1 Introduction↩︎

Classical machine learning is rapidly transforming research and industrial applications, while its quantum counterpart, qml, remains in its infancy. Nevertheless, expectations are high: although quantum advantage on classical data is contested, quantum advantage is widely anticipated when qml operates on quantum data ([1]–[3]).

For the successful development of vqas, a thorough understanding of the common properties of cost functions is essential. In particular, the topological and geometrical properties of qcls are of critical importance, both in the NISQ era and beyond ([4]).

In the classical setting, systematic analysis of the cost landscape has yielded insights that informed both architecture design and optimization ([5]–[9]). Research on the nature of the classical cost landscape around the minima proved that most local minima of large decoupled neural networks lie within a band-like structure, with the number of minima outside this band diminishing exponentially with network size ([10]). Furthermore, previous research found numerically that the Hessian of the cost landscape decomposes into a near-zero bulk and a small number of outliers ([11]), indicating the presence of flat directions. Complementing this structural picture, [12] established flat connectivity between distinct minima in deep networks, concluding that there are essentially no cost barriers in modern deep architectures.

The question of whether qcls exhibit comparable structural properties or not has received comparatively little attention, with most research interest in the qml community instead concentrated on bps, regions of extreme flatness where gradients vanish exponentially with the number of qubits ([13]–[15]). [16] directly linked this phenomenon to narrow gorges, which is the exponential narrowness of minima, proving that both always co-occur.

A particularly relevant contribution to understanding the similarities between classical and quantum cost landscapes comes from [4], who examine the connectivity of distinct minima in the QCL for a regression task with classically encoded quantum data in a three-qubit setting. Extending the concept of narrow gorges, they introduce ravines — low-cost connections between minima — and numerically observe their presence in most of their experiments, while noting that ravines do not imply a globally flat landscape and calling for further investigation. This study follows their definition and defines a ravine as a path between local minima \(\mathbf{p}_1\) and \(\mathbf{p}_{2}\) for which \(\mathcal{C}(\mathbf{p}_r) \lesssim \max \big(\mathcal{C}(\mathbf{p}_1), \mathcal{C}(\mathbf{p}_{2})\big)\) holds, with \(\mathcal{C}(.)\) as the cost function and \(\mathbf{p}_r\) as any point along the path.

The present study builds upon their analysis of ravines: motivated by the work of [12] and [4], we conduct a systematic analysis of low-cost connections between local minima of vqas and extend previous findings to quantum data and to circuits of greater width and depth. As a further contribution, we systematically explore and visualize the geometric and topological properties of qcls, and subsequently introduce a framework that leverages them to improve prediction performance through ensemble learning. Finally, a formal complexity analysis confirms that the presented ensembling approach yields substantial resource savings over naive qnn ensembling. Section 2 introduces the key methods, including the neb algorithm and the qnn filtering metric. The learning setup and neb ensembling procedure are detailed in Section 3, followed by the results and their discussion in Section 4. The limitations of the proposed framework are discussed in Section 5, and the paper concludes with an outlook and conclusion in Sections 6 and 7, respectively.

2 Methods↩︎

2.1 neb algorithm↩︎

The neb algorithm is designed to find a minimum-energy path between two points in a potential landscape. Originally applied to approximate activation energies in theoretical chemistry ([17]), the algorithm translates naturally to learning landscapes where the potential corresponds to the cost function \(\mathcal{C}\). Specifically, the refined tangent calculation from Henkelman and Jónsson is used, which improves numerical stability at kinks in the path ([18]).

To find the low-cost path, a linear interpolation between start and end point is initialized. This interpolation includes \(N\) pivot points \(\mathbf{p}_i\) whose position will be optimized during the neb process by steps in direction of \[\begin{align} \mathbf{F}(\mathbf{p}_i, k) = \mathbf{F}^{\mathrm{S}} (\mathbf{p}_{i}, k)\big|_{\parallel} - \mathbf{\nabla} \mathcal{C}(\mathbf{p}_i)\big |_{\perp}. \end{align}\] There are two contributions to the pivot position adjustment: The first contribution is attractive and acts tangentially to the neb path. Thus, it keeps neighboring pivots in proximity as a spring with constant \(k\) stretched between neighboring pivots. The force follows Hooke’s law as \[\begin{align} \mathbf{F}^{\mathrm{S}} (\mathbf{p}_{i}, k) = k[(\mathbf{p}_{i+1} - \mathbf{p}_{i}) - (\mathbf{p}_{i} - \mathbf{p}_{i-1})] \cdot \hat{\boldsymbol{\tau}}_{i}\hat{\boldsymbol{\tau}}_{i} \end{align}\] where \(\hat{\boldsymbol{\tau}}_i\) denotes the unit tangent to the path at pivot \(i\). The second contribution \(\nabla \mathcal{C}(\mathbf{p})\big |_{\perp}\) forces the pivot points in the direction of lower cost values while keeping distance to neighboring pivots by acting only orthogonal to the neb path.

Finally, the pivot position is adjusted with a step size parameter \(\eta\) as \[\begin{align} \mathbf{p}^{(t+1)} = \mathbf{p}^{(t)} + \eta \, \mathbf{F}. \end{align}\] The resulting pseudo code of the neb algorithm is shown in the Appendix 8.

[12] extended the NEB algorithm with an outer refinement loop to find low-cost connections between local minima: After each full NEB run, additional pivots are inserted where the linear interpolation of \(\mathcal{C}\) surpasses a deviation threshold, evaluated at \((M-1)\) positions between each pair of pivots. At maximum, one new pivot is added between two previous. The NEB is then rerun on this refined pivot set. To prevent the inserted pivots from redistributing uniformly due to the spring force, \(k\) is set to 0.

The present work builds on this iterative approach by introducing an adapted step scheduler. The underlying rationale is that newly inserted pivots are initialized between existing ones and thus already reside in the vicinity of a ravine, requiring fewer iterations to converge. This motivates our geometry-aware step scheduler, which progressively reduces the number of steps per neb cycle to allow convergence at reduced computational costs. The resulting aneb procedure is detailed in Algorithm 1, with its complexity analysis given in Section 2.2.

2.2 Complexity analysis of the aneb algorithm↩︎

The quantum complexity of the aneb algorithm (Algorithm 1) decomposes into two contributions: the complexity of the inner neb calls (Algorithm 16), and the overhead incurred between successive neb calls. One inner neb call together with the subsequent interpolation and pivot insertion step is referred to as one cycle \(c \in \{1, \dots, |S_{\mathrm{step}}|\}\), where \(S_{\mathrm{step}} = \{T_1, T_2, \dots, T_{|S_{\mathrm{step}}|}\}\) denotes the step scheduler with \(T_c\) steps assigned to cycle \(c\). Let \(p^{(c)}\) denote the number of pivot points at the start of cycle \(c\), and let \(G\) denote the number of circuit evaluations required to compute the gradient \(\nabla \mathcal{C}\) at a single point.

Inner neb complexity Each step of the inner neb requires evaluating the cost function \(\mathcal{C}\) at all \(p^{(c)}\) pivots for the tangent estimate, and computing \(\nabla \mathcal{C}\) at the \(p^{(c)} - 2\) interior pivots. Over \(T_c\) steps, cycle \(c\) therefore contributes \[\begin{align} T_c \cdot \left[ p^{(c)} + (p^{(c)}-2) \cdot G \right] \end{align}\] circuit evaluations.

Inter-cycle overhead Between successive neb calls, the cost function is evaluated along the linear interpolation of the current path to identify segments requiring refinement. Using \(M-1\) interpolation points between each pair of neighboring pivots, this amounts to \[\begin{align} (p^{(c)}-1)M + 1 \end{align}\] evaluations per cycle. This overhead is incurred for all cycles except the last.

Combined quantum complexity Summing both contributions over all cycles yields the total quantum complexity of the aneb: \[\begin{align} \mathrm{qco_{aNEB}} = \underbrace{\sum_{c=1}^{|S_{\mathrm{step}}|} T_c \cdot \left[ p^{(c)} + (p^{(c)}-2) \cdot G \right]}_{\text{inner NEB calls}} + \underbrace{\sum_{c=1}^{|S_{\mathrm{step}}|-1} \left[(p^{(c)}-1)M + 1\right]}_{\text{inter-cycle overhead}}. \label{eq:adapt-neb-complexity} \end{align}\tag{1}\] Note that eq. 1 is an upper bound, since the cost function values computed during the interpolation step could in principle be reused at the first step of the subsequent neb call, although this overlap is small and neglected here.

In practice, since \(G\) scales linearly with the number of parameters being optimized, the gradient evaluations dominate the complexity asymptotically and \(\mathrm{qco_{aNEB}}\) is well-approximated by retaining only the \(G\)-weighted terms.

Naive-emsemble complexity For comparison, the naive approach trains \(k \in \mathbb{N}\) independent models for \(T \in \mathbb{N}\) gradient steps each, giving \[\begin{align} \mathrm{qco_{naive}} = k \cdot T \cdot G. \label{eq:naive-complexity} \end{align}\tag{2}\] Optimizer-specific contributions are typically negligible relative to the gradient evaluations and thus are omitted.

Comparison A key advantage of the neb approach is that pivot points can be inserted progressively across cycles, as a ravine-like qcl structure ensures that even sparsely initialized neb settings converge to meaningful low-cost connections. Therefore, ensemble members along a converged path yield effective predictions despite being added in later neb cycles. Consequently, the initial pivot count can be kept small, with refinements added only where needed, reducing the dominant \(G\)-weighted gradient evaluations in the early cycles.

2.3 Ensemble learning↩︎

Ensemble learning is a well-established method for improving the prediction capabilities of classifiers. Rather than relying on a single classifier, a set of distinct classifiers is trained, and their predictions are combined into a joint ensemble prediction. The combination can be done in various ways, with averaging over individual predictions being a common approach ([19]).

The performance of an ensemble classifier can be characterized by two quantities: the prediction accuracy of the individual classifiers and the degree of independence between them ([20]). This reflects the intuition that a good ensemble consists of individually capable classifiers that nevertheless make sufficiently distinct errors ([21]).

2.4 Prediction-correlation metric↩︎

Ensemble performance depends on selecting base classifiers from favorable regions of a qcl. However, the choice of a suitable quantum circuit architecture and weight initialization presents a significant challenge due to the complex structure of qcls ([22]–[24]). To address this, we introduce a landscape quality metric that enables systematic evaluation of circuit and initialization choices prior to any optimization procedure. The metric builds upon the relative fluctuation \(\sigma\) introduced by [25], who proposed it as an optimization-free indicator of qcl learnability in a vqe setting. It combines the variance of the energy expectation value over the parameter space \(\Theta\) with a normalization factor to yield an inter-circuit comparable measure: \[\begin{align} \sigma \propto \sqrt{\underset{\theta \in \Theta}{\mathrm{Var}}(\langle H\rangle)}. \end{align}\] Zhang et al. demonstrate that this global metric captures key structural properties of qcls, including bps, weak expressibility, and ruggedness, which is corroborated through numerical simulations. The present work builds upon this metric and adapts it to the typical qml setting of multi-state classification with a vqa. To generalize the metric to this setting, the correlation between prediction vectors is employed as an effective measure of similarity.

A key distinction of the metric introduced here is its locality. Since gradient-based optimizers with a finite learning rate explore only a bounded neighborhood of the initialization point, the accessible region of parameter space is constrained by the product of the learning rate, the number of optimization steps, and the magnitude of the gradient, rendering distant regions of the landscape irrelevant to convergence behavior. Consequently, the structure of a qcl in the vicinity of the initialization is most consequential for optimization success, whereas a metric defined over the full parameter space \(\Theta\) may not faithfully reflect the learnability experienced in practice. The proposed metric therefore samples prediction vectors within a Gaussian-weighted neighborhood of a reference point in parameter space, effectively localizing the evaluation to the region relevant for training. Slow, large-scale variations in the prediction landscape are thus appropriately down-weighted.

The normalization factors from Zhang et al. are omitted here: the restriction to a single measured Pauli operator renders the Hamiltonian-norm factor redundant, while the Pearson correlation coefficient intrinsically normalizes by the sample variance, resulting in a direct measure of similarity. Combining these considerations yields the landscape quality metric \[\begin{align} \zeta(\theta;s) = \underset{\theta_1, \theta_2 \sim \mathcal{N}(\theta, s) }{\mathbb{E}} \Big[ R\big(\hat{y}(\theta_1),\hat{y}(\theta_2) \big) \Big], \end{align}\] where \(R(\cdot,\cdot)\) denotes the Pearson product-moment correlation coefficient between two prediction vectors \(\hat{y}\), each obtained for a given \(\theta\). The applied prediction function \(\hat{y}\) is defined in Equation 4 . The scale parameter \(s\) controls the width of the Gaussian-weighted neighborhood \(\mathcal{N}(\theta, s)\), which should be chosen to reflect the expected parameter displacement per optimization step. A low value of \(\zeta\) indicates high variability in the predictions within the local neighborhood, corresponding to a circuit and initialization that is suitable for optimization.

A key practical advantage of this metric is that it can be evaluated prior to training, at the stage where circuit architecture and weight initialization are selected. Therefore, one may repeat the sampling procedure until \(\zeta\) falls below a chosen threshold. Since no gradient evaluations are required, the computational overhead of this procedure is negligible relative to the subsequent gradient-based optimization.

3 Experimental setup↩︎

3.1 Learning problem↩︎

A controlled comparison across network architectures and optimization procedures requires a well-defined learning problem comprising a dataset, a prediction network, and a cost function, alongside the optimization procedures to be compared. The resulting classification process is depicted in Figure 2.

Datasets This study requires data sets that scale naturally with the number of qubits and offer tuneable difficulty are desirable for a meaningful comparison. The NTangled dataset satisfies both requirements: it provides a method to generate quantum states of various sizes whose ce is distributed around a specified target value ([1]). The ce of an \(n\)-qubit state \(\ket{\Psi}\) is defined as \[\begin{align} \mathrm{CE}(\ket{\Psi}) = 1-\frac{1}{2^n} \sum_{\alpha \in Q} \mathrm{Tr}[\rho_{\alpha}^2], \label{eq:ce} \end{align}\tag{3}\] where \(Q\) denotes the power set of \([n]\) and \(\rho_{\alpha}\) is the reduced density operator of subsystem \(\alpha\). Using quantum data natively avoids the encoding overhead that has been identified as a source of trainability challenges in several studies ([15], [26]).

The learning problem is formulated as a binary classification between two classes of quantum states with distinct ce values. Two datasets are considered: one with \(\mathrm{CE} \in \{0.05, 0.35\}\) and one with \(\mathrm{CE} \in \{0.15, 0.35\}\). We assume the latter, with the smaller separation between target ce values, is a harder classification task since the entanglement distributions of the states show a bigger overlap. The hyperparameters for the generation of input states, listed in Table 1, were chosen to minimize inter-class overlap in the ce distributions. Nevertheless, the labeling procedure introduces an expected false-label rate of approximately \(5\%\), which sets an upper bound of \(\approx95\%\) on the average accuracy achievable on unseen data.

Since estimating the ce requires access to all subsystem purities, multiple copies of the input state are necessary ([27], [28]). Throughout this study, two copies are provided, meaning that 3-qubit states are generated and concatenated into a single 6-qubit input, except where noted i.e. in the scaling experiments.

Finally, each dataset is split into 400 training and 600 test states, allowing for a precise evaluation on test data. To reduce dataset dependency, three data shuffle seeds are used and their results are aggregated in the evaluation.

Cost function To evaluate the predictions made by the networks, a binary cross entropy cost \(\mathcal{C}_{\mathrm{BCE}}\) is used.

\[\begin{align} \mathcal{C}_{\mathrm{BCE}} = -\left( y \log(\hat{y}) + (1 - y)\log(1 - \hat{y}) \right) \end{align}\]

3.2 Network architectures↩︎

qnns This study is oriented towards generalization across learning problems, requiring circuits that are both resource efficient and versatile. heas satisfy both criteria through their generic structure and low circuit depth ([29]).

Circuits are sampled to cover a broad range of architectural configurations. Each circuit consists of three parts: all circuits start by loading the copies of a quantum state onto the registers, followed by alternating rotational and entangling layers. Rotational layers draw one kind of parameterized gates (Pauli-\(X\), Pauli-\(Y\), their combination, or arbitrary rotation gates) while entangling layers adopt chain, cyclic ([4]), or all-to-all connected topology. Figure 3 illustrates the resulting layout. Excluding the scaling analysis, six circuits are evaluated, with depths ranging from 2 to 10 and parameter counts between 12 and 72.

The output is obtained by measuring the Pauli-\(Z\) expectation value on one of the first three qubits. This value is linearly rescaled with factor \(5\) and bias \(b\) and passed through a sigmoid function to improve gradient flow across the observable’s value range, yielding the prediction \[\begin{align} \hat{y} = \sigma\!\big(5 \cdot \langle Z \rangle_{\theta} + b\big) = \frac{1}{1 + \exp\!\Big(-\big(5 \cdot \langle Z \rangle_{\theta} + b\big)\Big)}. \label{eq:y-hat} \end{align}\tag{4}\] The bias parameter \(b\) is optimized along the rotational parameters of the qnns.

All qnns are simulated using PennyLane ([30]) with JAX ([31]) for computational acceleration, in a noise-free setting throughout.

mlp As a classical counterpart to the hea, a mlp was included to enable a fair classical comparison. To match the parameter counts of the sampled circuits, architectures with either one or two hidden layers were used, covering a range of 19 to 109 parameters. The input vector was constructed by separating the complex amplitudes of the quantum states into their real and imaginary parts. ReLU activations were applied in the hidden layers and a Sigmoid activation on the output. As overfitting was observed during preliminary experiments, the training dataset of the mlps is further partitioned into training and validation subsets at a ratio of 4:1, with early stopping conducted with the validation data results.

3.3 Training protocols↩︎

neb optimization protocol Two minima were first obtained via standard Adam optimization and subsequently used as initialization points for the adapted NEB algorithm (Algorithm 1).

Two hyperparameter configurations were employed, each targeting a distinct objective. The first configuration aimed to confirm the existence of a low-cost connection in the cost landscape, and was therefore allocated sufficient resources to converge to such a path, hence its label as the resource-abundant configuration. It uses a large number of cycles and iterations, inserts additional pivots wherever the deviation exceeds a predefined threshold, and sets the spring constant to zero. The latter ensures that pivots are not distributed uniformly along the path. Instead, all pivots and their interpolations should aim at the cost level of the initialization points.

The second configuration addresses whether low-cost paths are well-suited for practical ensemble prediction. Here, the emphasis shifts from convergence guarantees to computational efficiency: the goal is to identify sufficiently good points along the low-cost connection without excessive resource expenditure, hence its label as resource-scarce configuration. The exact parameter choices for both configurations are listed in Tables 3 and 4.

In both settings, the step scheduler is designed such that the first cycle comprises more iterations, providing a rough approximation of the potential ravine, which is then refined in subsequent cycles.

Ensemble predictions are obtained by evaluating the predictions of a set of pivot points along a low-cost path, each corresponding to a distinct classifier configuration, and averaging their predictions per data point, as illustrated in Figure 4.

Naive ensemble In the naive ensemble approach, each base classifier is trained separately, without coordination. At inference time, predictions are aggregated by averaging across ensemble members. To ensure a fair comparison, the number of independently trained classifiers equals the total number of pivots in the final neb path, and both approaches are allocated the same number of iterations using Adam as the optimizer which in the aneb case are only used by pivots added in the first cycle. Consequently, the training of one naive ensemble classifier is identical to one initialization point acquisition of the neb procedure.

3.4 Experimental baselines↩︎

This work investigates the potential of the neb procedure for ensemble prediction. To isolate the sources of any observed advantages, two axes of comparison are considered: whether qnns outperform mlps on the learning problem, and whether neb ensembling outperforms the naive ensemble approach. All four combinations are evaluated.

For the qnn results, the prediction-correlation metric \(\zeta\) introduced in Section 2.4 is used as a pre-training filter. Since \(\zeta\) can be evaluated before training with negligible computational overhead, it provides a principled criterion for selecting promising circuit and initialization combinations. Results are reported for different percentile thresholds of \(\zeta\), retaining only the configurations with the lowest values.

Since \(\zeta\) is specifically designed for qnns, mlps are instead filtered by a dedicated metric \(\chi\), defined as the minimum validation cost averaged across the classifiers of one ensemble. Ensembles are ranked by this quantity, and only those falling within the lowest \(x\)-th percentile are retained for analysis.

Crucially, this mlp filter \(\chi\) is computed post-training and directly reflects optimization outcomes, whereas the qnn filter is computed prior to training. Any performance advantage of qnns over mlps in the filtered results can therefore be attributed to model quality rather than a more favorable filtering procedure.

3.5 Complexity analysis↩︎

The presented framework is a quantum-classical hybrid algorithm. Given that the wall-clock time of operations on quantum hardware is significantly bigger than for operations on classical hardware, the complexity analysis focuses on the quantum complexity of the algorithms. The derivation for the aneb algorithm (Algorithm 1) is provided in Section 2.2. Following the parameter shift rule ([32]), the cost of a single gradient evaluation is assumed to be \(2d\), where \(d = \dim(\nabla \mathcal{C})\).

The quantum complexity of the entire neb workflow includes the complexity of the initialization point generation and the resources for the actual aneb procedure. Thus, the quantum complexity of the rs neb configuration evaluates to \[\begin{align} \mathrm{qco}_{\mathrm{NEB}} &= \mathrm{qco}_{\mathrm{init}} + \mathrm{qco}_{\mathrm{aNEB}} \big|_{\mathrm{rs}} \\ &= 2000d + (4600d + 3337) \\ &= 6600d + 3337, \end{align}\] where \(d\) denotes the dimensionality of the search space. For comparison, the quantum complexity of naive ensemble training is derived in Section 2.2 (Equation 2 ). The comparable case consists of \(9\) independently trained models with \(500\) steps each, yielding \[\begin{align} \mathrm{qco}_{\mathrm{naive}} = 9000d. \end{align}\]

In the high-\(d\) regime, the relative quantum complexity savings of the neb workflow approach \(\approx 27\%\) compared to naive ensemble training. This advantage arises because pivots are added incrementally across cycles: pivots inserted in later cycles only the resources required for the remaining cycles, rather than the full procedure, reducing the total number of gradient evaluations compared to running the full optimization for all classifiers as in the naive ensembling.

3.6 Scaling experiments↩︎

Since this study relies on classical simulations of quantum circuits, computational constraints limit the tractable circuit sizes. Nevertheless, understanding how the proposed methods scale with circuit width and depth is essential for assessing their viability on future noise-resilient hardware. Two questions are addressed: whether low-cost connections are still found for larger circuits, and how resource requirements scale with the search space-dimensionality and the number of qubits.

Due to computational restrictions, the scaling analysis is limited to the two best-performing circuits from the main experiments and a single data-shuffle seed. Depth scaling is examined by doubling and quadrupling the number of layers, while width scaling extends the analysis to 8- and 9-qubit circuits. For the width scaling experiments, the number of neb steps is tripled to allow for a more precise scaling analysis given the increased resource demands.

Changing the circuit width requires adapting both the dataset and the circuit architecture accordingly. For 8-qubit circuits, a duplicated 4-qubit input state is used. The generation follows the procedure from [1] with the parameters provided in Table 2 For 9-qubit circuits, the 3-qubit states from the main experiments are triplicated.

Figure 5 illustrates the procedure of analyzing the convergence behavior for different widths: the gathered training cost values are averaged over the pivots added in the respective cycle (Fig. 5 (a)). The resulting values are then used to conduct an exponential fit of the form \(f(x) = a\cdot\exp(bx) + c\) with parameters \(a,b,c \in \mathbb{R}\). This exponential model is motivated by the expected exponential convergence of gradient descent in the vicinity of a local minimum. It is further adopted for its simplicity, as it summarizes the decay rate in a single interpretable parameter \(b\), which can be directly compared across different settings. Since the point of convergence of all pivots changes each cycle with the addition of new pivots, each cycle can be regarded as an independent decay process. As the most relevant process is the decay into the ravine corresponding to the first cycle in which a pivot was added, this decay analysis is restricted to just the first cycle of each pivot. Given the decay parameter \(b\), the number of iterations required to reduce the cost value to a threshold of \(\delta=5\%\) of its starting value relative to the point of convergence can be computed as \[\begin{align} \mathrm{it}^{\star} = \frac{\log\delta}{b}. \end{align}\]

Although scaling experiments are limited in scope, they provide indicative evidence of whether the proposed framework remains viable beyond the primary experimental setting.

4 Results & discussion↩︎

4.1 Existence of ravines↩︎

Previous research suggested the existence of ravines in qcls in a 3-qubit setting when learning from classical data ([4]). We extend this result and confirm the existence of such ravines across the vast majority of tested six qubit settings. All cost values reported in this subsection are evaluated on the training set, using the resource-abundant neb configuration.

Figure 6 shows the normalized cost values along the optimized neb path. For low values of the prediction-correlation metric (\(\zeta_\text{pct} = 0.25\)), a clear distinction between the linear interpolation and the optimized path is apparent, corresponding to the finding of ravines. While the cost values along the linear connection form a plateau with steep ends, the average neb training cost remains below that of the initialization points at every point along the optimized neb path. This suggests not only the existence of ravines, but also that classifiers located within the ravine achieve better training performance than the initialization points, which were naively trained with comparable resources. One possible explanation is that, while ravines are clearly distinguishable from plateaus by their lower cost values, they are not entirely flat internally. Pivots already residing within a ravine continue to be optimized, ultimately converging to regions of even lower cost than those reached by the naive initialization points.

When aggregating over all circuit architectures and initializations (\(\zeta_\text{pct} = 1\)), this trend of finding ravines persists but is attenuated: the linear connection forms a less pronounced plateau with considerably higher variance, and the optimized neb path likewise exhibits higher variance, with cost values that do not consistently fall below those of the initialization points. Inspection of individual traces reveals isolated settings in which no low-cost connection could be identified by the neb procedure, as well as cases in which the linear interpolation is already flat at initialization.

For classical neural networks, low-cost connections between local minima are well established ([12], [33]–[35]). [12] find consistently that the barriers along such connecting paths vanish — turning them into proper ravines — in wider and especially deeper convolutional neural networks, hinting at a positive link between number of parameters and ravines in classical loss landscapes. Nevertheless, ravines in qcls have only been observed in small likely-underparameterized qnn circuits. Thus, it is important, and likely useful, to analyze whether ravines arise in larger overparameterized qnn circuits. Overparameterization necessarily leads to redundant dimensions in the loss landscape, which can embed local minima points into higher-dimension spaces with similar loss value, and thus may aid the formation of ravines.

Figure 7 provides an intuitive illustration of the ravine structure of a qcl for a single exemplary optimized neb path. The path is unfolded along the horizontal axis, with the cumulated distance between pivot points used as the parameterization, while the vertical axis encodes displacements orthogonal to the neb path. For most pivot points, the cost values are lowest along the path itself and increase rapidly away from it, confirming the narrow, channel-like geometry characteristic of a ravine. Nevertheless, some pivot points exhibit a broader low-cost neighborhood, suggesting the presence of side tracks or branching ravines merging into the optimised neb path.

4.2 Ensemble performance↩︎

After confirming the existence of ravines, this section examines the properties of the pivot points along the neb path optimized in the resource-scarce setting, with all reported metrics evaluated on the held-out test set.

Independence of single classifiers Each pivot point corresponds to a parameter configuration and thus defines a baseline classifier. Figures 8 (a, c) show the distribution of pairwise prediction-correlation coefficients across all pivots for one neb optimization after the optimization procedures, which is thus inherently different from the prediction-correlation metric \(\zeta\), computed before training. A positive correlation after training is found in almost all settings, which is expected since multiple baseline classifiers must agree in their predictions if they classify this state correctly. Nevertheless, the inter-prediction-correlation is substantially higher for both mlp variants than for their qnn counterparts, representing one of the key structural differences between the two network types. How far this observation generalizes beyond the present setting, and what theoretical mechanism underlies it, remain open questions that warrant further investigation.

Comparing the neb and naive approaches in the quantum case, the qnn neb classifiers exhibit significantly lower pairwise correlation (\(p<0.001\) for \(\zeta_{\mathrm{pct}} \in \{0.25, 1.0\}\)), which is beneficial for ensemble learning as described in Section 2.3. Further research is needed to establish whether this independence is an inherent property of ravines.

Statistical significance throughout this work is assessed using the nonparametric one-sided Mann-Whitney U test at a significance level of \(0.05\). This test compares two arbitrary distributions and evaluates whether one is statistically smaller or greater than the other.

Note that the peak of near-zero correlation can be attributed to classifiers that do not surpass the baseline accuracy of \(50\%\).

Accuracy of single classifiers Turning to the second ingredient for strong ensemble performance, the accuracy of the individual baseline classifiers, the classical networks dominate as shown in Figures 8 (b,d). For low values of the validation cost filter metric (\(\chi_\text{pct} = 0.25\)), the mlp classifiers exhibit two sharp peaks at high accuracy, outperforming the vast majority of qnn baseline classifiers with \(\zeta_\text{pct} = 0.25\), whose accuracy distribution is broad and diffuse. Nevertheless, the qnn neb classifiers achieve stronger individual performance than those from the naive approach (\(p<0.001\)).

Including all circuits and initializations (\(\zeta_\text{pct}, \chi_\text{pct} = 1\)), the picture becomes more ambiguous, with the notable exception of the naive qnns: all other approaches include a non-negligible number of classifiers at the \(50\%\) baseline accuracy. Furthermore, the dominance of the mlp networks is less pronounced, as reflected in a greater overlap of the distributions. This is also a consequence of the mlp filtering, which excludes poorly performing mlp configurations from the \(\chi_\text{pct} = 0.25\) subset.

Taken together, both filter metrics serve as strong indicators of base classifier performance, with the prediction-correlation metric \(\zeta\) being particularly noteworthy given that it is computed prior to any optimization, suggesting its utility as a general-purpose vqa performance indicator beyond this study.

Interestingly, the correlation filter \(\zeta\) appears to benefit the qnn neb approach even more strongly than the naive qnn approach. While both approaches seem to be optimized by taking this metric into account, the neb protocol has the advantage that neighboring pivots collectively drive each other deeper into the ravine, requiring less iterations to build strong models than the naive protocol which trains each qnn independently. A possible explanation is that \(\zeta\) helps maximize performance for both approaches, but only the neb approach leverages the valuable information of the initialized local minima to find further minima, which highlights the inherent advantage of our proposed protocol.

Accuracy of the ensembles Combining the individual strength and independence of classifiers, the ensemble performance will be examined in the following. Figure 9 shows the ensemble test accuracy distributions for all combinations of network types and optimization procedures across different threshold values \(\zeta_{\mathrm{pct}}\) and \(\chi_{\mathrm{pct}}\).

Most notably, the qnn neb approach outperforms all other classical and quantum methods for \(\zeta_{\mathrm{pct}}, \chi_{\mathrm{pct}} \in \{0.25, 0.5\}\), thus providing superior ensemble performance while lowering resource demands compared to the naive qnn by \(\approx 27\)% as shown in Section 3.5. This demonstrates the high potential of the comparatively independent qnn baseline classifiers and underlines the efficiency of the neb approach in exploiting this independence. The previously stated performance differences are statistically significant (\(p<0.001\)) for all stated pairwise comparisons except for qnn neb and qnn naive at \(\zeta_{\mathrm{pct}} = 0.5\), where the two methods yield comparable results (\(p=0.441\)).

For \(\zeta_{\mathrm{pct}}, \chi_{\mathrm{pct}} = 1\), both qnn approaches are significantly outperformed by their classical mlp counterparts (\(p<0.001\)), which can be attributed to the considerably larger performance spread of qnns across different circuit designs and initializations. While this sensitivity to architectural and initialization choices can be a disadvantage when only a single circuit and initialization are available, it simultaneously presents an opportunity: the provided prediction-correlation metric allows for an informed selection of suitable circuits and initializations at barely any additional computational overhead.

Comparing both qnn approaches directly, statistically significant performance differences are observed only for \(\zeta_{\mathrm{pct}} = 0.25\), where the qnn neb approach surpasses naive ensembling (\(p<0.001\)). This is consistent with the earlier finding that neb baseline classifiers exhibit a higher degree of independence in this regime compared to their naively trained counterparts, which in turn translates into a more diverse and therefore stronger ensemble. The single classifier accuracies are also higher for the qnn neb approach than for its naive counterpart. Nevertheless, this advantage diminishes for higher values of \(\zeta_{\mathrm{pct}}\), suggesting that the qnn neb is particularly effective when base classifiers are drawn from well-initialized, low-\(\zeta\) regions of the parameter space, where the neb path exploration yields the most diverse set of solutions.

Furthermore, the data suggest an underlying bimodal structure in the performance distributions of both qnns and mlps. This is an artifact of the limited number of data sets studied and is expected to dissolve into a unimodal distribution in the limit of a large number of data sets and circuits. Additionally, the presumption that the dataset with CE values of \(\{0.15, 0.35\}\) defines a harder learning problem compared to CE values of \(\{0.05, 0.35\}\) is confirmed by the results.

In terms of absolute performance, all configurations except the weakest mlp neb achieve ensemble test accuracies well above the random baseline of \(0.5\), demonstrating that the learning problem is consistently and reliably solvable across the methods studied. The highest observed median ensemble test accuracy of \(\approx88\%\) further suggests that the task difficulty is well-calibrated: while the problem is tractable for the strongest classifiers, none of the methods reaches a performance ceiling, leaving room for further improvement and meaningful differentiation between approaches.

4.3 Scaling analysis↩︎

While the previous subsections provided evidence for the existence of ravines and their potential to be leveraged for stronger ensemble performance, this subsection analyses how these properties scale to deeper and wider circuits, thereby providing an indication of how the approach may apply in practical settings with large-scale, noise-resilient hardware. To gather the results presented below, only a subset of circuits and data shuffle seeds was used due to limited computational resources. Details on the experimental configuration are provided in Section 3.6.

Depth scaling First, the scaling behavior with circuit depth is considered. Figure 10 shows the linear and neb-optimized paths for circuit depths scaled by factors of 1 (reference), 2, and 4. Note that the results shown, unlike those in the previous plots of this kind, originate from neb runs of the rs configuration, which gives rise to the characteristic spikes at the interpolation points between pivots.

While these neb runs cannot conclusively confirm the existence of ravines, since the path is sampled only at a small number of pivots, the resulting pivot configurations nonetheless exhibit the defining properties of ravines: low-cost local minima situated between the initialization points. Furthermore, a pronounced increase in the relative plateau height of the linearly connecting path is observed with increasing depth of the circuits. This can be attributed to the fact that deeper circuits admit local minima with considerably lower cost values, while randomly sampled parameter configurations yield approximately the same cost values regardless of circuit depth. This underlines that circuits with a larger number of parameters allow for a better approximation of the target classification function, and is further reflected in a systematic shift of the statistical properties of the individual pivot classifiers, as shown in Figure 11. As the spread of predicted probabilities across different samples increases with circuit depth (Fig. 11 (a)), the classifiers exhibit higher decision confidence, which in turn enables them to reach lower training cost values (Fig. 11 (c)). This improvement in predictive quality, however, comes at the cost of increased correlation between the predictions of different classifiers, which is precisely the trend observed in Fig. 11 (b).

Combining the individual classifiers into ensembles confirms the overall trend of improving performance with increasing circuit depth: Figure 12 shows that the ensemble test accuracy rises steadily with circuit depth, reaching up to \(91\%\) for the deepest configuration studied. However, this performance gain comes at a commensurate increase in computational cost, as the complexity of a single gradient evaluation scales linearly with the number of parameters meaning that the 2x and 4x depth configurations require roughly a factor of 2 or 4 more computational resources, respectively, for only modest accuracy improvements. This highlights an important practical consideration: slightly weaker baseline classifiers operating at shallower depth can yield comparable ensemble performance provided their predictions are sufficiently independent, effectively achieving similar results at a fraction of the computational cost. Beyond this trade-off, the results demonstrate that the qnn neb approach scales favorably with circuit depth and retains strong ensemble performance across all circuit regimes studied.

Qubit scaling Lastly, the scaling of the neb method with circuit width is analyzed, as this constitutes the crucial scaling factor for large-scale qml applications. To conduct this comparison, different datasets are required for different qubit counts, rendering a direct comparison of absolute performance values impractical. Nevertheless, the existence of ravines and the amount of computational resources required for convergence can still be analyzed to evaluate the scaling prospects of the neb approach.

Figure 13 shows the linear and neb-optimized paths for circuits of 6 (reference), 8, and 9 qubits. Two observations stand out: first, the neb pivots significantly reduce the cost values along the linear connection, presumably in the process of converging towards a ravine-like structure; and second, the plateaus along the linear connection are less pronounced than in the 6-qubit reference case. The latter can be explained by the fact that the local minima used as initialization points exhibit generally higher cost values for wider circuits, while randomly sampled points in parameter space yield approximately the same cost values as in the reference case.

The fact that the pivot points have not yet reached the cost level of their respective initialization points suggests that the neb procedure requires additional computational resources for wider circuits. This raises the question of how much extra resource expenditure is needed to compensate for the increased number of qubits. To address this, the number of optimization steps in the neb procedure was increased for the 8- and 9-qubit convergence experiments, allowing for a more accurate analysis of the convergence behavior. Further details on the experimental setup are provided in Section 3.6.

The metric \(\mathrm{it}^{\star}\) introduced in Section 3.6 is a measure for iterations necessary for convergence to a convergence point parameterized by \(\delta\). For \(\delta=0.05\), the resulting values of \(\mathrm{it}^{\star}\), shown in Figure 14, exhibit a substantial increase with the number of qubits for both approaches. Nevertheless, the resources required for convergence are consistently roughly twice as large for the naive approach as for the neb approach, a trend that holds across all qubit counts studied. Furthermore, the resources required for convergence in the second neb cycle are slightly below those of the first cycle, suggesting that previously explored landscape structure is effectively reused. The values for the exponential decay parameter \(b\) are shown in Figure 15.

Taken together, these results suggest that the neb method scales at least as favorably as the naive approach. In the specific configurations studied, the neb approach even exhibited faster convergence, which may be attributed to the fact that it combines gradient information from multiple pivots simultaneously, and thereby effectively leverages the geometrical structure of qcls.

5 Limitations↩︎

In this study, we conducted and analyzed several thousand optimization procedures using both neb and naive ensembling across multiple datasets, data splits, circuit designs, including various depths and widths, and initializations to improve generalizability. Numerical studies are, however, inherently limited to the data they generate: here, circuits of up to \(9\) qubits and \(288\) parameters. While modest on the scale of long-term applications, this scope was constrained by the computational cost of producing a statistically-meaningful amount of optimization runs. The simulations further assumed error-free gradient evaluation. Although current hardware does not satisfy this requirement, approaches such as the one presented here are intended for deployment once quantum hardware becomes fast enough for meaningful machine learning applications.

A further limitation of the proposed aneb ensembling framework is its dependence on the presence of ravine structures in the cost landscape. Although our observations of ravines are consistent with prior work ([4]), their existence has not been established for arbitrary circuit configurations. In cases where local minima are unconnected by such structures, the neb procedure will fail to identify low-cost connecting paths, potentially yielding ensemble predictions inferior to the naive baseline.

Finally, the filtering metric, which substantially improves qnn ensemble performance, may pose practical difficulties in settings where the circuit architecture is fixed or resampling is not feasible.

6 Outlook↩︎

Promising directions for future work include scaling this study to circuits of greater depth and width to verify that ravine structures persist beyond the parameter regimes accessible here. Complementing these empirical investigations with theoretical analysis would be equally valuable: a rigorous treatment could establish conditions for the existence of ravines and explain the notably-high independence observed among qnns in general and within the neb ensembling framework in particular. Such theoretical grounding would deepen our understanding of fundamental differences between quantum and classical machine learning.

Another natural extension is a systematic comparison of qcls with their classical counterparts regarding properties beyond ravines. The structure of local minima and their relationship to architecture choices has been studied extensively in classical settings ([7], [12]). Analogous investigations in the quantum regime remain largely open. While ravines already enable the tailored optimization procedures demonstrated here, a broader characterization of qcl properties could inform the design of additional improvements to qnn training.

Finally, given that the prediction-correlation metric \(\zeta\) is such a resource-light metric while being such a strong indicator of vqa performance, further research on the underlying mechanism of this phenomenon, as well as on its generalization to different qml models, are important future directions.

7 Conclusion↩︎

This study investigated the structure of qcls across a broad range of settings with respect to low-cost connections between local minima, commonly referred to as ravines ([4]). The neb algorithm identified ravines in the vast majority of configurations, extending previous findings to quantum data and to circuits with up to \(9\) qubits and \(288\) variational parameters.

Individual pivot points along the neb low-cost paths were shown to exhibit greater prediction independence than both quantum and classical baselines. The aneb ensembling procedure leverages this independence to construct a strong ensemble classifier, outperforming both quantum and classical alternatives when combined with a lightweight circuit initialization and filtering procedure prior to optimization. The introduced filtering metric itself proved to be a strong indicator of vqa performance more broadly, suggesting that its applicability extends well beyond the scope of this study. Our complexity analysis further shows that aneb achieves this while requiring substantially fewer resources than the naive ensemble approach. Finally, the depth and qubit scaling analysis provided promising evidence that ravines persist in larger circuits, and that the aneb convergence scales more favorably in terms of resource requirements than naive qnn ensembling.

Taken together, these results demonstrate that qcl-structure-informed algorithms can improve ensemble performance while reducing computational cost. The exploitation of landscape geometry is therefore a compelling direction for future research, with applications ranging from tailored optimization procedures to warm-starting strategies.

The data used in this study are available upon reasonable request. Equivalent datasets can be generated following the procedure of [1] with the code available at https://github.com/LSchatzki/NTangled_Datasets.

This work was supported by the Dieter Schwarz Foundation through the Fraunhofer Heilbronn Research and Innovation Centers HNFIZ.

The authors declare no competing interests.

References↩︎