1 Introduction↩︎

Quantum programs are executed by repeatedly evaluating circuits and measuring their outputs. Each shot samples the distribution induced by the circuit and the backend [1]; consequently, the shot count controls cost, queue usage, and the fidelity of the empirical distribution returned to the developer. Current execution workflows commonly expose shots as a fixed scalar budget, such as 1,000, 5,000, or 10,000 shots. This interface is simple, but it hides a software-engineering decision: the useful budget depends on circuit family, size, backend noise, and output sparsity. A single fixed value can therefore be excessive for one execution and insufficient for another.

This problem is aligned with the concerns of quantum software engineering (QSE). It is not only a statistical question about sample complexity, but also an execution-control question about how quantum software should be run in a systematic, measurable, and auditable way [2]. We focus on static quantum circuits: the circuit structure and parameters remain fixed while shots are collected. Workloads whose target distribution changes by design, such as optimizer-driven variational loops, are outside this scope. For a static circuit, however, the backend-induced distribution is fixed but unknown, so it is meaningful to ask whether the cumulative empirical distribution is still changing as new batches arrive.

We propose StableShots, a lightweight wrapper around any execution interface that returns batch-level counts. The method requires no circuit inspection, backend calibration data, or noise model. It accumulates counts, compares the current empirical distribution with an earlier cumulative distribution, and stops only after repeated evidence that the marginal effect of new batches is small. The decision is traceable from the observed counts, which makes the policy suitable for QSE settings where reproducibility and execution provenance matter.

This paper concentrates on one research question: how effectively does StableShots reach a target distributional-fidelity regime compared with fixed-shot baselines?¹ Our contributions are: (i) an auditable black-box shot-stopping rule for static circuit execution; (ii) a validation/test protocol for selecting and evaluating execution policies without using held-out traces during calibration; (iii) an empirical evaluation against fixed-shot baselines on QSimBench traces; and (iv) an open-source implementation and frontier-exploration tool.

2 Related Work↩︎

Analytical shot selection can be derived from concentration inequalities. Hoeffding’s inequality bounds deviations of sums of bounded independent random variables [3]. For empirical distributions over finite alphabets, Weissman et al. bound the \(L_1\) deviation between empirical and true distributions [4], and later work refines related concentration results [5]. Such bounds are valuable when formal guarantees are required, but their dependence on the output alphabet can make them conservative for full bitstring distributions. For \(\tau=0.05\) and \(\delta=0.05\), Hoeffding requires 82,707 shots at 4 qubits, 30.3 million at 8 qubits, and 179.8 billion at 14 qubits. Weissman is tighter but still reaches 2.27 million shots at 14 qubits. StableShots instead provides an empirical online policy: it does not certify closeness to the unknown distribution, but it decides whether further batches still materially change the observed distribution.

Adaptive measurement allocation has also been studied for variational and iterative workloads. Methods for VQE and related algorithms distribute shots across Hamiltonian terms, gradients, observables, or optimizer steps [6], [7]; learning-based approaches can infer measurement-reduction policies from previous executions [8]. These techniques exploit algorithmic or observable structure. StableShots is complementary: it targets a single static circuit and observes only streamed measurement counts. From a QSE perspective, this moves the shot budget from a pre-execution constant to an execution policy that can be calibrated, logged, and audited [9], [10].

3 StableShots↩︎

Let \(C\) be a fixed quantum circuit executed on backend \(Q\). After \(N\) shots, cumulative counts define an empirical distribution \(\hat{P}_N\) over bitstrings. We compare empirical distributions using Total Variation Distance: \[\mathrm{TVD}(\hat{P}_a,\hat{P}_b)=\frac{1}{2}\sum_x |\hat{P}_a(x)-\hat{P}_b(x)| . \label{eq:tvd}\tag{1}\]

StableShots executes \(C\) in batches of \(b\) shots. After each batch, it compares \(\hat{P}_N\) with \(\hat{P}_{N-\ell b}\), where \(\ell\) is the look-back in batches. If the resulting marginal TVD is at most \(\epsilon\) for \(k\) consecutive checks, execution stops; otherwise, execution continues until a maximum budget \(B\) is reached. The parameters have direct operational meaning: \(b\) controls decision granularity, \(\ell\) controls the comparison window, and \((\epsilon,k)\) control how conservative the stability criterion is.

The rule is intentionally heuristic. It does not claim that \(\hat{P}_N\) is within a certified distance from the unknown backend-induced distribution. Instead, it operationalizes a diminishing-returns signal: if several new batches have little effect on the cumulative empirical distribution, additional batches are unlikely to change the developer-visible result substantially. This makes the rule easy to implement and inspect. A run can be reconstructed from the batch sequence, the cumulative distributions, and the stopping predicate.

This design is deliberately conservative about what the wrapper assumes. It treats the circuit and backend as an opaque sampler and uses only the same information that a client already receives after execution: bitstring counts. Consequently, the method can be placed above different SDKs, simulators, or provider APIs without depending on compiler internals or hardware calibration schemas. The cost is that the stopping condition is local. It answers whether recent evidence has stabilized the cumulative empirical distribution, not whether a theorem certifies the final distribution. For the QSE use case addressed here, that distinction is useful: the policy is meant to support practical execution governance, where developers need an auditable stopping decision and an empirical record of the cost-fidelity trade-off.

For the experiments below we use the validation-selected profile b50_lb3_k5_eps0p005: batches of 50 shots, a three-batch look-back, five consecutive stable checks, and threshold \(\epsilon=0.005\). The external evaluation target is TVD \(\leq0.05\) against a high-shot empirical reference. The configuration should be interpreted as dataset- and target-scoped, not universally optimal.

4 Experimental Design↩︎

We evaluate on 180 QSimBench traces [11]: six circuit families (dj, qaoa, qft, qnn, random, and vqe), six sizes from 4 to 14 qubits, and five noisy IBM simulated backends. Each trace contains 20,000 shots. The 20,000-shot empirical distribution is used only as an offline reference; StableShots never observes it while deciding when to stop. We use the noisy-backend reference rather than an ideal simulator because the operational target of execution is the distribution induced by the selected backend, including noise [1].

All strategies are evaluated on prefixes of the same materialized trace. This makes fixed budgets and adaptive stopping comparable on identical sample paths, but it also means that lower-shot estimates and the 20,000-shot reference are not independent. We therefore interpret TVD as a controlled empirical proxy rather than as an oracle error measure.

Configuration choices are made only on validation traces. We use 100 structured backend-holdout repetitions. In each repetition and for each algorithm-size cell, one backend is held out as test and the remaining four backends are used for validation. Since the benchmark contains six algorithms, six sizes, and five backends, each repetition has 144 validation traces and 36 test traces. After validation selects a configuration, we evaluate it on the held-out traces. We compare against fixed-shot baselines at 5,000, 10,000, and 18,000 shots, which summarize low, medium, and near-cap execution regimes from the original baseline set.

The holdout protocol is designed to match the operational question. A developer rarely knows in advance which backend-noise profile will make a circuit easy or hard to sample, but a tool builder can calibrate a policy on previous executions and then deploy it on future executions from the same workload family. Holding out one backend per algorithm-size cell tests this setting more strictly than a random split over traces: every test trace belongs to a circuit family and size seen during validation, but its backend-specific sample path was not used to choose the policy. We report success rates at TVD thresholds 0.01, 0.05, and 0.10, median and mean shots, maximum TVD, and how often the policy reaches the 20,000-shot cap. Reporting cap use is important because a policy can achieve high success simply by behaving like a near-reference fixed budget.

5 Results↩︎

Table 1 reports held-out test performance over 100 repetitions, for 3,600 test evaluations. Fixed-shot baselines improve as the budget grows, but no fixed budget provides the same cost-fidelity trade-off. Fixed-5k has the lowest cost but reaches TVD \(\leq0.05\) on only 54.1% of evaluations. Fixed-10k improves success to 66.7%, while Fixed-18k reaches 83.3% success but spends more than twice the median shots of StableShots. The selected StableShots profile reaches TVD \(\leq0.05\) and \(\leq0.10\) on all held-out evaluations with median 7,650 shots.

2.4pt

The result should not be read as strict dominance. StableShots can spend more shots than a low fixed budget because it continues when the empirical distribution is still moving. Conversely, it can accept larger TVD than near-reference fixed execution because it stops once the marginal distributional change is small. Its role is to provide an adaptive operating point between arbitrary low-shot execution and always using a high fixed budget.

Table 2 shows that the adaptive behavior is trace-dependent. Size and algorithm family dominate cap use. Small circuits usually stop early, while 10–14 qubit traces often continue toward the cap. The dj and vqe families rarely need the cap, whereas qft, qnn, and random often approach it. Backend variation is smaller: median shots range from 7,000 to 8,550, and every backend group remains below the 0.05 target. The same profile reaches TVD \(\leq0.05\) on all 180 traces in the complete benchmark with median 7,700 shots, which is consistent with the held-out profile.

The cap-use pattern is a useful diagnostic rather than merely a cost statistic. When StableShots reaches the cap, the policy is reporting that local stability was not obtained under the configured threshold and patience. This is preferable to silently returning a low-shot estimate whose empirical distribution is still changing. In the evaluated traces, cap use is concentrated in larger and more diffuse distributions, where fixed low budgets also show poor reliability. For users who tolerate coarser fidelity, the policy can be made more aggressive: in the original validation grid, b50_lb2_k1_eps0p005 reached TVD \(\leq0.10\) on all traces with median 5,300 shots, but reached the stricter TVD \(\leq0.05\) target on only 87.8% of traces. This illustrates that StableShots is not a single universal budget-reduction recipe; it is a tunable execution policy whose configuration should be tied to the target fidelity regime.

2.6pt

These data clarify what the stopping signal captures. It is not merely a proxy for qubit count: circuits of the same size can stop at different budgets, and backend effects are visible but secondary. The policy therefore exposes a useful execution-level signal for quantum software tooling. Instead of asking developers to choose one budget for all workloads, a runtime or client library can record batch counts, apply a calibrated stopping rule, and report both the final distribution and the evidence that triggered termination.

6 Discussion and Limitations↩︎

The objective of StableShots is not to replace statistical guarantees with a universal empirical claim. The contribution is an execution policy and an evaluation workflow: the policy defines a transparent online signal, and the validation/test split prevents the reported configuration from being justified only on the traces used to choose it. This framing fits QSE practice because shot count becomes part of the software execution process: it can be parameterized, calibrated, logged, and revised as workloads or backends change.

Practical applicability. StableShots assumes that a client can obtain batches of counts from repeated executions of the same circuit. On some hardware services, submitting many tiny independent jobs would be inefficient because queueing and job-management overhead could dominate measurement savings. However, current quantum-cloud execution models increasingly support iterative workloads. IBM Runtime² sessions group iterative calls and prioritize subsequent jobs within the session, while caching data that can reduce repeated overhead. Amazon Braket reservations³ provide exclusive access to a selected device for a scheduled time window, making it possible to run multiple tasks without repeatedly competing in the global queue. These mechanisms do not eliminate the need to choose a practical batch size, but they make incremental execution plausible when batches are submitted inside a session, reservation, or provider-supported batch workflow.

Limitations. First, TVD is measured against a finite 20,000-shot reference, not the unknown backend-induced distribution. This is appropriate for noisy backends, where the noiseless ideal distribution is not the operational target, but the reference is still finite. Second, fixed budgets, adaptive prefixes, and references are derived from the same materialized traces. This improves comparability across policies but couples estimates and may favor longer prefixes. Third, the experiments use noisy simulated QSimBench backends; live QPU behavior, queueing, and backend drift require separate evaluation. Fourth, we assume that the backend-induced distribution is approximately stable during the execution of one circuit trace, so online stopping can be interpreted as convergence toward a fixed but unknown target distribution. This assumption is consistent with prior empirical evidence [9], but stronger temporal drift on live hardware could invalidate it. Fifth, StableShots targets static circuits. Within a variational iteration the parameters are fixed and StableShots could be used locally, but non-stationarity across optimizer steps would require additional mechanisms.

These limitations define the boundary of the present claim. The result is not that StableShots provides a distribution-free guarantee, nor that the reported parameter tuple should be used unchanged in all applications. The result is that a simple black-box stability signal can be selected without test leakage and can outperform common fixed-shot choices under a practical cost-fidelity objective. That is the core QSE contribution: execution cost is treated as a controllable software policy, and the policy is evaluated with the same empirical discipline expected for other quantum software tools.

A practical adoption path is to expose StableShots as an execution mode rather than as a replacement for fixed shots. A developer could request a target policy, maximum cap, and batch size; the runtime would then return the final counts, the consumed shots, the termination reason, and the stability trace. This keeps the familiar fixed-shot interface available for reproducibility-critical experiments, while adding an adaptive option for exploratory execution, benchmarking, regression testing, and service-oriented quantum workflows. It also makes failures informative: if a circuit repeatedly reaches the cap, the tool has evidence that the chosen target is too strict for the allowed budget or that the circuit-backend pair produces a distribution that stabilizes slowly.

This reporting discipline is important for empirical QSE. Shot count is often treated as an incidental parameter in experimental papers and tool evaluations, even though it can change both cost and measured quality. By making the stopping decision explicit, StableShots encourages authors and tool builders to report not only final accuracy but also how the measurement budget was consumed. In the long term, such execution metadata can support benchmark repositories in which shot policies are compared across workloads, reused under version control, and recalibrated when new backends or provider execution models become available.

7 Conclusions and Future Work↩︎

We presented StableShots, an online shot-stopping rule for static quantum circuit execution. The method monitors changes in cumulative empirical output distributions and stops when additional batches have repeatedly small marginal effect. On 180 QSimBench traces and 100 backend-holdout repetitions, the selected configuration reaches TVD \(\leq0.05\) on all held-out test evaluations with median 7,650 shots, while fixed-shot baselines either under-execute more often or spend substantially more measurements.

Future work should evaluate live QPU executions, independent high-shot references, and adaptive batch sizes. Richer policies could combine distributional change with support growth, uncertainty estimates, or backend-drift signals. For iterative workloads such as VQA and QML, StableShots could be used locally within each fixed-parameter evaluation while exploiting warm-start information across iterations. Another direction is to combine online stopping with circuit cutting and quantum multiprogramming, maintaining independent stopping states for subcircuits or co-scheduled circuits so that shots are spent where empirical distributions have not yet stabilized.

References↩︎

---

1. We performed a 75-configuration grid search over StableShots parameters and thresholds. We also empirically compared StableShots with scaled Hoeffding and Weissman policies, showing that plain Hoeffding and Weissman bounds are too conservative, recommending millions of shots even for small circuits, while it is also impossible to find a good scaling factor (even for each specific algorithm–size pair) that reaches results as good as StableShots. However, the strict 4-page limit does not leave enough space to discuss these experiments in detail. They will be presented in a future work. However, the full code, raw data, analysis scripts, and the frontier-exploration tool are freely available at: https://github.com/GBisi/stableshots↩︎

2. https://quantum.cloud.ibm.com/docs/en/guides/run-jobs-session↩︎

3. https://docs.aws.amazon.com/braket/latest/developerguide/braket-reservations.html↩︎