1 Introduction↩︎

Despite rapid advances in learning-based and foundation-model approaches [1]–[3], bundle adjustment (BA) remains the dominant refinement stage for achieving high geometric accuracy. In practice, BA often determines the final pose and calibration quality. However, in low-parallax or near-rotational regimes, the Schur-reduced camera system becomes ill-conditioned. This is a BA-level issue: image observations constrain bearing directions but weakly separate translation, point depth, and focal calibration when the camera baseline is small. In a simplified two-view model, the disparity satisfies \(d_{\mathrm{disp}}\approx fB/Z\), where \(f\) is focal length, \(B\) is baseline, and \(Z\) is scene depth; since \(\partial d_{\mathrm{disp}}/\partial Z\approx -fB/Z^2\), depth sensitivity vanishes as \(B\to 0\). This entanglement of translation, depth, and focal length creates weakly constrained directions in the linearized system that, after Schur point elimination, manifest as near-null camera-increment directions, regardless of the specific linear solver used. Classical Schur-based Levenberg–Marquardt (LM) may converge to solutions with small reprojection error while producing inaccurate relative poses and unstable calibration. This discrepancy between reprojection accuracy and geometric reliability persists across both classical and learning-based pipelines.

Significant effort has been devoted to improving scalability and numerical efficiency of BA, including square root BA [4], matrix-free formulations [5], and inverse-expansion methods such as Power BA [6] and its variable-projection extensions [7]. While these approaches enhance conditioning or computational efficiency, they do not directly address the instability of the Schur-LM update under weak geometric conditioning.

We therefore take a complementary solver-side perspective. Rather than modifying the BA objective or its linearization, we retain the classical Schur-reduced camera system and trust-region framework, and instead restrict each LM step to a geometrically informed low-dimensional search subspace. In this way, we directly stabilize the update direction while preserving the full objective and variable set, aiming to improve relative pose quality without sacrificing reprojection accuracy.

We refer to the resulting method as Gate-Guided CSS-BA, a selective LM scheme operating entirely within the classical trust-region (TR) framework. At each iteration, a gated support of informative camera blocks is formed, and Column Space Search (CSS) constructs a compact LM basis from this support. Camera blocks are ranked using a Schur-consistent predicted reduction score. A low-dimensional search subspace is then constructed from the camera-block coordinate subspaces spanned by a small high-gain subset of the Schur camera system.

By concentrating updates on well-conditioned directions, the method improves trust-region acceptance and stability in weak geometry while remaining compatible with standard Schur-based BA infrastructures.

Our contributions are summarized as follows:

• CSS instantiated on Schur-reduced BA. We adapt Column Space Search to the Schur camera system and construct a compact search basis from gate-guided block statistics without modifying the objective, residuals, or Jacobians.

• Direction-only restriction with full variables. Top-\(k\) gated blocks are used only to localize the LM step direction; all camera variables remain in the optimization problem.

• Stability gains in weak geometry. The method improves relative pose accuracy in low-parallax regimes and can reduce focal-length calibration error under the hardest weak-geometry settings, while preserving reprojection quality and lowering the effective solve dimension.

2 Related Work↩︎

Bundle adjustment (BA) is the standard nonlinear least-squares refinement in multiview geometry [8]. Modern systems rely on Schur-complement reduction and sparse LM solvers, as implemented in widely used libraries such as SBA [9], g2o [10], Ceres [11], and COLMAP [12].

Prior work on BA has primarily focused on scalability, numerical conditioning, and alternative factorizations. In this section, we review efficiency- and stability-oriented approaches and position our solver-side update strategy within this landscape.

2.1 Efficiency-oriented methods↩︎

A major line of work improves the scalability of BA by reworking its linear algebra backend. Agarwal et al. [13] replace sparse Cholesky with preconditioned conjugate gradients on the Schur-reduced system, enabling large-scale Internet reconstructions. Subsequent efforts focus on parallelization and memory efficiency, including multicore acceleration [14], out-of-core optimization [15], and distributed solvers [16], [17].

More recent approaches explore approximate or matrix-free formulations. Zhou et al. [18] reduce computational cost via stochastic approximations, while Safari [5] proposes a matrix-free scheme for shared-intrinsic settings. In parallel, inverse-expansion methods such as Power BA [6] and its variable-projection extension [7] improve efficiency by approximating the Schur complement. Recent GPU-oriented work further accelerates the BA linearization and solve stages [19], [20].

These works primarily target scalability and conditioning through alternative linear solvers or factorizations.

2.2 Stability-oriented methods↩︎

A complementary line of work focuses on improving numerical robustness of BA under ill-conditioning and weak initialization. Square Root BA replaces the standard Schur elimination with a QR-based null-space marginalization, improving conditioning and enabling accurate single-precision solves [4].

Variable projection (VarPro) methods analytically eliminate separable parameters to enlarge convergence basins. Hong et al. [21], [22] applied VarPro to BA to obtain initialization-free formulations, and Iglesias et al. [23] introduced expOSE, a projective factorization with exponential regularization. Weber et al. [6], [7] further extended this direction: PoBA approximates the inverse Schur complement via power-series expansion, while PoVar combines power expansion with VarPro for large-scale initialization-free optimization.

Other recent works explore robustness through probabilistic or calibrated formulations, such as ProBA [24] and calibrated BA without initialization [25]. As a representative stress case, spherical SfM [26] highlights the rank-deficiency arising under near-spherical motion, where classical LM can converge to geometrically unreliable solutions.

These approaches improve stability primarily by altering factorizations, reparameterizations, or eliminating subsets of variables. In contrast, our method retains the classical Schur-reduced system and objective, and instead stabilizes the solver by restricting each LM update to a geometrically informed search subspace.

2.3 Selective LM under Trust Region↩︎

Our approach is grounded in the trust-region (TR) interpretation of Levenberg–Marquardt (LM). Under standard assumptions, LM enjoys quadratic local convergence, and with rank deficiency one can still obtain fast local rates under suitable error-bound conditions [27]. The trust-region acceptance rule further ensures global descent even when the Jacobian is not locally full rank.

Motivated by this trust-region view, we adopt Column Space Search (CSS) [28] to construct a selective solver-side update policy within the classical LM framework. In its original form, CSS scores candidate coordinate directions using residual/Jacobian information, prioritizes data-consistent directions, and solves a reduced subproblem without modifying the objective.

Related lines of work restrict updates by selecting keyframes or states over time (e.g., incremental smoothing, double-window optimization, relative BA, and rolling-shutter BA) [29]–[32]. Unlike these approaches, which alter the active variable set or measurement support, our method keeps the full BA objective and variable space fixed, and restricts only the solver-side search subspace at each LM iteration.

In our formulation, a geometrically supported block set is identified per iteration, a compact search basis is constructed, and a damped LM subproblem is solved in this subspace before lifting the increment back to the full variable space. Because acceptance and damping updates follow the same trust-region rule as the baseline LM solver, accepted steps remain governed by the same objective-decrease test while reducing exposure to ill-conditioned directions under weak geometry.

3 Method: Gate-Guided CSS-BA↩︎

1 provides an overview of the proposed Gate-Guided CSS-BA pipeline. We design a solver-side Levenberg–Marquardt (LM) scheme tailored to the Schur-reduced bundle adjustment (BA) system. In classical Schur-LM, full-dimensional camera updates are computed at each iteration. Under weak geometric conditioning, such as low-parallax or near-rotational motion, the reduced camera system can become ill-conditioned, causing the LM step to move along poorly supported directions despite small reprojection error.

Rather than modifying the BA objective, altering its linearization, or eliminating variables, we retain the standard Schur-reduced formulation and trust-region mechanism. Instead, we restrict each LM step to a geometrically informed low-dimensional search subspace defined in the Schur-reduced camera increment coordinates.

Unlike keyframe or state-selection methods, the objective and variable set remain unchanged; only the search directions used to compute \(\Delta\mathbf{c}\) are constrained. The resulting increment is lifted back to the full camera space, so all camera variables remain in the optimization problem, although each iteration restricts the admissible update directions.

3.1 Preliminaries: Schur-Reduced BA↩︎

We consider the standard bundle adjustment objective \[E(\mathbf{x}) = \tfrac{1}{2}\sum_{i,j}\|\mathbf{r}_{ij}(\mathbf{c}_i,\mathbf{p}_j)\|_2^2, \qquad \mathbf{x}=[\mathbf{c};\mathbf{p}],\] where \(\mathbf{c}_i\) and \(\mathbf{p}_j\) denote camera and point parameters. Linearization at each LM iteration yields the damped normal equations \[\bigl(H+\lambda D_{\mathbf{x}}^{\top}D_{\mathbf{x}}\bigr) \Delta\mathbf{x} =-\mathbf{g}, \qquad H=J^\top WJ,\quad \mathbf{g}=J^\top W\mathbf{r} .\] Here, \(\mathbf{r}\) denotes reprojection errors, \(J\) the Jacobian of the reprojection errors, \(W\) a weighting matrix (e.g., from a robust loss), \(H\) the Gauss–Newton approximation to the Hessian, \(\lambda>0\) the LM damping parameter, and \(D_{\mathbf{x}}:=\operatorname{blkdiag}(D_c,D_p)\) the diagonal LM scaling matrix for camera and point variables.

Let \[H_{cc,\lambda}:=H_{cc}+\lambda D_c^\top D_c, \qquad H_{pp,\lambda}:=H_{pp}+\lambda D_p^\top D_p .\] Eliminating the point increments gives the damped Schur system \[\tilde{H}_\lambda\Delta\mathbf{c} =-\tilde{\mathbf{g}}_\lambda, \qquad \tilde{H}_\lambda :=H_{cc,\lambda} -H_{cp}H_{pp,\lambda}^{-1}H_{pc}, \quad \tilde{\mathbf{g}}_\lambda :=\mathbf{g}_c -H_{cp}H_{pp,\lambda}^{-1}\mathbf{g}_p . \label{eq:css:schur}\tag{1}\]

The candidate update \(\Delta\mathbf{x}=[\Delta\mathbf{c};\Delta\mathbf{p}]\) is accepted or rejected using the standard trust-region ratio \[\rho = \frac{E(\mathbf{x})-E(\mathbf{x}+\Delta\mathbf{x})}{ -\,\mathbf{g}^\top\Delta\mathbf{x} -\tfrac12\Delta\mathbf{x}^\top H\Delta\mathbf{x}}. \label{eq:rho}\tag{2}\]

We retain the baseline LM damping and trust-region update unchanged. Our modification targets only how \(\Delta\mathbf{c}\) is computed from the damped Schur system in 1 .

After Schur reduction, the linear solve is expressed over camera increments, and landmark information remains encoded through point elimination. This reduced system is naturally block-structured by camera blocks. As discussed in Sec. 1, under low-parallax or near-rotational motion this system becomes ill-conditioned, creating near-null camera-increment directions that cause the LM step to move along poorly supported directions. Intuitively, small baselines provide weak sensitivity to translation, depth, and focal-length changes; after point elimination, these weak modes appear as near-null directions in the reduced camera system.

3.2 Column-Space Search as LM Search-Space Design↩︎

Column Space Search (CSS) [28] is a general subspace strategy for nonlinear least squares. It scores candidate coordinate subspaces and computes an update in the span of a small, data-consistent subset.

In the Schur-reduced BA system, the unknown of the linear solve is the camera increment \(\Delta\mathbf{c}\). Therefore, we instantiate CSS over camera-block coordinate subspaces of \(\Delta\mathbf{c}\). At iteration \(t\), CSS scores camera blocks and selects a top-\(k\) support \(K_t\) as described in Sec. 3.3. The selected support is then used to construct a basis \(V_t\) for a low-dimensional search subspace.

This support should not be interpreted as selecting columns of the Schur matrix or as solving a separate BA problem on selected cameras. Each selected camera block defines a coordinate subspace of the camera-increment vector. Equivalently, using the block projector \(P_{K_t}\) introduced in Sec. 3.3, the support localizes the symmetric Schur operator on both sides: \(\tilde{H}_{\lambda,K_t}=P_{K_t}\tilde{H}_\lambda P_{K_t}\).

Let \(\mathcal{U}_t\subseteq\mathbb{R}^{n_c}\) be the search subspace at iteration \(t\), where \(n_c\) is the total dimension of the camera-parameter vector \(\mathbf{c}\). Let \(V_t\in\mathbb{R}^{n_c\times r_t}\) be a full column-rank basis of \(\mathcal{U}_t\), constructed as described in Sec. 3.3. We restrict only the camera-update direction by parameterizing \(\Delta\mathbf{c}=V_t\mathbf{y}_t\). Substituting this into the damped Schur system in Eq. 1 gives the projected LM system \[\bigl(V_t^\top\tilde{H}_\lambda V_t\bigr)\mathbf{y}_t = -\,V_t^\top\tilde{\mathbf{g}}_\lambda, \qquad \Delta\mathbf{c}=V_t\mathbf{y}_t . \label{eq:css:projLM}\tag{3}\]

After solving for \(\Delta\mathbf{c}\), we recover \(\Delta\mathbf{p}\) by back-substitution using \(H_{pp,\lambda}\). Acceptance and damping updates follow the same trust-region rule in Eq. 2 as the baseline LM solver.

The BA objective, residuals, and variable set remain unchanged. The increment \(\Delta\mathbf{c}=V_t\mathbf{y}_t\) is a full-sized camera vector, but its admissible direction is restricted to \(\operatorname{range}(V_t)\) at the current iteration. Thus, \(K_t\) is a basis-construction support, not an active-variable mask.

3.3 CSS Subspace Construction↩︎

To construct a stable search subspace, we first identify cameras that are geometrically reliable under weak-geometry conditions. These cameras provide directions that are better conditioned in the Schur-reduced system and are therefore suitable for constructing a stable LM step.

Let \(t\) denote the LM iteration index. We distinguish two camera index sets: \(\mathcal{S}\) is the geometry-aware support, computed once before optimization, and \(K_t\subseteq\mathcal{S}\) is the top-\(k\) basis seed selected by CSS scoring at iteration \(t\). Here \(k=\min(k_{\max},|\mathcal{S}|)\), with \(k_{\max}=10\) in our experiments.

The set \(\mathcal{S}\) collects cameras that pass geometric reliability tests.

At iteration \(t\), CSS scores the camera blocks in \(\mathcal{S}\) and selects \(K_t\) to construct the subspace basis; high-gain blocks are those with large local predicted decrease in the current damped Schur model.

Gate decides eligibility; CSS decides which eligible blocks are informative. The gate forms a reliable support \(\mathcal{S}\) using view connectivity, parallax, and rotational consistency. CSS then ranks only blocks within \(\mathcal{S}\) by their local Schur-LM predicted decrease and selects the top-\(k\) set \(K_t\). Together, gating restricts the candidate pool for basis construction to well-constrained cameras, while CSS selects the most useful eligible blocks for the current LM update.

Importantly, \(K_t\) determines basis directions only; it is not an active-variable mask on camera parameters.

Geometry-aware support \(\mathcal{S}\)↩︎

We identify geometrically informative cameras using diagnostics commonly employed in SfM and SLAM keyframe selection, such as parallax magnitude, rotational consistency, and view connectivity. These quantities detect weakly constrained configurations that often yield ill-conditioned Schur systems.

For camera pose \((R_i,\mathbf{t}_i)\) with center \[\mathbf{C}_i := -R_i^\top \mathbf{t}_i ,\] we define valid two-view edges \[(i,j)\in\mathcal{E}\Longleftrightarrow|\mathcal{P}_{ij}|\ge\tau_{\mathrm{shared}}, \label{eq:valid95edge}\tag{4}\] where \(\mathcal{P}_{ij}\) denotes the set of shared tracks observed in both cameras \(i\) and \(j\), with neighborhood \(\mathcal{N}(i)=\{j\mid(i,j)\in\mathcal{E}\}\).

For each shared track \(q\in\mathcal{P}_{ij}\) with 3D point \(\mathbf{X}_q\), define rays \[\mathbf{u}_{iq}=\frac{\mathbf{X}_q-\mathbf{C}_i}{\|\mathbf{X}_q-\mathbf{C}_i\|},\qquad\mathbf{u}_{jq}=\frac{\mathbf{X}_q-\mathbf{C}_j}{\|\mathbf{X}_q-\mathbf{C}_j\|},\] and pairwise median parallax \[\mathrm{Par}_{ij}=\operatorname{median}_{q\in\mathcal{P}_{ij}}\arccos\!\big(\operatorname{clamp}(\mathbf{u}_{iq}^\top\mathbf{u}_{jq},-1,1)\big).\]

We define the parallax-filtered neighborhood \[\mathcal{N}_E(i)=\{j\in\mathcal{N}(i)\mid\mathrm{Par}_{ij}\ge\tau_{\mathrm{par}}^{\mathrm{edge}}\}.\]

The camera-level parallax score is the median over parallax-filtered neighboring views, \[\mathrm{Par}_{i}=\operatorname{median}_{j\in\mathcal{N}_E(i)}\mathrm{Par}_{ij}.\] If \(\mathcal{N}_E(i)=\emptyset\), we set \(\mathrm{Par}_i=0\). For rotational consistency, let \(\bar R_i\) denote the mean rotation of cameras in \(\mathcal{N}(i)\). We compute the Rotation Agreement (RA) score as the geodesic distance between \(R_i\) and \(\bar R_i\):

\[\mathrm{RA}_i=\arccos\!\left(\operatorname{clamp}\left(\tfrac{\mathrm{tr}(R_i^\top \bar R_i)-1}{2},-1,1\right)\right). \label{eq:ra}\tag{5}\]

Here \(\mathrm{RA}_i=0\) means that \(R_i\) is aligned with the average orientation of its neighbors; larger values indicate lower local rotational consistency. We use RA as a consistency filter, while parallax provides the main baseline-related geometric test.

The geometry-aware support is \[\mathcal{S} = \left\{i\;\middle||\mathcal{N}_E(i)|\ge\tau_{\mathrm{nbr}},\quad\mathrm{RA}_i\le\tau_{\mathrm{ra}}, \quad\mathrm{Par}_i\ge\tau_{\mathrm{par}}^{\mathrm{cam}}\right\}.\]

Although similar geometric criteria are often used for keyframe selection in SfM and SLAM, here they serve only to define a stable support for subspace construction. No variables or measurements are removed from the optimization.

3.3.1 CSS Camera-Block Scoring for \(K_t\)↩︎

At iteration \(t\), blocks in \(\mathcal{S}\) are scored on the damped Schur system to select the top-\(k\) set \(K_t\). We partition the \(n_c\)-dimensional camera vector \(\mathbf{c} = [\mathbf{c}_1;\ldots;\mathbf{c}_B]\) into \(B\) camera blocks, where block \(i\) has dimension \(d_i\) and \(n_c = \sum_{i=1}^B d_i\). Let \(P_i \in \mathbb{R}^{n_c \times d_i}\) embed block \(i\) into the full vector.

Let \(\tilde{\mathbf{g}}_{\lambda,i}:=P_i^\top\tilde{\mathbf{g}}_\lambda\) and \(\tilde{H}_{\lambda,ii}:=P_i^\top\tilde{H}_\lambda P_i\) denote the corresponding blocks of the damped Schur system.

For each camera \(i\in\mathcal{S}\), CSS computes \[s_i=\tfrac12\tilde{\mathbf{g}}_{\lambda,i}^{\top}\tilde{H}_{\lambda,ii}^{-1}\tilde{\mathbf{g}}_{\lambda,i}, \label{eq:css:score}\tag{6}\] which is the local predicted decrease under single-block activation for the current damped Schur system.

We choose \(K_t\subseteq\mathcal{S}\) as the top-\(k\) indices by descending score.

3.3.2 Basis Construction from \(K_t\)↩︎

Given \(K_t\), define the block projector \[P_{K_t} \;=\; \sum_{i\in K_t} P_i P_i^\top, \label{eq:css:PK}\tag{7}\] and the localized Schur operator

\[\tilde{H}_{\lambda,K_t}=P_{K_t}\tilde{H}_\lambda P_{K_t}. \label{eq:css:HK}\tag{8}\]

Thus, top-\(k\) selection localizes the operator rather than defining a coordinate-only active-variable set.

Lanczos/Ritz basis. The localized operator \(\tilde{H}_{\lambda,K_t}\) concentrates the Schur curvature on the selected camera blocks. Lanczos is initialized from the negative reduced gradient restricted to the selected support.

Running \(m\)-step Lanczos on \(\tilde{H}_{\lambda,K_t}\) generates an orthonormal Krylov basis \(Q_m \in \mathbb{R}^{n_c \times m}\) and a tridiagonal matrix \(T_m \in \mathbb{R}^{m \times m}\). We take the top-\(k\) eigenvectors \(C_k\) of \(T_m\) and form \[V_K = Q_m C_k \in \mathbb{R}^{n_c\times k}. \label{eq:css:VK}\tag{9}\] These vectors capture the dominant curvature directions within the selected support and provide a compact basis for the LM step before adding the optional complement direction. This converts the selected camera-block support into smooth joint update directions, rather than optimizing the selected cameras independently.

With the default setting, we also consider one complement direction outside the selected support: \[z_g=(I-P_{K_t})(-\tilde{\mathbf{g}}_\lambda). \label{eq:css:vg}\tag{10}\] If \(\|z_g\|_2>\epsilon_g\), we orthogonalize and normalize it against \(V_K\) to obtain \(v_g\) and set \[V_t=[V_K,\;v_g], \label{eq:css:V}\tag{11}\] which has \(k+1\) columns. Otherwise, we set \(V_t=V_K\), which has \(k\) columns. The complete procedure is summarized in Algorithm 2.

This basis \(V_t\) is plugged into 3 to solve the projected LM step and lift a full-sized increment.

3.3.3 Implementation details.↩︎

Our implementation reuses the standard BA components: linearization, LM damping, Schur reduction, and manifold updates. CSS-BA only changes the camera-step subspace. In preprocessing, we construct the geometry-aware support set \(\mathcal{S}\) from the diagnostics \(\mathrm{RA}_i\), \(|\mathcal{N}_E(i)|\), and \(\mathrm{Par}_i\).

During LM iterations, each step scores blocks in \(\mathcal{S}\) by 6 , selects the top-\(k\) set \(K_t\), constructs the Ritz basis, solves the projected Schur-LM system, and lifts the full camera increment.

The method uses runtime hyperparameters \((k_{\max},m)\) and gate thresholds \((\tau_{\mathrm{shared}},\tau_{\mathrm{nbr}},\tau_{\mathrm{ra}},\tau_{\mathrm{par}}^{\mathrm{edge}},\tau_{\mathrm{par}}^{\mathrm{cam}})\). The concrete values used in our experiments are reported in Sec. 4; implementation-level details of prediction safeguards and hybrid fallback are provided in the supplementary material.

4 Experiments↩︎

We evaluate Gate-Guided CSS-BA on two complementary regimes designed to test stability and generality. First, we consider PhoneSweep [26], a low-parallax dataset with near-spherical camera motion, to assess robustness under geometric degeneracy and calibration ambiguity. Second, we evaluate on Bundle Adjustment in the Large (BAL) [13], which contains large-scale Internet-photo reconstructions with more diverse motion, to assess solver behavior beyond near-rotational regimes.

Our implementation is built in C++ on top of the publicly available PoBA codebase [6], ensuring that Schur reduction, linearization, LM damping, and manifold updates are identical across solvers. Unless otherwise stated, all methods share the same trust-region strategy and stopping criteria to isolate differences in search-space design.

Experiments were conducted on Ubuntu 22.04.3 LTS (Linux 5.4.0-190-generic) with dual-socket AMD EPYC 7742 CPUs (256 hardware threads), 1TB RAM, and a single NVIDIA A100-SXM4 GPU (40GB, CUDA 12.2).

4.1 Datasets↩︎

PhoneSweep [26] contains thirteen real-world sequences with near-spherical camera motion (close to pure rotation, extremely low translational parallax), a regime that induces rank deficiency and calibration ambiguity in BA and provides a challenging stress test for solver stability. BAL [13] provides large-scale Internet-photo reconstructions with diverse camera trajectories (ladybug, trafalgar, dubrovnik families), and serves as a complementary generality check under well-conditioned motion.

4.2 Evaluation Metrics↩︎

For PhoneSweep, we report relative rotation and translation accuracies, RRA@\(\tau\) and RTA@\(\tau\), defined as the percentage of camera pairs whose relative rotation or translation angular error is below the threshold \(\tau\in\{5^\circ,15^\circ,30^\circ\}\); higher is better. We also report AUC@30 (higher is better) and Absolute Focal Error \(\mathrm{AFE}=|\hat{f}-f_\mathrm{gt}|/f_\mathrm{gt}\) (lower is better). Formal definitions follow Ventura et al. [26] and are recalled in the supplementary material.

For BAL, we report per-instance objective-improvement statistics \(r_i = f_i^\mathrm{final}/f_i^0\) (smaller is better) and \(\Delta_i = \log_{10}(f_i^0/f_i^\mathrm{final})\) (larger is better), aggregated as family-wise means.

4.3 Experimental Setup↩︎

Common solver settings. We compare three solvers at the bundle adjustment stage: a standard Levenberg–Marquardt baseline (Normal-LM), Power Bundle Adjustment (PoBA) [6], and our CSS-BA. Normal-LM is the classical Schur-reduced LM solver that solves the full damped system 1 at each iteration without any subspace restriction. All solvers operate on the same Schur-reduced camera system.

We use identical trust-region parameters across solvers (initial radius \(40.0\), acceptance threshold \(\eta=0.1\)) and stop when the relative decrease in reprojection cost falls below \(10^{-2}\) or after at most \(150\) iterations.

Unless otherwise noted, we fix \(\tau_{\mathrm{nbr}}=2\) and \(\tau_{\mathrm{par}}^{\mathrm{cam}}=2.0\) across datasets and adapt only the rotation gate to the motion regime.

For PoBA, we use \(20\) power iterations per linear solve. For CSS-BA, we use top-\(k=10\) with \(32\) Lanczos iterations. At each iteration, support cameras are selected from the geometry-aware support \(\mathcal{S}\) using the CSS block-gain score.

PhoneSweep protocol. We evaluate on the iPhone13Mini and Nexus5X subsets. Each sequence is initialized using a general SfM pipeline with GLOMAP [33]. Due to the near-rotational, low-parallax motion pattern, we use a stricter rotation gate (\(\tau_{\mathrm{ra}}=8.0^\circ\)).

The original PhoneSweep benchmark assumes shared intrinsics across views. To isolate solver behavior, our primary comparison among Normal-LM, PoBA, and CSS-BA is conducted without shared intrinsics. We report SphericalSfM+BA only as a reference baseline, not as a directly matched configuration.

BAL protocol. We use the official BAL problem files and preserve the original camera parameterization, 3D points, and observations. All solvers are initialized from the same provided state and optimized with identical trust-region parameters and stopping criteria. Since BAL contains more diverse camera motion than PhoneSweep, we adopt a looser rotation gate (\(\tau_{\mathrm{ra}}=20^\circ\)) to avoid over-pruning informative cameras.

2pt

4pt

5 Results↩︎

5.1 PhoneSweep Results↩︎

Table 1 compares Normal-LM, PoBA, and CSS-BA on PhoneSweep. SphericalSfM+BA with shared intrinsics is shown as a reference.

Across both devices, CSS-BA substantially improves relative-pose accuracy over Normal-LM and PoBA. On iPhone13Mini, AUC@30 rises to \(79.97\) from \(13.34\) (Normal-LM) and \(42.62\) (PoBA). On Nexus5X, the gain is larger: \(83.78\) vs.\(12.06\) and \(33.01\). On Nexus5X, CSS-BA also reduces AFE from \(265.35\)/\(121.18\) to \(0.89\), indicating that the baselines largely fail to recover focal calibration under this extreme degeneracy.

Figure 3 shows that Normal-LM and PoBA often produce collapsed or fragmented trajectories, while CSS-BA recovers the characteristic near-spherical circular layout. CSS-BA operates without shared intrinsics or explicit motion constraints, yet approaches the strongly regularized SphericalSfM+BA baseline.

5.2 BAL Results↩︎

Table 2 shows that CSS-BA obtains objective reductions comparable to Normal-LM and PoBA on BAL across all three families. CSS-BA is not consistently the best solver on these less-degenerate problems, but the results suggest that the subspace restriction does not introduce severe degradation in standard large-scale BA settings. The Venice BAL family is reported in the supplementary material.

5.3 Ablation↩︎

Gate ablation. Table 3 ablates the geometric gate on PhoneSweep (GLOMAP init.). The gate is a stability filter: it helps most on the harder Nexus5X setting, while iPhone13Mini variants are close due to near-saturation. The AUC/AFE ordering is not strictly monotone with gate complexity because the update direction is determined by CSS ranking and the Ritz basis, not by the gate alone.

3.5pt

Top-\(k\) sensitivity. Table 4 reports AUC@30 and AFE for \(k \in \{4, 6, 8, 10, 12, 16\}\) on three PhoneSweep sequences under GLOMAP initialization. Metrics vary little across the range, supporting a fixed default \(k=10\) without per-scene tuning.

2.5pt

Runtime analysis. CSS-BA introduces per-iteration overhead from block scoring, Lanczos basis construction, and the projected solve. Table 5 reports representative wall-clock timings on two PhoneSweep sequences under the same 150-iteration cap. Aggregating these timed cases gives Normal-LM / PoBA / CSS-BA \(\approx\) 6495 / 1930 / 13142 s, so CSS-BA costs about \(2.0\times\) Normal-LM and \(6.8\times\) PoBA on this weak-geometry timing subset. Together with the accuracy gains reported above, this shows that CSS-BA trades throughput for improved pose stability in the targeted ill-conditioned regime. Calibration gains are strongest on Nexus5X and remain dataset-dependent.

On BAL (Table 2), the results show comparable objective improvement; the added subspace construction overhead is therefore mainly justified when weak geometry makes accuracy the primary concern.

3.5pt

6 Conclusion and Limitations↩︎

We introduced Gate-Guided CSS-BA, a solver-side Column Space Search variant of Schur-LM tailored to bundle adjustment under weak geometric conditioning. Operating on the Schur-reduced camera system, the method combines geometry-aware support selection with a subspace-restricted update while preserving the original objective, variable set, and trust-region mechanism.

On near-spherical, low-parallax PhoneSweep sequences, CSS-BA improves relative pose accuracy over standard LM and PoBA and, in the Nexus5X case, substantially reduces focal-length error. It approaches the strongly regularized SphericalSfM+BA reference in relative-pose and trajectory-layout metrics without shared intrinsics or explicit motion priors.

Because CSS-BA only changes the LM update subspace, it is compatible with standard robust estimators and outlier handling. Its main benefit is in ill-conditioned regimes where classical Schur-LM is prone to poorly supported updates; in well-conditioned cases, gains are more modest and subspace construction adds overhead. Adaptive gating and subspace sizing remain future work.

This supplementary material provides evaluation metric definitions, additional implementation details, robustness checks, and failure-case analyses.

7 Evaluation Metric Definitions↩︎

Relative pose errors. For a camera pair \((i,k)\), let \(R_{ik}=R_kR_i^\top\) be the ground-truth relative rotation and \(\mathbf{t}_{ik}=R_i^\top(\mathbf{t}_k-\mathbf{t}_i)/\|R_i^\top(\mathbf{t}_k-\mathbf{t}_i)\|\) be the ground-truth relative translation direction. We first define the angular errors \[\begin{align} e_R(i,k)&=\arccos\!\left(\operatorname{clamp}\left(\frac{\mathrm{tr}(R_{ik}^\top\hat{R}_{ik})-1}{2},-1,1\right)\right),\\ e_T(i,k)&=\arccos\!\left(\operatorname{clamp}\left(\mathbf{t}_{ik}^\top\hat{\mathbf{t}}_{ik},-1,1\right)\right). \end{align}\]

RRA / RTA. For a threshold \(\tau\), the reported relative rotation and translation accuracies are \[\mathrm{RRA}@\tau=\frac{100}{|\mathcal{P}|}\sum_{(i,k)\in\mathcal{P}}\mathbf{1}[e_R(i,k)\le\tau],\quad \mathrm{RTA}@\tau=\frac{100}{|\mathcal{P}|}\sum_{(i,k)\in\mathcal{P}}\mathbf{1}[e_T(i,k)\le\tau].\]

AUC@30. For each threshold \(\tau\), define the joint pose accuracy \[\mathrm{Acc}(\tau)=\frac{100}{|\mathcal{P}|}\sum_{(i,k)\in\mathcal{P}}\mathbf{1}[e_R(i,k)\le\tau\wedge e_T(i,k)\le\tau].\] AUC@30 is the normalized area under \(\mathrm{Acc}(\tau)\) over \(\tau\in[0,30^\circ]\).

AFE. We define the absolute focal error as \(\mathrm{AFE}=|\hat{f}-f_{\mathrm{gt}}|/f_{\mathrm{gt}}\), following Ventura et al. [26].

8 PhoneSweep Results with SphericalSfM Initialization↩︎

In the main paper, we evaluate CSS-BA on PhoneSweep using a general SfM initialization obtained with GLOMAP. As a complementary experiment, we additionally evaluate a spherical-motion-specific initialization produced by SphericalSfM [26] before bundle adjustment. This setting is better aligned with the near-spherical motion of PhoneSweep and allows us to test whether the proposed solver remains effective under a stronger motion-consistent initialization.

For this complementary SphericalSfM-initialized experiment, we use the following device-specific gate thresholds to account for the different PhoneSweep camera characteristics.

For Nexus5X, we use \(\tau_{\mathrm{nbr}}=2\), \(\tau_{\mathrm{par}}^{\mathrm{cam}}=1.8\), and \(\tau_{\mathrm{ra}}=7.1^\circ\). For iPhone13Mini, we use \(\tau_{\mathrm{nbr}}=2\), \(\tau_{\mathrm{par}}^{\mathrm{cam}}=0.8\), and \(\tau_{\mathrm{ra}}=7.1^\circ\). All other settings, including the top-\(k\) budget \(k_{\max}=10\) and the Lanczos iteration count \(m=32\), remain unchanged.

Quantitative results. 6 summarizes the quantitative results.

CSS-BA again improves relative-pose accuracy over Normal-LM and PoBA on both devices and reduces AFE among the three BA solvers. The SphericalSfM+BA reference remains stronger in focal calibration because it uses the shared-intrinsic assumption. These gains are obtained without the shared-intrinsic assumption used by the SphericalSfM+BA reference baseline, so the comparison still reflects a more general bundle-adjustment setting.

At the same time, the translation results suggest a more nuanced picture than rotation alone. For both devices, CSS-BA approaches the reference baseline much more closely at coarse angular tolerances (e.g., RTA@30) than at stricter ones (e.g., RTA@5), indicating that the translational geometry is largely recovered at a coarse level while fine-grained translation-direction accuracy remains harder to obtain. This behavior is consistent with the extremely weak-parallax nature of PhoneSweep.

Qualitative results. 4 provides a qualitative comparison on representative PhoneSweep sequences under SphericalSfM initialization. CSS-BA recovers outward-facing trajectories that remain close to the ground-truth circular layout.

3pt

9 Additional BAL Results on the Venice Family↩︎

To further assess solver behavior beyond the BAL families reported in the main paper, we additionally evaluate on the Venice family, which contains larger problem instances.

7 summarizes the results on a fixed subset of 12 Venice problems. CSS-BA remains competitive with Normal-LM and PoBA in this larger-scale setting. Although the gains are smaller than in the strongly degenerate PhoneSweep regime, the results indicate that the proposed solver does not break down on larger BAL instances and remains comparable to conventional solvers under more general geometry.

10 Failure Analysis of GLOMAP Initialization on PhoneSweep↩︎

To better understand the PhoneSweep results under GLOMAP initialization reported in the main paper, we inspect representative failure cases of the initialization itself. As illustrated in 5, some outward-looking sequences are incorrectly initialized as inward-facing, indicating that the initializer can produce a globally flipped camera rig before any downstream bundle adjustment is applied.

This observation suggests that part of the degradation in the GLOMAP-initialized setting originates from the initializer rather than from the downstream BA solver. To isolate this effect, we additionally report a filtered analysis in which sequences with clear outward/inward inversion failures are excluded. The resulting averages are shown in 8.

In this filtered setting, CSS-BA still matches or surpasses Normal-LM and PoBA across the reported pose and calibration metrics.

3pt

11 Additional Implementation Details↩︎

This section provides implementation details that are omitted from the main paper for brevity. Unless otherwise noted, the runtime uses a support size of \(k=10\) and \(32\) Lanczos iterations for the Ritz subspace construction. At each iteration, the solver scores blocks in the geometry-aware support \(\mathcal{S}\) and selects the top-\(k\) support \(K_t\) used to build the projected search subspace.

Unless otherwise noted, the runtime includes prediction safeguards and a hybrid fallback. These mechanisms are triggered only when the projected CSS step fails to provide a sufficiently reliable predicted decrease, and therefore do not affect ordinary successful iterations. Experiment-specific exceptions are stated in the corresponding sections.

Unless otherwise noted, experiments use the default gate configuration described in the main paper, and any experiment-specific threshold changes are stated explicitly in the corresponding section.

The gate restricts only which camera blocks are eligible for scoring and selection into the CSS subspace; it does not directly constrain the final search direction or determine which variables are allowed to move. After the projected solve, the recovered camera increment remains full-sized, so camera blocks outside the gate or outside the selected support can still receive nonzero updates.

References↩︎