1 Introduction↩︎

Vision-centric autonomous driving has emerged as a scalable and cost-effective paradigm for perception, thereby garnering escalating scholarly and industrial interest [1]–[5]. A fundamental challenge in this field is recovering accurate metric scene geometry and ego-motion from synchronized multi-camera observations under rigid camera configurations. Accurate geometric understanding is essential for downstream applications such as planning [2], simulation [6], and online HD map construction [7]. Recent visual geometry models have demonstrated impressive capabilities in camera pose estimation, depth prediction, and 3D reconstruction from image sequences [8]–[10]. However, directly applying these models to autonomous driving remains challenging due to rigid multi-camera configurations and the requirement for accurate metric geometry.

Despite substantial progress, reliable metric geometry estimation remains challenging for multi-camera autonomous driving. Existing methods are largely image-driven, using vehicle-side geometry, such as odometry and camera-rig calibration, only as auxiliary inputs or supervision rather than explicit representations for reasoning [8], [10]–[12]. As a result, metric scale is implicitly inferred from visual appearance, causing scale ambiguity and limited geometric accuracy. Recent geometry-prior methods [13]–[15] are mainly designed for generic visual scenes and treat images as independent observations, thus under-exploiting the structural constraints of synchronized rigid camera rigs. A camera pose in autonomous driving naturally consists of a time-varying ego trajectory and a static camera-rig configuration. However, existing methods typically entangle them into a unified pose representation, making it difficult to separately exploit motion dynamics and camera topology. Such entanglement further leads to costly global reasoning or post-processing, e.g., \(\mathrm{Sim}(3)\) alignment, for consistent metric-scale reconstruction [16], [17].

Motivated by this observation, we propose Trajectory-Rig Decoupled Metric Geometry Learning (TRIG), a metric-aware perception framework for multi-camera autonomous driving. TRIG explicitly decouples time-varying ego trajectories from static camera-rig configurations, allowing motion dynamics and camera topology to be modeled independently yet optimized jointly. This decoupling naturally supports an asymmetric temporal–spatial design. The model continuously aggregates trajectory cues but only sparsely injects rig constraints, leading to better scalability and computational efficiency.

Our contributions are three-fold:

• We propose TRIG, a trajectory-rig decoupled framework that leverages vehicle-side geometric priors for direct metric-coordinate geometry learning in multi-camera autonomous driving.

• We develop a decoupled pose modeling strategy, including pose encoding and pose supervision, to separately handle ego-motion and camera-rig geometry, improving metric scale consistency, pose stability, and global reconstruction quality.

• We introduce Sparse Temporal–Spatial Attention (STSA), which performs continuous trajectory aggregation and sparse rig-constraint injection for scalable cross-camera geometric reasoning. Experiments on five benchmarks show that TRIG achieves state-of-the-art results on metric depth prediction, 3D reconstruction, and ego-pose estimation.

2 Related Work↩︎

2.1 Visual Geometry Learning for Autonomous Driving↩︎

Vision-based 3D geometry perception has become an essential component of autonomous driving systems, aiming to recover scene structures from multi-camera observations. Early approaches mainly focused on depth estimation from monocular or multi-view images, providing dense depth maps but only partial 3D understanding of the scene geometry [18]–[20]. To achieve a more comprehensive representation, recent methods have explored bird’s-eye-view (BEV) perception and 3D occupancy prediction, where the environment is represented in a unified 3D space using voxel-based formulations. These approaches have demonstrated strong performance in autonomous driving perception tasks, including semantic occupancy and scene understanding [1], [21], [22].

However, voxel-based representations suffer from discretization errors and limited spatial resolution, which makes it challenging to accurately model fine-grained geometric structures. Recently, general visual geometry models have advanced 3D reconstruction by predicting dense point maps from image sequences [8], [9], [11], [12], [23]. Despite their impressive reconstruction capability, most existing approaches estimate geometry in relative coordinate systems and require additional scale recovery [8], [10]–[13], [23]. In contrast, TRIG focuses on directly learning metric geometry for rigid multi-camera driving systems by explicitly leveraging vehicle-side geometric priors.

2.2 Geometry-aware Visual Models with Camera Priors↩︎

Existing autonomous driving perception methods commonly exploit calibrated camera geometry through explicit projection using camera intrinsics and extrinsics. Conventional autonomous driving perception methods typically rely on explicit camera calibration and geometric projection, such as transforming image features into BEV space using intrinsics and extrinsics parameters [1], [20]. While effective under fixed camera configurations, these approaches often depend heavily on camera-specific assumptions, limiting their generalization to different sensor setups [24].

Recent visual geometry models have explored learning camera poses, depth, and 3D structures jointly from image sequences [8]–[10], [23]. Some approaches further incorporate geometric priors, such as camera poses, depth constraints, or external measurements, to improve reconstruction quality [13], [14], [25]. However, most existing methods treat camera poses as a unified geometric condition, without explicitly separating different sources of geometric variation. In multi-camera autonomous driving systems, camera poses naturally contain both time-varying ego-motion trajectories and camera-rig configurations, which provide complementary geometric information. TRIG explicitly decouples these two factors to better exploit vehicle-side priors for metric geometry learning.

Moreover, existing visual geometry models commonly rely on global self-attention to capture interactions among multi-view and multi-frame inputs, resulting in quadratic computational complexity with increasing input size. Recent works explore efficient attention or streaming strategies to improve scalability [11], [16], [17], [26]. Different from these approaches, TRIG introduces a sparse temporal–spatial attention mechanism that separates cross-camera spatial interaction from temporal feature aggregation, reducing computational overhead while maintaining effective geometric reasoning.

3 Preliminary↩︎

We adopt VGGT [8] as the basic visual geometry foundation model, which predicts camera parameters and dense scene geometry from multi-view images in a feed-forward manner. Given an image set \(\mathcal{I}\), a visual encoder first extracts image features and converts them into spatial tokens \(\mathbf{f}\). These tokens are combined with learnable camera tokens \(\mathbf{c}\) and register tokens \(\mathbf{r}\) to form the input representation of the transformer encoder:

\[(\hat{\mathbf{c}}, \hat{\mathbf{r}}, \hat{\mathbf{f}})=E(\mathbf{c},\mathbf{r},\mathbf{f}).\]

The encoder is constructed upon the Alternating Attention (AA) mechanism, which performs local and global feature interactions to capture both intra-view structures and cross-view geometric relationships. After several AA blocks, the refined spatial tokens are decoded by dense prediction heads to estimate depth maps and 3D point maps, while the camera tokens are used to regress camera intrinsics and extrinsics. Through this unified architecture, VGGT achieves joint estimation of scene geometry and camera motion from image sequences.

4 Methodology↩︎

This section details TRIG, a trajectory-rig decoupled framework for multi-camera geometry learning: 1) decoupled pose encoding for metric-aware geometry learning, 2) sparse temporal–spatial attention for efficient long-horizon reconstruction, and 3) decoupled pose supervision for decoupling motion and rig constraints during training.

4.1 Overview of TRIG↩︎

TRIG targets multi-camera geometry learning from a temporal sequence of synchronized surround-view images. Given an input clip \(\mathcal{I}=\{I_t^c \mid t=1,\dots,T,\;c=1,\dots,C\}\), where \(T\) and \(C\) denote the number of frames and cameras, respectively, each image \(I_t^c\in\mathbb{R}^{3\times H\times W}\) is associated with a camera pose \(P_{w \leftarrow c}^{t}\in SE(3)\). Here, \(P_{w \leftarrow c}^{t}\) denotes the pose of camera \(c\) in the world coordinate system at time \(t\). Instead of using \(P_{w \leftarrow c}^{t}\) as a single entangled condition, TRIG factorizes it into an ego-trajectory component and a camera-rig component:

\[P_{w \leftarrow c}^{t} = P_{w \leftarrow \mathrm{ego}}^{t} P_{\mathrm{ego} \leftarrow c}^{c} , \label{eq:pose95transform}\tag{1}\]

where \(P_{w \leftarrow ego}^{t}\in SE(3)\) is the ego-vehicle pose in the world coordinate system at time \(t\), and \(P_{ego \leftarrow c}^{c}\in SE(3)\) is the fixed extrinsic transform from camera \(c\) to the ego-vehicle coordinate system.

As shown in Fig. 2, the pose factors are encoded into trajectory and rig priors, \[z_{\mathrm{traj}}^t = \phi_{\mathrm{traj}}(P_{w \leftarrow \mathrm{ego}}^{t}), \qquad z_{\mathrm{rig}}^c = \phi_{\mathrm{rig}}(P_{\mathrm{ego} \leftarrow c}^{c} )\]

and injected into the image tokens extracted by a frozen visual backbone. TRIG alternates Rig Blocks and Traj Blocks to separately model cross-camera spatial constraints and cross-frame temporal motion cues. This decoupled design enables sparse temporal–spatial interaction over the token set \[\mathcal{X} = \{x_t^c\} \cup \{z_{\mathrm{rig}}^c\} \cup \{z_{\mathrm{traj}}^t\}.\] where \(x_t^c\) denotes the image tokens of view \(c\) at time \(t\). The resulting features \(\mathcal{X}\) are fed into the original VGGT DPT head and camera head for depth and camera prediction, respectively. These heads are kept unchanged and thus omitted in Fig. 2 for clarity.

4.2 Decoupled Pose Encoding↩︎

TRIG factorizes multi-camera poses into two complementary components: a temporal ego-motion trajectory and a static camera-rig configuration. Given the input pose sequence \(\mathbf{P}\in\mathbb{R}^{T\times N_C\times 9}\), we use the reference camera to represent the shared trajectory, i.e., \(\mathbf{P}_{\mathrm{traj}}=\{P^{t,0}\}_{t=1}^{T}\), and encode it as \(z_{\mathrm{traj}}^{t}=\mathrm{MLP}_{\mathrm{traj}}(P^{t,0})\). Similarly, the camera rig is extracted from the reference timestamp as \(\mathbf{P}_{\mathrm{rig}}=\{P^{0,c}\}_{c=1}^{N_C}\), and each camera-specific configuration is encoded by \(z_{\mathrm{rig}}^{c}=\mathrm{MLP}_{\mathrm{rig}}(P^{0,c})\). The final geometry-aware pose embedding is obtained by additive composition: \[z_{\mathrm{geo}}^{t,c}=z_{\mathrm{traj}}^{t}+z_{\mathrm{rig}}^{c}.\] This decomposition separates dynamic ego-motion from static multi-camera geometry, yielding structured geometric priors for surround-view reasoning.

Instead of replacing the original VGGT camera tokens, we inject the decoupled pose representation through residual adaptation. Specifically, the geometry feature is projected by a lightweight adapter and added to the original camera token: \[\tilde{z}_{\mathrm{cam}}^{t,c} = z_{\mathrm{cam}}^{t,c} + \mathrm{MLP}_{\mathrm{adapter}}(z_{\mathrm{geo}}^{t,c}),\] where \(z_{\mathrm{cam}}^{t,c}\) denotes the camera token produced by VGGT. Following OmniVGGT [13], the adapted camera tokens are inserted into each frame attention block during transformer refinement, enabling progressive geometric conditioning while preserving the original visual representation.

3pt

3pt

4.3 Sparse Temporal–Spatial Attention↩︎

Given the decoupled pose tokens, TRIG introduces a sparse temporal–spatial attention mechanism to aggregate geometric cues from long-horizon synchronized multi-camera sequences. Let \(\mathcal{X}\in\mathbb{R}^{B\times S\times P\times D}\) denote the input token tensor, where \(S=T\times N_C\), \(P\) is the number of tokens per observation, and \(D\) is the token dimension. Each observation contains image tokens and the injected decoupled pose tokens. In contrast to applying dense self-attention over all tokens at every layer, TRIG factorizes token interactions into three structured patterns: frame attention, global attention, and temporal attention, which respectively model local visual refinement, cross-camera communication, and ego-motion consistency.

4.3.0.1 Structured attention.

Frame attention performs self-attention independently within each camera observation by reshaping \(\mathcal{X}\) into \((BS)\times P\times D\). This operation refines local image and pose tokens without mixing information across different cameras or timestamps. Global attention flattens all observations and tokens into \(B\times (SP)\times D\), enabling communication across all frames and cameras. It provides the interaction path required for cross-camera geometric consistency. Temporal attention first reshapes \(\mathcal{X}\) into \(B\times T\times N_C\times P\times D\), and then applies self-attention along each camera stream with shape \((BN_C)\times (TP)\times D\). This design aggregates long-range temporal cues while preserving camera-specific viewing directions.

4.3.0.2 Rig and trajectory blocks.

Based on these attention patterns, we define two structured blocks. The Rig Block applies frame attention followed by global attention, i.e., \(\mathrm{RigBlock}(\mathcal{X})=\mathrm{GlobalAttn}(\mathrm{FrameAttn}(\mathcal{X}))\). Since the token set contains rig-aware pose tokens, the global interaction is conditioned on the fixed camera layout, encouraging cross-view geometric consistency. The Traj Block applies frame attention followed by temporal attention, i.e., \(\mathrm{TrajBlock}(\mathcal{X})=\mathrm{TempAttn}(\mathrm{FrameAttn}(\mathcal{X}))\). Unlike the Rig Block, the Traj Block constrains long-range aggregation to intra-camera observations, enabling temporal reasoning under ego-motion while preserving view-specific geometry.

4.3.0.3 Sparse temporal–spatial composition.

The transformer employs Traj Blocks across all layers to facilitate continuous temporal modeling, while sparsely interleaving Rig Blocks to enable global multi-camera interaction. Following the design of VGGT[17], we inject Rig Blocks at indices \(\mathcal{L}_{\mathrm{rig}} = \{0, 4, 11, 17, 23\}\) within our 24-layer transformer. The layer update is formulated as: \[\mathcal{X}^{l+1} = \begin{cases} \mathrm{RigBlock}^{l}\bigl(\mathrm{TrajBlock}^{l}(\mathcal{X}^{l})\bigr), & l\in\mathcal{L}_{\mathrm{rig}}, \\ \mathrm{TrajBlock}^{l}(\mathcal{X}^{l}), & l\notin\mathcal{L}_{\mathrm{rig}}, \end{cases}\] This sparse composition effectively decouples cross-camera spatial interaction from temporal feature aggregation, preserving essential geometric information flow while avoiding the prohibitive computational cost of global attention at every layer. Overall, this sparse composition separates cross-camera spatial interaction from temporal feature aggregation, thereby preserving essential geometric interaction paths while reducing unnecessary dense token interactions.

4.4 Decoupled Pose Supervision↩︎

To align the training objective with the trajectory-rig factorization of TRIG, we supervise relative poses instead of absolute camera poses. Relative transformations are invariant to the global coordinate frame and directly capture geometric consistency among observations. We divide relative pose pairs into two groups: cross-camera pairs at the same timestamp for rig supervision, and cross-time pairs within the same camera stream for trajectory supervision.

3pt 3pt 3pt

4.4.0.1 Rig supervision.

Rig supervision constrains the relative geometry between cameras at the same timestamp. For a camera pair \((c_i,c_j)\) at time \(t\), the predicted and ground-truth relative transformations are computed as \[\begin{align} \hat{T}^{(k),t}_{c_i\leftarrow c_j} &= \hat{T}^{(k),t,c_i}_{w\rightarrow c} \left( \hat{T}^{(k),t,c_j}_{w\rightarrow c} \right)^{-1}, \\ T^{t}_{c_i\leftarrow c_j} &= T^{t,c_i}_{w\rightarrow c} \left( T^{t,c_j}_{w\rightarrow c} \right)^{-1}. \end{align}\] Given that cameras share an identical ego pose at each timestamp, this term primarily constrains the fixed rig layout and promotes cross-camera consistency.

4.4.0.2 Trajectory supervision.

Trajectory supervision models ego-motion along each camera stream. For camera \(c\) and timestamps \((t_i,t_j)\), we define \[\begin{align} \hat{T}^{(k),c}_{t_i\leftarrow t_j} &= \hat{T}^{(k),t_i,c}_{w\rightarrow c} \left( \hat{T}^{(k),t_j,c}_{w\rightarrow c} \right)^{-1}, \\ T^{c}_{t_i\leftarrow t_j} &= T^{t_i,c}_{w\rightarrow c} \left( T^{t_j,c}_{w\rightarrow c} \right)^{-1}. \end{align}\] With fixed camera identity, this relative transformation mainly reflects ego-vehicle motion across time. For both rig and trajectory pairs, we adopt the relative pose loss from \(\pi^3\) [10], which combines translation regression and rotation geodesic error on \(SO(3)\). Unlike \(\pi^3\)’s unified relative pose supervision, TRIG partitions relative pairs according to the trajectory-rig decomposition, yielding \(L_{\mathrm{rig}}\) for cross-camera geometry and \(L_{\mathrm{traj}}\) for ego-motion.

Following VGGT, we also supervise the focal components in the predicted pose encoding and apply all pose losses to each refinement stage with stage-dependent weights. The final pose objective combines the rig, trajectory, and focal losses. This decoupled pose supervision explicitly separates cross-camera rig constraints from temporal ego-motion constraints, improving geometric consistency over unified pose supervision.

4pt

5pt

4pt

5 Experiments↩︎

We conduct extensive experiments to evaluate the performance of TRIG. To ensure fair comparison, we follow the experimental setup of DVGT, including benchmark selection and evaluation protocols. Unless otherwise specified, baseline results are taken from the DVGT paper, and TRIG is evaluated under the same setting. Detailed dataset statistics, implementation hyperparameters, and formal metric definitions are provided in the Appendix.

5.1 3D Reconstruction and Metric Depth↩︎

3D Reconstruction. As summarized in Table 1, TRIG establishes a new state-of-the-art in world-coordinate 3D Accuracy and Completeness across all five benchmarks. It consistently outperforms both general visual geometry models (e.g., CUT3R [28], VGGT [8]) and driving-specific architectures (e.g., DVGT [12]). This quantitative dominance is visually corroborated in Fig. 3, where TRIG yields significantly sharper geometries and cleaner boundaries than DVGT.

Crucially, unlike image-only foundation models that require post-hoc Umeyama [27] alignment to recover scale, TRIG leverages metric pose priors to define a deterministic scale reference, allowing its normalized predictions to be directly restored to metric scale without additional alignment. While recent prior-guided models (OmniVGGT [13], MapAnything [14]) attempt similar metric-aware predictions, they degrade severely in complex multi-camera setups (e.g., MapAnything’s 4.303 Completeness error on OpenScene). This exposes the vulnerability of entangled priors in wide-baseline surround-view configurations with impoverished appearance cues. By routing temporal ego-motion and static camera-rig topology through isolated pathways, TRIG constructs a stable metric reference frame, ensuring geometric stability where purely visual evidence falls short.

Metric Depth. Table 2 demonstrates that the structural advantages of our decoupled representation seamlessly transfer to dense depth estimation. TRIG dominates the primary metric error (Abs Rel) and inlier ratio (\(\delta_{1.25}\)), achieving substantial margins over DVGT (the strongest prior-free baseline) on highly dynamic datasets like KITTI and NuScenes.

On Waymo, while TRIG yields a marginally lower strict \(\delta_{1.25}\) than DVGT, it maintains a superior global Abs Rel. Because the hard-threshold \(\delta_{1.25}\) is disproportionately sensitive to isolated boundary artifacts, our consistently lower overall Abs Rel and 3D Completeness (Table 1) firmly prove that TRIG achieves a robust global metric consensus rather than overfitting to specific local thresholds. Ultimately, this joint improvement in depth and reconstruction confirms that decoupling dynamic and static poses is a potent geometric catalyst.

Ego-Pose Estimation. Following VGGT [8], we evaluate relative camera pose accuracy using AUC@\(30^\circ\). Table 3 shows TRIG secures the highest pose accuracy across KITTI, NuScenes, Waymo, and DDAD. Compared to DVGT, TRIG yields profound improvements (e.g., elevating KITTI from 87.6 to 99.1). Furthermore, TRIG circumvents the instability exhibited by recent prior-guided models. Whereas entangled prior injection causes OmniVGGT to degrade severely under wide-baseline configurations—dropping to 38.5 AUC on NuScenes—TRIG maintains a substantially higher AUC of 96.6.

These gains are critical for autonomous driving, where drifting poses render downstream reasoning unusable. Rooted in our decoupled supervision strategy, TRIG isolates inter-frame temporal constraints (ego-trajectory) from intra-frame spatial constraints (camera-rig). This explicitly optimizes ego-motion without corrupting the static surround-view layout. Consequently, TRIG delivers highly accurate, drift-resistant metric poses, ensuring the reconstructed scene is directly actionable in a driving-oriented 3D coordinate system.

5.2 Ablation Studies↩︎

We conduct extensive ablations across the DDAD, KITTI, NuScenes, and Waymo benchmarks to validate TRIG’s core architectural propositions. As summarized in Tables ¿tbl:tab:ablation?,  ¿tbl:tab:ablation95pose? and  5, we jointly assess metric depth, 3D reconstruction, and ego-pose estimation to capture holistic geometric impacts.

Sparse Temporal–Spatial Attention. We evaluate Sparse Temporal–Spatial Attention (STSA) with four variants: w/o Rig block, Frame-only, Global-only, and 3-layer rig block, where Rig blocks are inserted at layers 0, 11, and 23. As shown in Table ¿tbl:tab:ablation?, removing Rig blocks increases the reconstruction Accuracy error from 0.411 to 0.881. Frame-only performs even worse, suggesting that local refinement alone is insufficient for preserving global geometry. Although Global-only and 3-layer rig block improve upon these weaker baselines, both remain inferior to full STSA, highlighting the importance of jointly modeling temporal and cross-view interactions. Table 5 further demonstrates consistent improvements across DDAD, KITTI, NuScenes, and Waymo. In addition, Table 4 shows that STSA is more efficient than VGGT-style attention, achieving \(1.43\times\) and \(1.72\times\) speedups on \(6{\times}21\) and \(6{\times}40\) NuScenes inputs.

Decoupled Prior Encoding. To demonstrate the superiority of our decoupled strategy, we replace our distinct trajectory and rig embeddings with the entangled OmniVGGT pose prior. Table ¿tbl:tab:ablation? shows marked deterioration across all metrics, with point completeness worsening from 0.577 to 0.691. These results provide empirical support for our central hypothesis: ego-motion and camera-rig calibration represent intrinsically heterogeneous sources of information that warrant separate modeling. By routing them through decoupled pathways, TRIG smoothly integrates multi-modal geometric constraints without corrupting the foundation model’s original feature space.

Decoupled Pose Supervision. To assess our decoupled pose supervision, we replace it with the standard VGGT’s pose loss. As shown in Tables ¿tbl:tab:ablation? and ¿tbl:tab:ablation95pose?, this replacement consistently degrades both pose accuracy and geometric quality. Our full model substantially improves Pose AUC under all angular thresholds, especially from 76.33 to 86.92 at \(5^\circ\), indicating stronger resistance to pose drift. Meanwhile, it also improves dense depth and reconstruction, reducing Abs Rel from 0.076 to 0.069 and Completeness error from 0.722 to 0.577. These results indicate that separately supervising temporal ego-motion and spatial rig constraints is more effective than using a VGGT pose loss.

6 Conclusion↩︎

In this paper, we introduced TRIG, a novel metric geometry perception framework for multi-camera autonomous driving. By explicitly decoupling dynamic ego-trajectories from static camera-rig topologies, TRIG overcomes the geometric entanglement and scale ambiguity inherent in existing visual foundation models. To achieve this, we proposed a decoupled prior encoding module, a highly efficient Sparse Temporal–Spatial Attention (STSA) mechanism, and a decoupled pose supervision strategy. Extensive experiments across five large-scale driving benchmarks demonstrate that TRIG establishes new state-of-the-art standards in metric depth prediction, 3D scene reconstruction, and ego-pose estimation, providing a robust, scale-deterministic solution for end-to-end 3D scene understanding.

7 Dataset Details↩︎

We train our model on 5 public autonomous driving datasets. All datasets provide calibrated camera intrinsics and extrinsics, along with ground-truth depth and pose annotations. Table 6 summarizes the key statistics of each dataset.

8 Additional Implementation Details↩︎

8.1 Training Objective↩︎

The full training objective of TRIG consists of three components: a depth loss, a point map loss, and the Trajectory–Rig Decoupled pose loss: \[\mathcal{L} = \mathcal{L}_{\mathrm{depth}} + \mathcal{L}_{\mathrm{pmap}} + \mathcal{L}_{\mathrm{pose}},\] where the pose loss is further decomposed into a trajectory loss and a rig loss: \[\mathcal{L}_{\mathrm{pose}} = \mathcal{L}_{\mathrm{traj}} + \mathcal{L}_{\mathrm{rig}}.\] The trajectory loss supervises the inter-frame relative poses of the front camera, while the rig loss supervises the inter-camera relative poses at the reference timestamp \(t{=}0\). This decoupled formulation encourages the model to separately learn temporal ego-motion and static multi-camera rig geometry, reducing the ambiguity between vehicle motion and camera extrinsics.

8.2 Model Architecture↩︎

TRIG adopts a ViT-L backbone pretrained with DINOv2 as the image encoder. The image encoder is kept frozen during training to preserve the general-purpose visual representation learned from large-scale pretraining and to reduce the optimization cost. Given an input sequence with \(T\) timestamps and \(N_C\) cameras, the images are first encoded into visual tokens by the frozen ViT-L encoder.

To inject explicit pose-aware structure into the network, we separately encode the trajectory pose and the rig pose using two linear projection layers. Both projections map their corresponding pose representations into a 1024-dimensional latent space. The trajectory embedding has shape \((T, 1, 1024)\), while the rig embedding has shape \((1, N_C, 1024)\). We then combine them through broadcasting and summation: \[\mathbf{E}_{\mathrm{pose}} = \mathbf{E}_{\mathrm{traj}} + \mathbf{E}_{\mathrm{rig}},\] resulting in a pose embedding of shape \((T, N_C, 1024)\). This embedding provides each camera-view token with both temporal trajectory information and camera-specific rig information.

On top of the image encoder, we employ the proposed Sparse Temporal–Spatial Attention (STSA) module for efficient feature interaction across time and camera views. STSA consists of two types of transformer blocks:

• Trajectory Block. The trajectory block is the main building block of STSA and is repeated for 24 layers. Each trajectory block first applies frame-level self-attention to aggregate information within each timestamp, and then performs per-camera temporal attention to model ego-motion over time. This design captures temporal dynamics efficiently while avoiding the quadratic cost of full spatio-temporal attention over all frames and cameras.

• Rig Block. To model cross-camera geometry, we insert rig blocks at selected layers, specifically at indices \(\{0,4,11,17,23\}\). Each rig block performs global cross-view attention across cameras, enabling the network to aggregate multi-camera spatial information and reason about the static camera rig configuration.

During STSA processing, the pose prior is injected into the frame-level attention of each layer, following the spirit of OmniVGGT. This allows the model to condition feature interaction on the decoupled trajectory and rig embeddings, thereby improving its ability to distinguish temporal motion from multi-camera geometric structure.

For dense prediction, we follow the design of VGGT and feed intermediate tokens from the 4th, 11th, 17th, and 23rd transformer blocks into a DPT decoder for upsampling. The DPT decoder progressively fuses multi-scale transformer features and produces dense outputs such as depth maps and point maps.

3pt

8.3 Training Details↩︎

TRIG is trained on multiple public datasets for 40K iterations. We follow the data preprocessing protocol of DVGT. All input images are resized to \(518 \times 294\), and the patch size of the ViT encoder is \(14 \times 14\).

We use AdamW as the optimizer with a base learning rate of \(1 \times 10^{-4}\), weight decay of \(5 \times 10^{-2}\), and betas of \((0.9, 0.95)\). Bias and normalization parameters are excluded from weight decay. The learning rate is scheduled by OneCycleLR with cosine annealing, where the warm-up phase takes the first \(10\%\) of training iterations. The initial and final learning-rate division factors are both set to 100.

Training is performed with DeepSpeed ZeRO-1 optimization and bfloat16 mixed precision. We apply gradient norm clipping with a threshold of 1.0 for stable optimization. The model is trained on 64 NVIDIA H20 GPUs for approximately six days.

9 Evaluation Metrics↩︎

We evaluate our method’s performance on three primary tasks: 3D point map reconstruction, metric depth estimation, and ego-pose estimation.

3D Point Map Reconstruction. Following prior works, we measure overall geometric quality using Accuracy and Completeness. Let \(\mathcal{P} = \{p_i\}\) and \(\mathcal{G} = \{g_j\}\) denote the sets of valid points from the predicted point map \(\hat{P}\) and the ground truth \(P\), respectively. These metrics are calculated as: \[\begin{align} \mathrm{Accuracy} &= \frac{1}{|\mathcal{P}|} \sum_{p \in \mathcal{P}} \min_{g \in \mathcal{G}} \|p - g\|_2, \tag{2} \\ \mathrm{Completeness} &= \frac{1}{|\mathcal{G}|} \sum_{g \in \mathcal{G}} \min_{p \in \mathcal{P}} \|g - p\|_2, \tag{3} \end{align}\] where \(\|\cdot\|_2\) denotes the \(L_2\) distance. Accuracy measures how close each predicted point is to the nearest ground-truth point, reflecting reconstruction precision; Completeness measures how well the ground-truth geometry is covered by the prediction, indicating reconstruction recall. Both metrics are evaluated in the world coordinate frame at metric scale, without any post-hoc Sim(3) alignment.

Metric Depth Estimation. We evaluate per-camera depth using the Absolute Relative Error (Abs Rel) and the inlier ratio \(\delta_{1.25}\). Let \(\Omega\) be the set of valid pixels with available ground truth, and let \(u\) index a pixel within \(\Omega\). Given the predicted metric depth \(D(u)\) and ground truth \(D_{\mathrm{gt}}(u)\), the metrics are defined as: \[\begin{align} \mathrm{Abs\;Rel} &= \frac{1}{|\Omega|} \sum_{u \in \Omega} \frac{|D(u) - D_{\mathrm{gt}}(u)|}{D_{\mathrm{gt}}(u)}, \tag{4} \\ \mathrm{Acc}_\delta &= \frac{1}{|\Omega|} \sum_{u \in \Omega} \mathbb{I}\!\left( \max\!\left( \frac{D(u)}{D_{\mathrm{gt}}(u)}, \frac{D_{\mathrm{gt}}(u)}{D(u)} \right) < \delta \right), \tag{5} \end{align}\] where \(\mathbb{I}(\cdot)\) is the indicator function. Abs Rel captures the average proportional error, while \(\delta_{1.25}\) measures the fraction of pixels whose predicted depth falls within \(25\%\) of the ground truth.

Ego-Pose Estimation. Following VGGT, we report the Area Under the Curve (AUC) at a \(30^\circ\) threshold. We first compute the Relative Rotation Accuracy (RRA) and Relative Translation Accuracy (RTA) for all frame pairs within a sequence. RRA measures the geodesic distance between the predicted and ground-truth rotation matrices, while RTA evaluates the angular deviation between translation vectors. The accuracy at threshold \(\tau\) is the percentage of pairs satisfying \(\max(\mathrm{RRA}, \mathrm{RTA}) < \tau\), and the final AUC@\(30^\circ\) is obtained by integrating this accuracy: \[\mathrm{AUC@30^\circ} = \frac{1}{30} \int_0^{30} \mathrm{Acc}(\tau) \, d\tau, \label{eq:auc}\tag{6}\] where \(\mathrm{Acc}(\tau)\) represents the fraction of camera pairs with both angular errors smaller than \(\tau\).

Acknowledgments↩︎

The authors thank Carizon colleagues for their valuable discussions, technical support, and constructive feedback during this work.

References↩︎

---

1. \(\dagger\)These authors contributed equally to this work.↩︎

2. \(^{1}\)Carizon. E-mail: {lizhou.liao, intern.wentao.xu, handong.wang, lirong.yang, shuai.yang, weiwei.liu, chang.huang}@carizon.com.↩︎

3. \(^{2}\)ShanghaiTech University. All work was done during the internship.↩︎

4. \(*\)Corresponding author.↩︎

5. \(\ddagger\)Project leader.↩︎