1 Introduction↩︎

Learning to perceive the 3D structure of the world is integral to human development [1]–[5]. In contrast, in most existing vision, robotics or world models, the geometry of 3D space is imposed a priori, either through an explicit choice of a coordinate system, e.g., [6]–[9] or by distilling from human knowledge. One can therefore ask whether it is possible for a vision model to “discover” the 3D structure of the world with minimal intervention, e.g., purely from the statistics of visual data.

A prominent view in both cognitive science and philosophy is that to discover the geometry of Euclidean space, one must have agency, e.g., [4], [10], [11]. For example, Henri Poincaré hypothesized that “A motionless being could never have acquired the concept of space because, he would have had no reason to distinguish [changes of position] from changes of state. Nor would he have been able to acquire it if his movements had not been voluntary.” ([12] Chap. 4).

Modern vision foundation models present an opportunity to test this hypothesis. These models are, in a strict sense, “motionless beings.” Even when trained on video, they are passive observers of someone else’s movement. Inspired by Poincaré, we thus formulate the following problem:

1.0.0.1 The Poincaré Task:

Can a motionless observer, given only passive visual input, “discover” the structure of 3D space? In particular, does a vision model naturally organize its latent space in a way that reflects the special Euclidean group \(SE(3)\) of Euclidean transformations?

Note that the latent spaces of vision models associate features with the content of the scene, which is naturally present in 3D, while 3D space itself is not directly observable. Unlike prior works that either probe the 3D awareness of visual features by regressing local visible-surface-centric quantities such as depth and normals [13], or employ high-capacity decoders to extract pose [14]–[16], we test whether the feature space itself is organized to reflect Euclidean 3D structure.

The key characteristics of Euclidean space are its low-dimensionality (6D for position and camera orientation), and its homogeneity, e.g., relative displacements have the same meaning regardless of the position in space. This is seemingly in contrast to vision features which are both high-dimensional and fundamentally tied to visual content. It is thus not obvious a priori whether such a low-dimensional, homogeneous coordinate system resides in the visual features of even the most powerful models.

To answer this question, we define an appropriate protocol and test several vision foundation models on static scenes taken from diverse datasets. Our experiments show that information contained in several purely self-supervised image models, when probed correctly, possesses a remarkably strong alignment with the spatial structures of \(SE(3)\) — so these models are able to “see” \(SE(3)\). We build on this insight to propose efficient ways to regress camera motion and to enable closed-form latent visual navigation. The key contributions of this work are:

1. We formulate the problem of inferring the structure of 3D space from visual features, by comparing their structure to the group of Euclidean transformations \(SE(3)\), and introduce several probes to evaluate the performance of a range of encoders with respect to this task.

2. We show that while no features are directly isomorphic to \(SE(3)\) even locally, this structure can nevertheless be extracted using a lightweight Siamese decoder that we call the “Poincaré adapter,” which produces a homogeneous coordinate system.

3. We provide a theoretical analysis of the conditions under which self-supervised methods can recover geometric structure, and investigate the difficulty of recovering different components of \(SE(3)\) — for example, we show that rotation is easier to encode than translation. We also establish the necessary conditions for the emergence of geometric structure through a large ablation comparing different variants of vision features trained on the same dataset.

4. Building on this insight, we introduce a novel closed-form visual navigation approach, which exploits the linearization enabled by our Poincaré adapter.

Overall, our work investigates the conditions for the emergence of the 3D spatial structure within vision-based architectures. Furthermore, these insights are related to Visual Navigation Models [17]–[19] and Navigation World Models [20], [21] by identifying the geometry that underlies spatial reasoning and navigation in the latent space.

2 Related Work↩︎

Probing 3D Awareness. Our work is closely related to the recent literature on probing the 3D awareness of vision foundation models. For example, Probe3D [13] proposed a protocol and evaluated a range of vision models on depth and surface normals, as well as multi-view feature consistency (keypoint matching). Similarly, You et al. [22] demonstrated that multi-view feature consistency is strongly correlated with improved performance on various downstream tasks, while Amir et al. [23] showed that self-supervised ViT features serve as effective dense visual descriptors for keypoint correspondence. Huang et al. [24] evaluated a range of vision models by estimating multiple 3D properties from their features via shallow read-outs, while Man et al. [25] performed a similar analysis in the context of scenes, by considering both low-level geometric as well as semantic 3D tasks. Other works have probed for viewer-centric properties such as depth (e.g., [26], [27] among many others). Our work is fundamentally different from all these efforts, in that we focus on the underlying structure of the vision features themselves, and specifically on the presence of a homogeneous spatial coordinate system within them, which, we argue, is the hallmark of intrinsic geometric discovery.

Explicit 3D Reconstruction and Visual Odometry. The field of explicit 3D estimation has seen rapid progress with models like DUSt3R [15] and VGGT [16]. These approaches treat 3D reconstruction as a sequence-to-sequence problem, employing heavy Transformer architectures to “compute” geometry from features. Similarly, classical Visual Odometry (VO) [28] explicitly solves for pose. However, these approaches typically employ high-capacity decoders, which might demonstrate that geometric information exists within the features, but do not reveal whether visual features are organized and structured in a way that reflects Euclidean 3D space.

Closely related to our goals is RUST [29], which demonstrated that a generative model trained on static scenes naturally organizes its latent space into a geometric structure isomorphic to physical camera parameters. Our work broadens this by asking if discriminative models trained on arbitrary, diverse data (like ImageNet or YouTube) can also discover this geometric substructure for static scenes. Interestingly, related recent work by Mitchel et al. [30] shows that in the context of Novel View Synthesis, letting the network “discover” pose space through transferability, rather than imposing it a priori leads to better performance. Cognitive and Neuroscientific Foundations. Our investigation is also motivated by the work of Hénaff et al. [31], who demonstrated that the primary visual cortex (V1) transforms curved video trajectories into “straighter” paths—a hypothesis recently extended to learned world models [32] in the temporal domain. We ask whether vision foundation models perform a similar linearization to recover the 6D generators of rigid motion. Our finding that a “Visual Grid Code” emerges in passive, motionless observers challenges the prevailing view that grid-cell-like representations require active motor agency for path integration [33], [34].

Visual Navigation Models and Navigation World Models. Our work is also related to Visual Navigation Models (VNMs) [17]–[19], [35]–[37] and Navigation World Models [20], [21], [32], [38]. The former typically take a source and target frame and output a set of actions to reach the target from the source without building a map, while the latter (NWMs) are typically tasked with predicting the next visual state or observation given a current state and a simulated action. Although our work does not directly deal with action selection, these two problems are closely related to our pose linearization and latent space navigation (Section 4 below). Furthermore, both VNMs and NWMs have so far been driven primarily by architectural and scaling advances. Instead, our work aims to establish and improve the fundamental mechanisms that enable navigation and prediction in vision feature space.

3 Background and Motivation↩︎

The overall objective of our work is to study the structure of vision features and to relate it to the group of Euclidean transformations. To make the problem well-posed, we consider static scenes, where the mapping from camera pose to an image (and thus feature) is stable and well-defined.

3.0.0.1 Problem Statement.

Let \(\mathcal{S}\) be a static scene observed by a moving camera across time \(t\). An encoder \(f_\theta\) maps the image \(I_t\) at time \(t\) to a high-dimensional feature vector \(z_t = f_\theta(I_t)\). Our goal is to determine if these features reflect the structure of Euclidean transformations the camera undergoes.

In particular, we consider image acquisition as a sampling process \(\mathcal{U} \rightarrow \mathbb{R}^d\) from the camera pose space \(\mathcal{U} \subseteq SE(3)\) to the feature space \(\mathbb{R}^d\). For a static scene, this process is defined via composition \(P_t \rightarrow I_t \rightarrow z_t\), where \(P_t \in \mathcal{U}\) is the camera-to-world pose at time \(t\), \(I_t\) is the image associated with that pose, and \(z_t\) is the vision feature. After assembling the set of features \(\mathcal{Z} = \{z_t\}_{t=1..N}\) we then compare the properties of the point set \(\mathcal{Z}\) to the structure of \(SE(3)\), as defined by the set \(\{P_t\}\).

3.0.0.2 Topology Consistency and Submanifold Dimensionality.

We first evaluate whether the topology of the feature point set \(\{z_t\}\) locally respects the topology of \(SE(3)\) by comparing the nearest neighbors in the feature space to nearest neighbors in camera pose space (metric M1 below). We then evaluate whether \(\{z_t\}\) locally forms a 6D manifold (metric M2 below).

3.0.0.3 Geometry – Local Linearization and Lie Algebra.

Most importantly, we are interested in evaluating whether \(\{z_t\}\) (or a projection of it) captures the geometry of \(SE(3)\), or, more precisely, whether it is isomorphic to \(SE(3)\). Unfortunately, it is generally impossible to construct a global isomorphic representation of \(SE(3)\) with additive operations in \(\mathbb{R}^d\), regardless of the dimensionality \(d\) (see e.g., [39], [40]). We thus focus on recovering the structure of \(SE(3)\) locally. This aligns with the concept of “straightening” proposed by Hénaff et al. [31], which suggests that the visual cortex transforms non-linear pixel-space trajectories into straighter paths. This behavior has been investigated in the context of temporal straightening (e.g., [32], [41]–[43]), whereas we ask if the “straight” paths in latent space correspond to the generators of rigid body motion.

Formally, we let \(P_t, P_{t+s} \in SE(3)\) be the camera-to-world pose matrices at frames \(t\) and \(t+s\). The relative transformation between them is given by \(P_{rel} = P_t^{-1} P_{t+s}\). We map this relative pose to its tangent vector \(\Delta P\) in the Lie Algebra \(\mathfrak{se}(3)\) using the matrix logarithm: \[\label{eq:body95displacement} \Delta P = (\text{Log}(P_t^{-1} P_{t+s}))^{\vee} \in \mathbb{R}^6\,.\tag{1}\] Here, \(\Delta P\) is a 6-dimensional vector comprising 3 translational velocities and 3 rotational velocities (axis-angle), and \(\vee\) is the vee operator, which extracts the 6D twist coordinates from the \(4{\times}4\) matrix in \(\mathfrak{se}(3)\). We argue that if the latent space is geometrically structured, there should exist a fixed and uniform (i.e., applicable for every point in space) linear operator \(W\) such that: \[\Delta P \approx W(z_{t+s} - z_t).\] The requirement for \(W\) to be linear ensures that the latent space itself supports vector arithmetic isomorphic to the tangent space of \(SE(3)\).

3.0.0.4 Space Discovery.

To link this optimization problem to the original Poincaré task, suppose \(\Delta P = W (z_{t+s} - z_t)\). Since \(W \in \mathbb{R}^{6 \times d}\), this implies that transitions in the feature space \(z\) can be decomposed into a 6D row space of \(W\) (“changes of position”). Crucially, the modifications within this subspace are homogeneous: i.e., the difference vectors \(z_{t+s}-z_{t}\), after projection, can be captured by the same set of 6 basis vectors, regardless of \(t\). This means that, by isolating such a subspace an agent can, in principle, discover the primary degrees of freedom in the 3D physical world, and those degrees of freedom have a consistent meaning irrespective of the visual content observed.

4 Protocol↩︎

We evaluate a range of vision encoders on static scenes from ScanNet [44], ARKitScenes [45], TUM RGB-D [46], 12 Scenes [47], and 7 Scenes [48] datasets. As mentioned above, we introduce several metrics of increasing complexity to probe spatial awareness of vision models.

4.0.0.1 Metric 1 (M1): Topological Alignment.

We use the mutual \(k\)-nn alignment metric [49], [50] to evaluate the similarity between neighborhoods in the visual feature and camera pose spaces, respectively. While previous works have used it to compare signals as captured by different encoders (vision vs. text), here we adapt this metric to compare vision encoders directly to spatial transformations as captured by camera motion in a static scene. We describe our precise protocol in Appendix Sec. 9.1.

4.0.0.2 Metric 2 (M2): Intrinsic Dimensionality (ID).

We estimate the Local Intrinsic Dimensionality using MLE [51] and Two-NN [52] estimators. An intrinsic dimensionality closer to 6 suggests a representation that potentially captures the underlying degrees of freedom.

Note that both metrics M1 and M2 are training-free and directly evaluate the structural similarity between the feature space of different encoders and camera pose space. While the results depend on the choice of representation for camera pose, we found that the relative ordering of different vision encoders is remarkably robust across different representations.

4.0.0.3 Metric 3 (M3): Linear Equivariance.

We test if the feature space is globally Euclidean. For this, we train a linear regressor \(W\) to predict the relative pose change \(\Delta P\) from the feature difference: \(\Delta P \approx W (z_{t+s} - z_t)\) across frames separated by stride \(s\). Success here implies the manifold is natively flat. The regression targets are standardized (zero mean, unit variance) per component to account for scale differences between translational and rotational units (see Appendix 9.2 for details).

4.0.0.4 Metric 4 (M4): The Poincaré Adapter.

Since the metric M3 above imposes a restrictive linear constraint, we introduce a lightweight, trainable adapter \(\varphi_\theta\) (an MLP) in a Siamese manner: \[\Delta P \approx W (\varphi_\theta(z_{t+s}) - \varphi_\theta(z_t)) \label{eq:pose95prediction}\tag{2}\] We call the network \(\varphi_\theta\) a “Poincaré Adapter,” since its goal is to unroll the non-linear feature space to construct a homogeneous coordinate system, where changes of pose become linear. We emphasize that the adapter network \(\varphi\) is applied independently to any latent vector \(z\), thus allowing a restructuring of the latent space to reveal the \(SE(3)\) geometry. Please see Figure 1 for a qualitative illustration. The full training and implementation details are provided in Appendix 9.2. Although Eq. 2 is our main formulation, we investigate several alternatives in Section 6 and Appendix 9.5.

Following recent work (e.g., [53]), which has shown that intermediate layers of vision models tend to carry more geometric awareness, for all metrics M1-M4, we perform a layer sweep across the encoder blocks to achieve the best performance.

4.1 Application: Latent Space Navigation↩︎

As a proof of concept enabled by our study, we apply our approach to Latent Space Navigation. The key idea is that, if the visual latent space can be structured to reflect the local topology of \(SE(3)\), then an agent should be able to “imagine” the visual consequence of a physical movement without requiring explicit 3D reconstruction. This objective relates to both latent novel view synthesis [7], [54], and, more closely, to Navigation World Models [20], [21]. Unlike both of these frameworks, our goal is not to synthesize views, however, but rather to show that navigation can be greatly simplified (reduced to vector arithmetic) after identifying the appropriate camera-aligned linear feature subspace.

Formally, given the adapted feature \(g_t = \varphi_\theta(z_t)\) of a starting frame and a desired physical displacement (pose change) \(\Delta P\), our goal is to predict the feature representation of the destination frame, \(\hat{g}_{t+s}\) by inverting the Poincaré Adapter. We evaluate a completely training-free navigator (which we call Inverse Poincaré). Since our adapter projects features such that \(\Delta P \approx W(g_{t+s} - g_t)\), (where \(g_t = \varphi_{\theta}(z_t)\) as in Eq. 2 ), we can linearly invert this relationship: \[\hat{g}_{t+s} = g_t + W^\dagger \Delta P\,,\] where \(W^\dagger\) is the pseudo-inverse of the linear projection weights. We compare this approach to parametric correctors (e.g., Linear, MLP, and Attention layers) trained to predict \(\hat{g}_{t+s}\) from the concatenated input \([g_t; \Delta P]\). To give the networks a strong geometric foundation, the higher-capacity models (MLP and Attention) are constructed to predict a non-linear residual correction on top of the baseline prediction (i.e., \(\hat{g}_{t+s} = g_t + W^\dagger \Delta P + \text{Net}([g_t; \Delta P])\)).

To evaluate navigation success, we perform a nearest-neighbor retrieval task. We query the test set of unseen scene frames using the predicted \(\hat{g}_{t+s}\) via \(L_2\) distance in the adapted feature space, and measure Pose-Error Hits (Hit@\(\epsilon\)). For each frame, we compute the \(L_2\) distance between the raw 6D twist vectors (3D translation in meters + 3D axis-angle rotation in radians) of the target and retrieved relative poses. As this distance mixes translation and rotation, the thresholds \(\epsilon \in \{0.1, 0.2, 0.3, 0.5\}\) correspond approximately to a pure translation error of \(\epsilon\) m, a pure rotation error of \(\epsilon\) rad, or some combination thereof. Hit@\(\epsilon\) reports the fraction of retrievals for which this distance is below \(\epsilon\).

5 Main Results↩︎

We first evaluate a diverse set of vision foundation models on static scenes from ScanNet [44]. Our evaluation suite includes: patchified Raw Pixels (baseline), CLIP [58], DINOv2 [56], DINOv3 [57], CroCo [55] (3D-pretext SSL), V-JEPA 2.1 [21], against explicit geometric foundation models DUSt3R [15] and MoGe [59]. We also evaluated Depth Anything v3 [60], Perception Encoder [61], Metric3d v2 [62], and generative models: Stable Diffusion SD 1.5 DDIM and SD 2.1 DDIM features, [63], [64], LCM DDIM [63]–[65], SD-DINO VAE and SD-DINO UNet features [66], Stable Video Diffusion (SVD) [67], as well as several video models: VideoMAEv2 [68], RVM (Recurrent Video Masked autoencoders) [69] and the base variant from the 4DS family [70].

5.0.0.1 Topological Alignment (Metric M1).

We first evaluate if the neighborhood structure of the visual latent space mirrors the neighborhood structure of the physical world. We use the mutual \(k\)-nn alignment score described in Appendix 9.1, across both short and long temporal strides. A higher stride introduces larger viewpoint changes, reducing pixel overlap and testing global spatial awareness.

As shown in Figure 2, models explicitly trained with geometric objectives (DUSt3R, MoGe, MASt3R) achieve the highest alignment scores (\(\approx 0.35-0.40\)) at high strides. This highlights that their representations map visual inputs to a metric-aligned coordinate system. Interestingly, although most of these methods require a decoder to regress camera pose, our results show that the encoder itself tends to structure the latent space to align with the structure of the pose space.

Perhaps more surprisingly, among purely self-supervised vision models, DINOv2 and DINOv3 exhibit remarkably strong topological alignment (\(\approx 0.38\)), significantly outperforming other baselines and Raw Pixels. Crucially, this structure emerges purely from passive image-level self-supervision, without explicit depth or pose signals. We also note that video models (VideoMAE, RVM, 4DS) perform less successfully on this metric (\(\approx 0.15-0.25\)). We hypothesize that while these models capture temporal motion, their latent states are highly context-dependent (entangled with the generative trajectory) and fail to form a stable, globally consistent spatial map of the static environment.

5.0.0.2 Geometry of Raw Features (Metrics M2 & M3).

We estimate the Local Intrinsic Dimensionality (ID) using both Two-NN and MLE estimators. Theoretically, a perfectly space-aligned representation of a static scene viewed by a moving camera should have an ID of \(\approx 6\). As shown in Table 1, most encoders result in features with intrinsic dimensionality around 6 (we justify this effect theoretically in Section 6). We observe that DINOv2 and DINOv3 exhibit higher IDs (\(\approx 5.8 - 10.0\)), reflecting a richer semantic manifold that has expanded beyond simple pixel correlations. CroCo, which is pre-trained with explicit 3D cross-view completion, shows a lower ID (\(\approx 5.9\), MLE) closer to the physical degrees of freedom.

We then test if physical displacements \(\Delta P\) can be recovered via a global linear projection \(\Delta P \approx W(z_{t+s} - z_t)\). As reported in Table 1, models generally fail this test, yielding negative or near-zero \(R^2\) scores (e.g., DINOv2 \(R^2 \approx -0.28\), Pixels \(R^2\approx-5.29\)). This confirms that the “Manifold of Vision” is natively curved; standard vector arithmetic on raw features does not correspond to physical motion. We provide a full comparison across a wide range of models in Appendix 9.3.

5.0.0.3 Emergence of Linear Structure (Metric M4).

As shown above, Metrics M2 and M3 do not tend to differentiate vision encoders because they are, respectively, too easy (most models have ID close to 6) and too hard (no model possesses an easily linearizable subspace aligned with motion). We thus focus on the Poincaré Adapter described in Eq. 2 . Example input frame pairs (Figure 3) illustrate the typical viewpoint change across a stride of \(s{=}40\) frames in ScanNet [44].

Remarkably, by applying a lightweight Siamese MLP adapter \(\varphi\), we recover strong linear equivariance. As shown in Figure 4, DINOv2 achieves strong performance, reaching test set \(R^2 \approx 0.65\), averaged over 3 scenes from ScanNet. This implies that the latent space contains a locally Euclidean sub-manifold isomorphic to \(SE(3)\)—effectively a “Visual Grid Code”—which can be accessed via a non-linear projection. Notably, CroCo, despite being trained with explicit 3D tasks, underperforms DINOv2 (\(R^2 \approx 0.38\)), suggesting that massive passive observation (DINOv2’s scale) may be more effective for discovering spatial laws than smaller-scale explicit 3D pre-training.

We also note that topological alignment already emerges without any training (M1), and that raw pixels remain strongly negative \(R^2\) even with the adapter (Figure 4), confirming that the decodable structure arises in and is dependent on the properties of learned features.

To provide a complete picture of geometric decodability across different environments, we summarize the results of 5 different datasets in Table 2. We compare the three representative foundation models (DINOv2, V-JEPA, DUSt3R) and average decodability metrics within each dataset. Overall, we note that V-JEPA 2.1 [21] and DUSt3R [15] possess, on average, the strongest geometric awareness. We also remark that the difficulty of decoding the spatial structure is highly dependent on the environment, and present a scene difficulty analysis in Sec. 9.4.

5.0.0.4 Generalization and Scaling.

To understand how geometric awareness generalizes, we trained Poincaré adapters on varying numbers of ScanNet rooms (\(N \in \{5, 10, 15, \dots, 200\}\)) and evaluated them zero-shot on a fixed set of 31 held-out rooms. As shown in Figure 5, generalization improves monotonically with scale across multiple vision foundation models (DINOv2, V-JEPA, DUSt3R) despite their different pre-training objectives. For example, the fraction of rooms where the zero-shot adapter recovers a positive \(R^2\) signal grows from roughly 18% at \(N=5\) to 50% at \(N=200\), while the overall recoverable signal increases accordingly. Notably, these scaling curves do not plateau at \(N=200\), confirming that geometric awareness is a scalable, universal emergent property of vision transformers that transfers to unseen environments.

5.1 Latent Space Navigation↩︎

Building upon the linear structure revealed by the Poincaré Adapter, we test our Latent Space Navigation proof-of-concept using the intermediate features of DINOv2 (Layer 7) and the explicitly geometric DUSt3R (Layer 16). The results, averaged across multiple scenes, are summarized in 3. We include an Identity baseline that simply returns the source frame’s feature (\(\hat{g}_{t+s} = g_t\), i.e., no navigation at all) to calibrate the difficulty of the task. We compare the training-free “Inv.Poincaré” approach—which simply adds the pseudo-inverse scaled pose displacement to the current feature (\(g_t + W^\dagger \Delta P\))—against a learned Attention corrector for both backbones. As shown in Table 3, the zero-shot Inv.Poincaré approach yields highly competitive retrieval, significantly outperforming the static Identity baseline at all scales. While the Attention corrector achieves better regression metrics (\(R^2\)), it lags behind Inv.Poincaré in physical region retrieval, suggesting that simple vector arithmetic is sufficient to move the observer to the correct spatial neighborhood. A full comparison including additional correctors is provided in Appendix Table 8.

Cross-Room Generalization. We extend our navigation evaluation to 31 unseen test rooms to test generalizability. While the zero-shot pseudo-inverse (\(W^\dagger\)) correctly retrieves frames better than Identity in roughly 60% of rooms, adapting it using a Ridge-Refit (fitting only the linear mapping component of \(\Delta P \rightarrow \Delta g\) on the test room, given a frozen adapter) pushes navigation success to 90–94% across DINOv2, DUSt3R, and V-JEPA backbones. This demonstrates that the adapter discovers a geometrically structured manifold that is robust even across entirely different environments.

We also note that this linear structure allows for Multi-Step Navigation in the latent space. Given a source and a target frame from an unseen test room, we split the desired pose displacement \(\Delta P\) into \(S\) equal sub-steps and iteratively apply the adapter in an open-loop fashion to generate intermediate latent features, without generating any pixels. We then retrieve the nearest neighbor frame from the test set. We note that for moderate displacements (e.g., \(0.68\text{m}, 19^\circ\)) in the 12 Scenes dataset [47], the 4-step retrieved trajectory reaches the exact target frame. Even for extreme displacements (e.g., \(2.45\text{m}, 37^\circ\)), the path remains semantically coherent and rotation recovery remains highly precise (\(<5^\circ\) error), although translation drift accumulates. Figure 6 shows a representative trajectory, and further examples are provided in Appendix [sec:app:multi95step95nav]. This confirms the feasibility of open-loop latent planning over extended paths. We also explore extending the single-chart adapter to a Mixture-of-Experts (MoE) architecture, described in Appendix 9.11.

6 Analysis↩︎

Going back to the original Poincaré question formulated in the introduction, our findings suggest that the answer is: Yes, a motionless observer can discover space, but only up to a nonlinear unwrapping that a simple adapter can undo. In this section we provide a more in-depth analysis of both the theoretical underpinnings of this behavior and the practical conditions under which it arises. Full derivations, proofs, and detailed tables are deferred to the Appendix.

6.0.0.1 Probe design is important.

We compare the Poincaré adapter against four alternative formulations for pose recovery from frozen DINOv2-B features (Appendix 9.5, Table 6). Three design choices prove to be important. First, Lie algebra linearization: predicting the 6D twist vector via a linear readout \(W\) outperforms \(SE(3)\) matrix regression (\(R^2 = 0.61\) vs.\(-0.43\)), confirming that the adapted feature differences are naturally aligned with the tangent space of \(SE(3)\). Second, predicting absolute poses fails entirely (\(R^2 = -1.75\)), indicating that DINOv2 encodes relational structure between views, not scene-specific locations. Lastly, Siamese structure: applying the adapter independently to each feature before subtraction (\(\varphi(z_2) - \varphi(z_1)\)) outperforms operating on the raw difference (\(\varphi(z_2 - z_1)\)) by \(\Delta R^2 = +0.13\), with translation magnitude collapsing from \(+0.60\) to \(-0.08\) without it. The Siamese design enforces homogeneity: each feature is normalized into a common coordinate frame before comparison.

6.0.0.2 Why does the adapter work?

In the Appendix Sec. 9.7, we provide a formal analysis of the conditions under which the Poincaré adapter recovers SE(3) displacements. We show (Theorem 1) that any smooth encoder whose Jacobian has rank 6 admits a local linear readout. We then analyze global decodability: when does a single \(W\) work across all poses? We show (Theorem 3a–b) that the error is controlled by the curvature of the feature manifold, as measured by the variation of the feature encoder’s Jacobian across poses. A nonlinear adapter \(\varphi\) removes the feature-manifold curvature entirely by “unrolling” the curved manifold into a flat coordinate patch (Theorem 3c); a residual from the non-commutativity of \(\mathop{\mathrm{SE}}(3)\) remains, whose rotational component vanishes for pure translational displacements and which is small when the angular extent of the region is modest—consistent with the observed \(R^2 \approx 0.61\) where raw linear decoding fails (\(R^2 < 0\)). We note that on the training set, \(R^2\) exceeds \(0.9\), suggesting that the remaining gap is due to generalization rather than a structural ceiling of the problem formulation.

We also study the relative difficulty of recovering different \(SE(3)\) components by decomposing the Poincaré adapter’s output into rotation (3 DoF), translation direction (2 DoF), and translation magnitude (1 DoF). Across a sweep of 3,328 adapter configurations (varying bottleneck dimension, learning rate, and loss type) evaluated over multiple random seeds on frozen DINOv2-B features, a consistent pattern emerges (detailed in Appendix 9.6): rotation is reliably decodable (positive \(R^2\) in 88% of configurations), while translation magnitude is not (positive in only 27%, with \(6\times\) the seed standard deviation). With a well-tuned adapter, all components achieve comparable \(R^2\) (Table ¿tbl:tab:ablation?: \(0.63\) trans, \(0.59\) rot for pretrained DINOv2-B), but translation magnitude is far more sensitive to hyperparameters and initialization. This asymmetry has a geometric origin (Theorem 5): rotational optical flow depends only on pixel coordinates and focal length, while translational flow scales as \(1/Z\) with scene depth—introducing an additional source of Jacobian variation that makes translation harder to decode reliably. Translation magnitude is further confounded with depth via the rescaling invariance \((v, Z) \mapsto (\lambda v, \lambda Z)\), making \(\|v\|\) unrecoverable without metric priors.

6.0.0.3 Model training – topology is stable; geometry is fragile.

To identify the necessary training conditions, we train 25 DINOv2-B checkpoints from scratch on frames taken from a single walking-tour video (using the setup introduced in [71]). We perform systematic ablations across the main DINOv2 hyperparameters to investigate the necessary conditions for the emergence of geometric structure in the feature extractor (Table ¿tbl:tab:ablation?). We make several observations. First, mutual \(k\)-nn alignment (metric M1) is stable across recipes (M-\(k\)-nn \(\in [0.35, 0.42]\)), while Poincaré \(R^2\) ranges from \(0.08\) to \(0.47\)—even four identical runs yield \(R^2 \in [0.08, 0.35]\) with M-\(k\)-nn \(= 0.38 \pm 0.01\). The sole ablation that degrades topology is removing DINO self-distillation (M-\(k\)-nn drops to \(0.24\)); reducing batch size to 128 collapses manifold formation entirely (M-\(k\)-nn\(\,=\,0.15\)). Data scale is the dominant factor for geometry: the pretrained model (\(R^2 = 0.61\), 142M images) outperforms the best single-video checkpoint (\(R^2 = 0.47\)), with the gap concentrated in translation magnitude (Mag \(R^2 = +0.60\) vs.all negatives)—consistent with Theorem 5(d): metric scale requires statistical priors over object sizes that only diverse training can provide.

7 Conclusion, Limitations and Future Work↩︎

In this paper, we investigated whether vision features capture the structure of Euclidean space by studying the alignment between these features and the group \(SE(3)\) of rigid motions. We showed that, although no encoder reflects this structure directly, it is nevertheless possible to “unroll” the feature manifold using a relatively simple adapter network in some cases. It is remarkable that visual feature changes can be mapped to pose changes by a homogeneous low-dimensional adapter that is independent of location. Interestingly, self-supervised models, while only trained through passive observation, do form a representation of 3D space, decodable when probed correctly.

One limitation of our study is that we only considered static scenes where feature changes are only associated with camera motion, and thus do not test Poincaré’s distinction between “changes of position” and “changes of state”. It would be interesting to extend our formalism and approach to dynamic scenes, which are governed by both camera motion as well as scene motion from underlying physical processes, aiming for an understanding of more profound spatio-temporal symmetries or conservation laws.

8 Acknowledgements↩︎

The authors would like to thank Dima Damen, João Carreira, Daniel Zoran, Gabrijel Boduljak and Andrew Zisserman for the many useful comments, discussions and feedback on this work.

9 Appendix↩︎

9.1 Metric M1: Mutual \(k\)-nn protocol↩︎

For a fixed scene we sample 256 frames, at a given stride. We compute a feature vector for each frame, and also associate to it the corresponding camera pose (see Figure 3 for examples of such frame pairs). We then compute the \(k\) nearest-neighbors (\(k=10\)) in the feature space using cosine similarity, and compare these nearest neighbors to the nearest neighbors computed in the camera pose space. We use the 9D camera pose representation, comprising 3D translation and the first two columns of the rotation matrix, as defined in [40], in world coordinates. To reduce the impact of temporal sampling, we exclude temporal neighbors within 10 frames from the nearest neighbor computation in both visual feature and pose space. The mutual \(k\)-nn metric is defined as the average intersection (number of shared edges) between these nearest neighbor graphs (illustrated in Figure 7). The metric ranges from 0 (no alignment) to 1 (perfect alignment) between visual and spatial representations.

For video encoders that require, as input, a temporal window of frames, we sample the encoder-native set of frames around each anchor frame and pool all features within this temporal window to obtain a single feature vector. We also experimented with keeping only the features associated with a specific anchor frame, without a noticeable difference in the results.

Since this metric requires a single vector per frame, for encoders that contain a CLS token, we take features from that token. For others, we perform mean pooling over the patch tokens.

9.2 Metrics M3 and M4: implementation details↩︎

For both metrics, we construct frame pairs at a fixed temporal stride \(s\) and split them using a time-based strategy: for each scene, the first 80% of frames (ordered by timestamp) are used for training and the remaining 20% for testing, ensuring that no test frame appears during training. For M3 (Linear Equivariance), we fit a Ridge regression (scikit-learn, \(\alpha{=}1.0\), with intercept) from the raw feature displacement \(\Delta z = z_{t+s} - z_t\) to the 6D Lie-algebra pose target \(\Delta P\), and report the test \(R^2\) (uniform average over the 6 components). Table 1 reports results at the layer maximizing M3 for each model.

To train and evaluate the Poincaré Adapter (M4), we use the same temporal train/test split. The Siamese projection network \(\varphi_\theta\) is a 2-layer MLP: \(\texttt{Linear}(d, 64) \to \texttt{LayerNorm} \to \texttt{GELU} \to \texttt{Linear}(64, 20)\), followed by a bias-free linear readout \(W \in \mathbb{R}^{6 \times 20}\). The regression targets \(\Delta P\) are z-scored (zero mean, unit variance) per component on the training set before training, and predictions are un-normalized before computing the test \(R^2\). The adapter is optimized with AdamW (lr\(=\)​1e-3, weight decay\(=\)​1e-2) for 10 epochs with batch size 512 and gradient clipping at norm 1.0. Similarly to M1 described above, for video encoders we use a temporal window around each anchor frame and pool features within this window to obtain a single feature vector. We use a CLS token when available, or, mean pooled patch tokens, otherwise. The representative architecture of the “Poincaré Adapter” network is visualized in Figure 8.

9.3 Full Intrinsic Dimension and M3 Results for All Models↩︎

In Table 4, we report the Intrinsic Dimension (TwoNN and MLE) and the M3 metric (Max Test \(R^2\)) for 18 different models evaluated in our sweep, using the layer that maximized M3 performance for each model.

9.4 Dataset Summary & Difficulty Analysis↩︎

We summarize the different datasets and the key performance in Table 5. To understand why pose recovery varies across environments, we trained independent Poincaré adapters on 913 scenes spanning 5 diverse datasets (ScanNet, 12-Scenes, ARKitScenes, TUM RGB-D, 7-Scenes) using three representative encoders (V-JEPA 2.1, DUST3R, DINOv2-B). Consistently with the results presented in the main paper, we use a temporal 80%/20% train/test split separation. From a large-scale comparison across multiple environments, we observe that scene difficulty is driven, to a significant extent by data properties, in addition to the choice of foundation model. Specifically, success hinges on three factors: (1) Data Sufficiency: scenes with \(>1000\) pairs succeed 81% of the time, explaining why dense datasets like 12-Scenes achieve 100% success while sparse ones like 7-Scenes (often \(<50\) test pairs) artificially fail (\(9\%\) success). (2) Visual Richness: highly textured rooms provide the necessary semantic anchors, outperforming texture-less traversals. (3) Workspace Geometry: bounded rooms naturally support dense view overlap and closure, yielding much higher decodability than linear hallway traversals. A summary of the dataset difficulties is provided in Table 5.

Note that the intrinsic dimensionality of most features typically is close to the theoretical value 6 (see Theorem [thm:intrinsic95dim] below), while the \(R^2\) value for linear equivariance Metric M3 is negative or close to zero, suggesting that no model is natively aligned with rigid motion.

9.5 Ablation: Alternative Formulations for Pose Recovery↩︎

We compare the Poincaré adapter formulation against four alternatives for recovering camera pose from frozen DINOv2-B (Layer 7) features on ScanNet. All methods share the same feature backbone, train/test splits (time-based, 3 scenes), and adapter capacity (hidden_dim\(=\)​64, geo_dim\(=\)​20). Results are averaged over 15 random seeds.

9.5.0.1 Methods.

Let \(z_i = z(I_{g_i}) \in \mathbb{R}^{768}\) denote the frozen feature for image \(I\) at pose \(g_i \in \mathrm{SE}(3)\), and let \(\Delta P = \log(g_2 g_1^{-1}) \in \mathbb{R}^6\) be the Lie algebra target.

• Poincaré adapter (baseline): \(\hat{\Delta P} = W\bigl(\varphi(z_2) - \varphi(z_1)\bigr)\), where \(\varphi\) is a 2-layer MLP and \(W\) is a bias-free linear readout. Loss: MSE on z-scored Lie algebra targets. Siamese: \(\varphi\) is applied independently to each feature before taking the difference.

• Method A (unanchored relative): \(\hat{M} = \psi(z_2) \cdot \psi(z_1)^{-1}\), where \(\psi\) maps each feature to an SE(3) matrix (3D translation + 6D rotation representation), consistent with the target convention \(M_{\mathrm{gt}} = g_2 g_1^{-1}\). Loss: \(\|\log(\hat{M}^{-1} M_{\mathrm{gt}})\|^2\) (geodesic on SE(3)).

• Method B (absolute pose): \(\hat{W}_i = \psi(z_i)\) predicts the absolute camera-to-world pose; the relative pose is recovered as \(\hat{M} = \hat{W}_2 \cdot \hat{W}_1^{-1}\), consistent with \(M_{\mathrm{gt}} = g_2 g_1^{-1}\). Loss: geodesic on SE(3) for each absolute pose independently, i.e., \(\sum_i \|\log(\hat{W}_i^{-1} g_i)\|^2\). Note that the composition formula is used only at evaluation time; the training signal is purely on absolute poses.

• Method C (SE(3) MLP on \(\Delta\varphi\)): \(\hat{M} = \mathrm{MLP}_{\mathrm{SE3}}\bigl(\varphi(z_2) - \varphi(z_1)\bigr)\), using the same Siamese \(\varphi\) but outputting an SE(3) matrix instead of a Lie algebra vector. Loss: geodesic on SE(3).

• Method D (non-Siamese difference): \(\hat{\Delta P} = W \cdot \varphi(z_2 - z_1)\), where \(\varphi\) operates on the raw feature difference rather than on individual features. Loss: MSE on z-scored Lie algebra targets (same as baseline).

9.5.0.2 Results.

Table 6 summarizes the main results of this ablation.

Three findings emerge. (1) Relative changes over absolute poses. Method B, which predicts absolute camera pose from individual features, fails catastrophically (\(R^2 = -1.75\)), while all relative-change methods achieve positive \(R^2\) on at least some components. We attribute this failure to the fact that DINOv2 features do not encode scene-specific absolute camera coordinates—they encode relational structure between views. Even a hypothetically perfect composition formula cannot recover meaningful relative poses from absolute predictions that do not correlate with the true camera positions. (2) Linearization helps. Methods A and C operate on SE(3) directly rather than on the flat Lie algebra \(\mathfrak{se}(3)\), yet both perform worse than the linearized Poincaré adapter. Method C uses the same Siamese architecture and differs only in replacing the linear readout \(W\) with an MLP outputting SE(3) matrices, yet its \(R^2\) drops from \(0.61\) to \(-0.43\). This suggests that the feature differences \(\varphi(z_2) - \varphi(z_1)\) are naturally aligned with the Lie algebra structure, and that the linearization \(\Delta P \approx W \cdot \Delta\varphi(z)\) is not a lossy approximation but rather matches the geometry of the representation. (3) Siamese structure matters. Method D uses the same linear readout and loss as the baseline but applies \(\varphi\) to the raw difference \(z_2 - z_1\) instead of computing \(\varphi(z_2) - \varphi(z_1)\). This seemingly minor change drops \(R^2\) from \(0.61\) to \(0.48\), with the translation magnitude component collapsing from \(0.60\) to \(-0.08\). The Siamese architecture, in which \(\varphi\) normalizes each feature independently before subtraction, is essential for the pose signal to be linearly decodable from the difference.

9.6 Pose Recovery Components Difficulty Analysis↩︎

To further elaborate on our key metric M4 results and analyze the difficulty of recovering different components of camera pose, we decompose the Poincaré adapter’s overall \(R^2\) into three major parts: rotation (3 DoF), translation direction (2 DoF), and translation magnitude (1 DoF), where direction and magnitude together form the full translation (3 DoF).

We assess whether there is a difference in the difficulty of recovering these components. To this end, we conducted a hyperparameter sweep training different Poincaré adapters to optimize metric M4 following the protocol in 9.2. We trained 3,328 unique adapter configurations evaluated with up to 20 random seeds each, totaling 38,929 runs. All runs use frozen DINOv2-B features (Layer 7, CLS token, \(d = 768\)) extracted from 3 ScanNet scenes (\({\sim}1{,}200\) frames total, stride 2). The adapter consists of a Siamese nonlinear projection \(\varphi\) (2-layer MLP) followed by a bias-free linear readout \(W\); we sweep over bottleneck dimension \(\texttt{geo\_dim} \in \{10, 20, 40, 80, 160, 320\}\), number of mixture-of-experts heads \(K \in \{1, 2, 4\}\), learning rate \(\texttt{lr} \in [3{\times}10^{-4},\, 3{\times}10^{-2}]\), and loss type \(\in \{\text{MSE},\, \text{cosine},\, \text{rot-only}\}\). Evaluation pairs are sampled at a fixed temporal gap (stride) of \(s = 40\) frames, and \(R^2\) is computed on a held-out time-based test split.

9.6.0.1 Difficulty hierarchy.

Table 7 reports the fraction of adapter configurations achieving positive \(R^2\) (better than predicting the constant mean) and \(R^2 > 0.5\), aggregated over all runs.

A consistent ordering emerges: \(R^2_{\text{rot}} > R^2_{\text{trans}} > R^2_{\text{dir}} > R^2_{\text{mag}}.\) Rotation is positive in 88% of configurations; translation magnitude in only 27%. Moreover, translation magnitude has 6\(\times\) the seed variance of rotation (\(\sigma = 1.03\) vs.\(0.17\)), meaning that even the configurations that achieve positive \(R^2_{\text{mag}}\) in one seed often fail in another. While individual configurations occasionally achieve \(R^2_{\text{mag}}\) up to \(0.72\), the metric is not reliably linearly decodable: across all runs, the mean \(R^2_{\text{mag}}\) is \(-3.30\).

9.6.0.2 Why rotation is easier.

The difficulty hierarchy has a precise geometric explanation rooted in optical flow and formalized in Theorem 5 below. Intuitively, rotational flow is depth-independent, whereas the translational Jacobian depends on depth map (see Theorem 5 for details).

9.6.0.3 Joint training regularizes rotation.

Perhaps counter-intuitively, training the adapter to predict all 6 DoF jointly produces better rotation \(R^2\) than predicting rotation alone. Under matched conditions (same architecture, 5 seeds):

One possible interpretation is that the translation objective acts as a multi-task regularizer for the shared feature projection \(\varphi\), reducing the possibility of overfitting.

9.7 Theoretical Analysis↩︎

Before proceeding, we note that even without considering the specifics of the visual feature extractor, under fairly general conditions, the acquisition process itself imposes theoretical constraints on the structure in the feature domain. To state this formally, we frame the visual feature acquisition as a continuous generative process over time, and express these constraints with the theorem below (which is derived from basic principles, but which we state explicitly for clarity and completeness):

9.7.1 Geometric Decodability from Features – Setup and Notation↩︎

Throughout the theoretical analysis, relative pose is expressed in the body frame: \(\xi = \log(g_1^{-1} g_2)\), which describes the displacement as seen from \(g_1\). This is consistent with Eq. 1 of the main paper. Note that for the Poincaré adapter we define the target motion \(\Delta P\) in the local camera coordinate frame (the body frame). In this frame, a translation vector corresponds to an agent-centric action (e.g., “move forward 1 meter”) rather than a change in absolute map coordinates (e.g., “move North”). This choice is essential for homogeneity: a specific visual change (like the radial expansion of optical flow) should always correspond to the same displacement vector \(\Delta P\), regardless of where the agent is located in the world or which direction it is facing.

In Appendix 9.5, Methods A and B use the world-frame convention \(\log(g_2 g_1^{-1})\); the two are related by the adjoint action \(\log(g_2 g_1^{-1}) = \mathrm{Ad}_{g_1},\log(g_1^{-1} g_2)\).

Rendering model. Fix a 3D scene \(s\). A camera with pose \(g \in \mathop{\mathrm{SE}}(3)\) renders an image \(I_g = \mathcal{R}(g, s) \in \mathbb{R}^{H \times W \times 3}\). We write the Lie algebra decomposition \(\mathop{\mathrm{\mathfrak{se}}}(3) = \mathop{\mathrm{\mathfrak{so}}}(3) \oplus \mathbb{R}^3\), so a tangent vector \(\xi = (\omega, v)\) encodes infinitesimal rotation \(\omega \in \mathbb{R}^3\) and translation \(v \in \mathbb{R}^3\).

Encoder. Let \(z\colon \mathbb{R}^{H \times W \times 3} \to \mathbb{R}^d\) be a feature encoder (e.g., a vision transformer). Define the posed feature map: \[f\colon \mathop{\mathrm{SE}}(3) \to \mathbb{R}^d, \quad f(g) = z\bigl(\mathcal{R}(g, s)\bigr).\]

9.7.2 Local Geometric Decodability↩︎

9.7.3 Global Linear Decodability and Its Obstruction↩︎

The local readout \(W(g_0)\) from Theorem 1 depends on \(g_0\) because the Jacobian \(\mathrm{d}f_{g_0}\) varies with pose. Define the Jacobian field: \[J\colon \mathop{\mathrm{SE}}(3) \to \mathbb{R}^{d \times 6}, \quad J(g) = \mathrm{d}f_g.\]

9.7.4 Rotation–Translation Asymmetry↩︎

9.8 Vision Encoder Training, Performance Analysis↩︎

To understand the necessary and sufficient conditions for spatial structure to emerge in self-supervised features, we train DINOv2-B from scratch on a single walking-tour video (Amsterdam WalkingTours, \({\sim}\)​20 minutes) with systematic ablations of the training recipe. We evaluate 25 checkpoints spanning component ablations (removing individual losses or crop strategies), a prototype count sweep (64 to 65k), and multiple independent reruns of the same 65k-prototype baseline to measure training stochasticity. All checkpoints are trained for 62,500 iterations with effective batch size 192 (\(3 \times\) A100, batch_size_per_gpu \(=\) 64), except one checkpoint trained with batch size 128 (\(2 \times\) A100). Each checkpoint is evaluated on ScanNet (3 scenes, \({\sim}\)​1,200 frames) using two complementary metrics across all 12 transformer layers:

• Mutual \(k\)-nn (M1): a non-parametric measure of topological alignment between features and SE(3) camera poses, evaluated at frame stride 5 with oracle over layers.

• Poincaré adapter \(R^2\): a parametric measure of linear geometric decodability. Following the optimal configuration found in our hyperparameter sweep, we use \(s\in[10,40]\) range pairing with 30,000 pairs, a bottleneck dimension of 20, and train for 200 epochs averaging over 15 random seeds per checkpoint with oracle over layers.

Table ¿tbl:tab:ablation? reports both metrics. The central finding is that topology is stable while geometry is fragile.

9.8.0.1 Topology is stable; geometry is fragile.

Mutual \(k\)-nn alignment is remarkably stable: excluding the no_dino and (BS \(=\) 128) outliers, all trained checkpoints achieve mutual \(k\)-nn \(\in [0.35, 0.42]\), within 15% of each other and comparable to the pretrained model (\(0.37\)). In contrast, Poincaré adapter \(R^2\) varies from \(0.06\) to \(0.47\) across the same checkpoints—a significantly larger spread relative to the metric range. This dissociation reveals that DINOv2 training reliably produces features that are topologically aligned with camera pose, but the metric quality of this alignment (whether a linear adapter can quantitatively decode pose) is far more sensitive to training details and stochasticity.

9.8.0.2 DINO self-distillation shapes depth representation.

While no_dino achieves a reasonable overall \(R^2\) of \(0.31\), its best layer is L0 (the patch embedding), and performance collapses in deeper layers. This indicates that without the DINO objective, the network fails to build abstract geometric representations in its deeper layers, defaulting to low-level pixel correlations available at the input. Furthermore, no_dino is the sole ablation that substantially degrades mutual \(k\)-nn (\(0.24\) vs.\(0.38\) baseline), identifying self-distillation as a unique mechanism for topological alignment.

9.8.0.3 Batch size is critical for manifold formation.

The (BS \(=\) 128) checkpoint, trained with effective batch size 128 instead of 192, achieves mutual \(k\)-nn \(=\) \(0.15\)—near random weights (\(0.11\)) and far below all BS \(=\) 192 checkpoints (\({\geq}\,0.35\)). This suggests a sharp phase transition in self-distillation effectiveness as a function of batch diversity, where insufficient contrastive signal prevents the formation of a pose-aligned manifold.

9.8.0.4 Geometry is stochastic even under identical configs.

Four independent training runs of the same 65k-prototype configuration yield Poincaré \(R^2\) ranging from \(0.08\) to \(0.35\) (Table ¿tbl:tab:ablation?, reproducibility block), with the oracle layer migrating from L0 to L8 across runs. Their mutual \(k\)-nn scores, however, are nearly identical (\(0.38 \pm 0.01\)). This confirms that the topology-to-geometry gap reflects intrinsic training stochasticity: the feature manifold’s global shape is reproducible, but its local metric structure is not.

9.9 Full Latent Space Navigation Results↩︎

Table 8 provides the complete latent space navigation results including all corrector variants (Linear, MLP-2, MLP-3, Attention) for both DINOv2 and DUSt3R backbones. The condensed version in the main paper (Table 3) reports only the Identity baseline, Inv.Poincaré, and the Attention corrector.

9.10 Cross-Room Navigation Results↩︎

To rigorously test the generalizability of the learned geometric structures, we evaluated latent space navigation on a held-out set of 31 test rooms. A Poincaré adapter was trained on a separate set of rooms, and we evaluated navigation success using the Hit@0.1 metric. We compared a pure zero-shot inverse (using \(W^\dagger\) directly on unseen rooms) against a Ridge-Refit approach (which fits a direct inverse mapping \(\Delta P \rightarrow \Delta g\) on the test room). As shown in Table 9, while the zero-shot mapping succeeds in approximately 60% of rooms (beating the Identity baseline), the Ridge-Refit correction pushes this to 90–94% across all three major backbone families. This confirms that the geometric manifold is a robust, transferable property of the features.

9.11 Mixture-of-Experts Poincaré Adapter↩︎

To capture the full geometry of complex scenes, we extend the single-chart Poincaré adapter to a Mixture-of-Experts (MoE) architecture with \(K\) local charts. Each expert \(k\) possesses its own value projection \(W_V^k: \mathbb{R}^d \to \mathbb{R}^g\) and linear readout \(W_R^k \in \mathbb{R}^{6 \times g}\). The prediction is a gated combination of expert outputs:

\[\hat{y} = \sum_{k=1}^{K} \alpha_k(z_t) \cdot W_R^k \cdot (\varphi_k(z_{t+s}) - \varphi_k(z_t))\]

Gating is performed via prototype routing on the source feature \(z_t\) alone:

\[\alpha_k(z_t) = \text{softmax}\!\left(\frac{(W_{\text{route}} \, z_t) \cdot p_k}{\exp(\tau)}\right)\]

where \(W_{\text{route}} \in \mathbb{R}^{r \times d}\) is a shared linear projection, \(p_k \in \mathbb{R}^r\) are \(K\) learnable prototype vectors, and \(\tau\) is a learnable log-temperature scalar. The key design decisions are: (i) routing depends only on \(z_t\) (the observer’s current position), not on \(z_{t+s}\), enforcing that chart selection depends on where the observer is rather than where it is going; (ii) a shared routing projection reduces parameters by \({\sim}60\times\) compared to per-expert Q/K projections; and (iii) a learnable temperature allows soft-to-hard expert selection during training, avoiding the “uniform gating” trap. For \(K{=}1\), the routing layers are omitted entirely, ensuring exact numerical equivalence with the baseline single-chart adapter.

9.12 Multi-Step Latent Space Navigation↩︎

We provide qualitative examples of the open-loop Latent Space Navigation described in Section 5.1. In each case, given a source frame and a target frame, we compute the target pose displacement \(\Delta P\) and subdivide it into 4 equal sub-steps. The trajectory is integrated entirely in the frozen latent space by recursively predicting the next feature. We then query the nearest-neighbor image from the test set. Because the navigation operates open-loop, drift can accumulate on long paths. Figure 9 summarizes three navigation scenarios of increasing difficulty, with detailed strip visualizations shown in Figures 10, 11, and 12.

References↩︎