1 Introduction↩︎

Novel View Synthesis (NVS) aims to synthesize photorealistic images from novel viewpoints while preserving the geometric coherence of the observed scene. Neural Radiance Fields (NeRF) [1] and 3D Gaussian Splatting (3D-GS) [2] are representative per-scene optimization solutions that require a large number of observations for training. These methods struggle with sparse-view inputs and are limited to novel views near input viewpoints. Recently, feed-forward geometry methods [3]–[9] and generative approaches [10]–[15] have emerged as strong alternatives for such challenging scenarios.

Feed-forward geometry methods directly estimate explicit 3D representations from sparse-view input images in a single forward pass. One line of work [4]–[6], [9] predicts 3D Gaussians directly from inputs and renders novel views via Gaussian splatting. Another line of work [7], [8], [16] first estimates point cloud maps and then requires optimization to fit 3D Gaussians [17]. Despite their strong structural consistency in observed regions, these methods often fail to preserve fine-grained appearance details in low-overlap scenes.

Generative novel-view synthesis represents another powerful paradigm for handling sparse-view inputs, enabling dense and semantically rich scene synthesis. These methods incorporate camera [10]–[12], [14], [18], [19] or point trajectories [20]–[22] into video diffusion models to guide novel view generation. They exhibit remarkable capability even for scenes with low-overlap inputs, overcoming limitations of feed-forward geometry methods. However, these approaches still suffer from multi-view inconsistency, hallucinating structures that deviate from the input images, and limited camera controllability, failing to follow the intended camera trajectories as shown in [fig:teaser]. A natural remedy is to incorporate explicit geometry priors from feed-forward geometry models to enforce structural consistency between generated and input views.

Indeed, several studies [23]–[30] have explored this direction, combining geometry priors with generative models to achieve geometrically consistent and dense novel-view synthesis. These methods feed noisy rendered novel-view images [25], [27]–[30] or features [23], [24], [26] into a diffusion model for refinement. However, we observe that these input-level fusion methods in 1-(b) introduce geometric distortions and hallucinated structures in the final output, as noisy textures in the rendered images obscure structural information.

To overcome this limitation, we propose GeoNVS, which couples geometry priors with the diffusion model in feature space, enabling the diffusion model to leverage explicit 3D structural information during the denoising process. Our core component, the Gaussian Splatting Feature Adapter (GS-Adapter), bridges 3D Gaussians and the diffusion model through a feature embedding and rendering process, modulating geometrically inconsistent diffusion features. Specifically, GS-Adapter (1) embeds 2D input-view diffusion features into 3D Gaussians, (2) rasterizes geometry-aware novel-view features via Gaussian splatting, and (3) adaptively fuses them with novel-view diffusion features to produce geometrically corrected features. By anchoring diffusion features to explicit 3D structure, GS-Adapter ensures that generated pixels are geometrically aligned with the target camera viewpoint, leading to improved camera controllability.

Compared to previous methods that perform input-level fusion in 1-(b), our approach in 1-(c) leverages only the structural information encoded in 3D Gaussians, avoiding reliance on noisy view-dependent color information. Our modular design enables plug-and-play integration with diverse feed-forward geometry models without additional training, while existing methods [26], [29], [31], [32] depend on a specific geometry model, requiring retraining upon substitution. To validate the effectiveness of GeoNVS, we integrate GS-Adapter into SEVA [11] and CameraCtrl [12] and evaluate across indoor and outdoor scenes in both low- and large-overlap settings as well as long-trajectory generation, demonstrating consistent improvements in geometric consistency, camera controllability, and photorealistic quality. In summary, our key contributions are as follows:

• We propose GS-Adapter, which couples 3D Gaussian priors with the diffusion model in feature space, improving geometric consistency and camera controllability in video diffusion-based novel-view synthesis.

• Our modular design supports diverse combinations of feed-forward geometry models and video diffusion backbones without additional training, enabling flexible plug-and-play applicability.

• GeoNVS achieves state-of-the-art performance over existing generative NVS methods across diverse indoor and outdoor benchmarks, evaluated on 9 scenes and 18 settings.

2 Related Work↩︎

Sparse-view Novel View Synthesis. NeRF [1] and 3D-GS [2] achieve photorealistic novel-view synthesis given dense inputs, but overfit severely under sparse views. To resolve this, prior works introduce depth supervision [33]–[36] or normal consistency [37], [38], yet remain sensitive to constraint quality. Feed-forward geometry methods [3]–[6] directly predict 3D Gaussians using cost volumes [39]–[42] or monocular depth [43]–[45], and geometry foundation models [7]–[9], [16] further enhance reconstruction quality at scale. We leverage these priors to guide the diffusion model toward structural consistency.

Generative Novel View Synthesis. Diffusion models have evolved into powerful task executors by incorporating controllable conditioning signals [46], [47]. Camera-controlled video diffusion methods embed camera poses [14], Plücker coordinates [10]–[13], [18], [19], trajectory-based attention [15], or projected 3D point maps [48] to guide video generation. Recently, SEVA [11] demonstrated strong novel-view synthesis via a multi-view diffusion model trained on large-scale datasets. Building on SEVA and CameraCtrl, we enhance these baselines to achieve geometrically consistent generation and improved controllability.

Feature field distillation for 3D-GS. Beyond radiance fields, embedding semantic features into 3D representations [49], [50] has enabled various downstream applications [51]–[55]. Most existing methods [50], [56], [57] distill semantic features [58]–[61] into 3D Gaussians via per-scene optimization. Recently, Dr.Splat [62] introduced feed-forward feature distillation into 3D Gaussians, enabling fast 3D localization without per-scene optimization. LUDVIG [63] extends this to integrate DINOv2 [61] features into 3D scene geometry through graph diffusion for semantic segmentation. Unlike prior methods that distill static semantic features, diffusion features vary across denoising timesteps, making per-scene optimization infeasible. We therefore adopt a direct feed-forward lifting approach, extending it to preserve pixel-level feature fidelity rather than coarse instance-level distinctions sufficient for segmentation.

3 Method↩︎

In this section, we first provide preliminaries on the problem formulation, geometry prior construction, and Gaussian splatting background in 3.1. We then introduce GS-Adapter in 3.2, our core module that leverages 3D Gaussian priors to correct geometrically inconsistent diffusion features, thereby enforcing geometric consistency and improving camera controllability. Finally, we describe the integration of GS-Adapter into an existing video diffusion framework via multi-scale feature aggregation in 3.3.

3.1 Preliminary↩︎

Problem Formulation. Given \(P\) reference images \(\mathbf{I}_{\text{ref}} \in \mathbb{R}^{P \times hw \times 3}\) and their corresponding camera poses \(\boldsymbol{\pi}^{\text{ref}}\), the NVS task is to synthesize \(Q\) target views \(\mathbf{I}_{\text{tgt}} \in \mathbb{R}^{Q \times hw \times 3}\) for the target camera poses \(\boldsymbol{\pi}^{\text{tgt}}\). GeoNVS supports an arbitrary number of reference images, as both SEVA [11] and the geometry models natively handle monocular and multi-view inputs.

Geometry Prior. Our method requires 3D Gaussians \(\mathcal{G}\) as structural guidance for the diffusion model. We utilize feed-forward geometry models [4]–[8] to estimate this geometry prior from the reference images. For geometry models that estimate both camera poses and point clouds (VGGT [7] and Pi3 [8]), we align the predicted camera scale \(s^{*}\) to the target cameras by solving: \[s^{*} = \arg\min_{s} \left\| \mathbf{C}^{\text{gt}} - s\, \mathbf{C}^{\text{pred}} \right\|_{2}^{2} \label{eq:scale95optimize}\tag{1}\] where \(\mathbf{C}^{\text{gt}}\) and \(\mathbf{C}^{\text{pred}}\) denote the target and predicted camera centers, respectively. We then fit 3D Gaussians initialized from the estimated point clouds [17].

3D Gaussian Splatting. 3D Gaussian Splatting [2] represents a scene with \(N\) Gaussian particles composed of position \(\boldsymbol{\mu} \in \mathbb{R}^3\), rotation \(\boldsymbol{r} \in \mathbb{R}^4\) via quaternions, scale \(\boldsymbol{s} \in \mathbb{R}^3\), opacity \(\boldsymbol{\sigma} \in \mathbb{R}^1\), and color represented by spherical harmonics. The color at pixel \(p\) for view \(d\) is obtained by summing contributions of Gaussians: \[C(d,p) = \sum_{i=1}^{N} c_i(d)\, w_i(d,p), \quad w_i(d,p) = \alpha_i(d,p) \prod_{j=1}^{i-1} (1 - \alpha_j(d,p))\label{eq:rendering95color}\tag{2}\] where \(\alpha_i(d,p)\) is the product of the opacity \(\sigma_i\) and the 2D projected Gaussian density \(\mathcal{N}_i\) at pixel position \(p\). Similarly, feature rasterization is represented as: \[\mathbf{F}(d,p) = \sum_{i=1}^{N} \mathrm{f}_i\, w_i(d,p)\label{eq:rendering95feat}\tag{3}\]

3.2 GS-Adapter: Geometry-Guided Diffusion Feature Adaptation↩︎

Given the diffusion features \(\mathbf{F}^{t}\) at denoising timestep \(t\) and 3D Gaussians prior \(\mathcal{G}\), GS-Adapter is designed to obtain geometry-augmented features \(\hat{\mathbf{F}}^{t}\), formulated as \(\hat{\mathbf{F}}^{t} = \mathrm{\Phi}(\mathbf{F}^{t}, \mathcal{G})\). Diffusion features \(\mathbf{F}^{t} = [\mathbf{F}_{\text{ref}}^{t};\mathbf{F}_{\text{tar}}^{t}]\) consist of reference-view (input-view) features \(\mathbf{F}_{\text{ref}}^{t} \in \mathbb{R}^{P \times \frac{h}{n} \times \frac{w}{n} \times \text{C}}\) and novel-view features \(\mathbf{F}_{\text{tar}}^{t} \in \mathbb{R}^{Q \times \frac{h}{n} \times \frac{w}{n} \times \text{C}}\), where \(n\) denotes the downsampling factor of the Variational Autoencoder (VAE), and \(\text{C}\) denotes the channel dimension of the diffusion features. For convenience, we omit the superscript \(t\) hereafter. As illustrated in 2-(b), GS-Adapter consists of three major steps: (1) feature lifting: projecting reference-view diffusion features onto 3D Gaussians, (2) feature refinement: refining the rendered novel-view features to recover fine-grained details, and (3) feature fusion: combining the geometry-guided features with the original diffusion features to obtain the revised diffusion features.

Uplifting diffusion feature into 3D-GS. Since the diffusion features vary across timesteps and our method requires obtaining geometry-augmented features at every denoising timestep, per-scene optimization-based feature embedding approaches [50], [56], [57] are not feasible, as they cannot adapt to the continuously changing feature distribution during denoising. Inspired by [62], [63], we obtain uplifted diffusion features for each Gaussian as a weighted average of 2D reference diffusion features \(\mathbf{F}_{\text{ref}}\). Each 2D feature at the viewing direction \(d\) and pixel \(p\) contributes to the Gaussian feature \(\mathbf{f}_i\) proportionally to the rendering weight \(w_i(d,p)\) with normalization, where \(\mathcal{I}_i\) denotes the set of view/pixel pairs contributing to each Gaussian. The uplifted feature \(\hat{\mathbf{f}}_{i} \in \mathbb{R}^{\text{C}}\) is defined as: \[\hat{\mathbf{f}}_{i} = \frac{\mathbf{f}_{i}}{\lVert \mathbf{f}_{i} \rVert_{2}}, \qquad \mathbf{f}_{i} = \sum_{(d,p) \in \mathcal{I}_i} \frac{w_{i}(d,p)}{\sum_{(d',p') \in \mathcal{I}_i} w_{i}(d',p')} \mathbf{F}_{\text{ref}}(d,p) \label{eq:uplifting}\qquad{(1)}\] This feed-forward uplifting enables explicit participation of 3D Gaussians at every denoising step, allowing end-to-end training with the diffusion model.

Feature refinement. After uplifting \(\mathbf{F}_{\text{ref}}\) into the 3D Gaussians, we rasterize the geometry-constrained novel-view features \(\mathbf{G}_{\text{tar}} \in \mathbb{R}^{Q \times \frac{h}{n} \times \frac{w}{n} \times \text{C}}\) using 3 . Our next goal is to fuse these geometry-aware features with the original novel-view diffusion features \(\mathbf{F}_{\text{tar}}\). However, we observe that direct fusion alone is insufficient for recovering fine-grained details, as shown in 4, consistent with the findings of [63]. To compensate for the information loss during lifting and rendering, we augment the rendered features \(\mathbf{G}\) with Gaussian positional encoding (GS-PE) and refine them using a lightweight ResNet-based refinement network, \(\mathcal{R}\). Specifically, for each pixel \((d, p)\), we identify the most contributing Gaussian \(i^*(d,p) = \arg\max_i\, w_i(d,p)\), normalize its 3D center within the scene bounding box, and encode it alongside the rendering weight \(w^*(d,p)\) via sinusoidal encoding \(\gamma(\cdot)\) to obtain the positionally-encoded features \(\mathbf{G}'\): \[\mathbf{G}'(d,p) = \mathbf{G}(d,p) + \left[\,\gamma(\tilde{x})\;\|\;\gamma(\tilde{y})\;\|\; \gamma(\tilde{z})\;\|\;\gamma(w^*)\,\right] \label{eq:gs95pe} \tag{4}\] where \(\tilde{\mathbf{x}}(d,p) = (\tilde{x}, \tilde{y}, \tilde{z})\) denotes the normalized 3D center of \(i^*(d,p)\), and \(\gamma(v) = [\sin(\omega_k v),\, \cos(\omega_k v)]_{k=0}^{D'/2-1}\) with \(\omega_k = \omega_0^{-2k/D'}\) and \(D' = \text{C}/4\). This equips the refinement network with explicit 3D structural cues. Afterwards, we get refined features \(\hat{\mathbf{G}} = \mathcal{R}(\mathbf{G}')\). Because the original input-view features \(\mathbf{F}_{\text{ref}}\) serve as a reliable reference, we supervise \(\mathcal{R}\) by aligning the refined input-view features \(\hat{\mathbf{G}}_{\text{ref}} \in \mathbb{R}^{P \times \frac{h}{n} \times \frac{w}{n} \times \text{C}}\) with \(\mathbf{F}_{\text{ref}}\) during training via a cosine similarity loss: \[\mathcal{L}_{\text{feat}} = \left\| 1 - \frac{ \hat{\mathbf{G}}_{\text{ref}} \cdot \mathbf{F}_{\text{ref}}}{ \| \hat{\mathbf{G}}_{\text{ref}} \|_2 \, \| \mathbf{F}_{\text{ref}} \|_2 } \right\|_{2} \label{eq:feature95loss}\qquad{(2)}\] Adaptive feature fusion. Given the refined novel-view features \(\hat{\mathbf{G}}_{\text{tar}}\), we fuse them with the diffusion features \(\mathbf{F}_{\text{tar}}\) to obtain the geometry-augmented features \(\hat{\mathbf{F}}_{\text{tar}}\). A naïve approach is to concatenate \(\mathbf{F}_{\text{tar}}\) and \(\hat{\mathbf{G}}_{\text{tar}}\), then project them back to the original channel dimension via MLPs: \[\hat{\mathbf{F}}_{\text{tar}} = \mathrm{MLPs}\!\left(\mathbf{F}_{\text{tar}} \oplus \hat{\mathbf{G}}_{\text{tar}}\right) \label{eq:naive95fusion}\qquad{(3)}\] However, this fusion relies entirely on the geometry prior and fails once it is corrupted (3). We instead propose an adaptive fusion that incorporates geometry based on its local reliability. Specifically, we first obtain the geometry-attended feature \(\mathbf{F}_{\text{tar}}^{A}\) by cross-attending from \(\mathbf{F}_{\text{tar}}\) (query) to \(\hat{\mathbf{G}}_{\text{tar}}\) (key/value). We then predict a pixel-wise confidence weight \(\mathbf{W}_{\text{tar}} \in [-1, 1]\) via a gating MLP, and combine \(\mathbf{F}_{\text{tar}}\) and \(\mathbf{F}_{\text{tar}}^{A}\) weighted by \(\mathbf{W}_{\text{tar}}\): \[\mathbf{F}_{\text{tar}}^{A} = \mathrm{CrossAttn}\!\left(\mathbf{F}_{\text{tar}},\, \hat{\mathbf{G}}_{\text{tar}}\right), \qquad \mathbf{W}_{\text{tar}} = \mathrm{Tanh}\!\left(\mathrm{MLP}\!\left(\mathbf{F}_{\text{tar}} \oplus \mathbf{F}_{\text{tar}}^{A}\right)\right) \label{eq:gate}\qquad{(4)}\] \[\hat{\mathbf{F}}_{\text{tar}} = \mathbf{F}_{\text{tar}} + \mathbf{W}_{\text{tar}} \cdot \mathbf{F}_{\text{tar}}^{A} \label{eq:adaptive95fusion}\qquad{(5)}\] To verify that \(\mathbf{W}_{\text{tar}}\) behaves as intended, we measure the Pearson correlation between the absolute confidence weights, \(\left| \mathbf{W}_{\text{tar}} \right|\) and 3D-GS uncertainty [65] across 3 reflective scenes (3). The consistent negative correlation (\(r\approx{\color{red}-0.30}\)), which strengthens as denoising progresses and is most pronounced in reflective regions, supports our design claim: Adaptive Fusion downweights geometric guidance in regions where the geometry prior is unreliable.

3.3 Integration into Video Diffusion↩︎

Integrating 3D-GS into video diffusion. GeoNVS incorporates explicit 3D structural information into the denoising process by coupling 3D Gaussians with the diffusion model in feature space. At each denoising timestep \(t\), GS-Adapter embeds \(\mathbf{F}_{\text{ref}}^{t}\) into 3D Gaussians, rasterizes geometry-aware novel-view features, and adaptively injects them back into the diffusion model to correct geometrically inconsistent features throughout the entire denoising process.

Multi-scale feature aggregation. Multi-scale features play a crucial role in visual understanding [66]–[70] and generation [71], [72]. To effectively combine geometry-guided features with diffusion features across different spatial scales, we leverage multi-scale encoder features via a DPT-style [70] aggregation strategy, upsampling and hierarchically merging them to a unified feature of \(\mathbf{F} \in \mathbb{R}^{(P+Q) \times \frac{h}{n} \times \frac{w}{n} \times \text{C}}\) before feeding into GS-Adapter (see 2). The geometry-corrected features are then downsampled to the original resolutions and added through skip connections to the decoder. We validate this design in 4.6.

Loss function. We train the multi-scale fusion module and GS-Adapter alongside trainable LoRA [64] layers injected into the attention modules of the frozen video diffusion model. The total training objective is formulated by combining the latent space alignment loss [73] and the feature alignment loss from ?? : \[\mathcal{L} = \mathcal{L}_{\text{latent}} + 0.05 \mathcal{L}_{\text{feat}} \label{eq:total95loss}\qquad{(6)}\]

4 Experiment↩︎

4.1 Overview↩︎

We organize our experiments to validate three key claims of GeoNVS. First, GS-Adapter improves geometric consistency and camera controllability, quantified by camera pose error and Chamfer Distance, and further supported by a feature consistency metric ([subsec:geo_exp] [subsec:geo_feat_exp]). Second, GeoNVS achieves state-of-the-art photorealistic quality across diverse scenes, evaluated in 4.3 on 9 scenes under small/large-viewpoint and long-trajectory settings. Third, GS-Adapter supports plug-and-play integration with diverse geometry models without additional training, and is compatible with other diffusion backbones; we further show that its computational overhead can be substantially reduced via voxel-based Gaussian pruning with negligible performance degradation (4.6).

4.2 Implementation Details↩︎

We implement GeoNVS on two baselines, SEVA [11] and CameraCtrl [12], to demonstrate its compatibility. While the SEVA-based GeoNVS supports an arbitrary number of input and output views, the CameraCtrl-based GeoNVS is limited to a single reference view due to constraints of the baseline model.

Training setup. We use the DL3DV-10K [74] dataset to train GeoNVS. Using VGGT [7] and InstantSplat [17], we construct 3D Gaussians paired with 150K clips with varying reference-to-novel view splits \((P, Q) \in \{(1, 20),\;(3, 18),\;(6, 15),\;(9, 12),\;(12, 9)\}\) where the total number of views is fixed at 21. Please refer to Sec. E of the supplementary material for details on dataset preprocessing. Both SEVA- and CameraCtrl-based GeoNVS are trained at resolutions \((h, w)\) of \(384 \times 384\) and \(320 \times 576\) with initial learning rates of \(3 \times 10^{-5}\) and \(1 \times 10^{-4}\), respectively. Our model is fine-tuned using LoRA [64] on 8 NVIDIA A6000 GPUs (48GB). We set the rank and \(\alpha\) to 16 for SEVA, and 4 for CameraCtrl. The training process takes approximately 80 hours with a total batch size of 8.

Inference setup. We evaluate our method on the SEVA benchmark [11], consisting of 9 scenes [74]–[82] with input views \(P\) varying from 1 to 9. We include Tanks and Temples [78] under the CF-3DGS sampling protocol [83] for the view interpolation task (T&T O split). For fair comparison, all combined methods (generative + geometry prior) use the same best-performing geometry prior per dataset. Note that MVSplat360 [26] is limited to MVSplat [4] by design. We evaluate all methods on a single A6000 GPU. Please refer to Sec. B of the supplementary material for details, including evaluation dataset splits.

Input-level injection of geometry prior. As a baseline for input-level geometry conditioning into the diffusion model in 1-(b), we implement an input-level injection approach following prior work [27], [28], [84]. Given the VAE-encoded latents \(\mathbf{z}_\text{tar}\) of 3D-GS rendered novel-view images, we initialize the noisy latents as \(\mathbf{x}_\text{tar} \leftarrow \mathbf{x} \cdot \sigma_{t_0} + \mathbf{z}_\text{tar}\), where \(\sigma_{t_0}\) is the noise scale at the start timestep \(t_0 = \lfloor T \cdot s \rfloor\) determined by a strength \(s \in [0, 1]\). A larger \(s\) allows the diffusion model more freedom to deviate from the geometry prior, while a smaller \(s\) enforces stronger adherence to the rendered geometry; we set \(s = 0.2\). We compare this approach against our GS-Adapter, which modulates internal diffusion features with a geometry prior rather than relying on rasterized images.

4.3 Perceptual Quality↩︎

Metrics. We evaluate perceptual quality using PSNR, SSIM, and LPIPS — standard metrics in NVS — and report PSNR and SSIM as the primary metrics in the main manuscript; full results are in Sec. F of the supplementary material.

Robustness to viewpoint overlap. Following SEVA [11], we evaluate on Set NVS, which generates a set of target views in arbitrary order, decomposed into small- and large-viewpoint settings based on the CLIP [58] distance between reference and target views (threshold: 0.11). As shown in ¿tbl:tab:method95comparison? ¿tbl:tab:large95viewpoint? 5 6, our method consistently outperforms comparison methods across most datasets, improving over our baseline SEVA [11] on all benchmarks and demonstrating robustness to both small and large viewpoint changes.

Long trajectory generation. Trajectory NVS organizes target views as a smooth camera trajectory forming a video sequence. Leveraging the geometry prior from reference views, we adopt the two-pass sampling method from SEVA [11] to generate consistent, non-flickering long-term videos. As shown in [tbl:tab:long95traj95psnr] [tab:long95traj95geo], our method consistently improves over baseline SEVA by a large margin, with performance scaling favorably with more reference views.

4.4 Geometric Consistency and Camera Controllability↩︎

Metrics. To validate geometric fidelity, we reconstruct point clouds and estimate camera trajectories from generated videos using ViPE [86]. Camera controllability is evaluated by the translation error \(T_{\textit{err}}\)[cm] and rotation error \(R_{\textit{err}}\)[deg] between estimated and ground-truth camera poses following [12]. Geometric consistency is quantified by mean Chamfer Distance (\(CD\)) between point clouds reconstructed from generated and ground-truth videos [24], [87]. We exclude scenes where ViPE [86] fails to reconstruct the ground-truth video (\(T_{\textit{err}}\) or \(R_{\textit{err}}\) \(>\) 50).

Geometric consistency. As shown in [tab:long95traj95geo], GeoNVS substantially improves geometric consistency over the SEVA baseline across all datasets. This is further corroborated by ¿tbl:tab:geometric95consistency?, where our method consistently outperforms input-level geometry injection approaches — Difix3D[28] and SEVA with input-level injection — by a large margin, demonstrating that feature-space modulation via GS-Adapter is more effective than injecting geometry at the input level.

Camera controllability. GeoNVS substantially improves camera controllability over the SEVA baseline. As shown in [tab:long95traj95geo] 8, the gains in synthesis quality stem primarily from improved camera controllability and geometry enhancement, with our method being especially effective in reducing translation deviation. ¿tbl:tab:geometric95consistency? further supports this: GeoNVS achieves consistently lower pose errors than the SEVA baseline across all three datasets, while competing methods that inject geometry priors at the input level suffer catastrophic degradation in controllability, with translation and rotation errors far exceeding those of the baseline. These results indicate that anchoring diffusion features to explicit 3D structure is key to reliable camera controllability.

Geometry prior extrapolates to unseen regions. To evaluate whether geometry priors improve synthesis beyond observed regions, we construct co-visibility masks using VGGT-derived [7] point clouds following the co-visible mask construction in [17]. Specifically, we project only the input-view 3D points back onto each novel view, yielding per-pixel co-visibility masks; we measure PSNR\(_\text{V}\) on co-visible and PSNR\(_\text{U}\) on non-co-visible (unseen) regions. As shown in Table ¿tbl:tab:geometric95consistency?, our method consistently outperforms baselines on PSNR\(_\text{U}\) across all datasets, with notable gains of +1.06 on Mip360 and +2.08 on DL3DV over SEVA. This demonstrates that our geometry guidance, explicitly conditioned on visible regions, facilitates geometry-consistent extrapolation into unseen areas.

4.5 Feature analysis of GS-Adapter↩︎

To qualitatively validate GS-Adapter, we visualize intermediate features throughout the denoising process in 4. The initial diffusion features \(\mathbf{F}_\text{tar}^0\) are spatially incoherent due to high noise levels, whereas the rendered Gaussian features \(\mathbf{G}_\text{tar}\) consistently exhibit sharp structures derived from the 3D geometry prior. After refinement, \(\hat{\mathbf{G}}_\text{tar}\) recovers fine-grained details lost during lifting and rendering. Upon fusion, the geometry-corrected diffusion features progressively align with the underlying 3D structure as denoising advances, ultimately yielding photorealistic novel-view images with improved geometric consistency. These observations confirm that GS-Adapter effectively anchors diffusion features to explicit 3D geometry throughout the entire denoising process.

4.6 Ablation Studies↩︎

Compatibility with geometry priors. A key advantage of GeoNVS is its plug-and-play compatibility with diverse feed-forward geometry models without requiring any additional training. To validate this, we integrate GS-Adapter with four different geometry priors [4], [5], [7], [8] and evaluate using SEVA [11] as the generative backbone in ¿tbl:tab:ablation95lrms1?. Both fusion variants consistently improve over the SEVA baseline regardless of the geometry prior used. However, Naïve Fusion remains sensitive to prior quality: while VGGT yields strong gains on T&T (+3.37 dB), Pi3 degrades below the baseline on DTU (14.83 vs. 17.03), as observed in 3. Adaptive Fusion substantially mitigates this sensitivity, achieving robust and consistently superior results across all priors (19.02–19.18 dB on DTU; 19.22–19.26 dB on CO3D), and enabling flexible substitution of the geometry backbone without performance degradation.

GeoNVS with CameraCtrl. To demonstrate that GS-Adapter generalizes beyond SEVA, we integrate it into CameraCtrl [12] in [tbl:tab:ablation95lrms2]. Standalone geometry priors consistently underperform the CameraCtrl baseline, suggesting that geometry alone is insufficient in sparse-view settings. GeoNVS with Naïve Fusion improves over both the baseline and the standalone priors across most settings, achieving up to +1.66 PSNR gain on average over the baseline, suggesting that GS-Adapter provides complementary benefit even on top of a strong geometry prior and is applicable beyond a single diffusion backbone.

Analysis of model components. Table 1 validates the contribution of each design choice in GS-Adapter. Both the feature consistency loss \(\mathcal{L}_\text{feat}\) and Gaussian positional encoding (GS-PE) contribute to performance, as removing either component leads to consistent drops across benchmarks, indicating that explicit 3D structural cues and fine-grained supervision are both necessary for recovering details lost during the lifting-and-rendering process. RefineNet consistently yields performance gains, demonstrating its independent contribution beyond 3D-GS splatting. Finally, using multi-scale features together in GeoNVS provides consistent improvements, indicating that geometry-guided features across multiple spatial resolutions are complementary.

Comparison with input-level injection of geometry. As shown in ¿tbl:tab:geometric95consistency?, input-level injection of geometry priors causes severe degradation in camera controllability and geometric consistency, performing even worse than SEVA, a purely generative baseline. This failure stems from the fact that rasterized images introduce view-dependent color noise that corrupts the structural signal rather than reinforcing it. In contrast, the proposed GS-Adapter leverages only the structural information encoded in 3D Gaussians to modulate internal diffusion features, consistently reducing camera pose errors and Chamfer Distance across all datasets. 1 further reinforces this finding: input-level injection yields synthesis quality gains only when the geometry prior is near-perfect, whereas GS-Adapter consistently improves PSNR regardless of prior quality. These results collectively demonstrate that feature-level geometry modulation is both more robust and more effective than image-level injection.

Pose-free experiment. We extend GeoNVS to unknown pose scenarios by integrating ViPE [86], which outperforms VGGT [7] in robustness on complex scenes. Given poses estimated by ViPE from input images, we optimize 3D-GS using the VGGT point cloud, with scale aligned to ViPE poses solved via 1 .

r0.45 0.45pt

As shown in the right table, GeoNVS maintains high synthesis quality without ground-truth intrinsics or extrinsics. On T&T [78], the ViPE pose setting slightly surpasses the GT setting, likely because ViPE estimates poses more robustly than the noisy COLMAP [89], [90]-derived poses.

Runtime and memory efficiency. We measure inference time, peak GPU memory, and PSNR across 10 scenes from DL3DV [74] to analyze the computational overhead relative to SEVA (7). The primary bottleneck is the number of Gaussians from the geometry prior, which raises inference time to 4.19 sec/frame and peak memory to 18.03 GB. Applying voxel-based Gaussian pruning [88] reduces these to 1.89 sec/frame (2.22\(\times\) speedup) and 12.93 GB, on par with SEVA’s 12.81 GB, while maintaining a PSNR advantage over SEVA.

5 Conclusions↩︎

We present GeoNVS, a geometry-grounded novel-view synthesizer that enhances geometric fidelity and camera controllability using GS-Adapter. Unlike prior methods that inject geometry at the input level, GS-Adapter modulates internal diffusion features using 3D Gaussian priors in a plug-and-play manner, enabling zero-shot compatibility with any geometry model and diffusion backbone. Extensive experiments across 9 scenes and 18 settings demonstrate state-of-the-art performance, achieving 11.3% and 14.9% improvements over SEVA and CameraCtrl, with up to \(2\times\) reduction in translation error and \(7\times\) in Chamfer Distance. GeoNVS improves synthesis in non-co-visible regions by up to 2.08 dB, confirming that feature-level 3D grounding facilitates extrapolation beyond observed regions. Finally, GeoNVS supports pose-free scenarios via SfM methods [83], [86] and reduces inference overhead by \(2.2\times\) with negligible performance degradation.

Limitations and future work. While our proposed method peforms well in sparse-view and long-range trajectory NVS, its generation ability degrades in regions distant from input views. We attribute this to increasingly uncertain feature rendering in GS-Adapter as the distance from input views grows. Our method occasionally produces blurry textures due to information loss during feature lifting, which can reduce perceptual sharpness despite well-preserved structure (9). We leave addressing these limitations to future work.

Acknowledgements. This work was supported by the InnoCORE program of the Ministry of Science and ICT (26-InnoCORE-01), and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2026-25473963).

Supplementary Material↩︎

This supplementary material is organized as follows.

• 6 summarizes the mathematical notation used throughout the paper.

• 7 provides details on dataset splits and comparison methods.

• 8 presents additional experiments, including further analysis and ablations on input-level injection strength, feature uplifting, feature refinement, and adaptive fusion gating.

• 9 describes the detailed architectures of the multi-scale fusion module and RefineNet.

• 10 explains the training dataset preprocessing pipeline.

• 11 provides additional quantitative results in SSIM and LPIPS.

• 12 presents additional qualitative results across diverse settings.

• 13 discusses limitations and failure cases of GeoNVS.

6 Notation Table↩︎

 4 summarizes the mathematical notation used in the paper by category.

7 Detail of Evaluation Benchmark↩︎

We evaluate GeoNVS across 9 scenes and 18 settings drawn from 9 public benchmarks following SEVA [11]: OmniObject3D [82], RealEstate10K [80], LLFF [81], DTU [75], CO3D [76], WRGBD [77], Mip-NeRF 360 [79], DL3DV [74], and Tanks and Temples [78]. We detail the dataset splits and evaluation protocols of comparison methods in [subsec:supple_dataset_split] [subsec:supple_evaluation_method].

7.1 Dataset Splits↩︎

The data splits used in the evaluation tables include ReconFusion [23] (R), 4DiM [91] (D), SEVA [11] (S), pixelSplat [92] (P), ViewCrafter [85] (V), LongLRM [93] (L), and our own split adapted from CF-3DGS [83] (O).

7.2 Evaluation Details of Comparison Methods↩︎

MVSplat [4], HiSplat [6], DepthSplat [5]. We use these feed-forward methods to directly regress Gaussian splats \(\mathcal{G}\) from the input-view images \(\mathbf{I}_{\text{ref}}\). These methods require posed images (images with corresponding poses \(\boldsymbol{\pi}\)).

VGGT [7], Pi3 [8]. Using the point cloud and camera parameters predicted by the feed-forward methods, we initialize the Gaussian splats \(\mathcal{G}\) directly from the point cloud. We then apply the co-visibility-based redundancy elimination strategy of InstantSplat [17] to prune overlapping regions, and further optimize the 3D-GS representation [2] with the reference view images \(\mathbf{I}_{\text{ref}}\). The scale ambiguity between the predicted camera poses and the ground-truth point cloud is resolved using 1 .

ViewCrafter [85]. In the original implementation, ViewCrafter operates in an exploratory manner without any knowledge of target viewpoints, relying on dust3r [94] to estimate camera parameters and iteratively expanding the point cloud with generated frames. For evaluation, we instead provide ground-truth camera poses and use \(P\) reference views alongside 5 additional viewpoints for geometry estimation, then render along the ground-truth trajectory before passing to the diffusion model. Note that this setting provides an advantage over the original by eliminating camera estimation error, and thus can be regarded as an upper-bound approximation of ViewCrafter’s performance.

MotionCtrl [14]. We employ only the CMCM module from MotionCtrl for camera control, following the original inference protocol with 25 denoising steps and the same classifier-free guidance scale. All videos are generated at \(576 \times 1024\) resolution and resized to \(384 \times 384\) for evaluation.

CameraCtrl [12]. We use the SVD [73] based CameraCtrl officially provided by the author. The inference process follows the original paper, using 25 steps and the same conditional-guidance scale. All videos are generated at a resolution of \(320 \times 576\) and subsequently resized to \(384 \times 384\) for evaluation.

SEVA [11]. We adopt the officially released v1.0 model provided by the authors; our GeoNVS framework is similarly built upon SEVA v1.0. Inference is performed following the settings of the original paper, using 50 denoising steps with a CFG scale of 2.0. All videos are rendered at a resolution of \(384 \times 384\).

MVSplat360 [26]. We first render novel-view features from MVSplat and use these features as input to generate novel-view images using SVD, following the original implementation.

GenFusion [27]. Rather than adopting GenFusion’s direct 3D reconstruction pipeline, we first leverage geometry priors from a regression-based model to render novel-view RGB, depth (RGBD) images. These noisy RGBD predictions are then fed into GenFusion for refinement. All videos are generated at a resolution of \(320 \times 512\) and subsequently resized to \(384 \times 384\) for evaluation.

Difix3D [28]. We utilize the Difix3D model by providing a reference image together with the novel-view image rendered by the geometry prior (regression-based model). The reference image is chosen from the input-view set \(\boldsymbol{I}_{\text{ref}}\) based on the camera-center distance to the target novel-view viewpoint.

7.3 Geometry prior used for combined methods↩︎

For fair comparison, all combined methods [27], [28] and ours use the best-performing geometry prior per dataset in ¿tbl:tab:method95comparison?. 3 lists the geometry priors used in each benchmark for small-viewpoint NVS. Note that MVSplat360 [26] is an exception, as it is limited to MVSplat [4] by design.

8 Additional Experiments↩︎

8.1 Injection Strength in Input-Level Integration↩︎

To ensure a fair comparison with input-level injection approaches, we analyze performance across different inpaint strength levels \(s\), where smaller \(s\) enforces stronger adherence to the geometry prior and larger \(s\) approaches the purely

generative baseline [11]. As shown in Fig. ¿fig:fig:inpaint95ablation?, input-level injection performs best at small \(s\) on DL3DV-10K, where the geometry prior dominates, but degrades monotonically as \(s\) increases toward the generative baseline. On MipNeRF-360, performance is unstable across all \(s\) values, suggesting that noisy rasterized colors corrupt the structural signal regardless of inpaint strength. In contrast, GS-Adapter consistently outperforms input-level injection across all \(s\) values on both datasets, demonstrating that feature-level modulation yields more stable and consistently superior results regardless of the geometry prior strength.

8.2 Ablation on Feature Uplifting: Point Cloud vs 3D-GS↩︎

In the feature uplifting process, restricting each ray to consider only the single Gaussian with the highest rendering weight reduces the process to a hard assignment scheme, in which each pixel’s feature is attributed solely to the most opaque surface point along that ray, which is mathematically equivalent to projecting features onto a point cloud. In contrast, the soft assignment scheme in ?? aggregates all Gaussians along a ray as a rendering-weight-proportional weighted sum, following the same principle as alpha-compositing to continuously accumulate features across the volume. This formulation mitigates the fragility of hard assignment in regions where multiple Gaussians partially contribute to a single pixel, as a single dominant Gaussian may fail to capture the full structural signal of the scene. Since Gaussians with moderate rendering weights are entirely discarded under hard assignment, soft assignment ensures

that all contributing Gaussians accumulate feature information proportional to their geometric relevance, yielding a smoother and more consistent feature field upon rasterization into novel views and ultimately improving both geometric consistency and synthesis quality. We validate this in Figure above by varying \(k \in \{1, 16, 32, 64, 128, 255\}\) on Mip-NeRF 360 and DL3DV-10K, where \(k{=}1\) and \(k{=}255\) correspond to hard and full soft assignment, respectively. PSNR consistently improves with increasing \(k\) on both datasets, confirming the effectiveness of soft assignment.

8.3 Validity of Feature Splatting and Refinement↩︎

Feature splatting in the latent domain [56] implicitly assumes that diffusion latent features share the linear additive properties of RGB colors. However, certain feature channels may not correlate with geometric opacity, leading to a potential gap in the rendered features. Unlike prior works [62], [63] that apply feature splatting to discrete tasks such as segmentation, novel-view synthesis is particularly sensitive to this gap, as any feature distortion directly degrades perceptual output quality. To mitigate this, RefineNet is trained with the feature refinement loss (?? ) to recover fidelity lost during compositing. Beyond the ablation in 1, we further evaluate the effect of refinement by comparing (a) lifting-rendering (w/o refinement) against (b) lifting-rendering-refinement, with per-scene optimization applied to the VAE-encoded diffusion latent space. We report PSNR and SSIM between the decoded novel-view images and the ground truth across 540 randomly sampled scenes from the DL3DV [74] test set. As shown in the figure below, refinement consistently improves both metrics across all input-view settings, with gains widening as the number of input views increases (+1.37 dB PSNR at 9 views), demonstrating that it recovers feature fidelity lost during compositing.

8.4 Sensitivity to Classifier-free Guidance Scale.↩︎

SEVA [11] is sensitive to the classifier-free guidance (CFG) scale, which can induce color shifts or structural distortions regardless of viewpoint difficulty. As shown in the figure below, these artifacts appear in SEVA across varying CFG scales, whereas our method remains robust: the geometric conditioning provided by GS-Adapter stabilizes the generation and suppresses such distortions.

9 Model Architecture Details↩︎

Detailed descriptions of the multi-scale fusion module and the RefineNet \(\mathcal{R}\) used in the feature refinement of GS-Adapter are provided in 3.3 and 3.2.

9.1 Multi-Scale Fusion↩︎

The detailed architecture of the multi-scale fusion module in GeoNVS is illustrated in 10. This module aggregates multi-scale features from the diffusion module to produce a single feature map at a unified resolution.

9.2 RefineNet Architecture↩︎

The detailed architecture of the RefineNet module in GS-Adapter is illustrated in 11. This module takes the rasterized Gaussian feature \(\mathbf{G}\) as input and produces a refined feature representation \(\hat{\mathbf{G}}\).

10 Training Dataset Preprocessing↩︎

We construct our training dataset from DL3DV-10K [74] by preprocessing each scene into input–target view pairs annotated with pretrained 3D Gaussian representations (12). Images are resized to \(480 \times 270\) or \(960 \times 540\) resolution, and camera poses are converted to world-to-camera extrinsics with normalized intrinsics. For structured view selection in 13, we construct a pose graph over all frames using pairwise combined pose distances [42] and connect frames within a dynamic distance threshold that guarantees at least \(N_{\text{group}}=21\) near neighbors per frame. Anchor nodes are sampled proportionally to their graph degree, yielding \(K=3\) independent view groups per scene. Within each group, input views (\(P \in \{1, 3, 6, 9, 12\}\)) are selected via K-means clustering on camera position and orientation to maximize spatial diversity. The remaining frames serve as novel-view candidates and are divided into easy and hard subsets based on their minimum CLIP cosine distance [58] to the input views, where the bottom 60% by distance constitute easy samples. Both subsets are filtered by a frustum IoU threshold of \(\tau_{\text{IoU}} = 0.4\) to ensure sufficient overlap between input and target cameras. Each resulting input–target pair is then processed by VGGT [7] to obtain an initial point cloud, which is further optimized into 3D Gaussians given the reference images, parameterized by position \(\boldsymbol{\mu}\), rotation quaternion \(\mathbf{q}\), scale \(\mathbf{s}\), opacity \(\alpha\), and spherical harmonic coefficients.

11 Additional Quantitative Results↩︎

We provide additional SSIM and LPIPS results:

• Small-viewpoint set NVS in ¿tbl:tab:supple95smallnvs?, corresponding to ¿tbl:tab:method95comparison?.

• Large-viewpoint set NVS in ¿tbl:tab:supple95large95viewpoint?, corresponding to ¿tbl:tab:large95viewpoint?.

• Long trajectory NVS in ¿tbl:tab:supple95long95traj?, corresponding to [tbl:tab:long95traj95psnr].

12 Additional Qualitative Results↩︎

We provide additional qualitative results:

• Small-viewpoint set NVS in 14 15 16 17, corresponding to ¿tbl:tab:method95comparison?.

• Large-viewpoint set NVS in 18 19, corresponding to ¿tbl:tab:large95viewpoint?.

• GeoNVS with CameraCtrl in 21, corresponding to [tbl:tab:ablation95lrms2].

• GeoNVS with geometry priors in 20, corresponding to ¿tbl:tab:ablation95lrms1?.

• Trajectory NVS with ViPE [86] Reconstruction in 22 23, corresponding to [tab:long95traj95geo].

• Input-level injection vs Ours in 24, corresponding to ¿tbl:tab:geometric95consistency?.

13 Limitations and Failure Cases↩︎

As shown in Fig. 25, GeoNVS exhibits two main failure modes. First, geometry inconsistency arises in thin structures such as poles and fences, where sparse Gaussian representations fail to accurately capture fine-grained geometry, propagating errors into the rendered novel-view features. Second, blurry textures appear in heavily occluded regions invisible from any reference view, where insufficient geometric guidance leaves the diffusion model to hallucinate plausible but blurry content. Incorporating uncertainty-aware Gaussian representations or inpainting-based priors for occluded regions remains a promising direction for future work.

References↩︎