1 Introduction↩︎

High-quality novel view synthesis at real-time frame rates is a central objective in 3D vision, with applications ranging from virtual reality to robotics. Neural radiance fields (NeRFs) [1] and related neural field representations [2]–[5] have driven substantial progress in this area by representing scenes as continuous functions and rendering images through volumetric integration. Due to their continuous formulation and robust optimization behavior, neural field methods have demonstrated strong performance in challenging real-world settings [6]. However, their reliance on volumetric rendering often leads to high computational cost, limiting their applicability when real-time inference is required.

Recently, 3D Gaussian Splatting (3DGS) [7] has emerged as an alternative paradigm, providing an efficient alternative thanks to its rasterization-based nature. By representing scenes as collections of anisotropic Gaussian primitives, 3DGS achieves high-quality rendering at real-time frame rates. Despite this advantage, its optimization is highly dependent on the quality of the initial geometry. This limitation is particularly pronounced in sparse-view settings, where limited image observations often lead to incomplete or noisy point-cloud reconstructions. In such cases, Gaussian primitives may be poorly placed or missing in under-observed regions, making it difficult for optimization to recover accurate geometry and appearance.

Sparse-view reconstruction [8]–[10] remains especially challenging in this context. With only a small number of input views, traditional structure-from-motion pipelines [11], [12] may fail to recover sufficient scene structure, which directly affects downstream Gaussian Splatting optimization. While neural radiance fields can often produce plausible reconstructions from sparse observations due to their continuous volumetric representation, Gaussian Splatting typically lacks an equally flexible mechanism for reasoning about regions that are weakly supported by the initial point cloud.

In this work, we propose AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. Our key idea is to use a radiance field not as the final rendering representation, but as a source of additional supervision for Gaussian Splatting. Given sparse input views, we first train a radiance field and use it to synthesize images from nearby novel viewpoints. These synthetic views increase the effective supervisory coverage of the sparse input set and provide additional constraints during Gaussian optimization.

RadSplat [13] is closely related to our work, as it also leverages radiance fields to improve Gaussian Splatting. However, while RadSplat uses radiance-field information to improve Gaussian initialization, AugSplat uses radiance-field renderings as auxiliary training views. This makes our method complementary to initialization-based approaches.

We explore two variants of this strategy. Staged AugSplat first optimizes Gaussian Splatting using synthetic radiance-field views before switching to real images, while Dual AugSplat jointly trains on real and synthetic views with a decaying weight on the synthetic supervision. By using radiance fields to augment sparse supervision and Gaussian Splatting for efficient inference, AugSplat combines the robustness of implicit representations with the real-time rendering speed of explicit Gaussian primitives.

In summary, our contributions are as follows:

• We introduce AugSplat, a radiance-field-guided augmentation framework for improving Gaussian Splatting in sparse-view settings.

• We propose two training strategies, Staged AugSplat and Dual AugSplat, for incorporating synthetic radiance-field views into Gaussian Splatting optimization.

• We show that radiance-field-based view augmentation improves sparse-view Gaussian Splatting on Mip-NeRF360 [14] scenes. Staged AugSplat achieves the strongest average reconstruction quality, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training.

• We analyze training dynamics and show that AugSplat improves early-stage optimization rather than merely benefiting from longer training.

2 Related Work↩︎

Neural radiance fields (NeRFs) [1] represent scenes as continuous volumetric functions, enabling high-quality novel view synthesis by modeling both geometry and appearance. Subsequent work has improved reconstruction quality, scalability, and anti-aliasing for complex scenes [14], [15], while efficient neural representations such as Instant-NGP [16] have substantially reduced training and rendering costs. Nevertheless, many NeRF-based methods still rely on volumetric rendering and therefore do not generally provide the same real-time inference performance as rasterization-based Gaussian Splatting.

In AugSplat, we use this robustness not for final rendering, but to synthesize additional supervision for a real-time Gaussian representation.

Novel view synthesis from sparse observations is highly under-constrained, since large portions of the scene may be weakly observed or entirely unseen. NeRF-based methods address this problem using regularization, learned priors, or auxiliary geometric constraints. For example, RegNeRF [8] regularizes geometry and appearance in unobserved views, while SparseNeRF [17] leverages depth-ranking constraints to improve few-shot reconstruction.

More recently, sparse-view settings have also been studied for Gaussian Splatting. Methods such as SparseGS [18] and DNGaussian [19] introduce additional constraints, depth priors, or regularization strategies to improve Gaussian optimization from limited views. Our work is complementary to these approaches: rather than directly regularizing the Gaussian representation, we use a radiance field to generate additional synthetic training views that increase the effective coverage of the sparse input set.

3D Gaussian Splatting (3DGS) [7] represents scenes using anisotropic Gaussian primitives and renders them efficiently through rasterization. This formulation enables high-quality real-time novel view synthesis, but also makes optimization sensitive to the quality of the initial point cloud and the distribution of training views. This sensitivity becomes particularly problematic in sparse-view settings, where incomplete geometry can lead to missing or poorly placed Gaussian primitives.

A large body of work has extended Gaussian Splatting to applications such as human and avatar reconstruction [20]–[23], 3D generation [24]–[26], SLAM [27]–[30], dynamic scene reconstruction [31], [32], and open-vocabulary scene understanding [33], [34]. These works demonstrate the flexibility of Gaussian representations, while our focus is specifically on improving their robustness under sparse-view supervision.

Several recent methods combine neural field and Gaussian Splatting representations to exploit their complementary strengths. RadSplat [13] is closest to our work, as it also uses a radiance field to improve the robustness of Gaussian Splatting. Rather than deriving Gaussian primitives or representation-level priors from the radiance field, AugSplat treats the radiance field as a data generator: it renders nearby synthetic views and uses ensemble uncertainty to weight their contribution during optimization.

Other approaches integrate neural components more directly into Gaussian representations [35], [36], or use learned structures to predict Gaussian attributes in a view-adaptive manner [37]. In contrast, AugSplat keeps the Gaussian representation unchanged and instead uses radiance-field renderings as additional training views.

3 Method↩︎

Given a sparse set of calibrated input images, AugSplat first trains a radiance-field ensemble to obtain a continuous scene representation and uncertainty estimates. We then render nearby synthetic views from this ensemble and use the resulting images and confidence maps as auxiliary supervision for Gaussian Splatting. We consider two ways of incorporating this supervision: a staged strategy that uses synthetic views only during an initial warm-up phase, and a dual strategy that jointly optimizes real and synthetic supervision with a decaying synthetic loss weight.

3.1 Neural Radiance Fields for Improved Supervisory Coverage↩︎

A Neural Radiance Field (NeRF) represents a function \[f_\phi: \mathbb{R}^3 \times \mathbb{S}^2 \rightarrow \mathbb{R}^+ \times \mathbb{R}^3,\] mapping a 3D point \(\mathbf{x}\) and viewing direction \(\mathbf{d}\) to a volume density \(\sigma\) and color \(\mathbf{c}\). Given a camera ray, pixel colors are obtained by alpha compositing samples along the ray: \[\mathbf{c}_{\text{NeRF}} = \sum_{k=1}^{K} \tau_k \alpha_k \mathbf{c}_k,\] where \[\tau_k = \prod_{j=1}^{k-1} (1-\alpha_j), \quad \alpha_k = 1 - e^{-\sigma_k \delta_k}.\]

The radiance field parameters \(\phi\) are optimized by minimizing the photometric reconstruction loss over sampled training rays: \[\mathcal{L}_{\text{NeRF}}(\phi) = \sum_{\mathbf{r} \in \mathcal{R}_{\text{batch}}} \| \mathbf{c}^{\phi}_{\text{NeRF}}(\mathbf{r}) - \mathbf{c}_{\text{image}}(\mathbf{r}) \|_2^2.\]

We leverage neural radiance fields to increase the effective geometric coverage available during Gaussian Splatting optimization. Given a sparse set of input views, we train an ensemble of \(M\) radiance fields. The ensemble provides both synthetic color predictions and an uncertainty estimate, allowing us to downweight synthetic supervision in regions where the radiance field predictions disagree. We use this confidence weighting as a practical heuristic to reduce the influence of uncertain synthetic pixels.

We then sample a set of nearby novel camera poses \(\mathcal{C}_{\text{syn}} = \{ \mathbf{T}_i \}_{i=1}^{N_{\text{syn}}}\) by interpolating between existing training poses and applying small pose perturbations. We restrict these poses to remain close to the observed camera distribution, which helps maintain reliable radiance-field renderings while expanding scene coverage.

For each synthetic pose, we render one image from each ensemble member and aggregate the predictions using the per-pixel median: \[I_i^{\text{nerf}}(\mathbf{p}) = \operatorname{median}_{m=1}^{M} I^{\text{nerf}}_{i,m}(\mathbf{p}),\] where \(\mathbf{p}\) indexes image pixels. This yields a set of synthetic images \(\mathcal{I}_{\text{nerf}} = \{ I_i^{\text{nerf}} \}_{i=1}^{N_{\text{syn}}}\).

To estimate per-pixel confidence, we compute the variance of the ensemble color predictions at each pixel. Let \(V_i(\mathbf{p})\) denote the average RGB variance across ensemble members for synthetic view \(i\). We normalize this variance using per-image robust lower and upper bounds, computed from the ensemble variance map: \[v_{\text{lo},i} = P_5(V_i), \quad v_{\text{hi},i} = P_{95}(V_i),\] where \(P_5\) and \(P_{95}\) denote the 5th and 95th percentiles, respectively. The normalized variance is then given by \[V_{\text{norm},i}(\mathbf{p}) = \mathrm{clip}\left( \frac{V_i(\mathbf{p}) - v_{\text{lo},i}}{v_{\text{hi},i} - v_{\text{lo},i}}, 0, 1 \right).\]

The normalized variance is converted into a confidence weight: \[w_i(\mathbf{p}) = \left(1 - V_{\text{norm},i}(\mathbf{p})\right)^{\gamma}.\]

This yields a weighted synthetic dataset \(\mathcal{D}_{\text{nerf}} = \{ (I_i^{\text{nerf}}, w_i) \}_{i=1}^{N_{\text{syn}}}\).

Intuitively, the confidence weights encourage the Gaussian model to focus on stable, low-frequency structures in the synthetic views rather than overfitting uncertain high-frequency details produced by the radiance field. This is particularly useful during the early stages of Gaussian optimization, when the primitives are still coarse and the main objective is to recover a reliable geometric layout. As training progresses, supervision from real images can then refine high-frequency appearance details.

3.2 Radiance Field-Guided Gaussian Splatting↩︎

3D Gaussian Splatting (3DGS) represents a scene as a set of \(G\) anisotropic Gaussian primitives parameterized by position, opacity, color, scale, and rotation. Rendering is performed by rasterizing and alpha-compositing visible Gaussians: \[\mathbf{c}_{\text{GS}} = \sum_{g=1}^{G} \mathbf{c}_g \alpha_g \tau_g,\] with transmittance defined analogously to NeRF.

While enabling efficient real-time rendering, 3DGS relies on an explicit set of primitives initialized from reconstructed geometry. Although densification can add new primitives during training, the optimization remains strongly influenced by the initial point cloud and the camera views that provide gradient signal. In sparse-view settings, missing or inaccurate initial geometry can therefore lead to poor local optima.

We propose AugSplat, a framework that leverages radiance-field-generated views to improve the early stages of Gaussian Splatting optimization. Our key insight is that nearby synthetic views provide additional geometric and photometric constraints when Gaussian primitives are still coarse and highly deformable.

AugSplat uses two sources of supervision: synthetic views rendered from the radiance-field ensemble and real sparse input images. For synthetic supervision, we sample a minibatch \(\mathcal{B}_{\text{syn}}\) uniformly from the weighted synthetic dataset \(\mathcal{D}_{\text{nerf}}\) and minimize the per-pixel weighted photometric loss \[\mathcal{L}_{\text{nerf}} = \sum_{i \in \mathcal{B}_{\text{syn}}} \sum_{\mathbf{p}} w_i(\mathbf{p}) \left\| I_i^{\text{GS}}(\mathbf{p}) - I_i^{\text{nerf}}(\mathbf{p}) \right\|_2^2 .\] The synthetic images are sampled randomly during training, and the confidence weights reduce the influence of pixels where the radiance-field ensemble is uncertain.

For real-image supervision, each optimization step samples a single real training image \(I_i\) uniformly (\(i \sim \mathcal{U}\)) from the sparse training set. We use the photometric loss \[\mathcal{L}_{\text{real}} = (1 - \lambda)\,\| I_i^{\text{GS}} - I_i\|_2^2 + \lambda \,\mathcal{L}_{\text{SSIM}}(I_i^{\text{GS}}, I_i),\] where \(\mathcal{L}_{\text{SSIM}}\) denotes the SSIM-based dissimilarity term. We use the default value of \(\lambda=0.2\).

We consider two strategies for combining these supervision sources.

Staged AugSplat. Staged AugSplat first optimizes Gaussian Splatting using only the synthetic loss \(\mathcal{L}_{\text{nerf}}\) for a fixed warm-up period. After this phase, the synthetic supervision is removed and training continues using only the real-image loss \(\mathcal{L}_{\text{real}}\). This schedule uses radiance-field views to stabilize the early geometry, while relying on real images for the final appearance refinement.

Dual AugSplat. Dual AugSplat optimizes Gaussian Splatting using real and synthetic supervision in parallel: \[\mathcal{L}(t) = \lambda_{\text{nerf}}(t)\mathcal{L}_{\text{nerf}} + \lambda_{\text{real}}(t)\mathcal{L}_{\text{real}}.\] We use an exponentially decaying raw weight for the synthetic NeRF supervision: \[\tilde{\lambda}_{\text{nerf}}(t) = w_{\text{nerf},0} \exp\left(-\frac{\ln 4}{D}t\right) = w_{\text{nerf},0}4^{-t/D},\] where \(D\) is the quarter-decay period, i.e., the raw NeRF weight is reduced by a factor of four every \(D\) steps. The real-image loss uses a constant raw weight \(\tilde{\lambda}_{\text{real}} = w_{\text{real}}\). The final loss weights are normalized as \[\lambda_{\text{nerf}}(t) = \frac{ \tilde{\lambda}_{\text{nerf}}(t) }{ \tilde{\lambda}_{\text{nerf}}(t) + \tilde{\lambda}_{\text{real}} }, \quad \lambda_{\text{real}}(t) = \frac{ \tilde{\lambda}_{\text{real}} }{ \tilde{\lambda}_{\text{nerf}}(t) + \tilde{\lambda}_{\text{real}} }.\] This schedule gives synthetic views greater influence early in training, when Gaussian primitives are still coarse, and gradually shifts the optimization toward real-image supervision for appearance refinement.

3.3 Implementation Details↩︎

We implement our pipeline using Nerfstudio [38] and gsplat [39]. For the radiance-field prior, we use depth-nerfacto, which incorporates depth supervision to improve geometric consistency.

We train an ensemble of NeRF models using depth-nerfacto, leveraging the estimated poses and depth supervision. The ensemble is used to render synthetic views and compute per-pixel uncertainty estimates.

All Gaussian Splatting variants use the same initialization and base gsplat hyperparameters.

For Staged AugSplat, the initial synthetic warm-up phase uses batches of 20 NeRF-rendered images per step, rendered at \(8\times\) lower resolution. After \(T_{\text{stage}}=300\) synthetic warm-up steps, training switches to standard real-image supervision with one real image per step.

For Dual AugSplat, each optimization step uses both supervision sources in parallel: one real image at the standard training resolution and a batch of 20 NeRF-rendered images at \(8\times\) lower resolution. The synthetic loss is weighted by the exponentially decaying coefficient described above, while the real-image loss uses a constant raw weight.

Unless otherwise specified, we use 30 sparse input images per scene, with 26 images used for training and 4 for validation. We train an ensemble of \(M=5\) depth-nerfacto models and render \(N_{\text{syn}}=200\) synthetic views per scene. For confidence normalization, we use the 5th and 95th percentiles of each synthetic view’s variance map as robust lower and upper bounds, and set the confidence exponent to \(\gamma=1.5\). For Dual AugSplat, we use raw loss weights \(w_{\text{nerf},0}=2\) and \(w_{\text{real}}=1\), with the NeRF weight decaying by a factor of four every \(D=300\) steps. The AugSplat hyperparameters were selected pragmatically based on small-scale pilot experiments, dataset size, and training cost. We primarily tuned the relative loss weights and the synthetic warm-up duration, while using fixed values for the number of synthetic views and confidence exponent. We use a short synthetic warm-up period because NeRF-rendered views are most useful early in optimization, when Gaussian primitives are still coarse, while longer synthetic-only training can overfit to artifacts in the radiance-field prior. Similarly, in Dual AugSplat, the NeRF loss is initialized with a larger weight than the real-image loss but is decayed rapidly, encouraging synthetic views to stabilize early geometry without dominating later appearance refinement. The confidence-weighting parameters are chosen to robustly normalize per-image ensemble variance and suppress uncertain synthetic pixels while preserving stable low-frequency structure.

4 Experiments↩︎

We evaluate our method on the mip-NeRF 360 dataset [14], which consists of nine unbounded indoor and outdoor scenes. To simulate sparse-view settings, we subsample 30 images per scene. Of these, 26 images are used for training and 4 images are held out for evaluation.

To ensure that all methods operate under the same sparse-view reconstruction setting, camera poses, depth estimates, and the sparse point cloud are computed from the same selected 30 views. We obtain COLMAP-compatible reconstructions using VGGT [40]. The resulting poses and depth maps are used to train the depth-supervised radiance-field prior, and the same reconstruction is used consistently across all Gaussian Splatting variants.

We compare against standard 3D Gaussian Splatting implemented in the gsplat library.

We report PSNR, SSIM, and LPIPS on the held-out evaluation views. Since sparse-view Gaussian Splatting can overfit after reaching its best reconstruction quality, we evaluate models throughout training and report the checkpoint that achieves the lowest average error on the held-out evaluation views.

Following [41], we use the combined average error metric, defined as the geometric mean of MSE, \(\sqrt{1-\text{SSIM}}\), and LPIPS: \[\text{avg. error} = \left( 10^{-\text{PSNR}/10} \cdot \sqrt{1-\text{SSIM}} \cdot \text{LPIPS} \right)^{1/3}.\]

To further analyze training dynamics, we additionally report all methods at the iteration where standard GSplat achieves its lowest average error. This allows us to compare reconstruction quality at an equivalent optimization stage.

The effectiveness of AugSplat depends on scene characteristics. AugSplat yields its largest improvements on challenging scenes such as Bicycle and Flowers, where sparse-view initialization is more likely to miss or misplace geometry. On moderately complex scenes such as Kitchen and Stump, the dual formulation provides consistent gains in average error. On simpler scenes such as Room and Counter, performance remains comparable to the baseline, suggesting that standard Gaussian Splatting is already well constrained in these cases.

Average results are shown in Table ¿tbl:tab:avg95res?, and per-scene results are reported in Table 1.

To analyze optimization behavior, we compare all methods at the iteration where standard GSplat achieves its lowest validation average error. As shown in Table [tbl:tab:intermediate95steps], both AugSplat variants achieve lower average error than the baseline at this equivalent training stage, with Dual AugSplat obtaining the best overall result.

This suggests that the improvements are not simply due to longer training or checkpoint selection, but are already present during early optimization.

Figure 4 shows representative training curves for sparse-view scenes. We observe a recurring pattern: PSNR and SSIM often reach their highest values early in training and then gradually decrease, whereas LPIPS continues to improve over later iterations. As a result, the combined average error is minimized at an intermediate point that balances pixel-level fidelity, structural similarity, and perceptual similarity.

This behavior is consistent with sparse-view overfitting. After the initial optimization stage, the model can continue adapting to the limited training views, improving perceptual similarity according to LPIPS, while losing test-view fidelity as measured by PSNR and SSIM. AugSplat follows the same general trend, but it typically reaches better PSNR and SSIM during the early optimization phase. This explains the difference between Table [tbl:tab:intermediate95steps] and Table ¿tbl:tab:avg95res?: standard 3DGS may obtain its best average error at a later point where LPIPS is lower, but PSNR and SSIM have already degraded, whereas AugSplat often reaches its best average error earlier, when image fidelity is still higher. AugSplat generally maintains better PSNR and SSIM during the early optimization phase and achieves lower average error at the selected checkpoints.

Overall, our results demonstrate that NeRF-based augmentation improves Gaussian Splatting in sparse-view settings. The dual formulation provides a stable alternative that keeps real-image supervision active throughout training, while Staged AugSplat achieves the best average reconstruction quality.

While AugSplat improves Gaussian Splatting in many sparse-view scenarios, its effectiveness depends on the quality of the NeRF prior used for augmentation. In highly complex scenes, where the radiance field is less reliable, synthetic views might introduce inaccuracies that negatively affect optimization.

Furthermore, the staged formulation exhibits higher variance across scenes, occasionally leading to degraded performance when synthetic supervision dominates early training. Although the dual formulation mitigates this issue, it still relies on a careful balance between synthetic and real supervision.

Finally, AugSplat requires training a radiance-field prior in addition to Gaussian Splatting, increasing the overall training complexity.

5 Conclusion and Future Work↩︎

In this work, we introduced AugSplat, a framework that leverages neural radiance fields to improve Gaussian Splatting in sparse-view settings. By augmenting the training data with NeRF-generated views, we increase the effective supervisory coverage available during training and provide a stronger signal for early-stage optimization.

We proposed two training strategies, Staged AugSplat and Dual AugSplat. In our experiments, Staged AugSplat achieves the best average reconstruction quality, while Dual AugSplat provides a closely performing alternative that combines real and synthetic supervision throughout training. These results suggest that radiance-field-generated views are most useful during the early stages of Gaussian optimization, where they can provide additional geometric and photometric constraints.

Our experiments on the mip-NeRF 360 dataset show that AugSplat improves reconstruction quality in challenging sparse-view scenarios, while remaining competitive in better-constrained scenes. These results indicate that radiance fields can serve as useful auxiliary supervision for explicit Gaussian representations, helping mitigate some limitations of sparse-view Gaussian Splatting.

References↩︎