Reliability-Aware Monocular Depth Supervision
for Sparse-View Neural Reconstruction
June 27, 2026
Abstract
Sparse-view neural reconstruction is challenging in outdoor driving scenes, where cameras usually move along a narrow forward-facing trajectory and provide limited multi-view overlap. Although monocular depth estimators can provide dense geometric priors, their predictions are noisy, and not uniformly reliable across image regions. In this work, we study monocular depth supervision for sparse-view neural reconstruction. We use Depth Anything V2 as a dense monocular depth prior, align its predictions to metric depth using scale-shift fitting, and apply depth supervision selectively through photometric masks generated from an RGB-only baseline model. We evaluate this strategy on two representative scene representations: Mip-NeRF-360 and Splatfacto. On KITTISeq02 under an every2 sparse-view setting, masked monocular depth supervision gives only marginal rendering gains for Mip-NeRF-360 and does not improve metric geometry. In contrast, Splatfacto benefits more clearly, improving PSNR from 14.903 to 15.932 and reducing RMSE from 0.542 to 0.100. Additional KITTISeq05 experiments and matched-ratio mask ablations further show that the gains for Splatfacto come from selecting reliable low-error regions rather than simply reducing the number of depth-supervised pixels. Additional experiments on the Bicycle scene show that depth supervision can improve geometry while hurting RGB rendering quality when multi-view coverage is already strong. Overall, our results suggest that monocular depth priors are useful for under-constrained sparse-view reconstruction, but should be applied selectively and with moderate weighting.
1 Introduction↩︎
Novel view synthesis aims to reconstruct a scene representation from a set of posed images and render photorealistic images from unseen viewpoints. This problem is central to applications such as autonomous driving simulation, robotics, augmented reality, and digital twins. Recent neural scene representations, including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have achieved impressive rendering quality by optimizing differentiable scene representations directly from images [1], [2]. However, these methods still rely heavily on sufficient multi-view coverage. When the input views are sparse, the reconstruction problem becomes under-constrained, often leading to incorrect geometry and floaters [3]–[5].
This limitation is especially severe in outdoor driving scenes [5], [6]. Unlike object-centric datasets, where cameras move around the target object, autonomous-driving sequences are usually captured by a forward-moving camera along a narrow trajectory. As a result, nearby objects may have limited viewpoint coverage, distant regions provide weak parallax, and large sky regions, shadows, reflective surfaces, and dynamic objects further complicate reconstruction [5]. These challenges make sparse-view outdoor reconstruction an important but difficult setting for both NeRF-like and 3DGS-like methods [2], [5], [7], [8].
Depth supervision is a natural way to reduce this ambiguity. However, accurate ground-truth depth is expensive and often sparse, whereas monocular depth estimators such as Depth Anything V2 (DA-V2) provide dense but imperfect relative depth estimates [9]. This leads to a critical question: when can we trust monocular depth supervision in sparse-view neural reconstruction?
We study reliability-aware monocular depth supervision for outdoor sparse-view reconstruction in this paper. We fit scale and shift to metric depth for DA-V2 predictions and selectively apply depth supervision using photometric reconstruction error from an RGB-only baseline. We evaluate this strategy on two representative scene representations, Mip-NeRF-360 and Splatfacto [2], [6], [7], [9], [10].
Our contributions are summarized as follows:
We propose a reliability-aware monocular depth supervision strategy where aligned monocular depth priors are applied selectively based on photometric reconstruction error.
We assess the effect of this approach on NeRF-style implicit scene representations and Gaussian Splatting-style explicit scene representations.
Through matched-ratio mask ablations and an additional KITTISeq05 fragment, we show that low-photometric-error regions provide more robust supervision for Splatfacto, while Mip-NeRF-360 remains sensitive to noisy monocular depth.
We further show that depth supervision can improve the geometry but degrade the RGB rendering quality in object-centric scenes with stronger multi-view coverage.
2 Related Work↩︎
2.0.0.1 Neural scene representations.
NeRF represents a scene as a continuous radiance field and renders images through differentiable volume rendering [1]. Mip-NeRF-360 extends this idea to unbounded real-world scenes and is therefore a strong baseline for outdoor reconstruction [7]. More recently, 3D Gaussian Splatting represents scenes using explicit anisotropic Gaussians and achieves real-time high-quality rendering [2]. Recent feed-forward Gaussian Splatting methods further use learned multi-view or depth cues for sparse-view reconstruction [11]–[13]. In this work, we compare these two types of representations because depth supervision may affect an implicit density field and an explicit Gaussian representation differently.
2.0.0.2 Depth supervision for sparse-view reconstruction.
Sparse-view reconstruction remains challenging because photometric supervision alone does not fully constrain scene geometry. Prior works such as DS-NeRF and Dense Depth Priors for NeRF show that additional geometric constraints can improve reconstruction under limited views [3], [4]. Recent work on outdoor neural radiance fields explores the effect of different depth priors on the reconstruction of large-scale outdoor scenes [5].
2.0.0.3 Reliability-aware monocular depth priors.
While depth priors can reduce the geometric ambiguity, monocular depth estimates from recent foundation models [9], [14]–[18] are not equally reliable over the entire image because of scale ambiguity, dynamic objects, reflective surfaces, sky regions and occlusion boundaries [8], [9]. Thus, uniform monocular depth supervision may result in wrong geometry in unreliable regions [8]. We build on the line of work in depth-supervised sparse-view reconstruction but we focus on when to trust monocular depth supervision [3]–[5]. In contrast to prior work that has mostly focused on improving Gaussian Splatting with specific depth losses or inconsistency masks [8], [19]–[22], we explore a simple photometric-error-based reliability mask and investigate the effect of the same monocular depth prior on both implicit NeRF-style and explicit Gaussian representations. We specifically study the effect of reliability-aware photometric masking and depth-loss weighting for Mip-NeRF-360 and Splatfacto in sparse outdoor reconstruction.
3 Data↩︎
We evaluate our approach on two benchmark datasets, selected sequences from the KITTI [6] odometry benchmark and the outdoor object-centric Bicycle scene from Mip-NeRF-360 [7] dataset.
KITTI. KITTI comprises large-scale real-world outdoor driving scenes, making it well suited for evaluating novel view synthesis in autonomous driving contexts. Unlike object-centric scenes typical of standard NeRF and 3DGS benchmarks, vehicles in autonomous driving scenarios exhibit predominantly forward or turning motion along a fixed trajectory. To mitigate the influence of lighting variation and dynamic objects, we follow the broader evaluation protocol of prior outdoor NeRF work [5], which draws from selected KITTI odometry sequences 00, 02, 05, and 06. For each sequence, every 10th frame is held for the test set and the remaining frames are used for training. To assess robustness under sparse viewpoints, we simulate low-frequency imaging at 2.5 Hz by subsampling 50% of the training frames (retaining every second frame). In this work, we focus on KITTI sequence 02 (fragment 034) as the main evaluation setting and include representative KITTI sequence 05 experiments as an additional generalization check. Rather than relying on Structure-from-Motion (SfM) poses, which may introduce scale inconsistencies with ground-truth depth, we use the calibrated poses provided directly by the KITTI odometry benchmark. The same pose setup is used for both the NeRF and 3DGS baselines. DA-V2 is used as the monocular depth prior throughout.
Mip-NeRF-360. We further evaluate on the outdoor Bicycle scene from the Mip-NeRF-360 dataset, using 194 sparse views. Every 10th frame is held back for validation and testing. To accommodate GPU memory constraints, images are downscaled by 4× for 3DGS training, while depth priors are computed at full resolution and resized at load time.
Experimental Scope. We use KITTI sequence 02 (fragment 034) as the main evaluation setting for both Mip-NeRF-360 and Splatfacto, include representative KITTI sequence 05 experiments as an additional generalization check, and evaluate Splatfacto on the Mip-NeRF-360 Bicycle scene under sparse viewpoints.
4 Methods↩︎
4.1 Overview↩︎
Our objective is to reconstruct a scene representation that yields geometrically consistent depth estimates under sparse-view conditions and accurate novel-view renderings given a set of posed RGB training images. Using photometric supervision alone under-constrains sparse-view reconstruction, particularly in forward-facing outdoor scenes where the camera trajectory provides limited multi-view overlap.
Since monocular depth estimates can be noisy, scale-ambiguous, and unreliable in difficult regions like dynamic objects, reflective surfaces, occlusion boundaries, and sky regions, we avoid applying Depth Anything V2 (DA-V2) as a dense monocular depth prior uniformly across all pixels. Instead, we use a photometric reconstruction error-based reliability-aware depth supervision approach.
There are three steps in our approach. First, we use sparse reference anchors, like LiDAR depth in KITTI or COLMAP points in the Bicycle scene, to align each DA-V2 depth prediction to metric depth. To compute the per-pixel photometric reconstruction error, we train an RGB-only baseline model and render the training views. For depth supervision, pixels with low photometric error are regarded as more reliable regions. Third, we retrain the reconstruction model using a masked depth loss, in which only pixels chosen by the photometric reliability mask are subject to the aligned monocular depth prior.
We use the same framework for Splatfacto and Mip-NeRF-360. The two backbones represent scene geometry and render depth differently, but they have the same depth prior, mask-generation process, and masked depth-supervision objective.
4.2 Scale-Aligned Monocular Depth Prior↩︎
We use DA-V2 to predict a dense monocular depth map for each training image. Since DA-V2 predicts relative depth rather than metric depth, its raw output cannot be directly used as supervision. We therefore align each predicted depth map to a metric reference before training. For KITTI, the reference comes from valid LiDAR depth pixels. For the Mip-NeRF-360 Bicycle scene, we use sparse COLMAP points as anchor depth.
Given a monocular prediction (\(d_m(u)\)) and reference depth (\(d_r(u)\)) at valid anchor pixels (\(u \in \Omega\)), we solve a per-image scale-shift alignment: \[s^*, t^* = \arg\min_{s,t} \sum_{u \in \Omega} \left(s d_m(u) + t - d_r(u)\right)^2 .\]
The aligned depth prior is then \[d(u) = s^* d_m(u) + t^* .\]
For KITTI, we clip the aligned depth to the valid evaluation range of 80 meters. Pixels without valid reference depth are not used to estimate the scale and shift, but the resulting aligned dense map is used as a depth prior during training. This alignment step allows us to use the dense structure predicted by DA-V2 while keeping its scale approximately consistent with the reconstruction coordinate system.
4.3 Photometric Reliability Mask↩︎
We build upon prior work [5], [8] showing that depth supervision improves NeRF and 3DGS reconstruction for outdoor sparse-view conditions. However, monocular depth priors are often noisy and inconsistent, and existing approaches typically apply depth supervision uniformly across the image. We investigate the effects of a masked depth prior, where we first estimate where the RGB reconstruction is reliable. Given a rendered image \(\hat{I}\) from a depth-free baseline model and a ground-truth image \(I\), we compute the photometric error per-pixel: \[e(u) = \frac{1}{3} \sum_{c \in {R,G,B}} \left| \hat{I}_c(u) - I_c(u) \right|,\] where \(u\) denotes a pixel location and \(e(u)\in[0,1]\) for normalized RGB images.
Pixels are then classified using a threshold \(\tau\). We focus primarily on low-error masking (\(e(u) < \tau\)), where depth supervision is applied only to pixels that are accurately reconstructed by the RGB model.
We define the binary photometric reliability mask as
\[M_{\tau}(u) = \begin{cases} 1, & e(u) < \tau, \\ 0, & \text{otherwise}. \end{cases}\]
The motivation for this mask is that large photometric error often indicates regions where supervision is less reliable, such as dynamic objects, reflective surfaces, occlusion boundaries, or sky regions. Applying a monocular depth loss to these pixels may force the model toward an inaccurate geometry. In contrast, low-error pixels are more likely to correspond to regions that are already consistent with the RGB observations, so the aligned monocular depth prior can act as a stabilizing geometric regularizer.
During depth-supervised training, the photometric reliability mask is fixed after being calculated once from the RGB-only baseline.
4.4 Masked Depth Supervision Objective↩︎
Let \(M_{\tau}(u)\) denote the fixed photometric reliability mask and \(D(u)\) denote the depth-validity mask. The effective supervision mask is \[M_{\mathrm{eff}}(u) = M_{\tau}(u) \land D(u).\]
The depth loss is computed only on pixels selected by the effective supervision mask: \[\mathcal{L}_{\mathrm{depth}} = \frac{1}{N} \sum_{u} M_{\mathrm{eff}}(u) \left( \hat{d}(u) - d(u) \right)^2,\] where \(\hat{d}(u)\) is the rendered depth and \(d(u)\) is the aligned depth prior.
The overall training objective is \[\mathcal{L} = \mathcal{L}_{\text{rgb}} + \lambda_{\text{depth}}\, \mathcal{L}_{\text{depth}},\] where \(\lambda_{\mathrm{depth}}\) controls the relative weight of depth supervision. The RGB reconstruction loss is evaluated over the full image, while the depth loss is applied only within the effective supervision mask.
4.5 Training Pipeline↩︎
Our approach consists of three stages. First, we train a baseline reconstruction model without depth supervision (\(\lambda_{\mathrm{depth}} = 0\)) for 50,000 iterations. The resulting model is used to render all training views and estimate per-pixel photometric error. We then set a threshold for the error maps to generate binary photometric masks, which remain fixed for the remainder of training. Finally, the model is retrained with depth supervision enabled while applying the photometric masks to gate the depth loss. The threshold determines the fraction of pixels that contribute to depth supervision, while \(\lambda_{\mathrm{depth}}\) controls the relative weight of the depth loss.
To evaluate whether photometric-masked depth supervision generalizes across scene representations, we apply the same framework to both Mip-NeRF-360 and Splatfacto. In both cases, aligned DA-V2 depth maps are used as supervision and photometric masks are applied only to the depth loss, while RGB reconstruction is evaluated over the full image.
4.6 Backbone-Specific Implementation↩︎
4.6.0.1 Mip-NeRF-360.
For Mip-NeRF-360, depth supervision is incorporated through a mean squared error (MSE) loss between the rendered depth and the aligned depth prior. Pixels excluded by the fixed photometric mask are marked as invalid supervision and do not contribute to the depth loss. The RGB rendering objective remains unchanged.
4.6.0.2 Splatfacto-DA-V2.
We extend Nerfstudio’s Splatfacto [23] implementation to load aligned DA-V2 depth maps through the dataset configuration. During training, depth loss is computed only on pixels that satisfy both depth-validity and photometric-mask constraints. This follows the effective supervision mask and masked depth objective defined above. The RGB rendering objective remains the same.
Both Mip-NeRF-360 and Splatfacto therefore share the same depth prior, photometric-mask generation procedure, and hyperparameter sweeps, differing only in their underlying scene representations.
5 Experiments↩︎
5.1 Experimental Setup↩︎
We evaluate our approach on KITTISeq02 in an every2 sparse-view setting, i.e. one out of two training frames. We additionally evaluate representative settings on KITTISeq05 to test whether the observed trends generalize to another outdoor driving sequence. We compare Mip-NeRF-360 as an implicit NeRF-based backbone [7] and Splatfacto as an explicit Gaussian Splatting backbone [2], [10]. We use DA-V2 as the monocular depth prior [9], and each DA-V2 depth map is scale and shift aligned to the KITTI depth, and clipped at 80 m before supervision. We compare RGB-only training with masked DA-V2 supervision under different mask thresholds \(\tau\) and depth weights \(\lambda\). We vary \(\tau \in \{0.14,0.16,0.18,0.20,0.22,1.0\}\) and \(\lambda \in \{0,0.05,0.10,0.15\}\) for each experiment set. For the matched-ratio mask ablation, we use the best Splatfacto setting \(\tau=0.18,\lambda=0.10\) and compare the low-error mask against a high-error mask and random masks with the same number of supervised pixels. We evaluate the rendering quality with PSNR, SSIM and LPIPS [24], [25], where higher PSNR/SSIM and lower LPIPS indicate better novel-view rendering. When ground-truth depth is available, we evaluate geometry using AbsRel and RMSE on the rendered depth maps; for Splatfacto experiments, we report RMSE as the primary depth metric, with lower values corresponding to more accurate depth reconstruction. We also test Splatfacto on the object-centric Bicycle scene as an additional setting of a circular view.
5.2 Depth Prior Quality↩︎
First, we evaluate the aligned DA-V2 maps on valid KITTI depth pixels before using DA-V2 as supervision [6], [9]. The average absolute error of alignment is 4.22 m. This means that DA-V2 provides useful scene geometry, but is still noisier than the ground-truth LiDAR depth, especially on reflective surfaces, thin structures, and faraway areas.
5.3 Mip-NeRF-360 Results↩︎
Takeaway. Masked monocular depth supervision provides only weak and unstable regularization for Mip-NeRF-360: it slightly improves PSNR in some KITTISeq02 settings, but does not improve SSIM, LPIPS, or metric geometry. On KITTISeq05, both global and masked depth supervision degrade rendering and geometry compared with RGB-only training.
Table 1 summarizes the results of Mip-NeRF-360 on KITTISeq02 with the every2 sparse-view setting [6], [7]. For brevity, we report only a subset of the raw sweep (rather than listing each run). This is mostly for readability, as the full grid has many rows and would be overwhelming in the two-column report format. Furthermore, the depth loss is turned off when \(\lambda=0\), so the mask threshold does not affect training. Thus, for Mip-NeRF-360 we use the RGB-only run as a shared baseline across thresholds.
The RGB-only Mip-NeRF-360 baseline achieves 20.378 PSNR, 0.601 SSIM, 0.409 LPIPS, 0.0857 AbsRel and 2.703 RMSE. Adding masked DA-V2 supervision improves PSNR in several settings, but only marginally. The best PSNR is achieved with \(\tau=1.0\) and \(\lambda=0.15\), 20.607 PSNR, which is improved by 0.229 dB over RGB-only training. Other good choices are \(\tau=0.16,\lambda=0.15\) and \(\tau=0.14,\lambda=0.15\) with PSNR values of 20.581 and 20.488 respectively.
However, the improvement is uneven on all measures. Masked monocular depth supervision can improve PSNR a little, but SSIM and LPIPS are generally worse than the RGB-only baseline. The geometry metrics are also worse: the best-PSNR setting is AbsRel 0.107 and RMSE 3.580, worse than the baseline RGB-only. This suggests that for Mip-NeRF-360 the monocular depth prior is a weak regularizer for RGB reconstruction, but does not provide reliable improvements to the metric geometry. One possible reason is that the scale-shift aligned monocular depth remains noisy and its supervision may interfere with the geometry implicitly learned by volume rendering.
Figure 5: PSNR as a function of depth loss weight \(\lambda\) on KITTI Seq02, evaluated across six threshold values \(\tau \in \{0.14, 0.16, 0.18, 0.20, 0.22, 1.0\}\).. a — PSNR increases monotonically with \(\lambda\) across nearly all \(\tau\) values, showing consistent benefit from stronger depth regularization., b — PSNR peaks sharply at \(\tau{=}0.18\), \(\lambda{=}0.1\) (15.93 dB), showing sensitivity to depth supervision and threshold values.
| \(\tau\) | \(\lambda\) | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | AbsRel\(\downarrow\) / RMSE\(\downarrow\) |
|---|---|---|---|---|---|
| RGB-only | 0 | 20.378 | 0.601 | 0.409 | 0.0857 / 2.703 |
| 0.14 | 0.15 | 20.488 | 0.596 | 0.413 | 0.1113 / 3.663 |
| 0.16 | 0.15 | 20.581 | 0.596 | 0.415 | 0.1098 / 3.654 |
| 0.18 | 0.10 | 20.384 | 0.594 | 0.416 | 0.1044 / 3.532 |
| 0.20 | 0.15 | 20.381 | 0.594 | 0.417 | 0.1097 / 3.658 |
| 0.22 | 0.15 | 20.454 | 0.594 | 0.416 | 0.1088 / 3.603 |
| 1.00 | 0.15 | 20.607 | 0.595 | 0.412 | 0.1074 / 3.580 |
We further evaluate representative Mip-NeRF-360 settings on KITTISeq05. As shown in Table 2, both global depth supervision and low-error masked supervision reduce rendering quality and increase depth error compared with RGB-only training. The low-error mask performs better than global depth supervision, but it still does not recover the RGB-only performance. This supports the observation that monocular depth priors are not reliably beneficial for the implicit density representation.
| \(\tau\) | \(\lambda\) | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | AbsRel\(\downarrow\) / RMSE\(\downarrow\) |
|---|---|---|---|---|---|
| RGB-only | 0 | 17.068 | 0.546 | 0.529 | 0.1166 / 2.978 |
| 1.00 | 0.15 | 16.389 | 0.527 | 0.569 | 0.1527 / 4.803 |
| 0.18 | 0.15 | 16.612 | 0.530 | 0.563 | 0.1399 / 4.454 |
5.4 Splatfacto Results↩︎
Takeaway. Splatfacto benefits more clearly from masked monocular depth supervision in sparse forward-facing KITTI scenes. Matched-ratio ablations show that the low-error mask outperforms high-error and random masks, indicating that the gain comes from selecting more reliable pixels rather than simply reducing the number of depth-supervised pixels.
Table 3 summarizes the Splatfacto results in the same KITTISeq02 every2 sparse-view setting [2], [6], [10]. Similar to the Mip-NeRF-360 table, we provide a short summary instead of the full raw hyperparameters grid. The average RGB-only baseline and representative setting for each threshold are listed below. This makes the table easier to read and still retains the main trend needed for analysis.
In contrast to Mip-NeRF-360, Splatfacto gains more obvious advantages from masked monocular depth supervision. RGB-only Splatfacto baseline (mean): 14.903 PSNR, 0.433 SSIM, 0.446 LPIPS, 0.542 RMSE. The best rendering result is achieved when \(\tau=0.18\) and \(\lambda=0.10\), achieving 15.932 PSNR, 0.477 SSIM and 0.408 LPIPS. This increases PSNR by approximately 1.03 dB over the average RGB-only baseline and significantly decreases RMSE from 0.542 to 0.100.
The results also show a tradeoff between rendering quality and geometry quality. At \(\tau=0.18\), increasing the depth weight from \(\lambda=0.10\) to \(\lambda=0.15\) further improves the RMSE from 0.100 to 0.096 but decreases the PSNR from 15.932 to 15.588 and increases the LPIPS from 0.408 to 0.436. This shows that stronger depth supervision can increase the geometric consistency, but can also constrict the optimization and impair the photometric fidelity. In general, \(\tau=0.18,\lambda=0.10\) is the best balance for the rendering quality and depth regularization.
Splatfacto is more sensitive to the masked depth prior than Mip-NeRF-360. This is probably because Splatfacto has an explicit Gaussian representation and the depth supervision directly affects the placement of Gaussians and helps to reduce the poorly placed Gaussians or floaters in the sparse-view regions. In contrast, Mip-NeRF-360 represents geometry implicitly as a density field, which makes it more sensitive to noisy or mis-aligned monocular depth priors.
| \(\tau\) | \(\lambda\) | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | RMSE\(\downarrow\) |
|---|---|---|---|---|---|
| RGB-only | 0 | 14.903 | 0.433 | 0.446 | 0.542 |
| 0.14 | 0.10 | 15.463 | 0.454 | 0.436 | 0.105 |
| 0.16 | 0.10 | 15.452 | 0.454 | 0.434 | 0.106 |
| 0.18 | 0.10 | 15.932 | 0.477 | 0.408 | 0.100 |
| 0.18 | 0.15 | 15.588 | 0.459 | 0.436 | 0.096 |
| 0.20 | 0.05 | 15.171 | 0.431 | 0.463 | 0.129 |
| 0.22 | 0.15 | 15.143 | 0.443 | 0.452 | 0.103 |
| 1.00 | 0.10 | 15.494 | 0.448 | 0.434 | 0.101 |
To verify whether the photometric reliability mask is meaningful, we conduct a matched-ratio mask ablation at \(\tau=0.18,\lambda=0.10\), the best Splatfacto setting on KITTISeq02. The high-error and random masks select the same number of valid pixels as the low-error mask. As shown in Table 4, the low-error mask achieves the best PSNR, SSIM, LPIPS, and RMSE. This shows that the improvement is not merely due to using fewer depth-supervised pixels, but comes from selecting photometrically reliable regions.
| Mask type | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | RMSE\(\downarrow\) |
|---|---|---|---|---|
| High-error, matched ratio | 14.932 | 0.437 | 0.455 | 0.111 |
| Random, matched ratio | 15.036\(\pm\)0.341 | 0.442\(\pm\)0.013 | 0.456\(\pm\)0.017 | 0.109\(\pm\)0.002 |
| Low-error, \(\tau=0.18\) (ours) | 15.932 | 0.477 | 0.408 | 0.100 |
We also evaluate Splatfacto on KITTISeq05 using representative settings. Table 5 shows that the low-error mask again improves rendering quality and depth RMSE compared with both RGB-only training and global depth supervision. This indicates that the benefit of reliability-aware monocular depth supervision is not limited to a single KITTI fragment.
| Setting | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | RMSE\(\downarrow\) |
|---|---|---|---|---|
| RGB-only | 14.894 | 0.521 | 0.493 | 0.807 |
| Global depth, \(\lambda=0.10\) | 15.255 | 0.534 | 0.469 | 0.0962 |
| Low-error, \(\tau=0.18,\lambda=0.10\) | 15.903 | 0.548 | 0.445 | 0.0959 |
5.5 Object-Centric Bicycle Results↩︎
Takeaway. In the object-centric Bicycle scene [7], [10], depth supervision improves geometry as measured by RMSE, but it can hurt RGB rendering quality when multi-view coverage is already strong.
In contrast to KITTISeq02, which shows the camera moving forward for the most part in a large outdoor scene, the Bicycle scene captures a single object from viewpoints surrounding it. We investigate whether the advantage of masked monocular depth supervision persists when the RGB views already have stronger multi-view coverage.
Table 6 reports a compact summary of the hyperparameter sweep. We only show representative settings in the main text for readability. Since \(\lambda=0\) disables depth supervision, we summarize the RGB-only runs with an average baseline.
The RGB-only baseline achieves the best rendering quality with 17.731 PSNR, 0.570 SSIM and 0.240 LPIPS. Adding monocular depth supervision improves RMSE significantly from 1.479 to as low as 0.722, but generally decreases PSNR, SSIM and LPIPS. This indicates that in the Bicycle scene the monocular depth regularization mainly improves the geometric consistency but may over-regularize the RGB reconstruction. The gain in rendering quality is weaker compared to KITTISeq02 since the circular trajectory already imposes stronger multi-view constraints.
| \(\tau\) | \(\lambda\) | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | RMSE\(\downarrow\) |
|---|---|---|---|---|---|
| RGB-only | 0 | 17.731 | 0.570 | 0.240 | 1.479 |
| 0.14 | 0.05 | 17.415 | 0.524 | 0.277 | 0.830 |
| 0.14 | 0.15 | 17.036 | 0.477 | 0.309 | 0.722 |
| 0.16 | 0.05 | 17.415 | 0.521 | 0.278 | 0.775 |
| 0.18 | 0.05 | 17.466 | 0.520 | 0.279 | 0.751 |
| 0.20 | 0.05 | 17.345 | 0.519 | 0.278 | 0.778 |
| 0.22 | 0.10 | 17.106 | 0.491 | 0.297 | 0.725 |
| 1.00 | 0.05 | 17.593 | 0.520 | 0.275 | 0.763 |
5.6 Discussion and Failure Modes↩︎
Our results show that the effect of masked monocular depth supervision is dependent on the reconstruction backbone and scene configuration. On KITTISeq02, Splatfacto benefits more clearly than Mip-NeRF-360. The additional KITTISeq05 experiment shows the same overall trend: Splatfacto benefits from low-error masked depth supervision, whereas Mip-NeRF-360 degrades under both global and masked depth supervision. The matched-ratio ablation further confirms that low-error photometric regions are more effective than high-error or random regions for applying monocular depth supervision. This is likely due to the explicit Gaussian representation used in Splatfacto, where depth supervision can directly help with Gaussian placement and alleviate sparse-view geometry artifacts. In contrast, Mip-NeRF-360 learns geometry implicitly through volume density, thus being more sensitive to noisy or imperfectly aligned monocular depth.
The additional Bicycle experiment demonstrates that depth supervision is not always beneficial for rendering quality. In this object-centric circular-view setup, RGB-only Splatfacto gets the best PSNR, SSIM and LPIPS, while depth supervision mainly improves the RMSE metric. This suggests monocular depth priors can over-regularize reconstruction and degrade photometric fidelity when multi-view RGB coverage is already strong.
In summary, monocular depth priors are useful for sparse-view reconstruction, especially in under-constrained forward-facing scenes such as KITTISeq02, but must be used with care. A moderate depth weight can improve rendering and geometry, but too strong supervision may sacrifice RGB quality for better depth metrics. Future work could explore better scale alignment, uncertainty-aware masks, and adaptive depth-loss weighting.
6 Conclusion↩︎
In this work, we investigate masked monocular depth supervision for sparse-view neural reconstruction using DA-V2 priors with Mip-NeRF-360 and Splatfacto on KITTISeq02, representative KITTISeq05 experiments, and the object-centric Bicycle scene [6], [7], [10].
Our results show that the benefit of depth supervision depends strongly on the backbone and scene configuration. For Mip-NeRF-360, masked depth supervision gives weak and unstable gains: it slightly improves PSNR on KITTISeq02 but does not improve geometry, and degrades both rendering and geometry on KITTISeq05. For Splatfacto, it improves KITTISeq02 PSNR from 14.903 to 15.932 and RMSE from 0.542 to 0.100. Matched-ratio mask ablations and KITTISeq05 results further indicate that these gains come from selecting reliable low-error regions rather than simply reducing the number of supervised pixels. On Bicycle, depth supervision improves RMSE but reduces RGB rendering quality.
Overall, monocular depth priors are most useful with explicit Gaussian representations, but less reliable for implicit NeRF-style representations. Future work could improve scale alignment, uncertainty-aware masks, and adaptive weighting.