1 Introduction↩︎

Intrinsic decomposition is a fundamental task for image editing as it allows modification of object colors (called albedo) without altering shading and shadows [1]. Numerous algorithms have been proposed to decompose images [2], image collections [3], videos [4], lightfields [5], and neural radiance fields [6]. In this paper, we describe how to model, reconstruct and edit intrinsic decompositions of radiance fields represented as 3D Gaussian splats [7], allowing real-time navigation in the modified 3D scene.

1.0.0.1 Modeling intrinsic decompositions of Gaussian splats.

3D Gaussian splatting (3DGS) represents the radiance field of a scene as a collection of 3D Gaussian primitives that store localized view-dependent colors, typically optimized to reproduce the colors observed in multiple photographs of the scene. Prior work proposed to decompose 3D Gaussian splats by augmenting each primitive with intrinsic quantities, i.e, to separate the view-dependent color of each Gaussian into its albedo, shading and view-dependent residual values (or other material-lighting separations) [8]–[10]. But while intrinsic quantities often exhibit some spatial correlation (for example, object boundaries typically produce discontinuities in both albedo and shading), they can also require significantly different spatial resolution (for example, object textures produce high-frequency details in albedo but not in shading, while hard shadows produce sharp edges in shading that should not appear in albedo). We thus propose to model and reconstruct intrinsic decompositions with separate sets of Gaussians for the albedo, shading and residual components, allowing our approach to adapt the number and distribution of Gaussian primitives to the local characteristics of each signal.

1.0.0.2 Reconstructing intrinsic decompositions of Gaussian splats.

Decomposing observed radiance into intrinsic quantities is an ill-posed problem, for which numerous heuristic [1] and data-driven [11] priors have been proposed. We follow the latter approach and leverage a recent diffusion-based video decomposition model [12] to predict the albedo component of all images of a scene. Compared to single-image models, this video model produces consistent predictions across successive images, which is critical to fuse these predictions into a single 3D Gaussian splat. However, being free of shading, albedo images exhibit large uniform areas that hinder 3D reconstruction. We address this challenge by regularizing the 3D reconstruction of albedo Gaussians with per-image depth predictions computed on the original input images. Equipped with a consistent albedo field, we next optimize a view-independent shading field and a view-dependent residual field – each represented by its own set of Gaussians – such that rendering and compositing all three fields reproduces the input images.

1.0.0.3 Editing intrinsic decompositions of Gaussian splats.

Similar to prior work on intrinsic images, we demonstrate the value of our decomposition by modifying object colors and texture while preserving realistic shading and shadows (Figure [fig:teaser]). With our solution, users can orient a virtual camera towards the surface they wish to edit, and paint over the resulting image to modify its albedo. We then update the 3D Gaussians that represent the albedo field to capture these user edits while keeping the shading Gaussians untouched. The main difficulty resides in ensuring that the edits performed in one view stay consistent in other views, which is not guaranteed when a single view is used for Gaussian optimization. To address this difficulty, we reconstruct a proxy surface around the user edits and use it to reproject the edits in nearby views to further constrain the albedo update.

In summary, our contributions are:

• A representation of radiance fields as three separate Gaussian splats representing albedo, shading and view-dependent residuals.

• A method to reconstruct each of these intrinsic fields from multiple photographs, based on a video diffusion prior.

• A method that leverages this intrinsic decomposition to edit the albedo of a scene while keeping the shading untouched, allowing free-viewpoint navigation in the edited scene.

We demonstrate our approach by decomposing and editing radiance field reconstructions of three real-world scenes, and of one synthetic scene for which we have ground truth albedo.

2 Related Work↩︎

There is a vast literature on intrinsic image decomposition, typically separating the image into diffuse albedo and lighting layers. In later work, a residual layer was also introduced to better model view-dependent effects. We refer the reader to the excellent surveys covering this body of work [1], [11]. Early approaches depended on heuristic priors, for example assuming that the albedo is piecewise-constant while lighting is smooth, with mixed results. Recent approaches employ machine learning to obtain more powerful, data-driven priors [13], [14]. In particular, diffusion models offer rich priors learned from very large datasets, and recent work has demonstrated that fine-tuning a large diffusion model can achieve impressive intrinsic decompositions [15]–[17]. Since we are interested in the multi-view context, we employ a video-based diffusion model [12] to extract coherent intrinsic layers for multiple, densely-captured photographs of a scene.

Intrinsic decomposition is often a key component for relighting captured scenes. Early work combined forward rendering with heuristic priors [18], while subsequent methods adopted deep learning for better results [19]. The introduction of Neural Radiance Fields (NeRFs) quickly led to exploration of how to jointly perform 3D reconstruction, intrinsic decomposition and relighting [6], [20]–[23], but the computational and memory expense of the original method plus the additional terms is prohibitive for all practical purposes. 3D Gaussian Splatting (3DGS) [7] reduces the computational cost, and a large number of methods consider intrinsic decompositions of Gaussian splats. However, most of these (e.g., [8]–[10], [24], [25]) perform some form of inverse rendering under environment map lighting – often applied to an isolated object – and thus have difficulty handling the more involved context of full scenes with local lighting. We focus on such scenes and avoid the need for full inverse rendering by relying instead on a pre-trained diffusion model for intrinsic decomposition. While our decomposition does not allow relighting, it does support convincing albedo edits.

Several methods have been develop for editing NeRFs, and in particular modifying the color, e.g., for posterization [26] or palette-based color editing [27]. The speed of 3DGS has allowed the development of interactive editing tools for radiance fields, such as SplatShop [28], that allow editing of the geometry and “painting” on top of existing splats, with an emphasis on Virtual Reality interaction. ReCoGS [29] allows real-time re-coloring of splats using unprojection and fine-tuning to obtain the final result. With our intrinsic decomposition of 3DGS we can go much further than simple recoloring, since by separating the lighting from the albedo/texture we can perfom color and texture edits while preserving the lighting conditions in the scene.

3 Method↩︎

Our pipeline is illustrated in Fig. 2. We capture a scene with a video, which we feed to a video diffusion model to estimate albedo for each frame [12]. We also feed the video to a monocular depth estimator to obtain an initial guess of the scene geometry [30]. We then subsample the video to create a typical multi-view dataset used to create a radiance field (a few hundred images in most of our examples). Equipped with this multi-view dataset and accompanying albedo estimates, the second step of our method is to create an albedo field by optimizing a 3DGS representation from the albedo images. We regularize the optimization using the estimated depth. In the third step, we optimize a shading field and a residual field that we composite with the albedo field to reproduce the original input images. With this representation, the user can directly edit the albedo layer, e.g., by painting or by adding a texture. We then re-optimize the albedo Gaussians to capture the modified texture, allowing us to render the modified scene from any viewpoint while maintaining the original lighting effects. We next describe each step in detail.

3.1 Modeling intrinsic decompositions of Gaussian splats↩︎

We adopt the intrinsic image formation model that expresses each observed image \(I\) as the product of a diffuse albedo image \(A\) and a diffuse shading image \(S\), summed with a view-dependent residual \(R\) [6], [14] (Fig. 1): \[\label{eq:basic95im} I = A \times S + R.\tag{1}\] Prior work on intrinsic decomposition of Gaussian splats [8]–[10] (resp. neural radiance fields [6]) optimize a single set of Gaussians (resp. a single MLP) to express all intrinsic quantities. Instead, we propose to optimize three separate sets of Gaussians \(\mathcal{G}_{A}\), \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\) which, once splatted and composited according to Eq. 1 , reproduce all input images. Using three separate sets allows our approach to better capture the (possibly misaligned) sharp details of each intrinsic component.

Similarly to standard 3DGS [7], we model each Gaussian primitive within \(\mathcal{G}_{A}\), \(\mathcal{G}_{S}\) or \(\mathcal{G}_{R}\) with a position \(p\), color \(c\), opacity \(\alpha\) and a covariance matrix \(\Sigma\): \[g = \{c, \alpha, p,\Sigma\}.\] However, while the original method uses spherical harmonics to encode view-dependent radiance in \(c\), we only use spherical harmonics for the Gaussians \(\mathcal{G}_{R}\) that model the view-dependent residual, and resort to a simple triplet \(c = \{r,g,b\} \in \mathbb{R}^3\) for the Gaussians \(\mathcal{G}_{A}\) and Gaussians \(\mathcal{G}_{S}\) that model diffuse albedo and shading, respectively. For a given viewpoint, we render each set of Gaussians into its corresponding image by using the same rasterizer as in the original 3DGS implementation. The three sets of Gaussians \(\mathcal{G}_{A}\), \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\) can be seen as separate 3D fields, which we call albedo, shading and residual fields.

3.2 Reconstructing intrinsic decompositions of Gaussian splats↩︎

We now describe how to reconstruct the three sets of intrinsic Gaussians \(\mathcal{G}_{A}\), \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\) from multiple photographs of a scene. Given the ambiguity of the task, we first leverage a pre-trained video diffusion model to estimate the albedo for each input image. This data-driven estimate allows us to reconstruct \(\mathcal{G}_{A}\), which we then freeze to optimize \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\) to best reconstruct the images according to Eq. 1 .

3.2.1 Data-driven albedo estimation↩︎

To extract albedo for each input image, we employ DiffusionRenderer, a state-of-the-art material estimation method based on video diffusion models [31], which yields temporally consistent albedo estimates across all frames. Our initial experiments with single-image methods (e.g., [15]) revealed significant inter-frame inconsistencies that degrade 3D reconstruction quality.

In practice, video capture produces sequences containing hundreds of frames, whereas DiffusionRenderer is limited to processing clips of 57 frames at 24 FPS. Naïvely partitioning a long video into independent chunks and denoising each chunk separately introduces visible discontinuities in albedo predictions at chunk boundaries. To mitigate this issue, we adopt Brick Diffusion [32] and stagger the denoising process across neighboring chunks, enabling smooth transitions between the chunks.

Cosmos DiffusionRenderer operates in the latent space of the Cosmos Video Tokenizer [33]. However, the tokenizer’s asymmetric design complicates the direct application of brick diffusion. DiffusionRenderer denoises eight latent codes per step, where the first latent decodes to a single frame, and each subsequent latent decodes to a group of eight frames, yielding a total of 57 frames. Effective application of brick diffusion requires shifting the denoising window by both an integer number of latent codes and an integer number of frames simultaneously.

To address this, we process the sequence in chunks of 64 frames, corresponding to one start latent and eight regular latents. At each denoising step, we operate on 57 out of the 64 frames to denoise one start latent and seven of the eight regular latents. The window is then advanced by four latent codes, corresponding to a shift of 32 frames. After denoising, all start and regular latents are decoded to frames, and redundant frames are discarded. This strategy produces temporally consistent albedo predictions over long sequences with minimal boundary artifacts.

The video diffusion model expects and produces video at 24 FPS, which is far too dense for radiance field reconstruction. Hence the set of input images we use in our method is a subset of the video frames, it is obtained by subsampling the video sequence to a few hundred images taking every \(n\)th frame, with a preference for frames with low blur. We determined experimentally that sampling every \(n=5\) frame provided a sufficiently dense camera path for high quality novel view synthesis. We then run Structure-from-Motion (SfM) [34] on the extracted images to compute camera poses. Importantly, we apply the undistortion SfM applies on the input images to the albedo images as well.

3.2.2 Albedo reconstruction↩︎

As can be seen in Fig. 1, the albedo images typically contain piecewise constant color regions with an insufficient number of geometric cues, making it harder for the 3DGS algorithm to place Gaussians in 3D space. We address this challenge by regularizing the reconstruction against depth maps estimated for each input view [35], which we obtain using the Depth Anything V2 monocular depth estimator [30].

As in 3DGS, we start from a set of calibrated cameras and an initial point cloud typically produced by SfM. The set of albedo Gaussians \(\mathcal{G}_{A}\) is optimized using the same learning rates and activation functions as in the original implementation except for the albedo color, which we expect to be in \([0;1]\) so we use a sigmoid activation function. For regularization, we follow [35] and render a depth image \(D\) by replacing the color of the Gaussians by their depth. Given the predicted depthmap \(\tilde{D}\), we define the regularization term as: \[\mathcal{L}_{depth}=||\tilde{D} - D||_1.\] This regularization term encourages the placement of Gaussians close to the actual geometry of the scene and greatly improves densification in texture-free regions, such as the constant color patches frequently observed in the input albedo images. Depth regularization also helps snapping the user edits to the scene surface during albedo editing (Sec.3.3).

To improve multi-view consistency, we add another regularization term on the average scale \(\bar{s}\) of each Gaussian primitive to encourage the primitives to flatten and form sharp surface boundaries: \[\mathcal{L}_{scale}= \bar{s}.\]

Considering the predicted albedo images \(\tilde{A}\) and rendered albedo images \(A\), the total loss function for reconstructing the albedo radiance field is computed as: \[\label{albedo95loss} \mathcal{L} = (1 - \lambda) ||\tilde{A} - A||_1 + \lambda \mathcal{L}_{D-SSIM}(\tilde{A},A) + \lambda_d \mathcal{L}_{depth} + \lambda_s\mathcal{L}_{scale}.\tag{2}\]

An important benefit of our approach is that the albedo field is decoupled from shading and thus does not reconstruct lighting variations and shadows. This results in more uniform distribution of Gaussian primitives across the scene, which in turn allows more efficient editing of albedo and texture.

3.2.3 Shading and Residual Reconstruction↩︎

Once the albedo field is reconstructed, the \(\mathcal{G}_{A}\) splats can be rendered from any viewpoint to produce albedo images. Given an input image \(I\) and the rendered albedo image \(A\), we invert the image formation model (Eq.@eq:eq:basic95im ) to optimize the shading and residual Gaussians \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\). Denoting \(S\) and \(R\) the rendered shading and residual, we aim to minimize the reconstruction loss: \[\mathcal{L} = (1 - \lambda) ||I - (A \times S + R)||_1 + \mathcal{L}_{D-SSIM}(I , A \times S + R) + \lambda_d \mathcal{L}_{depth} + \lambda_s\mathcal{L}_{scale} \label{eq:full95reconstruction}\tag{3}\]

However, since jointly optimizing \(\mathcal{G}_{S}\) and \(\mathcal{G}_{R}\) is a challenging task, we perform this optimization in two phases. First, we ignore the residual term and only optimize \(\mathcal{G}_{S}\) using the diffuse image formation model \(I = A \times S\); these shading Gaussians are initialized from SfM points in the same manner as albedo Gaussians. Once this first optimization is completed, we freeze \(\mathcal{G}_{S}\) and optimize \(\mathcal{G}_{R}\) to capture residual non-diffuse effects.

Furthermore, since the shading of a scene is unbounded and can exhibit high dynamic range, we follow the recommendations of [13] and compute the photometric loss in the inverse domain, i.e., minimizing the difference between \(\frac{1}{1 + I}\) and \(\frac{1}{1 + A \times S + R}\). This transformation bounds the values in \([0;1]\), alleviating the need for an activation function on the color parameters.

If an albedo channel is dark, the shading can become arbitrarily large, which may lead to incorrect color reproduction. To alleviate this problem, we encourage the shading to be gray with an additional regularization term that computes, for each shading Gaussian, the range of its color parameters: \[\label{eq:col95reg} \mathcal{L}_{gray} = max(c) - min(c).\tag{4}\]

Residual Gaussians \(\mathcal{G}_{R}\) are created where they are most likely to be needed. Specifically, we iterate through the training views and compute the difference between the input image and the composited diffuse render (i.e., \(A \times S)\). We randomly select a subset of pixels with high difference and use their predicted depth as positions to spawn residual Gaussians. We model the color of these residual Gaussians with up to 3 bands of spherical harmonics to represent view-dependent effects. We use the same scheduling as in 3DGS to progressively add additional bands to the spherical harmonics and we keep the same scaling and depth regularizations as in albedo and shading reconstruction stages. Optimization then proceeds with the full image formation model (Eq. 3 ) but only the residual Gaussians are updated.

Each optimized field is composed of primitives with positions and shapes adapted to the content of each intrinsic layer. This can be seen in Fig. 3 where in the two rightmost images we have artificially reduced the scale of the Gaussians to better visualize the individual primitives. We can see that small and anisotropic Gaussian primitives are created in the shading layer to model shadows, and equivalently to model texture content in albedo.

3.3 Editing intrinsic decompositions of Gaussian splats↩︎

Once we have the full radiance field decomposed in three components, it is easy to edit the albedo separately while maintaining lighting and shadow effects. We provide an interactive interface allowing the user to perform albedo editing. To better comprehend the editing process, please see the supplemental video.

Users of our interface can edit the albedo in any input view. However, while we can minimize Eq. 2 on this view to update the albedo Gaussians accordingly, the resulting albedo field often degrades as soon as we move away from the view under which the editing was performed. Achieving multi-view consistency requires minimizing Eq. 2 over multiple edited views, which would be prohibitive to provide by users. Our solution consists in synthesizing these multiple views using depth-based reprojection of the user edits.

Unfortunately, 3DGS does not provide sufficiently reliable depth to perform such reprojection into neighboring views; for this reason we use a proxy plane to approximate the surface to be edited. The user first indicates the surface of interest by dragging a rectangular window in the image. We select, for each pixel in that window, the primitive that contributes most to its rendering. We then use RANSAC to fit the proxy plane over the center of these Gaussians. The plane can be optionally manipulated using standard 3D widgets, allowing the user to adjust its placement in the scene. Since a typical editing session involves adding or replacing the texture of a surface; we render the new texture over the proxy plane for pre-visualization.

Once the user is satisfied with the position of the textured proxy plane, we render a set of edited albedo images from multiple nearby viewpoints by compositing the plane with the albedo field, using the rendered depth to test for occlusions. We sample these virtual viewpoints on an hemisphere centered on the editing viewpoint. Finally, we use these multiple edited images to re-optimize the albedo Gaussians using Eq. 2 . Since the user edits might contain details not present in the original albedo, we spawn additional albedo Gaussians on the textured proxy plane by randomly sampling \(20\%\) of the pixels covered by the edit in the edited view. This editing process is illustrated in Fig. 4.

4 Implementation↩︎

Since we are performing lighting computation, we need to perform all our compositing operations in linear space. We thus apply a simple inverse gamma correction to the sRGB images used as input to obtain “pseudo-linear” RGB images used for processing. We apply the same gamma inversion to the albedo maps produced by DiffusionRenderer, which we observed to be non-linear. In addition, the albedo images can be quite blurry, resulting in floaters. We apply a simple floater reduction approach to improve our results. Our floater removal operates on each input view: We visit each pixel and accumulate all Gaussians until transmittance is saturated (“expected termination” depth). We then compute the standard deviation of the depths of these primitives and increment a counter for those that have depth less than \(n\) standard deviations of the average depth; larger floaters typically satisfy this condition for several pixels, resulting in a high value of the counter. We then remove primitives with a high counter value. This operation is performed before every densification step, which happens every 100 iterations from iteration 500 to iteration 15000 of the optimization.

We have implemented our system in an interactive viewer [36] shown in Fig. 5. The viewer includes interactive editing tools to select albedo Gaussians to be edited, and allows easy manipulation of the proxy plane to achieve the best results, including 3D widgets (please see supplemental video). We will release all source code of our system on publication.

5 Results and Evaluation↩︎

In Fig. 6 we show edits on three real scenes captured with a Canon EOS R6 in continuous shooting mode. The resulting sequence of still frames is dense enough to be processed as a video by DiffusionRenderer, which we use to extract albedo from the input images. Table. 1 provides a quantitative comparison on training times and primitive counts between our method and vanilla 3DGS [7] across our 4 scenes, all experiments ran on a H100 GPU. Our method requires an additional render call every time a new layer is reconstructed (i.e., two render calls when reconstructing a diffuse scene, three when enabling the residual layer) which yields slower rendering times that inevitably leads to longer training times.

We also show results on a synthetic scene that we use for the quantitative evaluation in the ablations (see Sec. 5.2). We rendered 288 images of this scene that we used as input photos for our intrinsic 3DGS pipeline. Our results on this synthetic scene (Fig. 7) illustrate the best possible outcome, since we use ground truth albedo and depth maps to reconstruct the intrinsic fields.

4pt

5.1 Comparisons↩︎

5.1.1 Intrinsic decomposition↩︎

We compare our intrinsic decomposition both quantitatively and qualitatively with two inverse rendering methods based on 3D Gaussian splatting on the synthetic scene shown as inset in Fig 11. R3GS [37] and GI-GS [38] both propose a full inverse rendering pipeline using 3DGS, the former uses raytracing directly on Gaussians in 3D space while the latter relies on GBuffers obtained by rasterizing different properties on Gaussians to speed up the computations. Table. 2 shows that on our synthetic indoor scene, our method produces higher quality albedo while being significantly faster than the others at the cost of generating \(\times\)​3.5 more primitives than GI-GS [38]. R3DG [37] also assumes a near natural white lighting so it cannot correctly represent the warmer, more realistic light placed in our synthetic scene; moreover it is designed for object-centric scenes so it fails to reconstruct our extended scene as shown in Fig 8.

5.1.2 Interactive edition↩︎

We choose to compare our work on an interactive editing task against two other 3DGS editing methods that can perform recoloring. ReCoGS [29] allows interactive tinting of Gaussians by reoptimizing the colors of the primitives affected by a user-specified mask. The second method we compare to is SplatShop [28], which provides a framework to select, transform and directly paint Gaussians that are intersected with a 3D brush. SplatShop is tailored for VR and the painting does not involve any optimization.

This recoloring application showcases two advantages of our method compared to previous work, as it allows to modify the albedo without altering the shading. This is illustrated in Fig. 9 where we recolored textured surfaces with a uniform red color, for a synthetic scene and a real scene. Setting a red tint in ReCoGS preserves both the shading and the texture information, while painting with a red brush in SplatShop erases both the shading and the texture. In contrast, our method allows to remove the texture without altering the shading and cast shadows. Note however that on the real scene, errors in the albedo prediction makes some albedo details appear in the shading, and as such remain visible in our edit, albeit much less than with ReCoGS. Our decomposition also yields a sharper boundary around the edited area, which we achieve by updating the parameters of the albedo Gaussians while keeping the shading and residual Gaussians untouched.

5.2 Ablation Study↩︎

We evaluate our method across different configurations on the quality of the reconstructed albedo field, shading field and fully composited renderings using the image formation model of Eq. 1 .

We use the synthetic scene shown as inset to perform quantitative evaluation, comparing the rendered results of each reconstructed field with the values of ground truth renderings.

Replacing ground truth albedo maps with predicted albedo maps gives a blurrier albedo reconstruction, which makes it harder to reconstruct shading and residual layers free of albedo variations (Fig. 13). We tried several other options, but the multi-view consistent albedo maps from DiffusionRenderer are currently the best option, even if quite blurry. Moreover, we observed that DiffusionRenderer does not produce linear outputs. While the gamma inversion we apply compensates for this nonlinearity, it is only an approximation and some color shift sometimes remains.

6 Limitations↩︎

While our method allows a reasonable intrinsic decomposition of 3D Gaussian Splatting, it comes with some limitations.

The first limitation is the “fuzzy” nature of the 3DGS representation itself, that makes editing complicated. We have demonstrated a practical solution for edits on planar surfaces, requiring some user interaction. This could be extended to simple non-planar shapes by fitting suitable proxies (e.g., cylinders or surfaces of revolution), but providing a truly general solution would be more complex. Methods that compute meshes from 3D Gaussian splats (e.g., [39], [40]) could potentially be used to initialize proxies, but accurate edits would still be challenging. Our method also inherits popping artifacts already present in standard 3DGS reconstruction, most notably seen in darker regions. These could be reduced by sorting the primitives as in StopThePop [41]. The second limitation is our dependence on the quality of diffusion models to extract albedo layers. While the layers can be used in a useful manner as shown in our results on real scenes, the predictions are often blurry, and sometimes can assign texture to shading. An example is shown in Fig. [fig:limitation]. Future progress in intrinsic image and video decomposition will directly benefit our method. Finally, as previously mentioned in section 5 our method is slower than 3DGS but a caching mechanism could be implemented to avoid re-rendering pretrained layers when training. At inference time this caching mechanism cannot be used but smarter memory layout of the Gaussians representing different layers could be used to increase rendering speed, since the current implementation consists of calling the renderer 3 times with different sets of Gaussians.

7 Future Work and Conclusion↩︎

In future work, we would like to investigate how to build on our ideas to allow full inverse rendering for radiance fields. Our explicit decomposition is promising, but many challenges remain in terms of the actual representation and the capacity to perform light transport.

We have presented a new method for intrinsic decomposition of 3D Gaussian splats, using diffusion model priors for albedo and depth. We introduce a new approach that first predicts an albedo field, and then jointly optimizes a shading and residual field. This intrinsic decomposition of 3D Gaussian splats allows easy editing of albedo and texture, while maintaining the appearance of shading, allowing for realistic edited results.

References↩︎