1 Introduction↩︎

Image compression is essential for real-time rendering & VR/AR [2] and high-density media storage [3], demanding realistic visual fidelity alongside computational efficiency. While traditional codecs like JPEG and PNG provide robust baselines, deep learning architectures [2] have been proposed to further improve the boundaries of rate-distortion performance. More Recently, have been posed as a promising alternative for compact image representation [1], [4]. By modeling local color and structural details with anisotropic Gaussian primitives, Gaussian-based image representation enables flexible, resolution-independent rendering and ultra-fast, GPU-friendly inference [1], [5], [6]. However, this flexibility incurs a storage cost. Each 2D Gaussian primitive consists of multiple unconstrained floating-point parameters (e.g., positions, covariance, and color). Accurately capturing high-frequency textures requires dense splat allocations, bloating the parameter count and degrading compression efficiency compared to SOTA VAE-based codecs [7], [8]. To address this, GI [1] proposed to use Vector Quantization (VQ) and Residual Vector Quantization (RQ) to compress these floating-point attributes into discrete codebooks. Yet, forcing a capacity-constrained global codebook to map the immense variance of natural image parameters inevitably leads to quantization errors and visual artifacts. This limitation demands the need for a representation that preserves an accurate reconstruction of Gaussians while constraining parameter entropy to improve codebook efficiency.

We propose to leverage a Cluster-based Codebook Quantizing method, CGVQ. Based on the principle: for a fixed codebook capacity, narrower parameter distributions are likely to achieve better quantization [9]. We therefore apply K-Means to divide 2D Gaussian primitives into homogeneous clusters and train cluster-specific codebooks for position, rotation-scale, and color, followed by efficient lookup table-based decoding. The localized multi-codebook design improves reconstruction quality with only a marginal compression overhead, achieving lower distortion than baseline models within specific bitrate regimes. Our main contributions are summarized as follows:

• We propose CGVQ, which partitions 2D Gaussian primitives into low-variance groups before quantization.

• We show that localized codebooks provide a better compression–fidelity trade-off than a global codebook baseline, improving PSNR/SSIM at comparable bitrate. This highlights the potential of utilizing K-Means clustering as a promising direction for future codebook quantization optimization.

Our method CGVQ is inspired and extended based on GI [1]. The code is available at this repository.

2 Method↩︎

Following GI, we represent the ground truth \(I\) as a set of \(N\) 2D Gaussian primitives \(G_i\): \(\mathcal{G} = \{G_i\}_{i=1}^{N},\quad G_i = \{\boldsymbol{\mu}_i,\, \mathbf{RS}_i, \mathbf{c}_i\},\) where \(\boldsymbol{\mu}_i \in \mathbb{R}^2\) is the 2D position, \(\mathbf{RS}_i \in \mathbb{R}^{2 \times 2}\) is the combined rotation-scale matrix, and \(\mathbf{c}_i \in \mathbb{R}^3\) is the weighted color feature [1]. In practice, the rotation-scale matrix \(RS_i\) is parameterized by two learnable attributes: a scale vector \(\mathbf{s}_i \in \mathbb{R}^2\) and a rotation parameter \(\mathbf{r}_i \in \mathbb{R}^1\). The matrix used for rendering is then obtained by a deterministic mapping \(\mathbf{RS}_i = F(\mathbf{r}_i, \mathbf{s}_i),\) where \(F(\cdot)\) constructs the corresponding rotation-scale matrix from the rotation and scale parameters.

To enable efficient per-cluster quantization, we group Gaussian primitives by feature similarity prior to codebook training. For each Gaussian primitive \(G_i\), we construct a clustering feature vector by concatenating its rotation, scale, and RGB color attributes: \(\mathbf{f}_i = \bigl[\,\mathbf{r}_i,\; \mathbf{s}_i,\; \mathbf{c}_i\,\bigr] \in \mathbb{R}^6,\) where \(\mathbf{r}_i \in \mathbb{R}^1\) denotes the rotation parameter, \(\mathbf{s}_i \in \mathbb{R}^2\) denotes the two scale parameters, and \(\mathbf{c}_i \in \mathbb{R}^3\) denotes the RGB color feature. The 2D position \(\boldsymbol{\mu}_i\) is deliberately excluded, as compression quality shows a high sensitivity to the spatial accuracy. We then apply K-Means on the normalized features \(\{\hat{\mathbf{f}}_i\}\) to partition \(\mathcal{G}\) into \(K\) disjoint clusters by minimizing the total intra-cluster variance: \[\{\hat{\mathcal{C}_k}\}^K_{k=1} \leftarrow \mathop{\mathrm{argmin}}_{\{\mathcal{C}_k\}} \sum_{k=1}^{K} \sum_{i \in \mathcal{C}_k} \bigl\|\hat{\mathbf{f}}_i - \boldsymbol{m}_k\bigr\|^2,\] where \(\boldsymbol{m}_k\) is the centroid of cluster \(\mathcal{C}_k\). The number of clusters \(K\) is selected manually with K between the range of 4 to 16, based on the number of primitives, to keep the balance between reconstruction fidelity and compression efficiency. Clustering is performed once after the initial training phase converges, and the resulting cluster assignment \(\{k_i\}\) is shared across all three parameter-specific quantizers, ensuring consistent grouping throughout the entire compression pipeline. As shown in Fig. [fig:codebook_training], for each cluster \(\mathcal{C}_k\), we train three independent codebooks, one per parameter type, to exploit the distinct statistical properties of each attribute: For each cluster, we model three components: (1) position \(\boldsymbol{\mu}\) using 16-bit floating-point QAT with a straight-through FakeVQ estimator, (2) rotation-scale \(\mathbf{RS}\) using a Uniform Vector Quantization (UQ) codebook, and (3) color \(\mathbf{c}\) using a Residual Vector Quantization (RQ) codebook for fine-grained residual refinement. All codebooks are trained end-to-end by minimizing a joint loss combining image reconstruction and vector quantization terms: \(\mathcal{L} = \mathcal{L}_{\text{recon}}(\hat{I},\, I) + \mathcal{L}_{\text{vq}}\) where \(\mathcal{L}_{\text{recon}}\) is a weighted combination of a pixel-wise L1 loss in image space and a perceptual L1 loss in feature space, with \(\lambda = 0.7\), and \(\mathcal{L}_{\text{vq}}\) is the commitment loss that regularizes encoder outputs toward their assigned codebook entries across all active codebooks.

As illustrated in Figure 2, each compressed cluster \(\mathcal{C}_k\) is decoded independently by looking up its three dedicated codebooks—FP16 for position \(\boldsymbol{\mu}\), UQ for rotation-scale \(\mathbf{RS}\), and RQ for color \(\mathbf{c}\)—using the stored quantization indices \(z_i^{\mathbf{RS}}\) and \(z_i^{\mathbf{c}}\) to reconstruct each primitive \(\hat{G}_i\). The decoded primitives from all \(K\) clusters are then composed into a single Gaussian set \(\mathcal{G}' = \bigcup_{k=1}^{K} \hat{\mathcal{C}}_k\) for rendering. To further reduce bitrate, we also apply Partial Bits Back Encoding [1] to the quantization index tables.

3 Evaluation↩︎

In our experiments, \(K\) is fixed to 16, which provides a suitable trade-off for scenes containing 10K to 25K Gaussian primitives. Rate-Distortion Efficiency: Compared with the GI baseline [1], CGVQ consistently demonstrates improved rate-distortion performance on the Kodak dataset. As shown in Fig. 4, CGVQ achieves higher PSNR and MS-SSIM than GI over the evaluated bitrate range, indicating better reconstruction quality at comparable bitrates. This advantage becomes more evident in the high-quality regime. In particular, for reconstruction quality above approximately \(30\) dB, CGVQ maintains a clear gain in PSNR while also preserving stronger structural similarity, suggesting a more favorable rate-distortion trade-off overall. Parameter Efficiency: We further assess parameter efficiency under a fixed Gaussian primitives. As shown in Fig. 3, with the same 15K Gaussian primitives, CGVQ improves PSNR by 1.68 dB\(\uparrow\) relative to GI. In addition, CGVQ at 15K points attains visual quality comparable to GI at 19K points, corresponding to a 21%\(\downarrow\) reduction in primitive count. The result of CGVQ stems from the principle that narrower parameter distributions are inherently easier to quantize accurately. By partitioning 2D Gaussian primitives into \(K\) homogeneous clusters based on appearance and anisotropy similarity, we effectively constrain the variance of parameters within each group. This allows our localized multi-codebook design to capture high-frequency details more precisely than global codebook approaches. Performance Trade Off: Shown in Table 1 the number of clusters reduces both encoding and decoding speed, with encoding FPS dropping from \(2.91\times10^{-2}\) to \(1.9\times10^{-3}\) and decoding FPS from 133.3 to 33.3 as the cluster count increases from 1 to 16. This shows that finer clustering introduces a clear runtime overhead, highlighting a trade-off between compression performance and computational efficiency. To conclude, CGVQ improves 2D Gaussian-based image compression by enabling more efficient quantization through cluster-specific codebooks, while introducing a clear trade-off between compression quality and computational efficiency.

References↩︎