1 Introduction↩︎

point cloud semantic segmentation has wide-ranging multimedia applications across various fields, such as autonomous driving [@li2025s4r; @fei2024enhancing], robotics [@soori2023artificial; @hu2024towards], and augmented reality [@sereno2020collaborative]. While fully-supervised learning has achieved numerous improvements in complex scene understanding [@zhang2024pointgt; @lai2022stratified], its effectiveness is constrained by the need for large-scale, expensive and fully-annotated datasets. To address this challenge, few-shot 3D point cloud semantic segmentation (FS-PCS) has recently attracted increasing attention, enabling models to generalize to unseen novel categories with just a few annotated samples [@an2024rethinking; @zhu2024no; @he2023prototype].

Existing FS-PCS methods typically adhere to the meta-learning framework [@vinyals2016matching; @snell2017prototypical], where prototypes derived from a handful of annotated support samples serve as benchmarks for segmenting unseen query point clouds. However, due to the inherent sparsity and non-structural nature of point clouds, prototypes derived from the limited support set are often misaligned and suffer from a small capacity, hindering effective generalization to novel categories. We term the prototype’s limited representational capacity as “small capacity". These limitations stem from two specific issues: i) the prototype’s limited representational capacity fails to cover the full intra-class diversity of a novel category, and ii) the prototypes suffer from misalignment with the query space due to the inter-set inconsistency between support and query sets.

A core limitation of existing methods is the prototype’s small representational capacity. This issue, which we term the “small-capacity" problem, originates from the intra-class diversity inherent in 3D data, as point cloud object shapes and appearances can vary dramatically even within the same category. Consequently, prototypes distilled from only a few support samples are inherently limited in the richness and diversity of their feature components, leading to an insufficient representational range to cover the entire distribution of a novel class. While existing methods attempt to mitigate this by employing multi-prototype strategies [@zhao2021few; @mao2022bidirectional], they merely increase the quantity of prototypes without fundamentally expanding the intrinsic representational capacity of the support features, thus still struggling to cover the full distribution of the novel class.

Simultaneously, these prototypes suffer from misalignment within the query feature space. This problem is a direct consequence of inter-set inconsistency, a natural distribution gap that originates from the different 3D scenes of the support and query sets. This gap introduces a notable bias when applying support-based prototypes to the query space. To address misalignment, prior works rectify this by “pushing" prototypes towards the query distribution through various alignment strategies [@ning2023boosting; @mao2022bidirectional]. Nevertheless, given the pronounced distribution shift in sparse 3D point clouds, the corrective force of such discriminative alignment is insufficient to bridge the gap, leaving the prototypes misaligned with query space.

So aforementioned analysis presents a fundamental dilemma: a well-aligned prototype is ineffective if its capacity is too small to cover class diversity, while a big-capacity prototype will fail if it is not properly aligned within the query space. Motivated by these important observations, a pertinent question arises: How can we leverage a few-shot support samples to construct aligned big-capacity prototypes for segmenting novel query classes?

In this paper, we explore expanding the effective representational capacity of prototypes in few-shot 3D learning scenarios. Our main idea is to enrich the prototype’s feature components to increase its capacity and align it towards the query feature space. To achieve this, we construct our prototype from two complementary feature sources. On one hand, we retain the traditional fully-supervised pre-training pipeline to learn representative features. These features provide a stable semantic core for the prototype, ensuring its discriminative power to segment unseen targets (Figure 1, a→b). On the other hand, we introduce a self-supervised diffusion process to provide the crucial generalizable features needed for prototype expansion. Motivated by the powerful generative capabilities of diffusion models [@ho2020denoising], we re-purpose a diffusion model’s pre-trained conditional encoder to provide these features. While these models recover a complete object shape from pure noise when guided by a condition vector, the conditional encoder learns to extract robust geometric priors. These priors serve as the rich generalizable features to expand the prototype’s representation capacity (Figure 1, a→c). To resolve the misalignment issue, dual prototypes are constructed from these two feature sources and then aligned and fused into a unified big-capacity prototype (Figure 1, d).

Under this prototype expansion setup, we introduce a novel framework, the Prototype Expansion Network (PENet), to effectively address the challenges in FS-PCS by harnessing two complementary feature sources. The overall architecture of PENet is depicted in Figure 2. Given the support and query point clouds, PENet first employs a dual-stream learner architecture to extract two sets of features: representative features from the Intrinsic Learner (IL) and generalizable features from the Diffusion Learner (DL) (cost-free). Based on initial prototypes generated from these features, a subsequent Prototype Assimilation Module (PAM) is designed to adopt a novel push-pull cross-guidance attention mechanism to iteratively align the two sets of prototypes with the query space, providing a more robust corrective force than single-step alignment methods. Subsequently, a prototype fusion step combines the aligned dual prototypes into a single big-capacity prototype. Additionally, to ensure the expanded prototype does not deviate from its source semantics, we introduce a Prototype Calibration Mechanism (PCM) that calibrates the final prototype’s representation by reconstructing source support set mask.

Our main contributions can be summarized as follows:

• We propose the PENet to overcome the critical limitations of small capacity and misalignment that hinder traditional prototypes in FS-PCS. To the best of our knowledge, this is the first work to explore diffusion model in FS-PCS.

• We design a dual-stream architecture that features an iterative PAM for robust prototype alignment and a PCM to ensure its semantic fidelity.

• Extensive experiments on the S3DIS and ScanNet benchmarks achieve state-of-the-art across various few-shot settings. Our work validates the value of using generative model components for discriminative few-shot learning.

2 Related Work↩︎

2.1 Few-Shot 3D Point Cloud Semantic Segmentation.↩︎

FS-PCS aims to segment novel classes using a minimal number of annotated support samples. The field was pioneered by AttMPTI [@zhao2021few], which introduced a multi-prototype transductive inference framework. Subsequent research has focused on mitigating the discrepancy between support and query distributions. Notably, in the broader few-shot learning domain, distribution calibration was proposed to address the biased distribution issue by hallucinating features via statistical transfer from base classes [@yang2021bridging]. Meta-transfer learning facilitates the adaptation of deep networks to novel tasks via weight transfer operations [@sun2020meta]. Similarly, semantic topology maintain consistent representations for novel classes by leveraging semantic constraints [@yang2021objects]. In the 3D domain, many works propose to adapt or align support prototypes by incorporating guidance from the query space [@zhang2023few]. Other approaches have focused on enhancing feature representations, for instance, by designing modules for bidirectional feature globalization [@huang2023part; @zhang2023few]. However, these methods typically rely on a single source of discriminative features. Our work departs from this by introducing a complementary feature stream to fundamentally expand the prototype’s representational capacity.

2.2 Diffusion Models for Representation Learning.↩︎

Diffusion models have recently achieved state-of-the-art results in generative tasks, including 3D point cloud generation [@lyu2023lion; @nichol2022point]. Beyond pure generation, their potential for representation learning and discriminative tasks has become an active area of research [@peebles2023scalable]. One line of work utilizes conditional diffusion models to synthesize labeled data for augmenting training sets in data-scarce scenarios [@gandikota2023erasing]. Another approach employs diffusion models directly as zero-shot classifiers, leveraging their learned data likelihood to predict class probabilities [@li2023your]. Furthermore, some studies have demonstrated that the intermediate features of the U-Net denoiser are discriminative and can be used for downstream tasks [@bar2022visual; @huang2024progressive]. In contrast to these methods, our work is the first to re-purpose the pre-trained conditional encoder of a 3D diffusion model as a dedicated feature learner within a few-shot segmentation framework.

3 Preliminary↩︎

3.1 Fundamental: Conditional Diffusion Models↩︎

Diffusion models [@ho2020denoising; @nichol2021improved] are powerful generative models that operate in two stages: a forward noising process and a reverse generation process. The forward stage is a Markovian process that incrementally adds Gaussian noise to clean data \(x_0\) over \(T\) timesteps, defined by \(q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t\mathbf{I})\). This allows the noised sample \(x_t\) to be generated directly from \(x_0\) in a closed form: \(q(x_t \mid x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t}x_0, (1-\bar{\alpha}_t)\mathbf{I})\), where \(\alpha_t = 1-\beta_t\) and \(\bar{\alpha}_t = \prod_{i=1}^{t}\alpha_i\), \(\beta_t\in (0, 1)\) is a variance hyperparameter.

The reverse stage learns a neural network \(p_\theta\) to reverse this process, approximating the true posterior \(q(x_{t-1} \mid x_t)\) by modeling it as a diagonal Gaussian, \(p_\theta(x_{t-1} \mid x_t) = \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t), \sigma_t^2\mathbf{I})\). In a conditional diffusion model [@zhang2023adding], this reverse process is guided by a condition vector \(v\), effectively modeling \(p_\theta(x_{t-1} \mid x_t, v)\).

In this work, we do not use the diffusion model for its generative capabilities directly. Instead, we leverage its pre-trained conditional encoder, the component responsible for producing the condition vector \(v\). The encoder enables the extraction of rich generalizable features even from the few samples in FS-PCS task.

3.2 Problem Setup↩︎

We follow the standard episodic paradigm for FS-PCS, as established in prior works [@zhao2021few]. Each task instance is framed as an N-way K-shot episode, which contains a support set \(\mathcal{S}\) and a query set \(\mathcal{Q}\). The support set \(\mathcal{S} = \{(X_{s}^{n,k}, Y_{s}^{n,k})\}_{n=1, k=1}^{N,K}\) provides \(N\) novel classes, where each class \(n\) is represented by \(K\) annotated samples (shots). The query set \(\mathcal{Q} = \{(X_q, Y_q)\}\) consists of an unannotated point cloud that contains instances of the \(N\) novel classes. The objective is to leverage the knowledge from the support set \(\mathcal{S}\) to predict a point-wise semantic mask for the query point cloud \(X_q\), segmenting it into one of the \(N\) target classes or a background category. During meta-training and meta-testing, the set of classes are disjoint (\(\mathcal{C}_{\text{train}} \cap \mathcal{C}_{\text{test}} = \emptyset\)), forcing the model to learn a generalizable segmentation capability rather than memorizing specific classes.

4 Methodology↩︎

In this section, we first detail our dual-stream Intrinsic and Diffusion Learners; then, we introduce the generation of dual prototypes; next, we present the Prototype Assimilation Module. Finally, we explain how these prototypes are fused and calibrated to construct our PENet framework, as shown in Figure 2.

4.1 Prototype Generation↩︎

4.1.1 Feature Extractors↩︎

Unlike prior few-shot methods that rely on a single shared feature extraction backbone [@zhu2023cross; @zheng2024few], our approach processes point cloud inputs through a dual-stream architecture with two distinct learners: the Intrinsic Learner (IL) and the Diffusion Learner (DL), as depicted in Figure 2. The IL extracts representative features by leveraging a fully-supervised pre-training paradigm, while the DL provides complementary generalizable features derived from a self-supervised diffusion process. The synergy between these two feature types provides the rich and diverse components necessary for our prototype expansion. Given the support/query point cloud \(X_{s/q} \in \mathbb{R}^{N \times D}\), we use the IL (\(\Phi_{IL}\)) and DL (\(\Phi_{DL}\)) to obtain their respective features, given by: \[\begin{align} F_{qi} = \Phi_{IL}(X_q) \in \mathbb{R}^{N_i \times D_i}, \quad F_{qd} = \Phi_{DL}(X_q) \in \mathbb{R}^{N_d \times D_d} \\ F_{si} = \Phi_{IL}(X_s) \in \mathbb{R}^{N_i \times D_i}, \quad F_{sd} = \Phi_{DL}(X_s) \in \mathbb{R}^{N_d \times D_d} \end{align}\] where the resulting \(F_{q/si}\) and \(F_{q/sd}\) represent the representative and generalizable features, respectively. Note that the learners operate at different resolutions, resulting in feature maps with different numbers of points (\(N_i, N_d\)) and channel dimensions (\(D_i, D_d\)).

4.1.2 Intrinsic Learner↩︎

Consistent with conventional FS-PCS frameworks, our method retains a fully-supervised feature learner as the Intrinsic Learner [@zhao2021few]. This learner is pre-trained via a standard supervised paradigm, enforcing a precise point-to-label mapping [@yu2021location]. This process endows the learner with strong discriminative capabilities, allowing it to extract representative features that form a stable semantic core for the prototype. Following the approach in [@zhao2021few], we adopt the widely used DGCNN [@wang2019dynamic] as the backbone for our IL to obtain these features.

4.1.3 Diffusion Learner↩︎

While the representative features from the IL provide a strong discriminative foundation, they struggle to generalize across substantial intra-class variations. To address this limitation, we introduce a parallel diffusion learner in the upper part of Figure 2. The core principle is that an encoder capable of guiding a generative process from pure Gaussian noise to a complete object learned the object’s generalizable geometric priors. These priors provide the rich feature components necessary to expand the prototype’s representational capacity. To this end, we re-purpose the pre-trained conditional encoder of the diffusion model as our DL. Unlike the IL, which processes the full point cloud, the DL takes diverse incomplete point cloud patches as input to absorb richer generalizable features. Inspired by PointDif [@zheng2024point], given an input point cloud \(X_{s/q}\), we first partition it into patches and apply a high-ratio random mask. The DL encoder, a multi-layer Transformer, then processes only the visible patches \(x_i^v\). To preserve crucial 3D spatial information, a position embedding function \(\psi(\cdot)\), realized by an MLP, generates a positional encoding \(Pos_i^v\) for the center coordinate \(C_i^v\) of each visible patch. This encoding is fused with the patch features \(F_i^v\) and fed into the transformer encoder to extract the latent generalizable features \(F_d\): \[F_d = \Phi_{DL}(\text{Concat}(F^v, \psi(C^v))),\] then generalizable features \(F_d\) with masked patch information are aggregated to provide the condition \(v\) that guides a denoising diffusion model \(\epsilon_\theta\) to progressively recover the original point cloud from a noisy version \(z_t\). The training objective is to minimize the following loss function: \[\mathcal{L}_{diff} = \mathbb{E}_{x_0, v, \epsilon \sim \mathcal{N}(0,I), t} \left[ ||\epsilon - \epsilon_\theta(z_t, t, v)||^2 \right],\] where \(z_t\) is the noisy point cloud at timestep \(t\), and \(\epsilon\) is standard Gaussian noise. To converge under this objective, the DL encoder is compelled to infer the complete object manifold from a highly incomplete input. This local-to-global inference task forces the encoder to learn the underlying structural patterns of a class, rather than surface-level details. These generalizable features provide the rich components necessary for prototype expansion, complementing the representative features from the IL.

4.1.4 Prototype Generation↩︎

Contrary to traditional few-shot models that generate prototypes from a single feature source [@zheng2024few], our method initializes two distinct sets of prototypes by integrating the complementary information from both the representative and generalizable feature streams. This dual-source approach ensures our initial prototypes are endowed with both discriminative power and robust generalizability. Initially, foreground and background prototypes are generated from the annotated support points for both the intrinsic features \(F_{si}\) and generalizable features \(F_{sd}\), using farthest point sampling and inverted sample clustering [@an2024rethinking]. This prototype generation, denoted as \(\text{ProtoGen}(\cdot)\), results in: \[\begin{align} p_c^i, p_{bg}^i &= \text{ProtoGen}(F_{si}, Y_s, L_s), \quad p_c^i, p_{bg}^i \in \mathbb{R}^{1 \times D_i} \\ p_c^d, p_{bg}^d &= \text{ProtoGen}(F_{sd}, Y_s, L_s), \quad p_c^d, p_{bg}^d \in \mathbb{R}^{1 \times D_d} \end{align}\] where \(Y_s\) and \(L_s\) are the labels and 3D coordinates of the support set points, respectively. The \(p_c, p_{bg}\) are the prototypes for a foreground class \(c\) and the background, respectively. Then the \(C\) foreground prototypes and the single background prototype are concatenated to form the complete prototype sets: \[\begin{align} P_i &= \text{Concat}(p_1^i, \dots, p_C^i, p_{bg}^i) \in \mathbb{R}^{(C+1) \times D_i} \\ P_d &= \text{Concat}(p_1^d, \dots, p_C^d, p_{bg}^d) \in \mathbb{R}^{(C+1) \times D_d} \end{align}\] where \(C+1\) is the total number of classes in the given \(N\)-way setting, including the background.

4.2 Prototype Assimilation Module↩︎

Having generated the dual prototypes, the critical challenge of prototype misalignment remains. This issue stems from the inter-set inconsistency between the support-derived prototypes and the query feature distribution, as discussed in Sec. I. To this end, we propose the Prototype Assimilation Module (PAM). As illustrated in Figure 3, PAM consists of push-pull blocks stacked to operate for M iterations. This mechanism functions figuratively as two interacting corrective forces applied iteratively: a pull force draws the intrinsic prototypes \(P_i\) towards the query space of diffusion features, while a symmetric push force drives the diffusion prototypes \(P_d\) towards the intrinsic feature space. Through an \(M\)-step process, the two prototype sets are jointly assimilated to better align with the query distribution. The core of this module is a cross-guidance channel-wise attention mechanism that directly updates the prototypes.

4.2.1 Pull Operation↩︎

Aligning Intrinsic Prototypes via Diffusion Space. The pull operation aims to refine the intrinsic prototypes \(P_i\) by leveraging the relational context of the diffusion space. First, we compute a channel-wise attention map \(\text{\textit{Attn}}_{\textit{pull}}\) where the Query and Key are derived from the diffusion-query features \(F_{qd}\) and diffusion-support features \(F_{sd}\), respectively: \[\textit{Query}_{\text{\textit{pull}}} = (F_{qd})^T W_q, \quad \textit{Key}_{\text{\textit{pull}}} = (F_{sd})^T W_k, \label{Qp}\tag{1}\] \[\begin{align} \textit{Attn}_{\textit{pull}} &= \text{softmax}\left(\frac{\textit{Query}_{\textit{pull}} \cdot \textit{Key}_{\textit{pull}}^T}{\sqrt{D}}\right), \end{align} \label{Ap}\tag{2}\] where \(W_q, W_k\) are learnable linear projection parameters. This attention map, capturing channel-wise dependencies within the diffusion space, then guides the update of the complementary intrinsic prototypes \(P_i\), which serve as the Value: \[\textit{Value}_{\textit{pull}} = P_i W_v.\] \[P_i^{\textit{update}} = (\textit{Attn}_{\textit{pull}} \cdot \textit{Value}_{\textit{pull}}^T)^T.\] This operation adjusts the intrinsic prototypes along their channel dimensions, infusing them with guidance from the diffusion space. The final aligned intrinsic prototype \(\hat{P}_i\) is then obtained via a residual connection and a small MLP \(\mathcal{F}_{\textit{pull\_mlp}}(\cdot)\): \[\hat{P}_i = P_i + \mathcal{F}_{\textit{pull\_mlp}}(P_i^{\textit{update}}). \label{Pb}\tag{3}\]

4.2.2 Push Operation↩︎

Aligning Diffusion Prototypes via Intrinsic Space. Symmetrically, the push operation refines the diffusion prototypes \(P_d\) using the relational context of the intrinsic space. An intrinsic channel-wise attention map \(\textit{Attn}_{\textit{push}}\) is first generated from the intrinsic-query features \(F_{qi}\) and intrinsic-support features \(F_{si}\): \[\textit{Query}_{\textit{push}} = (F_{qi})^T W'_q, \quad \textit{Key}_{\textit{push}} = (F_{si})^T W'_k.\] \[\textit{Attn}_{\textit{push}} = \text{softmax}\left(\frac{\textit{Query}_{\textit{push}} \cdot \textit{Key}_{\textit{push}}^T}{\sqrt{D}}\right).\] This map, encoding representative relationships, is then applied to the complementary diffusion prototypes \(P_d\) as the Value: \[\textit{Value}_{\textit{push}} = P_d W'_v.\] \[P_d^{\textit{update}} = (\textit{Attn}_{\textit{push}} \cdot \textit{Value}_{\textit{push}}^T)^T.\] Finally, a similar residual connection and MLP \(\mathcal{F}_{\text{push\_mlp}}(\cdot)\) are used to obtain the aligned diffusion prototype \(\hat{P}_d\): \[\hat{P}_d = P_d + \mathcal{F}_{\textit{push\_mlp}}(P_d^{\textit{update}}).\]

This cross-guidance attention mechanism enables a joint optimization of the dual prototypes. Furthermore, considering that the generalizable features from the diffusion encoder may operate at a different resolution, the attention map computed within the intrinsic space and the one computed within the diffusion space form a multi-scale correspondence. This allows the prototypes to absorb information from different granularities, leading to a richer final representation.

The final step is to merge these two complementary representations to construct the ultimate big-capacity prototype \(\hat{P}\): \[\hat{P} = \hat{P}_i + \hat{P}_d.\] The big-capacity prototype is equipped to cover the intra-class variations, enabling it to effectively match and identify morphologically diverse samples within the query set.

4.3 Prototype Calibration Mechanism↩︎

While the prototype assimilation and fusion processes expand the prototype’s representational capacity, they introduce a potential for semantic drift risk. As the prototypes continuously absorb generalizable features and align with the query space, the resulting prototype \(\hat{P}\) may deviate from the original class semantics defined by the support samples. To mitigate this issue, we introduce a Prototype Calibration Mechanism (PCM) which ensures the expanded prototype accurately traces back to its origin semantics. Inspired by [@he2023prototype], our calibration mechanism constrains the final prototype \(\hat{P}\) to accurately reconstruct the support set mask. The operator computes the similarity between \(\hat{P}\) and the intrinsic support features \(F_{si}\). We specifically interact with the intrinsic features because they preserve the original discriminative information of the support samples.

Specifically, for the given intrinsic support features \(F_{si}\) and the fused big-capacity prototype \(\hat{P}\), we compute a calibrated probability distribution \(\mathcal{P}_{\text{cal}}\) for each support point \(x\) over all classes \(c \in {C}\): \[\mathcal{P}_{\text{cal}}(x, c) = \frac{\exp\left(\frac{F_{si,x} \cdot \hat{P}^{c}}{\|F_{si,x}\| \cdot \|\hat{P}^{c}\|}\right)}{\sum_{k \in C} \exp\left( \frac{F_{si,x} \cdot \hat{P}^{k}}{\|F_{si,x}\| \cdot \|\hat{P}^{k}\|}\right)},\] where \(F_{si,x}\) is the intrinsic feature of the \(x\)-th support point, \(\hat{P}^{c}\) is the final prototype for class \(c\).

The calibration loss \(\mathcal{L}_{\text{cal}}\) is then formulated as the cross-entropy loss between this predicted probability distribution and the ground-truth support mask \(Y_s\): \[\mathcal{L}_{\text{cal}} = -\frac{1}{N_{s}}\sum_{x=1}^{N_{s}}\sum_{c\in\mathcal{C}} Y_{s,x}^{c} \log(\mathcal{P}_{\text{cal}}(x, c)),\] where \(Y_{s,x}^{i}\) is the one-hot ground-truth label. The overall training objective combines this calibration loss with the main segmentation loss \(\mathcal{L}_{\text{seg}}\), balanced by a hyperparameter \(\lambda\): \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{cal}}.\] This mechanism effectively anchors the prototype expansion process and prevents semantic drift.

5 Experiment↩︎

5.1 Experiment Setup↩︎

5.1.1 Datasets↩︎

We conduct a comprehensive evaluation of our proposed PENet on benchmark datasets covering both indoor and outdoor 3D few-shot semantic segmentation scenarios. For the indoor setting, we utilize two widely-used benchmarks: S3DIS [@armeni20163d] and ScanNet [@dai2017scannet]. S3DIS is composed of 272 point clouds from six large-scale indoor areas, encompassing 12 semantic categories and a clutter class. ScanNet is a larger collection of 1,513 scanned indoor scenes with 20 semantic categories. Following standard practice [@zhao2021few], we preprocess the raw scenes by partitioning them into \(1\,m \times 1\,m\) blocks and randomly sampling 2048 points as input. Each point is represented by a 9-dimensional vector, including its XYZ coordinates, RGB color, and normalized spatial coordinates.

To validate the model’s generalization capabilities in more complex and large-scale environments, we extend our evaluation to SemanticPOSS [@pan2020semanticposs], a challenging outdoor LiDAR dataset. The dataset was captured across campus and road environments at Peking University and contains 2,988 scans with 13 semantic categories (e.g., road, building, vegetation). For preprocessing, we partition each scene with \(6\,m \times 6\,m\) windows and a stride of 6 meters, resulting in 50,158 blocks. From each block, 2,048 points are randomly sampled, where each point is a 7D vector including XYZ, intensity, and normalized spatial coordinates. This configuration yields an average density of around 57 points/\(m^2\), which ensures sufficient geometric details for small objects (e.g., riders, trunks) while balancing GPU memory constraints.

For all datasets, we adopt the standard few-shot setting where classes are divided into two non-overlapping splits, S0 and S1, for cross-validation. For the indoor datasets S3DIS and ScanNet, we follow the class splits established in [@zhao2021few]. For SemanticPOSS, the splits are defined as follows: S0 consists of ‘person’, ‘trashcan’, ‘trunk’, ‘ground’, and ‘traffic sign’, while S1 includes ‘rider’, ‘building’, ‘pole’, ‘bike’, and ‘plants’. Given the sparsity of valid objects in outdoor scenes, we employ a category-indexed sampling strategy during meta-training. We pre-index all blocks by their constituent labels and explicitly sample blocks containing the target categories to construct valid N-way episodes. When one split is used for meta-training on base classes \(C_{\text{train}}\), the other is reserved for testing on novel classes \(C_{\text{test}}\).

5.1.2 Implementation Details↩︎

Our model is implemented in PyTorch and trained on NVIDIA A100 GPUs. We adopt a two-stage training paradigm consisting of feature learner pre-training and episodic meta-learning [@zhao2021few]. In the first pre-training stage, we train two distinct feature learners to extract complementary information. The Intrinsic Learner (IL), utilizing a DGCNN backbone [@wang2019dynamic], is trained using a standard fully-supervised semantic segmentation objective on the base classes (\(C_{train}\)) of the target dataset (e.g., S3DIS or ScanNet). Following common practice, it is optimized using the Adam optimizer with an initial learning rate of 0.001 to ensure it extracts discriminative representative features. In parallel, the Diffusion Learner (DL) is pre-trained on a large-scale 3D shape dataset (e.g., ShapeNet) via a self-supervised objective to capture generalizable geometric priors following [@he2023prototype]. Specifically, we partition each 3D shape into 128 patches and apply a high mask ratio of 80%. The DL’s transformer-based encoder is then compelled to generate a condition vector from the visible patches to guide a diffusion process in reconstructing the complete object. This reconstruction task allows the DL to learn robust structural priors without requiring semantic labels. In the second meta-learning stage, the parameters of both the pre-trained IL and DL are frozen to preserve their learned representations. The IL processes \(N_i\) = 2048 points, while the DL operates on \(N_d\) =128 patches. Both learners output features with a channel dimension of \(D_i =D_d\) = 384. The remaining components, including the Prototype Assimilation Module (PAM) and Prototype Calibration Mechanism (PCM), are trained end-to-end using the standard episodic paradigm on the support and query sets. The model is trained for 100 epochs with the Adam optimizer, utilizing an initial learning rate of 0.001 and a decay factor of 0.5 every 5,000 iterations. For the DGCNN backbone within the IL, the number of nearest neighbors \(k\) is set to 20. The number of iterations in the PAM is set to \(M=2\), and the loss balancing weight \(\lambda\) is set to 1. Data augmentation techniques, including random scaling, shifting, rotation, and Gaussian jittering, are applied to enhance robustness.

5.2 Comparison with State-of-the-Art Methods↩︎

Table ¿tbl:tab:s3dis? and ¿tbl:tab:scannet? shows the experimental results of our method compared with state-of-the-art (SOTA) approaches including FT [@zhao2021few], AttMPTI [@zhao2021few], BFG [@mao2022bidirectional], 2CBR [@zhu2023cross], PAP3D [@he2023prototype], QGPNet [@hu2023query], TFDR [@wang2024two], SQFI [@zheng2024few], Seg-PN [@zhu2024no], and SDSimPoint [@wang2025sdsimpoint].

5.2.1 Results on S3DIS↩︎

As shown in Table ¿tbl:tab:s3dis?, we compare PENet with SOTA methods on the S3DIS dataset. Our proposed PENet establishes SOTA performance across most few-shot settings. Specifically, in the challenging 2-way 1-shot and 3-way 1-shot scenarios, PENet achieves average mIoUs of 70.04% and 63.07%, surpassing the previous best method, SDSimPoint, by margins of +0.37% and +0.88%, respectively. The performance gains become more pronounced in the 5-shot settings, where PENet achieves 75.85% and 68.67% mIoU in the 2-way and 3-way scenarios, surpassing SDSimPoint by substantial margins of +3.43% and +3.03%. This demonstrates the effectiveness of our prototype expansion approach. Notably, we observe that PENet exhibits a wider performance gap between the two splits, with higher scores on the S1 split compared to S0. This trend suggests that our prototype expansion mechanism effectively enhances generalization. By training on the more morphologically complex classes in the S0 split (e.g., bookcase, chair), the resulting big-capacity prototype develops a superior capability to reconcile intra-class diversity. This enhanced capability subsequently leads to stronger segmentation performance when evaluated on the novel classes in the S1 split.

5.2.2 Results on ScanNet↩︎

Table ¿tbl:tab:scannet? shows that PENet’s superior performance is further validated on the more challenging ScanNet dataset. In the 2-way 1-shot setting, our method achieves an average mIoU of 68.86%, outperforming the previous SOTA, SDSimPoint, by +3.67%. The performance gains are consistent across all scenarios, including a notable +3.89% improvement in the 2-way 5-shot setting. These improvements on a complex dataset with a wider variety of classes underscore the robust generalization capability of our approach, which effectively leverages complementary features to better understand novel scenes.

5.3 Generalizability Validation↩︎

We evaluate PENet on more challenging tasks, including N-way settings with an increased number of novel classes and complex outdoor scenes.

5.3.1 Performance on Outdoor Scenes↩︎

To verify that our method’s effectiveness extends beyond constrained indoor environments, we conducted experiments on the SemanticPOSS outdoor dataset. Outdoor LiDAR scans present greater challenges due to their large scale and high object density. As shown in ¿tbl:tab:sota95semanticposs?, PENet demonstrates superior performance over the re-trained SOTA baselines across all settings. For instance, in the 2-way 5-shot setting, PENet achieves an average mIoU of 53.90%, outperforming the next best method by a margin of +5.86%. This performance advantage is even more pronounced in the difficult 3-way 1-shot setting, where our method surpasses the baseline by +6.69% mIoU. These results validate the generalization capabilities of our prototype expansion framework.

5.3.2 Performance on N-Way Settings↩︎

While most existing methods limit their evaluation to 2-way or 3-way tasks, we extend our analysis up to 6-way segmentation to test the model’s robustness. Figure 4 illustrates the performance of PENet on the S3DIS S1 split for both 1-shot and 5-shot scenarios. Notably, even in the highly challenging 6-way 1-shot setting, our model maintains a robust performance of over 50% mIoU, demonstrating that the expanded representational capacity of our method provides strong generalization.

5.4 Computation complexity↩︎

We provide an analysis of the computational complexity of PENet in comparison to prior SOTA methods in Table ¿tbl:tab:complexity?. The results show that while PENet achieves superior performance, its total parameter count is substantially larger. This increase is primarily attributed to the frozen pre-trained parameters of the DL. The DL is a critical component of our framework, responsible for extracting the rich generalizable priors necessary for prototype expansion, and this design choice incurs a higher parameter cost in exchange for performance gains. Crucially, the number of trainable parameters in PENet remains comparable to that of PAP3D, indicating an efficient meta-learning stage. For computational cost, PENet’s FLOPs are moderately higher than PAP3D’s but lower than the classic AttMPTI. In conclusion, our model balances performance with computational complexity.

5.5 Qualitative results↩︎

We present qualitative comparisons between PENet and the classic baseline, AttMPTI. Figure 5 illustrates several challenging 2-way 1-shot segmentation examples on S3DIS dataset. These visualizations demonstrate the enhanced segmentation capabilities of PENet, highlighting how to lead a more comprehensive understanding of novel targets. The results reveal that AttMPTI often struggles with incomplete segmentation, failing to capture the full extent of target objects, particularly when their surface features resemble the background. For instance, when segmenting objects like a ‘table’ or ‘board’, the baseline model frequently misclassifies the interior surfaces, resulting in hollow or fragmented predictions. This limitation stems from the small capacity of its prototypes, which cannot cover the full intra-class diversity of the object. In contrast, PENet consistently produces more complete and coherent segmentation masks. As highlighted by the colored circles, our big-capacity prototype effectively matches the entire object, demonstrating its enhanced ability to generalize from the support sample to the varied features of the query instance.

We further provide visualization results on the ScanNet dataset in Figure 6. Consistent with the results on S3DIS, PENet demonstrates superior segmentation performance with better object completeness and boundary accuracy than the baseline AttMPTI. To provide an objective evaluation, we also present a failure case in the last row of Figure 6. In this challenging scenario, the model fails to capture the complete structure of the target object. This limitation suggests that although the generative learner expands the prototype’s representational capacity, it may still fall short of fully covering the query space when facing novel categories with morphological complexity. Consequently, segmentation failures can occur in these boundary regions. However, despite this limitation, PENet still yields a more complete and identifiable prediction compared to the baseline, which almost completely misses the target.

5.6 Ablation Experiments↩︎

5.6.1 Impact of Different Modules↩︎

We conduct a component-wise ablation study to validate the effectiveness of each key module in PENet. Table ¿tbl:tab:ablation95modules? presents the results under the 2-way 1-shot setting, where uppercase suffixes from A to F denote different variants of our model.

We observe that the absence of either the Diffusion Learner (DL) or the Prototype Assimilation Module (PAM) leads to a marked decline in performance. Specifically, removing the DL (PENet-A) or the PAM (PENet-B) from the full model results in a sharp mIoU drop of 7.03% and 5.84%, respectively. This aligns with our analysis in Section I, confirming that the DL is critical for providing the generalizable features to resolve the prototype’s small-capacity limitation, while the PAM is essential for correcting the misalignment limitation by aligning prototypes with the query space. This mechanism is analogous to the distribution calibration strategy [@yang2021bridging], which expands the support distribution by borrowing statistics from base classes. Notably, PENet-E (excluding DL and PCM) also exhibits relatively lower performance. This decline stems from two critical factors: 1) the absence of the DL limits the prototype’s capacity to cover intra-class diversity; 2) without the PCM, the prototypes suffer from slight semantic drift during alignment. The optimal performance of 70.04% mIoU is achieved when all modules are integrated (PENet-F), validating the synergistic effect of our complete architecture.

5.6.2 Cross-Dataset Generalization Analysis↩︎

To investigate the universality of our proposed modules, we extended the ablation study to the outdoor SemanticPOSS dataset. The comparative results are reported in Table ¿tbl:tab:cross95dataset95ablation?.

The analysis reveals consistent performance gains from the DL across both domains, confirming that the geometric priors extracted by the generative encoder are generalizable to diverse scenes. However, we observe that while both modules enhance performance across domains, their relative impact varies due to domain-specific scenes. In the indoor S3DIS dataset, the DL plays a primary role, indicating that increasing prototype capacity is crucial for distinguishing complex indoor objects. In contrast, on the outdoor SemanticPOSS dataset, the PAM provides a more substantial gain of 39.61% mIoU compared to the DL of 37.36% mIoU. It indicates that outdoor scenes are unstructured and sparse, resulting in more severe misalignment within the query feature space. Therefore, the iterative push-pull alignment in PAM becomes the most critical factor in mitigating distribution shift, ensuring robust generalization in complex outdoor environments.

5.6.3 Ablation Study of PAM Structure↩︎

We further analyze the internal structure of our PAM to validate the contributions of its core components. The study includes ablating its symmetric push-pull blocks and varying the number of assimilation iterations, with results presented in Table 1 and Table 2, respectively.

Impact of Push-Pull Blocks. Table 1 shows the results of using only the push block, only the pull block, or the complete push-pull mechanism. The results indicate that both blocks are indispensable, as removing either one leads to a noticeable performance drop. Specifically, the pull block, which refines the intrinsic prototypes, appears more critical for performance on the S0 split, while the push block, which calibrates the diffusion prototypes, is more crucial for the S1 split. The full push-pull mechanism achieves the best result of 70.04% mIoU, demonstrating that the joint two complementary forces is necessary to effectively assimilate both prototype sets into the query distribution.

Impact of Iteration Number. We also investigate the impact of the iterative corrective force in PAM by varying the number of stacked push-pull blocks. The results in Table 2 show that a single iteration (\(M=1\)) yields a 2.82% mIoU drop, suggesting the corrective force is insufficient to bridge the gap between the support-derived prototypes and the query space. Conversely, increasing the iterations to \(M=3\) appears to cause over-correction, which also degrades performance by 1.97%. The optimal performance is achieved with \(M=2\), demonstrating that this configuration strikes the best balance for effective prototype assimilation.

5.6.4 Hyperparameter Sensitivity Analysis↩︎

To provide practical guidance for future deployment, we evaluate the sensitivity of PENet to three critical hyperparameters: the loss weight \(\lambda\), the number of neighbors \(K\), and the DL mask ratio. The results are summarized in Figure 7.

Impact of Loss Weight \(\lambda\). We first investigate the influence of the calibration loss weight \(\lambda\) (Eq. 19). As shown in Figure 7 (a) and (d), the performance initially improves as \(\lambda\) increases, peaking at \(\lambda=1.0\) (achieving 70.04% on S3DIS and 68.86% on ScanNet), before declining at \(\lambda=2.0\). This indicates an excessively small \(\lambda\) fails to prevent semantic drift, while an overly large \(\lambda\) imposes rigid constraints that hinder the prototype’s adaptability to novel query variations.

Impact of Neighbors \(K\). We also analyze the number of nearest neighbors \(K\) used in the DGCNN backbone of our IL. As illustrated in Figure 7 (b) and (e), the performance improves as \(K\) increases from 5 to 20, as a larger local receptive field captures richer geometric context. However, increasing \(K\) further to 40 leads to a performance drop (e.g., from 70.04% to 67.7% on S3DIS). This degradation suggests that an excessively large neighborhood may smooth out distinct local features, making \(K=20\) the optimal choice for our few-shot tasks.

Impact of Mask Ratio. Finally, we examine the mask ratio for the DL, varying it from 0% to 90%. As plotted in Figure 7 (c) and (f), the mIoU consistently rises with the ratio, reaching its zenith at 80%. This demonstrates that a high mask ratio is crucial for forcing the DL encoder to learn generalizable geometric priors from highly incomplete point clouds.

5.6.5 Visualization of Prototype Evolution↩︎

To intuitively understand how PENet improves prototype quality, we visualize the evolution of prototype distributions using t-SNE in Figure 8. In Figure 8 (a), the initial state displays both the intrinsic prototypes and the diffusion prototypes. The diffusion prototypes generated by the DL for the subsequent prototype expansion. However, at this stage, both prototypes are scattered and misaligned with the query feature distribution, reflecting the small-capacity and misalignment issues. Figure 8 (b) demonstrates the effect of the PAM. The both aligned prototypes have shifted towards the query clusters, proving that the iterative push-pull mechanism bridges the support-query gap. Finally, Figure 8 (c) shows the fused big-capacity prototype. Unlike the initial single-point prototypes, the final prototype occupies a central position within the query distribution, demonstrating an expanded representational capacity that can cover the intra-class diversity of the novel category.

6 Conclusion↩︎

In this paper, we propose PENet to address the limitations of small-capacity and misaligned prototypes in FS-PCS. PENet constructs big-capacity prototypes by employing a dual-stream architecture that extracts complementary representative features from an intrinsic learner and generalizable features from a repurposed diffusion learner. These resulting dual prototypes are then aligned by the PAM and regularized by the PCM to ensure robust alignment and semantic fidelity. Extensive experiments demonstrate that PENet establishes state-of-the-art on the S3DIS and ScanNet benchmarks. A main limitation of our work is that integrating the diffusion model into the framework introduces additional computational overhead, which could be a concern for large-scale applications or resource-constrained environments. Future work will focus on optimizing the framework’s efficiency, for instance, through model compression, quantization or by designing more lightweight generative feature extractors. Ultimately, our work validates the effectiveness of the prototype expansion approach and highlights the value of repurposing generative model components for discriminative few-shot learning tasks.

---

1. This work was supported in part by the National Key Research and Development Project of China (2020YFA0713503), the National Natural Science Foundation of China (61773330), the Natural Science Foundation of Hunan Province Project (2023JJ30598). (Corresponding author: Yan Zhou and Dongli Wang)↩︎

2. Qianguang Zhao and Richard Irampaye are with the School of Mathematics and Computational Science, Xiangtan University, Xiangtan 411105, China (e-mail: qg_zhao@smail.xtu.edu.cn; richarcive@gmail.com; hb.zhou@smail.xtu.edu.cn).↩︎

3. Dongli Wang and Yan Zhou are with the School of Automation and Electronics Information, Xiangtan University, Xiangtan 411105, China (e-mail: wangdl@xtu.edu.cn; yanzhou@xtu.edu.cn).↩︎

4. Jianxun Li is with the School of Automation, Shanghai Jiao Tong University, Shanghai 200232, China (e-mail: lijx@sjtu.edu.cn).↩︎