1 Introduction↩︎

Prenatal anomalies are functional or structural abnormalities that occur during fetal growth [1], [2]. These anomalies can seriously affect fetal survival and postnatal quality of life [3]. Therefore, accurate diagnosis is critical for pregnancy management. Ultrasound (US) is widely used for prenatal anomaly screening and diagnosis because it is non-invasive, radiation-free, and enables real-time imaging [4], [5]. However, the low incidence and high categorical diversity of prenatal anomalies lead to heterogeneous US appearances, making accurate identification difficult even for experienced sonographers. Consequently, prenatal anomaly screening becomes highly operator-dependent and susceptible to substantial inter-operator variability, underscoring the clinical value of automated methods for assisting screening and reducing diagnostic variability.

Learning-based method in prenatal US focuses on various tasks including plane detection [6], [7], biomarker measurement [8], and disease diagnosis [9], [10]. Zhang et al. [11] proposed a deep learning-based method to identify the standard fetal midsagittal plane in 3D US volumes for crown-rump length measurement. Lin et al. [12] developed a YOLOv3-based system to detect fetal brain standard planes and to classify abnormal images. Liang et al. [13] introduced medical knowledge to enhance anomaly recognition and achieved prenatal abdominal anomaly classification without relying on standard planes. These studies have shown strong performance in prenatal US tasks. However, most methods rely on large amounts of annotated data. Since prenatal anomalies have a low incidence and data access is constrained by patient privacy, constructing large-scale annotated datasets is costly. This hinders the further development of these methods.

Few-shot learning, e.g., few-shot object detection (FSOD), is considered an effective solution to alleviate data scarcity [14]–[17]. SMILe [16] introduces submodular mutual information as the learning objective to encourage well-separated feature clusters between base and novel classes, thereby reducing class confusion and catastrophic forgetting in FSOD. Zhu et al. [18] propose TRR-CCM, an US FSOD method that leverages circular channel Mamba and topological relationship reasoning to improve detection and localization accuracy. However, existing methods rely on large-scale annotated data for pretraining from the same modality. They also require fine-tuning on novel categories, which limits their practicality in resource-limited clinical settings.

Foundation vision models (FVMs), such as Contrastive Language-Image Pre-training (CLIP) [19] and Self-Distillation with No Labels (DINO) [20]–[22], have recently achieved strong representation learning ability through pretraining on large-scale datasets. Recent studies have explored applying these foundation models to training-free medical image segmentation [23]–[25], showing their potential in low-annotation or annotation-free settings. As shown in Fig. 1, although DINO is not trained on US data, it can still capture fetal structural features, demonstrating strong fine-grained representation ability.

In this study, we propose a training-free framework for prenatal anomaly classification and localization based on vision foundation encoder. Our framework does not require fine-tuning and does not rely on large amounts of annotated data, providing a practical solution for prenatal anomaly classification and localization. Our contributions are threefold. (1) To the best of our knowledge, this is the first study to perform multi-disease prenatal anomaly classification and localization in a training-free setting without relying on extensive annotated data. (2) We develop a unified framework that jointly incorporates memory bank construction, anomaly feature aggregation, and class-aware refinement to facilitate accurate and efficient localization of abnormal regions. (3) We validated our approach on a multi-center prenatal anomaly US dataset covering 9 categories, with 1,149 cases and 2,357 images. Experimental results demonstrated the effectiveness of our method.

2 Methodology↩︎

Fig. 2 illustrates the overall framework of the proposed training-free method, which relies on only a few annotated reference images for accurate anomaly classification and localization. Our model consists of three stages. In Stage 1, a memory bank was constructed to store class-wise and anomaly-related prototype representations. Then, Stage 2 introduced an anomaly feature aggregation strategy that selects candidate anomaly-related prototypes based on class-wise feature similarity and computes anomaly scores through anomaly-feature matching with anomaly-aware soft merging. Last, a class-aware refinement strategy was equipped to improve the anomaly category discrimination in Stage 3.

2.1 Multi-Granular Prototype Memory Bank Construction↩︎

A good vision foundation encoder \(E\) plays a vital role in extracting features for memory bank construction. Most recently, DINOv3 has shown powerful performance to provide visual features for multiple medical imaging tasks [26]. Inspired by this study, we employ DINOv3 [22] as the encoder \(E\) in our work.

Given a set of reference images \(\{I_i\}_{i=1}^{N}\) with anomaly bounding box annotations \(\{B_i = \{(x_1, y_1), (x_2, y_2)\}\}_{i=1}^{N}\), where \(N = C \times K\), \(C\) denotes the number of categories, \(K\) denotes the number of reference images per category, \((x_1, y_1)\) and \((x_2, y_2)\) denote the top-left and bottom-right coordinates of the bounding box, respectively. DINOv3 first partitions \(I_i\) into patches and processes them through \(L\) transformer layers, producing patch embeddings \(z^l, l\in L\) and a corresponding \([\mathrm{CLS}]^l\) token at each layer. \[E = \{layer^1, layer^2, \cdots, layer^L\},\] \[\{[\mathrm{CLS}]^l, z^l \} = layer^l(z^{l-1}),z^0 = I_i.\]

Class-wise Prototype Construction for Coarse-grained Recognition. For each reference image \(I_i\), we pass it through the encoder \(E\) and extract the \([\mathrm{CLS}]^L\) token from the last layer. After \(\ell_2\) normalization, the token is stored as a global class feature \(g_i \in \mathbb{R}^{D}\), where \(D\) denotes the feature dimension. Then, for each category \(c\in C\), class features from reference images belonging to the same category are averaged to form a class-wise prototype: \[\mathbf{p}_c = \frac{1}{K} \sum_{i \in \mathcal{I}_c}g_i,\] where \(\mathcal{I}_c\) denotes the index set of category \(c\). These class-wise prototypes are stored in the memory bank as class-wise representations.

Anomaly-Related Prototype Construction for Fine-grained Identification. For each reference image \(I_i\) with anomaly bounding box annotation \(B_i\), we pass \(I_i\) through the encoder \(E\) and extract \(\ell_2\)-normalized patch embeddings \(z^L\) from the last layer as dense feature map \(\mathbf{F_i} \in \mathbb{R}^{d \times h \times w}\). Due to the low resolution of the original feature maps, a high-resolution upsampling method, JAFAR [27], is adopted to enhance spatial details to obtain high-resolution feature map \(\mathbf{F_{hr}}_i\). For each image, we aggregate features within the annotated bounding box to form an anomaly-related prototype: \[\mathbf{q}_i = \frac{1}{|B_i|} \sum_{(x,y)\in B_i} \mathbf{F_{hr}}_i(x,y).\] These anomaly-related prototypes capture localized anomaly features and complement the class-wise prototypes stored in the memory bank, thereby forming a multi-granular prototype representation and improving anomaly localization.

2.2 Prototype-Guided Anomaly Feature Aggregation↩︎

During inference, given a query image \(I_q\), its class feature is extracted using the encoder \(E\) and matched with the class-wise prototypes to select top-\(M\) candidate categories \(\mathbf{p}_c\). The corresponding anomaly-related prototypes are then retrieved from the memory bank for anomaly aggregation.

Anomaly-Feature Matching. First, we retrieve candidate anomaly-related prototypes \(\{\mathbf{q}_{c,k}\}\). Then, we extract dense feature map \(\mathbf{F}_q\) from the query image and upsample to obtain high-resolution feature map \(\mathbf{F}_{hr}\). We compute the cosine similarity between each candidate prototype and \(\mathbf{F}_{hr}\) (see Equ. 1 ), resulting in one anomaly map per retrieved prototype. \[\label{equ:32cos95anomaly} \mathbf{A}_{c,k} = \cos\left(\mathbf{q}_{c,k}, \mathbf{F}_{hr}\right).\tag{1}\]

Anomaly-Aware Soft Merging. In US images, distinct anatomical structures can exhibit visually similar feature patterns, resulting in feature-level ambiguity that introduces noise and leads to false detections when features are matched independently. Meanwhile, we noticed that a local patch feature is often highly similar to its neighboring patches, which can be leveraged to strengthen region-level attention. Hence, for each anomaly map, our anomaly-aware soft merging strategy first selects the top-\(S\) high-similarity locations and computes their centroid \(\mathbf{f}_{c,k}\). Then, the cosine similarity between the centroid feature \(\mathbf{f}_{c,k}\) and \(\mathbf{F}_{hr}\) was evaluated to obtain the region-enhanced map: \[\mathbf{R}_{c,k} = \cos\left(\mathbf{f}_{c,k}, \mathbf{F}_{hr}\right),\] which can be used to softly merge the \(K\) anomaly maps: \[\mathbf{A}_c = \frac{1}{K}\sum_{k=1}^{K} \mathbf{R}_{c, k}\,\mathbf{A}_{c,k}\]

Anomaly Localization. Patches with anomaly values above the 95th percentile threshold in \(\mathbf{A}_c\) are retained to generate a binary mask. Then, the minimum enclosing rectangle is computed as the final anomaly regions \(\Omega\).

2.3 Class-Aware Refinement with Enhanced Memory Alignment↩︎

Based on the anomaly regions obtained in the previous step, this stage further refines their categories via a class-aware refinement strategy with the constructed memory bank. For each predicted anomaly region, we extract its region features from \(\mathbf{F}_{hr}\) and compute an average feature embedding: \[\mathbf{e}_i = \frac{1}{|\Omega_i|} \sum_{(x,y)\in \Omega_i} \mathbf{F}_{hr}(x,y),\] where \(\Omega_i\) denotes the bounding box of the \(i\)-th anomaly region. We then compute the cosine similarity between \(\mathbf{e}_i\) and the retrieved anomaly-related prototypes, and select the class with the highest similarity as the predicted label, with the similarity value serving as the confidence score.

3 Experiment Results↩︎

Materials and Implementation Details. With local Institutional Review Board approval (No. K-2023-067-H01), we collected a multi-center prenatal US dataset comprising 1,149 cases and 2,357 images, encompassing 9 categories spanning 3 anatomical regions (brain, heart, and abdomen). All images were annotated by sonographers with classification labels and bounding boxes: the anomaly class is annotated with the anomaly region, and the normal class is annotated with the region of interest. Specifically, the fetal brain images contain four types, including choroid plexus cyst (CPC, 122 cases / 226 images), ventriculomegaly (VM, 164 cases / 312 images), holoprosencephaly (HPE, 104 cases / 238 images), and normal brain (B, 183 cases / 329 images). The heart-related category contain single ventricle (SV, 29 cases / 197 images). The abdomen-related categories contain duodenal atresia (DA, 178 cases / 255 images), multicystic dysplastic kidney (MCDK, 77 cases / 261 images), abdominal wall defect (AWD, 51 cases / 181 images), and normal abdomen (A, 241 cases / 358 images). The dataset was split at the case level by sampling cases without replacement. Once the accumulated number of images exceeded 32, 32 images were randomly selected to form a fold, and the corresponding cases were removed from the pool. The process was repeated to construct five reference folds (Splits 1-5), each containing 32 images per category (1,440 images in total), while the remaining cases were used for testing (917 images in total). Within each reference set, we randomly selected 4, 8, 16, or all 32 images per category to construct the 4-shot, 8-shot, 16-shot, and 32-shot settings, respectively.

Our approach was implemented in PyTorch using an NVIDIA RTX 4090 GPU (24GB). Images were resized to \(512 \times 512\). We employed a pretrained DINOv3-Small backbone, producing \(64 \times 64\) feature map. The JAFAR upsampling method was applied to upsample features to \(256 \times 256\) without additional pre-training. For hyperparameters, we set \(M=2\) and \(S=16\). Performance was evaluated using mAP@0.5 with TorchMetrics. Codes can be found in GitHub.

Quantitative Analysis. We compare our approach with official implementations of FSOD methods, including DeFRCN [14], DiGeo [15], SMILe-FSOD [16], and TRR-CCM [18], which are fine-tuned on our reference sets, as well as training-free methods, including ProtoSAM [28] and MAUP [23]. Table 1 shows the results. Our method outperforms the comparative methods across most categories. Compared with the second-best method, our approach achieves an 11.41% higher mAP. Fig. [fig:few] shows the mAP performance with error estimation across splits. As the number of reference images increases, our method exhibits steady performance gains, demonstrating strong scalability and outperforming all compared methods overall. Table ¿tbl:tab:ablation? shows the ablation results. High-resolution (HR) upsampling improves mAP by 2.72%. Anomaly-aware soft merging (ASM) improves mAP by 7.81%. Class-aware refinement (CR) improves mAP by 5.47%. Using all components improves mAP by 15.49%. Fig. [tbl:fig:visual] shows the ablation results on different categories. It can be observed that each proposed strategy provides systematic performance improvements across all categories.

Qualitative Analysis. We visualize several typical cases in Fig. 3. For each query image (a)-(h), we show three candidate prototypes. We observe that, after aggregating the similarity maps from these candidate prototypes, the resulting anomaly map is well concentrated on the fetal region. The mask and bounding box obtained using the 95th-percentile threshold accurately localize the anomaly region. These results indicate that, even without training and using only a few reference images, our approach can accurately localize anomaly regions and correctly classify their categories, highlighting its effectiveness.

4 Conclusion↩︎

In this paper, we proposed a novel training-free framework for multi-class prenatal anomaly classification and localization. Our method requires only a few reference images, addressing the scarcity of abnormal data and eliminating the need for retraining. The proposed framework consists of three stages: (1) memory bank construction to store class-level and anomaly-related representations from reference images. (2) anomaly feature aggregation to localize abnormal regions via anomaly-aware soft merging. (3) class-aware refinement to improve anomaly category discrimination. We evaluated our method on a multi-center dataset using only a few reference images per class. The results demonstrated the effectiveness of our approach. In future work, we will extend our approach to additional anomaly types and perform comprehensive clinical validation.

References↩︎