1 Introduction↩︎

Life-limiting congenital anomalies (LLCAs) are severe fetal conditions associated with high mortality [1]. Different anomalies require distinct management strategies. For example, severe open spina bifida may warrant prenatal fetal repair [2], complex congenital heart defects often require staged surgery after birth [3], and anencephaly is typically managed with palliative care or pregnancy termination [4]. Therefore, accurate prenatal differentiation among anomaly types is critical for appropriate clinical decision-making. Ultrasound (US) is the primary modality for screening and diagnosis [5]. In clinical practice, a typical prenatal US examination involves acquiring images from multiple anatomical planes. Accurate diagnosis requires identifying the anatomical plane represented by each image and determining which planes provide diagnostically relevant evidence for a specific anomaly (Fig. 1). However, this diagnostic process is time-consuming and cognitively demanding, as different anomalies rely on distinct subsets of anatomical planes. Moreover, US interpretation is inherently subjective, leading to variability in diagnostic consistency [6]. These challenges motivate automated methods for multi-plane prenatal diagnosis.

Developing automated systems that capture these diagnostic capabilities remains challenging. Existing approaches often follow staged pipelines, such as standard plane detection followed by anomaly classification [7], [8]. While effective under curated conditions, these methods require fine-grained image-level annotations, which are costly and difficult to obtain. Moreover, they assume that standard anatomical planes can be reliably localized, an assumption often violated in clinical practice due to anatomical variability and abnormal presentations [9], [10]. Multi-Instance Learning (MIL) provides a natural alternative by learning from unordered image sets using case-level supervision [11]. However, most MIL methods aggregate image features directly through attention or ranking mechanisms without explicitly modeling anatomical plane identity or disease-specific plane selection [12]–[19]. As a result, heterogeneous anatomical planes are mixed within a unified representation, limiting the model’s ability to capture plane-level semantics and disease-specific relevance.

Motivated by the fundamental capabilities required for multi-plane diagnosis, we propose AnomExpert, a prototype-driven framework for prenatal US anomaly diagnosis using case-level supervision. Instead of replicating a staged pipeline, AnomExpert explicitly models two key capabilities: anatomical plane identification and disease-specific plane selection. Specifically, learnable plane prototypes organize unordered images into latent representations corresponding to anatomical planes without requiring plane annotations. A disease-aware sparse selection mechanism then adaptively identifies diagnostically relevant planes for each anomaly. This design enables the model to differentiate anatomical planes under case-level supervision and to selectively integrate disease-relevant planes for diagnosis. Our contributions are threefold: (1) a prototype-driven framework for weakly supervised anatomical plane identification without plane annotations; (2) a disease-aware sparse aggregation mechanism for adaptive diagnostic plane selection; and (3) extensive experiments demonstrating consistent improvements over representative MIL methods for prenatal US anomaly classification.

2 Methodology↩︎

As illustrated in Fig. 2, AnomExpert models two key capabilities of prenatal US diagnosis: anatomical plane identification and disease-specific plane selection. The framework first organizes unordered images into plane-level representations using learnable plane prototypes. These plane representations serve as candidate anatomical planes within each case. A disease-aware selection mechanism then identifies diagnostically relevant planes for anomaly prediction. The entire framework is trained end-to-end using only case-level labels.

2.1 Plane Prototype Learning↩︎

A key challenge in multi-plane US diagnosis is identifying anatomical planes without explicit plane annotations. To approximate anatomical plane identification under case-level supervision, we introduce \(K\) learnable plane prototypes. Each prototype is intended to capture a distinct anatomical plane pattern. During training, image features are softly assigned to these prototypes, encouraging images with similar anatomical content to cluster around the same prototype.

Image embedding: Given an input image \(x_i\), we extract a feature representation using a backbone network \(\phi(\cdot)\) followed by a projection head \(g(\cdot)\): \[f_i = g(\phi(x_i)) \in \mathbb{R}^{256}.\]

Prototype assignment: We maintain a set of learnable plane prototypes \(P=\{p_k\in\mathbb{R}^{256}\}_{k=1}^{K}\). For a mini-batch containing \(N\) images (summed across multiple cases), we compute cosine similarities between all image features and prototypes to form a score matrix \(S\in\mathbb{R}^{N\times K}\). Soft assignments are obtained using Sinkhorn-balanced normalization [20]: \[A=\mathrm{Sinkhorn}\!\left(\exp(S/\epsilon_s)\right) \in\mathbb{R}^{N\times K},\] where \(\epsilon_s=0.05\) controls assignment sharpness. Sinkhorn balancing encourages diverse prototype utilization and helps prevent prototype collapse.

Prototype loss: To encourage consistent image-to-prototype assignments, we optimize a prototype assignment loss: \[\mathcal{L}_{proto} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{k=1}^{K} A_{i,k}\log \mathrm{softmax}(S_{i,:}/\tau)_k, \label{eq:proto}\tag{1}\] where \(\tau\) is a learnable temperature parameter. Through this learning process, images with similar anatomical characteristics are grouped under the same prototype, enabling anatomical plane identification without plane labels.

Plane representation: Given a case with \(L\) images, we aggregate image features into plane representations using the corresponding assignment weights: \[v_k= \frac{\sum_{i=1}^{L} A_{i,k} f_i}{\sum_{i=1}^{L} A_{i,k}+\delta}, \quad k=1,\dots,K,\] where \(\delta\) is a small constant for numerical stability. This yields a structured plane representation set \(V=\{v_k\in \mathbb{R}^{256}\}_{k=1}^{K}\) for the current case, which serves as the basis for downstream disease-specific plane selection.

2.2 Disease-aware Plane Selection↩︎

Unlike conventional MIL approaches that aggregate features from all images, our method performs selection at the plane-level. To this end, we introduce learnable disease queries to estimate the relevance of each anatomical plane to each disease.

Disease queries: We define a set of learnable disease queries: \[Q = \{q_c \in \mathbb{R}^{256}\}_{c=1}^{C},\] where \(C\) is the number of categories.

Relevance estimation: Given the structured plane representations \(V=\{v_k\in \mathbb{R}^{256}\}_{k=1}^{K}\) obtained in Sec. 2.1, we compute cosine similarity scores between each disease query and all plane representations: \[u_{c,k} = \mathrm{sim}(q_c, v_k).\] To capture disease-specific preferences over anatomical planes, we introduce a learnable bias matrix \(B \in \mathbb{R}^{C \times K}\), which models prior associations between diseases and planes. The final relevance scores are obtained as: \[\tilde{u}_{c,k} = u_{c,k} + B_{c,k}.\]

Sparse plane selection and aggregation: For each disease \(c\), we retain the top-\(k\) most relevant planes: \[\Omega_c = \mathrm{TopK}(\tilde{u}_{c,:}),\] where \(\Omega_c\) denotes the selected plane indices. We then compute disease-specific representations by aggregating the selected planes using normalized weights: \[z_c = \sum_{k \in \Omega_c} \alpha_{c,k} v_k, \qquad \alpha_{c,\Omega_c} = \mathrm{softmax}(\tilde{u}_{c,\Omega_c}/T),\] where \(T=0.07\) is a temperature constant controlling the sharpness of selection. This mechanism ensures that anomaly prediction is based on a subset of diagnostically relevant anatomical planes rather than all available images.

Case-level prediction: The disease-specific representations are concatenated and fed into a classifier to obtain the final prediction: \[\hat{y} = \mathrm{Classifier}([z_1,\dots,z_C]).\]

Training objective: The model is trained end-to-end using case-level category label \(y\). We use cross-entropy loss for anomaly classification: \[\mathcal{L}_{cls} = \mathrm{CE}(\hat{y}, y),\] and jointly optimize the prototype assignment loss defined in Equation 1 : \[\mathcal{L} = \mathcal{L}_{cls} + \lambda \mathcal{L}_{proto},\] where \(\lambda\) balances plane prototype learning and disease classification.

3 Experiments and Results↩︎

3.1 Datasets and Experimental Setup↩︎

3.1.1 Datasets↩︎

In this study, we collected a multi-center prenatal US dataset from 24 medical centers with approval from the institutional review board; informed consent was waived due to the retrospective study design. The dataset comprises 3,654 cases with 61,460 images, averaging 17 images per case (range: 5–94, std\(=\)​13). Each case was assigned a single primary diagnosis. All diagnoses were confirmed by experienced fetal US specialists according to established clinical criteria. The dataset includes eight types of LLCAs and normal controls: Single Arterial Trunk (SA, n\(=\)​399), Single Ventricle (SV, n\(=\)​341), Alobar Holoprosencephaly (AH, n\(=\)​478), Anencephaly (n\(=\)​643), Encephalocele (n\(=\)​180), Bilateral Renal Agenesis (BRA, n\(=\)​335), Open Spina Bifida (OSB, n\(=\)​348), Thanatophoric Dysplasia (TD, n\(=\)​410), and Normal (n=520). Gestational ages ranged from 10 to 39 weeks (mean ± std: 20.7 ± 12.0 weeks). The dataset was randomly partitioned at the case level into training (n\(=\)​2,558), validation (n\(=\)​366), and testing (n\(=\)​730) sets to prevent data leakage. The splits were stratified to preserve class distribution, and gestational age distributions were comparable across subsets.

3.1.2 Implementation Details↩︎

We evaluated our method using two ImageNet-pretrained backbone architectures, ViT-tiny (ViT-t) and ViT-small (ViT-s), to assess robustness across different model capacities. The number of plane prototypes was set to \(K=30\), and the disease-aware selection retained the top-\(k=4\) planes for each category. The weight of the prototype assignment loss was fixed at \(\lambda=0.1\). Models were trained for 60 epochs using Adam (initial learning rate \(1\times10^{-4}\), weight decay \(1\times10^{-5}\)), and the checkpoint with the best validation performance was used for testing. We used cosine annealing for learning rate decay and a batch size of 8 (case-level). Input images were randomly cropped and resized to \(224\times224\). Additional data augmentations included horizontal flipping, color jittering, affine transformations, and random grayscale conversion.

3.1.3 Evaluation Protocol↩︎

We compared our method with nine representative multi-instance learning (MIL) approaches, including AttentionMIL [12], MeanNetMIL [13], MaxNetMIL [13], TransMIL [14], DSMIL [15], AdditiveMIL [16], Mamb-a2MIL [17], SAMIL [18], and AAcls-MIL [19]. All competing methods were implemented using their official open-source code and evaluated under the same data split. Performance was evaluated using Accuracy, macro-averaged Precision, Recall, F1-score, and area under the ROC curve (AUC). Each method was trained and evaluated over five independent runs with different random seeds. We report the mean and standard deviation across runs.

3.2 Results↩︎

3.2.1 Quantitative Results↩︎

Table 1 presents the quantitative comparison between AnomExpert and nine representative MIL methods on the test set. Overall, the ViT-small variant of AnomExpert achieves the best performance across all evaluation metrics. Specifically, it achieves 86.9% accuracy, 84.9% precision, 83.9% recall, 84.2% F1-score, and 97.9% AUC. Compared with the best-performing baseline, MaxNetMIL, AnomExpert improves accuracy by 1.4 percentage points (86.9% vs. 85.5%) and F1-score by 2.1 percentage points (84.2% vs. 82.1%). Furthermore, AnomExpert demonstrates strong parameter efficiency. Using the ViT-tiny backbone, it achieves competitive performance (83.8% accuracy and 97.3% AUC) with only 5.60M parameters, fewer than all baseline models. These results confirm the effectiveness of AnomExpert in improving performance and parameter efficiency over existing MIL methods.

3.2.2 Qualitative Results↩︎

Fig. 3 (a) shows test images assigned to selected plane prototypes. Images grouped under the same prototype exhibit consistent anatomical patterns, indicating that the learned prototypes approximate anatomical plane identity without requiring plane annotations. Fig. 3 (b) shows the class-wise plane selection heatmap. The heatmap reflects the average selection probabilities of planes across all test cases. Certain planes are selected more frequently, suggesting that they correspond to diagnostically informative anatomical planes. Together, these visualizations demonstrate that AnomExpert organizes images into plane representations and performs selective aggregation for prediction.

3.2.3 Ablation Study↩︎

Table 2 shows that removing prototype learning (\(\lambda=0\)) reduces F1-score by 1.5 points, while replacing Sinkhorn with softmax leads to a larger drop to 80.8%. For disease-aware selection, dense aggregation and removing the disease bias matrix both degrade performance. These results indicate that both plane identification and disease-aware selection contribute to overall performance. Fig. 4 illustrates the influence of key hyperparameters. We evaluated five candidate values for each hyperparameter, and the best performance was achieved with \(K=30\), top-\(k=4\), and \(\lambda=0.1\).

4 Conclusion↩︎

We proposed AnomExpert, a prototype-driven framework for prenatal US anomaly diagnosis under case-level supervision. By modeling anatomical plane identification and disease-specific plane selection, the method integrates key diagnostic capabilities into MIL. Experiments show consistent improvements over representative MIL methods, and ablation studies support the contribution of each component. These results highlight the importance of combining anatomical plane modeling and disease-aware plane selection for improving weakly supervised multi-plane medical image analysis. Future research could explore enhancing the scalability of AnomExpert for more diverse datasets and integrating it with clinical workflows for real-time anomaly detection and diagnosis.

References↩︎