Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentation
May 16, 2025
Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentation
jiaxiang.guo@tju.edu.cn
1 Introduction↩︎
Medical image segmentation is a vital component of computer-aided diagnosis [1]–[4]. A key challenge in this domain is the need for precise annotations, which must be performed by experienced experts. This manual annotation process is both laborious and time-consuming, creating a significant bottleneck for developing effective methods [5]. To address this limitation, semi-supervised medical image segmentation (SSMIS) has emerged as a promising and efficient approach. It leverages large volumes of unlabeled data to augment a limited set of labeled samples, reducing the reliance on exhaustive manual annotation [6]–[9].
Recently, managing the learning pace [10] of model training has garnered attention in semi-supervised learning, whether through explicit or implicit mechanisms. For instance, in the context of open-world semi-supervised learning, methods have been introduced to explicitly balance learning between seen and unseen classes. These techniques include using an adaptive threshold with distribution alignment [11] or employing an adaptive synchronizing margin loss and confidence-based clustering [12].
In the context of SSMIS, techniques implicitly incorporate the principles of curriculum learning [13]. For instance, using a confidence threshold on weakly perturbed samples to filter noisy pseudo-labels [14] inherently controls the learning pace, progressing from high-confidence to more challenging regions. Similarly, the approach proposed in [15] leverages confidence to generate samples tailored to different training stages by including or excluding low-confidence areas, which implicitly establishes a confidence-based curriculum.
The mean teacher (MT) strategy, widely recognized for enforcing consistency regularization [16]–[21], also implicitly incorporates a curriculum learning mechanism to some extent, as will be analyzed later. This underlying principle also extends to related methods like dual teacher [21] and dual student [22], although it is rarely acknowledged in their design.
In the MT framework, two models with identical architecture are instantiated as the teacher and student. The student is trained using both human-annotated labels and pseudo-labels generated by the teacher. The teacher model is updated via an exponential moving average (EMA) of the student’s weights, \[\label{eqn:EMA} \Theta^T_t= \omega \Theta^T_{t-1}+(1-\omega)\Theta^S_t,\tag{1}\] where \(\Theta^S_t\) is the weight of the student model at step \(t\), and \(\Theta^T_{t-1}\) is the weight of the teacher model at step \(t-1\). This strategy has demonstrated strong performance in SSMIS and has inspired numerous successful extensions [15], [23], [24].
Applying MT strategy on the supervised learning implicitly introduces a mechanism similar to self-paced learning [10] or curriculum learning [13]. Let \(\boldsymbol{y}_{\mathrm{gt}}\) denote the ground truth segmentation label of a sample \(\boldsymbol{x}\), \(\hat{\boldsymbol{y}}\) the student output, and \(\hat{\boldsymbol{y}}_{\mathrm{t}}\) the teacher-generated pseudo-label. The training objective for the student is \[\label{loss:seperate} \mathcal{L}=\mathcal{L}(\hat{\boldsymbol{y}},\boldsymbol{y}_{\mathrm{gt}})+ \lambda\mathcal{L}(\hat{\boldsymbol{y}},\hat{\boldsymbol{y}}_{\mathrm{t}}),\tag{2}\] which can be reformulated based on whether the teacher’s prediction agrees with the ground truth, i.e., \(\hat{\boldsymbol{y}}_{\mathrm{t}} = \boldsymbol{y}_{\mathrm{gt}}\) \[\begin{align} \mathcal{L} &= \mathbb{1}({\hat{\boldsymbol{y}}_{\mathrm{t}} = \boldsymbol{y}_{\mathrm{gt}}}) (1+\lambda) \mathcal{L}(\hat{\boldsymbol{y}},\boldsymbol{y}_{\mathrm{gt}}) \notag \\ &+ \mathbb{1}({\hat{\boldsymbol{y}}_{\mathrm{t}} \neq \boldsymbol{y}_{\mathrm{gt}}}) (\mathcal{L}(\hat{\boldsymbol{y}},\boldsymbol{y}_{\mathrm{gt}})+ \lambda\mathcal{L}(\hat{\boldsymbol{y}},\hat{\boldsymbol{y}}_{\mathrm{t}})), \label{eqn:mt1} \end{align}\tag{3}\] where \(\mathbb{1}(\cdot)\) is the indicator function, applied pixel-wise, that activates the corresponding term based on whether \(\hat{\boldsymbol{y}}_{\mathrm{t}}\) matches ground truth \(\boldsymbol{y}_{\mathrm{gt}}\). When the teacher label (temporally averaged signal) consensus with the ground truth, the loss simplifies to \[\label{eqn:mt2} \mathcal{L}= (1+\lambda)\mathcal{L}(\hat{\boldsymbol{y}},\boldsymbol{y}_{\mathrm{gt}}),\tag{4}\] indicating reinforcement of the ground truth signal. When the teacher label disagrees with the ground truth, the loss becomes \[\label{eqn:mt3} \mathcal{L}=\mathcal{L}(\hat{\boldsymbol{y}},\boldsymbol{y}_{\mathrm{gt}})+ \lambda\mathcal{L}(\hat{\boldsymbol{y}},\hat{\boldsymbol{y}}_{\mathrm{t}}),\tag{5}\] which reflects competing gradients from two sources, balancing the optimization of \(\hat{\boldsymbol{y}}\) in different directions.
Therefore, as illustrated in [eqn:mt1,eqn:mt2,eqn:mt3], signals from both the temporally averaged teacher and the ground truth label jointly control the learning pace, when the MT strategy is applied to supervised learning. During the early epochs of training, the teacher model tends to make confident predictions on easier regions, causing the optimization target 2 to focus more on these simpler tasks. As training progresses, the agreement between the teacher model and the ground truth gradually extends to more difficult regions, thereby guiding the student model to tackle increasingly challenging examples.
| ACDC | PROMISE12 | |||||
| DSC\(\uparrow\) | Jaccard\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) | DSC\(\uparrow\) | ASD\(\downarrow\) | |
| U-Net (w/o MT) | 91.44 | 84.59 | 4.30 | 0.99 | 84.76 | 1.58 |
| U-Net (w/ MT) | 91.70 | 85.02 | 1.15 | 0.29 | 85.22 | 1.55 |
To verify that the MT strategy introduces a self-paced learning effect, we conducted experiments in a fully supervised setting on the ACDC and PROMISE12 datasets. As shown in 1, applying the MT strategy in this supervised-only context improves segmentation performance. Furthermore, 1 visualizes the agreement between teacher-generated and ground truth labels across training iterations. The regions of agreement progressively expand from the object’s center to its more challenging borders as training advances. This observation partly supports the hypothesis that the MT strategy implicitly controls the learning pace, guiding the model from easier to more difficult regions.
However, when applied to unlabeled data, the standard MT strategy lacks this intrinsic self-paced mechanism due to the absence of a ground truth signal. To introduce a robust self-paced learning framework to SSMIS, this work utilizes two distinct signals to control the learning pace, the aforementioned temporally averaged teacher signal and a cross-architectural signal. The underlying hypothesis is that early in training, these two diverse signals are more likely to achieve consensus on simpler regions, which could be used to control the learning pace. Notably, this consensus-based mechanism is distinct from approaches that rely solely on prediction confidence to guide the learning pace.
Specifically, a framework of dual teacher student learning (DTSL) is proposed, which employs two teacher-student groups with distinct architectures. Within each group, the teacher and student share the same architecture, and the teacher model maintains an EMA of its corresponding student’s weights. For each student, both its in-group teacher (temporally averaged signal) and the cross-group student (cross-architectural signal) regulate the pace of learning by using the consensus of these two signals. To implement this, a consensus label generator (CLG) is proposed to create pseudo-labels from regions where these two signals agree. Agreement is quantified by measuring the Jensen-Shannon (JS) divergence [25] between the outputs of the two different models.
The main contributions of this work are as follows:
The curriculum learning effect of the MT strategy is first analyzed in the supervised learning context and then extended to the semi-supervised learning framework.
Different from confidence-based learning pace control, this work is the first to implement self-paced learning in SSMIS by leveraging two distinct signals, which provides greater flexibility in controlling the pace of learning and enhances model performance.
In the context of SSMIS, outputs from cross-architectural models were first explicitly leveraged for generating pseudo labels.
Extensive experiments across various methods and datasets demonstrate that the proposed DTSL framework outperforms state-of-the-art (SOTA) approaches in both supervised and semi-supervised medical image segmentation. Notably, applying DTSL in a semi-supervised setting outperforms standard supervised learning on three of the four datasets. To the best of our knowledge, this is the first such achievement. Ablation studies and hyperparameter optimization further validate the effectiveness of the proposed method.
2 Related Works↩︎
2.1 Medical image segmentation↩︎
Contemporary medical image segmentation aims to delineate regions of interest across various imaging modalities, such as magnetic resonance imaging (MRI) and computed tomography (CT) [26]–[29]. Convolutional Neural Networks (CNNs) have been foundational in this domain, with architectures like UNet [1], VNet [30], ResUNet [31], etc., achieving remarkable results. However, a major limitation of these supervised methods is their reliance on large quantities of precise, high-quality annotations—a process that demands substantial time and expertise [5].
2.2 Semi-supervised learning↩︎
Semi-supervised learning aims to train models using a combination of labeled and unlabeled data [9], [24]. The main approaches include pseudo-labeling, consistency regularization, and entropy minimization. Pseudo-labeling methods [32], [33] first perform supervised learning on labeled data, then generate pseudo-labels for unlabeled data, and finally refine these pseudo-labels using strategies such as random propagation. Consistency regularization methods [34]–[36] enforce the model to produce consistent predictions on unlabeled data under various perturbations. Entropy minimization methods [37], [38] aim to minimize the output entropy of the model for unlabeled samples.
A widely adopted strategy in semi-supervised learning is the MT framework [16], which encourages the student model to produce predictions consistent with those of the teacher model. This approach has inspired numerous successful algorithms [17], [18], [21], [22], [37]. Many of these MT strategy-based methods aims to generate high quality pseudo-labels. For example, Tri-U-MT[39] introduces a triple uncertainty-guided MT framework to improve prediction performance.
Similarly, co-learning has also proven to be an effective approach in SSMIS [40], [41]. These methods typically employ two sub-networks, often with distinct architectures, to generate pseudo-labels for each other. This mutual supervision process enables the models to iteratively learn more representative features from unlabeled data. For instance, CrossTeaching [42] leverages two architecturally different networks to perform cross-pseudo supervision. However, a key challenge for these frameworks is maintaining training stability. Without the grounding influence of a temporally-averaged teacher model, the pseudo-labels generated by the concurrently training networks can be noisy, potentially leading to error accumulation.
2.3 Self-paced learning↩︎
Self-paced learning [10] is a learning methodology inspired by cognitive science, which cautiously and adaptively prioritizes learning from simple and reliable examples, and gradually transitions to more difficult ones. Common self-paced learning methods [43], [44] are capable of autonomously evaluating the difficulty of samples, making the learning process dynamic and optimizable. Self-paced learning has been successfully applied to a variety of tasks, such as saliency detection [45] and object tracking[46].
This self-paced learning principle has also been applied in semi-supervised segmentation tasks, guiding models to learn by selecting sample regions from simple to complex. Methods such as FixMatch [47], UniMatch [14], CWMS [48],ABD [15], PriorsMatch [49], etc exemplify this approach. In these works, high-confidence predictions are retained as pseudo-labels, which implicitly controls the learning pace via a confidence-based curriculum. Although these methods have achieved good results, their reliance on prediction confidence as the sole proxy for sample simplicity is a limitation, as a model’s self-reported confidence can often be miscalibrated and unreliable.
3 Method↩︎
3.1 Problem setting↩︎
The training set \(D\) for semi-supervised segmentation consists of a small labeled subset \(D^{\ell} = \{(\boldsymbol{x}_i^{\ell}, \boldsymbol{y}_i^{\ell})\}_{i=1}^{N}\) and a large unlabeled subset \(D^u=\{\boldsymbol{x}_i^u\}_{i=N+1}^{M+N}\), where \(N \ll M\). For the labeled data, each image \(\boldsymbol{x}_i^{\ell}\) in \(D^{\ell}\) has a corresponding pixel-wise annotation \(\boldsymbol{y}_i^{\ell} \in \{0,1,\dots,K-1\}^{H\times W}\), where \(K\) is the total number of classes and class \(0\) usually denotes the background.
The objective of semi-supervised learning is to leverage both the labeled data \(D^{\ell}\) and the unlabeled data \(D^u\) to train a model that generalizes effectively, making accurate predictions on the unseen test set \(D^{t} = \{(\boldsymbol{x}_i^{t}, \boldsymbol{y}_i^{t})\}\).
3.2 Dual teacher student learning framework↩︎
As illustrated in 2, the proposed DTSL framework employs two teacher-student groups with distinct architectures. Within each group, the teacher and student models share the same architecture.
During training, the student models are updated via gradient descent on a composite loss function. Concurrently, the parameters of each teacher model are updated using the EMA strategy defined in 1 , based on their respective student’s weights.
The optimization objective for a labeled sample, \(\mathcal{L}^{\ell}\), is a composite loss consisting of two components, \[\label{eqn:totallosssupervised} \mathcal{L}^{\ell}=\mathcal{L}_{\mathrm{sup}}+ \mathcal{L}_{\mathrm{pace}},\tag{6}\] where \(\mathcal{L}_{\mathrm{sup}}\) is the conventional supervised loss and \(\mathcal{L}_{\mathrm{pace}}\) is the proposed loss term to regulate the learning pace. The supervised component \(\mathcal{L}_{\mathrm{sup}}^i\) for \(\mathrm{student}_i\) from group \(i, (i=0,1)\) is defined as a standard mixed loss of cross-entropy and Dice loss [50], \[\mathcal{L}^i_{\mathrm{sup}} = \frac{1}{2}( \mathcal{L}_{\mathrm{CE}}(\hat{\boldsymbol{y}}^i_s, \boldsymbol{y}_{\mathrm{gt}})+\mathcal{L}_{\mathrm{dice}}(\hat{\boldsymbol{y}}^i_s,\boldsymbol{y}_{\mathrm{gt}})),\] where \(\hat{\boldsymbol{y}}^i_s\) is the \(i\)-th student’s prediction and \(\boldsymbol{y}_{\mathrm{gt}}\) is the ground truth.
For unlabeled samples, the optimization objective \(\mathcal{L}_{\mathrm{pace}}\) consists only of the pace regulator \[\label{eqn:totallossunsupervised} \mathcal{L}^{u}=\mathcal{L}_{\mathrm{pace}}.\tag{7}\] In this context, \(\mathcal{L}_{\mathrm{pace}}\) serves a dual purpose: it controls the learning pace while simultaneously functioning as the mechanism for pseudo-label propagation.
3.2.1 Consensus label generator (CLG)↩︎
As mentioned in 1, the MT strategy, when applied in a fully supervised setting, introduces an implicit learning pace. This pace is regulated by the agreement between the temporally-averaged teacher model and the ground truth labels.
In this work, this self-paced concept is explicitly adapted for semi-supervised learning. We extend the pace-regulating mechanism to be governed by the agreement between two distinct signals, one from a temporally-averaged model and another from a cross-architectural model. This is implemented by the proposed CLG.
As illustrated in 2, to generate a pseudo-label for a given student, the inputs of CLG are sourced from its in-group teacher and the cross-group student. The consistency between these two predictions is used to create the pseudo-label for that student.
Let \(\boldsymbol{o}_1\) and \(\boldsymbol{o}_2\) denote the outputs of the two different models, representing the pixel-wise predicted categorical distributions. The JS divergence [25] is computed between \(\boldsymbol{o}_1\) and \(\boldsymbol{o}_2\) pixel-wisely, \[\mathrm{JS}(\boldsymbol{o}_1,\boldsymbol{o}_2) = \frac{\mathrm{KL}(\boldsymbol{o}_1||\boldsymbol{m}) + \mathrm{KL}(\boldsymbol{o}_2||\boldsymbol{m})}{2},\] where \(\boldsymbol{m}=\frac{\boldsymbol{o}_1+\boldsymbol{o}_2}{2}\) is the average distribution. JS divergence is a symmetric measure of similarity between two distributions, derived from the Kullback–Leibler (KL) divergence [51], bounded in \([0, 1]\) when using a base-2 logarithm.
Next, a threshold \(\kappa\in [0,1]\) is applied to the resulting JS divergence map to create binary masks for consistent (easy) and inconsistent (difficult) regions, \[\begin{align} \label{eqn:kappa} \mathrm{Mask}_{\mathrm{cons}}&=\mathbb{1}(\mathrm{JS}(\boldsymbol{o}_1,\boldsymbol{o}_2)<\kappa);\\ \mathrm{Mask}_{\mathrm{diff}}&=\mathbb{1}(\mathrm{JS}(\boldsymbol{o}_1,\boldsymbol{o}_2)\geq \kappa), \end{align}\tag{8}\] where \(\mathbb{1}(\cdot)\) denotes the the pixel-wise indicator function that returns \(1\) if the condition is true and \(0\) otherwise.
The CLG then creates the final pseudo-label. It uses the \(\mathop{\mathrm{argmax}}\limits\) of the average distribution of \(\boldsymbol{o}_1\) and \(\boldsymbol{o}_2\) for consistent regions (easy regions), and assigns the background class to all inconsistent regions (difficult regions), \[\begin{align} \label{eqn:CLG} &\mathrm{CLG}(\boldsymbol{o}_1,\boldsymbol{o}_2) \nonumber \\ =&\mathop{\mathrm{argmax}}\limits\left(\frac{\boldsymbol{o}_1+\boldsymbol{o}_2}{2}\odot \mathrm{Mask}_{\mathrm{cons}} + \mathrm{onehot}(\boldsymbol{0}) \odot \mathrm{Mask}_{\mathrm{diff}}\right), \end{align}\tag{9}\] where \(\mathrm{onehot}(\boldsymbol{0})\) represents the one-hot encoding of the background class.
This approach generates pseudo-labels in a self-paced manner, establishing a curriculum that begins with easy regions and gradually incorporates more difficult ones as training progresses. Early in training, the models are more likely to achieve consensus on simple regions. As the models become more accurate, their agreement extends to more challenging areas, allowing these harder regions to be progressively incorporated into the learning process.
3.2.2 Pace regulator: \(\mathcal{L}_{\mathrm{pace}}\)↩︎
The pace regulator \(\mathcal{L}_{\mathrm{pace}}\) consists of two components, a conventional semi-supervised loss \(\mathcal{L}_{\mathrm{semi}}\) that applies pseudo-labels to consistent (easy) regions, and a uniform regularization loss (URL) \(\mathcal{L}_{\mathrm{URL}}\) applied to inconsistent (difficult) regions. The URL encourages the model to produce high-entropy (uncertain) predictions in areas where the guiding signals disagree.
As part of \(\mathcal{L}_{\mathrm{pace}}\), we define a semi-supervised loss, \(\mathcal{L}_{\mathrm{semi}}\), based on the pseudo-labels generated by the CLG. This loss aligns with standard pseudo-labeling in semi-supervised learning frameworks. For \(\mathrm{student}_0\) from \(0\)-th group, the loss is computed against a pseudo-label generated from the consensus of its in-group teacher (\(\mathrm{teacher}_0\)) and the cross-group student (\(\mathrm{student}_1\)). Conversely, \(\mathrm{student}_1\) is supervised by the consensus of \(\mathrm{teacher}_1\) and \(\mathrm{student}_0\): \[\begin{align} \mathcal{L}_{\mathrm{semi}}^0 &= \mathcal{L}_{\mathrm{dice}}(\hat{\boldsymbol{y}}^0_s, \mathrm{CLG}(\hat{\boldsymbol{y}}^1_s,\hat{\boldsymbol{y}}^0_t)), \\ \mathcal{L}_{\mathrm{semi}}^1 &= \mathcal{L}_{\mathrm{dice}}(\hat{\boldsymbol{y}}^1_s, \mathrm{CLG}(\hat{\boldsymbol{y}}^0_s,\hat{\boldsymbol{y}}^1_t)), \end{align}\] where \(\hat{\boldsymbol{y}}^i_s\) and \(\hat{\boldsymbol{y}}^i_t\) denote the predictions of the student and teacher from group \(i \in \{0,1\}\), respectively.
Motivated by [52], an additional loss term is applied to handle inconsistent regions between a student and its cross-group teacher. This term, referred to as the URL, encourages the model to reconsider its discrepant predictions by increasing the prediction’s entropy, which is achieved by comparing the model’s output distribution to a discrete uniform distribution using KL divergence.
Specifically, taking \(\mathrm{student}_0\) as an example, the inconsistent regions is firstly identified by comparing its prediction \(\hat{\boldsymbol{y}}^0_s\) with the cross-groupe teacher’s prediction \(\hat{\boldsymbol{y}}^1_t\), \[\mathrm{Mask}^0_{\mathrm{diff}}=\mathbb{1}(\mathrm{JS}(\hat{\boldsymbol{y}}^0_s,\hat{\boldsymbol{y}}^1_t)\geq \kappa),\] where \(\mathbb{1}(\cdot)\) is the pixel-wise indicator function. The URL for \(\mathrm{student}_0\) is then computed only on these inconsistent regions, \[\label{eqn:URL} \mathcal{L}^0_{\mathrm{URL}} = \mathrm{Mask}^0_{\mathrm{diff}} \odot \mathrm{KL}(\hat{\boldsymbol{y}}^0_s, \mathrm{Uniform}),\tag{10}\] where \(\mathrm{Uniform}\) denotes the discrete uniform distribution over all classes.
Finally, the complete pace regulator \(\mathcal{L}_{\mathrm{pace}}\) is defined as the weighted sum of the semi-supervised loss \(\mathcal{L}_{\mathrm{semi}}\) and the uniform regularization loss \(\mathcal{L}_{\mathrm{URL}}\), \[\begin{align} \label{eqn:pace} \mathcal{L}_{\mathrm{pace}} &= \alpha \mathcal{L}_{\mathrm{semi}} + \beta \mathcal{L}_{\mathrm{URL}} \\\notag & = \alpha (\mathcal{L}^0_{\mathrm{semi}}+\mathcal{L}^1_{\mathrm{semi}}) + \beta (\mathcal{L}^0_{\mathrm{URL}}+\mathcal{L}^1_{\mathrm{URL}}), \end{align}\tag{11}\] where \(\alpha\) and \(\beta\) are hyperparameters that balance the contribution of the two corresponding terms.
In summary, by optimizing the composite losses defined in 6 and 7 , the proposed DTSL framework is trained in a self-paced manner, regulated by agreements not only between temporally averaged models but also between cross-architectural models.
4 Experiments↩︎
4.1 Datasets, metrics, and experimental settings↩︎
Datasets. The DTSL is evaluated on four benchmark datasets. These include two 2D datasets, ACDC [53] and PROMISE12 [54], and two 3D datasets, LA [55] and Pancreas-NIH [56].
The ACDC dataset contains multi-slice 2D cine cardiac MRI images from \(100\) patients, covering four classes: background, right ventricle, left ventricle, and myocardium. It is split by patient into \(70\) for training, \(20\) for validation, and \(10\) for testing. The PROMISE12 dataset, from the MICCAI 2012 challenge, contains \(50\) prostate MRI images from \(50\) patients and is split into \(35\) for training, \(5\) for validation, and \(10\) for testing. The LA dataset comprises \(100\) 3D gadolinium-enhanced MRI (GE-MRI) scans, split into \(80\) for training and \(20\) for testing. Finally, the Pancreas-NIH dataset is a 3D contrast-enhanced abdominal CT dataset of \(80\) volumes, which are split into 60 for training and \(20\) for testing.
We adopt the data augmentation strategy from BCP [24]. This approach defines two distinct levels of transformation: weak augmentation, which consists of standard geometric transforms (random rotation and random flips), and strong augmentation, which applies color jitter and Gaussian blur in addition to the weak augmentations. These two augmentation levels are applied to the inputs of all teacher-student models.
Metrics. Following previous works [15], [24], we adopt four popular evaluation metrics for the ACDC, LA, and Pancreas-NIH datasets:
the dice similarity coefficient (DSC) [57], [58], which measures the volume overlap, calculated as twice the intersection divided by the sum of the sizes of the predicted and ground truth segmentations;
the Jaccard index [59], which quantifies similarity by dividing the intersection by the union of the predicted and ground truth segmentations;
the 95% Hausdorff distance (95HD) [60], which measures the 95th percentile of the maximum distance between the predicted and ground truth surfaces, providing robustness to outliers;
the average surface distance (ASD) [61], which calculates the average distance between the boundaries of the predicted and ground truth segmentations.
For the PROMISE12 dataset, we adhere to its standard evaluation protocol by using DSC and ASD.
Experimental settings. As aforementioned, the two teacher-student groups employ distinct backbone architectures: U-Net [1] and ResU-Net [31] for the 2D datasets (ACDC, PROMISE12), and V-Net [30] and ResV-Net [40] for the 3D datasets (LA, Pancreas-NIH).
All experiments were conducted in PyTorch on an NVIDIA RTX 3090 GPU. We trained the 2D models (U-Net and ResU-Net) for 30k iterations and the 3D models (V-Net and ResV-Net) for 7.5k iterations. The Adam optimizer was used with an initial learning rate of \(\eta_0 = 1\times 10^{-3}\), which was decayed using a poly learning rate scheduler \(\eta=\eta_0\times(1- \mathrm{iter}/\mathrm{max\_iter})^{0.9}\). During training, 2D inputs were randomly cropped to \(256\times 256\), while 3D inputs were cropped to \(112\times 112 \times 80\) for the LA dataset and \(96\times 96 \times 96\) for the Pancreas-NIH dataset.
4.2 Performance gain is primarily attributed to the DTSL framework rather than unlabeled data↩︎
The DTSL framework can be applied in supervised learning using 6 and in semi-supervised learning by [eqn:totallosssupervised,eqn:totallossunsupervised]. To disentangle the performance gains from the framework itself versus those from incorporating unlabeled data, we conducted a comparative analysis across five experimental settings
vanilla supervised learning with limited labeled data;
supervised DTSL with limited labeled data;
semi-supervised DTSL with limited labeled data and complementary unlabeled data;
vanilla supervised learning with full labeled data;
supervised DTSL with full labeled data.
The results of this analysis are presented in the lower portion of each results [tab:ACDC,tab:Promise12,tab:LA,tab:Pancreas-NIH] for the four benchmark datasets.
These results suggested that, when comparing the performance of vanilla supervised learning, supervised DTSL, and semi-supervised DTSL under limited labeled data, the majority of the performance gain is observed in the transition from vanilla supervised learning to supervised DTSL, rather than from supervised DTSL to semi-supervised DTSL. This strongly suggests that the performance improvement is primarily driven by the self-paced learning strategy rather than the incorporation of additional unlabeled data.
Notably, the semi-supervised DTSL, trained with only limited labeled data, surpasses the performance of a standard supervised model trained on the full labeled dataset on three of the four benchmarks, namely ACDC, PROMISE12, and Pancreas-NIH. This is a interesting finding, as performance on the full dataset is typically considered the upper bound for semi-supervised methods. This result further validates the effectiveness of our proposed framework.
4.3 Comparison experiments on benchmark datasets↩︎
| Scans used | Metrics | |||||
|---|---|---|---|---|---|---|
| 2-7 | Labeled | Unlabeled | DSC\(\uparrow\) | Jaccard\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) |
| DTC (AAAI’21) [34] | 3 (5%) | 67 (95%) | 56.90 | 45.67 | 23.36 | 7.39 |
| URPC (MICCAI’21) [62] | 55.87 | 44.64 | 13.60 | 3.74 | ||
| MC-Net (MICCAI’21) [63] | 62.85 | 52.29 | 7.62 | 2.33 | ||
| SS-Net (MICCAI’22) [35] | 65.83 | 55.38 | 6.67 | 2.28 | ||
| SCP-Net (MICCAI’23) [64] | 87.27 | - | - | 2.65 | ||
| BCP (CVPR’23) [24] | 87.59 | 78.67 | 1.90 | 0.67 | ||
| ABD (CVPR’24) [65] | 88.96 | 80.70 | 1.57 | 0.52 | ||
| \(\beta\)-FFT (CVPR’25) [66] | 89.58 | 81.68 | 1.46 | 0.45 | ||
| Ours-DTSL | 90.09 | 82.43 | 1.74 | 0.54 | ||
| DTC (AAAI’21) [34] | 7 (10%) | 63 (90%) | 84.29 | 73.92 | 12.81 | 4.01 |
| URPC (MICCAI’21) [62] | 83.10 | 72.41 | 4.84 | 1.53 | ||
| MC-Net (MICCAI’21) [63] | 86.44 | 77.04 | 5.50 | 1.84 | ||
| SS-Net (MICCAI’22) [35] | 86.78 | 77.67 | 6.07 | 1.40 | ||
| SCP-Net (MICCAI’23) [64] | 89.69 | - | - | 0.73 | ||
| PLGCL (CVPR’23) [67] | 89.10 | - | 4.98 | 1.80 | ||
| BCP (CVPR’23) [24] | 88.84 | 80.62 | 3.98 | 1.17 | ||
| ABD (CVPR’24) [65] | 89.81 | 81.95 | 1.46 | 0.49 | ||
| SDCL (MICCAI’24) [52] | 90.92 | 83.83 | 1.29 | 0.34 | ||
| \(\mathrm{M^{3}HL}\) (MICCAI’25) [68] | 90.47 | 83.23 | 1.43 | 0.34 | ||
| \(\beta\)-FFT (CVPR’25) [66] | 90.54 | 83.24 | 1.51 | 0.49 | ||
| Ours-DTSL | 91.47 | 84.67 | 1.10 | 0.26 | ||
| U-Net (supervised) | 3 (5%) | 0 | 47.83 | 37.01 | 31.16 | 12.62 |
| Ours-DTSL (supervised) | 3 (5%) | 0 | 81.36 | 71.08 | 6.89 | 1.56 |
| Ours-DTSL (semi-supervised) | 3 (5%) | 67 (95%) | 90.09 | 82.43 | 1.74 | 0.54 |
| U-Net (supervised) | 7 (10%) | 0 | 79.41 | 68.11 | 9.35 | 2.70 |
| Ours-DTSL (supervised) | 7 (10%) | 0 | 87.81 | 79.24 | 4.55 | 1.23 |
| Ours-DTSL (semi-supervised) | 7 (10%) | 63 (90%) | 91.47 | 84.67 | 1.10 | 0.26 |
| U-Net (supervised) | 70 (All) | 0 | 91.44 | 84.59 | 4.30 | 0.99 |
| Ours-DTSL (supervised) | 70 (All) | 0 | 92.22 | 85.87 | 1.71 | 0.41 |
ACDC dataset. We compared our proposed DTSL framework against several SOTA methods on the ACDC dataset, including DTC [34], URPC [62], MC-Net [63], SS-Net [35], SCP-Net [64], PLGCL [67], BCP [24], ABD [65], SDCL [52], \(\mathrm{M^{3}HL}\) [68], and \(\beta\)-FFT [66]. 2 presents the average segmentation performance across all classes for the 5%, 10%, and full labeled data protocols. The results demonstrate that DTSL consistently outperforms all SOTA baselines across all four evaluation metrics.
3 presents representative segmentation results on the ACDC dataset under the semi-supervised setting with 10% labeled data. As shown in the figure, segmentations produced by the proposed method align more closely with the ground truth compared to other approaches, particularly in capturing fine-grained structures. This visual evidence supports the motivation of our self-paced framework, which is designed to control the learning pace from easy (e.g., bulk regions) to complex (e.g., fine structures). Notably, in the red-highlighted region at the bottom of the figure, only the propsoed method achieves an accurate segmentation.
Performance comparison with SOTA methods on the PROMISE12 dataset under semi-supervised learning using 20% labeled data. Supervised baselines are also included at the bottom for reference.
PROMISE12 dataset. The evaluation results on the PROMISE12 dataset are summarized in ¿tbl:tab:Promise12?. Compared with several SOTA methods, including CCT [69], URPC [62], SS-Net [35], SLC-Net [70], SCP-Net [64], ABD [65], and \(\beta\)-FFT [66], the proposed DTSL framework demonstrates consistent improvements in both DSC and ASD metrics. Notably, in terms of DSC, DTSL achieves a 2.69% improvement over the second-best method, and its ASD is also better than previous approaches.
| Scans used | Metrics | |||||
|---|---|---|---|---|---|---|
| 2-7 | Labeled | Unlabeled | DSC\(\uparrow\) | Jaccard\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) |
| UA-MT (MICCAI’19) [71] | 4 (5%) | 76 (95%) | 82.26 | 70.98 | 13.71 | 3.82 |
| SASSNet (MICCAI’20) [6] | 81.60 | 69.93 | 16.16 | 3.58 | ||
| DTC (AAAI’21) [34] | 81.25 | 69.33 | 14.90 | 3.99 | ||
| SS-Net (MICCAI’22) [35] | 86.33 | 76.15 | 9.97 | 2.31 | ||
| PS-MT (CVPR’22) [21] | 88.49 | 79.13 | 8.12 | 2.78 | ||
| BCP (CVPR’23) [24] | 88.02 | 78.72 | 7.90 | 2.15 | ||
| AD-MT (ECCV’24) [65] | 89.63 | 81.28 | 6.56 | 1.85 | ||
| SGRS-Net (MICCAI’25) [72] | 89.70 | 81.40 | 6.68 | 1.75 | ||
| Ours-DTSL | 90.26 | 82.31 | 5.34 | 1.94 | ||
| V-Net (supervised) | 4 (5%) | 0 | 52.55 | 39.60 | 47.05 | 9.87 |
| Ours-DTSL (supervised) | 4 (5%) | 0 | 85.71 | 76.22 | 8.60 | 2.72 |
| Ours-DTSL (semi-supervised) | 4 (5%) | 76 (95%) | 90.26 | 82.31 | 5.34 | 1.94 |
| V-Net (supervised) | 80 (All) | 0 | 91.47 | 84.36 | 5.48 | 1.51 |
| Ours-DTSL (supervised) | 80 (All) | 0 | 91.64 | 84.41 | 5.32 | 1.41 |
LA dataset. As shown in 3, the proposed method is compared against several SOTA baselines, including UA-MT [71], SASSNet [6], DTC [34], SS-Net [35], PS-MT [21], BCP [24], AD-MT [65] and SGRS-Net [72]. The proposed method achieves superior performance on the DSC, Jaccard Index, and 95% Hausdorff Distance (95HD) metrics. While its ASD performance is slightly inferior to that of AD-MT [65], the strong overall results demonstrate the effectiveness and robustness of our approach.
| Scans used | Metrics | |||||
|---|---|---|---|---|---|---|
| 2-7 | Labeled | Unlabeled | DSC\(\uparrow\) | Jaccard\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) |
| UA-MT (MICCAI’19) [71] | 6 (10%) | 54 (90%) | 66.34 | 53.21 | 17.21 | 4.57 |
| SASSNet (MICCAI’20) [6] | 68.78 | 53.86 | 19.02 | 6.26 | ||
| DTC (AAAI’21) [34] | 69.21 | 54.06 | 17.21 | 5.95 | ||
| ASE-Net (TMI’22) [73] | 71.54 | 56.82 | 16.33 | 5.73 | ||
| SS-Net (MICCAI’22) [35] | 71.76 | 57.05 | 17.56 | 5.77 | ||
| PS-MT (CVPR’22) [21] | 76.94 | 62.73 | 13.12 | 3.66 | ||
| BCP (CVPR’23) [24] | 73.83 | 59.24 | 12.71 | 3.72 | ||
| AD-MT (ECCV’24) [65] | 80.21 | 67.51 | 7.18 | 1.66 | ||
| SGRS-Net (MICCAI’25) [72] | 80.55 | 67.88 | 6.00 | 2.50 | ||
| Ours-DTSL | 83.07 | 71.24 | 4.79 | 1.30 | ||
| V-Net (supervised) | 6 (10%) | 0 | 55.60 | 41.74 | 45.33 | 18.63 |
| Ours-DTSL (supervised) | 6 (10%) | 0 | 73.32 | 63.42 | 8.59 | 5.13 |
| Ours-DTSL (semi-supervised) | 6 (10%) | 54 (90%) | 83.07 | 71.24 | 4.79 | 1.30 |
| V-Net (supervised) | 60 (All) | 0 | 82.60 | 70.81 | 5.61 | 1.33 |
| Ours-DTSL (supervised) | 60 (All) | 0 | 83.51 | 71.62 | 4.71 | 1.25 |
Pancreas-NIH dataset. 4 presents the results on the Pancreas-NIH dataset, comparing the proposed method against several SOTA baselines, including UA-MT [71], SASSNet [6], DTC [34], ASE-Net [73],SS-Net [35], PS-MT [21], BCP [24],AD-MT [65], SGRS-Net[72]. The results demonstrate that DTSL outperforms all these methods across all evaluated metrics.
4.4 Ablation studies and hyperparameter optimization↩︎
We conducted ablation studies on the ACDC dataset using the 10% labeled data, unless otherwise stated, to evaluate the effectiveness of each proposed DTSL module, different CLG strategies, and the impact of hyperparameter optimization.
Effectiveness of each module in DTSL. As presented in ¿tbl:tab:ablation?, the ablation study compares the performance of several configurations: a vanilla MT strategy; a ‘plain DTSL’ setup (dual pseudo-labels from teacher and across-group student models without CLG); DTSL enhanced with the CLG module in 9 ; and the full framework including the URL in 10 .
The results in ¿tbl:tab:ablation? indicate that employing dual teacher-student groups (Plain DTSL) significantly outperforms the standard single-group MT approach. We attribute this improvement to the implicit self-paced learning introduced by using dual pseudo-labels. Similar to our analysis of the supervised MT strategy in [eqn:mt1,eqn:mt2,eqn:mt3], consensus between the pseudo-labels reinforces the learning signal, while disagreement reflects competing gradients that blur the optimization target.
Furthermore, incorporating the CLG module, which acts as a precise pace-regulator, substantially enhances performance over the Plain DTSL. Finally, applying the URL on top of CLG provides an additional, though smaller, improvement.
Ablation study on the effectiveness of each DTSL component using the ACDC 10% dataset. The table compares the performance of: (1) a vanilla MT baseline, (2) a ‘Plain DTSL’ setup (dual pseudo-labels without CLG/URL), (3) DTSL enhanced with the CLG module, and (4) the full DTSL framework including both CLG and URL.
Results on different CLG strategies. To validate the choice of inputs for the CLG, we explored several alternative strategies. Taking \(\mathrm{student}_0\) as an example, the available candidate signals are its in-group teacher (\(\mathrm{teacher}_0\)), the cross-group student (\(\mathrm{student}_1\)), and the cross-group teacher (\(\mathrm{teacher}_1\)). We compared the following setups (a symmetric configuration is used for \(\mathrm{student}_1\)),
Strategy 1: consensus between \(\mathrm{student}_1\) and \(\mathrm{teacher}_1\);
Strategy 2: consensus between \(\mathrm{teacher}_0\) and \(\mathrm{teacher}_1\);
Strategy 3: consensus among \(\mathrm{student}_1\), \(\mathrm{teacher}_0\), and \(\mathrm{teacher}_1\);
Default: consensus between \(\mathrm{teacher}_0\) and \(\mathrm{student}_1\).
As shown in 5, the default setting achieves the best performance. This result validates the motivation to control the learning pace is by leveraging the consensus between the temporally-lagged signal and the cross-architectural signal.
| Strategy | DSC\(\uparrow\) | Jaccard\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) |
|---|---|---|---|---|
| Strategy 1 | 90.21 | 82.69 | 1.70 | 0.53 |
| Strategy 2 | 90.45 | 83.21 | 1.98 | 0.41 |
| Strategy 3 | 90.64 | 83.29 | 1.60 | 0.30 |
| Default | 91.47 | 84.67 | 1.10 | 0.26 |
Results on alternative network architectures. To explore the impact of architectural choice on DTSL, we evaluated several combinations for the two model group, U-Net/ResU-Net (default), U-Net/Swin-Unet [2], and ResU-Net/Swin-Unet. The comparison, conducted on the ACDC 10% labeled dataset, is presented in ¿tbl:tab:structure?. The results confirm that the default U-Net/ResU-Net configuration achieves the best performance among the tested combinations.
Impact of network architecture combinations on performance, evaluated on the ACDC 10% dataset. The default (U-Net/ResU-Net) is compared to U-Net/Swin-Unet and ResU-Net/Swin-Unet.
4.5 Hyperparameter optimization↩︎
Optimization of the threshold \(\kappa\). The threshold \(\kappa\), introduced in 8 , is a vital hyperparameter as it determines the boundary between consistent (easy) and inconsistent (difficult) regions based on JS divergence, which are critical for regulating the self-paced learning curriculum. Experimental results, presented in ¿tbl:tab:kappa?, show that performance generally peaks at a specific value; it increases as \(\kappa\) rises from 0.05 and then decreases at higher values. We found that the optimal value for \(\kappa\) is consistently around \(0.1\) across all four datasets.
Hyperparameter optimization for the JS divergence threshold \(\kappa\) in [eqn:kappa], evaluated on the ACDC 10% dataset. Optimal performance is achieved at \(\kappa = 0.1\) across all four datasets.
| ACDC (10% labled) | PROMISE12 (20% labeled) | LA (5%) | Pancreas-NIH (10% labeled) | ||||||||||||||||
| \(\alpha\) | DSC\(\uparrow\) | Jac.\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) | \(\alpha\) | DSC\(\uparrow\) | Jac.\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) | \(\alpha\) | DSC\(\uparrow\) | Jac.\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) | \(\alpha\) | DSC\(\uparrow\) | Jac.\(\uparrow\) | 95HD\(\downarrow\) | ASD\(\downarrow\) |
| 0.50 | 91.18 | 84.22 | 1.94 | 0.45 | 0.50 | 84.08 | 72.73 | 4.08 | 1.38 | 0.05 | 90.05 | 81.99 | 7.15 | 2.51 | 0.05 | 82.52 | 70.43 | 4.90 | 1.26 |
| 0.75 | 91.31 | 84.40 | 1.13 | 0.40 | 0.75 | 84.76 | 73.70 | 3.09 | 1.17 | 0.10 | 90.26 | 82.31 | 5.34 | 1.94 | 0.10 | 83.07 | 71.24 | 4.79 | 1.30 |
| 1.00 | 91.47 | 84.67 | 1.10 | 0.26 | 1.00 | 84.69 | 73.63 | 2.64 | 1.05 | 0.15 | 89.61 | 81.29 | 6.42 | 2.05 | 0.15 | 83.02 | 71.17 | 4.70 | 1.25 |
| 1.25 | 90.71 | 83.56 | 1.13 | 0.32 | 1.25 | 85.19 | 74.35 | 2.60 | 0.97 | 0.20 | 89.80 | 81.60 | 6.14 | 2.20 | 0.20 | 82.80 | 70.87 | 4.87 | 1.26 |
| 1.50 | 90.11 | 82.66 | 1.22 | 0.29 | 1.50 | 84.42 | 73.21 | 2.66 | 1.00 | 0.30 | 90.02 | 81.93 | 6.20 | 2.20 | 0.30 | 82.95 | 71.10 | 4.65 | 1.24 |
Optimization of \(\alpha\) in 11 . The pace regulator, \(\mathcal{L}_{\mathrm{pace}}\), is composed of \(\mathcal{L}_{\mathrm{semi}}\) and \(\mathcal{L}_{\mathrm{URL}}\), weighted by hyperparameters \(\alpha\) and \(\beta\), respectively. Since the full supervised loss \(\mathcal{L}^{\ell}\) also includes \(\mathcal{L}_{\mathrm{sup}}\), \(\alpha\) and \(\beta\) should be tuned as independent hyperparameters.
However, considering the unlabeled loss \(\mathcal{L}^{u}\) consists solely of \(\mathcal{L}_{\mathrm{pace}}\) and dominates the overall training objective in SSMIS, we found the relative ratio between \(\alpha\) and \(\beta\) to be more critical than their absolute values. To provide a clear analysis, we adopted a standard tuning strategy by fixing one hyperparameter while varying the other.
As shown in 6, with \(\beta=0.05\) fixed, the results demonstrate that the optimal choice for \(\alpha\) is around \(1.00\) for the 2D datasets (ACDC, PROMISE12) and \(0.10\) for the 3D datasets (LA, Pancreas-NIH).
Optimization of \(\omega\) of the EMA process. In the MT framework, the weights of the teacher model are updated via the EMA of the student model’s weights as defined in 1 , making the teacher a temporally smoothed version of the student. The update factor \(\omega\) controls this temporal smoothing, and we hypothesized it might be critical for regulating the learning pace.
To investigate this effect, we conducted experiments on the ACDC 10% dataset, varying \(\omega\) from \(0.90\) to \(0.99\). The results, presented in ¿tbl:tab:omega?, show that while optimal performance was achieved at \(\omega=0.95\), the overall performance variation across the tested range is minimal. This suggests that the DTSL framework is robust to the specific choice of \(\omega\).
Hyperparameter optimization for the EMA update factor \(\omega\) in [eqn:EMA], evaluated on the ACDC 10% dataset. Although optimal performance is observed at \(\omega = 0.95\), the minimal performance variation demonstrates that the DTSL framework is highly robust to this parameter.
5 Conclusion↩︎
Inspired by interpreting the MT strategy by self-paced learning, we propose the DTSL framework for SSMIS. The framework employs two groups of teacher-student models with distinct architectures. A CLG is introduced to produce pseudo-labels by leveraging both outputs from temporally averaged and cross-architectural models. These pseudo-labels guide the learning process from easy to complex regions. Additionally, a URL is applied to uncertain regions, encouraging the model to reconsider low-confidence predictions and further refine the learning pace. Experimental results demonstrated that our method achieves SOTA performance on four benchmark datasets. Remarkably, on three of the four datasets, our semi-supervised method with limited labeled data surpasses its fully supervised counterparts.