1 Introduction↩︎

Deep neural networks have achieved remarkable progress in computer vision and related fields in recent years [1], [2]. However, their success typically relies on the assumption that training and test data follow the same distribution, which rarely holds in real-world applications. In practice, factors such as illumination changes, weather variations, sensor noise, and differences in imaging devices often lead to distribution shifts. When faced with such shifts at inference time, model performance can degrade substantially, sometimes resulting in catastrophic failures [3]. Since it is infeasible to train models on all possible data distributions, this reliance on the training distribution limits their adaptability.

To mitigate distribution shifts, prior research has extensively explored Unsupervised Domain Adaptation (UDA), which addresses domain gaps by jointly training on labeled source data and unlabeled target data [4]–[7]. However, it requires access to both source and target data during training. In practice, target data is often unavailable beforehand, distribution shifts are unpredictable, and access to source data may also be restricted due to privacy or computational constraints. In response to this challenge, Test-Time Adaptation (TTA) has recently attracted growing attention in scenarios involving domain shifts caused by image corruptions [8], [9]. In this paradigm, a pre-trained source model, without access to its training data, is adapted to unlabeled test data through online training.

Early TTA studies [10]–[12] primarily focused on static adaptation settings. These methods assumed that all test samples were independently drawn from a single fixed target domain, implying that each test batch was independently and identically distributed (i.i.d.). Under this assumption, the model can gradually adapt to a stationary target distribution by minimizing entropy or updating normalization statistics. However, such an assumption is often unrealistic in practice, as real-world test data rarely follow a fixed distribution and instead exhibit complex and evolving patterns over time. To address this limitation, later works [13]–[15] proposed Continual Test-Time Adaptation (CTTA), which considers a more realistic dynamic adaptation scenario where the test distribution continuously shifts over time under changing environments. Furthermore, [16] introduced a more practical setting that not only follows the continual adaptation paradigm but also accounts for the fact that test data usually arrive as streaming sequences with strong temporal correlations. This often leads to severe class imbalance within short time windows. Termed Practical Test-Time Adaptation (PTTA), this setting presents two critical challenges: class imbalance and continuous domain shifts.

First, class imbalance occurs when some classes appear frequently within a short period while others are underrepresented or temporarily absent. This situation often comes from temporal correlation in real-world streaming data. For example, in autonomous driving, vehicles on highways predominantly observe cars, whereas in residential areas they mostly encounter pedestrians or non-motorized vehicles. Such temporally localized distributions lead to biased model updates because the model repeatedly learns from dominant classes [17]. As a result, gradient updates are dominated by majority classes, causing the model to overfit to short-term patterns while neglecting minority classes, ultimately degrading overall generalization performance.

Second, continuous domain shift refers to gradual changes in the test distribution as environmental conditions evolve over time. For instance, in autonomous driving, lighting changes when vehicles move from sunny roads to rainy streets or tunnels, causing shifts in input features over time [18]. Under continual domain shift settings, TTA methods that rely on pseudo-labels or entropy minimization tend to be unstable and highly sensitive to noisy signals. As the distribution drifts, pseudo-labels may degrade, producing noisy supervision that misguides gradient updates and reduces model reliability.

To jointly address class imbalance and continuous domain shift, we propose a Balanced and Prototype-Guided Test-Time Adaptation (BP-TTA) method, which combines batch-balanced sampling with prototype-guided adaptation. The following are our main contributions:

• We propose Batch-Balanced Sampling (BBS) to address class imbalance, which constructs a class-balanced memory bank from realistic test streams and dynamically reconstructs the current batch according to the class distribution of incoming samples, effectively mitigating class imbalance. The details are illustrated in Fig. 1 (a). This design ensures that each model update is guided by high-quality, balanced pseudo-labels, thereby improving adaptation robustness in dynamic streams.

• We propose Category Prototype-Guided Adaptation (CPGA) to handle continuous domain shifts, aiming to improve pseudo-label quality and stabilize TTA. As shown in Fig. 1 (b), CPGA dynamically maintains class prototypes during testing and incorporates feature-prototype similarity into model updates, enabling active adaptation in the feature space.

• We evaluate our method on three datasets under data streams with coexisting class imbalance and continually changing distributions, demonstrating superior performance over recent state-of-the-art methods.

2 Related Work↩︎

2.1 Test-Time Adaptation↩︎

Neural networks have become the leading topic of artificial intelligence, and recurrent models [19]–[21] and ODE-based methods [22]–[24] have been largely developed for complicate machine learning tasks. However, these models performed not well in dynamic scenarios with continual domain shifts. Unsupervised Domain Adaptation (UDA) aims to train a model using labeled source domain data and generalize it to an unlabeled target domain, addressing the challenge of distribution shift without requiring target domain annotations [4]–[7]. However, it may be infeasible when training data are unavailable due to privacy or storage concerns. To overcome this limitation, [10] proposed Test-Time Adaptation (TTA), which focuses on real-time model adjustment during the testing phase using unlabeled target data, without accessing the source data used for training. Early TTA approaches mainly focus on adapting normalization statistics or entropy minimization. For instance, BN-1 [11] adapts models by aligning batch normalization statistics to the target domain, which provides a simple yet effective way to partially mitigate distribution shifts. TENT [10] further updates model parameters by minimizing the prediction entropy on incoming test samples while simultaneously updating batch normalization statistics. In contrast to these partial-update strategies, AdaContrast [12] introduces a memory module combined with contrastive learning and performs adaptation on the entire network, enabling richer representation learning during test time. Overall, these methods demonstrate that models can effectively adapt to distribution shifts using only unlabeled test data, highlighting the promise of test-time learning as a practical and flexible solution for improving model robustness.

2.2 Real-World Test-Time Adaptation↩︎

While existing TTA methods have demonstrated strong potential for improving out-of-distribution generalization, most of them are typically developed and evaluated under relatively controlled experimental settings, where the assumptions about the test environment are simplified and the distribution shifts are often limited in scope. Consequently, recent studies on TTA have increasingly focused on extending adaptation methods to more realistic scenarios. Several key challenges have been identified in real-world applications, including adaptation with small-batch or single-sample updates [25], [26], class-imbalanced test data [27], continuously shifting target domains [13], [28], [29], adaptation on noisy data streams [30], mixtures of multiple distribution shifts [31], and constraints imposed by limited computational resources or deployment on edge devices [32], [33]. Empirical evidence shows that these real-world conditions can significantly degrade the performance of existing TTA methods. In this work, we address the combined challenges of continuous domain shifts and severe class imbalance by introducing a dynamic batch-balancing component combined with class prototype guidance.

2.3 Prototype Learning↩︎

Prototype-based methods represent each class by a compact and representative feature vector, known as a prototype, typically computed as the mean of embedded samples belonging to that class. While the idea of class prototypes has a long history in pattern recognition, Prototypical Networks [34] popularized its use in modern deep learning, particularly in few-shot learning, by classifying queries based on distances to class prototypes rather than parametric classifiers. Since then, prototype-based methods have been widely adopted in various scenarios, including domain adaptation [35], [36] and semantic segmentation [37], [38], due to their robustness and interpretability. In TTA setting, T3A [39] leveraged class prototypes to adjust classifier weights during test time. In contrast, our method utilizes prototype information to guide feature adaptation, rather than confining it to classifier recalibration, and is thus better suited to handling more complex and evolving test-time data distributions.

3 Methods↩︎

3.1 Problem Definition↩︎

Given a model \(f_{\theta}\) pre-trained on a labeled source dataset \(\mathcal{D}^{Source} = \{(\boldsymbol{x}^{Source}, \boldsymbol{y}^{Source})\}\), and a sequence of unlabeled target-domain batches provided over time \(\mathcal{D}^{Target} = \{\boldsymbol{x}^{Target}\}\). At each time step \(t\), the target samples are drawn from a distribution \(P^{Target}_{t}(\boldsymbol{x})\), which evolves over time. Moreover, due to temporal dependencies, the target samples are non-independent and identically distributed, and the underlying distribution evolves continuously over time. This setting is motivated by real-world dynamic scenarios, in which distribution shift and temporal correlation among samples often occur simultaneously.

To formally describe the model architecture, we define \(f_{\theta}(\cdot) = h(g(\cdot))\), where \(h(\cdot): \mathbb{R}^{B \times D} \rightarrow \mathbb{R}^{B \times C}\) is the linear classifier for the last layer and \(g(\cdot): \mathbb{R}^{B \times F \times H \times W} \rightarrow \mathbb{R}^{B \times D}\) is the feature extractor for the rest. Like most mainstream Test-Time Adaptation (TTA) methods, we adopt a Mean Teacher structure [40], where a teacher network guides the student network through consistency regularization, effectively smoothing the learning process over time. Both the student model \(f_{\theta^S}\) and the teacher model \(f_{\theta^T}\) are initialized with the parameters of the pre-trained source model. At each time step \(t\), \(f_{\theta^S_t}\) receives a batch of target samples \(\boldsymbol{x}_t\), produces predictions \(f_{\theta^S_t}(\boldsymbol{x}_t)\), and subsequently updates its parameters to adapt to future inputs, i.e., \(\theta_t^S \rightarrow \theta^S_{t+1}\). \({\theta^T_t}\) are updated as an Exponential Moving Average (EMA) of the student model parameters. The objective is to improve the performance of the model during inference on the continually evolving target domain in an online manner, without accessing any source data. We propose BP-TTA, which enhances robustness in Practical Test-Time Adaptation (PTTA) through two complementary components. An overview of BP-TTA is illustrated in Fig. 2.

3.2 Batch-Balanced Sampling↩︎

In dynamic scenarios with strong temporal dependencies, the class distribution within each incoming batch \(\boldsymbol{x}\) is often highly imbalanced. To mitigate class imbalance, we propose a Batch-Balanced Sampling (BBS) component that dynamically reconstructs balanced batches directly from realistic test streams.

Our approach maintains a class-aware memory bank \(\mathcal{M}\), which is initially empty and dynamically updated as new batches arrive. Due to the continuously changing test distribution, old samples in \(\mathcal{M}\) may lose their relevance or even hinder the model’s adaptation, while high-uncertainty samples can generate erroneous gradients that impair learning. To account for these factors, each sample stored in the memory bank \(\mathcal{M}\) is associated with a set of attributes \((\hat{y}_i, a_i, \gamma_i, u_i)\) [16]. Here, \(\hat{y}_i\) is the pseudo-label generated by the teacher model \(f_{\theta^T}\), and \(a_i\), \(\gamma_i\), and \(u_i\) denote the age of the sample, the confidence of the pseudo-label, and the uncertainty of the prediction, respectively. Specifically, \(\hat{y}_i\), \(\gamma_i\), and \(u_i\) are computed as follows: \[\begin{align} p(c|x_i) &= \mathrm{Softmax}\big(f_{\theta^T}(x_i)\big), \tag{1} \\ \hat{y}_i &= \arg\max_{c} p(c|x_i), \tag{2}\\ \gamma_i &= \max_{c} p(c|x_i), \tag{3} \\ u_i &= - \sum_{c=1}^{C} p(c|x_i) \log(p(c|x_i)), \tag{4} \end{align}\] where \(x_i\) denotes the \(i\)-th test sample in current batch and \(C\) is the number of categories. The sample age \(a_i\) is initialized to 0 and increases with time \(t\). Each incoming sample is selectively inserted into \(\mathcal{M}\) according to a selection score that accounts for both temporal recency and uncertainty: \[\mathcal{H}_i = \exp\Big(\frac{a_i}{K}\Big) + u_i, \label{eq:score}\tag{5}\] where \(K\) is the memory capacity. When the class-specific quota is exceeded, existing samples in the memory bank with higher selection scores are removed to accommodate new samples, ensuring that the memory bank retains more reliable and informative samples for each class.

Using an algorithm described in Algorithm 3, we dynamically reconstruct a balanced batch \(\hat{\boldsymbol{x}}_t\) with exactly \(N_{\text{per-class}}\) samples per class for model adaptation at each time step. We set \(N_{\text{per-class}} = \frac{N}{C}\) by default. When the number of categories is extremely large, we instead use \(N_{\text{per-class}} = \frac{N}{U_t}\), where \(U_t\) is the number of predicted pseudo-labels at time step \(t\). By eliminating redundant majority-class samples and adding high-quality memory samples to underrepresented minority classes, our method successfully reduces class imbalance. Specifically, BBS first partitions the current batch \(\boldsymbol{x}_t\) by pseudo-labels, grouping samples into class-specific sets. It then adjusts the processing approach for each class based on the representation of the class in the incoming batch. We choose the top-\(N_{\text{per-class}}\) highest-confidence samples for majority classes that surpass the target count, thereby downsampling without sacrificing quality. We retrieve additional samples from the corresponding class subset in the memory bank, giving priority to those with high confidence scores, for minority classes that fall short of the target. We take a direct sample from the memory bank if a class is not present in the current batch. To guarantee that each selected class reaches \(N_{\text{per-class}}\) samples in the final balanced batch, we replicate the highest-confidence sample in rare cases where even memory augmentation is insufficient.

Innovations of BBS. RoTTA [16] proposed Category-balanced Sampling with Timeliness and Uncertainty (CSTU), which also leverages a class-balanced memory bank to mitigates test distribution shift in PTTA for effective model adaptation. Compared to CSTU, BBS brings clear and substantial improvements. First, BBS adopts a simpler score formulation that jointly considers temporal recency and prediction uncertainty, as defined in Eq. 5 , while remaining equally effective in practice. Second, although CSTU takes both timeliness and category balance into account, its batches are composed solely of samples retrieved from the memory bank. In scenarios with frequent domain changes, CSTU still introduces temporal delay, preventing the constructed batches from faithfully reflecting the current target-domain distribution. In contrast, BBS directly operates on the incoming test batch, selectively filtering redundant majority-class samples and supplementing underrepresented minority classes with high-quality historical samples from the memory bank. The resulting balanced batch not only provides equal learning opportunities for all classes but also better reflects the distribution of the current test stream, leading to more stable and robust adaptation.

3.3 Category Prototype-Guided Adaptation↩︎

Under continuous domain shifts, pseudo-labels generated solely by entropy minimization or model predictions tend to become increasingly noisy and unreliable. We propose Category Prototype-Guided Adaption (CPGA) to stabilize TTA under persistent domain shifts. It dynamically maintains class prototypes using an exponential moving average and leverages sample-prototype similarity to guide proactive adaptation of the feature space. This design promotes tighter intra-class clustering and more discriminative inter-class separation, thereby improving pseudo-label quality under persistent domain shifts.

To preserve stable class-level structures under continuous domain shifts, we maintain a set of class prototypes \(\{\boldsymbol{p}_c \in \mathbb{R}^D\}_{c=1}^{C}\), each representing the centroid of a class in the feature space and providing class-level guidance. Leveraging the discriminative class knowledge already encoded in the source-trained model’s classifier, we initialize these prototypes as follows: \[\boldsymbol{p}_c^{(0)} = \mathbf{W}_c, \quad 1 \leq c \leq C,\] where \(\mathbf{W}_c \in \mathbb{R}^D\) is the \(c\)-th row of the classifier weight matrix and \(D\) is the dimension of feature embeddings. During TTA, these prototypes must track the evolving feature distribution to remain effective under continuous domain shift. For each balanced batch \(\hat{\boldsymbol{x}}_t = \{\hat{x}_i\}_{i=1}^{N}\) constructed in Section 3.2, we extract features \(\boldsymbol{z}_i = g_t(\hat{x}_i)\) and calculate the pseudo-label \(\hat{y}_i\) by Eq. 2 for \(1 \leq i \leq N\). To maintain prototype quality and prevent error accumulation from noisy pseudo-labels, we selectively update prototypes using only high-confidence samples through a momentum-based mechanism: \[\boldsymbol{p}_c^{(t+1)} = \alpha \boldsymbol{p}_c^{(t)} + (1 - \alpha) \boldsymbol{z}_i, \quad \text{if } \hat{y}_i = c \text{ and } \gamma_i \geq \tau, \label{eq:proto95update}\tag{6}\] where \(\alpha\in[0,1]\) is a smoothing factor, \(\hat{y}_i\) is the teacher’s pseudo-label, \(\gamma_i\) is the confidence score calculated by Eq. 3 , and \(\tau\) is a confidence threshold. A lower momentum value enables prototypes to respond promptly to distribution changes while maintaining stability against noisy updates. With the updated prototypes, we enforce class-level consistency in the feature space through a prototype-based supervision loss. The key idea is to measure how well each sample’s feature aligns with different class prototypes and use this alignment to guide the learning process. Given a sample \(x_i\) with extracted feature \(\boldsymbol{z}_i \in \mathbb{R}^{D}\), we compute cosine similarity between it and its assigned class prototype: \[\text{sim}(\boldsymbol{z}_i, \boldsymbol{p}_{\hat{y}_i}) = \frac{\boldsymbol{z}_i \cdot \boldsymbol{p}_{\hat{y}_i}}{\|\boldsymbol{z}_i\| \|\boldsymbol{p}_{\hat{y}_i}\|},\] where \(\text{sim}(\boldsymbol{z}_i, \boldsymbol{p}_{\hat{y}_i}) \in [-1, 1]\) measures similarity between sample \(\hat{x}_i\) and its assigned class prototype. Higher similarity indicates stronger alignment between the feature and its target class representation. To ensure that prototype updates are guided by reliable supervision, we compute the prototype loss only on high-confidence samples, defined as \(\mathcal{S} = \{i|\gamma_i \geq \tau\}\). The loss is expressed as minimizing the distance between features and their corresponding prototypes: \[\mathcal{L}_{\text{proto}} = \frac{1}{|\mathcal{S}|} \sum_{i \in \mathcal{S}} \left(1 - \text{sim}(\boldsymbol{z}_i, \boldsymbol{p}_{\hat{y}_i})\right).\]

Innovations of CPGA. Prototype-based methods update class prototypes during testing and leverage them to facilitate classification. Early approaches typically do not perform explicit alignment in the feature space of the feature extractor. For example, T3A [39] updates class prototypes based on the support set of each category and subsequently adjusts classifier weights according to these prototypes during the test time. TAST [35] further refines the support set by incorporating nearest-neighbor information, thereby mitigating the impact of noisy pseudo-labels. More recently, TSD [41] introduces Memorized Spatial Local Clustering to optimize alignment in the feature space of the feature extractor. However, it still relies on local neighborhood constraints and incurs considerable storage overhead for sample features. We propose the CPGA, which dynamically maintains class prototypes to adapt continual domain shift and employs a prototype alignment loss to directly guide features toward their corresponding class prototypes. This encourages a more discriminative feature space characterized by increased intra-class compactness and enhanced inter-class separability.

3.4 The Overall Optimization Objective↩︎

The overall optimization objective consists of two parts that jointly guide the adaptation process. For the balanced batch constructed at each time step, we first apply strong data augmentations and enforce a prediction consistency constraint between the student and teacher models: \[\mathcal{L}_{\text{sup}} = \frac{1}{N} \sum_{i=1}^{N} H\big(p_{\theta^{\text{S}}}(y \mid \text{Aug}(x_i)),\, p_{\theta^{\text{T}}}(y \mid x_i)\big),\] where \(H(\cdot, \cdot)\) denotes a consistency loss, and \(\text{Aug}(\cdot)\) applies data augmentation. This loss stabilizes the student model by requiring its predictions under perturbations to match the teacher’s outputs, thereby improving robustness under distribution shift.

To further enhance class-level consistency, we incorporate the prototype loss introduced in Section 3.3, which encourages sample features to align with their corresponding class prototypes. The total loss is defined as: \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{sup}} + \lambda \mathcal{L}_{\text{proto}}, \label{eq:loss}\tag{7}\] where \(\lambda\) is a regularisation coefficient.

Student model parameters are updated via gradient descent with gradient clipping to prevent instability: \[\theta_{t+1}^S \leftarrow \theta_t^S - \eta \nabla_{\theta_t^S} \mathcal{L}_{\text{total}}, \quad \|\nabla_{\theta_t^S} \mathcal{L}_{\text{total}}\| \leq \epsilon,\] where \(\eta\) is the learning rate and \(\epsilon\) is the clipping threshold. The teacher model is updated via exponential moving average: \[\theta^T_{t+1} \leftarrow (1 - \nu) \theta^T_t + \nu \theta_{t+1}^S,\] where \(\nu\) is the EMA momentum.

4 Experiments↩︎

4.1 Experiment Settings↩︎

4.1.1 Datasets↩︎

We follow the standard evaluation benchmarks introduced in [13] and evaluate our proposed method on CIFAR10-C, CIFAR100-C, and ImageNet-C. These datasets are built upon the corruption protocols proposed in [42]. Each dataset contains an image set of 15 corruption style including gaussian noise, shot noise, impulse noise, defocus blur, glass blur, motion blur, zoom blur, snow, frost, fog, brightness, contrast, elastic, pixelated, and jpeg. We adopt the same test-time augmentation strategy as CoTTA [13], applying transformations such as color jitter, gaussian blur, gaussian noise, random flips, and random affine operations to the input.

4.1.2 Implementation and setup↩︎

Following common practice in Test-Time Adaptation (TTA), we adopt WideResNet-28 [43] for CIFAR10-C and ResNeXt-29 [44] for CIFAR100-C, both sourced from the RobustBench benchmark [45]. For ImageNet-C, we use a ResNet-50 [1] model provided by the official torchvision library. All evaluations are conducted under the highest corruption severity (level 5). During the TTA process, the memory bank capacity \(K\) is set to 100, 200, and 1000 for CIFAR10-C, CIFAR100-C, and ImageNet-C datasets, respectively. EMA momentum \(\nu\) is fixed to 0.001 for all experiments. Smoothing factor \(\alpha\) is set to 0.3 for CIFAR10-C and 0.5 for CIFAR100-C and ImageNet-C. Threshold \(\tau\) is set to 0.8 for CIFAR10-C and CIFAR100-C, and 0.5 for ImageNet-C. \(\lambda\) and \(\epsilon\) are set to 10 and 1 throughout. To construct the PTTA test data stream, we sample from the Dirichlet distribution with concentration parameter \(\delta\) configured as 0.1, 0.01, and 0.001 respectively. Fig. 4 illustrates the class distribution of test streams generated with different \(\delta\) on CIFAR10-C. A smaller \(\delta\) leads to a more imbalanced class distribution, while a larger \(\delta\) results in a more balanced distribution.

For optimization, we adopt Adam optimizer with a learning rate \(\eta\) of 1e-3 and \(\beta=0.9\), and the batch size is consistently set to 64 across all datasets and methods. All experiments were implemented with PyTorch in Python 3.9 on GeForce 4090 GPUs.

4.1.3 Baselines↩︎

We evaluate our proposed BP-TTA against several representative TTA methods, including TENT [10], CoTTA [13], EATA [14], SAR [31], PALM [28], and SURGEON [33]. In addition, we include comparisons with TTA methods specifically designed for class-imbalanced scenarios, such as LAME [32], RoTTA [16], and DA-TTA [46]. For a fair evaluation, all baselines are re-implemented with parameters as mentioned in their paper and tested under identical experimental settings. In all result tables, the best scores are highlighted in bold, and the second-best scores are underlined.

4.2 Results↩︎

Table 1 presents the experimental results of BP-TTA and other TTA baseline methods for adaptation on sample streams with domain shift and class imbalance. The results show that our proposed BP-TTA method achieves the best performance across all baselines on the three datasets, outperforming the second-best method by margins of 3.08%, 1.84%, and 1.59%, respectively. It is observed that many previous methods exhibit poorer results under the Practical Test-Time Adaptation (PTTA) setting, for they tend to overfit local distributions when confronted with class-imbalanced test streams. Notably, although LAME shows favorable performance advantages on CIFAR10-C and even achieves optimal results on some target domains without updating the model’s parameters, it suffers from substantial performance degradation on CIFAR100-C and ImageNet-C. On the contrary, BP-TTA maintains stable adaptation capabilities across all target domains and achieves the best overall average performance.

4.3 Additional experiment on DomainNet-126↩︎

To validate the effectiveness of our approach under domain shifts not induced by corruption, we further conduct experiments on the DomainNet-126 dataset [47]. DomainNet-126 comprises 126 classes across four domains (real, clipart, painting, and sketch), each containing 18,523 images. We continually adapt a ResNet-50 model pre-trained on the sketch domain to correlatively sampled test streams of the rest domains. The architecture and pre-trained weights are consistent with those used in [12]. Notably, this architecture employs weight normalization, which decouples the weights into magnitude and direction components. This reparameterization modifies the gradient flow and leads to incompatibility with the sparsity mechanism and backpropagation process of SURGEON [33]. Therefore, we exclude SURGEON from this experiment. As shown in Table 2, BP-TTA achieves the best performance, demonstrating its strong capability in handling more challenging domain shifts.

4.4 Ablation Studies and Analysis↩︎

4.4.1 Effect of each component↩︎

We conduct an ablative study to further investigate the effectiveness of individual components. As shown in Table 3, using BBS alone significantly reduces error rates over the source model, demonstrating the effectiveness of balanced sampling in mitigating class imbalance during adaptation. CPGA alone underperforms the source model on CIFAR10-C and CIFAR100-C, since severe class imbalance in the test stream causes biased prototype updates, leading to error accumulation. This is mainly because the highly imbalanced test stream leads to biased and noisy category prototypes. The joint use of BBS and CPGA outperforms either component alone, as BBS provides balanced batches for reliable prototype updates, while CPGA further exploits these stable prototypes for effective adaptation.

4.4.2 Effect of batch size↩︎

As shown in Fig. 5 (a), batch size has a noticeable impact on the performance of BP-TTA. Compared to very small batch sizes, increasing the batch size significantly reduces the classification error on three datasets. However, when the batch size becomes excessively large, the performance tends to degrade. This is because a small batch size introduces large noise when updating the student network and category prototypes, making the adaptation unstable, while an overly large batch size slows down the model’s response to distribution shifts and may generate inaccurate pseudo-labels from an outdated teacher network. Therefore, choosing a moderate batch size can balance stability and response speed, achieving the best performance.

4.4.3 Effect of hyperparameters \(\tau\) and \(\alpha\)↩︎

We analyze the impact of the confidence threshold \(\tau\) and smoothing factor \(\alpha\) in Eq. 6 on the performance of BP-TTA. Taking CIFAR10-C as an example, Fig. 5 (b) presents the average error rates under different combinations of \(\tau\) and \(\alpha\). We observe that when the value of \(\tau\) ranges from 0.5 to 0.9, the results do not vary too much, indicating that the model does not heavily rely on precise parameter tuning. Regarding the smoothing factor \(\alpha\), lower values generally yield better results, as they allow class prototypes to respond more quickly to continual domain shifts, while large values may slow down the adaptation process and reduce the effectiveness of our method.

4.4.4 Sensitivity to weighting factor↩︎

We conduct a detailed hyperparameter search for weighting factor \(\lambda\), i.e., \(\lambda \in \{0, 0.1, 1, 10, 100\}\), to investigate the sensitivity of \(\lambda\) in Eq. 7 . \(\lambda\) is the hyper-parameter balancing \(\mathcal{L}_{\text{sup}}\) and \(\mathcal{L}_{\text{proto}}\). \(\mathcal{L}_{\text{sup}}\) provides direct supervision for samples with high-confidence pseudo-labels, and \(\mathcal{L}_{\text{proto}}\) encourages feature representations to align with their corresponding class prototypes, therefore promoting intra-class clustering and inter-class separation. Table 4 reports the average classification error rates obtained with different values of \(\lambda\). From the results, we observe that setting \(\lambda\) to 10 consistently yields the best performance across all three datasets, demonstrating the effectiveness of the prototype-guided loss.

4.4.5 Effect of domain change frequency↩︎

To further investigate how different methods respond to varying frequencies of domain changes, we introduce a parameter \(n\) to explicitly control the frequency of domain shifts in the test data stream. Specifically, \(n\) denotes the number of segments into which the sample sequence of each domain is divided, such that increasing \(n\) results in more frequent domain transitions throughout the test stream. When \(n=1\), the setting degenerates to the original domain shift scenario where each domain appears only once in sequence. Table 5 presents the average classification errors on ImageNet-C. As can be seen, RoTTA suffers a substantial performance drop when \(n\) increases, indicating its sensitivity to frequent domain changes. Meanwhile, BP-TTA exhibits a slower performance degradation, which proves BBS can effectively alleviate the negative impact of frequent domain shifts.

4.4.6 t-SNE visualization analysis↩︎

We perform a feature distribution analysis via t-SNE visualization [48] on the CIFAR10-C dataset to validate the effectiveness of our proposed CPGA component. Specifically, we replace the CPGA module in BP-TTA with the classifier adjustment module proposed in T3A [39], which is also a prototype-based method. For a fair comparison, all other experimental settings are kept identical. Fig. 6 presents the visualization results for the “Brightness” domain within the test stream. As illustrated in the figure, when using T3A, the extracted features from different classes are heavily intermixed, with substantial overlap across categories, making it difficult to form well-separated clusters. In contrast, the features obtained with our CPGA exhibit significantly improved separability, forming more compact and well-defined clusters for each class. These results suggest that CPGA promotes tighter intra-class cohesion and clearer inter-class boundaries within the local feature space by using \(\mathcal{L}_{\text{proto}}\).

4.4.7 Robustness to different Dirichlet concentration parameters↩︎

We vary the value of Dirichlet concentration parameter \(\delta\) on three corrupted datasets in Table 6. In this setting, the parameter \(\delta\) controls the temporal correlation of class labels in the test data stream, where a smaller \(\delta\) produces a stronger temporal dependency and leads to a higher degree of class imbalance. As we can see, as \(\delta\) decreases from 1e-1 to 1e-6, the average error rates of BP-TTA increase 3.68%, 2.68%, and 1.39% on CIFAR10-C, CIFAR100-C, and ImageNet-C, respectively. Despite the increasingly challenging imbalance conditions induced by smaller \(\delta\), the performance degradation of BP-TTA remains relatively limited. These results indicate that the proposed method can maintain stable performance under varying levels of temporal class imbalance, demonstrating its robustness to different Dirichlet concentration settings in the test stream.

4.4.8 Robustness to the distribution changing order↩︎

To verify that the effectiveness of the proposed method is not dependent on specific orders of data distribution shifts, we conducted experiments with ten distinct sequences of distribution changes on three datasets under the PTTA setting. We report the average classification error of ten different domain shift sequences on three datasets in Table 7. Specifically, Fig. 7 demonstrates the results on ImageNet-C. From the figure, we observe that DA-TTA can achieve competitive performance and even obtain the best results on several specific sequences. However, its performance varies significantly across different sequences, indicating that the method is highly sensitive to the order in which distribution shifts occur and may suffer from large performance fluctuations. In contrast, BP-TTA maintains consistently strong performance across all sequences, demonstrating stronger robustness to sequence ordering and achieving the best average performance.

4.5 Comparison with model variation↩︎

To validate the adaptive applicability of our proposed method across various models, we conduct additional experiments. In the CIFAR-C benchmark, we use ResNext-29 [44], WideResNet-28 [43], WideResNet-40 [43], and PreActResNet-18 [49] model from [45]. For simplicity, we refer to these models as RNXT29, WRN28, WRN40, and PARN18, respectively. However, the CIFAR100-C pre-trained model parameters for WRN28 is not available in [45], so we exclude WRN28 from the CIFAR100-C experiment. In the ImageNet-C dataset, we use ResNet-50 [1], WideResNet-50 [43], and ResNeXt-50 [44] provided by the official torchvision library. To simplify, we refer to these models as RN50, WRN50, and RNXT50, respectively.

4.5.1 Comparison on CIFAR10-C↩︎

Table 8 presents the average classification errors on CIFAR10-C under the PTTA setting with different models. Our proposed BP-TTA consistently outperforms all baseline methods across all models, achieving the lowest error rates of 21.80%, 15.76%, 15.87%, and 19.78% on WRN28, RNXT29, WRN40, and PARN18, respectively. Notably, although LAME achieves second-best performance on RNXT29 and WRN40, it suffers from significant performance degradation on PARN18, resulting in an error rate of 79.10%. In contrast, BP-TTA maintains stable adaptation capabilities across all model architectures and achieves the best overall average performance.

4.5.2 Comparison on CIFAR100-C↩︎

We also conduct experiments on CIFAR100-C with different models, and the results are summarized in Table 9. Our method demonstrates an average mean error of 36.64%, 39.15%, and 49.32% in each model framework. Although RoTTA achieves the best performance on PARN18, our method still attains highly competitive results with only a marginal increase of 0.22%.

4.5.3 Comparison on ImageNet-C↩︎

We further evaluate our method on ImageNet-C with different models, and the results are presented in Table 10. As can be seen, TENT and CoTTA suffer from severe model collapse on all three models, resulting in extremely high error rates. Our proposed BP-TTA demonstrates outstanding performance across all variations, achieving the lowest average error rates of 66.77%, 64.53%, and 62.16% on RN50, WRN50, and RNXT50, respectively.

5 Conclusion↩︎

In this paper, we propose Balanced and Prototype-Guided Test-Time Adaptation (BP-TTA), a method designed for dynamic scenarios involving continual domain shifts and class-imbalanced data streams. Specifically, we introduce a Batch-Balanced Sampling (BBS) component that reconstructs the incoming test data stream by selecting samples from majority classes while supplementing minority classes, enabling more reliable adaptation. Furthermore, we proposed a Category Prototype-Guided Adaptation (CPGA) component that maintains dynamic class prototypes to provide stable and discriminative class-level guidance under continual domain shifts. Extensive experiments demonstrate that our method consistently outperforms existing methods and achieves state-of-the-art performance.

References↩︎

---

1. This work was supported by the Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China under Grant No. JYB2025XDXM101 and the Natural Science Foundation of Sichuan Province under Grant No. 2026NSFSC1488. (Corresponding author: Tao He, email: tao_he@scu.edu.cn).↩︎

2. Shaoyang Huang, Yashi Zhu, Yichen Yu, and Zhang Yi are with the College of Computer Science, Sichuan University, Chengdu, 610065, China. Tao He and Lei Zhang are with the School of Artificial Intelligence, Sichuan University, Chengdu, 610065, China.↩︎

3. Shaoyang Huang and Yashi Zhu are co-first authors and contributed equally to this study.↩︎