1 Introduction↩︎

Modern Deep Neural Networks (DNNs) are usually trained on a source distribution and tested on data assumed to follow a similar distribution. A central objective of classical machine learning research has been to improve generalization under mild distributional shifts, commonly referred to as covariate shift, where the input distribution \(P_X\) changes but \(P(Y \mid X)\) stays the same. Many works therefore focus on domain generalization [1], domain adaptation [2], and robustness to small input perturbations [3]. In contrast, under a semantic shift [4] (for example when a model trained on animals is tested on objects) the predictions become unreliable. In such cases, the goal is not to generalize but to detect the shift and raise an alert. Thus, models should be invariant to covariate shift but sensitive to semantic shift.

In this work, we argue that detecting covariate shift itself is also an important problem that has been less studied. In many real applications, knowing that the data distribution has changed is critical. For example, detecting AI-generated data often relies on identifying small distribution differences [5], [6]. Similarly, in finance [7], healthcare [8], or sensor systems [9], gradual covariate shifts can signal that model predictions may become unreliable. Existing approaches are predominantly fully supervised. For example, recent work such as DisCoPatch [10] trains a dedicated discriminator to distinguish between real and synthetic images. While effective, such methods require labeled examples from both distributions, limiting their applicability when negative samples are unavailable or expensive to curate. This raises a fundamental question:

We propose to address covariate shift detection through the lens of Positive–Unlabeled (PU) learning. This field studies binary classification when only labeled positive examples and unlabeled data are available. This paradigm naturally arises in anomaly detection [11], medical diagnosis [12], or out-of-distribution detection [13], where negative labels are scarce or unreliable. Surprisingly, to the best of our knowledge, PU learning has not been systematically investigated for covariate shift detection.

However, directly applying classical PU risk minimization in the presence of covariate shift is problematic. Standard PU methods rely on unbiased risk estimators that assume sufficient separability between positive and negative distributions. Under covariate shift, the distributions may overlap significantly, leading to small total variation distance and high intrinsic Bayes risk. In this regime, global risk estimation becomes unstable and prone to high variance.

We argue that in covariate-shifted PU settings, the negative distribution should not be estimated globally. Instead, it should be progressively discovered through its local geometric structure. Motivated by the manifold hypothesis, we treat the unlabeled dataset as a mixture of low-dimensional submanifolds corresponding to positive and shifted negative components. Rather than directly estimating risk differences, we iteratively uncover the negative manifold by propagating reliable local neighborhood information from positive anchor points.

To operationalize this idea, we introduce Spectral PU Neighborhood Annotation (S-PUNA), a geometry-aware pseudo-labeling framework for covariate shift detection as illustrated in Figure 1. S-PUNA initializes from labeled positives and iteratively expands both positive and negative anchor sets using confidence-controlled \(k\)-nearest neighbor propagation in feature space. Unlike classical \(k\)-NN-PU [14] that propagates only positive labels, S-PUNA symmetrically constructs both classes, progressively refining the decision boundary.

A key challenge in iterative pseudo-labeling is preventing semantic drift. We address this through a novel spectral entropy stopping criterion. By monitoring the eigenvalue distribution of the covariance matrix of the discovered negative set, we measure its intrinsic dimensionality and structural complexity. The iterative expansion stops once the spectral entropy stabilizes, indicating that the negative manifold has been sufficiently captured without invading overlapping regions.

Finally, we train a DNN classifier to distill the non-parametric neighborhood structure to maximize latent separation and distinguish positive from shifted components.

Our contributions are summarized as follows:

• We are the first to use PU for detecting changes in covariates and to provide theoretical insights on this new problem.

• We introduce S-PUNA, a geometry-aware framework that progressively uncovers the shifted distribution through local manifold expansion.

• We propose a principled spectral entropy stopping criterion that detects early contamination and prevents pseudo-labeling drift under covariate overlap.

• We provide theoretical insights for why spectral entropy signals help prevent annotation contamination.

• We build new covariate shift benchmarks and demonstrate state-of-the-art performance, approaching fully supervised methods, while showing strong transferability across diverse shifts.

2 Related Works↩︎

Covariate Shift Detection. While the problem is often included within broader discussions of general OOD detection [4], isolating and detecting these non-semantic shifts remains an under-explored challenge. Fully supervised strategies like DiscoPatch [10] require the unrealistic assumption that the specific covariate shifts are represented in the training set. Meanwhile, standard OOD algorithms like [15] and spatial anomaly detection methods like PatchCore [16] and SimpleNet [17] excel in semantic or localized tasks but prove ineffective for image-level covariate shift detection (see Appendix 12 for detailed results).

Positive-Unlabeled (PU) learning. PU learning enables binary classification using only labeled positives and an unlabeled mixture [12]. Modern approaches rely on disambiguation-free empirical risk estimators. For instance, nnPU [18] prevents overfitting by enforcing non-negative risk constraints, DCPU [19] handles biased or disjoint positive sets, and DistPU [20] mitigates negative-prediction bias by aligning predictions with the class prior through entropy minimization. PU formulations have also been applied to open-world OOD detection by treating nominal data as positives and deployment data as an unlabeled mixture [21], [22]. Most existing methods assume a semantic shift between the P and N, which simplifies the PU framework. While some works [23]–[25] address PU under covariate shift, they focus on classifier generalization between train and test sets rather than using the PU framework for shift detection.

3 Preliminaries and mathematical Insights↩︎

3.1 Distribution Shift Detection↩︎

Notations and preliminaries. Let \(\mathcal{X}\) be the set of all possible inputs (usually points in \(\mathbb{R}^d\), i.e. \(d\)-dimensional vectors). Let \(\mathcal{Y} = \{1, 2, \dots, K\}\) be the set of possible class labels (there are \(K\) different classes). During training we receive a dataset \(\mathcal{D}_{\mathrm{train}} = \{(x_1,y_1), (x_2,y_2), \dots, (x_n,y_n)\}\) where each pair \((x_i, y_i)\) is supposed to come from the same joint distribution \(P_{\mathrm{train}}(X,Y)\). We train a classifier \(f_\theta\) by minimizing the average training loss: \[\hat{\theta} = \arg\min_{\theta} \; \frac{1}{n} \sum_{i=1}^n \ell\bigl( f_\theta(x_i),\;y_i \bigr)\]

Here \(\ell(\cdot,\cdot)\) denotes a loss function. Classical learning theory assumes that future test samples \((X,Y)\) follow the same distribution: \(P_{\mathrm{test}}(X,Y) = P_{\mathrm{train}}(X,Y).\) In practice this assumption rarely holds, leading to distribution shift. Detecting and handling such shifts is essential for reliable machine learning systems. Two main types are commonly distinguished: Semantic shift and Covariate shift.

Semantic Shift occurs when the label distribution differs between training and test data, i.e., \(P_{\mathrm{test}}(Y) \neq P_{\mathrm{train}}(Y).\) This setting is closely related to Out-of-Distribution (OOD) detection, where test samples are drawn from a distribution \(P_{\mathrm{test}}\) defined on the same input space \(\mathcal{X}\) but involving unseen semantic classes. The OOD literature typically distinguishes two regimes [26]: Far OOD where the new classes are semantically very different from those seen during training, and Near OOD where they remain semantically similar. Near-OOD detection is generally more challenging due to the stronger semantic overlap with the training classes.

Covariate Shift refers to a change in the input distribution while the conditional labeling function remains unchanged: \(P_{\mathrm{test}}(X) \neq P_{\mathrm{train}}(X), \qquad P_{\mathrm{test}}(Y \mid X) = P_{\mathrm{train}}(Y \mid X).\) In this case, the optimal classifier is theoretically unchanged, but empirical risk minimization may become biased since the training data no longer reflect the marginal distribution encountered at test time. As with semantic shift, one can distinguish between Near Shift, where \(P_{\mathrm{test}}(X)\) differs only slightly from \(P_{\mathrm{train}}(X)\), and Far Shift corresponding to larger changes in input statistics. This distinction primarily reflects the difficulty of the shift rather than a strict quantitative measure in \(\mathcal{X}\).

3.2 Background and Mathematical Insights↩︎

3.2.0.1 Classical Binary Classification.

Let \((X,Y) \sim P\) be a random pair taking values in \(\mathcal{X}\times\{-1,+1\}\), where \(P\) denotes the joint data-generating distribution. We denote by \(\pi_p = \mathbb{P}(Y=+1)\), and \(\pi_n = \mathbb{P}(Y=-1)=1-\pi_p\) , the class priors, and by \(P^+ = P_{X|Y=+1}\) and \(P^- = P_{X|Y=-1}\) , the class-conditional distributions.

Given a hypothesis class \(\mathcal{H}\) of measurable functions \(f:\mathcal{X}\to\mathbb{R}\) and a loss function \(\ell:\mathbb{R}\times\{-1,+1\}\to[0,1]\), the population (true) risk of a classifier \(f\) is defined as \[R(f) = \mathbb{E}_{(X,Y)\sim P}[\ell(f(X),Y)].\] Using the law of total expectation, this can be decomposed as \[R(f) = \pi_p \, \mathbb{E}_{X\sim P^+}[\ell(f(X),+1)] + \pi_n \, \mathbb{E}_{X\sim P^-}[\ell(f(X),-1)]. \label{eq:true-risk-decomposition}\tag{1}\] In practice, \(P\) is unknown and we observe an i.i.d.training sample such that \(D=\{(x_i,y_i)\}_{i=1}^n \sim P^n.\)
The empirical risk is defined as \[\hat{R}_n(f) = \frac{1}{n} \sum_{i=1}^n \ell(f(x_i),y_i).\] Empirical Risk Minimization (ERM) selects \(\hat{f} \in \arg\min_{f\in\mathcal{H}} \hat{R}_n(f).\)

3.2.0.2 Positive–Unlabeled (PU) Classification

In PU learning, we do not observe labeled negative examples. Instead, we observe: a labeled positive set \(\mathcal{X}_P = \{x_i^p\}_{i=1}^{n_p} \sim (P^+)^{n_p}\), and an unlabeled set \(\mathcal{X}_U = \{x_j^u\}_{j=1}^{n_u} \sim (P_X)^{n_u}\), where the marginal distribution satisfies \[P_X = \pi_p P^+ + \pi_n P^-. \label{eq:marginal-mixture}\tag{2}\] Similarly to [27] a key observation is that the negative component of the risk can be recovered from the mixture structure. Indeed, by linearity of expectation and 2 , \[\begin{align} \mathbb{E}_{X\sim P_X}[\ell(f(X),-1)] &= \pi_p \mathbb{E}_{X\sim P^+}[\ell(f(X),-1)] + \pi_n \mathbb{E}_{X\sim P^-}[\ell(f(X),-1)]. \end{align}\] Rearranging yields: \[\pi_n \mathbb{E}_{X\sim P^-}[\ell(f(X),-1)] = \mathbb{E}_{X\sim P_X}[\ell(f(X),-1)] - \pi_p \mathbb{E}_{X\sim P^+}[\ell(f(X),-1)]. \label{eq:negative-recovery}\tag{3}\] Substituting 3 into 1 gives the unbiased PU risk: \[R_{pu}(f) = \pi_p \mathbb{E}_{X\sim P^+} \big[ \ell(f(X),+1)-\ell(f(X),-1) \big] + \mathbb{E}_{X\sim P_X} [\ell(f(X),-1)]. \label{eq:pu-risk}\tag{4}\] Importantly, \(R_{pu}(f)=R(f)\) for all \(f\), hence minimizing \(R_{pu}\) is equivalent to minimizing the true risk.

An empirical estimator of 4 is \[\hat{R}_{pu}(f) = \frac{\pi_p}{n_p} \sum_{i=1}^{n_p} \tilde{\ell}(f(x_i^p)) + \frac{1}{n_u} \sum_{j=1}^{n_u} \ell(f(x_j^u),-1),\text{ where}\] \[\tilde{\ell}(f(x)) = \ell(f(x),+1)-\ell(f(x),-1).\]

3.2.0.3 Rademacher Generalization Bound

Let \(\mathcal{H}\) be a class of functions taking values in \([0,1]\). We can define its empirical Rademacher complexity: \[\mathcal{R}_n(\mathcal{H}) = \mathbb{E}_\sigma \left[ \sup_{f\in\mathcal{H}} \frac{1}{n}\sum_{i=1}^n \sigma_i f(x_i) \right],\text{ where \sigma_i are i.i.d.\;Rademacher variables.}\]

A standard uniform convergence result (see, e.g., Theorem 3.1 of [28]) states that for any \(\delta>0\), with probability at least \(1-\delta\), \[\sup_{f\in\mathcal{H}} |R(f)-\hat{R}_n(f)| \le 2\mathcal{R}_n(\mathcal{H}) + \sqrt{\frac{\log(1/\delta)}{2n}}. \label{eq:rad-bound}\tag{5}\] This inequality provides a distribution-dependent control of the generalization gap.

Applying 5 separately to the positive and unlabeled samples yields, with probability at least \(1-\delta\), \[\begin{align} R_{pu}(f) \le \hat{R}_{pu}(f) &+ 2\pi_p \mathcal{R}_{n_p}(\mathcal{H}) + 2\mathcal{R}_{n_u}(\mathcal{H}) \nonumber\\ &+ \pi_p \sqrt{\frac{\log(1/\delta)}{2n_p}} + \sqrt{\frac{\log(1/\delta)}{2n_u}}. \label{eq:pu-generalization} \end{align}\tag{6}\] Thus, the excess risk is controlled by the complexity of the hypothesis class and the sample sizes \(n_p\) and \(n_u\).

3.2.0.4 Bayes Optimal Risk and Total Variation

Assume now balanced classes: \[\pi_p=\pi_n=\frac{1}{2}.\] Define the regression function \(\eta(x)=\mathbb{P}(Y=+1\mid X=x).\)

We can define the Bayes classifier, as in [29], by \(f^*(x)=\mathbf{1}\{\eta(x)\ge 1/2\},\) which minimizes \(R(f)\) over all measurable classifiers.

Its risk is \[R(f^*) = \mathbb{E}[\min(\eta(X),1-\eta(X))].\] Using the identity \[\min(a,b)=\frac{1}{2}(a+b-|a-b|),\] and the fact that \[\|P^+-P^-\|_{TV} = \frac{1}{2}\int |p^+(x)-p^-(x)|dx,\] one obtains the classical result from statistical decision theory: \[R(f^*) = \frac{1}{2} \left( 1-\|P^+-P^-\|_{TV} \right). \label{eq:bayes-tv}\tag{7}\]

Equation 7 shows that the intrinsic difficulty of the classification problem is entirely governed by the total variation distance between the class-conditional distributions. If the distributions overlap significantly, \(\|P^+-P^-\|_{TV}\) is small and the Bayes error is large. If they are well separated, \(\|P^+-P^-\|_{TV}\) approaches \(1\) and the Bayes error approaches \(0\).

3.3 Implications for Covariate Shift Detection↩︎

Under covariate shift, the marginal distribution of \(X\) changes while \(P(Y\mid X)\) remains fixed. When \(P^+\) and \(P^-\) differ substantially, the total variation term in 7 captures this shift directly.

Combining 6 with 7 , we obtain the high-probability bound \[\begin{align} R_{pu}(\hat{f}) \le \frac{1}{2} \left( 1-\|P^+-P^-\|_{TV} \right) + 2\pi_p \mathcal{R}_{n_p}(\mathcal{H}) + 2\mathcal{R}_{n_u}(\mathcal{H}) \nonumber\\ + \pi_p \sqrt{\frac{\log(1/\delta)}{2n_p}} + \sqrt{\frac{\log(1/\delta)}{2n_u}}. \end{align}\] This inequality highlights two sources of difficulty: Intrinsic difficulty, governed by \(\|P^+-P^-\|_{TV}\), which reflects distributional overlap and the estimation error, governed by Rademacher complexity and sample size.

In extreme classical shift scenarios used in PU, where \(P^+\) and \(P^-\) have nearly disjoint supports, \(\|P^+-P^-\|_{TV}\approx 1\), and the intrinsic Bayes error is small. However, in our case \(\|P^+-P^-\|_{TV}\approx 0\), hence the covariate shift might complicate all the training.

4 Spectral-PU Neighborhood Annotation (S-PUNA)↩︎

In the covariate shift regime, the class-conditional distributions \(P^+\) and \(P^-\) may exhibit substantial overlap in the ambient space, leading to a small total variation distance \(\|P^+ - P^-\|_{TV}\). As shown in Section 3.2, the Bayes risk scales as \(R(f^*) = \tfrac12(1-\|P^+ - P^-\|_{TV}),\) implying that when \(\|P^+ - P^-\|_{TV}\) is small, the intrinsic classification problem is difficult.

To mitigate this effect, we propose Spectral–PU Neighborhood Annotation (S-PUNA), a progressive, geometry-aware pseudo-labeling framework. Rather than estimating the negative distribution globally, S-PUNA exploits the local manifold structure of the unlabeled data to iteratively uncover regions that are statistically inconsistent with the positive anchor distribution. This results in a gradual expansion of a reliable negative support estimate. The Appendix 7 gives a deeper theoretical overview of the soundness of our proposed approach.

4.1 Iterative \(k\)-NN-Based Pseudo-Labeling↩︎

Our method is a two-stage process as illustrated in Figure 1: an initialization phase of the \(k\)-NN, followed by symmetric manifold expansion.

Let us consider that we have two sets of pseudo-annotations, one for positive and one for unlabeled negative data, which we denote as: \(\hat{\mathcal{X}}_{U,P}^{(t_0)} = \emptyset, \qquad \hat{\mathcal{X}}_{U,N}^{(t_0)} = \emptyset\) for the first iteration \(t_0\). Then for the next iterations \(t=t_0+1,\dots,T\), we maintain the current annotated sets: \[\mathcal{X}_P^{(t)} = \mathcal{X}_P \cup \hat{\mathcal{X}}_{U,P}^{(t-1)}, \qquad \mathcal{X}_N^{(t)} = \hat{\mathcal{X}}_{U,N}^{(t-1)}.\]

4.1.0.1 Baseline and Sets Initialization.

All samples features \(\phi(x)\) are extracted from the sixth block of a pre-trained Vision Transformer (ViT) [30]. This specific intermediate representation is chosen for its efficacy in capturing the geometric nuances of covariate shifts (see Appendix 10 for a comparative analysis of feature layer sensitivity). Our technique starts by constructing a \(k\)-nearest neighbor (\(k\)-NN) index on Positive Domain (PD) training features, \(\mathcal{X}_{train}\). For each sample \(x\) in the unlabeled pool \(\mathcal{X}_U\), we compute an initial distance score \(s_{orig}(x)\) as the minimum distance from the sample to its \(k\)-nearest neighbors in \(\mathcal{X}_{train}\). We then initialize the positive (PD) and negative (covariate) sets for the first iteration \(t_0\) by ranking the pool according to these scores: \[\hat{\mathcal{X}}_{U,c}^{(t_0)}=\begin{cases}\{x\in\mathcal{X}_U\mid x\text{ is among top }\alpha\text{ lowest }s_{orig}(\cdot)\},&\text{if }c=P\\\{x\in\mathcal{X}_U\mid x\text{ is among top }\alpha\text{ highest }s_{orig}(\cdot)\},&\text{if }c=N\end{cases}\] where \(\alpha\) is a set small number of features used to seed the initial pseudo-labeled sets. For subsequent iterations \(t=t_0+1, \dots, T\), the current annotated sets are maintained as: \(\mathcal{X}_P^{(t)} = \mathcal{X}_P \cup \hat{\mathcal{X}}_{U,P}^{(t-1)}, \qquad \mathcal{X}_N^{(t)} = \hat{\mathcal{X}}_{U,N}^{(t-1)} \qquad and \mathcal{X}_U^{(t)} = \mathcal{X}_U^{(t-1)} / (\hat{\mathcal{X}}_{U,N}^{(t)} \cap \hat{\mathcal{X}}_{U,P}^{(t)} ).\)

4.1.0.2 Neighborhood Search and Scoring.

At each iteration \(t\), we compute two separate distance metrics for each candidate sample \(x \in \mathcal{X}_U^{(t)}\):

1. \(d_{P}(x)\): the mean distance from the sample features \(\phi(x)\) to its \(k\)-nearest neighbors in the positive bank \(\mathcal{X}_P^{(t)}\).

2. \(d_{N}(x)\): the mean distance from the sample features \(\phi(x)\) to its \(k\)-nearest neighbors in the negative bank \(\mathcal{X}_N^{(t)}\).

Using these distances we calculate a score for ranking: \[s_t(x) = d_{ood}(x) - d_{id}(x).\]

4.1.0.3 Set Expansion.

The set is progressively expanded by ranking the candidates based on \(s_t(x)\). We select the top \(\beta\) samples for each class to be added to the respective sets: \[\hat{\mathcal{X}}_{U,c}^{(t)}=\hat{\mathcal{X}}_{U,c}^{(t-1)}\cup\begin{cases}\{x\mid x\text{ has top }\beta\text{ highest }s_t(x)\},&\text{if }c=P\\\{x\mid x\text{ has top }\beta\text{ lowest }s_t(x)\}.&\text{if }c=N\end{cases}\] \(\beta\) is a fixed parameter we set during the whole algorithm that controls the number of samples added to the sets at each iteration.

This iterative expansion differs from classical \(k\)-NN-based PU approaches, which typically only propagate positive labels. Here, both classes are progressively constructed, leading to a symmetric refinement of the decision boundary.

4.1.0.4 Geometric Interpretation.

Under the cluster assumption, points belonging to the same class lie on a low-dimensional manifold. As iterations proceed, the negative set expands along directions of maximal discrepancy from the positive support. This progressively increases the effective separation between the empirical supports of \(P^+\) and \(P^-\).

4.2 Spectral Entropy Stopping Criterion↩︎

A key challenge in iterative pseudo-labeling is preventing semantic drift. To detect when the discovered negative set has saturated its intrinsic structure, we monitor its spectral complexity.

Let \(\Sigma_t\) denote the empirical covariance matrix of \(\phi(x)\) for \(x\in\hat{\mathcal{X}}_{U,N}^{(t)}\): \[\Sigma_t = \frac{1}{|\hat{\mathcal{X}}_{U,N}^{(t)}|} \sum_{x\in\hat{\mathcal{X}}_{U,N}^{(t)}} (\phi(x)-\mu_t)(\phi(x)-\mu_t)^\top,\] where \(\mu_t\) is the mean of the features \(\phi(x)\) for \(x\in\hat{\mathcal{X}}_{U,N}^{(t)}\). Let \(\{\lambda_1^{(t)},\dots,\lambda_d^{(t)}\}\) be its eigenvalues normalized such that \(\sum_i \lambda_i^{(t)}=1\).

We define the spectral entropy: \[H(\Lambda_t) = -\sum_{i=1}^d \lambda_i^{(t)} \log \lambda_i^{(t)}.\]

4.2.0.1 Interpretation.

Spectral entropy quantifies the dispersion of variance across principal directions. Low entropy indicates concentration on a narrow manifold; high entropy reflects a richer, more isotropic structure. As the negative manifold is progressively uncovered, \(H(\Lambda_t)\) increases.

We stop when \(H(\Lambda_t) < H(\Lambda_{t-1}),\) indicating that the geometric expansion has stabilized. This prevents the algorithm from invading regions where \(P^+\) and \(P^-\) overlap, which would artificially decrease \(\|P^+ - P^-\|_{TV}\).

4.2.0.2 Theoretical Justification.

This stopping criterion prevents semantic drift. As formalized in Lemma 1, this inevitably disperses the mixture’s spectrum and strictly decreases spectral entropy.

The full proof is provided in Appendix 7.

4.3 Knowledge Distillation into Deep Networks↩︎

After convergence, we obtain the pseudo-labeled dataset \[\mathcal{D}_{anno} = \mathcal{X}_P^{(t)} \cup \hat{\mathcal{X}}_{U,N}^{(t)}.\]

We train a parametric classifier \(g_\omega\) on top of embeddings \(\phi(x)\), e.g., a multi-layer perceptron. The training objective is: \[\mathcal{L}(\omega) = -\frac{1}{|\mathcal{D}_{anno}|} \sum_{(x,\hat{y})\in\mathcal{D}_{anno}} \mathrm{CE}\big(g_\omega(\phi(x)),\hat{y}\big).\]

4.3.0.1 Why Distillation Helps.

The \(k\)-NN procedure is a non-parametric estimator with low bias but high variance. Distilling its pseudo-labels into a DNN induces a smoother decision boundary, reducing variance while preserving local geometry. Importantly, the learned DNN is optimized to keep the local geometry that comes from the \(k\)-NN, allowing for a good separation between positive and negative.

5 Experimental results↩︎

In this section, we evaluate our proposed method S-PUNA and test its robustness under severe covariate shift across diverse datasets. We benchmark S-PUNA against classical positive-unlabeled (PU) risk minimization estimators as well as neighborhood-based baseline methods. Complete implementation details and hyperparameters are provided in the Appendix 8.3. In Section 5.3, we analyze 21 methods using ResNet and ViT architectures to evaluate how supervised and unsupervised techniques perform in detecting covariate shift.

5.1 Datasets and Metrics↩︎

We evaluate our proposed method across three primary image classification benchmarks characterized by significant covariate shifts: ImageNet-1K [31], TinyImageNet [31], and EuroSAT [32].

ImageNet-1K Benchmarks. Using ImageNet-1K as the Positive Domain (PD) baseline, we test our approach on four shifted datasets. ImageNet-V2 [33] (near shift) replicates the original data collection process while introducing subtle statistical biases. ImageNet-R [34] (far shift) provides adversarial and rendered versions (e.g., paintings, sketches) of ImageNet classes, preserving semantic content while altering the visual domain. ImageNet-C (far shift)[3] applies 15 types of algorithmic perturbations, including noise, blur, and weather effects, across five severity levels. Finally, GenImage (near shift) [35] consists of ImageNet classes regenerated via various generative models, introducing synthetic distributional shifts.

TinyImageNet Benchmarks. For TinyImageNet (PD), we consider two shifted variants: TinyImageNet-C [3] (far shift), which applies corruptions similar to ImageNet-C, and TinyImageNet-V2 (near shift), a filtered version of ImageNet-V2 containing only the 200 overlapping classes.

EuroSAT Benchmarks. To demonstrate the versatility of our method beyond object-centric domains, we evaluate it on the EuroSAT satellite dataset (PD). We consider two shifted variants: EuroSAT-C(far shift) [36], which simulates synthetic atmospheric and sensor-based degradations, and a new dataset introduced in this study, EuroSAT-D(near shift). The latter consists of regenerated versions of EuroSAT images to address synthetic generative shifts. Further details on dataset construction are provided in Appendix 8.4.

Metrics and Evaluation. We report standard covariate shift detection metrics: Area Under the Receiver Operating Characteristic curve (AUROC), Area Under the Precision-Recall curve (AUPR), and the False Positive Rate at a 95% True Positive Rate (FPR95).

In our analysis, we distinguish between near-shift datasets, which exhibit moderate covariate shifts that are distributionally close to the PD data (and thus more challenging to detect), and far-shift datasets, which represent severe shifts that are more easily identifiable. We provide a comprehensive discussion on this taxonomy in Appendix 14.

5.2 Baselines and Implementation Details↩︎

For evaluation, images are projected into a dense feature space extracted from the sixth block of a ViT pre-trained on in-domain data. We select this specific intermediate layer as it captures structural nuances of covariate shifts while mitigating the task-specific overfitting prevalent in deeper layers (see Appendix 10).

We benchmark S-PUNA against several state-of-the-art PU learning methods, including Dist-PU [20], DC-PU [19], and saPU [37], all of which are optimized using the same ViT feature embeddings. Additionally, we compare our approach against pseudo-labeling strategies such as LaGAM [38] and a naive \(k\)-NN baseline [14] that propagates labels from the positive set.

For S-PUNA, the iterative expansion process terminates when the absolute difference in spectral entropy between consecutive steps, \(H(\Lambda_{t}) - H(\Lambda_{t-1})\), falls below the threshold of \(0.0\). An ablation study regarding all of the hyperparameters of S-PUNA is provided in Appendix 9. Upon convergence, the learned structural information is distilled into a MLP via standard cross-entropy training. Results using a ResNet architecture instead of the ViT is detailed in Appendix 13.

5.3 From supervised to unsupervised Covariate shift detection↩︎

To assess the effectiveness of various detection paradigms, we conduct an extensive benchmark in Appendix 12, evaluating 14 OOD detection methods, 5 anomaly detection methods, and 3 diffusion-based techniques across both ResNet and ViT architectures. We further compare these against two supervised approaches: DiscoPatch [10] and DRCT [39].

Our results indicate that supervised techniques achieve superior performance on near-shift scenarios, with an average AUROC of 89.8%. However, on far-shift datasets, these methods are outperformed by the majority of unsupervised techniques. Among the unsupervised approaches, logit-based methods such as ReAct [40] and ASH [41] yield the weakest results, suggesting that high-level class probabilities often discard the subtle structural cues necessary for robust detection. While feature-based methods like MDS [42], hybrid approaches like ViM [43] and diffusion based methods like DiffPath [44] show marginal improvements, these findings highlight that current OOD detection methods struggle on covariate shift when restricted to purely unsupervised internal signals.

5.4 Results↩︎

ImageNet. Table 1 summarizes the covariate shift detection performance on ImageNet-1K for both near and far shift scenarios.

Classical PU-based baselines exhibit a significant performance gap between the two regimes. For instance, Dist-PU improves from \(84.48\%\) AUROC in near-shift cases to \(96.70\%\) in far-shift cases, confirming that the latter is considerably easier to identify. However, its near-shift performance remains insufficient, with a high FPR95 of \(35.05\%\). Similar trends are observed for saPU and LaGAM, while DC-PU struggles across both regimes, particularly regarding FPR95 (\(85.80\%\) near, \(89.38\%\) far).

In contrast, S-PUNA consistently outperforms all baselines. Under near-shift conditions, our method achieves \(97.89\%\) AUROC—surpassing the strongest baseline (Dist-PU) by over \(13\%\)—while simultaneously reducing the FPR95 from \(35.05\%\) to \(9.69\%\) (a \(-25.3\%\) improvement). S-PUNA also achieves the highest AUROC under far-shift (\(98.51\%\)). On average, our approach yields \(98.20\%\) AUROC (a \(+7.61\%\) gain over Dist-PU) and reduces the average FPR95 to \(8.52\%\), outperforming the best baseline by nearly \(18\%\). Unlike prior methods, our approach maintains high stability across both shift types, demonstrating superior robustness.

TinyImageNet. On the TinyImageNet benchmark (Table 2), most baselines perform exceptionally well under near-shift conditions; for instance, Dist-PU achieves \(99.96\%\) AUROC and \(0.15\%\) FPR95. S-PUNA surpasses these results, reaching perfect near-shift performance (\(100.00\%\) AUROC, \(0.00\%\) FPR95). Our method also provides the strongest far-shift results (\(99.64\%\) AUROC, \(1.20\%\) FPR95), slightly improving upon the best-performing baseline, LaGAM (\(99.59\%\) AUROC, \(1.39\%\) FPR95). On average, S-PUNA consistently improves all metrics across the board compared to the state-of-the-art.

EuroSAT. Results on the EuroSAT dataset (Table 3) reveal a critical vulnerability in traditional PU methods. While several baselines achieve high AUROC under near-shift conditions (Dist-PU: \(99.78\%\)), their performance degrades substantially under far-shift scenarios—Dist-PU, for example, drops to \(89.46\%\) AUROC with a significant \(49.97\%\) FPR95. In contrast, S-PUNA achieves perfect separation in the far-shift regime (\(100.00\%\) AUROC, \(0.00\%\) FPR95), marking a \(+10.54\%\) AUROC gain and a \(-49.97\%\) reduction in FPR95 over the best baseline. Averaged across both regimes, S-PUNA reaches \(99.90\%\) AUROC and \(0.33\%\) FPR95, substantially outperforming LaGAM’s average metrics (\(92.79\%\) AUROC, \(28.92\%\) FPR95). These results confirm the superior robustness of our geometry-aware approach across varying shift severities. Please refer to Appendix 11 for transfer experiments and robustness across shifts.

6 Conclusion↩︎

In this work, we investigated the efficacy of various paradigms for covariate shift detection. Our analysis demonstrates that while fully supervised approaches perform reasonably well, traditional unsupervised covariate shift detection methods—relying on internal signals like logits or feature statistics fail significantly in the near shift regime. Because covariate shifts preserve semantic content and introduce only subtle distributional changes, these unsupervised signals are unable to reliably distinguish shifted samples from in-distribution data.

To bridge this gap, we proposed a weakly supervised alternative based on the Positive–Unlabeled (PU) learning framework. To mitigate the instability of classical PU risk estimators under significant distributional overlap, we introduced Spectral PU Neighborhood Annotation (S-PUNA). This geometry-aware method progressively discovers shifted samples through local neighborhood expansion within the feature manifold.

Extensive experiments across multiple benchmarks confirm that S-PUNA consistently outperforms existing PU baselines and even surpasses fully supervised models in several settings. Our findings suggest that strictly supervised signals are not a prerequisite for reliable covariate shift detection; rather, by effectively leveraging the intrinsic geometric structure of feature representations, weakly supervised PU learning can achieve highly robust and state-of-the-art performance.

7 Theoretical Justification of the Spectral Entropy Criterion↩︎

7.1 Contamination Model for the Annotated Negative Set↩︎

To justify the spectral entropy stopping rule, we propose Lemma 1 in the main paper. In this section, we analyze the evolution of the covariance structure of the progressively discovered negative set to prove this Lemma.

Assume that at iteration \(t\), the annotated negative set is composed of:

• true negatives drawn from \(\mathcal{N}(\mu_N,\Sigma_N)\) with probability \(1-\varepsilon\),

• contaminated positives drawn from \(\mathcal{N}(\mu_P,\Sigma_P)\) with probability \(\varepsilon\),

where \(\varepsilon \in [0,1]\) denotes the contamination level induced by pseudo-labeling errors.

The resulting population distribution is therefore the Gaussian mixture

\[X \sim (1-\varepsilon)\,\mathcal{N}(\mu_N,\Sigma_N) + \varepsilon\,\mathcal{N}(\mu_P,\Sigma_P)\]

7.1.0.1 Population covariance of the contaminated set.

Let \(\Delta\mu := \mu_P - \mu_N\). The covariance of a two-component Gaussian mixture satisfies the classical decomposition:

\[\Sigma(\varepsilon) = (1-\varepsilon)\Sigma_N + \varepsilon\Sigma_P + \varepsilon(1-\varepsilon)\Delta\mu\Delta\mu^\top.\]

The first two terms correspond to the within-class covariance, while the third term is the between-class covariance, which is positive semi-definite and of rank at most \(1\).

Define the normalized eigenvalues of \(\Sigma(\varepsilon)\): \[\rho_i(\varepsilon) = \frac{\lambda_i(\Sigma(\varepsilon))}{\operatorname{tr}(\Sigma(\varepsilon))}, \qquad \sum_{i=1}^d \rho_i(\varepsilon)=1.\]

The spectral entropy is defined as

\[S(\varepsilon) = -\sum_{i=1}^d \rho_i(\varepsilon)\log \rho_i(\varepsilon).\]

We now show that \(S(\varepsilon)\) strictly decreases as soon as \(\varepsilon>0\), which formally justifies entropy-based stopping.

7.2 Special Case: Isotropic Covariances↩︎

Assume \[\Sigma_N=\Sigma_P=\sigma^2 I_d.\]

Then

\[\Sigma(\varepsilon) = \sigma^2 I_d + \varepsilon(1-\varepsilon)\Delta\mu\Delta\mu^\top.\]

Let \[\alpha(\varepsilon) = \varepsilon(1-\varepsilon)\|\Delta\mu\|^2, \qquad u=\frac{\Delta\mu}{\|\Delta\mu\|}.\]

Then \[\Sigma(\varepsilon) = \sigma^2 I_d + \alpha(\varepsilon)uu^\top,\] which is a rank-1 perturbation of a scaled identity.

7.2.0.1 Eigenvalues.

By the spectral theorem for rank-1 perturbations,

\[\lambda_1(\varepsilon) = \sigma^2+\alpha(\varepsilon), \qquad \lambda_i(\varepsilon)=\sigma^2 \quad (i=2,\dots,d).\]

The trace equals

\[\operatorname{tr}(\Sigma(\varepsilon)) = d\sigma^2+\alpha(\varepsilon).\]

The normalized eigenvalues are therefore

\[\rho_1(\varepsilon) = \frac{\sigma^2+\alpha(\varepsilon)}{d\sigma^2+\alpha(\varepsilon)}, \qquad \rho_i(\varepsilon) = \frac{\sigma^2}{d\sigma^2+\alpha(\varepsilon)}.\]

7.2.0.2 Pure case.

If \(\varepsilon=0\), then \(\alpha(0)=0\) and

\[\rho_i(0)=\frac{1}{d} \quad \forall i,\]

which yields

\[S(0) = -\sum_{i=1}^d \frac{1}{d}\log\frac{1}{d} = \log d,\]

the maximal entropy over the probability simplex.

7.2.0.3 Contaminated case.

For any \(\varepsilon\in(0,1)\) with \(\|\Delta\mu\|>0\), we have \(\alpha(\varepsilon)>0\), implying

\[\rho_1(\varepsilon)>\frac{1}{d} > \rho_i(\varepsilon) \quad (i\ge2).\]

Hence \(\rho(\varepsilon)\) is non-uniform and strictly majorizes the uniform vector.

Since Shannon entropy is strictly Schur-concave (because \(-x\log x\) is strictly concave), strict majorization implies

\[S(\varepsilon)<S(0).\]

Moreover, \(S(\varepsilon)\) is continuous in \(\varepsilon\), and a first-order Taylor expansion around \(0\) shows

\[S(\varepsilon) = \log d - C\varepsilon + o(\varepsilon), \quad C>0,\]

so entropy decreases immediately at infinitesimal contamination.

7.2.0.4 Interpretation.

Even arbitrarily small contamination injects a directional variance component, concentrating spectral mass on the leading eigenvalue and reducing entropy. Thus entropy is maximized at purity and strictly decreases once semantic drift begins.

7.3 General Case: Arbitrary Covariances↩︎

Let \[A(\varepsilon) = (1-\varepsilon)\Sigma_N + \varepsilon\Sigma_P,\]

\[B(\varepsilon) = \varepsilon(1-\varepsilon)\Delta\mu\Delta\mu^\top \succeq 0.\]

Then

\[\Sigma(\varepsilon) = A(\varepsilon)+B(\varepsilon).\]

7.3.0.1 Eigenvalue monotonicity.

By Weyl’s monotonicity theorem for Hermitian matrices,

\[\lambda_i(\Sigma(\varepsilon)) \ge \lambda_i(A(\varepsilon)) \quad \forall i.\]

Since \(\operatorname{tr}(B(\varepsilon))>0\) for \(\varepsilon\in(0,1)\), the largest eigenvalue satisfies

\[\lambda_1(\Sigma(\varepsilon)) > \lambda_1(A(\varepsilon)).\]

Thus the spectrum of \(\Sigma(\varepsilon)\) becomes strictly more dispersed.

7.3.0.2 Majorization argument.

Let \(\tilde{\rho}(\varepsilon)\) be the normalized eigenvalues of \(A(\varepsilon)\) and \(\rho(\varepsilon)\) those of \(\Sigma(\varepsilon)\). Ky Fan inequalities imply that the partial sums of the top \(k\) eigenvalues strictly increase under addition of \(B(\varepsilon)\).

Therefore \(\rho(\varepsilon)\) strictly majorizes \(\tilde{\rho}(\varepsilon)\), and by strict Schur-concavity of Shannon entropy,

\[S(\Sigma(\varepsilon)) < S(A(\varepsilon)).\]

Since \(A(\varepsilon)\to\Sigma_N\) as \(\varepsilon\to0^+\), we obtain

\[S(\varepsilon) < S(0) \quad \text{for all sufficiently small } \varepsilon>0.\]

7.4 Finite-Sample Concentration↩︎

In practice, we observe the empirical covariance \(\hat{\Sigma}_t\). By matrix concentration inequalities (e.g., matrix Bernstein bounds), its eigenvalues converge uniformly to those of \(\Sigma(\varepsilon)\):

\[\|\hat{\Sigma}_t-\Sigma(\varepsilon)\| = O_p\!\left(\sqrt{\frac{d}{n}}\right).\]

Hence the empirical spectral entropy \(\hat{S}_t\) concentrates around \(S(\varepsilon)\), ensuring that entropy drop is detectable with high probability before contamination becomes large.

In both isotropic and general Gaussian settings, the mechanism is identical: the Shannon entropy strictly decreases when the N data are contaminated.

Therefore, spectral entropy is maximized when the negative set is pure and strictly decreases as soon as contamination begins.

This provides a rigorous geometric justification for using spectral entropy as a principled stopping criterion: it detects the earliest onset of semantic drift through spectral anisotropy in feature space.

8 Details on Baseline Implementations↩︎

All baselines are evaluated under the same detection protocol for covariate shift across three in-distribution datasets: ImageNet, Tiny-ImageNet, and EuroSAT.

For each in-distribution dataset, we first sample a class-balanced positive set \(P\) from the training split. We then construct an unlabeled set \(U\) by combining two sources of data: (i) positive samples taken from the validation split of the in-distribution dataset (ImageNet validation set, Tiny-ImageNet validation set, or EuroSAT validation set), and (ii) samples originating from the considered covariate-shift dataset. The remaining part of the validation split is used as the test set and is never observed during the training procedure.

All PU baselines are trained using a Multi-Layer Perceptron (MLP) composed of a single hidden layer with 512 neurons. The MLP operates on frozen feature embeddings extracted from a pre-trained backbone network.

We consider two types of feature extractors: a Vision Transformer (ViT) and a ResNet-50. In both cases, the backbone networks are pre-trained on the corresponding in-distribution dataset (ImageNet for ImageNet and Tiny-ImageNet experiments, and EuroSAT for EuroSAT experiments).

8.0.0.1 ViT features.

For the Vision Transformer, we extract the CLS token representation from block 6 of the transformer encoder. This produces a feature vector of dimension 768, which is used as the input embedding for the MLP classifier.

8.0.0.2 ResNet-50 features.

For ResNet-50, we extract intermediate convolutional features from the output of the second residual block group (also called layer2). The resulting feature map is then processed using global average pooling to produce a fixed-dimensional embedding of size 512, which is used as input to the MLP.

8.1 Metrics↩︎

Each method is evaluated using three different random seeds. We report three standard metrics commonly used in distribution shift detection [26]: AUROC, AUPR, and FPR95.

Given the test split, we compute the AUROC, which measures the ability of a model to distinguish between positive and negative samples across all possible decision thresholds.

We also report the AUPR, which evaluates the trade-off between precision and recall across thresholds.

Finally, we compute FPR95, defined as the false positive rate when the true positive rate reaches \(95\%\). Concretely, the shift detection threshold is set at the \((1 - 0.95)\) quantile of the shift (covariate or semantic) scores, and the false positive rate is then measured on the in-distribution samples. Lower FPR95 values indicate better detection performance.

8.2 Construction of \(P\) and \(U\)↩︎

For each experiment, both the Positive distribution (In-Distribution ID) and covariate shift datasets are shuffled according to the current random seed. We then sample examples from the ID and covariate shift datasets to construct the unlabeled set \(U\).

Across all experiments, \(U\) is constructed such that the class prior is equal to \(0.5\). More precisely,

\[U = U_{\mathrm{ID}} \cup U_{\mathrm{Covariate}}, \quad \pi = \frac{|U_{\mathrm{ID}}|}{|U_{\mathrm{ID}}| + |U_{\mathrm{Covariate}}|} = 0.5.\]

For each dataset, the number of samples used for \(P\), \(U_{\mathrm{ID}}\), and \(U_{\mathrm{Covariate}}\) is summarized in Table 4.

All remaining samples that are not used to construct \(U\) are used as test data for computing AUROC, AUPR, and FPR95.

For corrupted datasets (ImageNet-C, Tiny-ImageNet-C, and EuroSAT-C), we evaluate each corruption severity separately. For a given severity level, we pool all corruption types associated with that severity.

8.3 Hyperparameters↩︎

Unless otherwise stated, all PU baselines in the main paper are trained using ViT features extracted from the CLS token of the sixth transformer block of a ViT-B/16 backbone [30]. We use the same MLP for all the baselines and the same embedding, and exactly the same data.

8.3.0.1 Dist-PU.

We use 30 warm-up epochs followed by 150 PU training epochs for ImageNet, and 15 warm-up and 15 PU epochs for EuroSAT and Tiny-ImageNet. The training batch size is 2048, and the learning rate is \(10^{-4}\). The entropy, Mixup, and mixed-entropy coefficients are set to \(\mu=0.01\), \(\nu=2.0\), and \(\gamma=0.02\), respectively. The Mixup Beta parameter is \(\alpha=6.0\).

8.3.0.2 DC-PU.

We use the logistic loss and train for 30 epochs on ImageNet and 10 epochs on Tiny-ImageNet and EuroSAT. Each epoch consists of 200 iterations with batch sizes \(B_P = B_U = 512\). The learning rate is \(3 \times 10^{-3}\) with learning rate milestones at 6000 and 9000 iterations and a decay factor \(0.1\). The exponential moving average parameter is \(\beta=0.9\). The penalty parameter is \(\tau = 10^{-2}\), the rollback margin is \(\gamma = 0.7\), and the Lagrange multiplier \(\Theta\) is randomly initialized with standard deviation \(0.01\).

8.3.0.3 SAPU.

Training is performed for 50 epochs with 200 iterations per epoch, using batch sizes \(B_P = B_U = 512\) and \(n_{\mathrm{bag}} = 4\). We use the logistic loss and the AdamW optimizer with learning rate \(3\times10^{-4}\) and weight decay \(10^{-4}\).

8.3.0.4 LaGAM.

For meta-validation, we use 250 in-distribution samples and 250 shifted samples. Training is performed for 80 epochs on ImageNet and 50 epochs on EuroSAT and Tiny-ImageNet. The batch size is 1024, and the meta-batch size is 256. Optimization is performed with AdamW using a learning rate \(3\times10^{-4}\) and weight decay \(10^{-4}\). The EMA coefficient for pseudo-label updates is linearly scheduled from \(0.95\) to \(0.8\). The meta-update is applied to the layer labeled classifier. The auxiliary Mixup loss uses weight \(1.0\) and Beta parameter \(\alpha=4.0\).

8.4 EuroSAT and Variant Datasets↩︎

The EuroSAT dataset [32] is a widely used benchmark for land use and land cover classification. It is built from open-access Earth observation data collected by the Copernicus Sentinel-2 satellite. The dataset contains approximately 27,000 labeled and geo-referenced image patches, each covering a \(64 \times 64\) pixel area with a spatial resolution of 10 meters per pixel.

In our experiments, we use the RGB version of EuroSAT, which consists of the red, green, and blue spectral bands encoded as standard JPEG images. This format makes the dataset compatible with standard deep learning architectures used in computer vision.

The dataset contains ten classes: Annual Crop, Forest, Herbaceous Vegetation, Highway, Industrial Buildings, Pasture, Permanent Crop, Residential Buildings, River, and Sea/Lake.

8.4.1 EuroSAT-C↩︎

We use EuroSAT-C [36] to evaluate robustness under corrupted input conditions. This dataset is derived from the EuroSAT test set by applying a set of synthetic corruptions including algorithmic noise, blur, simulated weather effects, and digital artifacts. Each corruption is applied at multiple severity levels.

These perturbations are designed to mimic realistic degradations encountered in satellite imagery, such as atmospheric disturbances or sensor noise. Evaluating models on EuroSAT-C therefore provides a systematic assessment of robustness to domain shifts in remote sensing scenarios.

8.4.2 EuroSAT-D↩︎

We also introduce EuroSAT-D, a synthetic dataset generated using a Stable Diffusion model fine-tuned on the EuroSAT RGB training set with Low-Rank Adaptation (LoRA).

To ensure semantic consistency and visual quality, generated images are filtered using a dual-model validation pipeline. We use a fully supervised ViT-Large model pretrained on ImageNet and fine-tuned on EuroSAT classification. Only generated samples for which both networks predict the target class with confidence greater than \(0.7\) are retained.

For the filtering step, the ViT-Large model is fine-tuned on \(224 \times 224\) inputs for 20 epochs with batch size 64 and a \(0.2\) validation split. Training uses a cosine annealing learning rate schedule with a 3-epoch warmup, base learning rate \(10^{-4}\), and weight decay \(10^{-4}\).

8.5 Results across Datasets↩︎

Across all datasets, S-PUNA consistently outperforms existing PU-learning baselines and approaches the performance of fully supervised classifiers. On the ImageNet benchmark (Table 5), classical PU methods such as Dist-PU, saPU, DC-PU, and LaGAM achieve moderate detection performance, with average AUROC values ranging between \(65.56\) and \(90.59\). In contrast, S-PUNA significantly improves this performance, reaching an average AUROC of \(98.20\) when the classifier is used. This result reduces the gap with the supervised classifier (AUROC \(98.61\)) to less than \(0.5\) points, while operating under the weaker PU supervision setting. A similar trend can be observed for AUPR and FPR95, where S-PUNA drastically reduces the false positive rate compared to existing PU methods.

A key observation is the strong performance of S-PUNA across different types of distribution shifts. In particular, the method remains robust on both near shifts (e.g., ImageNet-V2 or EuroSAT-D) and far shifts (e.g., ImageNet-C or EuroSAT-C). For example, on GenImage and ImageNet-C, S-PUNA achieves AUROC values above \(99\), demonstrating that the proposed approach effectively separates shifted samples even under severe covariate shifts. This stability across shifts contrasts with several PU baselines whose performance varies significantly depending on the dataset.

The comparison between the two variants of our method further highlights the benefit of the final classifier distillation step. The variant without classifier (S-PUNA w/o C) already provides strong performance, outperforming most PU baselines on several datasets. However, adding the classifier consistently improves the results, especially on challenging near-shift scenarios. For instance, on ImageNet-V2, the AUROC increases from \(92.65\) to \(96.38\), and the FPR95 drops from \(33.70\) to \(17.14\). Similar improvements are observed on EuroSAT-D, where the classifier significantly boosts AUROC from \(84.40\) to \(99.79\). This confirms that the distillation step helps transform the non-parametric neighborhood structure discovered during the PU phase into a smoother and more robust decision boundary.

Finally, results on Tiny-ImageNet (Table 2) and EuroSAT (Table 3) confirm the strong generalization of S-PUNA across datasets of different sizes and domains. In particular, S-PUNA often matches or surpasses the best PU baselines while remaining very close to the supervised upper bound. These results suggest that exploiting local neighborhood expansion within the PU framework allows the model to progressively uncover the negative distribution and produce reliable shift detection even with limited labeled supervision.

2pt

                                                        | **ImageNet-C**                                                                             |

                                                                                                                      |

9 Ablation Study↩︎

Table 8 analyzes the sensitivity of S-PUNA to its main hyperparameters on the ImageNet-R benchmark. Overall, the results indicate that the proposed method is robust to most hyperparameter choices, with only a few parameters having a significant impact on performance.

9.0.0.1 Effect of contamination tolerance \(\epsilon\).

The contamination tolerance parameter \(\epsilon\) controls how much label noise is tolerated during the pseudo-labeling process. Interestingly, the results show that this parameter has virtually no influence on the final performance. For \(\epsilon \in \{0, 0.05, 0.1\}\), the detection metrics remain identical, reaching \(97.34\%\) AUROC, \(98.49\%\) AUPR, and \(13.37\%\) FPR95. This behavior suggests that the contamination score estimated by S-PUNA naturally converges toward small values, making the method insensitive to small variations of \(\epsilon\). In practice, this indicates that the algorithm is able to control contamination in the pseudo-labeling process without requiring precise tuning of this parameter.

9.0.0.2 Effect of expansion parameter \(\beta\).

The parameter \(\beta\) controls the number of samples that are symmetrically added to the positive and shifted sets during each iteration of the algorithm. Unlike \(\epsilon\), this parameter has a noticeable impact on the detection performance. Increasing \(\beta\) progressively improves the results, with AUROC increasing from \(93.78\%\) for \(\beta=500\) to \(97.34\%\) for \(\beta=1500\), while FPR95 decreases from \(31.92\%\) to \(13.37\%\). This behavior suggests that larger expansion steps allow the algorithm to more effectively explore the local neighborhood structure of the feature space and progressively uncover the shifted distribution. However, the improvement saturates beyond \(\beta=1500\), indicating that excessively large expansions do not bring additional benefits.

9.0.0.3 Effect of neighborhood size \(k\).

The neighborhood size \(k\) determines the number of nearest neighbors used to estimate local structure during the manifold expansion stage. The results show that S-PUNA remains highly stable across a wide range of values. For \(k \in \{100, 300, 500\}\), AUROC remains close to \(97\%\) and FPR95 varies only slightly between \(13.37\%\) and \(15.50\%\). This stability suggests that the proposed geometric expansion strategy is robust to moderate variations in the local neighborhood size, which simplifies the practical use of the method.

9.0.0.4 Effect of initialization size \(\alpha\).

The parameter \(\alpha\) defines the number of samples used to initialize the positive and shifted sets before the iterative expansion process begins. The results indicate that S-PUNA is stable for moderate initialization sizes (\(\alpha=30\) or \(50\)). However, performance degrades significantly when \(\alpha\) becomes too large. In particular, setting \(\alpha=100\) leads to a substantial drop in performance, with AUROC decreasing to \(72.45\%\) and FPR95 increasing to \(89.42\%\). This degradation can be explained by the fact that overly large initial sets introduce noisy samples early in the process, which propagates errors during the neighborhood expansion phase. Therefore, smaller initialization sizes provide a cleaner starting point and lead to more reliable expansion dynamics.

4pt

10 Details on ViT layer selection and covariate shift↩︎

To determine which representation layer of the Vision Transformer (ViT-B/16) provides the most suitable features for covariate shift detection, we evaluate the CLS token extracted from each transformer block. The analysis relies on three complementary clustering metrics: the Silhouette score (higher is better), the Davies–Bouldin index (lower is better), and the Calinski–Harabasz index (higher is better).

For each experiment, we randomly sample 10,000 images from the ImageNet validation set and 10,000 images from either a covariate shift dataset (ImageNet-C, ImageNet-R, ImageNet-V2, GenImage) or a semantic shift dataset (OOD). We then compute clustering scores by comparing the feature representations of these two groups across all transformer blocks.

For clarity, we group the transformer layers into three regions: early blocks (before block 4), middle blocks (between blocks 4 and 7), and late blocks (after block 7). Figure 4 (a) shows that the best separation between distributions consistently occurs in the middle layers of the transformer.

Early layers mainly capture low-level visual features such as textures, colors, or simple edges. As a result, they are not well suited for detecting distribution shifts that occur at a higher semantic level, since many datasets share similar low-level statistics. On the other hand, the last layers of the transformer become highly specialized for the classification task. These representations strongly focus on semantic class information learned during training and may ignore subtle variations in the input distribution. Consequently, they are less effective for detecting covariate shifts that modify the input statistics without changing the semantic labels. The middle layers provide a better compromise: they capture intermediate-level structures that still preserve information about the input distribution while remaining sufficiently discriminative. Hence, across the three clustering metrics, the middle blocks (approximately layers 3–7) consistently obtain the most favorable scores. Based on this observation, we select block 6 for all experiments, as it provides a strong trade-off between distribution separability and representation stability. We performed similar experiments on ResNet-50 to select the correct representation.

Another important observation is that semantic shift datasets exhibit stronger separation from ImageNet than covariate shift datasets across most layers. This behavior is expected because semantic shifts introduce larger distribution changes, typically corresponding to different object categories. In contrast, covariate shifts preserve the class semantics while modifying only the input statistics, which makes them more difficult to detect in the feature space.

11 Generalization of results↩︎

We evaluate how well a detector trained on one type of distribution shift generalizes to other shifts. Table 9 reports AUROC and FPR95 when training on a single shifted dataset and testing on different ones.

Training on ImageNetV2 shows strong transfer to synthetic shifts, achieving 91.04 AUROC on GenImage with a very low 0.36 FPR95. However, performance drops on corruption-based shifts such as ImageNet-C (70.01 AUROC, 69.29 FPR95).

Similarly, training on ImageNet-R generalizes reasonably well to GenImage (87.33 AUROC), but remains weaker on ImageNet-C (68.16 AUROC).

Detectors trained on ImageNet-C exhibit poor cross-shift generalization overall, with AUROC dropping to 56.80 on GenImage and 62.79 on ImageNetV2, suggesting that corruption-based shifts do not transfer well to other types of distribution shift.

Finally, training on GenImage transfers well to other distribution shifts, achieving 89.69 AUROC on ImageNetV2 and 81.72 AUROC on ImageNet-R, while performance remains moderate on ImageNet-C (72.57 AUROC).

Overall, GenImage achieves the strongest cross-shift generalization, with the highest average AUROC of 81.33, followed by ImageNetV2 (80.69) and ImageNet-R (78.22). In contrast, detectors trained on ImageNet-C perform substantially worse, with an average AUROC of only 63.59 and the highest average FPR95 of 82.66.

2pt

12 Unsupervised covariate shift detection↩︎

To better understand how difficult it is to detect covariate shift in large-scale vision models, we evaluate several existing unsupervised shift detection approaches. The objective of this section is to study whether covariate shift can be reliably detected in a fully unsupervised setting, that is, without having access to labeled shifted data.

To design our evaluation protocol, we rely on the literature on generalized out-of-distribution (OOD) detection [4], [45]. This line of work studies how models behave when the test distribution differs from the training distribution and proposes a taxonomy of distribution shifts depending on where the change occurs in the joint distribution \(P(X,Y)\).

In practice, two main types of shifts are commonly considered. Semantic shift corresponds to situations where new classes or new semantic concepts appear at test time. In contrast, covariate shift refers to a change in the input distribution while the underlying labeling rule remains the same. In other words, the marginal distribution of inputs changes but the relationship between inputs and labels stays constant.

In this section, we focus on unsupervised approaches that do not require any labeled shifted data. These approaches can be divided into two main families:

• OOD detection methods, which compute a score from a trained classifier in order to determine whether a sample deviates from the training distribution.

• Anomaly detection methods, which attempt to model the distribution of normal data and identify samples that deviate from it.

12.1 OOD detection methods for unsupervised covariate shift detection↩︎

OOD detection methods aim to identify samples that do not follow the training distribution by computing a detection score from the outputs or the internal representations of a trained classifier. Typical approaches rely on confidence scores, energy-based scores, feature-space distances, or activation manipulation techniques.

These methods can be divided into two categories. Methods by design modify the training procedure of the classifier, for example, by introducing additional losses, confidence regularization, or auxiliary datasets containing outliers. Although these approaches can improve OOD detection performance, they require retraining the model and often rely on additional data.

In contrast, post-hoc methods operate on a pretrained model without modifying its training procedure. They compute an OOD score directly from the model outputs or intermediate features and can therefore be applied in a fully unsupervised way at test time. Because they are simple to use and do not require retraining the model, post-hoc methods are widely used in practice.

In our experiments, we focus on unsupervised post-hoc OOD detection methods. Most of the evaluated techniques are implemented using the OpenOOD benchmark framework [26], which provides standardized implementations of many state-of-the-art OOD scoring methods. In addition, we also evaluate several recent diffusion-based approaches that compute detection scores using generative models, as well as one supervised baseline for reference.

Table 10 presents the results obtained on four datasets using ViT-B/16 features. Several observations can be made from these results. First, unsupervised post-hoc methods perform poorly on near covariate shifts. For datasets such as ImageNet-V2 and GenImage, which are relatively close to the ImageNet training distribution, most methods obtain AUROC values between \(54\) and \(59\). This performance is close to random detection and indicates that these methods struggle to identify small changes in the input distribution.

Second, performance improves when the shift becomes larger. For example, on ImageNet-R, which introduces stronger visual changes through artistic renditions of objects, several feature-based methods achieve significantly higher scores. Methods such as ViM, MDS, and KNN reach AUROC values between \(85\) and \(87\), suggesting that distance-based representations in feature space can capture stronger distribution changes.

Third, the results on ImageNet-C show that corruption-based shifts are moderately detectable. Most methods obtain AUROC values around \(74\)–\(76\), which is better than near shifts but still far from perfect detection. This suggests that low-level perturbations such as noise, blur, or compression alter the input statistics enough to be partially detected but remain difficult to separate completely.

The results on GenImage highlight another limitation of these approaches. Despite the synthetic nature of this dataset, the AUROC values remain close to \(55\)–\(60\) for most methods, showing that generative images can remain close to natural images in feature space and therefore remain difficult to detect.

Overall, these results suggest that unsupervised post-hoc OOD detection methods are not well suited for detecting small covariate shifts. They tend to succeed only when the distribution change is sufficiently large.

Finally, the supervised baseline DisCoPatch achieves much higher performance on several datasets, for example, reaching AUROC values above \(90\) on ImageNet-V2 and GenImage. This result highlights the gap between supervised and unsupervised detection methods.

12.2 Anomaly detection methods for unsupervised covariate shift detection↩︎

Anomaly detection methods are another family of techniques that can potentially detect covariate shift. As discussed in the generalized OOD detection literature [4], [45], anomaly detection typically assumes a setting where only normal data are available during training, and the objective is to detect samples that deviate from this notion of normality at test time. Since covariate shift corresponds to a change in the input distribution, anomaly detection techniques are natural candidates for identifying such changes. In our evaluation we consider a broad set of anomaly detection methods.

Classical one-class and density-based methods such as OneClassSVM [58], LOF [59], ABOD [60], ECOD [61], and COPOD [62] estimate normality from the geometric or statistical structure of the feature space. Tree-based methods such as IsolationForest [63] detect anomalies using recursive partitioning and path-length statistics. Deep-feature statistical models such as DFM [64] and PaDiM [65] first extract deep features and then model their distribution in order to identify deviations. Memory-based and flow-based deep anomaly detectors such as PatchCore [16], FastFlow [66], and CFA [67] rely on nearest-neighbor matching, likelihood estimation, or feature transformation in order to capture more complex shifts. Finally, SimpleNet [17] learns a lightweight discriminator that separates normal features from noise-augmented features and uses the resulting score as an anomaly signal.

Table 11 presents the performance of these methods. In all methods, we use a ViT-B/16 as encoder for the feature extraction. Overall, the results show that classical anomaly detection methods also struggle to detect near covariate shifts. On ImageNet-V2, most methods obtain AUROC values close to \(50\), which corresponds to random performance. This indicates that small changes in the input distribution are extremely difficult to detect using standard anomaly detection techniques.

Performance improves slightly for stronger shifts. For example, on ImageNet-C and ImageNet-R several methods achieve AUROC values above \(80\). In particular, PaDiM, DFM, and CFA perform relatively well on these datasets, reaching AUROC values around \(84\)–\(86\) on ImageNet-C. This suggests that modeling the distribution of deep features can help capture larger changes in the input statistics.

However, performance remains inconsistent across datasets. For example, many methods perform reasonably well on ImageNet-C but fail on GenImage, where AUROC values remain around \(55\)–\(62\). This again highlights the difficulty of detecting subtle shifts when the feature distributions remain close.

In summary, both OOD detection methods and anomaly detection methods show limited ability to detect covariate shift in a fully unsupervised setting. Their performance improves only when the shift becomes large, while near shifts remain extremely difficult to identify.

13 ResNet Results↩︎

Table 12 reports the performance of S-PUNA using ResNet50 features across four ImageNet shifted datasets. We compare our approach with the weakly supervised positive-unlabeled (PU) learning baselines, and for reference, we also report a fully supervised classifier, which represents an upper bound since it relies on labeled supervision unavailable to PU methods.

Across all datasets, S-PUNA substantially outperforms existing PU approaches. On ImageNet-V2, the best PU baseline (Dc-PU) reaches \(55.72\%\) AUROC, whereas S-PUNA achieves \(84.64\%\) AUROC together with a large reduction of FPR95 to \(64.26\%\). This corresponds to an improvement of \(+28.92\) over the strongest PU baseline. On ImageNet-R the strongest PU baseline (LaGAM) obtains \(80.92\%\) AUROC while S-PUNA improves performance to \(88.76\%\) AUROC and reduces FPR95 from \(82.03\%\) to \(56.08\%\) corresponding to a gain of \(+25.95\). On GenImage S-PUNA also provides significant gains. While LaGAM reaches \(80.12\%\) AUROC, S-PUNA increases performance to \(93.72\%\) AUROC and lowers FPR95 to \(31.08\%\), yielding a large improvement of \(+13.60\) on AUROC and \(+42.48\) on FPR95. Finally, on ImageNet-C, S-PUNA nearly matches the supervised upper bound. It achieves \(99.34\%\) AUROC with \(2.59\%\) FPR95, outperforming all PU baselines and approaching the supervised classifier (\(99.36\%\) AUROC).

Overall, S-PUNA consistently delivers the strongest performance among PU methods. Averaged across datasets, it reaches \(91.62\%\) AUROC compared to \(77.48\%\) for the best competing PU baseline (LaGAM), corresponding to an improvement of \(+14.14\) while also reducing the average FPR95 from \(67.87\%\) to \(38.50\%\). Although the supervised classifier remains an upper bound (\(93.55\%\) AUROC), S-PUNA significantly narrows the gap while relying only on weak supervision.

2pt

                                                        | **ImageNet-C**                                                                             |

                                                                                                                      |

14 Near and Far Covariate Shift↩︎

In real-world deployments, test data often deviates from the training distribution; we refer to this mismatch broadly as distribution shift. Following the generalized OOD viewpoint, shifts can be described by where the change is manifested under the joint distribution \(P(X,Y)\), namely the input space \(X\) versus the label space \(Y\) [4]. In our setting, Covariate Shift primarily alters the input marginal \(P(X)\) while keeping the label space fixed, whereas Semantic Shift is associated with changes in label semantics or effective class composition, which typically induces changes in \(P(X)\) [4].

14.0.0.1 Near vs. Far Covariate Shift.

To further characterize the difficulty of detecting covariate shifts, we distinguish between near and far covariate shifts according to the distance between the input distributions \(P_{\text{ID}}(X)\) and \(P_{\text{Shift}}(X)\). Let \(d(\cdot,\cdot)\) denote a distance between distributions We define the magnitude of the covariate shift as \[\Delta_X = d\!\left(P_{\text{ID}}(X),\, P_{\text{Shift}}(X)\right).\]

A Near Covariate Shift corresponds to situations where \(\Delta_X\) is small, meaning that the shifted samples remain close to the training distribution in feature space. These cases typically arise from mild domain perturbations such as small changes in acquisition conditions or dataset re-sampling.

In contrast, a Far Covariate Shift occurs when \(\Delta_X\) becomes large, indicating that the shifted distribution lies further away from the training manifold. Such shifts often arise from strong corruptions, domain changes, or stylistic transformations that significantly alter the input statistics while preserving the semantic labels.

14.0.0.2 Empirical analysis of covariate shift magnitude.

To quantify these effects, Table 13 reports feature-space distances between ImageNet validation samples and several shifted datasets using embeddings extracted from a ViT backbone. Distances are computed using both cosine distance and mean \(L_2\) distance at two representation levels: an intermediate layer (block6) and the final layer (last).

Several observations can be drawn from these results. First, ImageNet-V2 consistently exhibits the smallest distances to the ImageNet validation set, with cosine distance \(0.0041\) and \(L_2\) distance \(0.0793\) at block6. This indicates that ImageNet-V2 remains extremely close to the original distribution and therefore constitutes a typical near covariate shift. Similarly, GenImage also remains relatively close to the ImageNet distribution, with moderate distances at block6 (cosine distance \(0.0144\)), suggesting another form of near-to-moderate shift.

In contrast, ImageNet-R produces significantly larger feature-space deviations, with cosine distance increasing to \(0.0366\) at block6 and reaching \(0.3440\) at the final layer. This indicates that stylized renditions of ImageNet classes create a much stronger distributional deviation, making ImageNet-R representative of a far covariate shift.

The strongest shifts are observed for ImageNet-C, where the distance from the training distribution increases with the corruption severity level. For instance, at block6 the cosine distance increases from \(0.0710\) at severity \(1\) to \(0.0735\) at severity \(5\), while the \(L_2\) distance rises from \(0.3309\) to \(0.3329\). At the final representation layer, this effect becomes even more pronounced, with cosine distance reaching \(0.3829\) at severity \(5\). These results indicate that ImageNet-C constitutes a progressively stronger far covariate shift as the corruption severity increases.

Overall, this analysis allows us to categorize the considered datasets according to the magnitude of the covariate shift:

• Near covariate shifts: ImageNet-V2 and GenImage.

• Far covariate shifts: ImageNet-R and ImageNet-C (all severity levels).

This distinction is important because near shifts are typically more difficult to detect: since the shifted samples remain close to the training manifold, classical distributional detectors often fail to identify them reliably. In contrast, far shifts generate stronger deviations in feature space and are therefore easier to detect. Our experiments demonstrate that S-PUNA remains effective in both regimes, including the particularly challenging near-shift scenario.

6pt

References↩︎