1 Introduction↩︎

With the rapid growth of social media and online sharing platforms, an increasing number of individuals are sharing images from their daily lives. While this enhances connectivity and information exchange, it also raises the risk that personal images may be repurposed to train commercial models without their owners’ consent. As deep learning methods [1]–[3] increasingly rely on massive datasets [4] to achieve stronger performance, the potential for privacy breaches has become a pressing concern. To mitigate this risk, the task of generating Unlearnable Examples (UEs) has been proposed, where imperceptible perturbations are added to images such that models fail to extract correct semantic information from them [5]–[7].

Despite recent progress, we show that existing UE defenses can be easily circumvented. Specifically, our empirical analysis demonstrates that simply applying a low-pass filter to protected data can substantially undermine their unlearnability (Figure 1), indicating that the generated perturbations depend predominantly on high-frequency components. Once these are suppressed, models can still extract semantic information from the residual low-frequency content, ultimately leading to privacy leakage. These results expose a fundamental vulnerability of current approaches and suggest that their protective capability has been overestimated.

In this paper, we argue that a key requirement for reliable UEs is to remain effective under spectral filtering, which in turn demands that the perturbations themselves function across the entire spectrum so that no semantic information can be extracted from any band. Otherwise, models may still exploit residual cues in bands left unaffected by perturbations. If perturbations act only on high-frequency regions, low-frequency structures such as object shapes remain informative and continue to support learning. Conversely, when perturbations concentrate on low frequencies, fine-grained details in the high-frequency range can still be exploited. This insight motivates a spectrum-agnostic design that regulates perturbation effects toward a balanced distribution across the spectrum, thereby preserving unlearnability even when certain frequency bands are suppressed by filtering.

To this end, we propose Full-spectrum Unlearnable examples via Spectral Equalization (FUSE), a novel method explicitly designed to generate spectrum-agnostic perturbations that maintain unlearnability even under spectral filtering. Specifically, FUSE incorporates two complementary components. (i) Random Spectral Masking (RSM), introduces randomized spectral band suppression during training to simulate diverse distortions. This mechanism ensures perturbations to remain effective regardless of which frequencies are suppressed. (ii) Cross-Band Guidance (CBG), further boosts unlearnability and utility of UEs by mutual guidance between low- and high-frequency signals, enhancing their consistency. This mechanism allows low-frequency components to inherit unlearnability from high frequencies and high-frequency components to preserve semantic fidelity from low frequencies, ensuring that both collectively retain complementary strengths. Together, RSM and CBG enable FUSE to generate spectrum-agnostic perturbations that effectively degrade semantic learnability, thereby offering reliable privacy protection across diverse scenarios.

Our main contributions are summarized as follows:

• We identify a critical vulnerability of existing unlearnable examples, showing that their effectiveness collapses under low-pass filtering.

• We propose FUSE, a novel spectrum-agnostic framework that distributes perturbation effects across the entire frequency range, preventing residual cues and preserving unlearnability even when specific bands are suppressed by filtering.

• Through extensive experiments across datasets, architectures, filtering scenarios, and defenses, we demonstrate that FUSE consistently outperforms prior approaches, achieving strong unlearnability while maintaining imperceptibility.

2 Related Work↩︎

Unlearnable examples (UEs) have recently emerged as an effective paradigm for safeguarding data privacy by adding crafted perturbations to training samples. The key idea is to render data unusable for model training while preserving visual imperceptibility, thereby preventing unauthorized exploitation of sensitive information. Models trained on UEs typically memorize spurious patterns introduced by perturbations rather than learning genuine semantic features, which results in severe performance degradation when evaluated on clean test data [5], [6].

Existing approaches to generating UEs generally focus on designing perturbations that mislead models into relying on spurious patterns rather than true semantic information. Some works achieve this by embedding signals that dominate the learning process and obscure meaningful features [5], [7], [9]–[13]. Others introduce perturbations that deliberately push samples toward incorrect decision regions, causing classifiers to associate images with misleading labels [14]–[16]. More recently, model-independent strategies have emerged, where perturbations are constructed without the use of surrogate models, for example by leveraging random convolutions or enforcing linear separability across classes [8], [17], [18]. While these methods provide effective protection in standard training scenarios, they generally overlook the impact of frequency characteristics, causing their unlearnability to diminish substantially under low-pass filtering. Beyond UEs, several recent works [19] underscore the importance of frequency-domain analysis, complementing our motivation for developing a spectrum-agnostic UE method.

3 Method↩︎

In this section, we present Full-spectrum Unlearnable Examples via Spectral Equalization (FUSE), a spectrum-agnostic framework designed to preserve unlearnability under diverse spectral filtering, as illustrated in Figure 2. Specifically, we adopt a Random Spectral Masking (RSM) strategy, which randomly removes a contiguous frequency band, thereby forcing the remaining bands to sustain the unlearnable effect. In addition, we introduce Cross-Band Guidance (CBG), which enforces mutual consistency between low- and high-frequency signals, strengthening low-frequency unlearnability while regulating high-frequency perturbations to preserve semantic fidelity.

3.1 Problem Setup↩︎

We formulate the problem in the context of image classification with deep neural networks (DNNs). Let \((\boldsymbol{x}, y)\) be a labeled example, where \(\boldsymbol{x} \in \mathcal{X}\) is an input image and \(y \in \mathcal{Y} = \{1, \dots, K\}\) is its class label, where \(K\) is the number of classes in datasets. We denote the clean training and test datasets as \(\mathcal{D}_{c}\) and \(\mathcal{D}_{t}\), respectively. A perturbation generator \(\mathcal{G}_{\psi}\) is applied to \(\mathcal{D}_{c}\) to produce an unlearnable dataset \(\mathcal{D}_{u}\) by adding generated perturbations \(\boldsymbol{x}' = \boldsymbol{x} + \boldsymbol{\delta}\), where \(\boldsymbol{\delta} = \mathcal{G}_{\psi}(\boldsymbol{x})\) and \(\psi\) denotes the learnable parameters of the perturbation generator. To capture scenarios where input images are subject to spectral filtering, we additionally consider the case where training data is processed with a low-pass filter. In this setting, a classifier is trained on low-pass filtered unlearnable data, and the effectiveness of \(\boldsymbol{\delta}\) is evaluated by the accuracy degradation of classifier on the clean test set \(\mathcal{D}_{t}\). The objective is to design perturbations that consistently prevent the model from learning useful semantic information, even when the perturbed data is modified by low-pass filtering.

3.2 Randomized Spectral Masking↩︎

To ensure that perturbations remain effective under spectral filtering, we propose Randomized Spectral Masking. Specifically, RSM enforces spectral invariance by combining randomized frequency dropout, which prevents reliance on particular bands, with entropy-based regularization that encourages dispersion of perturbation energy across the spectrum. In this way, UEs are discouraged from concentrating in narrow spectral regions and remain effective under diverse filtering conditions.

To avoid perturbations collapsing onto a narrow spectral region, we introduce a stochastic frequency dropout mechanism that, at each training step, randomly discards a contiguous portion of the frequency domain. This design ensures that the remaining frequency components can still achieve the unlearnable objective, thereby compelling perturbations to distribute their influence across the spectrum. By applying different random masks at every step, the model learns to generate spectrum-agnostic UEs that remain effective regardless of which frequencies are suppressed.

Given an input \(\mathbf{z}\), the random mask operator is defined as \[\mathcal{T}_{\text{spec}}(\mathbf{z}) = \mathcal{F}^{-1}\!\left( \mathbf{M} \odot \mathcal{F}(\mathbf{z}) \right),\]

where \(\mathcal{F}\) denotes the Fourier transform and \(\mathbf{M} \in \{0,1\}^{H \times W}\) is a binary mask that zeros out a randomly sampled contiguous spectral band at each training step.

By training with randomized frequency suppression, the perturbation is required to preserve its unlearnable effect regardless of which frequency subset remains. This design prevents over-reliance on a specific spectral region and instead enforces a broader distribution of perturbation influence. The associated invariance loss is defined as \[\mathcal{L}_{\text{inv-spec}} = \mathcal{L}_{\text{CE}}\!\left(f_{\theta}\!\left(\mathcal{T}_{\text{spec}}^{(k)}(\boldsymbol{x} + \boldsymbol{\delta})\right), y\right),\]

where \(f_{\theta}\) is the classifier and \(\mathcal{T}_{\text{spec}}^{(k)}\) denotes the operator with an independently random mask at step \(k\). By optimizing this loss, the perturbation is compelled to remain unlearnable even when part of the spectrum is dropped, thereby enhancing its effectiveness across the entire frequency range.

Randomized masking ensures invariance under diverse filtering, but it does not necessarily prevent perturbation energy from collapsing into narrow spectral regions. To address this issue, we introduce a regularizer that promotes a more balanced energy allocation by maximizing the entropy of the normalized power spectrum. Formally, We denote \(\mathcal{F}[\delta]\) as the 2D FFT of the perturbation \(\delta\). We apply \(\operatorname{fftshift}\) to convert the raw FFT output into a centered spectrum representation, where the zero-frequency component lies at the center: \(\hat{\boldsymbol{\delta}} = \operatorname{fftshift}(\mathcal{F}[\boldsymbol{\delta}])\). The normalized spectral energy distribution is: \[P(u,v) = \frac{|\hat{\boldsymbol{\delta}}(u,v)|^2}{\sum_{u,v} |\hat{\boldsymbol{\delta}}(u,v)|^2},\] where \(P(u,v)\) is interpreted as the probability mass assigned to frequency coordinate \((u,v)\). The spectral entropy is then defined as \[\mathcal{H}(\boldsymbol{\delta}) = - \sum_{u,v} P(u,v)\log P(u,v).\] A larger value of \(\mathcal{H}(\boldsymbol{\delta})\) reflects a more uniform distribution of perturbation energy across frequencies, whereas a smaller value indicates that energy is concentrated within a limited region. Importantly, this regularizer does not enforce a perfectly uniform distribution (which would lead to white-noise perturbations), but rather mitigates excessive spectral bias while preserving the unlearnable objective. The complete RSM objective integrates the invariance and entropy terms: \[\mathcal{L}_{\text{RSM}} = \mathcal{L}_{\text{inv-spec}} - \mathcal{H}(\boldsymbol{\delta}).\]

This formulation promotes perturbations that remain unlearnable even under strong spectral filtering, thereby mitigating the vulnerability of prior approaches whose unlearnability is largely confined to high-frequency components, and thus fails to generalize under low-pass filtering.

3.3 Cross-Band Guidance↩︎

While randomized spectral masking distributes perturbations across the spectrum, it does not explicitly constrain the interaction between low- and high-frequency components. In natural images, low-frequency components primarily capture global semantics such as object shape, whereas high-frequency components represent local details and texture. To prevent models from exploiting either band independently, we propose Cross-Band Guidance, which imposes mutual constraints between the two bands to ensure their complementary roles are jointly preserved.

Given a clean image \(\boldsymbol{x}\) and its perturbed counterpart \(\boldsymbol{x}'\), we define the frequency split radius \(r_c \in [0,1]\) in the normalized Fourier domain so that it remains independent of the input image’s resolution. For an image of spatial size \(H \times W\), the normalized radial frequency at location \((i,j)\) is computed as: \[r(i,j)= \sqrt{ \left(\frac{i - H/2}{H/2}\right)^2 + \left(\frac{j - W/2}{W/2}\right)^2 }. \label{eq:radius}\tag{1}\]

We construct binary circular masks for low- and high-frequency regions: \[M_{\text{low}}(i,j)=\mathbb{1}\!\left[r(i,j) \le r_c\right], \qquad M_{\text{high}} = 1 - M_{\text{low}}. \label{eq:mask}\tag{2}\]

The decomposed components are obtained by masking the Fourier transform of the image: \[\begin{align} \boldsymbol{x}_{\text{low}}, \boldsymbol{x}_{\text{high}} &= \mathcal{F}^{-1}\!\big(M_{\text{low}} \odot \mathcal{F}(\boldsymbol{x})\big),\; \mathcal{F}^{-1}\!\big(M_{\text{high}} \odot \mathcal{F}(\boldsymbol{x})\big), \\ \boldsymbol{x}'_{\text{low}}, \boldsymbol{x}'_{\text{high}} &= \mathcal{F}^{-1}\!\big(M_{\text{low}} \odot \mathcal{F}(\boldsymbol{x}')\big),\; \mathcal{F}^{-1}\!\big(M_{\text{high}} \odot \mathcal{F}(\boldsymbol{x}')\big). \end{align} \label{eq:decomp}\tag{3}\] A small \(r_c\) corresponds to coarse global structures, whereas a larger \(r_c\) incorporates finer details. By explicitly modeling this decomposition, CBG establishes two complementary pathways that capture global semantics and fine details, and their interaction is critical for achieving effective unlearnability. We evaluate the impact of \(r_c\) in the ablation study.

To couple the two bands, we introduce a semantic guidance mechanism that transfers the stronger unlearnability of high-frequency perturbations to the low-frequency branch. High-frequency perturbations naturally induce stronger confusion in the classifier, producing unstable or incorrect predictions. We therefore treat the high-frequency prediction as a soft target and use the following joint formulation: \[\hat{y}_{\text{high}} = f_{\theta}(\boldsymbol{x}'_{\text{high}}), \quad \mathcal{L}_{\text{guide}} = \mathcal{L}_{\text{CE}}(f_{\theta}(\boldsymbol{x}'_{\text{low}}), \hat{y}_{\text{high}}).\]

This design encourages the low-frequency component to remain calibrated with the semantics emphasized by the high-frequency band, rather than injecting arbitrary distortions. In effect, the low-frequency pathway provides stable semantic cues, while the high-frequency pathway is constrained to reinforce these cues. This prevents the model from exploiting high-frequency artifacts as an alternative shortcut and ensures that the perturbations across different bands contribute coherently to the unlearnability objective.

Since low-frequency bands encode global semantics that remain relatively stable under perturbations, whereas high-frequency bands are more vulnerable to distortions, the low-frequency branch serves as a reliable reference to regularize high-frequency perturbations. We realize this guidance by measuring a hybrid similarity that integrates feature-level and perceptual cues. Let \(\phi(\cdot)\) denote the feature extractor of the surrogate model \(f_{\theta}\), which is kept frozen during training, and let \(\cos(\cdot, \cdot)\) denote the cosine similarity. The hybrid similarity for each band is then defined as \[\mathrm{sim}(\boldsymbol{a},\boldsymbol{b}) = \mathrm{cos}\big(\phi(\boldsymbol{a}),\,\phi(\boldsymbol{b})\big) + \mathrm{SSIM}(\boldsymbol{a},\,\boldsymbol{b}).\] For the low- and high-frequency components, the band-wise similarities are \[s_{\text{low}} = \mathrm{sim}(\boldsymbol{x}_{\text{low}},\,\boldsymbol{x}'_{\text{low}}), s_{\text{high}} = \mathrm{sim}(\boldsymbol{x}_{\text{high}},\,\boldsymbol{x}'_{\text{high}}).\] To let the low-frequency branch guide the high-frequency branch, we match the two similarities via a consistency loss: \[\mathcal{L}_{\text{struct}} = \mathrm{MSE} (s_{\text{high}},\, \mathrm{sg}[\,s_{\text{low}}\,]),\] where \(\mathrm{sg}[\cdot]\) denotes the stop-gradient operator, ensuring that the low-frequency similarity serves as a fixed target without receiving gradients from \(\mathcal{L}_{\text{struct}}\). This consistency regularizer discourages perturbations from distorting one band independently and encourages balanced interaction between low and high frequencies.

By jointly enforcing semantic guidance and structural alignment in both feature and perceptual domains, CBG balances low- and high-frequency perturbations to enhance unlearnability without compromising perceptual plausibility. Finally, the complete CBG objective combines semantic guidance with structural alignment: \[\label{lambda} \mathcal{L}_{\text{CBG}} = \mathcal{L}_{\text{guide}} + \lambda \cdot \mathcal{L}_{\text{struct}},\tag{4}\] where \(\lambda\) balances the trade-off between enforcing semantic guidance and preserving structural fidelity. Through the integration of frequency decomposition, semantic guidance, and structural alignment, CBG strengthens low-frequency unlearnability while constraining high-frequency perturbations to maintain semantic fidelity, preventing the model from relying excessively on any single band.

3.4 Optimization Objective↩︎

In summary, our framework integrates two complementary components: Randomized Spectral Masking, which enforces spectrum-agnostic unlearnability by preventing perturbations from collapsing onto specific frequency bands, and Cross-Band Guidance, which strengthens low-frequency unlearnability while constraining high-frequency perturbations to preserve semantic fidelity. Together, these designs yield perturbations that remain effective across spectral filtering while maintaining visual coherence. Formally, the overall optimization is:

\[\begin{align} \arg\min_{\theta}\;& \mathbb{E}_{(\boldsymbol{x}, y) \sim \mathcal{D}_{c}} \left[ \min_{\psi}\mathcal{L}_{\mathrm{full}}\big( \boldsymbol{x}, y; f_\theta, \mathcal{G}_\psi \big) \right] \\ \text{s.t.}\;& \|\boldsymbol{\delta}\|_\infty \leq \epsilon \end{align} \label{bound}\tag{5}\]

where \(\epsilon = 8/255\) denotes the perturbation budget. The final training objective is defined as: \[\mathcal{L}_{\mathrm{full}}=\mathcal{L}_{\text{RSM}}+\mathcal{L}_{\text{CBG}}.\]

4 Experiments↩︎

In this section, we conduct extensive experiments to evaluate the effectiveness of our proposed FUSE framework across multiple datasets and architectures. We begin by examining its unlearnability under low-pass and unfiltered settings to validate our design motivation. We then compare FUSE with representative UE baselines under varying cutoff levels to assess effectiveness across different degrees of frequency suppression. We also present ablation studies, including hyperparameter analysis, to isolate the individual contributions of Random Spectral Masking and Cross-Band Guidance. Finally, we evaluate our method against several defense strategies.

4.1 Experimental Settings↩︎

Datasets and Models. We evaluate FUSE on CIFAR-10 [20], CIFAR-100 [20], SVHN [21]. We use ResNet-18 [22], ResNet-50 [22], VGG-11 [23], DenseNet-121 [24], and ViT [25] as the surrogate classifier and target model both in training and testing. We utilize the classification accuracy on the clean test set as the evaluation metric, where lower accuracy indicates stronger unlearnability and protectiveness. Unless otherwise specified, low-pass filtering is implemented in the Fourier domain, where the cutoff is defined by a normalized Fourier radius of 0.5 relative to the image size.

Baseline. In the experiment section, we choose five UE methods as baselines, which are Error-Minimizing Noise (EMN) [5], Linearly-Separable Perturbations (LSP) [8], Transferable Unlearnable Examples (TUE) [7], Game Unlearnable Example (GUE) [9], Provably Unlearnable Examples (PUE) [26].

max width=

Implementation Details. Our model is developed with the PyTorch framework and trained on a single NVIDIA L40S GPU. We employ the Adam optimizer [27] with an initial learning rate of 0.001. We set the first loop training step \(T\) to 10 in all experiments. We train our model for 50 epochs on CIFAR-10 and SVHN, and 60 epochs on CIFAR-100. We fix the frequency split radius in Eq. 2 to \(r_c=0.5\) and the balance coefficient in Eq. 4 to \(\lambda=0.5\) throughout all experiments. To ensure imperceptibility, we set the perturbation bound \(\epsilon\) to \(8/255\) in Eq. 5 following prior works [5], [9], [14]. Results are averaged over five runs with different random seeds.

4.2 Main Results↩︎

Low-pass Filtering across Backbones. We evaluate representative UE methods under low-pass filtering across multiple backbones on CIFAR-10, CIFAR-100, and SVHN. This setting tests whether perturbations remain effective after high-frequency information is suppressed. As shown in Table ¿tbl:tab:lowpass95results?, prior methods show substantial accuracy recovery under low-pass filtering; for example, LSP and TUE reach \(54.16\%\) and \(52.16\%\) on CIFAR-10 with ResNet-18. In contrast, FUSE achieves only \(10.13\%\), and shows similar robustness on CIFAR-100 and SVHN. These results demonstrate that FUSE produces spectrum-agnostic perturbations that overcome the low-pass vulnerability of prior methods.

Unfiltered Evaluation across Backbones. We further evaluate unlearnable examples on unfiltered images to assess their effectiveness without spectral filtering. As shown in Table ¿tbl:tab:lowpass95results?, FUSE achieves the lowest test accuracy across most dataset–architecture pairs, confirming that its perturbations remain effective even without spectral distortions. On CIFAR-10 with ResNet-18, FUSE reduces accuracy to \(9.41\%\), substantially lower than the UE baseline LSP at \(13.54\%\). On SVHN with ViT, FUSE reaches \(6.55\%\), approaching the random-guessing level for a ten-class task. Even in the rare cases where FUSE is not the best performer, its performance remains very close, indicating negligible learnability. These findings demonstrate that FUSE is not confined to manipulating low-frequency information; rather, it retains strong unlearnability across the entire spectrum. This highlights that our framework produces perturbations that are intrinsically effective under both low-pass and unfiltered settings, thereby reinforcing its generality beyond the limitations of prior methods and ensuring consistent protection across diverse scenarios.

4.3 Evaluation under Varying Cutoff Frequencies↩︎

To further assess effectiveness against spectral filtering, we evaluate different UE methods under low-pass filtering with varying cutoff frequencies. As shown in Figure 3 (a) and Figure 3 (b), our proposed FUSE consistently reduces test accuracy to near random-guessing levels across all cutoff values on both CIFAR-10 and CIFAR-100. In particular, FUSE maintains test accuracy below \(10\%\) on CIFAR-100, demonstrating that the generated perturbations remain effective even under strong spectral suppression. In contrast, baseline methods show a marked decline in unlearnability when the cutoff is small, indicating that their perturbations primarily exploit high-frequency information. Notably, on CIFAR-100, TUE and PUE exhibit a non-monotonic trend where test accuracy initially increases before dropping at higher cutoffs. This occurs because their perturbations lack unlearnable effects in the low-frequency region, while the preserved semantic information of clean images becomes dominant as the cutoff increases. These results highlight that FUSE produces spectrum-agnostic perturbations, successfully overcoming the spectral fragility of prior approaches.

4.4 Cross-Architecture Transferability↩︎

We further evaluate the cross-architecture transferability of FUSE under an unfiltered setting, which is essential for assessing whether UEs remain effective when applied to unseen target models. Using ResNet-18 as the surrogate model, we evaluate its ability to render a diverse set of target architectures unlearnable, including CNN-based models (ResNet-50, VGG-11, DenseNet-121), a vision Transformer (ViT), and more recent non-standard architectures (ConvNeXt, MLP-Mixer, and Swin-T). As shown in Table 1, FUSE consistently achieves the lowest test accuracy across all target architectures, outperforming prior UE methods. Notably, FUSE remains effective on ConvNeXt, MLP-Mixer, and Swin-T, which substantially broadens the architectural coverage beyond the original CNN-heavy setting. These results demonstrate that FUSE is not overfitted to its surrogate model or standard CNN-style targets, and preserves strong unlearnability under diverse architectural shifts.

4.5 Ablation Study↩︎

6pt

We then analyze the contribution of each module through ablation experiments (Table 2). When Random Spectral Masking is removed, accuracy at cutoff \(0.5\) increases sharply to \(57.35\%\), showing that perturbations collapse into high frequencies and lose their unlearnability once low-pass filtering is applied. Removing the entropy regularizer also degrades performance, indicating that without explicitly spreading perturbation energy across the spectrum, perturbations become more localized and fragile. In contrast, removing Cross-Band Guidance leads to a more pronounced degradation under low-pass filtering, reflecting the importance of regulating low-frequency perturbations. Overall, the full FUSE consistently achieves the lowest accuracy, demonstrating that RSM, entropy regularization, and CBG are complementary: RSM enforces frequency diversity, entropy prevents spectral concentration, and CBG stabilizes cross-band interactions, together yielding spectrum-agnostic perturbations that remain effective under filtering.

We also analyze the choices of the frequency split radius \(r_c\), as shown in Table 3. A small value (\(r_c=0.2\)) provides insufficient low-frequency semantics, leading to unstable guidance signals and weaker unlearnability. In contrast, a large value (\(r_c=0.8\)) overly reduces the distinction between low- and high-frequency bands, weakening cross-band interaction. The best performance is consistently achieved at a moderate split (\(r_c=0.5\)), where the separation is sufficient to enforce interaction while maintaining semantic stability.

To assess the effect of the structural consistency loss, we vary its weight \(\lambda\) in the CBG objective. We first conduct this analysis on CIFAR-10 under both unfiltered and low-pass settings, as shown in Figure 4 (a). When \(\lambda\) is very small, the consistency loss is too weak to sufficiently constrain high-frequency perturbations or stabilize their guidance to the low-frequency branch, leading to weaker unlearnability, especially under low-pass filtering. As \(\lambda\) increases, the structural consistency loss better aligns the two spectral branches and reinforces the transfer of unlearnable effects across frequency bands, improving protection in both settings. However, overly large \(\lambda\) makes the structural term dominate optimization, weakening the semantic guidance from the UE objective and reducing overall unlearnability. Therefore, moderate values provide the best trade-off between semantic guidance and structural alignment.

To further verify this trend beyond CIFAR-10, we additionally conduct ablation experiments on CIFAR-100 (Figure 4 (b)) and SVHN (Figure 4 (c)). The results consistently show that moderate values around \(\lambda=0.5\) achieve the best or near-best unlearnability across datasets, while very small or very large values lead to weaker protection. These results indicate that \(\lambda=0.5\) is a robust default choice for balancing semantic guidance and structural alignment in FUSE.

4.6 Resistance to Defense Strategies↩︎

To further evaluate robustness, we test different UE methods against a broad range of defense strategies. We first consider common training-time defenses, including data augmentation [28]–[30] and adversarial training [31]. We additionally include two model-free UE baselines, AR [18] and OPS [32]. As shown in Table 5, while some baselines retain partial effectiveness under certain defenses, their unlearnability generally collapses once different types of defenses are applied. In contrast, FUSE consistently maintains low test accuracy across all settings, often close to random-guessing levels, demonstrating that its spectrum-agnostic design prevents models from circumventing perturbations through either augmentation or adversarial retraining.

Beyond these general defenses, we further evaluate FUSE against representative UE-specific defense methods, including D-VAE [33], AN-SDA [34], and ORTHO PROJ [35]. These defenses are substantially different from simple frequency filtering or JPEG-based purification, as they aim to remove or suppress unlearnable perturbations through reconstruction, detection-based purification, or projection-based preprocessing. As shown in Table 5, FUSE still achieves the lowest downstream accuracy under all three defenses, whereas most baselines are largely neutralized. This result indicates that the advantage of FUSE is not limited to frequency-related purification settings, but also extends to stronger UE-specific defenses. ALL these findings show that FUSE provides comprehensive resistance to a broad range of defense mechanisms, including augmentation, adversarial retraining, and dedicated UE purification methods.

4.7 Resistance to JPEG Compression Defenses↩︎

We further evaluate the robustness of FUSE under different JPEG compression levels, combined with a low-pass filter. JPEG compression removes high-frequency components and introduces frequency artifacts, making it a challenging perturbation scenario for unlearnable examples. Since FUSE proposes full-spectrum unlearable perturbations, rather than restricting perturbations to a single frequency band, FUSE is robust to the JPEG compression compared with other UE methods. As shown in Table 6, FUSE consistently maintains near random-guess accuracy across a wide range of JPEG quality levels, demonstrating that our method remains effective even under strong frequency-domain distortions. Additional results against JPEG are shown in Section 9.5.

4.8 Extension to Vision-Language Models↩︎

We further provide an initial study on whether FUSE can affect vision-language models. Different from standard classifiers, vision-language models are trained with paired image-text data and cross-modal alignment objectives [36]. Recent studies have also shown the growing importance of multimodal visual representation learning and alignment [37], [38]. As a proof-of-concept, we evaluate FUSE on a pretrained CLIP image encoder with ResNet50\(\times\)​4 backbone on CIFAR-10. Specifically, we freeze the text encoder and most of the image encoder, and fine-tune only the last visual block and attention pooling layer while preserving the original CLIP alignment objective. This allows us to evaluate whether the generated perturbations can disrupt image-text alignment rather than only standard classification decision boundaries. As shown in Table 7, UE baselines remain close to the clean-trained CLIP model under both unfiltered and low-pass settings, leading to only minor accuracy drops. In contrast, FUSE achieves the largest degradation, reducing the CLIP accuracy by 4.28% under the unfiltered setting and 6.06% under low-pass filtering. These results suggest that FUSE can also affect vision-language alignment beyond standard image classification.

4.9 Training Time and Computational Resource↩︎

Most existing UE methods directly store and add perturbations without any learnable generator. Therefore, for a fair comparison, we evaluate computational efficiency against the generator-based baseline GUE. Specifically, we feed each generator with a total of 50000 images from CIFAR-10 using a batch size of 512 to measure training time, inference time, GFLOPs, and parameters. All experiments are conducted on a consistent hardware setup comprising a NVIDIA L40S GPU. As shown in Table 8, these results show that FUSE contains fewer parameters and achieves lower inference time and computational cost, highlighting its efficiency and practicality for real-world deployment.

5 Conclusion↩︎

In this paper, we introduced Full-spectrum Unlearnable Examples via Spectral Equalization (FUSE), a spectrum-agnostic framework for protecting training data against unauthorized learning. Unlike existing methods whose effectiveness critically depends on high-frequency components and thus collapses under low-pass filtering, FUSE explicitly distributes perturbations across the entire frequency range and enforces cross-band consistency. Through the integration of Random Spectral Masking (RSM) and Cross-Band Guidance (CBG), FUSE generates perturbations that remain unlearnable under diverse filtering operations while preserving semantic fidelity. Extensive experiments across multiple datasets and architectures demonstrate that FUSE achieves consistently stronger and more stable unlearnability than prior approaches. Our results highlight the importance of frequency-domain modeling for data protection and suggest promising directions for future work, such as designing privacy-preserving mechanisms that remain effective under distribution shifts and adaptive defenses.

Acknowledgement↩︎

This work is supported by the Vector Institute and the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grants program.

Impact Statement↩︎

Our work introduces Full-spectrum Unlearnable Examples via Spectral Equalization (FUSE), a framework designed to protect data privacy by generating perturbations that prevent unauthorized models from learning useful semantics. While our goal is to strengthen privacy protection, we recognize that unlearnable perturbations could, in principle, be misused in adversarial contexts to interfere with model training. We emphasize that FUSE should be applied solely for ethical data protection purposes, and any use must comply with relevant legal frameworks, community standards, and dataset licenses. All datasets used in this study (CIFAR-10, CIFAR-100, SVHN, and ImageNet) are publicly available benchmarks, and our usage complies with their respective licenses and intended research purposes

APPENDIX↩︎

6 The Architecture of Perturbation Generator↩︎

In the main paper, we devise a generator to produce transferable perturbations and craft unlearnable examples. We denote our perturbation generator as \(\mathcal{G}\), which employs a standard encoder-decoder architecture. This structure comprises three down-sampling convolution layers, four residual blocks [22], and three transposed convolution layers. The detailed architecture is shown in Table 9.

7 Algorithm Pseudo Code↩︎

To improves the clarity and reproducibility of FUSE, we have proposed the algorithm pseudo code in Algorithm [alg:Framwork], summarizing the training pipeline, the randomized spectral masking, and the cross-band guidance.

8 More Implementation Details↩︎

Implementation of FUSE. For the perturbation generator, we adopt the Adam optimizer [27] with an initial learning rate of 0.001. For surrogate models, we use the SGD optimizer [39] with an initial learning rate of 0.1, applied to ResNet-18, ResNet-50, VGG-11, DenseNet-121, and ViT. To evaluate unlearnability, we employ the cross-entropy loss as the training objective in unauthorized supervised learning. Our method is a class-wise UE method and we add perturbations to the entire training dataset, which is the common practice in UE literature, to make all user data unexploitable by unauthorized training.

Implementation of Baselines. For a fair comparison, we adopt representative unlearnable example (UE) methods as baselines. Specifically, EMN [5], LSP [8], and PUE [26] follow a class-wise design, where perturbations are generated from class-level aggregation so that samples within the same class share similar perturbation patterns. In contrast, TUE [7] and GUE [9] employ a sample-wise approach, where each image is perturbed independently. We reproduce all baselines using their official implementations or configurations reported in the original papers, and tune the perturbation budget to align with our setting (\(\epsilon=8/255\)) for consistency.

9 Additional Experiments↩︎

9.1 Additional Results on SVHN and ImageNet↩︎

To further evaluate effectiveness against spectral filtering, we extend our analysis to SVHN and ImageNet\(^*\) (the first 100 classes of ImageNet [4]) under low-pass filtering with varying cutoff frequencies. As shown in Figure 6 (a) and Figure 6 (b), FUSE consistently achieves near-random test accuracy across a wide range of cutoffs, indicating that its perturbations remain effective even under strong spectral suppression.

On SVHN (Figure 6 (a)), when the cutoff is as small as 0.2, most baselines still yield relatively high accuracy (above \(60\%\)), whereas FUSE immediately reduces performance to around \(10\%\), close to random guessing. As the cutoff increases, the accuracies of baseline methods rapidly decline and eventually converge to FUSE’s level. At higher cutoffs (e.g., \(>0.7\)), a few methods slightly outperform FUSE, but all methods remain in the vicinity of random-guessing accuracy, rendering the differences negligible.

On ImageNet\(^*\) (Figure 6 (b)), the contrast is even more pronounced. FUSE consistently suppresses accuracy below \(10\%\) across nearly all cutoffs, while baselines such as EMN and LSP remain significantly higher (\(30–50\%\)) under small cutoffs (\(0.2\)–\(0.4\)), clearly demonstrating their dependence on high-frequency perturbations. As the cutoff increases, their accuracy declines but still remains above FUSE. This pattern confirms that FUSE perturbations are not confined to a particular spectral band and remain effective under both strong and weak filtering.

Overall, these results highlight the spectrum-agnostic property of FUSE. By maintaining unlearnability across datasets of different scale and complexity (SVHN and ImageNet\(^*\)), FUSE preserves its unlearnable effect under low-pass filtering, whereas prior approaches degrade substantially when high-frequency perturbations are suppressed.

9.2 Ablation Studies on Perturbation Strength↩︎

We further investigate the role of perturbation strength by varying the bound \(\epsilon\) in Eq. 5 on CIFAR-10, CIFAR-100, and SVHN under both unfiltered and low-pass filtering conditions (cutoff \(=0.5\)). As shown in Table 10, increasing \(\epsilon\) consistently improves unlearnability within each dataset, as reflected by the monotonic reduction in test accuracy toward the random-guess level (10% for CIFAR-10 and SVHN, 1% for CIFAR-100).

On CIFAR-10, accuracy decreases from \(11.0\%\) at \(\epsilon=2/255\) to \(9.4\%\) in the unfiltered setting, and from \(15.5\%\) to \(10.1\%\) under low-pass filtering. On CIFAR-100, accuracy drops from \(1.6\%\) to below \(1.0\%\) as \(\epsilon\) increases, which is already close to random guessing, showing that even mild perturbations suffice to induce unlearnability. A similar trend is observed on SVHN, where accuracy decreases from \(11.3\%\) at \(\epsilon=2/255\) to \(8.0\%\) at \(\epsilon=16/255\) in the unfiltered setting, and from \(14.6\%\) to \(9.3\%\) with filtering.

These results confirm that stronger perturbations remain effective even when high-frequency components are suppressed. Nevertheless, larger \(\epsilon\) inevitably reduces visual imperceptibility, highlighting the inherent trade-off between perturbation strength and image quality. Importantly, the consistent patterns across datasets and filtering conditions indicate that FUSE’s unlearnability is not tied to a specific perturbation budget or frequency band.

To complement the quantitative results in Table 10, we further provide visualizations of FUSE-generated unlearnable examples under different perturbation strengths. As shown in Figure 7, small perturbations remain visually indistinguishable but provide weaker unlearnable effects. Larger perturbations produce stronger suppression of learnability but also lead to noticeable visual degradation.

These visual trends align well with the quantitative results: increasing \(\epsilon\) consistently lowers the test accuracy of the defender toward random-guess levels, while the perceptual distortion gradually increases. Together, the visual and numerical evidence illustrate the intrinsic trade-off between perturbation intensity and imperceptibility.

5pt

9.3 Cross-Dataset Transferability of UEs↩︎

We further evaluate cross-dataset unlearnability by generating perturbations on one dataset (CIFAR-10, CIFAR-100, or SVHN) and applying them to other datasets. Results are reported for both the unfiltered setting (Table 11) and under low-pass filtering at test time (Table 12). Since LSP [8] and TUE [7] generate fixed perturbations, we first resample these perturbations before applying them to the images to ensure a fair comparison. When the number of samples differs, we generate new perturbation samples using a uniform sampling strategy [7] or randomly cropping the saved perturbations to remove redundant portions, thereby ensuring compatibility with the target dataset.

Across both conditions, FUSE generally achieves the lowest transfer accuracy, indicating the strongest unlearnable effect. In particular, when perturbations are generated on CIFAR-100 and transferred to CIFAR-10, FUSE reduces accuracy to \(9.98\%\) in the unfiltered case, while baselines such as TUE and GUE exceed \(90\%\). Even after low-pass filtering, FUSE maintains low transfer accuracy (\(17.08\%\)), whereas competing methods remain above \(60\%\). A similar advantage is observed in the SVHN-to-CIFAR-100 transfer, where FUSE suppresses accuracy to \(5.25\%\) in the unfiltered setting and \(16.29\%\) with low-pass filtering, again far below other approaches.

Other transfer pairs show the same trend. For example, from CIFAR-10 to SVHN, FUSE achieves \(9.94\%\) (unfiltered) and \(11.09\%\) (low-pass), while the baselines range from \(8\)–\(25\%\) in the former but surge above \(90\%\) in the latter, exposing their vulnerability to spectral filtering. Similarly, when transferring from SVHN to CIFAR-10, FUSE maintains accuracy near \(10\%\) under both conditions, whereas TUE and GUE exceed \(90\%\).

Overall, these results demonstrate that FUSE-generated perturbations transfer more reliably across datasets and remain effective even under spectral suppression. This advantage arises because natural images share highly similar frequency distributions [40], [41], and our design ensures effectiveness across the entire spectrum. Consequently, when facing cross-dataset transfer settings, FUSE maintains strong unlearnability, whereas prior methods exhibit limited transferability in the unfiltered case and collapse once low-pass filtering is applied.

9.4 More Analyses of Different Unlearnable Percentages.↩︎

In addition to the main experiments, we further investigate how the proportion of perturbed training samples affects unlearnability. While the standard UE setting perturbs the entire training set, practical applications may involve scenarios where only a given percentage of the training data (\(p\)) is protected. To this end, we follow the different unlearnable percentages protocol and vary the perturbed-data ratio [5], [8]. Specifically, for each unlearnable percentage, we train two models. The first model uses both the clean subset and the unlearnable subset as its training data, and the second one only uses the clean subset. The difference between the performances of those two models represents how much semantic information the first model gains from the UEs. A small performance gap indicates the first model gains little information from the UEs. As shown in Table 13 (with a cutoff radius of 0.5) and Table 14 (unfiltered), existing UE methods reduce protection when only part of the data is perturbed. In contrast, FUSE consistently achieves accuracy closest to the clean baseline across most poisoning ratios, indicating that the model effectively ignores perturbed samples even when the unlearnable portion is limited.

To further address the challenge of mixed clean and perturbed classes, we additionally evaluate FUSE under a partial protection setting on CIFAR-10, where only one (bird) or multiple classes are perturbed, and the remaining classes stay clean. Following the protocol used in prior UE work such as EMN, we analyze the resulting confusion matrices and per-class accuracy. As shown in Figure 8 (a) and Figure 8 (b), the results show that the protected classes become effectively unlearnable, with their accuracy dropping to nearly 0% (similar to EMN), while the unprotected classes remain largely unaffected. Moreover, the confusion is clearly asymmetric: protected-class samples are misclassified into other classes, but clean samples are rarely mapped back to the protected classes.

9.5 More Analyses of JPEG Compression Defenses.↩︎

We further evaluate the robustness of FUSE under different JPEG compression levels on Imagenet, combined with a low-pass filter. As shown in Table 15, FUSE consistently maintains near random-guess accuracy across a wide range of JPEG quality levels, demonstrating that our method remains effective even under strong frequency-domain distortions. To verify this, we further demonstrate the impact of JPEG compression (Q=70) and low-pass filtering using t-SNE visualizations in Figure 9. The results show that under full-spectrum conditions, FUSE’s perturbations result in a low accuracy, lower than other methods. When low-pass filtering is applied after JPEG compression, FUSE’s performance remains close to random-guess level. These results further support our claim that FUSE is more robust to JPEG compression than other methods.

9.6 Ablation on Frequency Split Radius↩︎

The frequency split radius \(r_c\) controls the proportion of low-frequency components retained in the spectral decomposition of FUSE. To verify whether the optimal \(r_c\) is dataset-dependent, we conduct a comprehensive ablation study on four datasets of different resolutions and characteristics: CIFAR-10, CIFAR-100, SVHN, and ImageNet\(^*\). As shown in Table 16, the optimal \(r_c\) consistently lies in a narrow range around 0.5, and the performance variation within this range is small. This indicates that the choice of \(r_c\) is stable across datasets and not dataset-specific.

Furthermore, we applied Bayesian Optimization to automatically search for the optimal \(r_c\) on each dataset (Table 17). The automatically identified values also converge near 0.5, and the resulting accuracies differ from the fixed setting \(r_c\) = 0.5 by less than 0.1%–0.2%. These results confirm that the manually chosen \(r_c=0.5\) is close to the global optimum and not the result of manual tuning. While selecting the dataset-wise optimal \(r_c\) yields marginal improvements, the overall performance differences are small, demonstrating that FUSE is not sensitive to this hyperparameter. For simplicity and reproducibility, we therefore adopt \(r_c\) = 0.5 as a robust default choice that generalizes well across datasets.

9.7 Effect of the Number of Frequency Bands↩︎

To examine whether a simple high-/low-frequency split may be insufficient, we conducted an additional ablation study on ImageNet\(^*\) where the frequency domain is divided into three bands (low, mid, high). We further applied our CBG module (Section 3.3) to all pairwise band interactions (low-to-mid, low-to-high, and mid-to-high). The results are summarized in Table 18.

It can be observed that, while modest, three-band split indeed leads to stronger unlearnability. On the other hand, we note that both two-band and three-band designs achieve accuracies that are close to random guess, whereas latter incurs higher computational cost, indicating a trade-off between effectiveness and efficiency.

9.8 High-Pass Filtering Analysis↩︎

We further evaluate FUSE under high-pass filtering, which removes low-frequency components and retains only high-frequency details. As shown in Table 19, FUSE consistently maintains near random-guess accuracy across all cutoff values, demonstrating robustness to both low-frequency (Table ¿tbl:tab:lowpass95results?) and high-frequency removal and supporting our spectrum-agnostic claim. Notably, the observation that prior UE methods also remain effective under high-pass filtering is consistent with our analysis in the paper. In contrast, their failure under low-pass filtering (as shown in Table ¿tbl:tab:lowpass95results?, Figure 3 (a) and Figure 3 (b)) arises because the high-frequency components are suppressed.

9.9 More Ablation on structural similarity.↩︎

The structure guidance in Section 3.3 includes cosine similarity and structural similarity. To isolate the contribution of structural similarity, we conduct an ablation study on unfiltered CIFAR-10 Dataset by removing the structural similarity term while keeping all other components unchanged. As shown in Table 20, removing structural similarity leads to degraded the quality of UEs, reflected by lower PSNR/SSIM and higher LPIPS/MSE. These results confirm that the structural similarity term enhances visual fidelity and structural preservation, complementing the cosine-similarity term.

9.10 Robustness to Downstream Loss Mismatch↩︎

We further evaluate whether FUSE remains effective when the downstream model is trained with a loss different from the one used during perturbation construction. In this experiment, the FUSE perturbation construction is kept exactly the same as in the main paper, and only the subsequent model training loss is changed. Specifically, we consider Focal Loss with focusing parameter \(\gamma=2.0\) and class balancing factor \(\alpha=1.0\), and Label Smoothing Loss with smoothing factor \(\sigma=0.1\). As shown in Table 21, FUSE consistently achieves the lowest test accuracy under both downstream losses. These results indicate that FUSE is robust to loss mismatch and is not narrowly tied to a specific downstream training objective.

9.11 Sensitivity to Downstream Training Protocols↩︎

6pt

5pt

We further evaluate whether FUSE remains effective under different downstream training protocols. Specifically, we test a broad set of optimization-related factors, including weight decay, optimizer family, learning rate, scheduler choice, SGD momentum, and Adam-style beta parameters. All experiments are conducted on CIFAR-10 with a ResNet-18 backbone, and lower downstream accuracy indicates stronger unlearnability.

As shown in Tables ¿tbl:tab:wd95opt95sensitivity?–¿tbl:tab:opt95sensitivity95part2?, FUSE consistently maintains low downstream accuracy across all tested settings. The performance remains stable under different weight decay values, optimizers, learning rates, scheduler choices, momentum values, and beta parameters, suggesting that the effectiveness of FUSE is not narrowly tied to a single default optimization setup.

We additionally note that this question is closely related to the motivation of our method. Prior studies suggest that deep networks often rely on shortcut or easy-to-learn predictive cues rather than human-salient semantics [42]–[44], which also helps explain why UE methods can hinder semantic learning [45]. From the spectral perspective, our low-pass filtering experiments further show that prior UE methods become substantially weaker once their high-frequency cues are suppressed, since the downstream model can more easily recover useful semantic information from the protected samples. This observation directly motivates the spectrum-agnostic design of FUSE, which encourages perturbations to remain effective across spectral bands instead of relying on fragile high-frequency shortcuts.

More broadly, this robustness analysis is related to learning under imperfect, noisy, or shifting conditions, including noisy supervision, misclassified responses, source-free adaptation, uncertainty control, and distribution shifts [46]–[56]. These works highlight that downstream learning behavior can be sensitive to supervision quality, optimization protocols, and distributional structure. From another perspective, Random Spectral Masking can be interpreted as exposing the perturbation generator to different spectral domains during training, which is related in spirit to data augmentation and domain generalization under changing environments [57]. Together, these connections help contextualize why FUSE remains effective across different losses, optimizers, schedulers, regularization settings, and spectral filtering conditions.

9.12 Broader Connections to Robust Representation Learning↩︎

From a broader representation-learning perspective, our results also connect to studies of subgroup robustness, fairness, and structure-aware representation learning [58]–[60]. Recent work further shows that structural inductive biases can support transfer and adaptation across domains, including graph and scientific prediction settings [61]–[65]. This direction is also broadly related to semantic representation learning and contextualized visual feature modeling in retrieval systems [66]. Although these problems differ from unlearnable example generation, they share the common goal of understanding how models exploit stable or spurious structures under distributional changes. FUSE follows this perspective by explicitly controlling spectral structures so that protected samples remain difficult to learn under diverse downstream conditions. In addition, deploying UE protection in larger-scale or distributed learning systems may introduce efficiency and system-level challenges. Recent studies on efficient attention, imbalanced regression, and federated learning highlight the importance of scalable optimization and robust training under heterogeneous data or resource constraints [67]–[70]. Although these directions are orthogonal to FUSE, they suggest useful future extensions for applying spectrum-agnostic protection to broader real-world learning pipelines.

10 Visualization↩︎

10.1 Imperceptibility Analysis↩︎

We further compare the visual quality of FUSE with representative UE methods under the same perturbation budget (\(\epsilon=8/255\)). Following prior UE works, we report standard image-fidelity metrics averaged over 100 CIFAR-10 samples, including PSNR, SSIM, LPIPS, and MSE. As shown in Table 22, the visual quality of FUSE-perturbed images is comparable to existing UE methods. In particular, FUSE achieves the best PSNR and LPIPS among all compared methods, while remaining competitive on SSIM and MSE. These results suggest that the proposed full-spectrum perturbation does not compromise imperceptibility compared with prior UE methods.

10.2 Test Accuracy Curves↩︎

In this section, we evaluate the test accuracy curves of unlearnable examples under low-pass filtering, using ResNet-18 [22] models trained on the perturbed CIFAR-10 dataset. As shown in Figure 10, most existing UE methods (e.g., EMN [5], GUE [9], and PUE [26]) still retain non-trivial accuracy after filtering, and some even exhibit early accuracy peaks, which may lead to semantic leakage. In contrast, our proposed FUSE consistently maintains accuracy close to the random-guessing level of 10% from the first epoch and remains stable throughout training. These results suggest that FUSE achieves stronger and more persistent unlearnability, even under spectrum-suppressed conditions introduced by low-pass filtering.

10.3 Visualization of Perturbations under Low-Pass Filtering↩︎

To further illustrate the effect of low-pass filtering on UEs, we visualize perturbed images from CIFAR-10, CIFAR-100, and SVHN under varying cutoffs (Figure 11). Notably, when the cutoff is 0.1, images become nearly structureless and contain little semantic content. For other cutoff values, semantic structures such as object shapes and digit outlines remain recognizable, indicating that low-frequency information alone can still support partial learning. At the same time, our perturbations are visually indistinguishable from their clean counterparts, demonstrating that FUSE preserves imperceptibility while distributing perturbation effects across the spectrum.

References↩︎