1 Introduction↩︎

Unlearnable examples (UE) are a proactive, user-side defense that lets people keep their publicly shared photos from being learned by unauthorized face-recognition (FR) models. Such photos are scraped without consent to train models that can later identify their subjects, and a single company has already amassed billions of web-scraped faces to build one [1]; modern face recognition is highly accurate [2] and remains effective with only a few images per identity [3], so a single public album is enough. To prevent this, before sharing the owner adds an imperceptible, error-minimizing perturbation that plants a spurious shortcut between image and label, so a model trained on the data learns the shortcut instead of identity and its accuracy drops to the random-guess level [4]. This source-side guarantee (protect once, at upload time) has made UE an active line of privacy research [5]–[8].

UE have been strengthened along several methodological axes and carried over to new settings. Later work made the shortcut survive adversarial training [5] and random perturbations [6], hold up under data augmentation [9], and resist spectral filtering [10]; for faces, the approach was extended to a generative protector, trained once, that lowers recognition accuracy [11]. Across these advances the intended guarantee is the same: a face perturbed once, at upload time, remains unlearnable to any party that later collects it [4], [11].

However, in a realistic recognition pipeline this guarantee collapses (1), because the attacker does not train on the uploaded image itself. Shared photos place the face (often small, tens of pixels across [12], [13]) within a larger scene, and an attacker who scrapes one at scale [1] first applies a transform before training: extracting the face and rescaling it to the model’s input [2], [14]. This transform breaks the protection in two ways. Spatially, the perturbation’s budget is spread over the whole image, but the transform keeps only the face region and discards the rest. Spectrally, whatever survives is high-frequency, and the rescaling acts as a low-pass filter that attenuates it. Passed through this pipeline, existing perturbations retain almost none of their effect, and the attacker recovers the accuracy it would have without any protection (1).

To address this, we propose LPID (Localized, Pipeline-coupled Identity Defense), which restores the protection by building the attacker’s transform into the perturbation itself. LPID acts on the standard objective at the two points the transform attacks: it confines the perturbation to the face region the attacker extracts [15], and optimizes its generation through a differentiable model \(T\) of that transform [16], [17]. Optimizing through \(T\) steers the perturbation’s energy into the band the transform preserves, so the resulting error-minimizing perturbation reaches, and remains a learnable shortcut on, the very input the attacker trains on from scratch. Because this robustness is a property of the transform rather than of any particular face, LPID protects users unseen at protection time (identities held out from any training set, since the perturbation is re-optimized for each new album). Our main contributions are:

• We propose LPID, a perturbation robust both to the attacker’s transform and to users unseen at protection time. LPID confines the perturbation to the face region the attacker extracts and optimizes its generation through a differentiable model \(T\) of that transform, the intervention point the standard objective had overlooked. It needs no knowledge of the protected user’s identity; re-optimized per album, it applies unchanged to any unseen user (4).

• We establish why the protection works. A spectral analysis explains why existing perturbations are erased by the transform while ours survives (93% of the perturbation energy in the resize-surviving band, % uncoupled), and a factorized ablation identifies which design choice is responsible for the protection (4.2).

• Extensive experiments show that LPID is effective in practice: across the attacker’s transform, on identities unseen at protection time, and against several attacker architectures, LPID holds the attacker below \(10\%\) across the crop+resize family, at \(32.7\) dB PSNR and \(0.161\) LPIPS ([sec:main-result] [sec:transfer]).

2 Related Work↩︎

2.1 Unlearnable Examples↩︎

Unlearnable examples make training data useless to a model that trains on it, a form of availability poisoning [18]–[20] within the broader data-poisoning literature [21]. Like adversarial examples [22], [23] they use small bounded perturbations, but to minimize rather than maximize the training loss. Huang  [4] craft the original UE with a min-min bi-level objective that injects an error-minimizing perturbation, inducing a shortcut [24] the model learns in place of real features; this perturbation is optimized at a single fixed resolution against a surrogate and is not robust to adversarial training [5].

To restore that robustness, REM [5] adds an inner maximization (a costly min-min-max optimization), while SEM [6] instead trains against random perturbations; both still operate on the whole image at a fixed resolution [5], [6]. TUE [7] adds an optimizable class-wise separability term to make the perturbation transferable across datasets, and LSP [8] removes the surrogate entirely, synthesizing model-free, class-wise linearly separable perturbations, though with a larger, more visible perturbation.

For faces, Segue [11] trains a class-conditional generator once to emit error-minimizing perturbations, with a distortion layer (adversarial training, blur, flip, sharpness, JPEG) for robustness; because it conditions on identity, it needs a label for each protected user and cannot cover users absent from its training set.

2.2 Facial Privacy Protection↩︎

Face-recognition systems can identify a person from a single photograph without their consent [2], [25]–[27], and recognizers have already been built at scale from faces scraped off the web [1]. Facial privacy protection lets individuals defend their own images against this unauthorized recognition [16], [28]; a complementary line instead replaces the face with a synthetic one via generative anonymization [29], [30], changing the published image rather than disrupting training.

Existing defenses differ mainly in when the protective perturbation takes effect [4], [28]. Test-time cloaks perturb an image so that an already-deployed recognizer misidentifies it [31], [32]; such a cloak protects only that image, leaves the person’s other photos usable, and does nothing to stop a recognizer from being trained on the collected data [16], [28]. Training-time protections alter the images before they are collected, so that a model trained on them generalizes poorly to clean faces [4], [5], [11].

Across all of these, the perturbation is crafted and evaluated at a single fixed resolution, and prior work shows that simple input preprocessing (grayscale, JPEG, or resizing [33], [34]) already removes much of their protection. We trace the resize case to the transform in the attacker’s extraction pipeline and address it directly (3). We take UE, REM, TUE, and LSP as our per-sample baselines and Segue as our generative baseline (5).

3 Preliminaries and Threat model↩︎

3.1 Unlearnable examples: the standard objective↩︎

Standard unlearnable examples assume the attacker trains on the protected image \(x+\delta\) as is—the assumption our method overturns. They add an imperceptible, error-minimizing perturbation \(\delta\) (bounded, \(\|\delta\|_\infty\le\epsilon\)) that injects a spurious shortcut between image and label, so a model trained on the protected data learns the shortcut in place of identity [4], by solving the min-min bi-level optimization problem \[\min_\theta \; \min_{\|\delta\|_\infty \le \epsilon} \; \sum_i L\big(f_\theta(x_i + \delta_i),\, y_i\big), \label{eq:ue}\tag{1}\] where \(L\) is the cross-entropy loss and \(f_\theta\) a surrogate recognizer, alternating an inner minimization that crafts \(\delta\) against the current \(f_\theta\) with an outer step that trains it. Here \(y_i\) is only the within-album grouping of images by identity (an arbitrary local index per identity), not membership in any pretrained label set; since the objective depends only on this local grouping and not on any fixed identity vocabulary, the same procedure runs unchanged on a new user’s album, whose images simply receive fresh local indices. Robust variants add an inner maximization (REM [5]) or train against random noise (SEM [6]) to survive adversarial training, but all inherit this same assumption.

3.2 Threat model↩︎

Attacker. The attacker collects the published photos and trains its own face recognizer from scratch, holding no pre-trained model [4], [11]. It scrapes the images [1] (possibly already recompressed by the hosting platform at upload) and applies the preprocessing every modern face recognizer requires [2], [14]. It detects and aligns the face [14], [35], crops it out, and resizes the crop to the recognizer’s fixed input resolution \(s\) (\(112\) for ArcFace [2]), optionally followed by Gaussian blur [11], before training a classifier. We call this detect–crop–resize sequence the attacker’s extraction pipeline, and write C+R\(s\) for its crop-and-resize core. Crucially, detection and alignment merely locate the face; the crop-and-resize is the step that actually reshapes the pixels carrying the perturbation, and it is unavoidable, since every recognizer needs a fixed input resolution [2]. This is why our method models exactly this crop-and-resize as a differentiable transform \(T\) and couples the perturbation to it.

We score protection by the attacker’s model, an identity classifier over the collected people (25 identities here). Its clean test accuracy is the fraction of held-out photos it labels with the correct person, where lower means better protection. Protection lowers this accuracy rather than hiding the face, which stays fully visible; random-guess accuracy is \(\approx4\%\). Unlike the aligned, full-frame faces on which recognizers are usually benchmarked [36], [37], the scraped photos here hold a small face within a larger scene [12], [13].

Defender. The defender must make a face unlearnable before publishing it, without knowing how the attacker will later process the image. Given a face of native size \(s_0\) at a known box \((x_0,y_0)\) with label \(y\) in the published image, the defender adds the perturbation \(\delta\) at budget \(\epsilon=8/255\), confined by a mask \(m\) to the face box and crafted against a surrogate \(f_\theta\) (a ResNet-18 [38]) that shares no weights with the attacker’s network; the perturbation is committed once, before any recognizer has seen it.

Because it is committed blindly, the protection must survive two unknowns. First, the defender does not know the attacker’s input size \(s\), resize kernel, or the platform’s JPEG quality (each fixed after publication and never disclosed), so protection must hold across the whole family of plausible resize operators, not one known target (4.2 shows why a single generation target suffices). Second, it must protect unseen users, whose identities are disjoint from the protector’s training set and are therefore unknown when the protection is built, an identity-disjoint (subject-independent) protocol [16], [28], [39]; our per-sample perturbation is re-optimized on each new album, so an unseen identity is handled exactly as a seen one, whereas a generative protector conditioned on an identity label cannot cover a user it was never trained for.

4 LPID: Localized, Pipeline-coupled Identity Defense↩︎

4.1 Extraction-aware noise generation↩︎

We build the attacker’s extraction pipeline into the standard UE objective. As 3 describes, the attacker does not train on the protected image \(x+\delta\) but on the face it extracts from it; we model that extraction as a differentiable operator \(T\) (2; defined below) and craft the perturbation through it. In the standard objective 1 , \(x+\delta\) then becomes \(T(x+m\odot\delta)\), giving the LPID objective \[\min_\theta \; \min_{\|\delta\|_\infty \le \epsilon} \; \sum_i L\big(f_\theta( T(x_i + m_i \odot \delta_i) ),\, y_i\big). \label{eq:lpid}\tag{2}\] This adds two ingredients to 1 , one per mismatch.

Localization: \(m\) is a binary mask (one on the face box the attacker crops, zero elsewhere), so \(m\odot\delta\) confines the perturbation to the region the extraction keeps, following the region-masking of [15].

Coupling: rather than crafting \(\delta\) in pixel space, we optimize it through the differentiable operator \(T\) [16], [17], so the gradient that shapes \(\delta\) is the one the attacker’s own pipeline will pass. We instantiate \(T\) as a differentiable model of the extraction, a crop of the face box followed by a bilinear resize to the surrogate’s input resolution. The attacker’s own output resolution is unknown to the defender, and 4.2 shows why generating at a single fixed resolution suffices. Given the box at top-left \((x_0,y_0)\) with native side \(s_0\), \(T\) composes the crop (a \(0/1\) selection of the box) with the resize: \[T(z) = R_{s_0 \rightarrow 224}\big(\, z[\,:,\, y_0{:}y_0{+}s_0,\; x_0{:}x_0{+}s_0\,]\,\big), \label{eq:T}\tag{3}\] with align_corners\(=\)false, where output pixel \(o\) reads input coordinate \(u(o)=(o+\tfrac12)\, s_0/224-\tfrac12\) and interpolates its two neighbours. Both operations are linear in the pixels, so \(T = R\,S_{\text{box}}\) is a fixed linear map (differentiable, four-tap-sparse Jacobian). The crop \(S_{\text{box}}\) confines the gradient to the face box, and the resize \(R\) means the surrogate receives \(R\delta\), so the inner minimization concentrates the perturbation energy in the band the resize preserves; 4.2 makes this precise.

4.2 Why LPID survives the transform↩︎

Coupling is what makes \(\delta\) survive. It concentrates the perturbation’s energy in the frequency band the resize keeps. We make this precise with standard sampling theory [40], [41] (introducing no new theory), then confirm it with a factorized ablation (3).

Because \(T\) is linear, \(T(x+\delta)=Tx+T\delta\) (3 ), and the surrogate only ever receives \(T\delta\). A resize is a low-pass filter whose cutoff is half the smaller of its input and output resolutions [40], [41]; it transmits the band below that cutoff and attenuates the rest, echoing the frequency perspective on CNN generalization and robustness [42], [43]. Let \(V\) be the face-box subspace band-limited to the native rate \(s_0/2\), the largest band any resize of the face can keep. Optimizing \(\delta\) through \(T\) gives the inner loop no gradient toward the attenuated high-frequency components, so \(\delta\)’s energy accumulates in \(V\); an uncoupled, pixel-space perturbation instead spreads its budget across the full band, most of which the resize discards.

We quantify the concentration by the surviving-band energy fraction, with \(P_V\) the ideal low-pass projection onto \(V\): \[\rho_{\mathrm{LF}} := \frac{\|P_V\,\delta\|^2}{\|\delta\|^2}. \label{eq:rholf}\tag{4}\] Coupling raises this fraction from \(\rho_{\mathrm{LF}}=0.33\) (uncoupled) to \(0.93\) (coupled) at the native cutoff (3a), and the crop+resize columns of 1 bear this out against the baselines.

This also explains why the defender can generate at a single resolution without knowing the attacker’s. Energy placed below the native cutoff \(s_0/2\) is kept by any resize that preserves that band (in particular by any target at least the native size, which merely up-samples), so protection does not require matching the attacker’s resolution. 1 confirms this across the full range of native face sizes and attacker resolutions we evaluate (5.1).

A factorized ablation confirms that coupling, not localization, is responsible (3b). We toggle localization (whole image the face box) and coupling (crafted in pixel space through \(T\)) on our own perturbation and measure the attacker’s clean test accuracy. Whole-image, uncoupled noise gives no protection (attacker \(70.3\), near the clean \(71.1\)); adding localization alone recovers only part (\(25.4\)); adding coupling drives the attacker to \(2.3\), near random. Coupling is the decisive factor, and localized\(+\)coupled equals whole\(+\)coupled because \(T\)’s crop already zeros the off-box gradient.

4.3 Algorithm↩︎

4 solves 2 with the standard alternating min-min optimization of unlearnable examples [4], adding only the operator \(T\) to each step. Per minibatch, an inner projected-gradient (PGD) [44] step updates the perturbation \(\delta\) (a sign-gradient step taken through \(T\), then clamped to the \(\epsilon\)-ball and masked to the face box), and an outer stochastic-gradient (SGD) [45] step trains the surrogate \(f_\theta\). The clamp is applied before \(T\) so that \(T\) stays the linear operator of 3 . We use the surrogate and budget of 3 (ResNet-18, \(\epsilon=8/255\)) with \(K\) inner PGD steps per outer step; \(K\), the step sizes, and the epoch budget \(E\) are given in 5.1. As 4.2 shows, the surviving band is fixed by the native face size, not by the generation resolution.

5 Experiments↩︎

5.1 Setup↩︎

Datasets. We build composites by pasting CASIA-WebFace [46] faces at native sizes \(\{50,75,100,150,200\}\) (small to near-frame-filling) at random positions on Places365 [47] backgrounds on a \(224\times224\) canvas; the smaller sizes match the small-face regime of scraped imagery [12], [13]. We use an 80/20 train/test split per identity.

Baselines. We re-implement all baselines, UE [4], REM [5], TUE [7], LSP [8], and Segue [11], from their papers and, where available, official code, with the reported hyperparameters and no added resolution augmentation. All \(\ell_\infty\) methods use \(\epsilon=8/255\) and a ResNet-18 surrogate; LSP uses its native \(\ell_2\) budget and is model-free. Segue’s generator is conditioned on a training identity (3); since in a real deployment the users to protect are not known when the protector is built, we query it with a nearest-identity pseudo-label in this realistic unknown-user setting (1), and grant it the ground-truth label only where a control fixes a known user ([sec:aligned] [sec:fullres-control]).

Extraction. We recall C+R\(s\) from 3, a crop of the face box then a resize to \(s\times s\) (bic.\(=\) bicubic kernel). We evaluate the endpoints \(s=112,224\) of the \(\{112,160,224\}\) recognizer family. JPEG85\(\rightarrow\)C+R112 applies q85 compression before extraction (platform recompression); \(+\)blur adds Gaussian blur after the resize.

Metric. The metric is the attacker’s clean test accuracy (%, lower is better; 3), with the lowest per column in bold. All results are on unseen identities (3), averaged over 3 seeds; each table and figure states its identity count (random-guess \(\approx\)​4% for 25, \(\approx\)​2% for 50).

Implementation details. LPID crafts \(\delta\) for \(E=60\) epochs with \(K=20\) inner PGD steps of size \(\alpha=0.8/255\), through a bilinear \(T\) at target \(224\). The ResNet-18 surrogate and the attacker are trained from scratch with SGD at learning rate \(0.025\); the attacker uses a \(30\)-epoch schedule, extended to a fixed \(80\) epochs for the cross-architecture study (3).

5.2 Main result: existing UE break under the extraction pipeline↩︎

1 reports every method under the realistic small-face extraction pipeline. Applied natively over the whole published image, the per-sample UE give essentially no protection. The attacker’s crop discards the budget outside the face box, leaving the attacker at 69–75%, near the 72–75% clean accuracy (the spatial mismatch; 5). Localizing them to the face box the attacker crops, matching LPID’s spatial coverage (the face-only rows), does not rescue them: at C+R224 only the error-minimizing pair improves appreciably (UE 70.1 to 24.4, REM 74.8 to 36.9), and at the recognizer’s 112 input the resize attenuates the high-frequency content of every localized baseline, so the attacker still reaches 37–74% (UE 37.0, REM/TUE/LSP 73–74). Localization removes the spatial mismatch but not the spectral one, which coupling addresses, as the ablation in 4.2 shows.

For these unseen users, Segue has no valid identity to condition on and gives essentially no protection (\(\sim\)​72%, 5.1). LPID, a per-sample method localized to the crop box and coupled to the differentiable transform, holds the attacker below \(7\%\) across the crop+resize family (4.9–6.9%), the sole lowest method in every crop+resize column. For UE and REM, which protect strongly when no extraction is applied (5.6), this collapse is attributable to extraction itself; TUE and LSP do not protect even at full resolution, so extraction is not what causes their failure. The upload-JPEG column is a separate axis. Recompression at upload (q\(\approx\)​70–85 [48]) attenuates even the resize-surviving band, degrading LPID to \(20.8\%\) at q85, a limitation it shares with all \(\epsilon\)-bounded perturbations [33], [34].

5.3 Imperceptibility↩︎

2 reports PSNR/SSIM/LPIPS on the protected faces to control for a possible confound in perturbation budget; PSNR (dB) and SSIM [49] measure pixel and structural fidelity to the clean face (higher is better), while LPIPS [50] is a learned perceptual distance (lower is better). LPID (32.7 dB, 0.917, 0.161) is comparable to UE on PSNR and SSIM and less perceptible than Segue on LPIPS (0.161 ). LPID’s protection does not rely on a larger perturbation. Like UE/REM/TUE/Segue it respects \(\epsilon=8/255\), whereas LSP’s \(\ell_2\) budget yields a larger residual (max\(|\delta|=43.7/255\)). 6 shows the corresponding textures. All are imperceptible at native resolution, but the per-sample UE and LSP are high-frequency grain while LPID is the low-frequency structure the resize preserves.

5.4 Generalization to aligned faces↩︎

To rule out that LPID’s protection is an artifact of our small-face-in-scene composites, we evaluate it in the standard aligned-face setting with no compositing: raw CASIA-WebFace faces at \(224\) (25 unseen identities), with the attacker resizing to the \(112\) recognizer input. LPID limits the attacker to 6.3%, compared with 35.7% for UE and 21.2% for Segue given its ground-truth label (clean 75.3%). The protection therefore transfers to real aligned faces and is not a property of the composite construction. In both settings the background is discarded by the attacker’s crop and is absent from the extracted face; protection is determined only by the perturbation’s survival through the resize.

5.5 Transferability↩︎

3 evaluates each method’s perturbation against four attacker architectures (ResNet-18 [38], MobileNet-V2 [51], DenseNet-121 [52], VGG-16 [53]) trained from scratch at C+R112, testing whether LPID’s protection depends on the ResNet-18 surrogate it was crafted on. LPID limits the attacker to 4.4–9.1% accuracy across all four architectures, the lowest of every method and far below the per-sample baselines (33.2–76.3%); its protection is therefore not specific to the surrogate. Segue is omitted here; it is evaluated in 1.

5.6 Full-resolution control↩︎

To confirm the collapse of 1 is caused by extraction rather than weak baselines, 7 reports the attacker’s clean test accuracy on full-resolution faces (224, no crop+resize), the standard UE setting (50 unseen identities). UE and REM protect strongly here (7.4 and 18.0, clean 70.0), so our reimplementations are faithful and their collapse to 69–75% under extraction (1) is attributable to the pipeline. TUE and LSP do not protect even at full resolution (69.7, 68.3), so we restrict the extraction-causes-the-collapse argument to UE and REM. Given the true label, Segue also protects here (3.4), matching LPID (1.6); the realistic case, where that label is unavailable, is 1.

6 Conclusion↩︎

Unlearnable examples are crafted in pixel space, assuming the perturbation reaches the model unchanged, but a face-recognition attacker first applies a transform that existing UE do not model: it crops the face and resizes it, so the crop discards the perturbation outside the box and the resize attenuates what remains. We showed that existing per-sample UE collapse under this pipeline and proposed LPID, which generates extraction-aware noise, localized to the face the attacker crops and optimized through the crop+resize, so that, by construction, the optimization concentrates the surviving energy in the resize-surviving band. On identities unseen at protection time, LPID attains the lowest attacker accuracy of all methods across the crop+resize family, showing that the protection holds in the realistic deployment where the users to protect are unknown in advance. These results indicate that face-privacy perturbations should be evaluated under the attacker’s full extraction pipeline, and that coupling the perturbation to the extraction operator, rather than to a fixed set of distortions, is what lets the protection survive it. We hope this extraction-aware perspective offers a practical step toward facial privacy protection in the real world.

References↩︎