Visual generative models have recently made significant progress, encompassing diverse paradigms such as diffusion [1]–[6] and visual autoregressive models [7]–[12] across both image [13]–[19] and video generation [20]–[25] domains. Despite their structural differences, these models share a common reliance on large-scale, uncurated Internet data for pretraining. Consequently, they inevitably absorb undesirable semantics, including copyrighted entities, specific artistic styles, and unsafe content. Since data curation and retraining models from scratch are prohibitively expensive, concept erasure has become a practical and necessary solution for safe alignment and compliant deployment.

The central goal of concept erasure is to precisely remove the target concept, i.e., erasure efficacy, while preserving unrelated visual content and the model’s generative prior, i.e., prior preservation. Existing fine-tuning [26]–[29], editing-based [30]–[33], and inference-time [34]–[36] methods have made progress toward this trade-off. However, most of them are developed for specific generative architectures, such as U-Net-based [37]–[39] or DiT-based diffusion models [40], [41]. Their effectiveness often relies on architecture-specific decisions about where to intervene and how to construct the erasure objective. Consequently, transferring them to heterogeneous generative frameworks usually requires non-trivial redesign rather than direct plug-and-play deployment.

A more desirable framework should be architecture-agnostic and readily applicable to diverse visual generative models. Inference-time methods are especially promising for this goal because they intervene during generation without retraining or permanently editing model parameters. Our previous work, AdaVD (Adaptive Value Decomposer) [42], offers preliminary evidence for this direction by showing that value orthogonal decomposition can effectively balance erasure efficacy and prior preservation across a broad range of U-Net-based diffusion models. However, AdaVD was not originally formulated as a universal framework, and its adaptation to heterogeneous generative architectures remains underexplored. In this work, we systematically generalize value orthogonal decomposition into a universal concept erasure framework for diverse visual generative models.

To build a universal inference-time concept erasure framework, three key questions must be addressed: where to erase, what to erase, and how to erase safely. The first question, “where to erase,” requires identifying a common semantic space in which concept erasure can be consistently performed across different generative architectures, forming the foundation of a universal framework. The second question, “what to erase,” involves accurately locating the target semantics within this space. This is non-trivial because explicit concepts are often associated with specific object tokens, whereas implicit safety-related concepts may be scattered across ordinary contextual tokens and heterogeneous text encoders such as CLIP and T5. The third question, “how to erase safely,” focuses on preserving irrelevant content and the model’s generative prior. This challenge is particularly pronounced in DiT-based image generators, video generators, and other models with strong multimodal interactions, where coarse interventions can easily disrupt the generative distribution, causing structural collapse in images or degraded visual consistency in videos.

To address these challenges, we introduce Uni-AdaVD, a universal inference-time concept erasure framework for diverse visual generative models. For “where to erase,” Uni-AdaVD uses the value space of multimodal attention layers as the common intervention space across different generative models, inspired by AdaVD. For “what to erase,” we propose Encoder-aware Target Representation Construction (ETRC), which introduces encoder-specific probing to collect scattered semantic features and align target semantics across heterogeneous text encoders. For “how to erase safely,” we combine Orthogonal Value Decomposition (OVD) with Layer-wise Adaptive Erasing Shift (LAES), which adaptively controls erasure strength across network depths to remove target semantics while preserving non-target content and generative priors.

Our main contributions are summarized as follows:

• Universal Inference-time Concept Erasure: We propose Uni-AdaVD, a universal inference-time concept erasure framework that performs concept erasure via value orthogonal decomposition across diverse visual generative models, including U-Net-based, DiT-based, autoregressive image generation, and video generation models.

• Adaptive Localization and Erasure: We introduce encoder-specific probing to localize scattered target semantics across heterogeneous text encoders, together with a layer-wise adaptive erasing shift to adaptively control erasure strength for prior-preserving concept removal.

• Extensive Evaluation and Robustness: Comprehensive experiments demonstrate that Uni-AdaVD achieves effective concept erasure across diverse generative architectures while preserving non-target content, with strong robustness against both black-box and white-box adversarial prompt attacks.

1 Related Work↩︎

1.1 Concept Erasure in U-Net-Based Diffusion Models↩︎

Early concept erasure studies mainly focused on U-Net-based latent diffusion models. In these models, text conditions are usually injected into visual features through cross-attention layers. Based on how the target concept is removed, existing methods can be divided into three main groups: fine-tuning, editing-based, and inference-time methods.

Fine-tuning methods remove target concepts by fine-tuning the base model or learning auxiliary parameters. ESD [26] uses the conditioned and unconditioned predictions of a frozen diffusion model as negative guidance to fine-tune the model, thereby discouraging the updated model from generating the target concept. Concept Ablation (CA) [27] redirects a target concept toward a predefined anchor concept and replaces its original semantics with a safer or more general one. To improve parameter efficiency, SPM [29] learns lightweight concept-specific adapters instead of updating the full model. MACE [28] extends this line to mass concept erasure by combining closed-form cross-attention refinement, concept-specific LoRA fine-tuning, and interference-aware multi-LoRA integration. A closely related line formulates concept removal as a machine unlearning problem. SalUn [43] identifies the parameters that are most important for forgetting and focuses the updates on them. AdvUnlearn [44] incorporates adversarial training to improve resistance to concept recovery attacks, while STEREO [45] uses a two-stage optimization process to improve robust erasure. ReCARE [46] further emphasizes the preservation of benign concepts that frequently co-occur with the target concept.

Editing-based methods directly change selected model parameters without standard iterative fine-tuning. UCE [31] uses a closed-form solution to update cross-attention parameters and supports the editing of several concepts at the same time. RECE [32] also removes target concepts through lightweight post-hoc parameter editing. These methods are usually more efficient than fine-tuning methods. However, they still change the original model weights, and their editing rules often depend on specific model components.

Inference-time methods suppress target concepts during inference without updating the original model parameters. Safe Latent Diffusion (SLD) [34] changes classifier-free guidance during the denoising process to reduce unsafe generation. SAFREE [35] finds unsafe semantic directions in the text embedding space and moves the prompt representation away from these directions. It also adjusts the denoising process based on the input prompt. PGCE [47] uses concept prototypes to suppress broad and complex unsafe concepts during inference. These methods avoid model retraining and permanent parameter changes. However, they still depend on specific inference-time conditioning mechanisms, which limits their direct transfer across heterogeneous generative architectures.

1.2 Concept Erasure in DiT-Based Diffusion Models↩︎

Recent text-to-image models, such as SD v3 [17] and FLUX [48], use diffusion transformers and flow-matching objectives. Unlike U-Net-based diffusion models, these models often use dual-stream or single-stream transformer blocks to process text and image tokens. These changes affect both the generation process and the interaction between text and visual features. Therefore, many concept erasure methods designed for U-Net-based models cannot be directly applied to DiT-based models.

Existing methods for DiT-based models mainly include fine-tuning and inference-time methods. Among the fine-tuning methods, EraseAnything [40] uses bi-level optimization with LoRA adaptation and attention-map regularization to remove target concepts while preserving unrelated content. Z-Erase [49] focuses on single-stream diffusion transformers and separates the effects of text and image tokens. It also uses adaptive erasure modulation to balance concept removal and content preservation. DVE [41] is an inference-time method for flow-matching models. It removes concept-specific components from the velocity field along a differential direction defined by the difference between the target and anchor concepts.

1.3 Concept Erasure in Autoregressive Models↩︎

Autoregressive visual generators, such as VAR [10], Switti-AR [52], and Infinity-2B [53], generate images token by token or scale by scale. Unlike diffusion models, they do not use an iterative denoising process. Therefore, concept erasure methods based on noise prediction, velocity fields, or denoising trajectories cannot be directly applied to autoregressive models.

Existing concept erasure methods for autoregressive visual generators are mainly fine-tuning. For instance, EAR [54] fine-tunes autoregressive models with Windowed Gradient Accumulation and Thresholded Loss Masking to remove target concepts while reducing damage to unrelated content. VCE [55] builds contrastive image pairs and uses DPO-based training to suppress unsafe concepts while preserving non-target generation.

2 Preliminaries↩︎

Text-conditioned visual generators inject textual semantics into the visual generation process through attention modules. Let \(\mathbf{Q}\in\mathbb{R}^{N_{\mathrm{q}}\times d}\), \(\mathbf{K}\in\mathbb{R}^{N_{\mathrm{k}}\times d}\), and \(\mathbf{V}\in\mathbb{R}^{N_{\mathrm{k}}\times d}\) denote the query, key, and value matrices, respectively, where \(N_{\mathrm{q}}\) and \(N_{\mathrm{k}}\) are the query and key/value sequence lengths, and \(d\) is the attention feature dimension. The attention output is \[\operatorname{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \operatorname{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V}. \label{eq:attn95formula95general}\tag{1}\] Since the output is a weighted linear combination of value vectors, the value space provides a natural interface for controlling injected textual semantics, while \(\mathbf{Q}\) and \(\mathbf{K}\) mainly determine attention routing and allocation [56]. We omit layer indices in the main text unless necessary.

We next summarize this attention interface in U-Net-based, DiT-based, and autoregressive generators.

2.1 U-Net-Based Generators↩︎

In U-Net-based latent diffusion models, let \(\mathbf{Z}_t\) denote the noisy visual features at denoising step \(t\), and let \(\mathbf{C}\) denote the contextualized text features. The queries are computed from noisy image features, whereas the keys and values are computed from text features: \[\mathbf{Q} = \mathbf{W}_{\mathrm{Q}}\mathbf{Z}_t,\quad \mathbf{K} = \mathbf{W}_{\mathrm{K}}\mathbf{C},\quad \mathbf{V} = \mathbf{W}_{\mathrm{V}}\mathbf{C}, \label{eq:unet95qkv}\tag{2}\] where \(\mathbf{W}_{\mathrm{Q}}\), \(\mathbf{W}_{\mathrm{K}}\), and \(\mathbf{W}_{\mathrm{V}}\) are learnable projection matrices. This formulation makes value-space intervention directly applicable to U-Net-based models.

2.2 DiT-Based Generators↩︎

DiT-based generators represent the visual state at denoising step \(t\) as an image-token sequence and often adopt tighter multimodal coupling. In joint-attention architectures such as FLUX and SD v3, modality-specific queries, keys, and values are first computed as \[\begin{align} &\mathbf{Q}_{\mathrm{I}} = \mathbf{W}_{\mathrm{Q},\mathrm{I}}\mathbf{Z}_t,\quad \mathbf{K}_{\mathrm{I}} = \mathbf{W}_{\mathrm{K},\mathrm{I}}\mathbf{Z}_t,\quad \mathbf{V}_{\mathrm{I}} = \mathbf{W}_{\mathrm{V},\mathrm{I}}\mathbf{Z}_t,\\ &\mathbf{Q}_{\mathrm{T}} = \mathbf{W}_{\mathrm{Q},\mathrm{T}}\mathbf{C},\quad \mathbf{K}_{\mathrm{T}} = \mathbf{W}_{\mathrm{K},\mathrm{T}}\mathbf{C},\quad \mathbf{V}_{\mathrm{T}} = \mathbf{W}_{\mathrm{V},\mathrm{T}}\mathbf{C}, \end{align} \label{eq:joint95qkv95def}\tag{3}\] where the subscripts \(\mathrm{I}\) and \(\mathrm{T}\) denote the image and text branches, respectively. Accordingly, \(\mathbf{Q}_{\mathrm{I}}, \mathbf{K}_{\mathrm{I}}, \mathbf{V}_{\mathrm{I}} \in \mathbb{R}^{L_{\mathrm{I}}\times d}\) and \(\mathbf{Q}_{\mathrm{T}}, \mathbf{K}_{\mathrm{T}}, \mathbf{V}_{\mathrm{T}} \in \mathbb{R}^{L_{\mathrm{T}}\times d}\), where \(L_{\mathrm{I}}\) and \(L_{\mathrm{T}}\) denote the image and text sequence lengths. These features are concatenated along the sequence dimension as \(\mathbf{Q}=[\mathbf{Q}_{\mathrm{I}};\mathbf{Q}_{\mathrm{T}}]\), \(\mathbf{K}=[\mathbf{K}_{\mathrm{I}};\mathbf{K}_{\mathrm{T}}]\), and \(\mathbf{V}=[\mathbf{V}_{\mathrm{I}};\mathbf{V}_{\mathrm{T}}]\), so that \(\mathbf{Q}, \mathbf{K}, \mathbf{V}\in\mathbb{R}^{(L_{\mathrm{I}}+L_{\mathrm{T}})\times d}\). The resulting attention is \[\operatorname{JointAttention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \operatorname{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V}. \label{eq:joint95attn95formula}\tag{4}\]

Compared with U-Net-based cross-attention, joint attention introduces stronger text-image coupling. As shown in Fig. 1, swapping \(\mathbf{V}\) changes the generated subject identity much more than swapping \(\mathbf{K}\), supporting the value space as the primary intervention interface. In practice, we intervene only on the text value component \(\mathbf{V}_{\mathrm{T}}\) to better preserve visual priors.

2.3 Autoregressive Generators↩︎

Autoregressive text-to-image generators synthesize images token by token or scale by scale rather than through iterative denoising. Let \(\mathbf{H}_k\) denote the visual hidden states at decoding step \(k\). Text conditions are still injected through attention: \[\mathbf{Q} = \mathbf{W}_{\mathrm{Q}}\mathbf{H}_k,\quad \mathbf{K} = \mathbf{W}_{\mathrm{K}}\mathbf{C},\quad \mathbf{V} = \mathbf{W}_{\mathrm{V}}\mathbf{C}. \label{eq:ar95qkv}\tag{5}\] Therefore, autoregressive generators are also compatible with the same value-space intervention view.

Overall, although U-Net-based, DiT-based, and autoregressive generators differ in visual parameterization and update dynamics, they all admit attention-based text conditioning, which motivates a unified value-space formulation for concept erasure across architectures.

3 Method↩︎

This section presents Uni-AdaVD (Universal Adaptive Value Decomposer), a universal inference-time framework for concept erasure in text-conditioned visual generators. Despite their architectural differences, U-Net-based, DiT-based, and autoregressive models all rely on attention-mediated text injection, which exposes the text value space as a shared intervention interface. As illustrated in Fig. 2, Uni-AdaVD builds on this observation by constructing encoder-aware target representation (ETRC, Section 3.1, Fig. 2 (a)), removing target-aligned components via Orthogonal Value Decomposition (OVD; Section 3.2, Fig. 2 (b)), and adaptively modulating the erasure strength through Layer-wise Adaptive Erasing Shift (LAES; Section 3.3, Fig. 2 (c)) to better preserve non-target priors.

3.1 Encoder-aware Target Representation Construction↩︎

After establishing the text value space as the common intervention interface, we next construct target representations tailored to different text encoders. This is necessary because semantic information is not distributed uniformly across encoder architectures. In particular, CLIP-family and T5-family text encoders exhibit markedly different token interaction patterns, which in turn lead to different semantic distributions.

To characterize this difference, we visualize the encoder attention maps in Fig. 3. CLIP exhibits a causal attention pattern, in which each token attends only to itself and preceding tokens; as a result, contextual information progressively accumulates toward later positions. By contrast, T5 shows dense bidirectional interactions over valid tokens, indicating that semantic information is distributed across the sequence rather than concentrated near its end.

We further examine this behavior through token-level value masking, as shown in Fig. 4. Specifically, we partition prompt tokens into content tokens, which describe the subject and its attributes, and end-of-text tokens, and then zero out the corresponding value vectors under three settings: \([\mathbf{0}, \mathbf{V}_{\mathrm{EOT}}]\), \([\mathbf{V}_{\mathrm{content}}, \mathbf{0}]\), and \([\mathbf{V}_{\mathrm{content}}, \mathbf{V}_{\mathrm{EOT}}]\). In CLIP-based SD v1-4, masking content tokens effectively removes explicit concepts such as “Snoopy”, whereas masking [EOT] tokens is more effective for suppressing implicit concepts such as “Nudity”. This suggests that, under causal attention, explicit concepts are concentrated near the last subject token, while implicit concepts accumulate toward the later part of the sequence. In T5-based FLUX, by contrast, neither content-token masking nor end-of-text-token masking reliably removes the target concept, indicating that semantic evidence is distributed across valid tokens.

Based on these observations, we propose Encoder-aware Target Representation Construction. Let \(\mathbf{C}_t = [\boldsymbol{c}_{t,1}, \boldsymbol{c}_{t,2}, \dots, \boldsymbol{c}_{t,L}]^T \in \mathbb{R}^{L \times d_c}\) denote the contextualized token features of the target concept to be erased, where \(L\) is the text sequence length and \(d_c\) is the text-encoder feature dimension. Let \(t_s\) denote the index of the last subject token. We extract an anchor \(\boldsymbol{c}_a\) from the target concept and then broadcast it to construct a target feature sequence \(\tilde{\mathbf{C}}_t\).

For CLIP, following the observed causal accumulation, we extract the semantic anchor \(\boldsymbol{c}_a\) from the end of the subject-related token span. For explicit concepts, we set \(\boldsymbol{c}_a = \boldsymbol{c}_{t,t_s}\), which provides the most localized representation. For implicit concepts, we instead average the token features from the last subject token \(t_s\) to the [EOT] token, i.e., \(\boldsymbol{c}_a = \frac{1}{N_{\mathrm{implicit}}} \sum_{i=t_s}^{i_{\mathrm{EOT}}} \boldsymbol{c}_{t,i}\), where \(i_{\mathrm{EOT}}\) denotes the index of the [EOT] token and \(N_{\mathrm{implicit}} = i_{\mathrm{EOT}} - t_s + 1\) is the span of implicit semantic tokens. After obtaining \(\boldsymbol{c}_a\), we preserve the start token \(\boldsymbol{c}_{t,1}\) and define \(\tilde{\mathbf{C}}_t^{\mathrm{CLIP}} = [\boldsymbol{c}_{t,1}, \boldsymbol{c}_a, \dots, \boldsymbol{c}_a]^T\).

For T5, since the bidirectional mechanism disperses semantic evidence across the sequence, no individual token provides a sufficiently stable anchor. Therefore, unlike CLIP, we do not distinguish between explicit and implicit concepts for T5-based encoders, and instead use a unified sequence-level construction for both. Let \(\mathcal{I}_{\mathrm{valid}} \subseteq \{1,2,\dots,L\}\) denote the set of valid token indices excluding padding and end-of-text tokens, and let \(N_{\mathrm{valid}} = |\mathcal{I}_{\mathrm{valid}}|\). We then compute \(\boldsymbol{c}_a\) as the mean of all valid token features, i.e., \(\boldsymbol{c}_a = \frac{1}{N_{\mathrm{valid}}} \sum_{i \in \mathcal{I}_{\mathrm{valid}}} \boldsymbol{c}_{t,i}\), and define \(\tilde{\mathbf{C}}_t^{\mathrm{T5}} = [\boldsymbol{c}_a, \dots, \boldsymbol{c}_a]^T\). This design treats the target concept as a sequence-level representation, reducing positional uncertainty introduced by bidirectional token interaction.

The reconstructed target feature sequence \(\tilde{\mathbf{C}}_t\) is then projected into the value space, yielding the target value sequence \(\tilde{\mathbf{V}}_t \in \mathbb{R}^{L \times d_v}\) in cross-attention and the textual target value sequence \(\tilde{\mathbf{V}}_{\mathrm{T},t} \in \mathbb{R}^{L \times d_v}\) in joint attention, where \(d_v\) is the value feature dimension. These projections are computed using the corresponding value projection matrices \(\mathbf{W}_{\mathrm{V}}\) and \(\mathbf{W}_{\mathrm{V},\mathrm{T}}\). The resulting target value sequences serve as the reference for the subsequent value-space intervention.

AdaVD studied target value construction only for the CLIP text encoder and used a single construction strategy without distinguishing different types of target concepts. In this extended version, we first examine how target semantics are distributed in the value spaces of different text encoders, including both CLIP and T5. We further divide target concepts into explicit and implicit concepts according to their semantic forms and token-level distributions. Based on these findings, we design different target representations for explicit CLIP concepts, implicit CLIP concepts, and concepts encoded by T5. Therefore, the proposed encoder-aware construction extends AdaVD from a CLIP-specific target representation to a more general design that supports heterogeneous text encoders and different types of concepts.

3.2 Orthogonal Value Decomposition↩︎

Given the encoder-aware target representation constructed above, Uni-AdaVD removes target semantics by projecting text-derived value vectors onto the orthogonal complement of a target concept subspace. This yields a unified value-space operation: the target-aligned component is removed, while the residual component is retained for subsequent generation.

For a single target concept, let \(\boldsymbol{v}^i \in \mathbb{R}^{d_v}\) denote the text-derived value vector at textual token position \(i\), and let \(\hat{\boldsymbol{v}}^i \in \mathbb{R}^{d_v}\) denote the corresponding target concept vector obtained from the target value representation, where \(i \in \{1,2,\dots,L\}\) and \(d_v\) is the value feature dimension. We remove the target concept by projecting \(\boldsymbol{v}^i\) onto the orthogonal complement of \(\operatorname{span}(\hat{\boldsymbol{v}}^i)\): \[\begin{align} \boldsymbol{v}_{\mathrm{R}}^i &= P_{\operatorname{span}(\hat{\boldsymbol{v}}^i)^\perp}(\boldsymbol{v}^i) \\ &= \left(\mathbf{I}_{d_v} - \frac{\hat{\boldsymbol{v}}^i(\hat{\boldsymbol{v}}^i)^T}{(\hat{\boldsymbol{v}}^i)^T\hat{\boldsymbol{v}}^i}\right)\boldsymbol{v}^i \\ &= \boldsymbol{v}^i - \frac{(\hat{\boldsymbol{v}}^i)^T\boldsymbol{v}^i}{(\hat{\boldsymbol{v}}^i)^T\hat{\boldsymbol{v}}^i}\hat{\boldsymbol{v}}^i, \end{align} \label{eq:ovd95single}\tag{6}\] where \(P_{\operatorname{span}(\hat{\boldsymbol{v}}^i)^\perp}\) denotes the orthogonal projection onto the complement of \(\operatorname{span}(\hat{\boldsymbol{v}}^i)\), and \(\mathbf{I}_{d_v} \in \mathbb{R}^{d_v \times d_v}\) is the identity matrix. This operation removes the component of \(\boldsymbol{v}^i\) aligned with the target direction \(\hat{\boldsymbol{v}}^i\) while preserving the orthogonal residual \(\boldsymbol{v}_r^i\). For encoders with a dedicated structural start token, the intervention is omitted at that position (\(i=1\)).

The same formulation extends naturally to multiple target concepts. Given \(n\) target concepts, let \(\hat{\boldsymbol{v}}^{h,i} \in \mathbb{R}^{d_v}\) denote the target concept vector for the \(h\)-th concept at textual token position \(i\), and define the target subspace at token position \(i\) as \(\mathcal{S}^i = \operatorname{span}\{\hat{\boldsymbol{v}}^{1,i}, \hat{\boldsymbol{v}}^{2,i}, \dots, \hat{\boldsymbol{v}}^{n,i}\}\). After Gram–Schmidt orthogonalization, we obtain an orthonormal basis \(\{\boldsymbol{o}^{h,i}\}_{h=1}^r\) of \(\mathcal{S}^i\), where \(r \le n\) denotes the effective rank of the target direction set. The erased value vector is then computed as \[\begin{align} \boldsymbol{v}_\mathrm{R}^i &= P_{(\mathcal{S}^i)^\perp}(\boldsymbol{v}^i) \\ &= \left(\mathbf{I}_{d_v} - \sum_{h=1}^r \boldsymbol{o}^{h,i}(\boldsymbol{o}^{h,i})^T\right)\boldsymbol{v}^i \\ &= \boldsymbol{v}^i - \sum_{h=1}^r \left((\boldsymbol{o}^{h,i})^T\boldsymbol{v}^i\right)\boldsymbol{o}^{h,i}, \end{align} \label{eq:ovd95multi}\tag{7}\] where \(P_{(\mathcal{S}^i)^\perp}\) denotes the orthogonal projection onto the complement of the target subspace \(\mathcal{S}^i\). By removing the components of \(\boldsymbol{v}^i\) lying in the target subspace, OVD suppresses one or multiple target concepts while preserving non-target semantics orthogonal to that subspace.

This operation is instantiated according to the underlying attention design. For generators that realize text conditioning through cross-attention, including the U-Net-based and autoregressive models considered in this work, OVD is directly applied to value vectors projected from text features. For joint-attention generators such as FLUX and SD v3, we intervene only on the text value component

\[\mathbf{V}_{\mathrm{T}} = \left[\boldsymbol{v}_{\mathrm{T}}^1, \boldsymbol{v}_{\mathrm{T}}^2, \dots, \boldsymbol{v}_{\mathrm{T}}^\ell\right]^T \in \mathbb{R}^{\ell \times d_v}\] while keeping the visual value component \[\mathbf{V}_{\mathrm{I}} = \left[\boldsymbol{v}_{\mathrm{I}}^1, \boldsymbol{v}_{\mathrm{I}}^2, \dots, \boldsymbol{v}_{\mathrm{I}}^m\right]^T \in \mathbb{R}^{m \times d_v}\] unchanged. As illustrated in Fig. 2 (c), this design confines the intervention to prompt-induced semantics and helps preserve visual priors.

3.3 Layer-wise Adaptive Erasing Shift↩︎

Although OVD effectively removes target-aligned components, using a fixed erasing shift across all tokens and network layers may also suppress weakly related or non-target information. To better balance erasure efficacy and prior preservation, we introduce Layer-wise Adaptive Erasing Shift (LAES). LAES adjusts the removed target-aligned component according to the similarity between the current value vector and the target representation. It also uses a layer-specific threshold to account for different semantic distributions across network depth.

For a single target concept, let \(\boldsymbol{v}^i \in \mathbb{R}^{d_v}\) denote the text-derived value vector at token position \(i\), and let \(\hat{\boldsymbol{v}}^i \in \mathbb{R}^{d_v}\) denote the corresponding target concept vector. LAES revises Eq. 6 by introducing a token-wise shift factor \(\delta_\ell(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i) \in \mathbb{R}\): \[\boldsymbol{v}_{\mathrm{R}}^i = \boldsymbol{v}^i - \delta_\ell(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i) \frac{(\hat{\boldsymbol{v}}^i)^T \boldsymbol{v}^i}{(\hat{\boldsymbol{v}}^i)^T \hat{\boldsymbol{v}}^i} \hat{\boldsymbol{v}}^i, \label{eq:aes95single}\tag{8}\] where \(\ell\) denotes the layer index. When \(\delta_\ell(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i)=1\), Eq. 8 reduces to standard OVD.

We define the shift factor using a sigmoid function: \[\delta_\ell(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i) = \frac{s}{1+\exp\left(-p\left(\cos(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i)-\epsilon_\ell\right)\right)}, \label{eq:aes95delta}\tag{9}\] where \(\cos(\hat{\boldsymbol{v}}^i,\boldsymbol{v}^i) = \frac{(\hat{\boldsymbol{v}}^i)^T \boldsymbol{v}^i}{\|\hat{\boldsymbol{v}}^i\|_2\|\boldsymbol{v}^i\|_2}\), \(s>0\) controls the maximum shift factor, \(p>0\) controls the sharpness of the transition, and \(\epsilon_\ell\) is a layer-specific similarity threshold. Larger cosine similarity leads to stronger erasure, while weakly aligned tokens receive smaller shifts.

LAES extends naturally to multi-concept erasure. Let \(\{\hat{\boldsymbol{v}}^{h,i}\}_{h=1}^n\) denote the target concept vectors of \(n\) concepts at token position \(i\), and let \(\{\boldsymbol{o}^{k,i}\}_{k=1}^r\) be the orthonormal basis of their span, where \(r \le n\). Writing each target concept vector as \(\hat{\boldsymbol{v}}^{h,i} = \sum_{k=1}^{r} w_{hk}^i \boldsymbol{o}^{k,i}\) with coefficient matrix \(\mathbf{W}^i = [w_{hk}^i] \in \mathbb{R}^{n \times r}\), we define a basis-wise shift factor by aggregating the concept-specific shifts according to the absolute basis coefficients: \[\delta_{\ell,k}^i = \frac{\sum_{h=1}^{n} |w_{hk}^i|\, \delta_\ell(\hat{\boldsymbol{v}}^{h,i},\boldsymbol{v}^i)}{\sum_{h=1}^{n} |w_{hk}^i|}, \qquad k=1,\dots,r. \label{eq:aes95basis95delta}\tag{10}\] The resulting LAES-adjusted multi-concept operator is \[\boldsymbol{v}_{\mathrm{R}}^i = \boldsymbol{v}^i - \sum_{k=1}^{r} \delta_{\ell,k}^i \left((\boldsymbol{o}^{k,i})^T \boldsymbol{v}^i\right)\boldsymbol{o}^{k,i}. \label{eq:aes95multi}\tag{11}\] When all \(\delta_{\ell,k}^i = 1\), Eq. 11 reduces to the multi-concept OVD operator in Eq. 7 .

The semantic roles of different network layers are not the same. Some layers contain stronger target-related information, while other layers mainly encode general visual structures or non-target semantics. Therefore, using one global threshold for all layers may cause insufficient erasure in target-sensitive layers and excessive intervention in other layers. To address this issue, Uni-AdaVD estimates a separate threshold \(\epsilon_\ell\) for each layer. The threshold is determined based on the difference between the target similarity and the non-target similarity at that layer. A detailed analysis of the layer-wise similarity distributions and threshold settings is provided in the Supplementary Material. In this way, LAES adapts the erasing strength to the semantic characteristics of each network layer.

This layer-wise design is another key extension over the Adaptive Erasing Shift used in AdaVD. AdaVD uses a fixed threshold for all network layers and does not model layer-wise differences between target and non-target similarities. In contrast, LAES estimates a separate threshold for each layer and adjusts the erasing shift according to its semantic separation. As demonstrated by the results, this design provides a better balance between concept erasure and prior preservation.

Particularly, for FLUX, OVD and LAES are applied only to the dual-stream transformer blocks, where text and image value sequences remain separated. We do not apply LAES to the single-stream blocks because the text and image representations are tightly mixed in these layers. Restricting the intervention to the text value branch of the dual-stream blocks helps reduce direct changes to visual information and better preserves the original generative prior.

4 Experiments↩︎

In this section, we first introduce the experimental setup, including the evaluated models, datasets, protocols, and metrics. We then compare Uni-AdaVD with existing methods on multiple types of target concepts, including implicit safety concepts, explicit instances, and artistic styles. These comparisons include quantitative evaluations, qualitative results, and robustness analysis under adversarial prompt attacks. Next, we extend Uni-AdaVD to text-to-video generation models to evaluate its transferability beyond image generation. Finally, we conduct ablation studies to examine the effects of the main components and parameter settings. More implementation details and additional experimental results are provided in the Supplementary Material.

4.1 Experimental Setups↩︎

(a) Quantitative Results on the FLUX Architecture

(b) Quantitative Results on the SD v3 Architecture

(c) Quantitative Results on the Switti-AR Model

4.1.1 Models and Implementation Details↩︎

We evaluate the proposed Uni-AdaVD on a range of image and video generation models with different architectures. These models include the U-Net-based diffusion model SD v1-4 [14], the DiT-based diffusion models FLUX [48] and SD v3 [17], and the autoregressive model Switti-AR [52]. We also evaluate two text-to-video models, CogVideoX [25] and ZeroScopeT2V [57]. More implementation details are provided in Supplementary Material.

4.1.2 Datasets and Evaluation Protocols↩︎

For explicit concept erasure, we follow the evaluation protocol of SPM [29]. For implicit concepts, we use the I2P benchmark [58] to evaluate NSFW concept erasure in image generation. We use SafeSora [59] to evaluate safety concept erasure in text-to-video generation. The COCO dataset [60] is used to measure non-target prior preservation and general generation quality. For robustness evaluation, we consider adversarial prompts from Ring-A-Bell [61], MMA [62], P4D [63], and UnlearnDiff [64].

4.1.3 Evaluation Metrics↩︎

For explicit concepts, we use CLIP Score [65] (CS) to measure the semantic similarity between the prompt and the generated images. For concepts that should be erased, a lower CS indicates stronger erasure. For implicit concepts, we use NudeNet [66] with a confidence threshold of \(0.6\) to detect nudity-related content. We use the Q16 classifier [67] to evaluate other NSFW categories. Lower detection counts or violation rates indicate better erasure performance. To evaluate prior preservation, we report Fréchet Inception Distance (FID) [68], Structural Similarity Index Measure (SSIM) [69], and Learned Perceptual Image Patch Similarity (LPIPS) [70].

4.2 On NSFW Concept Erasure↩︎

4.2.1 Results on SD v1-4↩︎

We first evaluate nudity erasure on the U-Net-based SD v1-4 using I2P and COCO. As shown in Table 5, the original SD v1-4 model produces 406 nudity detections, while Uni-AdaVD reduces this number to 26. Uni-AdaVD achieves the best erasure performance among all evaluated inference-time methods. Compared with AdaVD, Uni-AdaVD reduces the detection count from 95 to 26, showing a clear improvement in erasure effectiveness. For prior preservation, Uni-AdaVD also achieves the best or second-best results on most metrics among the inference-time methods. Moreover, it provides better overall prior preservation than most fine-tuning and model-editing methods. To further compare the balance between erasure effectiveness and prior preservation, we present a trade-off scatter plot in Fig. [fig:comparison]. Lower ASR indicates stronger target suppression, while lower FID indicates better preservation of the original generation capability. As shown in the figure, Uni-AdaVD is located close to the lower-left corner and achieves the most favorable trade-off among the compared methods. The qualitative results in Fig. 6 further support these findings. On I2P prompts, Uni-AdaVD reliably suppresses nudity-related content, while methods such as CA and SPM may still leave visible target-related details. On COCO prompts, the outputs of Uni-AdaVD remain close to those of the original model in object structure, lighting, color, and local texture. These results show that Uni-AdaVD effectively removes the target concept while preserving the general generation quality of SD v1-4.

1pt

4.2.2 Results on FLUX and SD v3↩︎

We next evaluate Uni-AdaVD on the DiT-based diffusion models FLUX and SD v3. As shown in Table ¿tbl:tab:comprehensive95erasure95results? (a) and (b), Uni-AdaVD achieves the lowest nudity detection counts among all evaluated inference-time methods on both models, which is consistent with the results on the U-Net-based SD v1-4 model. When all competing methods are considered, Uni-AdaVD achieves the second-best erasure performance on both FLUX and SD v3, only behind the fine-tuning EA method. Uni-AdaVD also shows strong prior preservation. On both models, it achieves the highest CS among the compared erasure methods. On FLUX, Uni-AdaVD further obtains the best FID, SSIM, and LPIPS among the inference-time methods. Its results on the remaining preservation metrics are also competitive with those of fine-tuning and model-editing methods. Moreover, we also visualize some generative examples in Fig. 7. As observed, Uni-AdaVD successfully suppresses the nudity concept on both models while keeping the COCO outputs close to the original models in object contours, textures, and background consistency.

4.2.3 Results on Switti-AR↩︎

To evaluate the applicability of Uni-AdaVD beyond diffusion models, we further conduct experiments on the autoregressive image generator Switti-AR. As shown in Table ¿tbl:tab:comprehensive95erasure95results? (c), Uni-AdaVD achieves the lowest nudity detection count among all compared methods. It also obtains the best prior preservation performance. Fig. 8 also visually compares nudity erasure and prior preservation, where Uni-AdaVD effectively suppresses nudity-related content while largely preserving the layout, visual structure, and non-target details of the original outputs.

4.2.4 Extension to Broad Safety Concepts↩︎

We further extend the evaluation from nudity to a broader range of implicit NSFW concepts. Following the I2P benchmark, we use the Q16 safety classifier to measure the violation rate across seven unsafe categories on SD v1-4 and FLUX. As reported in Table 1, Uni-AdaVD consistently reduces unsafe generation across different concepts and model architectures. On SD v1-4, it achieves the lowest violation rate in all seven categories, while on FLUX, it obtains the best results in six of the seven categories and the second-best result in the remaining one. In particular, the violation rates for harassment/violence content are reduced from 17.6%/38.2% to 6.7%/4.5% for SD v1-4 and from 19.7%/23.8% to 12.3%/12.0% for FLUX, respectively. These results show that Uni-AdaVD can effectively suppress a wide range of implicit safety concepts across both U-Net-based and DiT-based diffusion models.

4.2.5 Robustness Under Adversarial Attacks↩︎

Considering the potential risks of malicious prompt engineering in real-world deployments, we evaluate our defensive robustness against four adversarial attacks: Ring-A-Bell, MMA, UnlearnDiff, and P4D. Table 2 reports the Attack Success Rate (ASR) on SD v1-4, where a lower value indicates stronger defense. Uni-AdaVD achieves the lowest overall ASR among all inference-time methods, reducing the score from 42.59 for AdaVD to 27.17. Uni-AdaVD also outperforms several fine-tuning and model-editing methods, including ESD, CA, SPM, UCE and RECE, in terms of overall ASR. Some fine-tuning methods, such as SalUn, STEREO, and ReCARE, achieve lower overall ASRs, but they require additional optimization and modify the original model parameters. In contrast, Uni-AdaVD improves adversarial robustness without retraining or weight updates, showing a strong balance between defense effectiveness and deployment efficiency.

4.3 On Instance Concept Erasure↩︎

We evaluate the instance erasure performance of Uni-AdaVD on the DiT-based FLUX and SD v3 models, as well as the autoregressive Switti-AR model¹. Here, we investigate single-concept (“Snoopy”), dual-concept (“Snoopy”, “Mickey”) and triple-concept (“Snoopy”, “Mickey”, “SpongeBob”) erasure. Table 3 reports the performance comparison on FLUX. While results on SD v3 and Switti-AR are provided in the Supplementary Material.

For single-concept erasure, Uni-AdaVD obtains the second-best CS for the erased “Snoopy” concept, while achieving the lowest FID on all four non-target concepts. For dual-concept erasure, Uni-AdaVD achieves the second-best CS on “Snoopy” and remains competitive on “Mickey”. At the same time, it obtains the lowest FID on all three retained concepts. A similar trend is observed in the triple-concept setting. Overall, Uni-AdaVD maintains effective target suppression as the number of erased concepts increases and consistently provides the strongest preservation of non-target concepts. This shows that the proposed value-space intervention supports both single- and multi-concept erasure without causing large changes to unrelated generation. More detailed experimental results are provided in Supplementary Material.

4.4 On Artistic Style Erasure↩︎

We next evaluate Uni-AdaVD on artistic style erasure, including “Pencil Sketch”, “Van Gogh”, “Stained Glass”, “Pixel”, and “Gothic”. Table 4 reports the corresponding quantitative results on FLUX, where a lower CS for the target style indicates stronger erasure, and a lower FID for the remaining styles indicates better preservation of non-target generation. Overall, Uni-AdaVD ranks first on five of the fifteen reported metrics and second on the remaining ten. Some methods achieve strong target suppression but introduce large changes to non-target styles. For example, EA obtains the best CS for “Pencil Sketch”, but its FID values are clearly higher than those of Uni-AdaVD. In contrast, Uni-AdaVD consistently maintains effective style erasure and strong non-target preservation, showing a favorable balance across different artistic styles. Experimental results on SD v3 are provided in Supplementary Material.

4.5 Extension to Video Generation↩︎

We further extend our Uni-AdaVD from image generation to text-to-video generation and evaluate safety concept erasure on two representative models ZeroScopeT2V and CogVideoX. These two models use different attention mechanisms. For ZeroScopeT2V, Uni-AdaVD is applied to the value vectors in cross-attention modules. For CogVideoX, which uses joint attention, the intervention is restricted to the text-token part of the value sequence. Table 5 reports the violation rates for five safety categories. Uni-AdaVD achieves the best result in eight of the ten model-category combinations and the second-best result in the remaining two. Notably, VideoEraser is specifically designed for concept erasure in video generation models, whereas Uni-AdaVD is a general framework developed for diverse image and video generative architectures. Despite this broader design, Uni-AdaVD outperforms VideoEraser in most of the evaluated settings. It also consistently outperforms the inference-time SAFREE baseline across all categories on both models. These results demonstrate that the proposed value-space intervention transfers effectively to video generation. Additional qualitative results are provided in Supplementary Material.

4.6 Ablation Study↩︎

We conduct ablation studies to examine the main design choices of Uni-AdaVD. Specifically, we analyze the intervention space, the hyperparameter settings of the Value Decomposer, and the denoising timesteps at which the erasure operation is applied.

First, we investigate where the erasure operation should be applied in the DiT-based FLUX model, whose joint-attention blocks maintain separate text and image feature branches. We compare interventions on the key, value and both key and value components within either the text or image branch. As shown in Table 6, applying the intervention to the text-side key-only provides limited target suppression. Intervening in either the value or key+value components of the text branch can effectively erase the target concept. The text-side key+value setting achieves the lowest target CS of 19.95, while the value-only setting obtains a similar CS of 20.10. However, their prior-preservation performance differs considerably. The text-side value-only setting achieves FID values of 9.25 and 16.43 on the non-target concepts “Dog” and “Cat”, whereas the corresponding values increase to 78.22 and 88.35 when both key and value are modified. The qualitative results in Fig. 9 support this observation. Both text-side value and key+value interventions effectively suppress “Snoopy”. However, modifying both key and value causes visible changes to the appearance and structure of the non-target concept. In contrast, intervening only in the text-side value component preserves the non-target subject and scene more faithfully. Image-side interventions, especially on the value component, may even cause severe visual corruption or structural collapse. Therefore, we apply Uni-AdaVD only to the text-derived value vectors, as this setting provides the best balance between target erasure and non-target prior preservation.

Second, we analyze the hyperparameters of the Value Decomposer, including the steepness \(p\), the scaling factor \(s\), and the similarity threshold \(\epsilon_\ell\) used in Eq. (9 ) to compute the adaptive shift factor. Fig. 10 shows the results under different combinations of \(p\) and \(s\). The scaling factor \(s\) has a clear influence on erasure strength: increasing \(s\) progressively suppresses the visual characteristics of the target concept. In contrast, changing \(p\) within the evaluated range produces relatively small differences in the erasure results. For the non-target concept, the generated images remain visually consistent across most parameter combinations, indicating that both \(p\) and \(s\) have limited influence on prior preservation. Overall, Uni-AdaVD is more sensitive to \(s\) in terms of target erasure, while its preservation of non-target concepts remains relatively stable. We further compare a fixed threshold \(\epsilon=0.5\) with the proposed layer-specific thresholds \(\epsilon_\ell\) in Table 7. The layer-specific thresholds achieve better performance on both target erasure and non-target preservation compared to the fixed threshold. These results show that a single global threshold cannot adequately capture the different semantic distributions across network layers. In contrast, the layer-specific thresholds provide stronger target suppression while better preserving non-target concepts.

Third, for diffusion-based generators, we study the effect of applying concept erasure at different denoising timesteps. As shown in Fig. 11, effective target suppression mainly depends on the intervention during the initial denoising stage, particularly timesteps 1-6 (in total of 30). When erasure is applied during these early steps, the target concept can be effectively removed regardless of whether the intervention is continued in the subsequent steps. In contrast, applying the intervention only after the first six steps cannot reliably suppress the target concept. This indicates that the main semantic structure of the generated content is established at the beginning of the denoising process. Considering erasure effectiveness, computational efficiency, and prior preservation, we therefore apply Uni-AdaVD only during timesteps 1-6.

5 Conclusion↩︎

In this paper, we have presented Uni-AdaVD, a universal inference-time framework for concept erasure across diverse visual generative models. Uni-AdaVD uses the value space of multimodal attention as a shared intervention space and combines Encoder-aware Target Representation Construction, Orthogonal Value Decomposition, and Layer-wise Adaptive Erasing Shift to suppress target-aligned semantics without fine-tuning or modifying the original model weights. The proposed design supports heterogeneous text encoders, explicit and implicit concepts, and both single- and multi-concept erasure. Extensive experiments on U-Net-based diffusion models, DiT-based diffusion models, autoregressive image generators, and text-to-video models demonstrate effective target removal, strong preservation of non-target content, and robustness against adversarial prompt attacks. These results show that value-space intervention provides a practical and transferable solution for improving the safety and controllability of modern visual generative models.

6 Implementation Details↩︎

6.1 On Implementation↩︎

For evaluation, we follow the SPM protocol for explicit concept assessment, using 80 instance templates, 30 art style templates, and 25 celebrity templates. For each concept-template pair, we generate 10 samples with fixed initial noise. For implicit concept evaluation, we use the I2P benchmark for image safety assessment and the COCO-30k reference set for preservation evaluation. For video safety evaluation, we use the SafeSora dataset.

To ensure fair and consistent comparisons across methods, we adopt the predefined seeds and classifier-free guidance (CFG) scales provided by public benchmarks whenever available. For the SafeSora benchmark and our customized template generation, we use a fixed seed (seed 0) and a default CFG scale of 7.5, which is reduced to 3.5 for FLUX and CogVideoX. For all baselines and erased models, outputs are generated from identical initial latent noises (for diffusion models) or visual token prefixes (for autoregressive models) under the same dataset case identifiers.

All experiments are conducted on NVIDIA RTX 5880 GPUs. Unless otherwise specified, baseline implementations follow the default configurations provided in their official code repositories. As a training-free method, Uni-AdaVD operates exclusively at inference time. Table 8 summarizes the intervention spaces and the main hyperparameters (\(p, \epsilon_\ell, s\)) across all evaluated architectures.

8pt

Generation Settings: Image diffusion models utilize the DPM-Solver with 30 denoising steps. For video generation, videos are rendered at 16 frames, and a temporal truncation strategy is strictly applied, limiting the value-space intervention to the first 40 denoising steps. Across all architectures, self-attention and temporal attention modules remain completely unaltered.

6.2 Additional Hyperparameter Analysis↩︎

This section analyzes the effect of the layer-specific similarity threshold \(\epsilon_\ell\) used in Eq. (9). A smaller threshold makes the gating function more permissive, which can strengthen target removal but may also suppress non-target tokens that are semantically close to the target concept. In contrast, a larger threshold restricts the intervention to more strongly aligned tokens, improving prior preservation but potentially weakening erasure. A single fixed threshold is difficult to apply across all layers because semantic activation patterns vary with network depth in DiT architectures. As illustrated in Fig. 12 for the 19 joint-attention blocks of FLUX, prompts containing the target concept, such as “Snoopy”, generally exhibit higher cosine similarity to the target representation than non-target prompts, such as “Mickey”. However, both the similarity values and the separation between target and non-target prompts vary across layers. We therefore assign a layer-specific threshold \(\epsilon_\ell\), shown by the black dashed line, according to the target-non-target similarity gap at each layer. This provides an adaptive decision boundary that better distinguishes target-related semantics from non-target priors throughout the network.

7 Additional Experiments and Results↩︎

7.1 Additional Experiments Under Other Implicit NSFW Concepts↩︎

Fig. 13 presents additional examples of erasing other implicit NSFW concepts, including self-harm, hate, violence, sexual content, illegal activity, and shocking scenes, on both SD v1-4 and FLUX. In these cases, Uni-AdaVD reduces unsafe visual cues and often redirects the outputs toward safer alternatives, while largely preserving scene coherence and basic prompt relevance.

7.2 Additional Robustness Experiments Under Adversarial Attacks↩︎

The main manuscript quantitatively evaluates the robustness of Uni-AdaVD against adversarial prompt attacks on SD v1-4. To provide further analysis, we present qualitative examples for these attacks and extend the evaluation to the DiT-based SD v3 and FLUX models.

Fig. 14 illustrates examples against four adversarial attack frameworks: Ring-A-Bell, MMA, UnlearnDiff, and P4D. We consider both white-box attacks (UnlearnDiff and P4D), which use gradient information and internal model access, and black-box attacks (Ring-A-Bell and MMA), which rely on prompt perturbations or transferability. Under these attacks, the original models are frequently induced to generate nudity-related content. In contrast, Uni-AdaVD suppresses the target unsafe concept in most of the presented examples while largely preserving the scene layout and general visual structure. In some cases, the generated content is redirected toward a safer scene that remains semantically related to the input prompt. These qualitative results further support the adversarial robustness observed in the main experiments. They also show that the proposed value-space intervention can be transferred from the cross-attention modules of U-Net-based models to the joint-attention modules of SD v3 and FLUX, providing effective protection under both black-box and white-box adversarial prompts.

7.3 Extended Results on Instance and Art Style Erasure↩︎

This section presents additional quantitative and qualitative results for instance and artistic style erasure. In addition to the CS and FID metrics reported in the main manuscript, we further include SSIM and LPIPS to evaluate structural and perceptual preservation.

We first present supplementary results for instance concept erasure across multiple architectures. For FLUX, the qualitative comparisons and extended quantitative results are reported in Fig. 15 and Table 9. For SD v3, the qualitative comparisons and corresponding quantitative results are provided in Fig. 16 and Table 10, respectively. We further extend this evaluation to the visual autoregressive setting, with complete results for Switti-AR reported in Fig. 17 and Table 11.

Besides, to further verify the erasure precision of Uni-AdaVD, we evaluate Uni-AdaVD on neighbor-concept erasure following the Neighbor-Aware Localized Concept Erasure benchmark [72]. We consider three fine-grained clusters: dog breeds, bird species, and flower categories. As shown in Fig. 18, we test three representative cases across SD v1-4, FLUX, and SD v3.

We next evaluate art style erasure. For FLUX, the qualitative comparisons and extended quantitative metrics are shown in Fig. 19 and Table 12, respectively. For SD v3, the qualitative and quantitative results are provided in Fig. 20 and Table 13. Overall, these extended results provide additional evidence that Uni-AdaVD suppresses the targeted concepts while largely preserving non-target content.

7.4 On Celebrity Erasure↩︎

In this section, we evaluate the capability of Uni-AdaVD to erase a specific celebrity identity by targeting the concept “Bruce Lee” on both the FLUX and SD v3 architectures. To assess prior preservation, we additionally examine the generative quality of unrelated non-target identities.

Table 14 reports the quantitative results on FLUX. While EA and ESD achieve lower CS on the target identity, they are also associated with noticeably higher FID scores on non-target identities such as “Marilyn Monroe”, “Melania Trump”, and “Anne Hathaway”. By contrast, Uni-AdaVD achieves a competitive CS for target removal while maintaining lower FID scores on the evaluated non-target identities.

The qualitative comparisons in Fig. 21 show a similar pattern. Although most baselines suppress the target identity in the top rows, methods such as ESD and SAFREE often introduce structural artifacts in non-target generations. Uni-AdaVD suppresses target identity cues while largely preserving the semantic consistency and visual quality of non-target identities.

8 On Transferability to Other T2I Models↩︎

8.1 Generalization to PixArt Architectures↩︎

To further validate the architectural generalizability of Uni-AdaVD, we extend our evaluation to the PixArt family, focusing specifically on the high-resolution PixArt-\(\Sigma\) [73] model. Unlike U-Net-based models and joint-attention DiT variants, PixArt-\(\Sigma\) adopts a pure DiT backbone paired with a T5 text encoder, injecting textual conditions exclusively through cross-attention rather than joint-attention blocks.

Because semantic conditioning in PixArt is mediated entirely by standard cross-attention, we apply our intervention directly to the text value space (\(\mathbf{V}_\mathrm{T}\)) within its transformer blocks. As shown in Fig. 22, we evaluate both instance erasure and artistic style erasure on PixArt-\(\Sigma\). In the instance-erasure setting, Uni-AdaVD effectively suppresses the target concept (“Snoopy”) while preserving the structural and semantic integrity of non-target concepts such as “Teddy” and “SpongeBob”. Likewise, in the style-erasure setting, the targeted “Van Gogh” brushstrokes are neutralized without degrading the generation of other distinctive artistic styles, including “Monet” and “Picasso”.

These results show that our value-space intervention remains effective and adaptable for DiT-based T2I models that rely on pure cross-attention conditioning, further demonstrating the broad transferability of the Uni-AdaVD framework.

8.2 Generalization to Infinity-2B↩︎

While the main text focuses on Switti-AR, we provide supplementary qualitative results on Infinity-2B to further validate the transferability of Uni-AdaVD across AR architectures. Relative to Switti-AR, Infinity-2B operates at a substantially larger scale, with 2 billion parameters, and adopts a more sophisticated discrete token prediction process, thereby providing a more challenging testbed for our Inference-time intervention.

Fig. 23 presents concept-erasure results on both AR models. The left panel illustrates the disentanglement of subject and style: erasing the target instance produces a generic animal while preserving the artistic style, whereas erasing the target style removes the characteristic brushstrokes without altering the subject identity.

The right panel further examines multi-subject interactive scenes. When intervening on either target concept, Uni-AdaVD successfully suppresses the specified concept while preserving the non-target entity. These results further demonstrate the robustness of our value-space intervention across heterogeneous AR generation pipelines.

9 Visualization on T2V Models↩︎

Following the analysis presented in the main manuscript, this section provides additional qualitative examples of safety concept erasure on video generation models. As illustrated in Fig. 24, we evaluate the erasure of specific severe safety violations: “Animal Abuse” and “Sexual” on ZeroScopeT2V and CogVideoX. While the original unprotected models synthesize frames with explicit violations under malicious prompts, Uni-AdaVD successfully neutralizes the targeted harmful semantics across the generated sequences.

10 Failure Case Study↩︎

Despite its overall effectiveness, Uni-AdaVD still encounters certain concepts that are difficult to erase completely. Fig. 25 presents representative failure cases together with their successful corrections.

For artistic style erasure on FLUX, generations associated with “Pencil Sketch” may exhibit only partial suppression when a relatively conservative scaling factor is used (e.g., \(s=5\)), still retaining residual grayscale strokes and sketch-like textures. This behavior likely stems from the strong visual prior of certain artistic styles in the pretrained model: the default parameter can attenuate the target style, but may not be sufficiently strong to fully remove the latent representation from the corresponding style manifold. Increasing the scaling factor to \(s=8\) effectively overcomes this strong prior, leading to a much more complete suppression of sketch-related cues, as shown on the right side of Fig. 25.

A similar phenomenon is observed for implicit concept erasure on SD v1-4. When the erasing strength is set conservatively (e.g., \(s=1\)), residual unsafe semantics may still remain in the generated image. Implicit concepts, such as nudity triggered by specific contextual combinations in the prompt, can be particularly resistant to mild erasing shifts. By moderately increasing the scaling factor to \(s=1.5\), Uni-AdaVD more effectively suppresses the remaining target semantics and produces a benign visual output.

References↩︎

---

1. We do not repeat the instance erasure experiments on the U-Net-based SD v1-4 model. As discussed earlier, SD v1-4 uses a CLIP text encoder, and for explicit instance concepts, the target value construction in Uni-AdaVD is the same as that used in AdaVD.↩︎