Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
June 25, 2026
Abstract
Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.
Warning: This paper contains model-generated content that may be disturbing.
1 Introduction↩︎
Autoregressive image generation models [1]–[4] produce images by sequentially predicting discrete visual tokens from a codebook, where each embedding corresponds to a quantized visual pattern. As a highly valuable text-to-image generation model, autoregressive image generation models possess distinct advantages compared to other image generation models, mainly in the following three aspects: 1) The unified and consistent structure of autoregressive models across image generation and Large Language Models (LLMs) [5], [6] has driven significant progress in recent unified multimodal models [7]–[11]. These advanced models can both understand and generate multimodal content. 2) In contrast to diffusion-based generation models that require multiple denoising steps, token-level autoregressive image generation speed can be significantly accelerated by adopting parallel processing of tokens. 3) The unified structure, which is identical to that of LLMs, equips autoregressive image generation models with promising capabilities in understanding and following text instructions. Existing studies [12], [13] have revealed the issue of harmful image generation in diffusion-based models. Correspondingly, this raises an interesting question: how safe are the images generated by such autoregressive-based unified multimodal models?
Extensive research [12]–[15] has been proposed to improve the safety of diffusion-based [16], [17] image generation models. However, these methods typically operate in continuous spaces such as the latent semantic space of images during the diffusion generation process, and thus cannot generalize well to the discretized representations of autoregressive generation. In this work, we propose iterative self-improving codebooks for safe autoregressive image generation. Unlike previous methods, our approach is built on two core pillars: 1) We leverage the capability of autoregressive-based unified model itself to simultaneously generate and understand images. This enables it to provide feedback for unsafe generation without an annotated dataset and without human annotation, thereby identifying unsafe outputs. 2) We fix the inherent discretized representations within the codebooks to eliminate harmful mappings while maintaining image quality.
Concretely, our iterative self-improvement method consists of two steps: In the first step, we use the unified model itself to identify unsafe generations in responses to both harmful and harmless prompts. Based on the model’s understanding, we then construct corresponding harmful and safe image-text pairs. Using this paired data, we construct Harmful Space by comparing the differences in visual features between harmful and safe embeddings during the model’s inference process. The unified model’s internal codebook is then updated using this harmful space, which effectively eliminates harmful image generation. In the second step, we adaptively fine-tune the model codebook within the null space of the harmful space using safe image-text pairs. This not only preserves the high quality of the generated images, but also prevents the reintroduction of additional harmful information during training, since the training parameters are constrained to the space orthogonal to harmful information. These two steps are then repeated until no further improvement is observed, ultimately obtaining a safe model codebook. Figure [fig0] presents a visual comparison of the image safety and quality produced by the unified model before and after applying our proposed method.
Extensive experiments are conducted to verify the effectiveness of our method. Specifically, we evaluate its ability to mitigate harmful generations on eight harmful-prompt datasets such as I2P [12] and CoPro [18], [19], and assess whether the original capabilities of the models are preserved on various standard benchmarks after applying our method. We further validate the iterative self-improving capability of the method for specific harmful concepts, showing that iterative removal outperforms single-turn removal under the same data volume. Additionally, we verify the applicability of the approach across five unified multimodal generation models, including the Janus [8], [9] and VILA-U [10]. Moreover, experiments on out-of-distribution (OOD) data demonstrate that the proposed method generalizes well. Our contributions can be summarized as follows:
We address the safety of image generation in autoregressive unified multimodal models and are the first to systematically explore this challenge in such frameworks;
We propose iterative self-improving codebooks for safe autoregressive image generation, which use the model’s own capabilities to enhance the safety of generated images;
Extensive experiments demonstrate the effectiveness of our method. Specifically, the safety of unified models is improved iteratively without using an annotated dataset or human annotation.
2 Related Work↩︎
2.1 Image Autoregressive Generation↩︎
Image autoregressive generation models predict and generate each subsequent image element based on the previous one. Several earlier pixel-based autoregressive image generation methods [3], [20]–[22] have been proposed, generating images pixel by pixel by converting 2D images into 1D sequences via raster scan and treating individual pixels as visual elements, which is computationally expensive. Inspired by the token-by-token generation paradigm in NLP tasks [23]–[27], VQ-VAE [28] uses Vector Quantization (VQ) technology to map continuous image representations to the closest vectors in a fixed-size codebook, compressing and quantizing image representations. This enables more efficient processing of high-resolution image content. The structural consistency between image autoregressive generation models and LLMs [5], [6], [24] makes them attractive for the development of unified multimodal models. Recent works have proposed unified multimodal generation and understanding models [7]–[11], [29], which can both understand and generate multimodal content.
2.2 Image Generation Safety↩︎
Image generation models may be misused intentionally or unintentionally [30]. Such misuse involves generating harmful content that may be offensive, threatening, or otherwise cause anxiety. Recent studies on mitigating the generation of harmful content have focused on diffusion-based generation models. Pre-trained data filtering methods [31]–[33] can play a certain role, but are only effective for filtering overtly harmful content such as pornographic material. Furthermore, the resources required to retrain the model make this approach extremely costly when addressing issues discovered after training. Post-generation content filtering methods [32], [34]–[36] use NSFW detectors to detect generated data and filter out inappropriate content. However, they also introduce biases that the detectors have learned and are easy to circumvent [34], [37]. Methods for constructing safe diffusion models [37]–[39] eliminate specific harmful concepts by fine-tuning the parameters of diffusion models. The method of directional guidance during inference [12], [13], [40] generates safe images by guiding the diffusion model in a specific latent semantic space. SAFREE [40] integrates filtering across both textual embeddings and visual latent spaces, ensuring coherent safety checking while preserving the fidelity, quality, and safety of the generated outputs. SAFREE demonstrates excellent performance in both continuous image generation and video generation within the diffusion framework. However, the issue of image generation safety in autoregressive text-to-image generation has not yet been fully studied.
2.3 Self-Improving and Iterative Generation↩︎
One of the effective techniques to improve capabilities in LLMs is enabling them to introspect on their own outputs and correct errors [41]. To achieve model self-improvement, recent methods have attempted to reuse knowledge already stored in pre-trained models via few-shot prompting [42]–[46]. Prompt tuning combined with feedback can effectively elicit improved responses. Additionally, numerous studies have sought to fine-tune LLMs to acquire self-improvement capabilities [47]–[50]. However, no relevant research has yet been conducted on the safety of image generation in autoregressive unified multimodal models.
In diffusion model-based T2I generation, existing studies have leveraged self-improvement to generate images that better align with users’ subtle intentions [51]–[55]. However, diffusion models lack internal feedback, so incorporating human feedback to align generated outputs with user expectations has become a trend. The HIVE framework [54] uses RLHF to fine-tune a diffusion-based image editor. Users rank multiple outputs to train a reward model, which guides generation to faithfully follow instructions. DiffChat [55] utilizes LLMs for multi-turn conversations and employs reinforcement learning to refine prompts based on aesthetics, content completeness, and user preferences. However, all these methods require external feedback, our goal is to use the model’s internal feedback for self-improvement and iterative generation of safer images.
3 Method↩︎
3.1 Problem Formulation↩︎
Autoregressive unified multimodal understanding and generation models may produce harmful images. Given a text prompt \(x\) that is semantically benign, for example, "A painting of a gorgeous woman.", the image \(y\) generated by a unified multimodal model may still contain harmful content, such as nudity. Our goal is to eliminate the representation of harmful concepts in the multimodal model by modifying its codebook, so that the resulting image \(y^+\) generated from the same prompt \(x\) no longer contains unsafe content.
To achieve this goal, we propose the construction of an iterative and self-improving safe codebook. This codebook can be directly integrated into the unified multimodal model at inference time, ensuring safe and high-quality image generation. Moreover, the safe codebook is expected to be effective across multiple harmful concepts. For example, it should mitigate both nudity and violent content simultaneously. In other words, the safe codebook should be capable of self-improvement to adapt to incremental harmful concepts.
3.2 Iterative Self-Improving Codebook↩︎
In this section, we describe how to construct an iterative self-improving safe codebook in an autoregressive unified model. The framework is illustrated in Figure 1. Panels (a) and (b) correspond to Section 3.2.1, where we explain the process of building a harmful space based on the relevant harmful and safe image-text pairs, and removing the inappropriate information in the codebook that falls within this harmful space. Panel (c) corresponds to Section 3.2.2, where we describe adaptive fine-tuning of the model codebook based on the null space of the harmful space. Finally, we can repeat Step 1(Section 3.2.1) and Step 2(Section 3.2.2), these two steps, until no additional improvements are observed. The details of this iterative self-improving algorithmic process are provided in Appendix 6.
3.2.1 Projecting Codebook into Harmless Space↩︎
A unified multimodal model generates images via autoregressive token-to-token prediction. It produces a sequence of tokens, which are subsequently decoded into an image by a decoder. Specifically, given an input prompt \(x\), the text is encoded into an embedding that serves as the conditioning input for model inference. Based on this embedding, the model predicts the first token, which corresponds to the first image patch. The embedding vector of this token, retrieved from the codebook, is then fed back into the model as the input to predict the next token. In the codebook, the embedding vectors associated with tokens represent the feature expressions of image patches within the unified multimodal model. After \(K\) tokens are predicted through the iterative process, the sequence of \(K\) tokens is decoded to obtain the final semantically coherent image.
First, paired safe and harmful image-text pairs are obtained using the approach illustrated in Figure 1(a). Specifically, given a set of \(N\) text prompts that share a similar specific harmful attribute, denoted as \(X\), we generate the corresponding image set \(Y\) using the multimodal model.
Then, we use the same model \(\Gamma\) to assess whether each image in \(Y\) contains harmful content. Based on this assessment, we identify a subset of prompts \(X^u\) whose corresponding generated images are classified as harmful.
For each harmful prompt in \(X^u\), following the harmful–benign prompt pairing strategy of datasets such as ViSU [56] and CoPro [18], [19], and guided by predefined prompt templates, we use the unified model to replace unsafe terms with safe alternatives while minimally altering other semantics. This approach constructs a semantically similar but safer version with minimal modification, resulting in a new set of safe prompts \(X^s\). These safe prompts, when passed through the model, generate the corresponding safe images \(Y_s\). The final dataset used for constructing the harmful space can be represented as: \(S = \{(x_i^u, x_i^s), (y_i^u, y_i^s)\}_{i=1}^n\).
Second, as illustrated in Figure 1(b), we construct the harmful space and remove harmful information associated with this space from the original codebook. Specifically, we utilize the obtained prompt pairs \((x_i^u, x_i^s)_{i=1}^n\), to extract their semantic representations during the unified multimodal model’s inference process. Specifically, we extract the embedding features of the \(K\) tokens used in the image generation process for each prompt from the model’s codebook. This can be expressed as: \[(F_i^u, F_i^s) = \Gamma(x_i^u, x_i^s), \quad i = 1, \dots, n\] where \(F_i^u \in \mathbb{R}^{K \times D}\) represents the embedding features of the harmful prompt \(x_i^u\) in the codebook of the model \(\Gamma\) and \(F_i^s \in \mathbb{R}^{K \times D}\) represents the embedding features of the corresponding safe prompt \(x_i^s\). \(K\) denotes the total number of tokens used to represent each prompt and \(D\) is the dimensionality of the embedding feature of each token.
We aim to represent the feature vector corresponding to a specific harmful concept in the model by leveraging these paired safe and harmful feature representations. Specifically, we compute the difference between the feature embeddings of each pair of prompts (safe and harmful), and use this resulting difference vector as the feature vector characterizing the particular harmful concept. For pairs of prompts, the difference vector matrix \(E\) is computed: \[E = \frac{1}{n} \sum_{i=1}^{n} \left( F_i^u - F_i^s \right), \quad E \in \mathbb{R}^{K \times D}\] For \(n\) prompt pairs, each producing a difference matrix \(E_i\) that represents the same harmful concept, we compute a unified representation for this specific concept by taking the average. Since each difference matrix \(E_i\) is computed from a pair of prompts that are semantically similar that differ only in the inclusion of harmful content, the resulting unified difference matrix \(E\) is expected to capture the core features associated with the harmful information. In contrast, differences in background or benign semantics should be minimal and largely cancel out through averaging over multiple samples.
To find the main information corresponding to harmful content, we apply Singular Value Decomposition (SVD) to the matrix \(E\) to extract its principal components, which we use to define the harmful subspace corresponding to the concept. The SVD is performed as follows: \[E = U \cdot \Sigma \cdot V^\top, \quad U \in \mathbb{R}^{K \times K},\;V \in \mathbb{R}^{D \times D}\] Here, \(\Sigma\) is a diagonal matrix containing \(K\) singular values, and \(V\) represents the singular directions in the feature dimension. Since the singular values in \(\Sigma\) are arranged in descending order, we select the top-\(k\) right singular vectors from \(V\) corresponding to the largest \(k\) singular values. Denoting these vectors as \(V_k = [v_1, v_2, \ldots, v_k], \quad V_k \in \mathbb{R}^{D \times k}\), we construct the harmful space projection matrix \(P = V_k \cdot V_k^\top, \quad P \in \mathbb{R}^{D \times D}\).
For harmful spaces corresponding to multiple harmful concepts, such as those containing both sexual and violent content, we follow the same procedure to obtain their respective difference matrices \(E^{sex}\) and \(E^{vio}\), and the corresponding top-\(k\) singular vectors \(V_k^{\text{sex}}\) and \(V_k^{\text{vio}}\). We then concatenate the singular directions: \(V_k = [ V_k^{\mathrm{sex}} \mid V_k^{\mathrm{vio}} ],\quad V_k \in \mathbb{R}^{D \times 2k}\). Using this combined matrix, we construct an expanded harmful space \(P'\) that captures both sexual and violence-related features. When additional harmful concepts are introduced, the construction of the extended harmful space follows the same procedure.
We utilize the constructed harmful space to remove harmful embedding features from the codebook of the unified multimodal model. This is achieved by subtracting the projection of the original codebook \(W\) onto the harmful subspace \(P\), thereby ensuring that the resulting embeddings are orthogonal to \(P\) and thus free from harmful information. This process can be expressed as: \[W^{\text{proj}} = W^{\text{ori}} \left( I - \lambda \cdot \mathrm{Proj}_P \left( W^{\text{ori}} \right) \right)\] where \(W^{\text{proj}}\) denotes the projected codebook from which relevant information of the harmful space has been removed via projection.
3.2.2 Adaptive Codebook Fine-Tuning In Harmless Space↩︎
As illustrated in the adaptive codebook fine-tuning process shown in Figure 1(c), this section provides a detailed explanation of the reasons for and the specific procedure of the fine-tuning. The projected codebook obtained by removing harmful information from the original codebook using the constructed harmful subspace effectively eliminates the undesired content. However, this hard projection may also lead to degrading image quality, potentially degrading the visual quality and detail of the generated images. To address this, we further propose an image-adaptive fine-tuning step to the projected codebook. This fine-tuning aims to enhance the quality and fidelity of image generation while ensuring that no new harmful information is reintroduced during the process.
To theoretically ground this approach, we leverage null space concepts: For two matrices \(A\) and \(B\). If and only if \(BA=0\), then \(B\) lies in the null space of \(A\), meaning that \(B\) contains no information present in \(A\). See Adam-NSCL [57] for more details.
Following this principle, if we project an additional perturbation matrix \(\Delta\), to be added to the codebook, into the null space of the difference matrix \(E\), then \(\Delta\) will not introduce any harmful information captured by \(E\). Therefore, we optimize a perturbation \(\Delta \in \mathbb{R}^{\dim(W)}\) with the same dimensionality as the codebook \(W\), while enforcing that it remains in the null space of \(E\) during the optimization process. This allows us to enhance the visual quality and detail of the generated images without introducing any additional harmful information.
To ensure that the perturbation \(\Delta\) lies in the null space of the difference matrix \(E\), we use the previously computed SVD decomposition of \(E\). From the decomposition, we have identified the top-\(k\) singular vectors in \(V\) representing the dominant (harmful) directions. To obtain the null space, we select the singular vectors in \(V\) corresponding to singular values in \(\Sigma\) that are close to zero. These vectors form the matrix \(V_{null}\). The corresponding projection matrix onto the null space is then given by: \(Q = V_{\text{null}} \cdot V_{\text{null}}^\top, \quad Q \in \mathbb{R}^{D \times D}\). This matrix \(Q\) can be used to project \(\Delta\) into the null space of the difference matrix E, ensuring that the perturbation does not reintroduce harmful information. For more detailed derivations on null space construction, please refer to [58].
During the adaptive fine-tuning process, we form training image–text pairs using safe prompts and their corresponding target images. During the forward pass, as the model generates tokens autoregressively from the prompt, we compute a cross‑entropy loss between each generated token and the corresponding token from the encoded target image. This loss guides gradient‑based updates to the corresponding entries in the perturbation \(\Delta\). Critically, after each update, we project \(\Delta\) using the null‑space projection matrix \(Q\) to ensure the injected information remains orthogonal to the harmful subspace. After iterating over the training set, we obtain the final \(\Delta\). Formally, the fine‑tuning objective can be written as: \[L = \arg\min\limits_{\Delta} ( f(W^{\text{proj}} + \Delta \cdot Q) - Y_{\text{target}} )\] where \(f(\cdot)\) denotes the multimodal model image generation function and \(Y_\text{target}\) represents the target (reference) image. After training, the projected codebook \(W^{\text{proj}}\) is further updated by the learned perturbation, resulting in the final safe codebook \(W^{\text{safe}}\): \[W^{\text{safe}} = W^{\text{proj}} + \Delta \cdot Q\] Images generated using this final safe codebook are free from harmful content while still preserving the original high-quality visual details.
4 Experiments↩︎
| Category | I2P | CoPro | ViSU | |||
|---|---|---|---|---|---|---|
| 2-3 | Baseline | Safe-CB | Baseline | Safe-CB | Baseline | Safe-CB |
| Sexual | 0.12 | 0.04 | 0.022 | 0.013 | 0.14 | 0.05 |
| Violence | 0.39 | 0.21 | 0.28 | 0.18 | 0.47 | 0.32 |
| Hate | 0.38 | 0.19 | 0.19 | 0.14 | 0.32 | 0.16 |
| Self-harm | 0.40 | 0.23 | 0.26 | 0.19 | 0.37 | 0.22 |
| Illegal activity | 0.25 | 0.17 | 0.31 | 0.21 | 0.26 | 0.18 |
| Harassment | 0.31 | 0.15 | 0.28 | 0.20 | 0.30 | 0.22 |
| Shocking | 0.53 | 0.32 | 0.27 | 0.19 | 0.33 | 0.22 |
4.1 Experiments settings↩︎
Dataset and Benchmark: We evaluate the effectiveness of our method on datasets containing various harmful prompts, including the I2P [12], CoPro [18], and ViSU [56] datasets. These three datasets include 7 categories of harmful content: sexual, violence, hate, illegal activity, harassment, self-harm, and shocking. Additionally, we evaluate our method on datasets focused on the sexual category of harmful content, including P4D [59] , MMA-Diffusion [60], UnlearnDiffAtk [61], and UD [62]. Notably, we also perform evaluations on the MPUP [63] dataset, which contains multimodal pragmatic unsafe prompts. Furthermore, the model’s preserved generation and reasoning capabilities are evaluated on the Geneval [64] and MMMU [65] benchmarks, respectively.
Models and Evaluation Metrics: Experiments are conducted on various unified multimodal models, including the Janus series [8], [9], VILA-U [10], Emu3 [11], LlamaGen [66], and OmniMamba [29]. Unless otherwise specified, all experiments are conducted on Janus models. When evaluating the generated images, the Nudenet 1 detector and Q16 classifier [36] are used to detect whether the images contain pornographic content or harmful content (e.g., violence), respectively. "Baseline" refers to the outputs of the original unified multimodal models, while "Safe-CB" represents the outputs after applying our Safe-CodeBook. We further evaluate the models after applying our method on the COCO-30k caption dataset [67] by computing the FID [68] between the generated and natural images, which we denote as \(FID_g\).
4.2 Performance on Various Datasets and Benchmarks↩︎
In this section, we investigate the effectiveness of our method in improving the safety of generated images across different benchmark datasets. For I2P, CoPro and ViSU datasets, we categorize their prompts containing harmful content into 7 classes and report the results for each class separately. The detailed experimental results are shown in Table 1. For the I2P dataset, we test all 4703 inappropriate prompts. For CoPro and ViSU datasets, which contain a large number of prompts per category, we randomly select 1000 prompts from the test set for each category to generate images and conduct experiments. In Table 1, for each dataset, "Baseline" denotes the detection results of whether images generated by the original model contain harmful content, while "Safe-CB" represents the detection results of images generated after applying our method. The experimental results show that our method significantly improves the safety of generated images across multiple datasets and different categories.
Meanwhile, to evaluate the mitigation effect on the more common pornographic content, we conduct further experiments on five datasets: P4D, MMA-Diffusion, UnlearnDiffAtk, UD, and MPUP. These datasets contain prompts from various sources that may generate pornographic images, and the degree of potential pornographic content in images generated by these source-specific prompts varies. Notably, the MPUP dataset is a Multimodal Pragmatic Unsafe Prompts dataset where each prompt consists of both a text prompt and a visual prompt. Its harmful content is categorized into four distinct types: hate speech, physical harm, fraud, and porn. The detailed results are shown in Appendix 7.1. Our method also effectively mitigates unsafe generation across these datasets, significantly improving the safety of images generated from various potentially inappropriate prompts. Additionally, we visualize the mitigation effect of our safe codebook on harmful content in Figure [fig0], demonstrating that it enables the generation of safer images without compromising image quality.
Inherent Generative and Understanding Capabilities of the Model: As shown in Figure 2, we visualize the Janus model’s inherent image generation capability and image understanding-based text generation capability both before and after applying our method. The results demonstrate the content generated by the model across the relevant benchmarks. We use the Geneval benchmark to evaluate the model’s image generation capability and the MMMU benchmark to assess the model’s image understanding-based text generation capability. The experimental results indicate that Safe-CB does not cause significant damage to the model’s inherent capabilities.
Furthermore, to evaluate prompt–image alignment after applying our method, we conduct experiment on benign prompts from the MS‑COCO dataset, supplementing evaluation with CLIP‑Score and TIFA metrics. The results as shown in Table 3, the hard projection codebook (before applying \(\Delta\)) leads to a noticeable drop in alignment. After fine‑tuning with \(\Delta\), useful visual information in the codebook is effectively recovered, and Safe‑CB achieves near‑preserved semantic consistency on benign prompts.
Capability of the Model in Identifying Unsafe Images: The safe and harmful image-text pairs used to construct the harmful space are identified by the model’s judgment of generated content. Therefore, the ability of the unified multimodal model to understand and judge whether the generated images contain harmful content is crucial. We verify this judgment ability of the unified model through experiments, with detailed experimental data and analysis provided in Appendix 7.3. The experimental results show that the unified model exhibits excellent ability to understand and judge whether generated images contain harmful content on common benchmarks. The model’s judgment results demonstrate consistency with human judgment outcomes. This enables the unified model to effectively classify data into corresponding safe and harmful image-text pairs. Additional analysis on the impact of human annotations for subtle harmful concepts is provided in Appendix 7.4.
| Models | Sexual | Violence | Hate | Self-harm | Illegal | Harassment | Shocking | \(FID_g\) \(\downarrow\) | |
|---|---|---|---|---|---|---|---|---|---|
| Janus-pro(1B) | Baseline | 0.12 | 0.39 | 0.38 | 0.40 | 0.25 | 0.31 | 0.53 | 68.83 |
| Safe-CB | 0.04 | 0.21 | 0.19 | 0.23 | 0.17 | 0.15 | 0.32 | 70.66 | |
| Janus-pro(7B) | Baseline | 0.09 | 0.32 | 0.28 | 0.36 | 0.32 | 0.23 | 0.47 | 67.90 |
| Safe-CB | 0.05 | 0.18 | 0.18 | 0.19 | 0.15 | 0.14 | 0.28 | 68.27 | |
| VILA-U | Baseline | 0.11 | 0.34 | 0.33 | 0.41 | 0.26 | 0.30 | 0.51 | 69.66 |
| Safe-CB | 0.03 | 0.16 | 0.20 | 0.20 | 0.16 | 0.16 | 0.31 | 69.87 | |
| Emu 3 | Baseline | 0.13 | 0.38 | 0.36 | 0.38 | 0.28 | 0.29 | 0.49 | 66.42 |
| Safe-CB | 0.05 | 0.22 | 0.17 | 0.22 | 0.18 | 0.17 | 0.30 | 68.13 | |
| LlamaGen | Baseline | 0.14 | 0.40 | 0.33 | 0.37 | 0.27 | 0.28 | 0.50 | 71.22 |
| Safe-CB | 0.06 | 0.24 | 0.19 | 0.21 | 0.19 | 0.15 | 0.28 | 72.45 | |
| OmniMamba | Baseline | 0.13 | 0.40 | 0.34 | 0.36 | 0.28 | 0.30 | 0.47 | 70.66 |
| Safe-CB | 0.04 | 0.21 | 0.18 | 0.24 | 0.21 | 0.18 | 0.27 | 70.87 | |
| CLIP-Score \(\uparrow\) | TIFA \(\uparrow\) | |
|---|---|---|
| Baseline | 30.48 \(\pm0.29\) | 0.7925 \(\pm0.0028\) |
| (w/o fine-tune) | 26.11 \(\pm0.59\) | 0.7521 \(\pm0.0043\) |
| (w/ fine-tune) | 30.19 \(\pm0.42\) | 0.7894 \(\pm0.0039\) |
4.3 Performance on Different Models↩︎
In this experiment, we evaluate the effectiveness of our method in improving the safety of generated images across different models and model sizes. As shown in Table 2, we conduct experiments on five categories of models, including the Janus series, VILA-U, Emu3, LlamaGen, and OmniMamba. The Janus series comprises two models of different sizes: Janus Pro 1B and Janus Pro 7B. Meanwhile, we randomly selected 1000 samples from the COCO-30k dataset to calculate the FID between the generated images and natural images, denoted as \(FID_g\). This metric reflects the quality of images generated by different models both before and after applying our method. Although the sizes of the visual codebooks maintained during the inference process vary across different models, for image generation in these token-by-token unified multimodal models, the safe codebook constructed through our method can effectively enhance the safety of the generative models without causing significant degradation in image quality.
4.4 Iterative Self-Improvement of Harmful Concepts↩︎
We report the iterative self-improving capability of our method for specific concepts in Table 4. As the number of iterations increases, the proportion of generated images detected as containing harmful content decreases. We conduct experiments on four categories of harmful content. For each category, we used 100 pairs of data for self-improvement in the first iteration and added 100 more pairs of data in each subsequent iteration. The column "\(0\)" represents the detection results of harmful content in images generated by the original model. Similar experimental conclusions are observed across different categories of harmful content: as the number of iterations increases, the data used to construct the harmful space expands, leading to improved refinement of the spatial details and boundaries for specific harmful concepts. Based on this improved harmful space, the resulting safe codebook gradually enhances the safety of generated images, and the effect generally reaches saturation after 3 turns. The visual analysis of the experiments is presented in Appendix 8.
Compared with the "No turn" that uses the same data but only performs one-time removal, multiple iterative removals can converge to better results more quickly with less data. As shown in Table 4 by the comparison between the fifth and eighth columns of the experiment, when using 300 prompts to construct the harmful space, three iterations achieve better results than no turn. Additionally, using 200 prompts for two iterations already yields results similar to those of using 300 prompts without iteration. For the above conclusions, we further visualize the effects of some prompts with and without iteration in Table 5. As the number of iterations increases, harmful content is gradually and effectively removed. In contrast, the "No turn" approach may fail to eliminate such content.
| Iterations | 0 | 1(100) | 2(200) | 3(300) | 4(400) | 5(500) | No turn(300) |
|---|---|---|---|---|---|---|---|
| Sexual | 0.1278 | 0.0656 | 0.0457 | 0.0405 | 0.0401 | 0.0412 | 0.0472 |
| Violence | 0.3981 | 0.2944 | 0.2294 | 0.2047 | 0.2106 | 0.2023 | 0.2345 |
| Self-harm | 0.4019 | 0.3086 | 0.2489 | 0.2245 | 0.2275 | 0.2219 | 0.2439 |
| Shocking | 0.5327 | 0.4425 | 0.3416 | 0.3233 | 0.3371 | 0.3256 | 0.3445 |
| Prompt | Turn 0 | Turn 1(100) | Turn 2(200) | Turn 3(300) | No Turn(300) |
|---|---|---|---|---|---|
| “A natural portrait of an unclothed woman with soft.” | |||||
| “A close up of a person sitting with a laptop while blood splatters all over their face.” |
4.5 More Studies and Analysis↩︎
We compare our safe-CB construction method with the method of modifying model weights. Detailed experimental results and analysis are provided in Appendix 9. The method using our safe-CB can better remove harmful content while preserving the model’s original excellent generation capabilities, whereas the method of directly modifying model weights significantly degrades the model’s generation performance. Experimental results and analysis on OOD prompts are presented in Appendix 10, demonstrating that our safe-CB generalizes well to such data. Hyperparameter studies are included in Appendix 11, which illustrate the effect of varying the number of top-\(k\) principal component vectors used to define the harmful space. Additionally, we discuss the limitations of directly fine-tuning the codebook, in comparison to our method, in Appendix 12. Appendix 13 provides visual examples and analysis, including harmful images from the original unified model, safer images produced by our Safe‑CB method, and results from the safe‑model baseline that modifies model weights directly.
5 Conclusion↩︎
In this work, we explore the safety of images generated by autoregressive visual generation models. Leveraging the fact that unified multimodal models built on autoregressive architectures possess both text and image generation and understanding capabilities, we propose iterative self-improving codebooks for safe generations. Extensive experiments are conducted to verify the effectiveness of our method. The safe codebook approach enables iterative improvement of model safety without additional external feedback. Meanwhile, it performs well across various popular models and datasets, and an interesting advantage is its ability to generalize to novel datasets. For research on self-improving models, self-labeling risk and error propagation are inherent limitations since the underlying models are inherently imperfect. For future work, we will explore safety risks in multimodal generation, especially when models generate images and text together. Risks may arise in three forms: inappropriate text, inappropriate images, or combined content that triggers misunderstandings despite each part being normal alone.
Impact Statement↩︎
Our work addresses the critical safety challenges inherent in unified multimodal models by providing an autonomous mechanism for secure autoregressive image generation. Our methodology enables models to identify and mitigate harmful content through the iterative refinement and self-improvement of internal codebook representations. By reducing the reliance on human annotation and external datasets, this work fosters the development of more reliable and ethical generative systems. These advancements contribute to the broader goal of responsible artificial intelligence deployment and protect users from exposure to inappropriate visual content in practical applications.
APPENDIX
6 Detailed Algorithmic Procedure↩︎
In this section, we present the detailed algorithmic procedure for constructing the safe-codebook in a readable algorithm format, as shown in Algorithm 3.
All of our experiments are conducted on NVIDIA A40 or RTX 3090 GPUs. During the codebook fine‑tuning stage for perturbation \(\Delta\), we use a learning rate of \(1e‑4\), the Adam optimizer, and \(5\) training epochs. Image generation is performed with temperature \(1.0\) and a classifier‑free guidance scale of \(7.5\).
7 Experiment results on Various Datasets and Analysis of Model Understanding Ability↩︎
7.1 Experiment results on Various Datasets↩︎
| Dataset | P4D | MMA-Diffusion | UnlearnDiffAtk | UD | MPUP | |||
|---|---|---|---|---|---|---|---|---|
| 6-9 | Hate | Phy. | Porn | Fraud | ||||
| Baseline | 0.14 | 0.25 | 0.20 | 0.08 | 0.17 | 0.41 | 0.37 | 0.22 |
| Safe-CB | 0.02 | 0.03 | 0.05 | 0.01 | 0.12 | 0.27 | 0.19 | 0.17 |
In this section, we present the experimental results of our safe codebook on additional datasets containing harmful prompts, as shown in Table 6. The analysis of the experiments is provided in Section 4.2 of the main text.
7.2 Semantic Fidelity Evaluation↩︎
For semantic fidelity evaluation of generated images, we use CLIP-Score [69] and TIFA [70] on the COCO‑30k dataset, evaluating 1000 samples. To evaluate prompt–image alignment after applying our method, we conduct experiment on benign prompts from the MS‑COCO dataset, supplementing evaluation with CLIP‑Score and TIFA metrics. The results as shown in Table 3, the hard projection codebook (before applying \(\Delta\)) leads to a noticeable drop in alignment. After fine‑tuning with \(\Delta\), useful visual information in the codebook is effectively recovered, and Safe‑CB achieves near‑preserved semantic consistency on benign prompts.
| MPUP Dataset | Hate Speech | Physical Harm | Pornography | Fraud |
|---|---|---|---|---|
| Baseline | 0.17 | 0.41 | 0.37 | 0.22 |
| 0.12 | 0.27 | 0.19 | 0.17 | |
| 0.10 | 0.26 | 0.19 | 0.14 |
7.3 Analysis of Model Understanding Ability↩︎
In this experiment, we explored the accuracy of different methods for judging whether generated images contain harmful content. The experimental results are presented in Table 8. We conduct experiments on four categories of harmful content, including sexual, violence, self-harm, and shocking. The methods used to judge whether generated images contain harmful content include the unified multimodal model itself, detectors corresponding to harmful content, and human judgment. We perform the experiment using 500 images generated from the validation set prompts of the ViSU dataset. The values in the experimental results represent the number of images judged to contain harmful content by different methods. From the experimental results, it can be seen that the judgment results of the unified multimodal model itself exhibit good consistency with human judgment results, which confirms the rationality of using the model’s own judgment to identify whether generated images are harmful and further construct corresponding harmful and safe data pairs.
| Categories | Unified Model | Detector | ||||
|---|---|---|---|---|---|---|
| 2-7 | Accuracy | Recall | Cohen‘s Kappa | Accuracy | Recall | Cohen‘s Kappa |
| Sexual | 0.83 | 0.88 | 0.55 | 0.71 | 0.76 | 0.51 |
| Violence | 0.91 | 0.84 | 0.62 | 0.68 | 0.72 | 0.47 |
| Self-harm | 0.89 | 0.90 | 0.64 | 0.83 | 0.86 | 0.54 |
| Shocking | 0.78 | 0.81 | 0.52 | 0.74 | 0.79 | 0.51 |
7.4 Impact of Human Annotations on Subtle Harmful Concept Recognition↩︎
For subtle harmful concepts not covered in common benchmarks, the model’s judgment is indeed imperfect. Consequently, when high‑quality human annotations are available for complex or ambiguously defined harmful concepts, model safety improves to a measurable extent. As shown in Table 7 on the MPUP dataset, gains are marginal for clearly defined concepts such as “porn” and “physical harm”, where the model already performs reliably, but become noticeable for more nuanced concepts such as “hate speech” and “fraud”. Defining more complex harmful concepts remains an open‑ended challenge, which we plan to explore in future work.
8 Visual Analysis of Iterative Self-improving↩︎
This part corresponds to the visualization description and analysis of the experiment in Section 4.4 of the main text. The visualization results are shown in Figure 4. We visualize the detection of inappropriate ratios of generated images for four categories of harmful content: sexual, violence, self-harm, and shocking. The results show that similar experimental phenomena are observed for different categories of harmful content. As the number of iterations increases, the ratio of generated images detected as containing harmful content decreases, and it basically reaches saturation after 3 iterations.
Moreover, compared with the "No turn" method of one-time removal without iteration, when using the same 300 pairs of data, multiple iterations can achieve better results. Additionally, when using only 200 prompts for two iterations, the results can reach those of using 300 prompts for one-time removal.
9 Comparison with Weight Modification Method↩︎
In this experiment, we investigate and compare the effectiveness of two approaches, our Safe Codebook construction method and the direct model weight modification method in enhancing the safety of generated images while preserving the model’s inherent capabilities. The results as shown in Table 9, we evaluate both approaches across four key dimensions: the safety of images generated by using the sexual prompts from the I2P dataset, the FID, image quality on the Geneval benchmark, and the model’s understanding capabilities on the MMMU benchmark. For the "Safe-Model" approach, we adapt the mitigation method from [71] to improve image generation safety in unified multimodal models. The method was originally developed for LLMs and works by modifying their inherent reasoning weights to reduce hallucination outputs. The experimental results indicate that our safe codebook construction method not only better enhances the safety of generated images but also causes no significant damage to the model’s image generation and understanding abilities. In contrast, because the "Safe-Model" approach modifies model weights, it could degrade the model’s image generation and understanding abilities.
| Method | Sexual-classifier | \(FID_g\) | Geneval | MMMU |
|---|---|---|---|---|
| Baseline | 0.1278 | 68.83 | 0.64 \(\pm 0.0181\) | 29.1 \(\pm 0.3774\) |
| Safe-Model | 0.0746 | 92.13 | 0.54 \(\pm 0.0158\) | 26.7 \(\pm 0.3189\) |
| Safe-CB | 0.0440 | 70.66 | 0.62 \(\pm 0.0179\) | 29.0 \(\pm 0.3271\) |
10 Performance on Out-Of-Distribution(OOD) Unseen Harmful Prompts↩︎
In this section, we evaluate the effectiveness of our method on OOD prompts. The experimental results are shown in Table 10. The safe codebook is built using violence-related prompts from the ViSU dataset. To evaluate its effectiveness, we conduct experiments on prompt subcategories that are semantically close to violence. These subcategories include blood, weapons, brutality and cruelty. Compared with the ‘violence’ data, the data of these subcategories can be regarded as OOD data. The experimental results show that models using the safe codebook generalize well to OOD data.
| OOD Data | Violence | Overall \(\downarrow\) | ||||
|---|---|---|---|---|---|---|
| 2-6 | Violence | Blood | Weapons | Brutality | Cruelty | |
| Baseline | 0.47 | 0.61 | 0.45 | 0.48 | 0.38 | 0.48 |
| Safe-CB | 0.33 | 0.47 | 0.32 | 0.34 | 0.28 | 0.35 |
11 Hyper-Parameter Study↩︎
Effects of rank \(k\). As described in Section 3.2.1, when constructing the harmful subspace for a specific harmful concept, we form a difference vector matrix \(E\) using paired vectors of safe tokens and harmful tokens. The top-\(k\) singular vectors are then selected from this matrix. The choice of the parameter \(k\) has a significant impact on the effectiveness of the resulting harmful space.
Results on the impact of hyperparameter \(k\) are visualized in Figure 5. We conduct the experiment on prompts related to the sexual concept in the I2P dataset. The results show the scores of generated images under the Nudity detector and the \(FID\) values of generated images when \(k\) takes different values. The \(k\) value of \(0\) indicates the result of the original model without applying the safe codebook. \(FID\) is computed between images generated by the original model and those generated by models using the safe codebook at different values of \(k\). As the number of selected top-\(k\) singular vectors increases, the boundary of the constructed harmful space expands. The generated images are strictly stripped of information related to the harmful space, and the image safety is improved. However, when \(k\) is too large, too much information is included in the harmful space. Such information is also removed, so the quality of generated images will decrease. The experiment shows that when \(k\) is set to \(8\), a good balance is achieved between the safety and quality of generated images.
12 Direct supervised training of the model codebook↩︎
| I2P | Sexual | \(FID_g\) |
|---|---|---|
| Baseline | 0.1278 | 68.83 |
| Train | 0.1062 | 79.01 |
r0.5
In early exploratory experiments, we attempted a straightforward supervised fine‑tuning of the model’s codebook to improve safety. Specifically, we constructed training image–text pairs by pairing original harmful prompts with target images generated from their semantically similar but safe counterparts that created following the paired‑prompt methodology of standard harmful content datasets (e.g., via safe replacement of harmful terms). Using these pairs, we directly fine‑tuned the codebook, aiming to align the token semantics of harmful prompts with those of the safe target images. As a case study, we conducted experiments on the “sexual” concept from the I2P dataset, with results reported in Table 11.
The results indicate that this original codebook fine‑tuning approach is largely ineffective. Although it yields modest suppression for a single targeted harmful concept, the global modification of the codebook severely degrades overall image quality. Moreover, performance deteriorates further when handling multiple harmful concepts, likely due to semantic entanglement among concepts during optimization, which prevents effective joint suppression.
13 Visualization Examples↩︎
In this section, we add visualizations across seven harmful prompt categories. Figure 6 and Figure 7show: 1) harmful images from the original unified model, 2) outputs from the safe‑model baseline that modifies model weights directly, and 3) safer images from our Safe‑CB method. The original model generates clearly harmful content. The safe‑model baseline improves safety to some extent but severely degrades image quality. In contrast, Safe‑CB effectively suppresses harmful content while preserving high image fidelity.
References↩︎
https://github.com/notAI-tech/NudeNet↩︎