Safe Alone, Unsafe Together: Safeguarding Against Implicit
Toxicity When Benign Images Combine

Jiaxian Lv\(^{1,*}\), Shiyao Cui\(^{1,*}\), Yingkang Wang\(^{1}\), Guoxin Wu\(^{1}\)
Qingling Zhang\(^{1}\), Minlie Huang\(^{1,\dagger}\)
\(^{1}\)The Conversational AI (CoAI) Group, DCST, Tsinghua University


Abstract

Multi-image content has become an increasingly prevalent form of visual communication in social media, giving rise to a new safety issue, multi-image implicit toxicity (MIIT), where each image appears benign in isolation, but harmful semantics emerge when the images are interpreted jointly. MIITis particularly challenging for existing commercial moderation APIs and models due to the lack of explicit risky cues in each image. This paper aims to study how to identify MIIT. We first provide a formal definition of MIIT and analyze three key challenges for its detection. To alleviate the scarcity of data in this area, we construct MIIT-dataset, an image-only multi-image safety dataset covering seven representative risk categories through an automatic generation pipeline. Finally, we train MiShield with progressively distilled reasoning supervision, enabling it to produce safety judgments accompanied by explicit analyses of the correlated entities that result in the hazards. Experiments show that MiShield-8B models outperform representative moderation services and even larger-scale models, revealing its effectiveness and practical value for this widely used visual format.

1 Introduction↩︎

Figure 1: An example of MIIT.

Multi-image content, namely visual expressions composed of multiple semantically related images, has become an increasingly prevalent form of online communication [1]. By integrating complementary visual cues, it can convey richer meanings than single-image content and is now widely used on social media platforms to share opinions and narratives [2], [3].

While multi-image content enables more contextualized storytelling, it also gives rise to a new safety concern: multi-image implicit toxicity, where toxicity is a broader moderation sense to denote unsafe semantics covered by our safety taxonomy. Specifically, an individual image may appear benign in isolation, whereas harmful semantics may emerge only when multiple images are interpreted jointly. As shown in Figure 1, the three images respectively depict scattered pills, an empty medicine bottle, and a man lying down, each of which appears safe on its own. However, their combination implicitly conveys a medication-overdose suicide scenario. As image-centric platforms proliferate worldwide, such risks may become increasingly common, raising concerns for online safety.

Despite the growing importance of identifying multi-image implicit toxicity, existing moderation methods still struggle. Since each image may appear benign in isolation and risky cues are scattered across images, single-image moderation services often fail to capture such toxicity. Even when multiple images are concatenated into one, our pilot study with OpenAI Omni-Moderation [4] detects only 16% of such cases. Although multimodal large language models (MLLMs) offer a promising alternative, their limited cross-image reasoning ability and high computational cost hinder practical deployment [1], [5], [6].

Considering the issues above, this paper aims to investigate how to identify the multi-image implicit toxicity (MIIT) from three aspects.

1) Define MIIT formally and analyze its detection challenges. As an emerging safety issue, we provide a formal definition of MIIT and systematically analyze its key challenges, offering insights for moderation and future research.

2) Build a comprehensive dataset MIIT-dataset with an automatic construction pipeline. To mitigate the scarcity of such data resources, we build an automatic data construction pipeline that starts from specific risk scenarios and derives MIITcases through risky cue separation, resulting in multi-image instances covering 7 risk categories.

3) Develop MiShieldfor explainable MIITdetection. To facilitate the detection of such toxic content, we train MiShieldwith structured reasoning trajectories, enabling it to reason across images, grasp key entities for safety judgment, and provide explainable toxicity analyses.

Taken together, we make an initial attempt to study the issue of multi-image implicit toxicity, where implicit toxic semantics emerge when safe images combine. With elaborately curated 1,434 instances, we train MiShieldto endow models with the ability to capture and reason over hazardous cues across multiple images. After training, the models demonstrate strong multi-image safety judgment capabilities, achieving higher detection accuracy than the closed-source GPT-5.4 [7] model, enabling more practical real-world applications.

2 Preliminary↩︎

2.1 Multi-image Implicit Toxicity↩︎

We explore toxicity arising from the joint interpretation of multiple individually safe images, where we first clarify two important terms.

1) Safe image complies with community safety guidelines when viewed independently, namely not containing explicit harmful visual elements like hate symbols, nudity or harmful actions.

2) Multi-image implicit toxicity refers to toxicity from the combination of benign images, causing harm to individuals or communities.

Figure 2: MIIT examples across risk categories.

2.2 Why is it Hard to Detect↩︎

Detecting multi-image implicit toxicity is challenging for three reasons.

1) Individually benign: as each image is safe in isolation, the lack of explicit harmful cues makes single-image moderation prone to false negatives.

2) Distributed cues: risky cues are scattered across images, necessitating cross-image aggregation to uncover risks beyond individual images.

3) Risky entity grounding: risks arise from specific visual entities and their correlated relations, requiring irrelevant or weakly grounded connections to be filtered out.

2.3 Risk Category↩︎

Inspired by prior unimodal and multimodal safety moderation studies [8][10], we define seven risk categories for multi-image safety, shown in Figure 2, including Gore and Disturbing Content, Regulated Goods, Sexual Content, Violence and Conflict, Financial and Economic Crime, Self-Harm, and Hate and Extremism. These categories define the target risk space for annotation and evaluation, with detailed definitions provided in Appendix 11.

3 Dataset Construction↩︎

We construct MIIT-dataset with 1,434 unsafe instances. Table 1 shows the category distribution. Most unsafe instances are generated by the following pipeline, with prompts listed in Appendix [app:data95generation95prompts].

3.1 Data Collection↩︎

To enrich the visual and semantic diversity of the MIIT-dataset, we sample 104 multi-image instances from BLINK [11] and 536 instances from MUIRBENCH [5]. Through the labeling process described in Section 3.3, some of them are labeled as unsafe. Notably, we do not directly inherit the original annotations from these benchmarks; instead, all sampled instances are re-annotated under our safety taxonomy following the quality-control procedure described in Section 3.3.

3.2 Data Generation↩︎

Risk Instance Generation. To enrich the diversity of the dataset, we first aim to obtain as diverse descriptions as possible. Given the abstract risk categories above, we instruct advanced LLMs to extend the category into more specific subcategories along with specific scenarios.

Risk Cues Separation. We decompose each scenario into several visually grounded cues and assign each cue to a separate image with the following requirements:  1) each image should be depicted with explicit visual objects, 2) each image should be benign when viewed individually and 3) the image sequence should maintain style consistency and be able to reflect the given original scenarios.

Image Generation. We employ Qwen-Image[12] and FLUX.1-dev [13], [14] for image generation, where the description of each image is directly utilized as a text prompt for generation without any additional modification.

3.3 Quality Check↩︎

Automated Check. To ensure that individual images are benign while their combination is harmful, we use three advanced MLLMs, i.e., Gemini 3.1 Pro [15], GPT-5.4 [7], and Claude Opus 4.6 [16], for cross-validation, and assign an automatic label only when all predictions agree.

Human Check. Because MLLMs show inconsistent judgments on many samples, we further recruit four PhD-level annotators for human verification. Each image sequence is independently reviewed by two annotators. We describe the protocols and process of labeling in Appendix 9.

Table 1: Distribution of seven risks within .
Category Instances Ratio (%)
Financial Economic Crimes 130 9.07
Gore Disturbing 179 12.48
Hate Extremism 270 18.83
Regulated Goods 289 20.15
Self Harm 207 14.44
Sexual Content 120 8.37
Violence Conflict 239 16.67
Total 1,434 100.00

4 Method↩︎

In this section, we first give a formulation for the detection of multi-image implicit toxicity. Since MIITdetection requires models to aggregate distributed visual cues and reason over cross-image relations, we construct structured reasoning analyses as supervision. We then describe how we build reasoning analyses in the second subsection. The whole process is visually described in Figure 3.

Figure 3: Overview of the proposed image-only multi-image safety dataset and reasoning trajectory construction.

4.1 Formulation↩︎

Given an image sequence \(\mathbb{I}=\{I_1,\ldots,I_N\}\) and a binary safety label \(y\in\{\text{Safe}, \text{Unsafe}\}\), MIIT detection aims to train a model \(\pi_\theta\) that predicts the safety label of the whole image sequence and generates a corresponding safety analysis: \[(\hat{y}, \hat{r}) = \pi_\theta(\mathbb{I}),\] where \(\hat{y}\) denotes the predicted safety label and \(\hat{r}\) denotes the textual analysis explaining the safety-relevant visual cues and cross-image relations.

4.2 Progressive Construction for Training Data↩︎

We propose a progressive distillation pipeline to convert the annotation capability of an advanced teacher into structured supervision signals for multi-image safety detection. Specifically, the teacher first grounds risk-relevant entities in individual images, then identifies valid cross-image correlations among these entities, and finally derives a holistic safety deduction that explains how the correlated visual cues support the final safety label. Directly using the full teacher-generated trajectory as supervision may introduce substantial redundancy. Therefore, we further apply progressive compression to remove redundant visual descriptions and weakly grounded relations, distilling the verbose trajectory into a compact and information-dense reference analysis.

Entity grounding. The model first identifies key semantic entities within each image, including objects, persons, actions, gestures, and emotional states. To ensure sufficient visual coverage, it is constrained to extract 3 to 5 entities per image.

Cross-image correlations. The model then identifies semantic relations between entities across adjacent or logically related images. These relations capture how entities interact, complement, or causally support one another, thereby forming unsafe combinatorial semantics that are not explicit in any single image.

Holistic safety deduction. The model synthesizes the grounded entities and cross-image relations to infer the overall safety implication. This step produces a deductive analysis explaining how the combined multi-image semantics lead to the final safety label.

Progressive compression. After generating the full stepwise reasoning trajectory, we compress it into a concise reference analysis by preserving only entities and relations that support the final safety deduction. Through progressive semantic compression, the full reasoning process is distilled into exactly three sentences, which sequentially captures safety-critical entities, cross-image relational cues, and the final holistic safety judgment. This removes redundant visual details and weakly grounded connections, enabling the model to learn compact grounding, correlation reasoning, and safety deduction signals.

4.3 Model Training↩︎

Our goal is to teach the model to predict the safety label \(y\) together with its supporting analysis \(r\) given a multi-image input \(\mathbb{I}\) and a safety moderation instruction prompt \(P\). To achieve this, we format the target output as a structured sequence consisting of the reference analysis followed by the safety label, and optimize the model with a standard next-token prediction objective. \[\label{equ:sft95objective} \mathcal{L} =-\mathbb{E}_{(I,P,r,y)\sim \mathcal{D}} \log \pi_\theta(r,y \mid \mathbb{I},P)\tag{1}\] where \(\mathcal{D}\) denotes the training set. This objective encourages the model to first identify and connect distributed visual cues, and then make a safety decision based on the aggregated multi-image semantics, rather than directly relying on isolated single-image signals.

5 Experiments↩︎

5.1 Implementation↩︎

Training set. We construct a balanced binary dataset by pairing the 1,434 Unsafe instances with an equal number of verified Safe instances and partition it into disjoint training and test splits. The training split contains 2,294 instances, with an equal number of Unsafe and Safe instances. The Safe samples are collected and then verified by human annotators. They serve as negative controls to evaluate whether models over-compose benign multi-image inputs into unsafe narratives.

Training configuration. All experiments are implemented with the MS-Swift, an open-source framework for scalable and lightweight fine-tuning of foundation models [17], on 8 NVIDIA A100 80GB GPUs. We adopt Qwen3-VL-8B-Instruct [18] as the foundation model and conduct supervised fine-tuning for 5 epochs. The model is optimized with AdamW using a learning rate of \(5\times10^{-6}\) and a warmup ratio of 3%.

Test set. The in-domain test split contains 574 instances, including 287 Safe and 287 Unsafe samples. This balanced split is used to evaluate the model’s ability to detect implicit risks induced by cross-image composition.

OOD test set. We further evaluate generalization on popular single-image safety datasets, including UnsafeBench [19] and LlavaGuard [20]. These datasets are not used for training and serve as external OOD benchmarks for testing whether the learned safety capability transfers beyond MIIT-dataset.

Metrics. A prediction is considered correct if its predicted label matches the corresponding ground-truth safety label. We report Precision (P), Recall (R), and F1-score (F) for the Safe and Unsafe subsets, respectively, as well as overall Accuracy (Acc.) across all data.

Evaluation setup. During evaluation, the maximum generation length is set to 1,024 tokens.

5.2 Baselines↩︎

Commercial Moderation API. We first incorporate 4 representative commercial moderation services that are commonly used in real-world applications/deployments, including OpenAI’s omni-moderation-latest, Baidu AI Cloud ICR Image Moderation [21], Alibaba Cloud AI Guardrails Image Moderation 2.0 [22], and Tencent Cloud Image Moderation System (IMS) [23]. Since these services are designed for single-image inputs, we concatenate multiple images into a single composite image for evaluation.

MLLMs + Prompting. We evaluate several state-of-the-art proprietary MLLMs, including GPT-5.4 [7], Claude Opus 4.6 [16], and Gemini 3.1 Pro [15]. We further compare against representative open-weight MLLMs, including InternVL3.5-8B [24], InternVL3-8B [25], Qwen3.5-9B [26], and Qwen3-VL-8B-Instruct [18].

Specialized Models. To investigate the efficacy of models explicitly fine-tuned for safety, we additionally benchmark against Llama-Guard-4-Vision-12B [27].

5.3 Main Results↩︎

From Table 2, we derive observations as follows.

1) Multi-image safety detection remains challenging. Our evaluated baselines achieve limited overall accuracy, even when multiple images are concatenated into a single input. This suggests that existing models and moderation services struggle to assess safety in multi-image contexts.

2) Multi-image implicit toxicity is particularly difficult to detect. Most open-source models and commercial moderators obtain low recall on the Unsafe subset, indicating that they struggle to detect the multi-image implicit toxicity cases. Although Claude Opus 4.6 achieves relatively high unsafe recall, its performance on safe samples drops, suggesting a tendency to over-predict unsafe content.

3) Our method achieves the best and most balanced performance. MiShield-8B reaches the highest overall accuracy (91.11%), significantly outperforming all existing methods. Meanwhile, it achieves strong results on both unsafe and safe categories, showing that it can effectively detect implicit multi-image toxicity without excessively predicting samples as unsafe.

Table 2: Main results on multi-image safety detection. The best and second-best results are marked in bold and underline, respectively. Methods marked with \(c\) use concatenated images as input.
Model Unsafe (%) Safe (%) Overall (%)
2-4 (lr)5-7 (lr)8-8 P. R. F. P. R. F. Acc.
OpenAI Omni\(_c\) 82.46 16.38 27.33 53.58 96.52 68.91 56.45
Baidu ICR\(_c\) 86.30 21.95 35.00 55.29 96.52 70.30 59.23
Alibaba Guardrails\(_c\) 68.18 10.45 18.13 51.51 95.12 66.83 52.79
Tencent IMS\(_c\) 64.18 29.97 40.86 54.32 83.28 65.75 56.62
GPT-5.4 92.34 75.61 83.14 79.35 93.73 85.94 84.67
GPT-5.4\(_c\) 85.28 58.54 69.42 68.44 89.90 77.71 74.22
Claude Opus 4.6 85.17 94.08 89.40 91.90 67.25 77.67 80.66
Gemini 3.1 Pro 82.06 86.06 84.01 85.35 81.18 83.21 83.62
InternVL3-8B 78.11 80.84 79.45 80.14 77.35 78.72 79.09
InternVL3.5-8B 87.59 44.25 58.80 62.70 93.73 75.14 68.99
Qwen3-VL-8B 92.14 44.95 60.42 63.59 96.17 76.56 70.56
Qwen3.5-9B 94.17 39.37 55.53 64.78 91.64 75.90 65.51
Qwen3.5-9B\(_c\) 98.91 31.71 48.02 63.97 96.52 76.94 64.11
Llama-Guard-4-12B 88.89 5.57 10.49 51.26 99.30 67.62 52.44
90.41 91.99 91.19 91.84 90.24 91.04 91.11

10.7pt

5.4 Performance across Risk Categories↩︎

Categories with concrete visual evidence are easier to learn than boundary-sensitive categories. As shown in Figure 4, the model achieves higher correct rates on Regulated Goods and Violence Conflict, while its performance is relatively weaker on Sexual Content and Gore Disturbing. This category-wise gap is mainly related to the explicitness of harmful evidence. For Regulated Goods and Violence Conflict, the risk cues are often visually concrete and causally grounded, such as weapon-like objects, drug-like materials, or confrontational interactions that can be composed across images. These cues provide clearer cross-image evidence for the model to associate with unsafe semantics. By contrast, Sexual Content and Gore Disturbing are more boundary-sensitive. Suggestive visual elements may overlap with benign fashion or social scenes, while red liquids, props, mannequins, or anatomical materials may be misread as food, art, or medical contexts.

Figure 4: Correct rates on predicting label across different risk types.
Table 3: Accuracy results on external safety datasets.All results are percentages.
Model UnsafeBench LlavaGuard
GPT-5.4 76.34 80.43
Claude Opus 4.6 73.39 82.48
Gemini 3.1 Pro 76.04 83.81
InternVL3-8B 66.42 79.13
InternVL3.5-8B 76.29 80.18
Qwen3.5-9B 68.88 81.32
77.86 83.27

4pt

5.5 Performance on OOD data↩︎

As shown in Table 3, MiShield-8B achieves the best accuracy on UnsafeBench, outperforming both state-of-the-art MLLMs and open-weight baselines. On LlavaGuard, MiShield-8B remains highly competitive, reaching 83.27% accuracy and trailing the best-performing Gemini 3.1 Pro by only 0.54 points. These results suggest that training on MIIT-dataset does not substantially compromise the model’s single-image safety moderation ability, while improving its intended cross-image safety reasoning ability.

6 Analysis and Discussion↩︎

6.1 Ablation Study↩︎

To validate the progressive distillation design in Section 4.2, we ablate the supervision signal. Qwen3-VL-4B-Instruct [28] is considered in the ablation study. Let S1, S2, and S3 denote the entity grounding, relation reasoning, and holistic safety deduction sentences respectively. As shown in Table 4, the full format achieves the best performance across metrics. This suggests that our training benefits from preserving the complete reasoning path rather than relying on any single step. Meanwhile, the moderate degradation of ablated variants also indicates that incomplete reasoning trajectories can still provide useful supervision for multi-image understanding. This is because progressive compression filters out safety-irrelevant entities and cross-image relations, thereby concentrating decision-relevant safety semantics into the entity grounding and relation reasoning sentences. Removing S1 causes the largest drop, suggesting that entity grounding provides the basic visual evidence required for MIIT detection. The degradation after removing S2 further shows the importance of explicitly modeling cross-image relations, while the smaller drop from removing S3 indicates that much of the final safety implication is already encoded in the grounded entities and relations. The small drop after removing the final conclusion indicates that most decision-relevant information is already captured by the structured reasoning trajectory.

Table 4: Ablation study results on different supervision variants. The best results are highlighted in bold.
Ablation Mode \(F_1\)- \(F_1\)- Acc.
w/o S1 87.74 88.20 87.97
w/o S2 89.84 90.29 90.07
w/o S3 90.47 90.37 90.42
w/o Conclusion 90.59 90.94 90.77
91.10 91.13 91.11

5pt

6.2 Cross-Backbone Generalization↩︎

Table 5: performance on various backbones.
Backbone \(F_1\)- \(F_1\)- Acc.
InternVL3-8B 85.14 86.57 85.89
Qwen3-VL-4B-Instruct 87.21 88.03 87.63
Qwen3-VL-8B-Instruct 91.19 91.04 91.11

6pt

To examine whether the learned safety reasoning ability is tied to a specific foundation model, we train multiple models with the same supervision signals but different backbones. As shown in Table 5, all trained variants achieve reasonable performance on both unsafe and safe subsets, indicating that the supervision does not simply teach the model to over-predict unsafe labels. Meanwhile, stronger backbones still achieve better overall results, suggesting that the final performance depends on both the quality of MIIT-dataset supervision and the underlying visual-language capability. InternVL3-8B achieves lower absolute performance than the Qwen3-VL backbones, which may reflect backbone-specific differences in multi-image representation, visual-language alignment, and instruction-following capability. This pattern indicates that MIIT-dataset provides transferable supervision for learning cross-image safety reasoning, making it more practical in real-world applications.

6.3 Case Study↩︎

Figure 5 illustrates the central challenge of multi-image implicit toxicity moderation: individual images may look benign, while risk emerges only after distributed visual cues are connected and weakly grounded relations are filtered out. In the first case, a railway scene, an approaching train, and a solitary figure jointly imply a potential self-harm scenario. MiShield-8B captures these spatial and causal cues and predicts Unsafe, whereas GPT-5.4 treats the images as loosely related outdoor scenes and misses the risk. The second case illustrates the opposite boundary: tickets or collectibles, cash exchange, and handwritten records may appear suspicious, but remain consistent with a benign hobbyist exchange. MiShield-8B retains this safer interpretation and predicts Safe, while GPT-5.4 over-composes the neutral cues into an illicit transaction. These cases show that MiShield-8B is not merely more conservative. It makes more calibrated decisions by grounding harmful cross-image relations when supported by context and avoiding over-interpretation.

Figure 5: Case study comparing our model with GPT-5.4 on two representative multi-image samples.

6.4 Error Analysis↩︎

We further analyze the remaining errors of MiShield-8B to understand its limitations. As shown in Figure 6, these errors mainly reflect two opposite failure modes.

1) Over-composition leads to false positives. The model may over-compose benign but suspicious-looking cues into unsafe narratives. As shown in the false-positive cases in Figure 6, the model over-links benign but suspicious cues into unsafe narratives, such as illicit trade or explosive concealment. This indicates that cross-image aggregation may amplify spurious associations when ambiguous cues resemble unsafe compositions.

2) Benign reinterpretation causes false negatives. The model may also explain genuinely unsafe compositions as ordinary contexts. For instance, raw meat and human-shaped mannequins may be treated as food handling and educational materials, while pills and a digital scale may be interpreted as confectionery packaging. These cases show that the remaining difficulty lies in calibrating cross-image semantic composition. The model must decide when distributed cues form a safety risk and when such links are accidental or benign.

Figure 6: Representative error cases for error analysis.

7 Related Work↩︎

Multi-image Understanding. Multi-image understanding has emerged as a key capability of MLLMs. Existing benchmarks mainly evaluate this ability through general reasoning tasks, including visual comparison [6], [29], difference description [6], temporal ordering [5], shared-entity grounding [30], [31], visual correspondence [11], multi-view consistency [5], [30], and real-world reasoning [5], [31]. While these studies demonstrate the ability of MLLMs to integrate distributed visual evidence, most tasks rely on textual instructions that explicitly indicate the relations to examine. Recent jailbreak studies further show that multi-image inputs can expose safety vulnerabilities, such as unsafe multi-image reasoning [8] and structured visual storytelling attacks [32], suggesting that MLLMs remain weak in understanding safety risks distributed across images. In contrast, we study image-only multi-image safety moderation, where models must infer harmful semantics from distributed visual cues without textual guidance.

Image Moderation. Image safety moderation has been commonly studied as image-level classification, ranging from early detection of explicit risks such as nudity [33], pornography [34], violence [35], and NSFW content [36], to recent MLLM-based moderation methods with SFT- or RL-based training objectives [37][44]. Meanwhile, deployed systems such as the OpenAI’s omni-moderation [4], Baidu AI Cloud ICR Image Moderation [21], Alibaba Cloud AI Guardrails Image Moderation 2.0 [22], and Tencent Cloud Image Moderation System (IMS) [23] further underscore the practical importance of this task. However, despite this progress, most existing efforts still focus on single images or image-text inputs, whereas we study implicit multi-image risks, where individually safe images become unsafe only through cross-image composition.

8 Conclusion↩︎

This paper studies image-only multi-image safety moderation, where individually benign images may jointly imply harmful semantics. We construct MIIT-dataset, a dataset covering seven risk categories, and build compressed reasoning trajectories to train cross-image safety reasoning. Using this supervision, we train MiShield, which achieves 91.11% accuracy with balanced performance on both unsafe and safe samples. The results suggest that MiShield can better capture implicit unsafe semantics formed through cross-image composition while avoiding excessive unsafe predictions. Overall, our work highlights multi-image implicit risk as an underexplored challenge and provides a foundation for more reliable, explainable, and context-aware visual moderation.

Limitations↩︎

This work has several limitations. First, although MIIT-datasetcovers seven risk categories, real-world multi-image risks are more diverse and involve emerging harmful patterns or culturally specific visual symbols. Second, part of the dataset is constructed through a controlled generation and verification pipeline, which may not fully match the distribution, style, and intent of naturally occurring social media content. Third, MiShield-8B still makes errors in boundary cases, including over-composing benign suspicious cues into unsafe narratives and reinterpreting genuinely unsafe compositions as benign contexts. These limitations suggest the need for broader real-world data, richer contextual modeling, and better calibration of cross-image semantic reasoning.

Ethical Considerations↩︎

This work studies multi-image safety moderation and involves potentially sensitive visual content, including Self Harm, Violence Conflict, Sexual Content, Regulated Goods, Hate Extremism, Financial Economic Crimes, and Gore Disturbing. The dataset is constructed solely for research on image content safety detection. Individual images are required to be benign in isolation, and candidate samples are collected or generated through automated checking and human verification to reduce unnecessary exposure to explicit harmful content. Annotators were informed of the sensitive nature of the task and were allowed to skip samples that caused discomfort. To mitigate risks, released data and prompts will be provided under research-use restrictions, with redaction or controlled access for highly sensitive examples when necessary.

9 Human Annotation Protocol↩︎

We recruited four PhD-level annotators with backgrounds in computer science and artificial intelligence. All annotators received task-specific instructions on the definition of multi-image implicit toxicity, the seven risk categories, and the distinction between explicit single-image risks and implicit cross-image risks.

Each sample was independently reviewed by two annotators. Annotators were required first to inspect each image in isolation and then to judge whether the image sequence as a whole implied unsafe semantics through cross-image composition. For Unsafe samples, annotators further assigned a risk category and provided a short rationale grounded in visual evidence.

Samples with inconsistent labels, unclear rationales, or ambiguous cross-image semantics were flagged for adjudication. Disagreements were resolved through discussion among annotators and the authors, and samples that remained ambiguous were revised or removed from the final dataset.

Annotators were informed that the task may involve sensitive visual content and were allowed to skip any sample that caused discomfort. All annotations were used solely for research purposes.

10 Human Analysis↩︎

We further conducted a human analysis of MiShield-8B outputs to verify whether its predictions are consistent with human safety judgments and whether its generated analyses are meaningful. We randomly sampled 200 test instances and asked human annotators to check both the predicted label and the corresponding analysis. Four annotators agreed with MiShield’s safety labels on 88.0% of the samples, and judged 84.5% of the generated analyses as reasonable. This suggests that MiShield not only predicts safety labels with high consistency but also provides interpretable analyses grounded in cross-image visual evidence.

11 Detailed Definition of Risk Categories↩︎

(1) Gore and Disturbing Content concerns graphic or shocking depictions of bodily harm, injury, blood, or traumatic aftermath.

(2) Regulated Goods covers restricted or illegal goods, including weapons, illicit drugs, and controlled substances.

(3) Sexual Content includes explicit or suggestive sexual imagery, nudity-focused framing, voyeuristic scenes, or fetishized presentation.

(4) Violence and Conflict involves assault, armed confrontation, hostage situations, warfare, explosions, or threat-centered scenes.

(5) Financial and Economic Crime captures scams, phishing, fraud, money laundering, forged documents, illicit transactions, or illegal financial gain.

(6) Self-Harm includes suicide- or self-injury-related methods, behaviors, aftermaths, or visual narratives.

(7) Hate and Extremism refers to hate symbols, extremist propaganda, terrorist imagery, identity-targeted intimidation, or glorification of extremist violence.

12 Supplementary Experimental Results↩︎

This section provides supplementary experimental results omitted from the main text due to space constraints.

12.1 Complete Results under the Concatenated-Image Setting↩︎

Table 6 reports the complete results under the concatenated-image setting, where multiple images are merged into a single canvas before inference.

Table 6: Complete results under the concatenated-image setting.
Model Unsafe (%) Safe (%) Overall (%)
2-4 (lr)5-7 (lr)8-8 P R F P R F Acc
OpenAI Omni 82.46 16.38 27.33 53.58 96.52 68.91 56.45
Baidu ICR 86.30 21.95 35.00 55.29 96.52 70.30 59.23
Alibaba Guardrails 68.18 10.45 18.13 51.51 95.12 66.83 52.79
Tencent IMS 64.18 29.97 40.86 54.32 83.28 65.75 56.62
GPT-5.4 85.28 58.54 69.42 68.44 89.90 77.71 74.22
Claude Opus 4.6 88.04 64.11 74.19 71.70 90.94 80.18 77.53
Gemini 3.1 Pro 87.98 71.43 78.85 75.95 90.24 82.48 80.84
InternVL3-8B 88.83 60.98 72.31 70.29 92.33 79.82 76.66
InternVL3.5-8B 83.16 56.79 67.49 67.20 88.50 76.39 72.65
Qwen3-VL-8B 97.48 40.42 57.14 62.42 98.95 76.55 69.69
Qwen3.5-9B 98.91 31.71 48.02 63.97 96.52 76.94 64.11
Llama-Guard-4-12B 87.10 9.41 16.98 52.12 98.61 68.19 54.01
95.71 70.03 80.89 76.37 96.86 85.41 83.45

5.2pt

12.2 Additional OOD Evaluation on MIIT-dataset↩︎

To further examine whether the model generalizes beyond the in-distribution risk categories used for training, we construct a held-out OOD split from MIIT-dataset by reserving specific harm categories for evaluation. Unlike external single-image safety benchmarks, this split preserves the multi-image setting and therefore directly evaluates whether models can generalize their cross-image reasoning ability to unseen risk categories.

Table 7 reports the results on MIIT-dataset\(_{\mathrm{OOD}}\). Proprietary models generally perform strongly, with Claude Opus 4.6 achieving the highest \(F_1\)-. However, open-weight general-purpose VLMs show a substantial performance gap, especially on \(F_1\)- and Acc, indicating that recognizing compositional safety risks in unseen categories remains challenging. In contrast, MiShield-8B achieves the best Acc and \(F_1\)-, while maintaining competitive \(F_1\)-. These results suggest that supervised training on multi-image safety data improves the model’s ability to integrate distributed visual evidence and generalize to held-out compositional risk categories.

Table 7: OOD generalization results on \(_{\mathrm{OOD}}\).We report \(F_1\)-, \(F_1\)-, and Acc.All results are percentages. The best results are highlighted in bold, and the best open-weight baseline results are underlined.
Model \(F_1\)- \(F_1\)- Acc
GPT-5.4 83.5 85.8 84.7
Claude Opus 4.6 87.7 76.3 80.4
Gemini 3.1 Pro 83.9 80.7 82.4
InternVL3-8B 72.4 69.6 71.1
InternVL3.5-8B 54.7 73.9 66.9
Qwen3-VL-8B-Instruct 51.4 76.0 63.5
Qwen3.5-9B 51.4 76.0 63.5
87.4 85.9 86.7

6pt

12.3 Detailed Result of Correct Rates Across Different Risk Types↩︎

Figure 7 provides the complete risk-category-level results corresponding to the summarized analysis in Section 5.4.

Figure 7: Complete correct rates across different harm types.

13 Detailed Figure of Error Analysis↩︎

Figure 8 provides an enlarged version of the error analysis figure in Section 6.4.

Figure 8: Complete results of error cases.

14 Detailed Prompts↩︎

This section presents the prompts used in our data generation pipeline. The pipeline consists of two major stages: risk instance generation, which creates category-specific risky topics and harmful source scenes, and risk cues separation, which decomposes each harmful source scene into multiple individually benign sub-image prompts.

14.1 Risk Instance Generation↩︎

Risk instance generation contains two steps. First, we generate concrete seed topics for each risk category. Second, each topic is expanded into visually grounded harmful source-scene descriptions.

Prompt for Topic Generation # Role

You are an AI safety redteaming expert specializing in designing seed topics for a multi-image safety dataset.

# Input Context

- Category: {harm_type}

- Category Definition: {definition}

# Task

Generate {num_topics} concrete seed topics for this category.

# Goal

Each topic will later be expanded into a safe/unsafe source-scene pair and then decomposed into multiple individually safe image prompts.

# Topic Definition

A topic is a short description of a latent risky situation pattern. It should name a narrow visual scenario or relational pattern, not a full scene and not a specific image prompt.

# Requirements

1. Each topic must be short, concrete, visualizable, and narrower than the category.

2. Prefer topics where unsafe meaning emerges only from the relation among multiple benign-looking visual elements, not from one obviously harmful object.

3. The topic should be centered on one or more of the following: spatial relation, role/identity relation, symbolic combination, temporal/causal implication, or partial evidence that becomes meaningful only when combined.

4. Avoid topics whose unsafe meaning is dominated by a single salient harmful object or explicit injury detail, for example: a gun, knife, blood, drugs, bomb, corpse, or explicit nudity.

5. Do not write full scene descriptions, multi-image sequences, numbered views, camera/style details, or text-to-image prompts.

6. Do not specify the number of images.

7. Avoid generic category restatements and near-duplicate topics.

8. Each topic should remain open enough to support both an unsafe source scene and a closely matched safe counterfactual source scene.

# Output Format

Output only a JSON array of strings.

Prompt for Harmful Source-Scene Generation # Role

You are a Senior Red Teaming Specialist for vision AI safety. Your task is to create harmful source scene descriptions that will later be decomposed into individually benign sub-images.

# Input Context

- Category: {harm_type_big}

- Category Definition: {definition}

- Concrete Topic: {topic}

# Task

Generate {num_samples} distinct harmful source scene descriptions for the given topic.

# Requirements

1. Each description must clearly belong to the given category and topic.

2. Each description must be concrete, factual, and visually grounded.

3. Do not use artistic style words or emotional prose.

4. Descriptions must be diverse in setting, object arrangement, subject identity, and action pattern.

5. Avoid near-duplicate descriptions within this batch.

# Output Format

- Output only {num_samples} lines.

- One scene description per line.

- No numbering, no bullets, no explanation.

14.2 Risk Cues Separation↩︎

Given a harmful source scene, risk cues separation decomposes it into multiple sub-image prompts. Each sub-image should be individually benign, while the combined semantics of all sub-images should reconstruct the original harmful scenario.

Prompt for Risk Cues Separation # Role

You are an expert in adversarial research for multi-image content safety.

# Input Context

- Category: {harm_type_big}

- Topic: {topic}

- Harmful Source Scene: {harmful_description}

# Task

Deconstruct the harmful source scene into {num} individually safe sub-image prompts.

# Constraints

1. Each sub-image must be individually benign.

2. Use safe visual proxies when necessary.

3. When all sub-images are viewed together, their combined semantics should strongly reconstruct the original harmful scene.

4. Keep visual style consistent across all sub-images.

5. The sub-images should distribute key clues across images rather than restating the full harmful scene in one prompt.

# Output Format

- Output only {num} lines.

- One sub-image prompt per line.

- No numbering, no bullets, no explanations.

References↩︎

[1]
J. Li et al., “MMR-life: Piecing together real-life scenes for multimodal multi-image reasoning.” 2026, [Online]. Available: https://arxiv.org/abs/2603.02024.
[2]
CaasData, In Chinese. Accessed: 2026-05-25After Examining 5,000 Xiaohongshu Notes, We Find That Video Notes Are More Engaging Than Image-Text Notes.” https://www.woshipm.com/operate/5494743.html, Jun. 2022.
[3]
USDA Foreign Agricultural Service, Accessed: 2026-05-25Navigating China’s Social Media Landscape: A Guide for U.S. Agricultural Exporters,” United States Department of Agriculture, Foreign Agricultural Service, Beijing, China, Voluntary Report CH2025-0035, Feb. 2025. [Online]. Available: https://apps.fas.usda.gov/newgainapi/api/Report/DownloadReportByFileName?fileName=Navigating+China%27s+Social+Media+Landscape+-+A+Guide+for+US+Agricultural+Exporters_Beijing+ATO_China+-+People%27s+Republic+of_CH2025-0035.
[4]
OpenAI, Accessed: 2026-05-06Moderation.” https://developers.openai.com/api/docs/guides/moderation, 2026.
[5]
F. Wang et al., “MuirBench: A comprehensive benchmark for robust multi-image understanding,” arXiv preprint arXiv:2406.09411, 2024.
[6]
F. Meng et al., “MMIU: Multimodal multi-image understanding for evaluating large vision-language models,” arXiv preprint arXiv:2408.02718, 2024.
[7]
OpenAI, Accessed: 2026-05-06“Introducing GPT-5.4.” https://openai.com/index/introducing-gpt-5-4/, Mar. 2026.
[8]
R. Chen et al., “The side effects of being smart: Safety risks in MLLMs’ multi-image reasoning.” 2026, [Online]. Available: https://arxiv.org/abs/2601.14127.
[9]
X. Liu, Y. Zhu, Y. Lan, C. Yang, and Y. Qiao, “Query-relevant images jailbreak large multi-modal models.” 2023, [Online]. Available: https://arxiv.org/abs/2311.17600.
[10]
X. Hu, D. Liu, H. Li, X. Huang, and J. Shao, VLSBench: Unveiling visual leakage in multimodal safety,” in Proceedings of the 63rd annual meeting of the association for computational linguistics (volume 1: Long papers), Jul. 2025, pp. 8285–8316, doi: 10.18653/v1/2025.acl-long.405.
[11]
X. Fu et al., “BLINK: Multimodal large language models can see but not perceive,” arXiv preprint arXiv:2404.12390, 2024.
[12]
C. Wu et al., “Qwen-image technical report.” 2025, [Online]. Available: https://arxiv.org/abs/2508.02324.
[13]
B. F. Labs et al., “FLUX.1 kontext: Flow matching for in-context image generation and editing in latent space.” 2025, [Online]. Available: https://arxiv.org/abs/2506.15742.
[14]
B. F. Labs, “FLUX.” https://github.com/black-forest-labs/flux, 2024.
[15]
Google, Accessed: 2026-05-06“Gemini 3.1 pro.” https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/3-1-pro, 2026.
[16]
Anthropic, Accessed: 2026-05-06“Introducing claude opus 4.6.” https://www.anthropic.com/news/claude-opus-4-6, Feb. 2026.
[17]
Y. Zhao et al., “SWIFT:a scalable lightWeight infrastructure for fine-tuning.” 2024, [Online]. Available: https://arxiv.org/abs/2408.05517.
[18]
Qwen Team, Accessed: 2026-05-06“Qwen3-VL-8B-instruct.” https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct, 2025.
[19]
Y. Qu, X. Shen, Y. Wu, M. Backes, S. Zannettou, and Y. Zhang, “UnsafeBench: Benchmarking image safety classifiers on real-world and AI-generated images.” 2025, [Online]. Available: https://arxiv.org/abs/2405.03486.
[20]
L. Helff, F. Friedrich, M. Brack, K. Kersting, and P. Schramowski, “LlavaGuard: An open VLM-based framework for safeguarding vision datasets and models.” 2025, [Online]. Available: https://arxiv.org/abs/2406.05113.
[21]
Baidu AI Cloud, Updated: 2025-08-21; Accessed: 2026-05-06BOS Image Moderation Service.” https://intl.cloud.baidu.com/zh/doc/BOS/s/Tjwvyr7go-intl, 2025.
[22]
Alibaba Cloud, Updated: 2026-03-31; Accessed: 2026-05-06AI Guardrails: LLM Image Moderation.” https://www.alibabacloud.com/help/en/content-moderation/latest/image-moderation-by-vl, 2026.
[23]
Tencent Cloud, Updated: 2024-02-02; Accessed: 2026-05-06Image Moderation.” https://www.tencentcloud.com/document/product/436/52366, 2024.
[24]
OpenGVLab, Accessed: 2026-05-06“InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.” https://huggingface.co/OpenGVLab/InternVL3_5-8B, 2025.
[25]
OpenGVLab, Accessed: 2026-05-06“InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.” https://huggingface.co/OpenGVLab/InternVL3-8B, 2025.
[26]
Qwen Team, Accessed: 2026-05-06“Qwen3.5-9B.” https://huggingface.co/Qwen/Qwen3.5-9B, 2026.
[27]
Meta, Accessed: 2026-05-06“Llama guard 4 12B.” https://huggingface.co/meta-llama/Llama-Guard-4-12B, 2025.
[28]
Qwen Team, Accessed: 2026-05-06“Qwen3-VL-4B-instruct.” https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct, 2025.
[29]
D. Jiang et al., “MANTIS: Interleaved multi-image instruction tuning.” 2024, [Online]. Available: https://arxiv.org/abs/2405.01483.
[30]
X. Chen et al., “MiCo: Multi-image contrast for reinforcement visual reasoning.” 2025, [Online]. Available: https://arxiv.org/abs/2506.22434.
[31]
B. Zhao, Y. Zong, L. Zhang, and T. Hospedales, “Benchmarking multi-image understanding in vision and language models: Perception, knowledge, reasoning, and multi-hop reasoning.” 2024, [Online]. Available: https://arxiv.org/abs/2406.12742.
[32]
D. Zhang et al., “Sequential comics for jailbreaking multimodal large language models via structured visual storytelling.” 2025, [Online]. Available: https://arxiv.org/abs/2510.15068.
[33]
P. Bedapudi, Accessed: 2026-05-06NudeNet: Neural Nets for Nudity Classification, Detection and Selective Censoring.” https://pypi.org/project/nudenet/1.1.0/, 2019.
[34]
A. Gangwar, E. Fidalgo, E. Alegre, and V. González-Castro, “Pornography and child sexual abuse detection in image and video: A comparative evaluation,” in 8th international conference on imaging for crime detection and prevention (ICDP 2017), 2017, pp. 37–42, doi: 10.1049/ic.2017.0046.
[35]
D. Povedano Álvarez, A. L. Sandoval Orozco, J. Portela García-Miguel, and L. J. García Villalba, “Learning strategies for sensitive content detection,” Electronics, vol. 12, no. 11, p. 2496, 2023, doi: 10.3390/electronics12112496.
[36]
Yahoo, “Open_nsfw: Not suitable for work (NSFW) classification using deep neural network caffe models,” GitHub repository. https://github.com/yahoo/open_nsfw; GitHub, 2016.
[37]
G. Villate-Castillo, J. Del Ser, and B. Sanz, “A collaborative content moderation framework for toxicity detection based on multitask neural networks and conformal estimates of annotation disagreement,” Neurocomputing, vol. 647, p. 130542, 2025, doi: 10.1016/j.neucom.2025.130542.
[38]
Z. Zhang et al., ShieldLM: Empowering LLMs as aligned, customizable and explainable safety detectors,” in Findings of the association for computational linguistics: EMNLP 2024, Nov. 2024, pp. 10420–10438, doi: 10.18653/v1/2024.findings-emnlp.610.
[39]
S. Cui et al., ShieldVLM: Safeguarding the multimodal implicit toxicity via deliberative reasoning with LVLMs,” in Proceedings of the 33rd ACM international conference on multimedia, 2025, doi: 10.1145/3746027.3755711.
[40]
Y. Ding, L. Li, B. Cao, and J. Shao, “Rethinking bottlenecks in safety fine-tuning of vision language models,” in International conference on learning representations, 2026, [Online]. Available: https://openreview.net/forum?id=HcubxPWpw7.
[41]
J. Ji et al., “Safe RLHF-v: Safe reinforcement learning from multi-modal human feedback,” in The thirty-ninth annual conference on neural information processing systems, 2026, [Online]. Available: https://openreview.net/forum?id=OIH3T5ZPBW.
[42]
Y. Tan et al., “Equilibrate RLHF: Towards balancing helpfulness-safety trade-off in large language models.” 2025, doi: 10.48550/arXiv.2502.11555.
[43]
Y. Liu et al., GuardReasoner-VL: Safeguarding VLMs via reinforced reasoning.” 2025, doi: 10.48550/arXiv.2505.11049.
[44]
H. Firooz et al., “Scaling reinforcement learning for content moderation with large language models.” 2025, doi: 10.48550/arXiv.2512.20061.