Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement. Traditional moderation systems often rely on fragmented black-box
classifiers that are difficult to maintain and lack transparency. In this paper, we present UNIVID, a UNIfied VIsion-language model for video moDeration. Unlike standard
classification models, UNIVID generates policy-aware captions that serve as an interpretable intermediate representation, enabling human-verifiable decisions and multi-task reusability. While existing open-source and commercial VLMs often
suffer from safety-guardrail refusals and lack fine-grained policy alignment, we develop a specialized training data recipe that combines expert human-refined labels with synthetic data to align the model with our safety guidelines. By integrating UNIVID
as the core captioner, we design a novel end-to-end video moderation system that reduces violation leakage by 42.7% and overkill rate by 37.0% relatively. Meanwhile, by replacing over 1,000 policy-specific models with a single UNIVID backbone,
we recycled extensive computation resources while reducing engineering maintenance overhead. To our knowledge, this is one of the first reports of a high-efficiency captioning VLM successfully supporting industrial-scale moderation and cross-functional
business.
The rapid growth of short-video platforms such as Reels and YouTube Shorts necessitates accurate and operationally efficient moderation systems [1], [2]. Prior systems rely on thousands of specialized, end-to-end classification models [3], each targeting a specific policy (e.g., violence, regulated activities). However, this approach faces three bottlenecks: (1) Lack of Interpretability, as black-box scores offer little to no
rationales for human auditors [4], [5]; (2) Tedious Maintenance, where policy
updates require retraining thousands of individual models [2]; and (3) Resource Inefficiency, as these models lack the semantic
flexibility to support cross-functional business such as advertisement and recommendation safety.
Vision-Language Models (VLMs) offer a promising alternative by transforming video into natural language descriptions, i.e., video captions. Why using a Captioning VLM for moderation? First, captions act as a unified, human-readable
bridge that provides explicit evidence for policy violations [6].
Second, a single VLM can replace thousands of specific classification models [7], simplifying our system infrastructure. Finally, these
policy-aware captions can be cached and reused by downstream tasks, creating a multi-purpose content understanding feature [8].
Despite their potential, existing open-source [9]–[11] and
commercial VLMs [12], [13] often fall short of industrial moderation requirements.
They frequently refuse to describe sensitive or violative content [5], [14]
due to internal safety triggers. Furthermore, since these models are not grounded using our internal platform policies, they often fail to capture the precise enforcement boundaries, leading to inaccurate descriptions. Finally, the scale of commercial VLMs
makes real-time inference for full-traffic moderation economically impractical.
To address these limitations, we develop UNIVID, which adopts the same architectural design as LLaVA-OV [11] while employing a
task-specific training recipe for moderation. We use a hybrid training strategy that combines expert annotations with high-quality synthetic data, aligning the model with our detailed moderation policies. On top of UNIVID, we construct a
multi-stage moderation system: a high-throughput Risk Filter leveraging UNIVID embeddings for early screening; a Moderation Actor deploying two finetuned variants of UNIVID to support moderation decisions; and a Trend Governance
module that reuses cached UNIVID captions to detect emerging risks.
Our UNIVID-centric moderation system has been fully integrated into our platform, yielding the following contributions:
Unified System: We propose the industry deployment of a unified video moderation infrastructure built on UNIVID, significantly reducing engineering overhead and simplifying end-to-end troubleshooting.
Data & Evaluation Pipeline: We design a human-in-the-loop training recipe to ensure factual and policy alignment. Furthermore, we introduce CapBench for evaluation, which decomposes captions into atomic events to
evaluate violation recall across key safety domains.
Platform Safety Governance: By integrating UNIVID as the core captioner, our new moderation system reduces violation leakage by 42.7% and overkill rate by 37.0% relatively. Beyond that, UNIVID also achieves 81% matching
accuracy on the beta simulation of Brand & Ads applications.
Figure 1: Model architecture of UNIVID following LLaVA-OneVision [11]. We construct in-house data
recipe focusing on safety violation content.
Table 1: Overview of UNIVID training stages.
Stage
Data Source
Type
# Samples (M)
PT
LLaVA pretrain, our internal data
Single sentence caption
1.6
FT
Synthetic and human-refined caption
Caption
3.2
Synthetic VQA
General VQA (e.g., summary, topic, keywords, etc.)
2.0
CFT
Hybrid caption
Caption
0.1
Our UNIVID model is built upon the LLaVA family of multimodal large language models [9]–[11], [15], with a particular focus on the LLaVA-OneVision [11] architecture, which has demonstrated strong scalability and flexibility for video content understanding. We choose Mistral-v0.3-7B [16] as the LLM \(f_{\phi}\), as its permissive license ensures production compliance and its parameter scale is optimized for full-traffic deployment. We follow LLaVA-OneVision [11] for design choices on components shown in Figure 1.
To be more specific, for a video input \(V\), and the associated caption \(C\), we first extract frames uniformly from the video, providing a series of video frames \(X_{1...N}\), where \(N\) is the maximum supported frames for our model. After appropriate pre-processing, the image is fed into vision encoder \(g\), token
compression module \(TC\), and projection \(W\). This process eventually convert visual information into tokens \(H_v\), which is then concatenated with
encoded text instructions \(H_t\). For a caption \(C\) with length \(L\) we aim to maximize the following:
Constructing a large-scale training set for video moderation poses several challenges. Firstly, collecting real-world violative video is difficult and legally sensitive. Therefore, no open-source dataset is available. Moreover, we cannot completely rely
on proprietary or open-source VLMs, which are safety-aligned and frequently refuse to generate sensitive labels, e.g., CSAM1.
To address these limitations, we develop an in-house dataset tailored to the specific content styles and moderation requirements of our platform (see Table 1). Following the standard LLaVA instruction-tuning recipe [10], we leverage GPT-4o [13] to generate one-paragraph captions and VQA pairs2. The VQA pairs encompass both
general visual understanding and moderation-specific queries. Examples of training data are shown in Appendix 6.1.
To optimize the trade-off between annotation costs and data quality, we internally train annotators to perform human-in-the-loop refinement of the raw GPT-4o outputs. The refinement includes two dimensions: (1) Factual Correction:
Rectifying hallucinations and adding missing details regarding subjects, objects, actions, backgrounds, and OCR. (2) Policy Grounding: Ensuring that violation rationales are grounded in our internal safety policy playbook. For violative
content, annotators are required to map the video to the corresponding pre-defined policy title.
Furthermore, to better capture edge cases in harmful trends, we use proprietary VLMs to normalize and enrich the violation rationale description. This step specifically targets multilingual OCR and adversarial visual hacking techniques (e.g.,
split-screen effect).
Table 2: Comparison of violative video captioning benchmarks.
Following the LLaVA setup, our model training paradigm includes three stages: (1) Pretraining for modality alignment, (2) Instruction-tuning on caption and VQA data, (3) Continue finetuning on high-quality caption data to further enhance the model’s
generation ability (see Table 1).
In the pretraining stage, we freeze the whole model except the MLP projector to align the vision and text modality. In the next two finetuning stages, we train both the projector and LLM decoder. The model training procedure takes 120 hours on 32 H100
GPUs. We also develop UNIVID-1B as a lightweight variant by replacing the decoder with our proprietary LLM.
UNIVID functions as the core of our moderation system across following three stages.
Our risk filter module performs early-stage screening over full video traffic to identify potential policy violations while maintaining low latency and high throughput. We first generate video captions using UNIVID, augmented by auxiliary
signals including OCR, titles, and risk embeddings predicted by lightweight policy-specific modules. Our fusion network integrates these multimodal inputs into a shared embedding, which is connected to multiple MLP policy heads. Each policy head applies a
calibrated decision threshold to identify high-risk videos and route them to downstream stages.
Our earlier moderation system suffered from precision dilution and high maintenance overhead. This is due to the reliance on multiple independent components (e.g., neural scoring models, vector-based retrievers, and heuristic rules) operating in
parallel. Moreover, such a fragmented system complicates the threshold calibration and makes end-to-end troubleshooting tedious. To address these limitations, we transition to a recall-and-rank architecture composed of two models: UNIVID-Lite for primary
enforcement and UNIVID-RAG for leakage mitigation.
2.4.0.0.1 UNIVID-Lite
UNIVID-Lite serves as a unified actor model, providing a single decision layer for the moderation pipeline. It is finetuned on 1 million in-house videos based on the UNIVID-1B backbone. The training data consists of human-annotated moderation examples
balanced at a positive-to-negative ratio of 1:5 to reflect the natural distribution of production traffic. Each sample includes video frames together with auxiliary textual signals which are concatenated directly into the instruction prompt (see Appendix
D.2). UNIVID-Lite is trained with an autoregressive generation objective: rather than using a separate classification head, the model is prompted to produce a structured natural language output indicating the moderation decision (Approve or Violation)
along with the corresponding violated policy. This formulation allows the model to inherit the general reasoning capabilities of the pretrained UNIVID backbone while specializing for binary enforcement.
2.4.0.0.2 UNIVID-RAG
To mitigate leakage — violations missed by the primary actor — we introduce UNIVID-RAG, which augments the moderation decision with retrieval from a Violation Knowledge Base (VKB). The VKB contains approximately 100,000 structured violative events
derived from past cases labeled by human moderators. When new violations are identified, Gemini-2.5-Pro converts the raw annotations into structured violative events capturing the policy title and violation rationale, which are then added to the VKB. The
knowledge base is updated continuously to reflect the latest labeling.
At inference time, UNIVID-RAG retrieves the top-3 most semantically similar violative events from the VKB using cosine similarity over UNIVID caption embeddings. The retrieved events are inserted directly into the prompt as in-context examples,
providing the model with concrete prior cases to reason against (see Appendix D.3). This retrieval-augmented approach is specifically designed to improve coverage of hard or low-frequency violations that the primary model may miss, at the cost of a
moderately higher violation rate due to increased sensitivity.
Short-form video platforms are increasingly targeted by emerging trends, such as dangerous “hot water pouring challenges”. To identify these real-time issues, we implement a lightweight adaptation strategy. By reusing cached fusion embeddings from the
backbone, we train an MLP trend head to recognize these shared semantic videos via few-shot samples (< 50). This approach allows our moderation system to remain agile, capturing short-lived or rapidly evolving threats.
Existing video safety benchmarks are often limited to surveillance contexts [19], [20] or synthetic video sources [21], failing to capture the broad spectrum of user-generated content.
While the recent KuaiMod [18] benchmark addresses some of these gaps, it is localized to Chinese content and employs compliance-driven masking of key visual
components, such as human faces. Such anonymization brings a distribution shift that deviates from the high-fidelity and unmasked real-world video posts.
To evaluate multimodal understanding across both general and harmful contexts, we design our in-house benchmark CapBench, which assesses VLMs on two dimensions: descriptive accuracy and violation recall. Inspired by the prior
Dream-1k [17] benchmark, we employ a fine-grained metric that decomposes video captions into atomic “evidence.” CapBench includes two subsets:
violative and healthy sets. The violative subset is further divided into five domains: Violence, Sexual Abuse, Mental Health, Regulated Activity, and Integrity. Details are shown in Appendix 6.3.
For each video, we establish a human-refined ground-truth caption, which is then segmented by Gemini-2.5-Pro into verifiable events. We then prompt Gemini as judge to compute two symmetric metrics based on these segments: (1) Recall: Measures the
coverage of ground-truth events within the predicted caption. (2) Precision: Measures the rate of hallucination or the proportion of predicted events supported by the ground truth. For violative samples, events serving as violation rationales are annotated
with violation labels (e.g., “The three men ride together on a single red motorcycle.”), enabling separate evaluation of violative and non-violative recall. Compared to previous work in Table 2, our benchmark offers clearer insights into caption credibility and provides a unified scoring framework applicable to both violative and healthy video captioning.
We validate the captioning and violation recognition ability of UNIVID on our internal CapBench. We evaluate the leading commercial VLMs, specifically GPT-4o [13] and Gemini-2.5-Pro [12] to explore the upper bound performace. Additionally, we benchmark
LLaVA-OneVision-1.5 8B [22], which shares a similar architecture with UNIVID but has not been finetuned on our in-house
moderation dataset. The model generation outputs are constrained to 150 words to align with our online deployment, as shown in Appendix 9.
As demonstrated in Table 3, UNIVID-7B substantially outperforms LLaVA-OV 8B, despite their similar architectures. Notably, UNIVID-7B
also achieves higher violation recall across all domains compared to GPT-4.1 [13]. This gap is partially due to the safety guardrails of proprietary models,
which refuse generation in certain cases (12.9%), yielding empty captions.
We further conduct ablation studies upon training data recipe. We find that removing hybrid training data leads to consistent violation recall degradation across all domains (-16.8% average). We observe a more pronounced decline when training with
human-annotated data only, where violation recall further decreases to 26.1%, indicating limited generalization under narrowly distributed supervision using GPT-4o. Note that other ablation results on model architecture are presented in Appendix 7.1.
Table 4: Comparison of VLM deployment cost (per 1M videos). Note that we are unable to deploy LLaVA-OV due to compliance constraints.
UNIVID has delivered substantial improvements in moderation accuracy on production-scale traffic. Based on our simulated evaluation results, our new system achieved a 42.7% relative reduction in view-weighted violation leakage, falling from
0.255% to 0.146%. Simultaneously, the overkill rate decreased from 35.4% to 22.3%. Meanwhile, UNIVID replaces over 1000 policy-specific models with a single multimodal backbone, with 1900 A30 GPUs recycled. Consequently, our system reduces
engineering overhead and simplifies long-term maintenance.
To support full video traffic, we deploy UNIVID model with FP8 quantization on H100 GPUs, reaching 5.7 QPS per device. As shown in Table 4,
our production inference cost is approximately $180 per 1M videos, achieving a 15\(\times\) reduction compared to commercial VLMs (> $3,000).
This section presents our simulated evaluation results for downstream systems that integrate UNIVID, including our video moderation main system and cross-functional applications.
As shown in Table 5, integrating UNIVID embeddings consistently improves violation recall (especially +23.9% on high-view leakage), highlighting
enhanced sensitivity to rare but high-risk content.
After launching the Risk Filter with UNIVID embedding, our main moderation system observes an 11.5% increase in violation hit volume. Meanwhile, the moderation precision improves substantially, with violation precision from 35.9% to 75.6%,
demonstrating better coverage and decision quality in live traffic.
Our simulated moderation ablation in Table 6 demonstrates the effectiveness of UNIVID-Lite as a unified action layer in live moderation
traffic. Compared to the recall-only pipeline, UNIVID-Lite reduces the violation rate (1.38% → 1.34%) while improving precision (+9.4%) and leakage recall (+11.8%), indicating more accurate enforcement with fewer unnecessary actions.
Introducing retrieval augmentation further increases leakage recall to 53.6%, improving coverage of hard or low-frequency violations. This gain comes at the cost of slightly reduced precision and a slightly higher violation rate, reflecting the
trade-off between maximizing recall and enforcing conservatively. We further present our case study in Appendix 7.1. Overall, UNIVID-Lite offers strong precision-oriented actions, with RAG serving
as a complement when higher recall is prioritized.
Figure 2: Visualization of trend detector embeddings.
We report the simulated moderation results of few-shot learning in Table 5. On the school dangerous behavior trend, incorporating UNIVID
embeddings substantially improves moderation performance, boosting violation recall from 56.7% to 86.7%. The embedding visualization in Figure 2 further shows that UNIVID embeddings produce clearer separation
between trend and non-violative cases. This suggests that UNIVID provides semantically rich representations that generalize in low-data regimes, enabling rapid adaptation to emerging trends without degrading precision.
Beyond moderation, UNIVID also generates fine-grained video keywords that are consumed by the advertising ranking system as semantic targeting signals, enabling brand-safe matching and custom-lineup construction. This integration achieved
81% accuracy in the Brand & Ads applications in our simulation tests.
Table 5: Simulated online ablation of UNIVID embedding on the risk filter and few-shot trend detector. Scores are reported at 65% precision threshold.
Video moderation is crucial for protecting users from detrimental content and maintaining a healthy platform, especially for minors [23]–[26]. Plenty of efforts have been made to construct a video moderation ecosystem. Early work uses traditional ML methods on
handcrafted features for hate speech, toxicity, or fake news detection [24], [27], typically in binary or multi-class setups. Similar patterns appear in visual moderation, where porn or anomaly detection in images/videos uses engineered visual features and classical classifiers [28], [29].
Table 6: Simulated online ablation results of UNIVID-Lite and UNIVID-RAG on the moderation actor module.
Multimodal language models (MLMs) have demonstrated impressive capabilities in understanding images [30]–[32] and short videos [33]–[36]. These models generally comprise three modules: a vision encoder, a large language model (LLM) decoder, and an adapter that aligns image-text
modalities [10], [36]–[39]. Prior work explores MLM-based moderation models [18] and data generation frameworks validated
on open-source VLMs [40]. These approaches focus on offline evaluation, without considering real-world deployment at production
scale, while UNIVID is trained under our compliance regulations and operates in full-traffic production.
In this paper, we present UNIVID, a unified vision-language model that reframes video moderation from fragmented black-box classifiers to an interpretable, caption-driven framework. Built upon UNIVID, we construct a unified
moderation pipeline comprising a high-throughput Risk Filter, a Moderation Actor, and a Trend Governance module. This UNIVID-based system has been deployed on our global-scale short-video platform, operating over full video traffic under
strict compliance constraints. In our simulated production experiments, it achieves a 42.7% relative reduction in violation leakage and a 37.0% reduction in overkill rate, while attributing 81% accuracy on the Advertisement downstream business.
The deployment of UNIVID for industrial-scale video moderation involves handling highly sensitive content, necessitating rigorous ethical and operational safeguards. First, we prioritize annotator welfare by enforcing strict daily exposure
limits to violative content. Our Standard Operating Procedure (SOP) includes regular rotation shifts to prevent prolonged psychological strain and ensure mental well-being. For particularly sensitive categories like Child Sexual Abuse Material (CSAM), we
implement dedicated protocols where access is restricted to a minimal, specially trained group of annotators operating under enhanced security and psychological support frameworks.
Regarding data privacy and retention, our system adheres to strict internal compliance standards. To minimize data footprint, deleted and private videos are permanently purged from our systems within a strictly defined retention window, while
non-violative video data is retained only for a limited period. All data used for training and evaluation is stored in siloed environments with internal access controls. Any identified illegal content is handled through our internal reporting channels
without permanent retention in any environment.
To prevent dual-use and adversarial risks, we have established a strict non-disclosure policy for our technical assets. We will not release the source code, model weights, or the specific training datasets used in this paper. Instead, we provide an
overview of the system architecture and training objectives without offering a direct template for adversarial attacks. We are committed to ensuring that the specific enforcement logic remains protected against exploitation by bad actors.
Finally, we address bias and algorithmic fairness through our data recipe and model design. During the annotation process, our guidelines explicitly instruct annotators to avoid using or emphasizing terms related to specific ethnic groups to prevent the
encoding of societal prejudices. Furthermore, we are committed to the continuous improvement of the model’s multilingual capabilities, ensuring that safety enforcement is equitable and consistent across different linguistic and cultural contexts.
Currently, UNIVID does not incorporate reinforcement learning methods such as Group Relative Policy Optimization (GRPO). The policy-aware captions can serve as a form of reasoning trace indicating how a video may violate specific policies;
however, platform policy guidelines are not directly encoded as reward signals and therefore are not explicitly bound to this reasoning process.
Our system processes videos via frame sampling rather than full temporal modeling. As a result, videos that embed policy-violating content in only a single frame may evade detection if those frames are not selected. Therefore, the effectiveness of our
system depends on keyframe selection quality, which remains a practical limitation under latency and computational constraints.
Table 7: Distribution of safety domains and policies in CapBench.
Domain
Policy Category
# Samples
Violence
Violent Behaviors; Shocking & Graphic Content
2,700
Sexual Abuse
Exploitation & Abuse; Nudity & Sexual Activity
4,339
Mental Health
Mental Health; Harassment & Hateful Behavior
1,248
Regulated Activity
High-Risk & Regulated Activities
4,656
Integrity
Harmful Misinformation; Deceptive Behaviors
624
We show the detailed statistics of CapBench in Table 7. Note that since a video can violate multiple policies simultaneously, the sum of policy-specific
counts exceeds the total number of unique samples.
The evaluation prompts for both stages are presented below, including event extraction and entailment judgment.
You are given a description of a video clip: [caption]
Your tasks:
1. Extract at most 10 key events from the description.
2. From these events, identify those that may violate the video platform content policy labels: [policy_list]. Only include events that can serve as sufficient evidence of the listed policy violations. You will be given a document of policy instructions to
help you better understand the policy definition (if any).
Requirements for Events:
- An event must include an action or motion.
- Merge semantically similar and disjunctive actions into a single event. DO NOT repeat same events.
- Every event is represented by a brief sentence within 20 words, with a subject with key attributes, a predicate and optionally an object. Include the action or attributes (such as minor’s age, revealing outfit) that may violate the content policy!
- Every event must be atomic, meaning that it cannot be further split into multiple events.
- Scene cuts and camera movements are NOT events! - Substitute pronouns by the nouns they refer to.
Output Format:
Return a Python dictionary string with the following keys:
- “events”: List[str]
the list of all extracted events.
- “violative_events”: Dict[str, List[str]]
mapping from each policy name in policy_list to the list of events that violate it. If no events violate a policy, use an empty list.
Materials:
Policy Instructions: [policy_doc]
You are given a video description and a list of events.
Your task is to classify the relationship between the video description and each event into one of three classes: - entailment: The video description entails the event.
- contradiction: Some detail in the video description contradicts the event.
- neutral: The relationship is neither entailment nor contradiction.
Video Description: [caption]
Events:[events]
Output Format:
Return a JSON list in the following format:
{"event": "copy an event here",
"relationship": "put class name here"}
Requirements: - Classify every event independently.
- Use only one of the three predefined class names.
- Do NOT provide any additional text, explanation, or formatting.
- Output only the JSON.
As shown in Table 9, we also examine the impact of visual and resolution components. Intuitively, replacing the vision encoder with less performant
EvaCLIP [41] results in a decrease in violation recall (-9.5%). For resolution setting,
given the on-par performance but substantially higher token cost, we choose not to use AnyRes in our model design.
As shown in Appendix 9.2, we fuse video frames with auxiliary textual features as model input, including titles, user profiles, OCR, and ASR, to provide enriched semantic context.
We find that UNIVID-Lite exhibits a robust capacity for contextual inference, enabling the detection of latent risk signals that extend beyond explicit keywords. In High-Risk Driving scenarios, the model identifies danger by synthesizing
environmental cues (such as dashboard speedometers) even in the absence of overt reckless actions. Similarly, in Severe Bullying cases, UNIVID-Lite effectively perceives hostile intent conveyed through suggestive or indirect language rather
than explicit insults. Furthermore, in the domain of Serious Harm, the model recognizes subtle precursors to danger, successfully identifying implicit threats that often precede manifest harmful behavior.
Table 8: Examples of violative events from the knowledge base for UNIVID-RAG.
Policy
Violative Events
Nudity & Sexual Activity
The video shows two youths performing a dance that includes sexually suggestive movements, such as hip thrusting, squatting, and sticking their tongues out.
High-Risk & Regulated Activities
The video shows multiple people riding on the exterior of moving vehicles, including on the sides and tops of trucks within a convoy.
Mental Health
The video contains a real-life clip of a person standing on a train platform as a train passes, followed by animated scenes depicting bloodstains on the platform.
Harassment & Hateful Behavior
The video displays a person’s full email address (xxx@gmail.com) and a verification code on a smartphone screen.
Table 9: Ablation results of UNIVID-7B on CapBench.
Given image frames uniformly sampled from a video clip, describe the video (not the individual images) in detail, focusing on the main subjects, their actions, and the background scene. You should also pay attention to details that may violate the
trust-and-safety platform content policy. Don’t describe feelings or atmosphere.
Your output should be a single coherent paragraph. Maximum length: 150 words.
The online inference prompt of UNIVID-Lite is as follows:
Region: [region]
Title: [title]
User nickname: [nickname]
ASR Text: [asr]
OCR Text: [ocr]
Bio Text: [profie]
Based on the video frames and related content above, please indicate the severity of any inappropriate, disruptive, or harmful content.
The online inference prompt of UNIVID-RAG is as follows:
Region: [region]
Title: [title]
User nickname: [nickname]
ASR Text: [asr]
OCR Text: [ocr]
Bio Text: [profie]
Violative Events for Reference:
1. [policy title], violation reason: [reason]
2. [policy title], violation reason: [reason]
3. [policy title], violation reason: [reason]
Based on the video frames, the above content, and the judgment logic of the similar cases, please indicate the severity of any inappropriate, disruptive, or harmful content.
Z. Wang et al., “Filter-and-refine: A MLLM based cascade system for industrial-scale video content moderation,” in Proceedings of the 63rd annual meeting of the
association for computational linguistics (volume 6: Industry track), 2025, pp. 873–880.
[2]
H. Liang et al., “Embedding-based retrieval in multi-modal content moderation,” in Proceedings of the 48th international ACM SIGIR conference on research and
development in information retrieval, 2025, pp. 4264–4268.
[3]
J. Shi et al., “CPFD: Confidence-aware privileged feature distillation for short video classification,” in Proceedings of the 33rd ACM international conference
on information and knowledge management, 2024, pp. 4866–4873.
[4]
A. Levi, O. Levi, S. Mishra, and J. Morra, “AI vs. Human moderators: A comparative evaluation of multimodal LLMs in content moderation for brand safety,”2025 IEEE/CVF
International Conference on Computer Vision Workshops (ICCVW), pp. 6024–6032, 2025, [Online]. Available: https://api.semanticscholar.org/CorpusID:280545981.
[5]
H. Bao et al., “VModA: An effective framework for adaptive NSFW image moderation,”arXiv preprint arXiv:2505.23386, 2025.
[6]
G. Huang, Q. Peng, G. Xu, Y. Lu, and Y. Shen, “LLaVAShield: Safeguarding multimodal multi-turn dialogues in vision-language models.” 2025, [Online]. Available: https://arxiv.org/abs/2509.25896.
[7]
M. Wang et al., “From native memes to global moderation: Cros-cultural evaluation of vision-language models for hateful meme detection,”arXiv preprint
arXiv:2602.07497, 2026, [Online]. Available: https://api.semanticscholar.org/CorpusID:285451876.
[8]
H. Wang et al., “VIPS: Learning-view-invariant feature for person search,”Sensors, vol. 25, no. 17, p. 5362, 2025.
[9]
H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,”Advances in neural information processing systems, vol. 36, pp. 34892–34916, 2023.
[10]
H. Liu et al., “Llava-next: Improved reasoning, ocr, and world knowledge.” 2024.
[11]
B. Li et al., “LLaVA-OneVision: Easy visual task transfer,”arXiv preprint arXiv:2408.03326, 2024.
[12]
G. Comanici et al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,”arXiv
preprint arXiv:2507.06261, 2025.
D. Lee, J. Jang, J. Jeong, and H. Yu, “Are vision-language models safe in the wild? A meme-based benchmark study,”arXiv preprint arXiv:2505.15389, 2025.
[15]
H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition, 2024, pp. 26296–26306.
J. Wang, L. Yuan, Y. Zhang, and H. Sun, “Tarsier: Recipes for training and evaluating large video description models,”arXiv preprint arXiv:2407.00634, 2024.
[18]
X. Lu et al., “Vlm as policy: Common-law content moderation framework for short video platform,” in Proceedings of the 31st ACM SIGKDD conference on knowledge
discovery and data mining v. 2, 2025, pp. 4682–4693.
[19]
W. Sultani, C. Chen, and M. Shah, “Real-world anomaly detection in surveillance videos,” in Proceedings of the IEEE conference on computer vision and pattern
recognition, 2018, pp. 6479–6488.
[20]
T. Hassner, Y. Itcher, and O. Kliper-Gross, “Violent flows: Real-time detection of violent crowd behavior,” in 2012 IEEE computer society conference on computer vision
and pattern recognition workshops, 2012, pp. 1–6.
[21]
X. Liu et al., “Video-safetybench: A benchmark for safety evaluation of video lvlms,”arXiv preprint arXiv:2505.11842, 2025.
[22]
X. An et al., “LLaVA-OneVision-1.5: Fully open framework for democratized multimodal training,” in arXiv, 2025.
[23]
R. Gorwa, R. Binns, and C. Katzenbach, “Algorithmic content moderation: Technical and political challenges in the automation of platform governance,”Big Data &
Society, vol. 7, 2020, doi: 10.1177/2053951719897945.
[24]
V. U. Gongane, M. Munot, and A. Anuse, “Detection and moderation of detrimental content on social media platforms: Current status and future directions,”Social Network
Analysis and Mining, vol. 12, 2022, doi: 10.1007/s13278-022-00951-3.
[25]
S. Udupa, A. Maronikolakis, and A. Wisiorek, “Ethical scaling for content moderation: Extreme speech and the (in)significance of artificial intelligence,”Big Data &
Society, vol. 10, 2023, doi: 10.1177/20539517231172424.
[26]
V. Lai et al., “Human-AI collaboration via conditional delegation: A case study of content moderation,”Proceedings of the 2022 CHI Conference on Human Factors
in Computing Systems, 2022, doi: 10.1145/3491102.3501999.
[27]
A. Naseeb et al., “Machine learning- and deep learning-based multi-model system for hate speech detection on facebook,”Algorithms, 2025, doi: 10.3390/a18060331.
[28]
W. Wang et al., “Validating multimedia content moderation software via semantic fusion,”Proceedings of the 32nd ACM SIGSOFT International Symposium on Software
Testing and Analysis, 2023, doi: 10.1145/3597926.3598079.
[29]
K. Yousaf and T. Nawaz, “A deep learning-based approach for inappropriate content detection and classification of YouTube videos,”IEEE Access, vol. 10, pp.
16283–16298, 2022, doi: 10.1109/access.2022.3147519.
[30]
D. A. Hudson and C. D. Manning, “Gqa: A new dataset for real-world visual reasoning and compositional question answering,” in Proceedings of the IEEE/CVF conference on
computer vision and pattern recognition, 2019, pp. 6700–6709.
[31]
Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh, “Making the v in vqa matter: Elevating the role of image understanding in visual question answering,” in
Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6904–6913.
[32]
K. Marino, M. Rastegari, A. Farhadi, and R. Mottaghi, “Ok-vqa: A visual question answering benchmark requiring external knowledge,” in Proceedings of the IEEE/cvf
conference on computer vision and pattern recognition, 2019, pp. 3195–3204.
[33]
F. Caba Heilbron, V. Escorcia, B. Ghanem, and J. Carlos Niebles, “Activitynet: A large-scale video benchmark for human activity understanding,” in Proceedings of the
ieee conference on computer vision and pattern recognition, 2015, pp. 961–970.
[34]
V. Patraucean et al., “Perception test: A diagnostic benchmark for multimodal video models,”Advances in Neural Information Processing Systems, vol. 36,
pp. 42748–42761, 2023.
[35]
K. Li et al., “Mvbench: A comprehensive multi-modal video understanding benchmark,” in Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition, 2024, pp. 22195–22206.
[36]
M. Maaz, H. Rasheed, S. Khan, and F. S. Khan, “Video-ChatGPT: Towards detailed video understanding via large vision and language models,” in Proceedings of the 62nd
annual meeting of the association for computational linguistics (ACL 2024), 2024.
[37]
J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” in International
conference on machine learning, 2023, pp. 19730–19742.
[38]
D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “MiniGPT-4: Enhancing vision-language understanding with advanced large language models,”arXiv preprint
arXiv:2304.10592, 2023.
[39]
B. Lin, B. Zhu, Y. Ye, M. Ning, P. Jin, and L. Yuan, “Video-llava: Learning united visual representation by alignment before projection,”arXiv preprint
arXiv:2311.10122, 2023.
[40]
Z. Wang et al., “Reasoning-enhanced domain-adaptive pretraining of multimodal large language models for short video content governance,” in Proceedings of the
2025 conference on empirical methods in natural language processing: Industry track, 2025, pp. 1104–1112.
[41]
Q. Sun, Y. Fang, L. Wu, X. Wang, and Y. Cao, “EVA-CLIP: Improved training techniques for CLIP at scale.” 2023, [Online]. Available: https://arxiv.org/abs/2303.15389.
![Figure 1: Model architecture of UNIVID following LLaVA-OneVision [11]. We construct in-house data recipe focusing on safety violation content.](/article/2606.05748/figures/assets/masp_model_plot.png "fig:")
