1 Introduction↩︎

Recent vision–language models (VLMs) can generate detailed textual descriptions from images [1]–[5] and synthesize images from textual prompts [6], [7]. Despite their impressive generative capabilities, these models frequently produce hallucinated content: captions may mention objects or attributes absent from the image, while generated images may fail to faithfully reflect the input text. Such inconsistencies raise concerns about reliability, interpretability, and downstream usability in real-world applications.

Understanding inconsistencies between images and text is a fundamental problem underlying many vision–language tasks. Image classification, object detection, segmentation, and image–text retrieval can all be viewed as measuring consistency between visual evidence and textual descriptions [8]–[10]. Similarly, in human–robot interaction and instruction following, systems must identify whether textual instructions correctly correspond to observed visual environments [11], [12]. While prior studies have explored hallucination detection [13]–[15] or factuality evaluation, existing benchmarks primarily focus on coarse correctness judgments or text-only correction, providing limited insight into why a mismatch occurs and where it is grounded in the image.

To address this limitation, we introduce GAVEL (Grounded cAption Error Verification and Localization), a new benchmark for grounded vision–language inconsistency understanding. Given an inconsistent image–text pair, the task requires models to: (1) explain the inconsistency in natural language, and (2) localize the visual regions related to the discrepancy. By jointly evaluating explanation and localization, GAVEL enables a more fine-grained and interpretable analysis of vision–language alignment than prior benchmarks. The proposed benchmark offers several advantages. First, it evaluates whether a model can identify the underlying cause of a mismatch rather than merely predicting that an inconsistency exists. Second, grounding explanations in visual regions provides interpretable evidence that facilitates human understanding and diagnosis. Third, the task naturally supports practical applications such as automatic caption correction, hallucination analysis, and iterative refinement in generative systems, where identifying specific erroneous components is more useful than coarse binary feedback. To support this benchmark, we curate a dataset with diverse image–text inconsistencies collected from both image-to-text and text-to-image generation pipelines. The dataset contains grounded annotations describing the inconsistency together with corresponding visual evidence, enabling systematic evaluation of grounded vision–language understanding.

In addition, we establish GAVEL, including benchmark protocols and evaluation metrics tailored to grounded inconsistency explanation and localization. The benchmark jointly measures explanation quality and localization performance, enabling detailed analysis of model capabilities in identifying and grounding vision–language discrepancies. We believe GAVEL will serve as a valuable testbed for developing more reliable, interpretable, and grounded vision–language systems. Our contributions are summarized as follows:

• We introduce GAVEL, a new benchmark for grounded vision–language inconsistency understanding that jointly requires discrepancy explanation and visual localization, providing a foundation for future research on reliable and interpretable vision–language alignment.

• We construct a dataset containing diverse image–text inconsistencies from both image-to-text and text-to-image generation pipelines, together with grounded annotations and explanatory descriptions.

2 GAVEL Construction↩︎

Our goal is to construct a dataset in which each instance is represented as \((I, D, E, G)\), where \(I\) denotes an image, \(D\) denotes a single-sentence textual description, \(E\) denotes a textual explanation describing any image-text misalignment, and \(G\) denotes the visual grounding information, i.e. , bounding-box coordinates. In this section, we describe the dataset construction process and present its statistics.

2.1 Data Preparation↩︎

The challenge in constructing the dataset is how to collect hallucinated image-caption pairs. For the test split, we employ the negative pairs from Haldec [13], which provide sentence-level hallucination existence annotation, wherein we employ eight models, including Qwen2 [16], Llama-4 [3], LLaVA-Next [17], ShareGPT [2], CogVLM [4], GPT-4o [18], Stable-Diffusion [6], and an image generation model used in GPT.

To construct the training set, we design a semi-automatic pipeline that leverages VLMs to generate and select likely negative image–caption pairs, which are then sent to human annotators for further annotation. An overview of the data construction process is provided in the appendix. We first generate captions using a vision–language model (VLM) or a text-to-image model, and then send the resulting image–caption pairs to GPT-5 mini [18] to evaluate their alignment. The intuition is that pairs with lower alignment scores are more likely to be negative examples. We subsequently forward these low-scoring pairs to human annotators to obtain complete annotations.

2.2 Annotation↩︎

Given image–sentence pairs, we ask annotators to (i) explain any textual misalignment when the pair is incorrect, and (ii) provide a bounding box corresponding to the image region that grounds the error. To ensure annotation quality, we employ a three-stage verification process. First, each sentence is assigned to both an annotator and a reviewer, with any errors corrected during review. Subsequently, additional reviewers randomly inspect samples to further ensure the overall quality of the annotations.

2.3 Stats of the dataset↩︎

Overall Stats. Our human-annotated training dataset consists of 30,309 images with 35,249 sentences with detailed annotations in total, while the testing dataset consists of 2,569 images with 5,606 sentences. Thus, the training is conducted on 35,000 pairs, and testing is conducted on 5,000 pairs. Also, we confirm that our way of the negative pairs for the training data is working well, considering that the ratio of "No hallucination" pairs is only 7.3%.

Sentence Stats. Each sentence consists of 24.82 words on average, and the vocabulary covers 27,173 unique words.

Bounding box stats. On average, 1.32 bounding boxes are assigned per image. This suggests that visual grounding is often concentrated on a single region, whereas annotators frequently provide annotations for multiple regions—for example, in cases of hallucinations during object counting. We further visualize the distribution of bounding box sizes in the appendix. The results confirm that the annotated boxes span both small and large regions, indicating that our task effectively evaluates a model’s localization ability across a wide range of spatial scales.

Hallucination Categories. Figure 2 illustrates the hallucination type distributions in the train and test sets. To obtain this categorization, we use GPT-5 to classify each hallucination instance into predefined hallucination types [13] based on the annotated caption error and its corresponding explanation. The resulting distribution shows that our test set covers diverse fine-grained hallucination types, making it suitable for robustly evaluating models’ abilities in caption error verification, explanation, and localization.

Examples. In Fig.  1, we visualize examples of images, hallucinated captions, misalignment description, and visual grounding. More examples are provided in the appendix.

3 Experiment↩︎

3.1 Experimental Settings↩︎

We evaluate our method against a diverse set of strong baselines, including both proprietary and open-source vision-language models across different model scales.

Closed-source Models. We consider recent state-of-the-art proprietary models, including GPT-5 and GPT-5-mini  [19].

Open-source Models. We include a diverse set of open-source vision-language models across different parameter scales. For comparable-scale evaluation, we consider two representative 8B models: InternVL3.5-8B-Instruct [20] and Qwen3-VL-8B-Instruct [5]. To further examine scaling effects, we include larger models such as InternVL3.5-30B-A3B-Instruct and Qwen3-VL-30B-A3B-Instruct.

Evaluation Metrics. We evaluate bounding box predictions using IoU-based accuracy under thresholds of 0.3, 0.5, and 0.7. For textual explanations, we use GPT-5 as an automatic evaluator to assign scores from 0 to 5, which are multiplied by 20 and reported on a 0–100 scale for both average and category-wise performance.

5pt

3.2 Results↩︎

Hallucination Grounding Results. To evaluate current multimodal models on the proposed Hallucination Grounding benchmark, we compare representative closed- and open-source models in Table 1. The benchmark measures visual grounding with A@0.3, A@0.5, and A@0.7, and textual explanation quality with LLM scores across hallucination categories. Results show that Hallucination Grounding remains challenging. GPT-5 achieves the best A@0.3, indicating stronger coarse localization, while Qwen3-VL-30B performs best at stricter IoU thresholds, with the highest A@0.5 and A@0.7. For textual explanations, GPT-5 obtains the highest LLM score, followed by Qwen3-VL-30B. Category-level results reveal notable variation across hallucination types. Overall, our benchmark offers a diagnostic setting for analyzing hallucination localization and explanation quality.

Analysis of Training Set Learnability. We further examine whether the training split can support model learning. Using the VisionLLM framework [21] as the base model, we fine-tune it on our training set and evaluate the trained model on the test set. As shown in Table 2, we report the epoch-7 checkpoint. Compared with the base model without training, our trained model achieves consistent improvements across grounding metrics. This shows that the dataset contains learnable and generalizable grounding signals, suggesting that our benchmark provides both reliable evaluation data and effective training supervision for future model development.

Failure Analysis of Visual Grounding. We further analyze whether visual grounding errors are related to bounding-box size bias. For each valid prediction, we compute the predicted-to-ground-truth area ratio, \(\mathrm{Area}(B_{pred})/\mathrm{Area}(B_{gt})\), for GPT-5, GPT-5 mini, Qwen3-VL-30B, and Qwen3-VL-8B. Figure 3 shows the log-scale distribution of this ratio, where 0 indicates equal box size. Most models have median ratios close to 1.00, suggesting no systematic over- or under-sizing. However, GPT-5 mini tends to produce larger boxes, while Qwen3-VL-8B is more concentrated around the ground-truth size. Together with the grounding accuracy results, this suggests that grounding failures stem not only from box-size bias, but also from inaccurate localization.

Qualitative comparison. Figure 4 presents qualitative comparisons between InternVL and GPT-5 on hallucination detection and grounding. For each example, we show the input caption, ground-truth explanation and bounding box, and predictions from both models. Red boxes indicate localized visual evidence, while highlighted text marks the key claim identified by each model. These examples show that models may fail in different ways: identifying the hallucinated phrase but localizing an imprecise region, or giving a partially incorrect explanation despite a plausible bounding box.

4 Conclusion↩︎

GAVEL shows the need to move beyond image- or response-level hallucination assessment by jointly evaluating detection, explanation, and localization. Beyond benchmarking, it provides training data for improving hallucination understanding and grounding across diverse hallucination categories. This direction is crucial for building more reliable, interpretable, and practical vision-language systems.

5 Limitations↩︎

This work primarily focuses on GAVEL as a fine-grained benchmark for evaluating hallucination detection, explanation, and localization in vision-language models. Although we provide a training split and conduct preliminary experiments showing that the dataset contains learnable grounding signals, we do not fully explore its potential for model training. More systematic studies on training strategies, data scaling, data mixture design, and model architectures are left for future work.

Acknowledgements↩︎

This work was supported by JST PRESTO, Japan, Grant Number JPMJPR2523. This work was partly achieved through the use of SQUID at D3 Center, The University of Osaka.

6 Appendix↩︎

6.1 Additional Dataset Information↩︎

Payment to Annotators. The annotations were conducted by a professional annotation company. We compensated the annotators at a rate of approximately $0.3 per annotated pair, based on the exchange rate at the time of the study.

Dataset Construct Pipeline. Due to space limitations, the main paper describes the dataset construction process only in text. Here, we provide an additional illustration of the construction pipeline in Figure 5. The pipeline consists of two main stages: negative image–caption pair selection and human annotation. In the first stage, we generate candidate image–caption pairs from two directions. For text-to-image generation, a caption is used to synthesize an image, and the resulting pair is evaluated for semantic alignment. For image-to-text generation, a VLM is prompted to generate a caption for a given image, and the generated pair is similarly assessed. We then use GPT-5 mini to filter these candidates according to their alignment scores. Pairs judged to be well aligned are discarded, while low-alignment pairs are regarded as likely hallucinated examples and forwarded to human annotators. In the second stage, annotators verify whether each candidate pair is truly hallucinated. For incorrect pairs, they provide a textual explanation of the misalignment and localize the corresponding image region with a bounding box. This procedure produces the final annotations used in GAVEL.

Analysis of bounding boxes size. As shown in Fig. 6, bounding-box sizes are grouped into logarithmic intervals. Most instances are concentrated in the \(10^4\)–\(10^5\) pixel range, accounting for 47% of the training set and 54% of the test set. In addition, bounding boxes in the \(10^5\)–\(10^6\) pixel range account for 30% and 20% of the training and test sets, respectively, while extremely small bounding boxes below \(10^3\) pixels constitute only a small fraction. Overall, the training and test sets exhibit similar bounding-box size distributions.

6.2 Baseline Details↩︎

We build our model upon the VisionLLM training framework and extend it with task-specific coordinate tokens for spatial prediction. Instead of introducing an additional bounding-box regression head, we formulate localization as an autoregressive token generation problem.

Specifically, we augment the original tokenizer with a discrete coordinate vocabulary \[\mathcal{P} = \{\tau_i \mid i \in [-512, 512]\},\] where each \(\tau_i\) corresponds to a textual token <p\(i\)>. This results in 1025 additional coordinate tokens. After adding these tokens to the tokenizer, we resize the model’s token embedding matrix to match the new vocabulary size. During supervised fine-tuning, both the input token embeddings and the language modeling head are kept trainable, allowing the model to learn representations and output probabilities for the newly introduced coordinate tokens.

For each ground-truth bounding box, the annotation is first converted from \((x,y,w,h)\) to \((x_1,y_1,x_2,y_2)\). Each coordinate is then normalized with respect to the original image resolution and quantized into the coordinate-token range. Given a coordinate value \(v\) along an axis of size \(S\), we compute its discrete index as \[q(v, S) = \mathrm{round} \left( 512 \cdot \mathrm{clip} \left( \frac{v - S/2}{S/2}, -1, 1 \right) \right).\] Here, \(S\) denotes the image width for horizontal coordinates and the image height for vertical coordinates. A bounding box is then serialized as \[\left( \tau_{q(x_1,W)}, \tau_{q(y_1,H)}, \tau_{q(x_2,W)}, \tau_{q(y_2,H)} \right),\] where \(W\) and \(H\) are the original image width and height, respectively.

During data construction, the bounding-box placeholder <BBX> in the assistant response is replaced by the corresponding serialized coordinate tokens. The user prompt, visual tokens, and assistant response are formatted following the Qwen/VisionLLM chat format. The supervised fine-tuning loss is computed only over the assistant response tokens, while tokens belonging to the system and user prompts are masked out. Thus, the model is trained to generate both natural language responses and discrete spatial coordinates in a unified autoregressive manner.

6.3 LLM-as-a-Judge Evaluation Prompt↩︎

We use an LLM-as-a-judge protocol to evaluate the semantic consistency between the predicted hallucination analysis and the ground-truth annotation. The judge assigns a score from 1 to 5, where 5 indicates full semantic consistency and 1 indicates an irrelevant or incorrect prediction. The judge is required to output only a valid JSON object containing the score and a short reason.

The system prompt is:

You are an evaluator. Output only a valid JSON object.

For samples with hallucination annotations, we use the following prompt:

Compare the following two texts:

Prediction:
{prediction}

Ground truth:
{ground_truth}

Consider whether the prediction agrees with the ground truth in meaning,
including the hallucination conclusion and the explanation.

Rate the prediction from 1 to 5:
5: fully consistent.
4: mostly consistent with only minor differences.
3: partially consistent but missing important details.
2: weakly related or largely inconsistent.
1: irrelevant or wrong.

Reply with a JSON object only:
{
  "score": 1-5,
  "reason": "one short sentence"
}

For samples whose ground-truth label is no hallucination, we use the following prompt:

The ground truth hallucination label is:
NO HALLUCINATION.

This means the prediction should clearly indicate that there is no
hallucination and that the information is grounded or correct.

Prediction:
{prediction}

Judge only whether the prediction correctly expresses no hallucination
or an equivalent meaning, such as no error, description matches the image,
or everything is correct.

Ignore wording differences and focus on whether it clearly indicates
no hallucination.

Rate the prediction from 1 to 5:
5: clearly and correctly states no hallucination, or that the information
   is grounded and correct.
4: basically indicates no hallucination but with minor ambiguity.
2-3: unclear or ambiguous about hallucination status.
1: clearly wrong, e.g., claims hallucination or provides hallucinated
   or incorrect information.

Reply with a JSON object only:
{
  "score": 1-5,
  "reason": "one short sentence"
}

In implementation, both the prediction and ground-truth text are truncated to at most 1500 characters before being provided to the judge.

6.4 Computational Resources↩︎

All training experiments were conducted on a computing cluster with 16 NVIDIA A100 GPUs, each equipped with 40 GB of GPU memory. Inference was performed on a single NVIDIA A100 GPU with 80 GB of memory.

6.5 Additional Qualitative Examples↩︎

We provide additional qualitative examples in Figs. 7–10 to further illustrate the fine-grained hallucination detection and localization setting in GAVEL. Each example contains an image–caption pair, the ground-truth hallucination annotation, and model responses from representative vision-language models. The responses include both textual explanations and localized visual evidence, allowing us to compare whether models can correctly identify hallucinated content, explain the inconsistency, and ground the relevant image regions.

References↩︎