1 Introduction↩︎

Vision-language models (VLMs) achiam2023gpt?, bai2025qwen3?, chen2024internvl?, liu2023visual? have achieved strong performance on standard multimodal taskschen2025unireal?, liu2025step1x?, song2024processpainter?, wan2024grid?, peng2025cia? such as visual question answering antol2015vqa?, kuang2025natural? and image captioning ghandi2023deep?. However, many practical scenarios an2024etpnav?, liu2024volumetric?, khan2023visual?, sapkota2025vision?, saleh2024forest? require more than understanding each image independently: they require comparing task-relevant visual attributes across multiple related images. Such attributes may include object count, scene depth, relative distance, or spatial layout. We refer to this capability as cross-image comparative reasoning, which arises naturally in applications such as comparing depth changes across views for navigation or count differences across scenes for monitoring.

Despite its practical relevance, cross-image comparative reasoning remains under-studied and poorly supported by current VLMs. As shown in Fig. 1, even strong recent models often struggle with tasks such as cross-image counting, depth comparison, relative distance judgment, and spatial correspondence, suggesting that this is a systematic limitation rather than an isolated failure case. One reason is that existing benchmarks largely emphasize either single-image perception tong2024cambrian?, fu2024blink? or general multi-image understanding wang2024muirbench?, cheng2025evaluating?, including temporal ordering, narrative comprehension, and image retrieval. While these settings evaluate whether a model can integrate information across images at a holistic semantic level, they provide only limited coverage of fine-grained comparative reasoning, where success depends on accurately comparing specific metric attributes across images.

Beyond benchmark coverage, improving this capability also poses a distinct supervision challenge. A natural strategy is to optimize models only for final-answer correctness chen2025mico?, wan2026remot?, feng2025onethinker?. However, we find that outcome-only optimization is often insufficient for cross-image comparative reasoning: models may arrive at correct answers through shortcut patterns or accidental guessing while producing poorly grounded or internally inconsistent reasoning. In our analysis (Table 4), up to 17.1% of predictions with correct final answers are accompanied by factually invalid reasoning chains, revealing a gap between answer accuracy and reasoning reliability. A natural alternative is to use an LLM as a judge gu2024survey?, liu2025noisyrollout? to assess the reasoning trace, but this is also less suitable for our setting (Fig. 2). The core issue here is not subjective preference over reasoning style, but structured reasoning correctness: whether the model correctly identifies task-relevant attributes in each image, whether intermediate pairwise judgments are correct and consistent with the final answer, and whether all pairwise conclusions satisfy global consistency. Because these properties are explicit and verifiable, they are better handled by direct structured supervision than by holistic free-form judging.

Motivated by this observation, we present CoRe, a framework for studying and improving cross-image comparative reasoning in VLMs. To enable scalable and verifiable supervision, we first construct CoRe-20K, a large-scale triplet-based dataset in which each sample consists of three related images and their associated pairwise comparison questions. Rather than relying on manual annotation or VLM-generated pseudo-labels, CoRe-20K is built automatically from structured visual metadata through a multi-expert pipeline that extracts task-relevant metrics, filters unreliable or trivial triplets, and generates comparison questions with deterministic ground-truth labels. On top of this data, we propose TriSR, a structured reward framework for triplet-based reasoning that decomposes supervision into three complementary signals: Attribute Grounding, Judgment Alignment, and Triplet Consistency. These rewards are optimized jointly with final-answer correctness under GRPO, encouraging models to produce not only correct answers but also grounded and globally consistent reasoning. To evaluate this capability systematically, we further construct CoRe-Bench, a benchmark specifically designed for fine-grained cross-image comparative reasoning. CoRe-Bench covers four task dimensions—counting, depth, distance, and spatial relations—across diverse real-world source domains. Experiments show that CoRe substantially improves performance on this challenging setting over strong VLM baselines, while remaining competitive on broader vision-language benchmarks. Taken together, our results suggest that exploiting the verifiable intermediate structure of comparative reasoning is a promising direction for improving multi-image reasoning in VLMs.

Our contributions are three-fold: (1) we identify fine-grained cross-image comparative reasoning as a distinct and under-served capability of VLMs; (2) we propose TriSR, a structured reward framework that exploits verifiable intermediate structure in triplet-based comparison; and (3) we construct CoRe-20K and CoRe-Bench to enable scalable training and systematic evaluation for this setting.

2 Related Work↩︎

Visual language models and evaluation benchmarks. Visual language models (VLMs) have advanced rapidly in recent years, evolving from early vision-language alignment frameworks such as BLIP li2022blip? and Flamingo alayrac2022flamingo? to instruction-following multimodal systems including LLaVA liu2024improved? and InstructBLIP dai2023instructblip?. More recent proprietary and open-source models, such as GPT-4o hurst2024gpt?, Gemini comanici2025gemini?, and Qwen3-VL bai2025qwen3?, have further strengthened multimodal perception and reasoning. Correspondingly, a broad range of benchmarks have been proposed to assess VLM visual capabilities. Single-image benchmarks, such as MME fu2023mme?, CV-Bench tong2024cambrian?, MMTBench ying2024mmt?, and BLINK fu2024blink?, mainly evaluate perception and reasoning within individual images. More recently, multi-image benchmarks such as MuirBench wang2024muirbench? and MMRB cheng2025evaluating? have expanded evaluation to settings involving multiple related images, covering tasks such as retrieval, temporal ordering, and narrative understanding. However, these benchmarks primarily assess semantic integration across images at a holistic level, rather than fine-grained comparative reasoning across related images.

Data construction. Constructing reliable training data for cross-image comparative reasoning is non-trivial, as labels must capture precise metric relationships between images rather than semantic content within a single image. Existing efforts assran2025v?, liu2025spatial?, dave2022tclr?, sevilla2021only? typically face a trade-off between supervision quality, scalability, and label verifiability. For example, video-text datasets bain2021frozen?, grauman2022ego4d? provide only coarse clip-level descriptions, lacking the metric granularity required for explicit comparative supervision. Manual annotation can produce high-quality labels but is prohibitively labour-intensive and difficult to scale across diverse visual attributes and scene types. More recently, some approaches have used VLMs as annotators to generate pseudo-labels for comparison or reasoning data; however, this strategy risks propagating the very reasoning errors that training aims to correct, and in practice may yield high format-error rates and limited output validity wan2026remot?. A more principled alternative is to derive comparison labels programmatically from structured metadata, which guarantees label verifiability without human annotation effort. Our CoRe-20K dataset adopts this metadata-driven strategy through a multi-expert collaborative pipeline, constructing over 20,000 high-quality triplet-based comparison samples across four metric dimensions: counting, depth, distance, and spatial relations.

Reasoning Enhancement via Reinforcement Learning. The application of reinforcement learning (RL) to enhance the reasoning capabilities of large language models has attracted considerable research attention zheng2025group?, dong2025agentic?, wan2026remot?, chen2024measuring?, shao2024visual?, zhang2023multimodal?, wu2025reinforcing?, liu2025seg?, zhang2025critique?, li2025star?, feng2025video?, liu2025spatial?. A prominent line of work, represented by DeepSeek-R1, employs rule-based RL through the Group Relative Policy Optimization (GRPO) algorithm guo2025deepseek?, optimizing models directly against outcome-level reward signals. While outcome-level supervision effectively drives answer accuracy, we observe that in cross-image comparative reasoning it frequently produces models that arrive at correct answers through erroneous or shortcut reasoning—a failure mode that outcome-only rewards neither diagnose nor suppress. A natural remedy is to incorporate LLM-as-Judge frameworks gu2024survey?, which leverage powerful language models to assess intermediate reasoning quality and provide richer training signals beyond answer correctness. However, we empirically find that LLM-based evaluation exhibits a substantial alignment gap with human judgments on cross-image visual reasoning chains, rendering it unreliable as a direct reward signal for this task. This motivates our proposed TriSR framework, which decomposes holistic reasoning evaluation into three structured, verifiable criteria—Attribute Grounding, Judgment Alignment, and Triplet Consistency—transforming subjective quality assessment into a principled and reliable scoring process that yields more effective reward signals for training.

3 Method↩︎

3.1 Problem Setup↩︎

We study cross-image comparative reasoning over image triplets. A training example is defined as \[\label{eq:training95example} x = (I_1, I_2, I_3, m, \mathcal{Q}, \mathcal{A}),\tag{1}\] where \(I_1, I_2, I_3\) are three related images, \(m\) denotes the task dimension (e.g., counting, depth, distance, or spatial relation), \(\mathcal{Q}\) is the set of pairwise comparison questions, and \(\mathcal{A} = \{A_1, A_2, A_3\}\) denotes the structured metadata associated with the three images. From the metadata, we derive a unified task-relevant attribute representation \[\label{eq:attribute95representation} a_i = f_m(I_i, A_i), \qquad i \in \{1,2,3\},\tag{2}\] where \(a_i\) may be numerical (e.g., count, depth, distance) or categorical (e.g., spatial relation state), depending on the task. For the three image pairs \[\label{eq:image95pairs} \mathcal{P} = \{(1,2), (1,3), (2,3)\},\tag{3}\] a deterministic task-specific comparator \(g_m\) produces the ground-truth pairwise labels \[\label{eq:ground95truth95labels} y^*_{ij} = g_m(a_i, a_j), \qquad (i,j) \in \mathcal{P}.\tag{4}\] We denote the full target as \(y^* = \{y^*_{ij}\}_{(i,j)\in\mathcal{P}}\).

Given a triplet query \(q\), the model produces an output \[\label{eq:model95output} o = (t, \hat{y}),\tag{5}\] where \(t\) is the reasoning trace and \(\hat{y}\) is the final predicted answer. Our goal is not only to maximize final-answer accuracy, but also to enforce that the reasoning trace is grounded in correct per-image attributes, aligned with the predicted answer, and globally consistent across the triplet.

3.2 Metadata-Driven Triplet Construction↩︎

Comparison triplets. Unlike standard single-image instruction data, our setting requires supervision over comparisons across multiple related images. We therefore organize each sample as a comparison triplet \(T=(I_1,I_2,I_3)\), where all three images come from the same source dataset and share a common task dimension \(m\). Each triplet is paired with three coupled comparison sub-questions over \((I_1,I_2)\), \((I_1,I_3)\), and \((I_2,I_3)\). This triplet-level formulation is crucial: it not only supports pairwise supervision, but also exposes higher-order consistency structure across the three comparisons.

Multi-expert collaborative pipeline. As shown in Fig. 3, we construct triplets automatically from structured metadata using a modular pipeline with three expert modules.

Metric Extraction Expert \((f_m : (I_k, A_k) \rightarrow v_k)\). Given image \(I_k\) and its task-specific annotation \(A_k\), this expert derives a unified metric representation \(v_k\). Because source datasets provide heterogeneous annotation formats, we instantiate task-specific extractors and map them into a common comparison space. Concretely, for counting, we read instance counts from crowd annotations; for depth, we compute the median depth of the queried region from ground-truth depth maps; for distance, we derive ego-to-object Euclidean distance from 3D annotations; and for spatial relations, we infer the target relation label from object-level bounding box annotations. This step decouples supervision from VLM-generated pseudo-labels and yields deterministic comparison targets.

Quality Control Expert \((\phi_m : (v_1,v_2,v_3)\rightarrow\{0,1\})\). Not every image triplet produces reliable or informative supervision. We therefore retain a triplet only if all three pairwise comparisons satisfy task-specific validity constraints. Let \[\label{eq:diff95measure} \delta_{ij} = d_m(v_i, v_j)\tag{6}\] denote the task-specific difference measure between images \(I_i\) and \(I_j\). For numerical tasks, \(d_m\) is the absolute metric difference; for categorical tasks, it is replaced by task-specific validity rules. We accept a triplet if \[\label{eq:validity95cond} \phi_m(v_1,v_2,v_3)=1 \quad \Longleftrightarrow \quad \delta_{ij}\in\mathcal{T}_m,\;\forall (i,j)\in\mathcal{P},\tag{7}\] where \(\mathcal{T}_m\) denotes a task-adaptive admissible set.

For numerical tasks, \(\mathcal{T}_m=[\theta_{\min}^m,\theta_{\max}^m]\) serves two purposes. The lower bound \(\theta_{\min}^m\) suppresses label noise by excluding pairs whose true metric differences are too small relative to annotation uncertainty. The upper bound \(\theta_{\max}^m\) removes overly easy comparisons whose answers may be inferred from superficial cues without genuine comparative reasoning. For categorical tasks such as spatial relations, we instead enforce validity and diversity constraints to avoid degenerate triplets in which all pairwise relations collapse to the same label.

Question Generation Expert. For each retained triplet, we instantiate a triplet-level prompt containing the three pairwise comparison sub-questions. Questions are generated from a curated template library, while the correct options are determined deterministically from the extracted metric representations. To reduce answer-position bias, we randomly shuffle the candidate options before assigning labels, ensuring that correct choices are approximately uniformly distributed across answer positions.

CoRe-20K and CoRe-Bench. Using this fully automatic pipeline, we construct CoRe-20K, a collection of 20,000 comparison triplets with balanced coverage across four task dimensions: counting, depth, distance, and spatial relations, each contributing 5,000 triplets. Since each triplet contains three pairwise sub-questions, the full collection corresponds to 60,000 comparison QA items.

From this pool, we reserve 1,000 triplets per task dimension to form CoRe-Bench, yielding 4,000 held-out evaluation triplets in total. The remaining 16,000 triplets are used for training. This split preserves balanced coverage across all tasks while ensuring that evaluation is conducted on disjoint triplets.

3.3 TriSR: Structured Reward for Triplet Reasoning↩︎

Cross-image comparative reasoning exposes a useful property: its intermediate reasoning steps are not purely stylistic, but partially verifiable. A correct solution should (1) identify the task-relevant attribute in each image, (2) derive pairwise judgments consistent with both the ground truth and the final answer, and (3) satisfy global consistency across the three pairwise comparisons. We therefore design TriSR, a structured reward framework that supervises these three aspects directly.

Structured output parsing. To make intermediate supervision executable, we prompt the model to produce a structured response containing: (1) per-image attribute estimates \(\{\hat{v}_k\}_{k=1}^3\), (2) pairwise comparative judgments inferred in the reasoning trace \(\{\tilde{y}_{ij}\}_{(i,j)\in\mathcal{P}}\), and (3) final predicted answers \(\{\hat{y}_{ij}\}_{(i,j)\in\mathcal{P}}\). A lightweight rule-based parser extracts these fields from each sampled response. TriSR then assigns rewards based on the extracted structure.

Attribute Grounding. The first requirement of comparative reasoning is that the model correctly grounds the task-relevant attribute in each image. We define an Attribute Grounding reward \[\label{eq:reward95ag} R_{\text{ag}}(o) = \frac{1}{3}\sum_{k=1}^{3} s_m(\hat{v}_k, v_k),\tag{8}\] where \(s_m(\cdot,\cdot)\) is a grounding score between the model-estimated attribute \(\hat{v}_k\) and the metadata-derived target \(v_k\). We use \[\label{eq:score95numerical} s_m(\hat{v}_k, v_k) = \exp\!\left( -\frac{|\hat{v}_k - v_k|}{\max(|v_k|,\epsilon)} \right),\tag{9}\] where \(\epsilon>0\) avoids division by zero. This reward encourages the model to ground its reasoning in correct per-image attribute estimates rather than relying on ungrounded heuristics.

Judgment Alignment. Even when the final answer is correct, the reasoning trace may imply the wrong pairwise conclusion, or may contradict the model’s own final prediction. We therefore define a Judgment Alignment reward that jointly measures factual correctness and internal consistency: \[\label{eq:reward95ja} R_{\text{ja}}(o) = \frac{1}{2|\mathcal{P}|} \sum_{(i,j)\in\mathcal{P}} \left( \mathbf{1}[\tilde{y}_{ij}=y^*_{ij}] + \mathbf{1}[\tilde{y}_{ij}=\hat{y}_{ij}] \right).\tag{10}\] The first term checks whether the comparative judgment stated in the reasoning trace matches the ground-truth answer; the second checks whether that judgment is consistent with the model’s final selected answer. Maximum reward is obtained only when the reasoning trace is both correct and self-consistent.

Triplet Consistency. Pairwise comparisons over three images induce higher-order logical structure. For example, if image \(I_1\) is judged larger than \(I_2\) on a given metric and \(I_2\) is judged larger than \(I_3\), then the comparison between \(I_1\) and \(I_3\) should be globally compatible with those two intermediate judgments. We capture this property with a Triplet Consistency reward.

We define a task-specific verifier \[\label{eq:verifier} \tau_m(\hat{y}_{12}, \hat{y}_{13}, \hat{y}_{23}) \in \{-1,0,1\},\tag{11}\] where \(+1\) indicates logical consistency, \(-1\) indicates contradiction, and \(0\) denotes cases in which no non-trivial consistency check can be established.

For ordinal comparison tasks, we map each answer to a signed relation \(s_{ij}\in\{-1,0,+1\}\). If \(s_{12}\) and \(s_{23}\) jointly imply a unique relation between \(I_1\) and \(I_3\), we verify whether the predicted \(s_{13}\) matches that implication. For interval-based answers, we convert each answer to a quantitative range \(\Delta_{ij}\) and check whether the composed interval \(\Delta_{12}+\Delta_{23}\) is compatible with \(\Delta_{13}\) up to quantization tolerance. The resulting reward is \[\label{eq:reward95tc} R_{\text{tc}}(o) = \tau_m(\hat{y}_{12}, \hat{y}_{13}, \hat{y}_{23}).\tag{12}\] Importantly, \(R_{\text{tc}}\) depends only on the model’s own predictions and therefore acts as a self-consistency signal that can penalize logically incompatible outputs even when individual pairwise answers are considered in isolation.

Composite reward. We combine the structured rewards with standard task accuracy. Let \[\label{eq:reward95task} R_{\text{task}}(o) = \frac{1}{|\mathcal{P}|} \sum_{(i,j)\in\mathcal{P}} \mathbf{1}[\hat{y}_{ij}=y^*_{ij}]\tag{13}\] denote the average pairwise answer accuracy within a triplet. The final reward is \[\label{eq:reward95total} R(o) = R_{\text{task}}(o) + \lambda_{\text{ag}} R_{\text{ag}}(o) + \lambda_{\text{ja}} R_{\text{ja}}(o) + \lambda_{\text{tc}} R_{\text{tc}}(o),\tag{14}\] where \(\lambda_{\text{ag}}, \lambda_{\text{ja}}, \lambda_{\text{tc}}\) control the relative contribution of each structured component. This formulation encourages the model to produce answers that are accurate, well-grounded, internally consistent, and globally coherent across the triplet.

3.4 Optimization with GRPO↩︎

We optimize the model with Group Relative Policy Optimization (GRPO) guo2025deepseek?. For each query \(q\), we sample \(G\) responses \(\{o_i\}_{i=1}^{G}\) from the old policy \(\pi_{\theta_{\text{old}}}\), evaluate each response with the composite reward \(R(o_i)\), and compute group-normalized advantages: \[\label{eq:advantage} \hat{A}_i = \frac{R(o_i)-\mathrm{mean}(\{R(o_j)\}_{j=1}^{G})}{\mathrm{std}(\{R(o_j)\}_{j=1}^{G})+\epsilon}.\tag{15}\] The policy is then updated by maximizing \[\label{eq:objective95grpo} J(\theta) = \mathbb{E}_{q,\{o_i\}} \left[ \frac{1}{G}\sum_{i=1}^{G} \min\!\left( r_i \hat{A}_i,\, \mathrm{clip}(r_i,1-\varepsilon,1+\varepsilon)\hat{A}_i \right) - \beta D_{\mathrm{KL}} \right],\tag{16}\] where \[\label{eq:importance95ratio} r_i=\frac{\pi_{\theta}(o_i\mid q)}{\pi_{\theta_{\text{old}}}(o_i\mid q)}\tag{17}\] is the importance ratio, and \(\beta D_{\mathrm{KL}}\) regularizes the updated policy toward the reference distribution. Since the reward combines both outcome correctness and structured reasoning signals, GRPO encourages the model not only to obtain the correct final answer, but also to arrive there through grounded and logically consistent comparative reasoning.

4 Experiment↩︎

We evaluate CoRe from three complementary perspectives: (1) benchmark-level performance on CoRe-Bench, (2) diagnostic analysis of structured reasoning quality, and (3) generalization to broader VLM benchmarks.

4.1 Experimental Setup↩︎

Hyperparameters. We adopt Qwen3-VL-4B-Thinking bai2025qwen3? as our base model, as it provides a strong balance between reasoning capability and computational efficiency. The model is kept in Thinking mode to retain its intrinsic chain-of-thought capability. Each sample is formatted as <think>...</think> <answer>ans</answer>, and the task accuracy reward is computed solely against tokens within the <answer> block. For reinforcement learning, we adopt GRPO guo2025deepseek? with a composite reward that combines outcome accuracy with the three proposed structured rewards, namely attribute grounding, judgment alignment, and triplet consistency. The corresponding reward weights are set to \(\lambda_{ag} = 0.2\), \(\lambda_{ja} = 0.2\), and \(\lambda_{tc} = 0.3\). In addition, three auxiliary rewards are incorporated as standard training stabilizers: a format reward that encourages syntactically valid answer structure, a cosine CoT length reward that penalizes excessively verbose reasoning chains, and a repetition penalty that discourages degenerate token repetition. These auxiliary components are provided by the ms-swift training library and are not contributions of this work; we therefore omit their implementation details. A KL regularization coefficient of \(\beta = 0.01\) constrains reward drift from the reference policy. Training uses a batch size of 4 with 4 rollouts per sample, a learning rate of \(1 \times 10^{-6}\) with cosine decay schedule, and AdamW optimization, running for 1 epochs on \(8\times\) A800 GPUs with mixed precision.

Evaluation Protocols. We follow the default inference configuration of Qwen3-VL and evaluate all models using the VLMEvalKit toolkit duan2024vlmevalkit?. For all benchmarks, we unify the prompting format and answer extraction rules. Each question is decoded through the model’s reasoning head and parsed from the final <answer> token. To comprehensively capture model capability under the triplet structure, we define two complementary metrics: Overall Accuracy (Ov.), which marks a triplet as correct only if all three pairwise sub-questions are answered correctly, emphasizing global metric consistency; and Partial Accuracy (Par.), which assigns a proportional score based on the ratio of correctly answered sub-questions within a triplet, reflecting localized reasoning ability under partial understanding.

4.2 Main Results↩︎

Quantitative Results. Table 1 reports overall and partial accuracies on CoRe-Test-Bench across four comparative reasoning dimensions. Existing general-purpose VLMs perform poorly on this benchmark, with average overall accuracy remaining below 7% for all baselines. In particular, Qwen3-VL-4B–CoT achieves 6.6% average overall accuracy and 25.8% partial accuracy, while InternVL3-8B and LLaVA-OneVision-8B obtain only 2.1% and 2.2% average overall accuracy, respectively. These results indicate that current VLMs struggle substantially with cross-image metric comparison, especially on counting and distance reasoning, where most models achieve near-zero or single-digit overall accuracy. In contrast, our CoRe-4B-CoT achieves 22.2% average overall accuracy and 54.0% partial accuracy, substantially outperforming all baselines. Compared with the strongest baseline, Qwen3-VL-4B–CoT, our model improves overall accuracy by 15.6 absolute points (from 6.6% to 22.2%) and partial accuracy by 28.2 points (from 25.8% to 54.0%), respectively. The improvements are consistent across all four task dimensions, with particularly large gains on depth (33.0% vs.%) and distance (24.8% vs.%), suggesting that the proposed training strategy is especially effective for cross-image comparative reasoning.

Generalization. To examine whether the gains are tied to the triplet benchmark itself, we construct CoRe-OOD-Binary, a two-image out-of-distribution benchmark from external datasets that are disjoint from CoRe-Bench. As shown in Table 2, CoRe improves over the base model on all four OOD dimensions.

4.3 Ablation Study↩︎

To analyze the contribution of each TriSR component, we conduct ablation experiments on a 10% subset of CoRe-20K-training for efficiency; absolute accuracies are therefore lower than those in Table 1, but the relative trends remain informative.

Effect of TriSR reward components. Table 3 shows that training with \(R_{\text{task}}\) alone already yields substantial gains over the base model (43.6% vs.% partial accuracy), confirming that outcome-based fine-tuning provides a strong baseline. The full TriSR framework further improves partial accuracy to 45.7%, demonstrating that structured intermediate supervision offers complementary gains beyond final-answer correctness. Ablating each component individually reveals consistent degradation: removing \(R_{\text{tc}}\) causes the largest drop (45.7%\(\to\)​44.3%), followed by \(R_{\text{ja}}\) (44.5%) and \(R_{\text{ag}}\) (45.1%), confirming that all three components contribute independently.

Reasoning quality analysis. Table 4 provides a finer-grained view based on manual inspection of 300 randomly sampled triplets, categorizing predictions by the joint correctness of the final answer and reasoning chain. The base model shows that nearly half of its correct predictions are accompanied by invalid reasoning, with a CA+WR rate of 11.5% and a Wilson 95% confidence interval of [8.4, 15.7]. This reveals a substantial gap between answer accuracy and reasoning reliability. Training with \(R_{\text{task}}\) alone exacerbates this issue, increasing CA+WR to 17.1% with a Wilson 95% confidence interval of [13.2, 21.8], indicating that outcome-only supervision reinforces shortcut reasoning. In contrast, CoRe reduces CA+WR to 8.3% with a Wilson 95% confidence interval of [5.7, 11.9], while increasing CA+CR from 12.6% to 38.4%. These results confirm that TriSR’s structured rewards steer the model toward predictions that are both correct and verifiably grounded.

4.4 Evaluation on Other VLM Benchmarks↩︎

To assess the generalizability of CoRe beyond our proposed benchmark, we evaluate it on a diverse set of vision-centric benchmarks, including CV-Bench tong2024cambrian?, BLINK fu2024blink?, RealWorldQA (RW-QA) Grok15vpreview?, MMT-Bench (MMT) Ying2024MMTBenchAC?, MMStar Chen2024AreWOMMStar?, MMVP Tong2024EyesWSMMStar?, MME-RealWorld (MME-RW) Zhang2024MMERealWorldCY?, \(V^*\) Bench (\(V^*\)) Wu2023VGVStar?, and HRBench (HR8K) Wang2024DivideCAHRBench?. Results are shown in Table 5. CoRe demonstrates strong transferability across general VLM benchmarks and achieves the best performance among open-source models on 5 out of 9 benchmarks, including CV-Bench, BLINK, RW-QA, MMT, and MMStar. Compared with the Qwen3-VL-4B-CoT baseline, CoRe improves accuracy by 0.2%, 1.1%, 0.2%, 1.1%, and 0.1% on these five benchmarks, respectively. CoRe also remains competitive on the remaining benchmarks, ranking second on \(V^*\) and tying for the best result on HR8K. Although CoRe does not achieve the top score on MMVP and MME-RW, its performance remains strong, indicating that training for comparative reasoning does not lead to catastrophic forgetting on broader multimodal evaluation tasks. Notably, CoRe-4B matches or surpasses several larger 7B- and 8B-scale open-source models on multiple benchmarks, suggesting that targeted data construction and structured reward design can be more important than raw model scale for improving fine-grained comparative reasoning.

4.5 Qualitative Analysis↩︎

We visualize representative examples from CoRe-Bench in Fig. 4, showing side-by-side comparisons between Qwen3-VL and CoRe on four cross-image reasoning tasks. The results show that Qwen3-VL often makes errors when comparing subtle visual differences across related images, whereas CoRe produces more reliable comparative reasoning and correct final predictions.

Additional analyses, including training strategy comparison, triplet design validation, and limitations, are provided in the supplementary.

5 Conclusion↩︎

We present CoRe, a comprehensive framework for cross-image comparative reasoning in vision-language models. Through a multi-expert collaborative pipeline, we construct CoRe-20K, a large-scale dataset derived from structured visual metadata. Combined with TriSR’s structured reasoning rewards and GRPO optimization, CoRe achieves state-of-the-art performance on CoRe-Bench and multiple VLM benchmarks. Our analysis further reveals that outcome-only supervision is insufficient for this setting, as correct answers can coexist with factually invalid reasoning chains, highlighting the importance of supervising verifiable intermediate structure.