DialogueVPR: Towards Conversational Visual Place Recognition
May 08, 2026
Abstract
Inspired by how humans communicate spatial information, language-guided geo-localization has gained significant traction for its intuitive and practical value. Despite this progress, most methods still rely on a static, one-shot retrieval paradigm, which fails to handle the ambiguity and incompleteness inherent in real-world natural language descriptions. We propose a paradigm shift to reasoning retrieval and introduce Dialogue Place Recognition (DlgPR), which casts localization as an interactive, dialogue-driven reasoning process. To support this new task, we present DlgQuest-Cities, the first large-scale dialogue-based benchmark for place recognition, and a unified reasoning framework that couples a cross-modal multi-level retriever with an intelligent questioner, DQ-pilot. DQ-pilot is trained in a curriculum: supervised fine-tuning on a curated DQ-cities-20k subset followed by reinforcement refinement on a harder DQ-cities-10k split via GRPO. Two task-aligned metrics guide learning: a Discriminative Difficulty Index (DDI) for curriculum sampling and a Positional Retrieval Gain (PRG) reward that directly measures retrieval improvement induced by a question. Experiments show this reasoning-based approach significantly outperforms baselines. The code and model are available at https://github.com/Graysonggg/DlgPR.
1 Introduction↩︎
Accurately perceiving and determining one’s location remains a fundamental challenge for both humans [1] and intelligent agents [2]. Solving this problem underpins a wide range of applications, including precise pedestrian navigation in urban environments, autonomous robot operation in dynamic scenes, and localization correction in GPS-denied areas such as urban canyons [3]. Motivated by these demands, community’s recent research has explored a more intuitive paradigm—place recognition driven by natural language descriptions [3]–[7]. Reflecting everyday human interactions, these approaches holds strong practical value: a passenger verbally guiding a taxi driver [8], identifying a place through spoken directions, or describing the surroundings in an emergency call [9], or commanding a home service robot through natural language [10].
Recent language-driven localization methods, such as Text2Pose [4] and Text2Loc [5], primarily focus on identifying individual locations within 3D point clouds. However, constructing and storing large-scale 3D maps remains costly, hindering practical deployment. Instead, recent work [8], [11] frames the problem as a large-scale retrieval task by correlating natural language with expansive, readily available visual data like satellite or street-view images. Despite progress, most language-guided localization methods still follow a static retrieval paradigm, where a fixed textual query is processed once to return the best-matching location. The fundamental limitation of this design lies in its passivity: it fails to handle the ambiguity inherent in real-world descriptions. When the initial input is vague or incomplete—such as an imprecise verbal account (the “user description dilemma”) or an erroneous recollection—these systems cannot actively seek clarification or gather additional information. Consequently, single-turn, non-interactive retrieval remains fragile in dynamic, real-world scenarios.
To transcend these constraints, we argue that geo-localization should evolve from passive retrieval to an advanced paradigm of reasoning Retrieval. An intelligent agent must move beyond passive matching toward active understanding, reasoning, and interaction with uncertain environments and ambiguous human instructions.
To drive this paradigm shift, we introduce Dialogue Place Recognition (DlgPR)—a new task that reformulates localization as an iterative, collaborative dialogue. In DlgPR, the system transforms from a passive retriever into an active reasoner: it analyzes candidate locations, proactively engages the user with targeted questions to obtain discriminative evidence, and incrementally refines its belief about the correct place as the dialogue history becomes richer and the information more complete. Specifically, we develop a unified reasoning framework composed of a Cross-Modal Progressive Learning (CMPL) Retriever and an intelligent Multimodal Large Language Model, Dialogue-Quest-Pilot (DQ-pilot). The CMPL retriever is responsible for iteratively integrating information from the evolving dialogue to refine its search and retrieve relevant candidate locations. These candidates are then passed to DQ-pilot, which acts as the reasoning core—diagnosing ambiguity and generating questions to maximize information gain. This synergy transforms the system from a passive retriever into an active reasoner, enabling efficient and precise localization by incrementally refining its belief as the dialogue unfolds.
Our main contributions are summarized as follows:
We propose a novel task, Dialogue Place Recognition (DlgPR), which shifts the paradigm from static retrieval to active, dialogue-driven reasoning. To facilitate research on this new task, we construct DlgQuest-Cities (DQ-cities), the first large-scale benchmark dataset for dialogue-based place recognition.
We develop DlgQuest, a unified and effective reasoning framework featuring a cross-modal retriever (CMPL) and an MLLM agent (DQ-Pilot). Crucially, to train this framework, we introduce a novel curriculum learning strategy guided by two task-aligned metrics—a Discriminative Difficulty Index (DDI) and a Positional Retrieval Gain (PRG)—enabling the agent to learn progressively from basic perception to advanced reasoning. Extensive experiments demonstrate the superiority of our approach.
2 Related Work↩︎
2.1 Natural Language-Driven Visual Perception and Localization↩︎
Geo-localization [12]–[22], NetVLAD?, SALAD?, BoQ?, SuperVLAD?, EMVP?, SelaVPR? predicts a query’s location by retrieving similar images from a geo-tagged database. Recently, multi-modal retrieval incorporating natural language has emerged in this field [7], [11], [23]–[26]. For example, [8] introduces scene text, breaks through the limitation of text length, and, for the first time, introduces an interpretability framework, ensuring the localization process is no longer a black box. Meanwhile, [27] enhances the model’s spatial perception capabilities by learning phrases that describe fine-grained spatial relationships in natural language through Blending Spatial Matching. In 3D localization, [4] uses natural language instructions for position matching in point clouds, while [5] advances this by directly fusing textual semantics with geometric features for end-to-end position regression. For indoor recognition, [26] refines ranking using discriminative text filtered from images. Despite their success, these text-driven geolocation tasks remain largely static and lack dynamic interaction capabilities.
2.2 Interactive Retrieval↩︎
Cross-modal interactive retrieval has been actively explored in text-to-image [28]–[31] and text-to-video domains [32], [33], encompassing various interaction formats [34]–[36]. For example, [33] diversifies question generation, while PlugIR [29] decouples dialogue understanding from retrieval via LLMs, enabling compatibility with black-box models. Furthermore, LLaVA-ReID [31] generates questions maximizing information gain through forward-looking supervision.
Ultimately, interactive retrieval aims to replicate human-like logical reasoning. However, current multi-turn dialogue methods primarily perform reactive information aggregation based on explicit feedback, lacking deeper proactive reasoning capabilities. In contrast, our work pioneers the first multi-modal interactive reasoning task in the field of geolocation.
2.3 Visual Reinforcement Learning↩︎
The advent of the OpenAI’s o1 [37] and DeepSeek-R1 reasoning model [38] introduced the paradigm of incorporating visual reasoning into visual tasks. Reinforcement learning (RL) is pivotal for endowing models with reasoning capabilities, and Group Relative Policy Optimization (GRPO) [39], characterized by its verifiable rewards, has emerged as a prominent RL methodology. Building on this, VLM-R1 [40] developed multiple verifiable reward functions to fine-tune Vision-Language Models (VLMs). Subsequently, Visual-RFT [41] formulated simple yet effective reward functions for diverse visual tasks, further enabling efficient learning under data-scarce conditions. Existing research demonstrates that, compared to Supervised Fine-Tuning (SFT), GRPO facilitates deeper reasoning, offers greater interpretability through its reasoning process, and exhibits superior generalization under limited supervision. Therefore, our proposed framework, DlgQuest, employs both SFT and GRPO to achieve active, reasoning-based geolocation.
3 DlgQuest-Cities↩︎
3.1 Overview↩︎
To support the dialogical reasoning required by our proposed DlgPR task, we construct the DlgQuest-Cities (DQ-cities) dataset. This new benchmark is built upon the widely-used GSV-Cities collection [42], augmenting its rich geo-tagged imagery with multi-layered annotations tailored for dialogue-based localization. Each location in DlgQuest-Cities is annotated with information specifically designed for interactive spatial reasoning. Specifically, the dataset includes: (1) Initial ambiguous place captions, simulating users’ vague or uncertain verbal queries based on incomplete memories; (2) Fine-grained place descriptions, offering comprehensive visual semantic details that serve as the factual foundation for multi-turn reasoning; (3) Region-level annotations, where bounding boxes are paired with corresponding textual descriptions to provide localized evidence for spatial grounding; and (4) Multi-turn, goal-oriented dialogues, in which each question is purposefully designed to differentiate visually similar locations and progressively resolve ambiguity. DQ-cities in total consists of 106,880 location images and 30k carefully selected conversation samples. Each fine-grained description has an average of 154.6 words, with the maximum reaching up to 262 words.
3.2 Dataset Construction↩︎
The rich annotations in DlgQuest-Cities are generated via an automated, multi-stage pipeline designed to produce the textual and dialogical data needed to train DQ-pilot. This pipeline, illustrated in Fig.1, is specifically engineered to synthesize strategy-aware dialogues for each place. It consists of four principal stages: (1) text-modality expansion, (2) text-driven region-level annotation, (3) Chain-of-Thought (CoT) based dialogue generation guided by GPT-4o and (4) Curriculum sampling based on discrimination difficulty.
Step 1: Text Modality Expansion. This initial stage is responsible for creating the foundational textual layers for each place. To emulate a user’s initial query, the pipeline first generates an initial ambiguous caption (e.g., “I see a bank with a telephone booth beside it.”). Following this, the system expands the caption into a long-form, fine-grained place description. This detailed narrative serves as a fact-rich foundation for subsequent dialogue generation. The generation process is constrained by a structured prompt (see Appendix) with task-specific rules: it focuses strictly on static elements (e.g., buildings, signage, spatial relationships), disregards transient objects (e.g., cars), and emphasizes features informative for place recognition. This procedure ensures the resulting descriptions provide reliable, factually grounded information for downstream reasoning.
Step 2: Region-Level Visual Evidence Construction To ground the dialogue in specific visual details, DlgQuest-Cities incorporates region-level annotations. We begin by extracting salient noun phrases from the fine-grained descriptions. Unlike approaches such as FG-CLIP [43] that often rely on simple nouns, we employ a greedy expansion strategy with spaCy to capture maximally descriptive phrases, including rich adjectival modifiers and prepositional clauses (e.g., “the red brick bank with green awnings”). These descriptive phrases serve as more effective text prompts for an open-vocabulary detector (YOLO-World), enabling it to localize the corresponding objects with greater precision. The final output is a set of structured annotations, where each annotation links a specific image region (the bounding box) to its corresponding textual phrase. This step enriches the visual evidence base of the place, effectively avoiding the omission of key details by the teacher model.
Step 3: Chain-of-Thought Dialogue Generation This stage constructs the interactive reasoning samples that power the DlgPR training process. For each place, we synthesize a five-round dialogue sequence, where each round simulates one reasoning–questioning cycle of the teacher model.
At each dialogue round \(i\), the pipeline assembles a decision-making context composed of: (1) a compact and distinctive candidate set—comprising the target image \(I_{\mathrm{t}}\), positive samples \(I_{\mathrm{p}}\) from the same location, and two challenging negatives \(I_{\mathrm{n1}}, I_{\mathrm{n2}}\) retrieved by trained CMPL; (2) the evidence base \(\mathcal{E}_i = \{(I_{\mathrm{t}}, t_{\mathrm{t}}, B_{\mathrm{t}}), (I_{\mathrm{p}}, t_{\mathrm{p}}, B_{\mathrm{p}}), (I_{\mathrm{n}}, t_{\mathrm{n}}, B_{\mathrm{n}})\}\), where \(t\) and \(B\) denote the textual descriptions and bounding boxes obtained in Steps 1 and 2; and (3) the accumulated dialogue history \(D_i\).
Next, the teacher model (GPT-4o) is prompted to execute a four-step chain-of-thought before composing the next question:
Analyze Dialogue History: summarize confirmed and ruled-out evidence contained in \(D_i\);
Validate Candidates Against Dialogue: compare each candidate’s evidence in \(\mathcal{E}_i\) with \(D_i\) and eliminate inconsistent ones;
Identify Key Visual Differentiators: examine the remaining candidates within \(\mathcal{E}_i\) to pinpoint region-grounded, text-anchored cues that most clearly distinguish them;
Formulate a Strategy: Design a question that targets the most decisive visual uncertainty to maximize information gain.
Finally, the teacher’s internal deliberation and proposed question are wrapped in \(\langle think \rangle \langle /think \rangle\) and \(\langle question \rangle \langle /question \rangle\).
Step 4: Discriminative Difficulty-Aware Curriculum Sampling To ensure DQ-pilot learns progressively from simple to complex scenarios, we introduce a curriculum-aware sampling strategy. This strategy is guided by a unified Discriminative Difficulty Index (DDI), a weighted score combining two complementary metrics: Semantic Ambiguity (SA) and Retriever-Informed Difficulty (RID).
Semantic Ambiguity (SA). SA quantifies the intrinsic ambiguity of a candidate set. Given positive and negative textual embeddings \(t_{\mathrm{p}}\) and \(t_{\mathrm{n}}\), and their corresponding visual embeddings, we compute: \[\begin{align} \mathrm{SA} = {}& \alpha \cdot \mathrm{sim}\!\big(\phi_T(t_{\mathrm{t}}), \phi_T(t_{\mathrm{n}})\big) \\ & + (1-\alpha) \cdot \big(1 - \mathrm{sim}(\phi_T(t_{\mathrm{t}}), \phi_T(t_{\mathrm{p}}))\big). \end{align}\] where \(\phi_T(\cdot)\) is the text encoder of CMPL. A higher SA indicates stronger semantic overlap and thus greater ambiguity among candidates.
Retriever-Informed Difficulty (RID). RID measures the empirical difficulty of a dialogue turn by quantifying the rank improvement of positive samples after answering the generated question. Let \(r_j^{(i-1)}\) and \(r_j^{(i)}\) be the rank of a positive item \(j \in \mathcal{P}\) before and after dialogue round \(i\). The Positional Retrieval Gain (PRG) normalizes the observed rank improvement against the maximum possible improvement: \[\mathrm{PRG}_i = \frac{G^{(i)} - G^{(i-1)}}{G^{*} - G^{(i-1)}},\] where gain \(G\) is the sum of nDCG-style [44] contributions \(c(r)=1/\log_2(r+1)\) over all items in \(\mathcal{P}\), and \(G^{*}\) represents the ideal total gain if all positive items occupied the top ranks (\(G^{*}=\sum_{k=1}^{|\mathcal{P}|} c(k)\)). We then set \(\mathrm{RID}_i = 1 - \mathrm{PRG}_i\), so that minimal rank improvement (low PRG) corresponds to high empirical difficulty.
DDI-based Curriculum Sampling. We first filter out low-quality dialogues (e.g., with minimal rank changes, \(\mathrm{PRG}_i < \tau_1\)) and overtly noisy ones using automated metrics. This automated screening is complemented by a brief manual inspection, primarily focused on borderline cases, to ensure overall data integrity. For the resulting filtered pool, we compute the final difficulty score: \[\mathrm{DDI} = w_{\mathrm{sa}}\cdot\mathrm{SA} + w_{\mathrm{rid}}\cdot\mathrm{RID}.\] Using a threshold on the DDI score, we construct a two-stage curriculum:
Stage 1 (For Supervised Fine-Tuning): We sample 20k instances as DQ-cities-20k, prioritizing low-DDI samples (\(\sim\!70\%\)). This stage focuses on learning fundamental visual grounding and core reasoning patterns.
Stage 2 (For Reinforcement Learning): We sample 10k instances as DQ-cities-10k, prioritizing high-DDI samples (\(\sim\!70\%\)). This stage challenges the model with highly ambiguous and hard-to-distinguish cases.
This entire pipeline, from description generation to curriculum sampling, produces the final 30k dialogue rounds in DQ-Cities. The resulting dataset is not only rich in content but also structured to facilitate progressive learning, advancing the model from basic visual grounding to robust, evidence-backed reasoning. More dataset statistics and construction details are provided in Appendix.
4 Method↩︎
4.1 The Dialogue Place Recognition Framework↩︎
The DlgPR framework reframes place recognition as a dynamic, interactive reasoning process, departing from traditional static retrieval. It orchestrates two core components: a multi-modal retriever, CMPL, that iteratively refines the search and a dialogue agent, DQ-pilot, that generates discriminative questions to resolve ambiguity.
The process begins when an initial user query, \(d_0\), yields a coarse set of candidate locations \(C_0\) via the CMPL retriever. To disambiguate these candidates, the system enters an iterative loop. At each round \(t\), DQ-pilot analyzes the current candidates \(C_t\) to formulate an optimal question \(q_t\). Upon receiving the user’s answer \(a_t\), the framework aggregates the dialogue history into an enriched textual query \(d_{t+1} = \text{concat}(d_0, a_1, \dots, a_t)\) for the CMPL retriever. This updated query \(d_{t+1}\) enables CMPL to perform a more informed retrieval, producing a refined candidate set \(C_{t+1}\). This cycle of question-answering and retrieval progressively narrows the search space, achieving robust localization by resolving ambiguities through natural conversation.
4.2 Cross-Modal Progressive Learning Retriever↩︎
To support dialogue-driven reasoning, our Cross-Modal Progressive Learning (CMPL) retriever incrementally refines visual-textual alignment from local to global granularity.
Progressive Feature Alignment. We extract hierarchical visual patches \(V^{(l)}\) and text tokens \(T^{(l)}\) from intermediate layers \(P = \{p_3, p_6, p_9, p_{12}\}\). To highlight geographically relevant cues, \(V^{(l)}\) is refined into \(V_{s}^{(l)}\) via a saliency filtering module (SFM) that dynamically selects discriminative tokens based on attention weights, supervised by an auxiliary loss \(L_{vpr}\) [45].
To bridge modality structures, we introduce a shared fine-grained extractor \(E_f\) and learnable instance-concept queries \(Q^{(l)}\). Acting as semantic anchors, they distill \(V_{s}^{(l)}\) and \(T^{(l)}\) into unified representations: \[F_v^{(l)} = E_f(Q^{(l)}, V_{s}^{(l)}), \quad F_t^{(l)} = E_f(Q^{(l)}, T^{(l)}).\]
Hierarchical Similarity Distribution Matching. We apply an SDM loss [46] at multiple granularities to minimize the bidirectional KL-divergence between the predicted similarity distribution \(p\) and the ground-truth \(q\). For an image anchor \(F_{v,i}\), its predicted distribution across \(B\) batch texts is: \[p_{v \to t, i, j} = \frac{\exp(s_{i,j} / \tau)}{\sum_{k=1}^{B}\exp(s_{i,k} / \tau)},\] where \(s_{i,j}\) is the similarity score and \(\tau\) is the temperature. The target \(q\) is normalized from binary batch labels.
Hard-Negative Isolation (HI). To further improve geometric separability, we propose a Hard-Negative Isolation (HI) loss that applies localized repulsion to the most confusing negatives within each batch. For an image–text pair \((F_{v,i}, F_{t,i})\), the hardest negatives \(j^*\) and \(k^*\) are selected by similarity, and the margin-based triplet objective \(L_{\text{hi}} = [d(F_{v,i},F_{t,i})^2 - d(F_{v,i},F_{t,j^*})^2 + \alpha]_+ + [d(F_{t,i},F_{v,i})^2 - d(F_{t,i},F_{v,k^*})^2 + \alpha]_+\) enforces discriminative separation across modalities.
| Method | Round | |||||||||||||||||||||||||||||||
| R@1 | R@5 | R@1 | R@5 | R@1 | R@5 | R@1 | R@5 | R@1 | R@5 | |||||||||||||||||||||||
| Initial | round0 | / | ||||||||||||||||||||||||||||||
| Qwen2.5-VL-7B | round3 | |||||||||||||||||||||||||||||||
| round5 | ||||||||||||||||||||||||||||||||
| Qwen2.5-VL-72B | round3 | |||||||||||||||||||||||||||||||
| round5 | ||||||||||||||||||||||||||||||||
| PlugIR | round3 | |||||||||||||||||||||||||||||||
| round5 | ||||||||||||||||||||||||||||||||
| DlgQuest (SFT) | round3 | |||||||||||||||||||||||||||||||
| round5 | ||||||||||||||||||||||||||||||||
| DlgQuest (SFT+GRPO) | round3 | |||||||||||||||||||||||||||||||
| round5 | 58.4 | 76.5 | 59.3 | 79.6 | 61.8 | 80.1 | 58.6 | 76.7 | 71.4 | 86.6 |
1.5mm
Overall Objective. We apply this hierarchically. The global loss (\(L_{gs}\)) uses cosine similarity between [CLS] tokens. For local losses (\(L_{ls}^{(l)}\)) from a set of intermediate layers \(P = \{p_3, p_6, p_9, p_{12}\}\), the score \(s_{i,j}^{(l)}\) is the mean similarity across all local tokens. The final training objective integrates hierarchical alignment and hard-negative isolation:
\[L_{\text{total}} = \lambda_{gs} L_{gs} + \lambda_{h} \sum_{l \in P} \left( L_{ls}^{(l)} + L_{hi}^{(l)} \right) + L_{vpr}.\]
4.3 Intelligent DQ-pilot↩︎
The DQ-pilot acts as a strategic visual reasoner, trained to formulate discriminative questions that enhance retrieval performance. Its training proceeds in two progressive stages: (1) Supervised Fine-Tuning (SFT) to establish foundational reasoning abilities, and (2) Reinforcement Learning (GRPO) to refine its question-generation strategy with task-aligned rewards.
Supervised Fine-Tuning (SFT). In the first training stage, DQ-pilot is fine-tuned on a carefully selected DQ-cities-20k subset of the DQ-Cities dataset using the standard next-token prediction objective. Each training instance
corresponds to a single dialogue turn, where the input consists of the current dialogue history \(Q_i\), the associated candidate set represented by <image> tokens, and an instruction that specifies the
Questioner’s reasoning goal and response format. The output is a structured reasoning trace followed by a well-formed discriminative question that effectively differentiates visually similar locations. Through this next-token prediction process, DQ-pilot
learns to connect accumulated dialogue context with spatial ambiguity and to formulate questions that progressively guide the retriever toward the correct place. This stage establishes the model’s foundational reasoning and dialogue abilities, providing a
solid initialization for subsequent reinforcement refinement.
Reinforcement Learning via GRPO. To further enhance strategic behavior beyond imitation, we refine the SFT-initialized model on the more challenging DQ-cities-10k subset using GRPO reinforcement learning.
Format Reward (\(R_{\text{fmt}}\)). To ensure consistent reasoning structure and interpretability, we define a binary reward verifying adherence to the required
<think></think><question></question>template: \[R_{\text{fmt}}(y) = \begin{cases} 1, & \text{if } y \text{ matches the required format},\\ 0, & \text{otherwise.} \end{cases}\]Retrieval Reward (\(R_{\text{prg}}\)). We reuse the Positional Retrieval Gain (PRG) from Sec [sec:ddi95sampling] as a task-aligned measure of how effectively a generated question improves localization. Given the retriever’s updated ranks at round \(t\), the retrieval reward is defined as \[R_{\text{prg}} = \mathrm{PRG}_t,\] which directly quantifies retrieval improvement induced by the model’s question.
Final Objective. The scalar reward used for GRPO optimization is a weighted combination of these two components: \[R = \alpha\,R_{\text{prg}} + \beta\,R_{\text{fmt}},\] where \(\alpha, \beta > 0\) balance task performance and structural consistency. This reinforcement phase encourages DQ-Pilot to move beyond supervised imitation—learning to generate concise, discriminative, and retrieval-effective questions that actively steer the reasoning process within DlgPR.
| Method | |||||||||||||||||||||||||||||||||||||||||||||
| R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | |||||||||||||||||||||||||||||||
| CLIP [47] | |||||||||||||||||||||||||||||||||||||||||||||
| Long-CLIP [48] | |||||||||||||||||||||||||||||||||||||||||||||
| FG-CLIP [43] | |||||||||||||||||||||||||||||||||||||||||||||
| Flair [49] | |||||||||||||||||||||||||||||||||||||||||||||
| CMPL(Ours) | 71.9 | 88.3 | 92.9 | 69.3 | 89.3 | 93.4 | 69.5 | 87.3 | 92.4 | 72.5 | 90.2 | 94.8 | 82.5 | 95.1 | 97.4 |
3pt
| Configurations | R@1 | R@5 | R@10 |
|---|---|---|---|
| Baseline | 71.6 | 88.0 | 95.3 |
| + Token Selection | 71.9 | 88.5 | 95.9 |
| + Progressive hsdm | 72.3 | 89.0 | 96.4 |
| + HI (Hard-negative Isolation) | 72.7 | 89.5 | 97.0 |
| Full (All components) | 73.2 | 90.0 | 97.5 |
| Setting | R@1 | R@5 |
|---|---|---|
| DQ-pilot | 60.5 | 77.8 |
| w/o DDI Curriculum (Random) | 59.6 | 77.2 |
| w/o GRPO (SFT-30k) | 59.1 | 76.6 |
| w/o GRPO (SFT) | 58.1 | 75.9 |
5 Experiments↩︎
5.1 Experimental Setup↩︎
Dataset. All the experiments are conducted on our proposed DQ-cities dataset, and the evaluation is carried out for five representative cities from various continents. Each sample begins with a vague initial description (in the 0th round), and the questioner completes the retrieval through iterative dialogues. Table 1 reports results up to Round 5. Evaluation Metrics. We evaluate the performance using cumulative Recall@K up to round \(r\), where \(k \in \{1, 5\}\), as the primary evaluation metric. In addition, the BRI index [29] is introduced as an indicator to measure the efficiency of each round of questioning.
Implementation Details. Our CMPL adopts a CLIP ViT-B/16 backbone and is trained using the proposed CMPL framework with fine-grained long descriptions as input, the number of learnable queries for each layer is set to 16. To handle long texts, we apply linear interpolation to the positional embeddings of tokens that exceed the original context length in the text encoder. The DQ-pilot is based on Qwen2.5-VL-7B-Instruct, fine-tuned with LoRA for parameter-efficient adaptation training follows our curriculum learning strategy : (1) Supervised Fine-Tuning (SFT) on low-DDI samples, and (2) GRPO-based reinforcement optimization on high-DDI samples. All the experiments are conducted on two A100s.
5.2 Main Results↩︎
Interactive Reasoning Retrieval. As summarized in Table 1, our fine-tuned DQ-pilot markedly surpasses both the original Qwen2.5-VL series and prior interactive retrieval methods. Compared to its 7B backbone, our model improves R@1 by 9.2% and 13.4% after 3 and 5 dialogue rounds, respectively, and even outperforms the much larger Qwen2.5-VL-72B by 7.3%. This indicates that our progressive alignment strategy and reward-optimized training effectively boost interactive reasoning beyond mere model scaling. We further include specialized interactive retrievers, PlugIR [29] as reference baselines to compare against established dialogue-driven retrieval pipelines. Our method achieves substantial gains in both R@1 and R@5 while maintaining the lowest BRI score, demonstrating superior interaction efficiency. It’s also worth noting that the results highlight the advantage of our SFT+GRPO fine-tuning strategy: SFT provides structured reasoning alignment from supervised dialogues, while GRPO further promotes the model to achieve deeper reasoning. For more examples, please refer to the appendix.
Retriever Performance. We also evaluate the retriever under ideal conditions using complete long descriptions, representing the upper bound of static retrieval. As shown in Table 2, our retriever significantly surpasses the Clip-based models including the recent state-of-the-art fine-grained image-text alignment models, validating its strong fine-grained cross-modal alignment.
5.3 Ablation Studies↩︎
To quantify the contribution of each core component, we conducted detailed ablation experiments. For DQ-pilot, regarding the Discriminative Difficulty-Aware Sampling strategy, we conducted two additional sets of control experiments: The first group trained the SFT model using the combined dataset of the two parts, while the second group performed SFT + GRPO training with the same sample quantity using a random sampling strategy. The results in Table 4 demonstrated that compared to imitation learning (SFT), the GRPO strategy could guide the model to perform deeper reasoning, and the curriculum setting guided by DDI was reasonable. For CMPL, Table 3 reports the results of the ablation experiments. Here, Baseline indicates the use of only the \(L_{gs}\) loss. We sequentially add the vpr loss and salient feature selection to the baseline to illustrate the importance of the salient location patches in the scene localization task. Then, we apply the multi-layer progressive CMPL, local-sdm loss, and HI loss, achieving the performance, which indicates that fully exploiting the alignment between fine-grained features is necessary.
6 Conclusion↩︎
We present DlgPR, a new paradigm that transforms traditional static geo-localization into an interactive, reasoning-driven process. Built upon our large-scale benchmark DQ-Cities and a curriculum guided by the DDI, our reasoning framework featuring the intelligent questioner DQ-Pilot—learns to iteratively refine spatial understanding through dialogue. Extensive experiments demonstrate that this interactive reasoning approach significantly enhances localization robustness and efficiency, highlighting the importance of active questioning for real-world geo-localization. In future work, we plan to explore more adaptive dialogue policies, tighter retriever–questioner co-training, and real-time deployment strategies for embodied agents in open environments.