1 Introduction↩︎

Worldwide image geo-localization [1]–[4] aims to infer the geographic location of an image captured anywhere on Earth. Unlike region-specific localization, which is constrained to predefined environments [5]–[7], global geo-localization requires models to reason over diverse visual cues, cultural context, architectural styles, and geographic priors at worldwide scale. Recent progress in multimodal large language models (MLLMs) has substantially improved image understanding [8], generation [9], and cross-modal reasoning [10], [11], motivating their use for real-world geo-localization [1], [12].

Despite these advances, most existing geo-localization methods still formulate the task as a static recognition problem, where geographic attributes are inferred directly from a single image [13] (Fig. 1A) or a pre-captured panoramic observation [14] (Fig. 1B) via regression or retrieval-based mappings. Such formulations assume that sufficient localization cues are available in fixed visual inputs, leaving no mechanism for interaction or evidence refinement, and thus struggle to resolve ambiguity in visually complex scenes.

This static paradigm contrasts with human localization behavior. In the absence of maps, GPS, or external positioning systems [15], humans localize by progressively observing the surroundings, selecting informative viewpoints, integrating evidence over time, and reasoning about spatial layout and commonsense context (Fig. 1C) [16], [17]. Localization is therefore naturally sequential: turning, zooming, and revisiting landmarks can reduce uncertainty when initial evidence is insufficient [18]. This view is consistent with active-vision and active-sampling accounts in cognitive science, where gaze and attention are guided by task goals, information gain, and curiosity-driven sampling rather than by passive stimulus exposure alone [19], [20]. It is also consistent with spatial-memory evidence that humans maintain scene, location, and heading representations across viewpoint changes [21]–[23]. This discrepancy raises a central question: can MLLMs perform global geo-localization through embodied and interactive visual reasoning that better reflects human localization behavior?

To address this gap, we introduce ERGeoBench (Embodied Reasoning Geo-localization Benchmark), the first benchmark that brings real-world panoramas into embodied simulation. ERGeoBench leverages 2,207 globally distributed panoramas and transforms them into an egocentric perception setting via a controllable camera model, enabling agents to actively acquire views through yaw, pitch, and zoom actions without access to global maps or privileged viewpoints. Under this setting, geo-localization becomes a progressive reasoning process in which agents select informative viewpoints, accumulate visual evidence, and iteratively infer their location in a human-like manner. Importantly, the embodied-view setting is not a higher-cost duplicate of panorama-view evaluation: panorama-view measures passive reasoning with near-complete visual access, whereas embodied-view measures active evidence selection, cross-view memory, and action-conditioned hypothesis revision under partial observability.

ERGeoBench adopts a task- and capability-oriented evaluation design. It evaluates models under three visual information conditions—single-view, panorama-view, and embodied-view geo-localization—covering increasingly rich perception and action settings. In addition, ERGeoBench provides fine-grained evaluation across four capability dimensions: foundational perception, spatial awareness, commonsense reasoning, and geo-localization reasoning. This design supports diagnosis beyond coarse accuracy metrics and reveals strong correlations between geo-localization performance and the other three capabilities, suggesting that accurate localization emerges from integrated perception and reasoning rather than isolated pattern matching.

Using a unified embodied-agent protocol that combines egocentric observations, action history, and in-context examples, we evaluate nine leading proprietary and open-source MLLMs on ERGeoBench. Our experiments reveal three consistent findings: (1) MLLMs perform well in high-level semantic geo-localization but remain weak at fine-grained metric localization; (2) maintaining spatial consistency across actively selected views is a major bottleneck; and (3) effective embodied reasoning benefits from accumulating visual evidence, highlighting the importance of realistic egocentric perception.

In summary, our contributions are threefold:

• We propose ERGeoBench, the first benchmark to systematically evaluate embodied geo-localization reasoning from real-world panoramas, with explicit emphasis on active evidence acquisition, spatial-memory consistency, and hypothesis refinement across sequential egocentric views.

• We introduce a fine-grained, capability-oriented evaluation framework spanning three task settings and four reasoning dimensions.

• We conduct large-scale evaluations and analyses of state-of-the-art MLLMs, providing insights toward building more human-like location-aware embodied agents.

2 Related Work↩︎

Visual Geo-localization. Traditional methods primarily treat geo-localization as a retrieval or classification problem. Retrieval-based frameworks match queries against geotagged databases but scale poorly in sparse regions, while classification-based approaches are limited by coarse granularity. Recently, MLLMs [1], [2], [12] have introduced a reasoning-driven paradigm, leveraging world knowledge to infer locations from subtle visual cues (e.g., architectural styles). However, these methods remain tethered to static, passive observation, lacking the dynamic capabilities for real-world interaction.

Evaluation Benchmarks. Geo-localization evaluation has shifted from simple distance metrics to diagnostic frameworks. IMAGEO-Bench [24] and GAEA-Bench [13] systematically evaluate geospatial bias and conversational reasoning, with GAEA integrating OSM metadata for environmental grounding. Others, such as GRE Suite [1] and GLOBE [12], emphasize reasoning chain quality through reinforcement learning (e.g., GRPO). While existing benchmarks demonstrate the cue-extraction ability of MLLMs, they focus on passive single-view inference. As shown in Table 1, ERGeoBench advances across single-view, panorama-view, and embodied-view settings, enabling active perception and progressive geo-localization through controllable exploration and multi-step evidence accumulation.

Embodied Geo-localization and Our Positioning. A critical gap exists in evaluating embodied geo-localization, where human-like agents must actively explore—e.g., rotating to find signage or zooming to resolve visual ambiguity—to achieve precise localization. To bridge this, we introduce ERGeoBench, a unified benchmark for MLLMs as embodied agents. Unlike prior static benchmarks, ERGeoBench covers single-view, panorama-view, and active embodied-view settings. Crucially, we decouple evaluation into four diagnostic dimensions: Foundational Perception (FP), Spatial Awareness (SA), Common Sense (CS), and Geo-localization Reasoning (GR). As shown in Fig. 2, ERGeoBench provides a controllable and perceptually realistic platform for researching location-aware agents.

3 Problem Formulation↩︎

3.1 Problem Setting↩︎

We formulate the problem of Embodied Reasoning Geo-localization as a sequential decision-making process. Let \(\mathcal{D} = \{(\mathcal{S}_i, \mathcal{Q}_i, \mathcal{A}_i)\}_{i=1}^N\) denote a dataset of \(N\) scenarios sampled from diverse global urban environments. Each scenario \(\mathcal{S}_i\) is represented by a high-resolution \(360^\circ\) equirectangular panorama \(P_i\), which serves as the latent environment state. Given a natural language query \(\mathcal{Q}_i\), the model must generate a response \(\hat{a}_i\) that matches the ground-truth answer \(\mathcal{A}_i\) based on an observation sequence \(\Omega_i\).

We evaluate model performance across four capability dimensions \(\mathcal{C}\): (i) Foundational Perception (\(\mathcal{C}_{perc}\)), (ii) Spatial Awareness (\(\mathcal{C}_{spat}\)), (iii) Common Sense Reasoning (\(\mathcal{C}_{common}\)), and (iv) Geo-localization Reasoning (\(\mathcal{C}_{geo}\)).

3.2 Visual Information Conditions↩︎

We simulate a human-like embodied agent that interacts with the environment under an embodied multi-view setting. Unlike passive observation, the agent explores the panorama through a controllable action space defined by yaw (\(\psi\)), pitch (\(\phi\)), and zoom (\(\zeta\)): \[\mathcal{V} = \{v = (\psi, \phi, \zeta) \mid \psi \in \Psi, \phi \in \Phi, \zeta \in \mathcal{Z}\}\] where \(\Psi\) represents the horizontal gaze direction, \(\Phi\) represents the vertical inclination, and \(\mathcal{Z}\) represents discrete zoom levels. The zoom level \(\zeta\) inversely determines the field-of-view (FOV), allowing the agent to adaptively trade off between capturing global contextual cues with a wide FOV and inspecting fine-grained local details with a narrow FOV during active exploration.

For any action \(v_t \in \mathcal{V}\) selected at step \(t\), the visual observation \(I_{v_t}\) is rendered via perspective projection from the latent panorama \(P\): \[\begin{align} I_{v_t} &= \text{Proj}(P; \psi_t, \phi_t, \text{FOV}(\zeta_t)), \\ &\quad \text{with } \text{FOV}(\zeta) = \text{FOV}_{base} \cdot 2^{-\zeta} \end{align}\] Here, \(\zeta=1\) corresponds to the base FOV (e.g., \(90^{\circ}\)). The agent operates sequentially: at step \(t\), given the interaction history \(H_t = \{(v_\tau, I_{v_\tau})\}_{\tau=1}^t\), the policy \(\pi(H_t, \mathcal{Q})\) decides whether to output a final geo-localization answer \(\hat{a}_i\) or select the next viewpoint \(v_{t+1}\) to acquire additional evidence. Thus, yaw, pitch, and zoom are treated as task-conditioned sensing actions rather than image augmentations: yaw changes the sampled sector, pitch targets overhead or street-level cues, and zoom resolves visible but unreadable details. The resulting observe–evaluate–update–act loop directly tests active sampling and cross-view memory.

3.3 Evaluation Metrics↩︎

To evaluate the multifaceted capabilities of MLLMs in geo-localization, we adopt a hybrid evaluation strategy that measures both semantic classification accuracy for geographic attributes and metric localization precision for physical coordinates. To provide a unified ranking index, we introduce the Geo-localization Score (GLS), a normalized metric ranging from 0 to 100.

Formally, GLS is defined as the arithmetic mean of three component scores, representing semantic understanding, hit-rate precision, and error magnitude, respectively: \[\text{GLS} = \frac{1}{3} \left( \mathcal{S}_{sem} + \mathcal{S}_{met} + \mathcal{S}_{err} \right)\]

Each component is calculated as follows:

Semantic Alignment Score (\(\mathcal{S}_{sem}\)). This component evaluates whether the model correctly predicts hierarchical geographic labels. Let \(y_i^{(k)}\) be the ground-truth label at granularity level \(k \in \{\text{street}, \text{city}, \text{country}\}\) and \(\hat{y}_i^{(k)}\) be the predicted label parsed from the model response. The score is the average accuracy across these three levels: \[\mathcal{S}_{sem} = \frac{1}{3} \sum_{k} \left( \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(y_i^{(k)} = \hat{y}_i^{(k)}) \right) \times 100\] where \(N\) is the total number of samples and \(\mathbb{I}(\cdot)\) is the indicator function.

Metric Precision Score (\(\mathcal{S}_{met}\)). This component evaluates physical proximity. First, we define the localization error \(d(g_i, \hat{g}_i)\) between the ground-truth GPS \(g_i=(\phi_i, \lambda_i)\) and the prediction \(\hat{g}_i=(\hat{\phi}_i, \hat{\lambda}_i)\) using the Haversine formula: \[\begin{align} d(g_i, \hat{g}_i) = 2R \cdot \arcsin \Bigg[ \bigg( \sin^2\left(\frac{\Delta\phi}{2}\right) & \\ + \cos \phi_i \cos \hat{\phi}_i \sin^2\left(\frac{\Delta\lambda}{2}\right) \bigg)^{1/2} &\Bigg] \end{align}\] where \(\phi\) and \(\lambda\) denote latitude and longitude in radians, \(\Delta\phi = \hat{\phi}_i - \phi_i\), \(\Delta\lambda = \hat{\lambda}_i - \lambda_i\), and \(R \approx 6,371\) km is the Earth’s mean radius.

Based on this geodesic distance, \(\mathcal{S}_{met}\) is calculated as the average “hit rate” across five dynamic thresholds \(\mathbb{T} = \{1, 25, 200, 750, 2500\} \text{ km}\). Following the multi-threshold design in nuScenes [25], this rewards models that achieve fine-grained localization while maintaining robustness at larger scales: \[\mathcal{S}_{met} = \frac{1}{|\mathbb{T}|} \sum_{\tau \in \mathbb{T}} \left( \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}\left( d(g_i, \hat{g}_i) \le \tau \right) \right) \times 100\]

Error Penalization Score (\(\mathcal{S}_{err}\)). While \(\mathcal{S}_{met}\) measures the success rate within thresholds, \(\mathcal{S}_{err}\) evaluates the magnitude of deviations for the entire dataset. We utilize the Median Distance Error (MdDE), denoted as \(E_{med}\), instead of the Mean Error to ensure robustness against long-tail outliers (e.g., antipodal failures).

To map the unbounded error to a normalized \([0, 100]\) scale, we apply logarithmic normalization. This reflects the heavy-tailed nature of geospatial errors, penalizing short-range deviations (e.g., 1km vs. 10km) more strictly than long-range fluctuations: \[\mathcal{S}_{err} = \max\left(0, \;1 - \frac{\ln(E_{med} + 1)}{\ln(D_{max} + 1)}\right) \times 100\] where \(D_{max}\) is set to half of Earth’s circumference (20,037.5 km).

4 ERGeoBench↩︎

To instantiate the problem formulation above, we introduce ERGeoBench, a large-scale benchmark designed to evaluate MLLMs in realistic, globally diverse, and interactive environments, as shown in Fig. 2.

4.1 Task Settings Implementation↩︎

We implement three information conditions using a unified panorama database to assess varying levels of agency. The Single-view setting serves as a static baseline, strictly limiting observation to a fixed \(90^\circ\) FOV street-level perspective (Pitch \(\approx 0^\circ\)). The Panorama-view setting provides the full equirectangular projection, serving as a passive information upper bound with maximal evidence availability. In contrast, the Embodied-view setting enables active exploration via a continuous action space (Yaw, Pitch, Zoom) subject to realistic kinematic constraints: lateral rotations are restricted to \(|\Delta\psi| \geq 45^\circ\) to prevent insignificant jitter, vertical observation is bounded to \(\phi \in [-60^\circ, 60^\circ]\), and magnification is adjustable within \(\zeta \in [1\times, 5\times]\), forcing agents to deliberatively trade off broad context for fine-grained details. This design makes panorama-view and embodied-view complementary: the former tests whether a model can exploit complete visual evidence, while the latter tests whether it can obtain and integrate the right evidence through sequential actions.

4.2 Data Collection Pipeline↩︎

Our data collection pipeline consists of three stages (as shown in Fig. 3): global sampling, diversity-aware filtering, and visual projection.

4.2.0.1 Global Sampling & Filtering.

We sourced candidate locations from the top 1,000 populated cities worldwide, excluding regions with restricted Street View coverage. To ensure high information density, we focused on a 5km radius around city centers and utilized OpenStreetMap (OSM) data. We applied a weighted scoring mechanism \(P_r\) to select representative street nodes: \[P_r = w_d \cdot S_d(r) + w_l \cdot S_l(r)\] where \(S_d(r)\) prioritizes roads closer to the city center, and \(S_l(r)\) favors major arterial roads over backstreets. We set \(w_d=0.7\) and \(w_l=0.3\) to balance centrality and road significance.

4.2.0.2 High-Fidelity Projection.

To minimize edge distortion common in \(360^\circ\) imagery, we established a precise mapping model between spherical coordinates and perspective FOV images. We rendered standard views with a fixed focal length to strictly simulate human visual observation, eliminating the “fisheye” artifacts often present in raw panoramic data.

4.3 Capability-Oriented Evaluation↩︎

ERGeoBench evaluates models across four hierarchical capabilities, moving from local perception to global reasoning.

4.3.0.1 Geo-localization Reasoning.

As the integrative capability, geo-localization reasoning requires the model to synthesize foundational perception, spatial awareness, and commonsense reasoning to infer geographic attributes such as country, city, street-level location, and GPS coordinates. This moves beyond static pattern matching toward evidence-based localization.

4.3.0.2 Foundational Perception.

This dimension assesses fine-grained visual recognition via a 5-Slot Framework (Architecture, Signage, Vegetation, Terrain, Infrastructure). Task Design: We generate Multiple Choice and True/False questions targeting specific FOVs. Quality Control: We use Target View Declaration to bind each question to a visible region, then apply layered verification rather than relying on a single model as ground truth: candidate annotations are checked by an independent VLM and manually verified by the authors, while inconsistent or ambiguous samples are corrected, discarded, or regenerated. This design reduces hallucination and mitigates potential self-referential bias from model-assisted annotation.

4.3.0.3 Spatial Awareness.

This dimension evaluates egocentric geometric reasoning across 7 sub-capabilities. We first assess the model’s grasp of intrinsic geometric properties through Yaw Estimation, 2D Positioning, Depth Estimation, and Distance perception. Beyond absolute metrics, we evaluate relative spatial understanding via Left/Right and Front/Back relational reasoning tasks, which require the agent to ground objects within its egocentric frame. A critical contribution is the Cross-view Reasoning task, where the model must deduce an object’s position relative to a target view after a rotation (e.g., “If I turn 90 degrees right, where is the object?”). This explicitly tests the agent’s ability to maintain a consistent 3D mental map across viewpoint changes.

4.3.0.4 Common-sense Reasoning.

This dimension focuses on Functional Affordance Alignment. Models must identify Points of Interest (POIs) that satisfy abstract user needs (e.g., “I am thirsty” \(\to\) Convenience Store). Hierarchical Complexity: Tasks range from Single-View identification to Dual-View combination (street-level entrances + skyline signage) and Cross-View accessibility comparison. Embodied Integration: The tasks explicitly require models to infer visibility and reachability, simulating a pedestrian deciding where to go based on visual evidence. For reachability-style questions, ground truth is represented by discrete POI labels or constrained comparative answers, avoiding subjective continuous thresholds while preserving the functional reasoning signal.

5 Experiments↩︎

5.1 Experimental Setups↩︎

Model Selection. We benchmark 9 representative MLLMs to ensure a comprehensive evaluation, comprising 5 proprietary and 4 state-of-the-art (SOTA) open-source models. The proprietary models include GPT-4o [26], the Gemini series (Gemini-2.0-Flash, 2.5-Pro, and 3-Flash) [27]–[29], and Qwen-VL-Max [30]. The open-source group consists of the Qwen series (Qwen2.5-VL-7B, Qwen3-VL-8B) [31], [32] and the InternVL series (InternVL2.5-8B, InternVL3-8B) [33], [34]. All models are configured with a temperature of \(0.1\) and a maximum generation length of \(4,096\) tokens to ensure deterministic and complete responses. All settings use a unified structured prompt and JSON schema. The full prompt is provided in Appendix 8.

5.2 Benchmark Results↩︎

5.2.0.1 Overall Results.

As shown in Table 2, there is a clear performance gap between proprietary and open-source models. Proprietary models, particularly the Gemini series and GPT-4o, consistently achieve GLS scores above 60.0, whereas open-source models generally lag behind. This gap suggests that global geo-localization is knowledge-intensive and benefits from broad pretraining, robust visual perception, and instruction-following ability. Among all evaluated models, Gemini-2.5-Pro achieves the highest overall GLS across the three settings, while Gemini-3-Flash leads many fine-grained distance-threshold metrics, establishing strong baselines for embodied geo-localization agents.

5.2.0.2 Impact of Active Exploration.

Comparing performance across the three task settings reveals the efficacy of embodied reasoning. The Panorama setting, which provides an omniscient field-of-view, consistently yields higher performance than the restricted Single-view baseline, serving as an information upper bound. Notably, we observe that capable embodied agents can bridge this information gap. Top-performing models like Gemini-3-Flash achieve GLS scores in the Embodied setting that approach or even surpass their Panorama performance. This suggests that through intelligent viewpoint selection, agents can effectively gather salient visual cues to compensate for restricted initial observations. The gain is most informative at fine-grained localization thresholds: for strong proprietary models, embodied exploration improves Acc.@1km over panorama-view evaluation (Appendix 11.2), indicating that sequential evidence accumulation can help resolve street-level ambiguity even when a panorama contains the same global scene. However, this active exploration capability is non-trivial; weaker models exhibit performance degradation in the Embodied setting compared to Panorama, likely due to disorientation caused by ineffective navigation actions.

5.2.0.3 Semantic Understanding vs. Metric Precision.

A fine-grained analysis of the capability dimensions exposes a divergence between high-level semantic reasoning and low-level metric localization. While leading models achieve high accuracy in hierarchical classification (e.g., country-level inference), their performance drops sharply at fine-grained distance thresholds (e.g., 1km accuracy). This indicates that while MLLMs have mastered regional inference, precise coordinate regression remains a significant challenge. Furthermore, the substantial discrepancy between average and median errors across all models confirms the heavy-tailed nature of geospatial prediction errors, where antipodal failures are common. This validates the necessity of our GLS metric, which utilizes log-normalization to provide a robust assessment of model reliability against such outliers.

5.3 Fine-grained Capability Analysis↩︎

To identify the specific strengths and weaknesses of current MLLMs, we perform a diagnostic evaluation across three foundational dimensions: Perception, Spatial Awareness, and Common Sense (as shown in Table [tab:fine95grained]).

5.3.0.1 Foundational Perception: A Solved Problem?

The results in Table [tab:fine95grained] indicate that current MLLMs possess robust visual recognition capabilities. Most proprietary models achieve average scores above 70.0, with GPT-4o leading in complex categories such as Infrastructure (85.70) and Vegetation (80.46). Even open-source models like InternVL3 demonstrate competitive performance, particularly in Terrain classification (94.03). This suggests that the foundational “what is in the image” problem is largely solved for high-quality street views, serving as a reliable basis for downstream reasoning tasks. To check that this conclusion is not an artifact of GPT-4o-assisted annotation, Appendix 11.1 reports additional GPT-5 and o1 evaluations; GPT-5 outperforms GPT-4o across all three capability dimensions, which weakens a simple self-preference explanation.

5.3.0.2 The Spatial Awareness Bottleneck.

A sharp performance drop is observed when moving from perception to spatial reasoning. The average scores for Spatial Awareness are consistently lower than those for Perception (e.g., GPT-4o drops from 82.78 to 43.86), highlighting this dimension as a critical bottleneck. Tasks requiring precise geometric inference, such as 2D Positioning and Yaw Estimation, prove exceptionally difficult, with many models scoring below 20.0. Notably, Gemini-2.0-Flash emerges as a significant outlier, achieving the highest average spatial score of 52.55 and outperforming GPT-4o by a large margin (8.69 points). This suggests that newer, efficient architectures may have retained superior geometric grounding capabilities.

5.3.0.3 Commonsense and Functional Reasoning.

In the Common Sense dimension, which tests the ability to map visual cues to user needs (e.g., identifying points of interest), we observe distinct behavior patterns. While models generally perform well in Single-View identification, performance degrades in Cross-View settings where spatial consistency is required. Once again, Gemini-2.0-Flash demonstrates dominance, particularly in Single-View reasoning (96.31), indicating a highly effective intent-to-location inference capability. However, the general decline in Cross-View scores across all models reinforces the finding that maintaining semantic consistency across viewpoint changes remains a major challenge for embodied agents.

6 Conclusion↩︎

We introduce ERGeoBench, a comprehensive evaluation framework designed to assess MLLM-based embodied agents across three progressively interactive settings and four fine-grained capability dimensions. Through extensive experiments, we identified key challenges, including difficulties in low-level metric localization and spatial reasoning, and the struggle to maintain semantic consistency across actively selected views. By highlighting these areas for improvement, we hope ERGeoBench will inspire and guide future research toward building more human-like and location-aware embodied agents.

Limitations↩︎

A key limitation is that our evaluation is conducted in a panorama-based simulated environment rather than through physical deployment in live 3D environments. This reflects a trade-off between reproducibility, global coverage, and logistical feasibility. In particular, our current embodiment focuses on rotation, pitch, and zoom at a fixed panorama node; it does not yet evaluate translational movement between street nodes, obstacle-aware navigation, or dynamic scene interaction. While real-world testing is important for deployment, simulated benchmarks provide standardized and repeatable evaluation at global scale. Future work could incorporate continuous video streams, dynamic scenes, embodied movement over street graphs, and standardized real-world test suites to further bridge the gap between benchmark evaluation and practical navigation agents.

Acknowledgement↩︎

This work is supported by the National Key Research and Development Program of China under Grant 2024YFC3308500, Beijing Municipal Natural Science Foundation under Grant L251042, National Natural Science Foundation of China under Grant 62406036, China Postdoctoral Science Foundation under Grant 2025M781457, and also sponsored by the State Key Laboratory of Networking and Switching Technology under Grant NST20250110.

Impact Statement↩︎

This work aims to advance the evaluation of embodied geo-localization in multimodal large language models. The benchmark is intended to support safer and more reliable research on location-aware agents, but geo-localization technologies also raise privacy and misuse concerns, including unwanted inference of sensitive locations from visual content. ERGeoBench uses publicly available street-view-style panoramas and evaluates model capability rather than enabling identification of private individuals. We encourage future work to pair progress in geo-localization with privacy-preserving evaluation, misuse analysis, and safeguards against inferring sensitive personal locations.

7 Appendix Overview↩︎

This appendix provides implementation details and additional analyses that complement the main paper. It has two goals. First, it documents the unified prompt and agent protocol used for single-view, panorama-view, and embodied-view geo-localization. Second, it reports supplementary diagnostic analyses, including prompt refinement, annotation reliability, model-bias checks, panorama-versus-embodied comparison, and out-of-distribution robustness.

8 Prompt Design and Agent Protocol↩︎

8.1 Design Rationale↩︎

The prompt is designed to turn geo-localization from one-shot visual recognition into a structured, evidence-seeking reasoning process. Instead of directly asking the model to predict a location, the final prompt decomposes each step into four operations: structured observation, evidence evaluation, hypothesis update, and next-action planning. This design is motivated by four recurring failure modes observed with a minimal prompt: single-cue overcommitment, premature guessing, weak uncertainty handling, and poor continuity across sequential views.

8.2 Five-Category Observation Framework↩︎

For consistency with the benchmark definition in the main paper, we organize visual evidence using the same 5-Slot Framework: Architecture, Signage, Vegetation, Terrain, and Infrastructure. This design keeps the prompt-level observation slots consistent with the benchmark taxonomy and the evaluated categories.

8.3 Unified Mode-Specific Prompting↩︎

The same output schema is used for all three visual settings. The only prompt-level difference is a mode line that specifies whether the model can physically change the rendered view or should treat the action as an attention intention. This keeps the reasoning protocol comparable across single-view, panorama-view, and embodied-view settings.

8.4 Structured Output Schema↩︎

The model is required to produce a JSON object with four fields: structured_observation, evidence_evaluation, hypothesis_update, and next_action, as shown in Fig. 4. The structured schema supports automatic parsing while also requiring the model to expose evidence, uncertainty, and the intended verification action.

8.5 Verification-Oriented Action Protocol↩︎

The prompt constrains actions to be verification-oriented rather than arbitrary exploration. Yaw and pitch are used to search for additional evidence, while zoom is used only when a visible target is not sufficiently clear. Typical action targets include readable street signs, shop names, language scripts, lane markings, curb standards, overhead wires, streetlight structures, and roofline or skyline features. The prompt also specifies that pitch-down actions should be preferred for road-standard cues such as lane markings or crosswalks, while pitch-up actions should be used when overhead infrastructure or roof-mounted features may disambiguate the region.

At step \(t\), the model receives the current observation, a short history of recent views, and the current best hypothesis. It then predicts a new hypothesis and either stops or emits \(\Delta\psi\), \(\Delta\phi\), and a zoom level for the next view. The run terminates when the model chooses to stop or when the confidence is sufficiently high. Conceptually, this implements a closed loop: \[\label{eq:closed95loop95appendix} I_t, H_t, \hat{y}_{t-1} \rightarrow \left(o_t, e_t, \hat{y}_t, a_t\right),\tag{1}\] where \(I_t\) is the current observation, \(H_t\) is the recent view history, \(\hat{y}_{t-1}\) is the previous hypothesis, \(o_t\) is the structured observation, \(e_t\) is the evidence evaluation, \(\hat{y}_t\) is the updated location hypothesis, and \(a_t\) is the next action.

8.6 Prompt Refinement Ablation↩︎

We compare the final structured prompt against a minimal base prompt. The base prompt directly asks the model to geolocate the image and output country, city, street/area, coordinates, and confidence. The final prompt adds five-category scene decomposition, explicit evidence evaluation, step-wise hypothesis updating, and verification-oriented actions.

9 Data Construction and Annotation Quality Control↩︎

9.1 Multi-Stage Quality-Control Pipeline↩︎

ERGeoBench uses model assistance for scalable construction, but the final annotations are not produced by a single model in a one-shot manner. The construction pipeline combines manual filtering, model-assisted generation, independent consistency checking, and human verification. Incorrect, ambiguous, or visually unreliable samples are corrected, discarded, or regenerated.

9.2 Ground Truth for Functional and Spatial Tasks↩︎

For common-sense reasoning tasks, the ground truth is represented as a discrete point-of-interest or functional label, and evaluation uses fuzzy text matching to allow minor naming variations. For spatial-awareness tasks, ground truth is represented using constrained natural-language answers, including coarse depth bins and discrete comparative outcomes such as left/right, front/back, and cross-view object relations. A judge model then applies explicit matching rules. We do not define reachability using pixel-level thresholds, 3D reconstruction, or path-planning distance, because ERGeoBench is designed as a reproducible panorama-based diagnostic benchmark rather than a full physical navigation simulator.

10 Cognitive and Neurological Motivation↩︎

Embodied geo-localization is naturally an active perception problem. Humans rarely localize from a single static glance; instead, they sample the scene, attend to diagnostic cues, maintain a spatial representation, and update hypotheses as new evidence becomes available. ERGeoBench operationalizes this process through a closed-loop protocol that asks the model to observe, evaluate evidence, update its hypothesis, and plan a targeted next action. This design is consistent with research showing that eye movements in natural behavior are shaped by task goals [19], that agents actively sample information under curiosity or expected-utility considerations [20], and that navigation depends on cognitive maps, place/grid representations, head-direction signals, and scene-memory networks [17], [21]–[23].

11 Supplementary Experiments and Analyses↩︎

11.1 Additional OpenAI Model Evaluation for Bias Checking↩︎

To test whether the benchmark favors GPT-4o because GPT-4o is involved in candidate annotation, we additionally evaluate GPT-5 and o1 under the same protocol. The results do not support a GPT-4o-specific advantage: GPT-5 outperforms GPT-4o in all three geo-localization settings, while o1 also exceeds GPT-4o in single-view geo-localization.

11.2 Value of Embodied Evaluation Beyond Panorama Evaluation↩︎

Panorama-view evaluation provides near-complete visual access at once, whereas embodied-view evaluation measures active viewpoint selection, temporal evidence accumulation, and spatial consistency under sequential egocentric observations. The embodied setting is therefore not intended to replace panorama evaluation; it diagnoses additional agentic abilities. Quantitatively, several strong models match or exceed their panorama GLS in the embodied setting, and fine-grained Acc.@1km improves consistently for the models shown below.

11.3 Out-of-Distribution Wilderness Stress Test↩︎

ERGeoBench mainly targets representative urban geo-localization. To test sparse-cue robustness, we additionally evaluate a small OOD set containing 20 extreme wilderness scenes. All tested models show large GLS drops, indicating that wilderness scenes remain challenging even for strong proprietary MLLMs.

12 Additional Reproducibility Details↩︎

12.1 Rendering and Inference Configuration↩︎

All three settings are implemented with a unified script and shared structured-output schema. The single-view setting renders a fixed perspective view, the panorama-view setting sends a scaled panorama or a zoomed panorama crop, and the embodied-view setting renders sequential perspective observations from the underlying panorama. Unless otherwise specified, inference uses temperature \(0.1\) and a maximum generation length of \(4096\) tokens. Panorama images are optionally resized so that the long side does not exceed \(1800\) pixels to reduce token cost, and JPEG quality is set to \(92\) for model input.

12.2 Trajectory Logging↩︎

For embodied evaluation, each trajectory record stores the step index, yaw, pitch, zoom, structured observation, complete JSON output, and optional saved image name. The recent trajectory history is summarized and passed back into the next prompt, enabling the model to reason over visual continuity rather than treating each observation independently. The output trajectory is serialized as JSON for downstream evaluation and reproducibility.

12.3 Parsing and Invalid Outputs↩︎

Model outputs are parsed by extracting the first valid JSON object from the response. Predictions with missing location fields, placeholder coordinates, or forbidden unknown-like answers are treated as invalid under the single-pass benchmark protocol. This design intentionally evaluates not only visual geo-localization ability, but also whether a model can reliably follow a structured response protocol under multimodal reasoning constraints.

References↩︎