1 Introduction↩︎

Navigation is a fundamental prerequisite for robots to physically assist humans in real-world environments. While recent research has achieved remarkable success in visual navigation within indoor environments [1]–[13], the deployment of general-purpose mobile robots requires capabilities that extend beyond these structured confines. Real-world applications, such as last-mile delivery, search and rescue, and campus security, demand agents capable of seamless indoor-to-outdoor transition. These hybrid scenarios present a transition gap: agents need to discover exits and adapt to scale and appearance shifts while maintaining robust physical interaction with the environment.

Despite this need, existing evaluation platforms are highly limited. Indoor benchmarks cover increasingly diverse indoor scenes, including scanned scenes in MP3D [14]/HM3D [15] and procedural or artist-designed scenes in ProcTHOR [4]/RoboTHOR [16]; however, they remain indoor-only and do not evaluate connected indoor-to-outdoor transitions under continuous robot dynamics. In contrast, outdoor benchmarks typically rely on offline, non-interactive datasets (e.g., KITTI [17], nuScenes [18], TouchDown [19]). Critically, the transition zone, where agents must reason about tasks such as exiting buildings or traversing curbs, remains underexplored. Existing benchmarks rarely evaluate these transition behaviors in connected indoor-to-outdoor scenes, where agents must continue acting across the boundary under realistic robot dynamics and traversability constraints.

To bridge this gap, we introduce NavVerse, a unified benchmark designed specifically for indoor-to-outdoor navigation in continuous robot simulation. Unlike prior works that decouple visual planning from control, NavVerse builds a physics-enabled simulation benchmark that unifies multiple action interfaces, supports different robot embodiments, and provides safety-related diagnostics for full-stack navigation evaluation. This enables the evaluation of agents not just on path efficiency, but also on physical viability, failure modes, and safety-related metrics across varying terrain conditions.

NavVerse systematizes evaluation across three environment types (indoor, outdoor, and indoor-to-outdoor) and three tasks: Object Navigation (ObjNav), Vision-and-Language Navigation (VLN), and novel Place Navigation (PlaceNav). Most notably, we introduce PlaceNav as a long-horizon navigation to place-level goals (e.g., locating a nearby restaurant). This novel yet naturalistic challenge bridges the gap between room-scale and city-scale navigation, requiring agents to transition from indoor spaces to external targets, a routine real-world capability unsupported by existing benchmarks. Solving PlaceNav requires understanding place-level semantics, grounding urban topology priors, and conducting long-horizon target search under kinodynamic and safety constraints. Evaluation metrics cover task success, path efficiency, and safety-aware execution.

We conducted extensive evaluations of four representative navigation baselines on NavVerse: an end-to-end visual-language-action (VLA) model, a transformer-based RL policy, a VLA-RL model, and a modular method. Our results indicate: i) Cross-context adaptation remains difficult: the end-to-end model performs best among zero-shot baselines, while all evaluated baselines struggle in hybrid indoor-to-outdoor scenarios that require exit finding, boundary crossing, and post-transition adaptation; and ii) A realistic navigation benchmark must account for kinodynamic constraints and robot dynamics: indoor benchmarks can produce unrealistic behaviors such as sliding against obstacles, while outdoor benchmarks often ignore terrain topology such as curbs and steps and assume perfect action execution. These assumptions can lead deployed policies to collide with obstacles, fall, or choose unsafe terrain. In contrast, NavVerse uses physics simulation to evaluate realistic action outcomes and safety metrics to quantify safety-aware behavior.

To sum up, our contributions are threefold: i) Physics-enabled indoor-to-outdoor benchmark: We integrate residential, commercial, outdoor, and connected hybrid scenes in continuous robot simulation, with scalable scene/task generation, verified episodes, and executable robot evaluation; ii) Rich task spectrum: We introduce tasks ranging from standard navigation to the novel, long-horizon PlaceNav. These are assessed via task success, path efficiency, and safety-aware execution metrics that expose physical instability and transition failures; and iii) Navigation baseline findings: We evaluate classical modular, RL, and VLA methods, revealing that representative baselines show some zero-shot transfer to unseen tasks but still struggle with indoor-to-outdoor transitions and safety-aware decision making.

2 Related Work↩︎

Simulation platform. Existing simulators generally prioritize either visual or physics fidelity. Visual-fidelity platforms: Platforms like Habitat-Sim [24], [29], [30] (utilizing scanned environments like Matterport3D [14] and HM3D [15]) offer high-throughput egocentric rendering for vision-based navigation, while AI2-THOR [23] supports interactive indoor scenes. However, benchmarks on these platforms often simplify physical execution using discrete actions or lightweight dynamics. Physics-fidelity platforms: Frameworks like MuJoCo [31] and Gazebo [32] excel at continuous control and contact dynamics but lack visually rich, large-scale navigation environments. To bridge this gap, Isaac Sim [28] serves as a powerful backend by combining GPU-accelerated visual rendering with articulated physics and continuous control. Built upon Isaac Sim, our NavVerse delivers both visual and physics fidelity, offering a comprehensive embodied navigation platform that unifies photorealistic scenes with executable robot motion, terrain annotations, and diagnostic metrics.

Indoor and outdoor navigation benchmarks. Most existing navigation benchmarks study indoor and outdoor navigation as separate settings, so the transition between the two remains less explored. Indoor benchmarks have mainly studied Object Navigation (ObjNav) [4], [14]–[16], [24], [29], [30], [33], and Vision-and-Language Navigation (VLN) [21], [22], [26], [27], [34]–[39]. These benchmarks have provided strong testbeds for indoor semantic understanding and instruction following, but they are typically centered on indoor scenes rather than connected transitions to outdoor space. Outdoor benchmarks often rely on street-view imagery and panoramic observations [19], [20], or pre-recorded trajectories [3], [40], [41]. Virtual Community [42] moves toward simulated outdoor environments using Google 3D Tiles and street-view imagery. These benchmarks extend navigation evaluation to larger-scale environments, but many of them are built around panoramic observations and non-interactive navigation interfaces, which makes it difficult to study continuous robot execution, safety, and embodiment-dependent traversability. To bridge these gaps, NavVerse connects diverse interactive scenes (indoor, outdoor and indoor-to-outdoor) with continuous robot-execution protocol, offering a comprehensive embodied navigation simulation platform, see Table 1. It pioneers a new research direction centered on the practical challenges of indoor-to-outdoor transitions.

3 NavVerse: Indoor-to-Outdoor Navigation Benchmark↩︎

In this section, we introduce NavVerse, a comprehensive embodied navigation benchmark, spanning indoor, outdoor and indoor-to-outdoor scenarios. The benchmark construction overview is shown in Fig. 2, including scene generation, task definition and evaluation. Note that NavVerse is built upon Isaac Sim, offering both photorealistic scenes and physics-aware robot execution.

3.1 Scene Generation↩︎

NavVerse covers indoor, outdoor, and indoor-to-outdoor scenes. To ensure a robust evaluation of generalization capabilities, the training, and evaluation splits feature mutually exclusive scenes. We defer implementation details for asset generation, mesh processing, object placement, storefront construction, terrain modeling, and indoor-to-outdoor alignment to Appendix 7.

Indoor Scenes. Our indoor scenes are built uponGRScenes meshes from GRUtopia [43]. To promote scene diversity, we curate 100 scenes covering apartments and commercial spaces such as groceries, hospitals, and offices. These indoor scenes feature diverse object categories and cluttered layouts, demanding both semantic reasoning and physics-aware collision avoidance capabilities.

Outdoor Scenes. Our outdoor scenes are built on 50 meshes from Virtual Community [42]. We enhance these meshes across several key dimensions, transforming sparse urban geometry into semantically rich, physics-backed, and interactive environments.

\(\bullet\) Real-to-sim urban layouts and physics-aware terrains. The scenes preserve real-world spatial layouts and urban scale through Google 3D Tiles geometry and map-aligned urban structure. Furthermore, to enable kinodynamic constraints and surface-dependent traversability, we segment terrain into map-derived material classes and refine the geometry of roads, sidewalks, and open areas with realistic elevation changes. This allows our benchmark to evaluate true trajectory feasibility rather than simple semantic goal recognition or topological routing.

\(\bullet\) Rich object and photorealistic storefronts population. We populate the layout lanes and sidewalks with semantically reasonable objects sampled from 121 Objaverse [44] object categories, supporting up to 200 cars and 1,000 sidewalk objects per scene. To support PlaceNav and language-guided tasks, we also generate and place 101 diverse storefront assets (restaurants, cafes, banks) at map-aligned building frontages.

Indoor-to-Outdoor Scenes. To evaluate connected cross-context navigation, we integrate indoor scenes into buildings in the outdoor world, resulting in 50 indoor-to-outdoor scenes.

\(\bullet\) Door-to-facade scene assembly. We construct indoor-to-outdoor scenes with two key steps: i) identify a road-facing building and create a facade opening as the exit; and ii) place a compatible indoor layout behind the building. To align the indoor entrance with the opening, we need to adjust the entrance height, and remove blocking door or wall meshes to connect the spaces.

\(\bullet\) Consistent physical execution. The connected indoor and outdoor components share the same robot-execution protocol, physics setup, sensing, and control stack across the full episode.

These indoor-to-outdoor navigation scenes task agents with traversing corridors, identifying exits, and crossing into open streets toward outdoor goals. This introduces unprecedented challenges for embodied navigation, requiring rapid search adaptation amid drastic shifts in lighting, appearance, environmental structure, and spatial scale.

3.2 Task Definition↩︎

NavVerse provides comprehensive evaluation across three task types under three scene types.

Object Navigation. Object navigation (ObjNav) requires the agent to locate and approach specific object categories within the environment. While indoor scenes focus on fine-grained object recognition, outdoor scenes challenge the agent to identify landmarks over much larger distances.

Place Navigation. Place navigation (PlaceNav) requires agents to reach functional destinations, such as restaurants, cafes, or banks, rather than localized visual objects. Embedded within the urban layout, PlaceNav challenges agents to reason over road topology, building frontages, storefront semantics, and long-range visibility cues to locate the target destination.

Vision-and-Language Navigation. In Vision-and-Language Navigation (VLN), the agent is tasked to follow natural language instructions to reach a goal. This evaluates the agent’s ability to ground linguistic cues in visual observations and execute complex, multi-step trajectories.

3.3 Benchmark and Evaluation↩︎

Episode generation. NavVerse generates episodes with three steps: i) offline sampling of start-to-goal paths on the navigation mesh (NavMesh); ii) online filtering via simulated robot execution to discard infeasible paths and record successful sensor observations and poses; and iii) human validation utilizing VLMs for VLN instruction generation and a web interface for manual quality control. This process guarantees high-quality, executable episodes, culminating in a benchmark of 4,027 ObjNav episodes, 2,973 PlaceNav episodes, and 3,000 VLN episodes. Detailed episode numbers by scene type and task are provided in Table 6.

Evaluation setup. We evaluate all models on the evaluation split, reporting both full task-level summaries and domain-specific diagnostic subsets for ObjNav and PlaceNav. Experiments utilize a simulated Boston Dynamics Spot robot with an RGB-D sensor. To preserve each baseline’s native structure, model outputs are converted to the NavVerse interface and executed using a shared waypoint follower managed by a PID controller. Detailed observation, action, control, and episode specifications are provided in Appendix 6.1–6.4.

Evaluation metrics. We report metrics along three axes: task completion, exploration efficiency, and safety. Task completion includes Success Rate (SR) and Success weighted by Path Length (SPL). Exploration efficiency includes Coverage Efficiency (CE), which measures the amount of newly covered space per unit travel distance and is especially important for large outdoor and indoor-to-outdoor scenes. Safety metrics include Collision Rate (CR), Average Distance to Obstacles (ADO), and Navigable Surface Ratio (NSR): CR quantifies collision with obstacles, ADO measures clearance from nearby obstacles, and NSR measures whether the robot remains on valid or kinodynamically preferred navigable surfaces. Success uses a geometric goal tolerance of \(1.0+r_{\mathrm{robot}}\) meters, which is \(1.6\) m for the Spot robot used in our experiments. Episodes terminate on success, timeout, fall, or stop at wrong goal. Formal metric definitions are provided in Appendix 6.5.

4 Experiments↩︎

4.1 Baselines↩︎

We evaluate four representative baselines. SGImagineNav [11] represents modular map-based semantic navigation; we connect its planned waypoints to the NavVerse controller. PoliFormer [8] is an indoor RL policy and tests transfer from indoor training to heterogeneous indoor/outdoor execution. UniNaVid [2] is a VLA navigation model evaluated across object-, place-, and language-conditioned tasks. LongNav-R1 [45] is an end-to-end VLA navigation policy optimized via RL within HM3D ObjNav [25]. All methods are evaluated under the same physics-enabled execution protocol. SGImagineNav directly uses the NavVerse waypoint interface, while PoliFormer, UniNaVid, and LongNav-R1 output discrete actions that are converted into single-step waypoints and executed by our shared waypoint follower. For VLN, we report UniNaVid as the only baseline among the evaluated methods that natively accepts route-style language instructions.

4.2 Multiple Navigation Task Evaluation↩︎

To provide a comprehensive evaluation of existing methods, we report success rate and path efficiency in Table ¿tbl:tab:result95task95scene?, and fine-grained safety and efficiency metrics in Table ¿tbl:tab:allscene95metrics95main?.

3pt max width=0.9

UniNaVid (VLA) achieves the best zero-shot task-completion performance. Table ¿tbl:tab:result95task95scene? shows that UniNaVid achieves the strongest task-completion performance among comparable zero-shot baselines on both ObjNav and PlaceNav. However, the low absolute SR across all tasks, including 10.67% SR on UniNaVid VLN, shows that physically grounded navigation remains far from solved.

Existing methods demonstrate divergent success, efficiency, and safety behaviors. Tables ¿tbl:tab:result95task95scene? and ¿tbl:tab:allscene95metrics95main? reveal that existing methods excel only in orthogonal dimensions: the VLA model UniNaVid leads in task success and search efficiency (SR, SPL, CE), LongNav-R1 ensures safety via obstacle clearance (high ADO), and SGImagineNav optimizes local interaction safety (low CR, high NSR). This divergence underscores that current baselines remain insufficient for holistic deployment. Crucially, NavVerse isolates these architectural blind spots, motivating the development of next-generation models capable of simultaneously balancing success, safety, and efficiency, all of which are non-negotiable requirements for real-world navigation.

PlaceNav presents more pronounced indoor-to-outdoor transition challenges than VLN and ObjNav. As shown in Table ¿tbl:tab:result95task95scene?, the performance of the leading model (UniNaVid) degrades sharply when transitioning to indoor-to-outdoor episodes. Success rates drop by 8.91% for ObjNav (17.33% to 8.42%) and by 6.67% for VLN (14.00% to 7.33%). The degradation is most severe in PlaceNav, where performance plummets by an absolute 14.01%, dropping from 17.65% in pure outdoor scenes to a mere 3.64% on indoor-to-outdoor episodes. The reasons are twofold: i) VLN primarily requires instruction-following capabilities that translate effectively across domains, resulting in the most stable performance across environments; ii) conversely, PlaceNav necessitates topology-aware point-of-interest searching, which diverges sharply between indoor and outdoor layouts, imposing much steeper adaptation bottlenecks than ObjNav.

4.3 Indoor-to-Outdoor Transition Gap↩︎

To understand the challenges of indoor-to-outdoor transitions, we conduct a detailed failure mode analysis in Tables [tbl:tab:stagewise95transition95success] and ¿tbl:tab:termination95reasons?. This evaluation pinpoints failures across the complete navigation sequence, which requires an agent to start indoors, reason over structural transitions, locate an exit, cross the boundary, and execute an outdoor goal-conditioned search.

2.0pt max width=

Indoor and outdoor scenes stress distinct failure modes, shifting from local execution challenges indoors to exploration and goal-grounding limits outdoors. Table ¿tbl:tab:termination95reasons? indicates that indoor ObjNav failures are dominated by local execution constraints in confined spaces: falls cause 31.3% of PoliFormer and 43.0% of SGImagineNav failures. Outdoors, the bottleneck shifts to exploration and grounding. PoliFormer times out in 95.1% of failed outdoor ObjNav and 84.8% of PlaceNav episodes, while LongNav-R1 mostly fails due to incorrect goals (95.0%/87.1% on ObjNav/PlaceNav). And 82.9% of UniNaVid outdoor VLN failures are wrong-goal issues. Post-exit metrics in Table [tbl:tab:stagewise95transition95success] mirror these trends, confirming that these vulnerabilities persist after transitioning outdoors.

Indoor-to-outdoor episodes introduce an exit-finding bottleneck. Table [tbl:tab:stagewise95transition95success] shows that many indoor-to-outdoor episodes fail before the agent reaches the outdoor side. On ObjNav, UniNaVid and LongNav-R1 are Not Reached Outdoor in 48.42% and 47.37% of episodes, respectively. Even SGImagineNav, which achieves the highest reached-outdoor rate, still fails to reach outdoor space in 25.26% of ObjNav episodes and 30.91% of PlaceNav episodes. For VLN, UniNaVid is also Not Reached Outdoor in 37.33% of episodes. This failure mode is invisible in pure indoor or pure outdoor evaluation, showing the importance of indoor-to-outdoor scenes.

Reaching outdoor space causes a systematic efficiency drop. Table [tbl:tab:stagewise95transition95success] further shows that all methods have lower coverage efficiency after the exit step than before it. For example, UniNaVid drops from 1.07 to 0.84 on ObjNav, from 1.26 to 0.81 on PlaceNav, and from 1.14 to 0.80 on VLN; the same pre-to-post exit CE drop appears for the other ObjNav and PlaceNav methods. We observe that after exiting the indoor area, agents often exhibit inefficient behaviors such as persistent in-place rotation and back-and-forth motion. This suggests limited adaptation to changes in scene scale and topology: although outdoor regions provide more open space to explore, agents fail to adjust their exploration strategy and cover the space less efficiently after reaching outdoor space.

Post-exit failures are method-specific. Among outdoor-reached failures, different methods fail for different reasons. PoliFormer and SGImagineNav are mainly limited by timeouts after reaching outdoor space. LongNav-R1 instead mostly fails through wrong-goal termination, with wrong-goal failures accounting for 74.5% on ObjNav and 72.2% on PlaceNav, suggesting that it often calls STOP without sufficient evidence that it has reached the correct goal. UniNaVid shows a mixed pattern: wrong-goal failures are prominent, especially on VLN, but timeout remains substantial on ObjNav and PlaceNav. These results indicate that reaching outdoor space is not sufficient; agents also need to adapt their exploration strategy and maintain reliable goal grounding after the exit step.

4.4 Diagnostic Analyses↩︎

Longer routes expose compounding errors. Table ¿tbl:tab:result95task95difficulty? groups episodes into task-specific path-length tertiles, yielding balanced Easy, Medium, and Hard splits within each task. Across ObjNav, PlaceNav, and VLN, the Hard split consistently reduces success for the strongest baselines compared with the easier splits. This suggests that long-horizon navigation in NavVerse is limited not only by goal recognition, but also by compounding errors in exploration, memory, terrain selection, instruction grounding, and closed-loop control. Difficulty thresholds are computed separately within each task using the \(1/3\) and \(2/3\) quantiles of path length. For ObjNav, the thresholds are 8.07 m and 28.28 m; for PlaceNav, 24.66 m and 39.21 m; and for VLN, 11.61 m and 32.61 m.

Embodiment dynamics matter even for oracle paths. Table [tbl:tab:robot95performance95main] provides a controlled oracle-trajectory analysis over embodiment and friction. The legged robot completes the tested oracle trajectories under both friction settings, while the wheeled robot has substantially lower SR and velocity. This diagnostic supports the need for physics-enabled evaluation: a navigation benchmark that ignores embodiment dynamics can overestimate kinodynamic executability under realistic robot embodiments.

Instruction type affects efficiency and semantic grounding. Table [tab:instruction95type95placenav] compares UniNaVid on the same PlaceNav episodes under three instruction formats. Route-style VLN instructions modestly improve efficiency over Store Category prompts, increasing overall SPL from 3.30% to 4.84%, mainly through outdoor episodes. However, they do not improve indoor-to-outdoor SR, which remains 3.64%, suggesting that route descriptions alone do not solve exit finding or post-transition adaptation. Intention Driven instructions introduce an additional intent-to-place grounding step, reducing overall SR to 4.07% and indoor-to-outdoor SR to 0.00%.

5 Conclusion↩︎

We introduced NavVerse, a physics-enabled navigation benchmark that unifies indoor, outdoor, and indoor-to-outdoor evaluation under a consistent protocol. NavVerse supports three complementary tasks, Object Navigation, Place Navigation, and Vision-and-Language Navigation, to stress goal conditioned reaching, long horizon semantic search, and instruction grounding and following. Experiments across RL, end-to-end VLA, and modular baselines reveal a clear indoor-to-outdoor transition gap, motivating methods that are robust to perception shifts, terrain variability, spatial re-anchoring, and long-horizon semantics in cross-context navigation settings.

Limitations and future work. NavVerse has several limitations that suggest concrete next steps. First, most environments are currently single floor meshes. Future direction should expand to multi floor layouts and overpass or skybridge structures to better reflect realistic urban navigation and vertical transitions. Second, the benchmark currently contains limited dynamics and interaction. We aim to introduce more dynamic agents such as pedestrians and vehicles, and more interactive objects such as doors and movable obstacles, to enable evaluation of socially aware and interaction aware navigation. Finally, outdoor-to-indoor transition is not explored in this work, we leave this setting to future extensions.

6 Benchmark Interface and Evaluation Protocol↩︎

6.1 Observations↩︎

At each decision step, the agent receives onboard sensing and proprioception:

• RGB-D. We provide a forward-facing RGB image and a depth image rendered at \(640\times480\) and \(10\) FPS. The depth range is \([0.1, 1000]\) meters. We do not add an additional depth noise model.

• Robot state and geometry. We provide the ground-truth robot base pose, the camera-to-body transforms, camera intrinsics, and robot size. We do not provide IMU measurements.

RGB, depth, and pose are published together every time an image is rendered. Most experiments in this paper use the Boston Dynamics Spot robot as the primary embodiment. We also instantiate a wheeled Turtlebot platform and compare its executable navigation performance against Spot in Table [tbl:tab:robot95performance95main], highlighting how embodiment and contact dynamics affect trajectory execution. The benchmark interface is designed to support additional platforms, including quadrupeds, humanoids, and bipedal robots, but these additional embodiments are not experimentally evaluated in this paper.

6.2 Actions↩︎

NavVerse supports three action types:

• Waypoint path. A list of waypoints in the world frame, \(W=\{(x_i,y_i,\psi_i)\}_{i=1}^N\), where \(\psi\) is yaw.

• Discrete actions. A discrete primitive action space that approximates waypoint control using fixed step-size motions. Each primitive is converted into a single-step waypoint in the world frame defined in Table 2.

• Velocity commands. A body-frame velocity \((v_x, v_y, \omega)\). This action type can directly drive the locomotion policy and bypass the PID-based controller.

3pt

6.3 Action-to-control bridging↩︎

Waypoints are executed by a shared low-level stack, which runs at \(50\) Hz:

• Path tracking. Given a waypoint path \(\mathbf{W}\) in the world frame, we track the path with lookahead distance \(d_{\mathrm{look}}=0.3\) m. A waypoint is considered reached when the position error is below \(0.03\) m and the yaw error is below \(3^\circ\).

• PID-based controller. The controller computes body-frame velocity \((v_x, v_y, \omega)\) from the tracking error between the current robot pose and the next lookahead target. We clip it to \(|v_x|\le 1.5\) m/s, \(|v_y|\le 0.3\) m/s, and \(|\omega|\le 1.5\) rad/s.

• Locomotion policy. The clipped velocity is passed to a learned locomotion policy with the same structure as the default Isaac Lab locomotion policy. We train this policy on flat ground, rough ground, and stairs to support diverse terrain execution.

• Action execution One discrete action corresponds to approximately 1 s of simulated time, as actions are executed by the locomotion/controller stack until the commanded motion primitive completes. The agent can also output waypoint targets, which can be streamed to the controller without blocking on the completion of the previous waypoint execution. This enables non-blocking control and better overlaps perception and planning with low-level execution.

6.4 Episodes↩︎

An episode starts from an initial robot pose and proceeds step-by-step as the agent outputs actions, until it either succeeds or meets a termination condition. For ObjNav and PlaceNav, the goal is a category that may have multiple instances in the scene; reaching any valid instance counts as success. For VLN, the agent is provided a natural-language instruction, and success is evaluated using the same geometric goal criterion as ObjNav and PlaceNav. In addition to the agent observation, the simulator can output privileged ground-truth signals for logging and training (e.g., SR/SPL, distance to goal, BEV images, goal positions and sizes, and reference trajectories); these signals are not required by the benchmark action interface and are not used by the evaluated policies unless explicitly stated.

6.5 Metrics↩︎

NavVerse provides the following metrics: Distance to Goal (DTG), Success Rate (SR), Success weighted by Path Length (SPL), SoftSPL, Navigation Error (NE), and Oracle Success (OS).

Let \(P\) be the executed trajectory length accumulated in 3D from the robot base \((x,y,z)\) trajectory. Let \(L\) be the shortest path length computed as the length of the path between the start and the goal. Let \(\mathbf{p}_t\) be the robot base position at time step \(t\), and let \(T\) be the termination time step.

• Distance-to-goal. The distance-to-goal at time step \(t\) is \[d_t = \max\!\left(0,\, \left\lVert \mathbf{p}_t - \mathbf{p}_{\mathrm{goal}}\right\rVert_2 - r_{\mathrm{goal}}\right),\] where \(\mathbf{p}_{\mathrm{goal}}\) is the goal center position and \(r_{\mathrm{goal}}=\tfrac{1}{2}\max(x_{\mathrm{size}}, y_{\mathrm{size}})\) is computed from the goal mesh size in the \(x\)-\(y\) plane.

• Success. An episode is successful if the final distance-to-goal satisfies \[d_T \le 1.0 + r_{\mathrm{robot}}.\] where \(r_{\mathrm{robot}}\) is the radius of the robot. We use \(r_{\mathrm{robot}}=0.6\) m for Boston Dynamics Spot.

• Success Rate (SR). \[\mathrm{SR} = \frac{1}{N}\sum_{k=1}^{N}\mathbb{I}\!\left[d_{T}^{(k)} \le 1.0 + r_{\mathrm{robot}}\right].\]

• Success weighted by Path Length (SPL). \[\mathrm{SPL} = \frac{1}{N}\sum_{k=1}^{N}\mathbb{I}\!\left[d_{T}^{(k)} \le 1.0 + r_{\mathrm{robot}}\right]\cdot \frac{L^{(k)}}{\max(L^{(k)}, P^{(k)})}.\]

• Soft Success (SS). We use the soft success score \(S_p\) to measure partial progress toward the goal: \[S_p = \max\left(0,\, 1 - \frac{d_T}{d_0}\right),\] where \(d_T\) is the final distance to the goal and \(d_0\) is the initial distance. We report \[\mathrm{SS} = \frac{1}{N}\sum_{k=1}^{N} S_p^{(k)}.\]

• SoftSPL. We define \[\mathrm{SoftSPL}^{(k)} = S_p^{(k)} \frac{L^{(k)}}{\max(L^{(k)},P^{(k)})},\] \[\mathrm{SoftSPL} = \frac{1}{N}\sum_{k=1}^{N}\mathrm{SoftSPL}^{(k)}.\] Here \(d_T\) is the distance between the agent’s final position and the goal, and \(d_0\) is the distance between the start position and the goal.

• Navigation Error (NE). Navigation error is the distance-to-goal at episode termination: \[\mathrm{NE} = \frac{1}{N} \sum_{k=1}^{N} d_T^{(k)}.\]

• Oracle Success (OS). Oracle success indicates whether the agent reaches the goal tolerance at any time: \[\mathrm{OS} = \frac{1}{N} \sum_{k=1}^{N} \mathbb{I} \left[ \min_{t\in[0,T_k]} d_t^{(k)} \le 1.0+r_{\mathrm{robot}} \right].\]

• Coverage Efficiency (CE). Let \(\mathcal{G}(\tau)\) be the set of unique \(1\) m-resolution occupancy-grid cells visited by the executed trajectory \(\tau\), and let \(P\) be the executed path length. Coverage efficiency measures newly explored space per unit travel cost: \[\mathrm{CE} = \frac{|\mathcal{G}(\tau)|}{\max(P,\varepsilon)}.\] This metric captures whether an agent explores large open spaces efficiently rather than repeatedly revisiting the same local region.

• Collision Rate (CR). Let \(c_t \in \{0,1\}\) indicate whether the robot has a non-ground collision with obstacle geometry at time step \(t\). We report the trajectory-level collision ratio: \[\mathrm{CR} = \frac{1}{N}\sum_{k=1}^{N}\mathbb{I}\!\left[\exists t,\; c_t^{(k)}=1\right].\] Lower CR indicates safer local motion and better obstacle avoidance.

• Average Distance to Obstacles (ADO). Let \(d^{\mathrm{obs}}_t\) be the distance from the robot base to the nearest obstacle or non-traversable geometry at time step \(t\). We define \[\mathrm{ADO} = \frac{1}{T}\sum_{t=1}^{T} d^{\mathrm{obs}}_t.\] Higher ADO indicates a larger clearance margin during navigation.

• Navigable Surface Ratio (NSR). Let \(n_t \in \{0,1\}\) indicate whether the robot base lies on a valid or kinodynamically preferred navigable surface at time step \(t\) according to the scene’s terrain annotations. We define \[\mathrm{NSR} = \frac{1}{T}\sum_{t=1}^{T} n_t.\] Higher NSR indicates that the robot more consistently stays on surfaces suitable for safe traversal.

6.6 Termination labels↩︎

An episode terminates when one of the following conditions is met:

• Perception error. The agent calls stop but the goal tolerance is not satisfied, i.e., stop is issued and \(d > 1.0 + r_{\mathrm{robot}}\) (equivalently \(d>1.6\) m for Spot).

• Fall. The angle between the projected gravity vector in the robot frame and the world \(z\)-axis exceeds \(45^\circ\).

• Timeout. Simulation time exceeds \(300\) seconds.

6.7 System Setup and Runtime↩︎

• Simulation stack. We use Isaac Sim 5.1.0 with Isaac Lab 2.3.0.

• Hardware. We use a single RTX 4090 for development and smoke tests, and four NVIDIA L40S GPUs for large-scale evaluation.

• Physics timestep and rendering. The simulator advances with a fixed physics step of 0.005 s (200 Hz). The rendering throughput is approximately 20 FPS on an RTX 4090.

• Parallelization. We support parallel evaluation across multiple simulator instances. In our setup, a single NVIDIA L40s (48G VRAM) can run up to three simulations concurrently.

• Memory footprint. Peak GPU memory usage is around 9 GB for indoor scenes and 13 GB for outdoor and indoor-to-outdoor scenes under the default observation configuration.

7 Scene Construction↩︎

This section provides the implementation details behind the scene-generation summary in Section 3.1.

7.1 Scene Sources and Inventory↩︎

• Indoor scenes. We use 100 indoor scenes from GRScenes in GRUtopia [43], covering apartments, hospitals, cafes, supermarkets, offices, and other everyday environments. The split uses 90 indoor scenes for training and 10 for evaluation.

• Outdoor scenes. We select 50 cities from the Virtual Community dataset [42] to construct our benchmark, with 40 scenes for training and 10 for evaluation. Each scene covers a 1 km\(\times\)​1 km area. Terrain across outdoor scenes is segmented into five classes: buildings, sidewalks, roadways, water bodies, and green areas. On average, each city contains 546 buildings, 36.4 km of road network, and 199,530 m\(^2\) of navigable area.

• Indoor-to-outdoor scenes. The benchmark includes 50 indoor-to-outdoor scenes created by embedding indoor layouts into outdoor cities through door-to-facade connections. Indoor-to-outdoor scenes follow the same scene-disjoint split protocol as the outdoor set.

• Object and storefront inventory. We curate 121 Objaverse [44] object categories suitable for urban environments, such as vehicles, chairs, and trash cans. Outdoor scenes populate lanes with up to 200 cars and sidewalks with up to 1,000 objects per scene. Indoor-to-outdoor scenes sample objects and cars within a 50 m radius of the indoor anchor to control episode difficulty. We additionally add 30 contextual-object groups and 101 generated storefront textures, as detailed in Sections 7.4 and 7.5.

• BEV map examples. Figure 3 shows representative BEV maps for indoor, outdoor, and indoor-to-outdoor scenes. Indoor scenes cover diverse layout types and contain rich object-category compositions. Outdoor scenes span large urban areas, with object instances and storefront goals distributed according to the underlying map structure. In indoor-to-outdoor scenes, many objects and storefronts are placed around the indoor-area anchor, increasing local goal density near the indoor region and improving task diversity for connected episodes.

7.2 Indoor Scene Processing↩︎

• Geometry preprocessing. We develop an automated mesh-optimization pipeline to reduce scene complexity and accelerate simulation and navigation. The pipeline applies three geometry operations in Blender: decimation to reduce face count, dissolve to remove redundant edges and vertices, and remeshing to regularize the mesh while preserving the overall structure. The optimized meshes keep the original textures and materials. Table 3 summarizes the reduction in mesh complexity. This optimization substantially speeds up NavMesh construction, reducing build time from 705 ms to 107 ms, and also reduces scene loading time.

  Table 3: Average face count before and after decimation.

  Scene Type	Raw Faces	Decimated Faces	Decimation (%)
  GRScenes	21,774,867	2,197,731	89.91

7.3 Outdoor Terrain and Road Modeling↩︎

• Terrain generation. We start from the imported city terrain mesh and densify it to provide sufficient geometric resolution for road modeling and ground-level execution. This is done through a lightweight cleanup and subdivision procedure that increases vertex density while preserving the overall terrain shape.

• Road depression. We extract vehicle road centerlines and widths from OSM, using tagged widths when available and otherwise deriving widths from lane count and road type. We convert these road attributes into road footprint polygons. We imprint the road footprints onto the terrain mesh and apply a uniform depression depth of \(0.2\) m inside road regions. This produces physically traversable roadbeds that align with the map geometry.

• Surface semantics and materials. We label terrain regions into classes such as sidewalks, roadways, water bodies, and green areas. We assign class-specific appearance and physics materials so that policies experience different friction and contact behavior across surfaces.

7.4 Contextual Objects↩︎

We enrich outdoor and indoor-to-outdoor scenes with contextual object groups to increase visual realism, provide local semantic cues, and diversify object-level goals. Unlike random object scattering, contextual placement is conditioned on the semantic type of each map region. For each region, we sample only object groups that fit the local context, such as bike racks and bicycles in bike-parking areas or bus shelters, briefcases, and umbrellas near bus stops. Contextual objects are instantiated from a curated library of 30 contextual-object groups. Each group consists of a small set of assets that co-occur in realistic activities, and objects may be stacked based on their sizes and support surfaces, such as a bag on a bench or toys on a table. Table 4 shows representative examples of the group definitions. To avoid excessive clutter, the placement of a new group is constrained to be at least \(8.0\,\mathrm{m}\) away from any existing contextual group.

6pt

7.5 Storefronts↩︎

We create generated storefront assets to enrich outdoor building facades and provide visually distinctive goal locations for PlaceNav and language-guided tasks. We generate 101 storefront textures covering place categories such as restaurants, cafes, and banks, together with corresponding relative depth maps using Gemini 3 Pro Image [46]. The generated assets are manually inspected for visual plausibility and semantic consistency. Each accepted storefront is converted into a 3D mesh by depth-based extrusion in Blender. We place storefront meshes at OSM-specified store locations by ray-casting to the nearest building facade, favoring vertical exterior faces. We orient each storefront toward the closest road, scale it to a target height, and align its base to the terrain. We carve a matching opening in the facade, shift laterally to resolve corner or overlap conflicts, and accept a placement only if the cut succeeds, the storefront faces the road, and it remains unoccluded by surrounding geometry.

7.6 Indoor-to-Outdoor Scene Assembly↩︎

We construct indoor-to-outdoor scenes by embedding indoor layouts into buildings in the outdoor city environment. For each scene, we first search for a suitable road-facing building facade with enough space for an exit and nearby traversable street or sidewalk area. We create a facade opening to serve as the outdoor exit. We then place a compatible indoor layout behind the selected building facade. We align the indoor entrance with the facade opening and adjust the entrance height to match the local outdoor terrain. We remove blocking door, wall, or facade meshes around the connection, producing a traversable passage between the interior and the street. This door-to-facade connection allows the agent to transition smoothly between the interior and the street without teleportation or viewpoint switching. The indoor and outdoor components are executed in the same Isaac Sim scene, so the robot uses the same physics, sensing, and control stack across the full trajectory. The resulting indoor-to-outdoor scenes support episodes that start inside an indoor room, pass through interior corridors, exit into the city, and continue toward outdoor goals. These episodes introduce abrupt changes in visual context, lighting, environmental structure, and spatial scale, enabling evaluation of whether agents can preserve the goal and adapt their search behavior after crossing from indoor layouts into outdoor space.

7.7 NavMesh Generation↩︎

We generate navigation meshes with the C++ RecastNavigation library and provide a lightweight Python wrapper for NavMesh building and query APIs.

• Adaptive settings. We use two NavMesh presets to support scenes with different scales. Indoor scenes use a finer voxelization with smaller cell size and height to capture narrow traversable regions. Outdoor scenes use a coarser resolution for efficiency. Table 5 summarizes the NavMesh build parameters used in this work.

• Episode sampling and reference paths. We use the NavMesh to sample feasible start positions and to generate reference paths from the start to each goal instance. These paths respect traversability constraints encoded by the NavMesh build parameters, including maximum walkable slope and maximum climb, and therefore reflect what the robot can physically traverse.

6pt

8 Task Generation↩︎

8.1 Episode Sampling↩︎

Episodes are generated by sampling a starting position and a goal category, then constructing a set of valid goal instances and their oracle reference paths.

• Goal discovery and instances. We traverse scene prims and group them into goal categories with scene-specific matching rules (e.g., regular expressions over prim names or prim paths).

• NavMesh-based start sampling and reference paths. We sample start positions from the traversable region of the scene NavMesh and compute a path from the sampled start position to each candidate goal instance center. For each scene, we sample goal categories uniformly. We retain instances only if the path lies within a prescribed range (\(5.0\,\mathrm{m} \le L \le 50.0\,\mathrm{m}\)).

• Episode initialization. If at least one instance is retained, we form an episode. The start yaw is initialized to face the next waypoint on the closest-instance reference path, plus a random perturbation sampled uniformly from \([-30^\circ, 30^\circ]\).

• Indoor-to-outdoor task sampling. For indoor-to-outdoor scenes, the start position is constrained to lie inside the embedded indoor region. Candidate outdoor goal instances are filtered to a bounded neighborhood around the indoor region to avoid excessively long or irrelevant indoor-to-outdoor tasks.

8.2 Online Verification and Quality Control↩︎

After offline sampling, we verify each candidate episode by executing a rollout that tracks the reference waypoints under full physics. We log observations and poses along the rollout for traceability. An episode is accepted if it fulfill all of these conditions:

• Oracle feasibility. A candidate episode must achieve oracle success with respect to at least one goal instance at some time during the trajectory.

• Episode duration. The rollout duration must exceed \(5.0\) s in simulation time to avoid degenerate episodes, and must be no more than \(100\) s during task generation to avoid overly difficult episodes and keep dataset construction efficient.

• Traveled distance. The executed trajectory length during verification must exceed \(5.0\) m but less than \(50.0\) m.

• No termination. Episodes that terminate due to failure conditions (e.g., out-of-bounds, fall, stuck) are rejected.

8.3 VLN Instruction Generation and Filtering↩︎

For VLN, we attach a natural-language instruction to each verified episode. Instruction generation proceeds in two stages, and episodes that fail automatic alignment or human checks are excluded from VLN evaluation.

• Template instructions. We generate a template instruction by converting a simplified reference path into a compact sequence of turn-and-move directives, followed by an arrival statement.

• Trajectory-to-frame alignment. Using the recorded rollout poses, we align directives to frames by matching simplified waypoints to nearby executed poses.

• VLM refinement. We refine the template using a vision-language model by providing a sparse set of keyframes sampled along the rollout (similar to Figure 6) and requesting a concise instruction that emphasizes visually distinctive landmarks while avoiding exact distance values. The prompt details are provided in Box [box:vlm_prompt].

• Human filtering. We additionally perform human checks to remove episodes with poor visual quality (e.g., bad lighting) or other issues (e.g., wrong VLM instruction) that hinder instruction-following evaluation.

Describe the trajectory of the robot in the images. Improve the given instruction to:
1) include unique landmarks you see in the images for guidance, describe color, shape, etc. briefly. Only mention landmarks that can be clearly observed.
2) make the expression more diverse.
3) replace the exact distance value with more general description.
4) do not mention (img x) in the output.
5) in some cases, if you don't directly see the target in the image, just say "you will arrive at [target name]".
6) do not mention the red lines in the images, those are pointing to the waypoints.
7) if the robot is following the road, mention it in the output.
8) make the instruction compact, only include the key landmarks.
9) return the output only, no other text.
10) make it within 100 words.

Improve this instruction: {PROMPT_INSTRUCTION}

Example output: Go forward until you see the red building, then turn right and go until you see the blue car. You should be able to see the target mailbox on your right.

9 Benchmark Statistics↩︎

9.1 Episode Overview↩︎

3pt

• Split Protocol. We construct train and eval splits at the scene level, and scenes do not overlap across splits. This prevents layout leakage and ensures that policies must generalize to novel environments rather than memorizing scene-specific geometry and appearance.

• Counts. The released dataset contains 10,000 episodes, including 4,027 ObjNav episodes, 2,973 PlaceNav episodes, and 3,000 VLN episodes. We separate them into two splits, Train and Eval. Both splits are obtained after episode-quality filtering, so the task and scene counts are not exactly balanced. In particular, the dataset contains more indoor ObjNav episodes, while outdoor and indoor-to-outdoor scenes contain fewer ObjNav episodes and relatively more PlaceNav episodes. VLN episodes are generated from the same underlying navigation tasks by producing language instructions along the reference path. Some candidates are removed because they contain too few distinctive landmarks for instruction following, or because the VLM-based instruction refinement fails quality checks, resulting in 3,000 VLN episodes in total.

9.2 Goal Distribution↩︎

• ObjNav. ObjNav includes 136 goal categories. Figure 4 (a) summarizes the target distribution. Several categories appear frequently because they correspond to common indoor objects that occur repeatedly across interior layouts. In contrast, outdoor scenes contribute a larger variety of categories, producing a more diverse and long-tailed distribution. This long tail increases the difficulty of perception, since many targets have limited exposure compared to the most frequent indoor categories.

• PlaceNav. PlaceNav includes 17 storefront goal categories. Figure 4 shows that the distribution concentrates on common place types (e.g., restaurants, convenience stores, cafes), which aligns with real-world urban statistics and OSM-derived semantics. This indicates that the benchmark captures realistic place frequencies while still covering a broad set of storefront types for evaluation.

9.3 Instruction Vocabulary↩︎

• Action verbs. Figure 5 (top) summarizes the verb distribution in VLN instructions. High-frequency verbs emphasize forward progress and route following (e.g., continue, move, arrive, pass, turn), indicating that most instructions describe incremental motion primitives and local heading changes. The presence of mid-frequency verbs such as follow, approach, and reach suggests that instructions also include goal-directed segments and landmark-conditioned transitions, while the long tail (Other) reflects additional fine-grained phrasing that appears less often.

• Landmark and object nouns. Figure 5 (bottom) shows the noun distribution used for describing landmarks and targets. Frequent nouns (e.g., building, road, street, window) capture global outdoor structure and traversable space, while indoor-oriented nouns (e.g., table, door, aisle) provide local cues for fine navigation and goal localization. The broad coverage across both structural landmarks and object-level references indicates that the VLN set supports multi-scale grounding.

10 Feasibility Analysis↩︎

We conduct a feasibility study with SGImagineNav [11] to isolate which components limit end-to-end performance. We evaluate three settings: No GT (no privileged goal or trajectory), GT Goal (oracle goal location provided to the planner), and GT Trajectory (execution driven by an oracle reference path/waypoints).

Table 7 reports ObjNav and PlaceNav results across scene types. GT Goal substantially improves SR/SPL (e.g., ObjNav all-scenes SR from 6.42% to 27.83%), while GT Trajectory achieves 100% SR in all domains, indicating the stack can succeed when the commanded route is correct. SPL remains below 100% under GT Trajectory (e.g., 90.16% on all-scenes ObjNav) due to tracking error from the PID controller and the learned locomotion policy, which introduces small deviations under physics. The large gap between No GT and GT Goal suggests that traversability under physics, narrow indoor passages, and outdoor terrain variability are major challenges even when the goal is known. Indoor-to-outdoor episodes remain difficult without oracle information: No GT achieves only 4.21% SR on ObjNav and 3.64% on PlaceNav, while GT Goal and GT Trajectory substantially improve success.

Figure 6 provides qualitative diagnostics for the GT Trajectory setting by pairing a BEV overview with egocentric rollout frames from the same episode. The BEV panels show the oracle route, and the RGB frames illustrate the observations encountered during execution across indoor, outdoor, and indoor-to-outdoor scenes, including typical landmarks, viewpoint changes, and visibility variations along the path.

References↩︎