1 Introduction↩︎

With their advanced reasoning and planning capabilities, Large Language Models (LLMs) have become a primary driver for embodied agents [1]–[6]. Early frameworks primarily utilized these models to execute diverse household tasks within text-based environments built upon underlying physical simulators [7]–[12] via textual perception and tool calling. However, pure-text interactions inherently lack the spatial and physical grounding critical for complex execution. This limitation has driven the widespread adoption of Vision-Language Models (VLMs) to endow agents with rich visual perception [13]–[16]. By leveraging high-fidelity rendered imagery, modern vision-driven environments enable precise spatial reasoning and significantly expand the range of executable tasks [17]–[19]. Yet, a critical bottleneck persists: existing benchmarks often decouple simulation from real-world complexities across two fundamental dimensions—active exploration and communicative interaction (see 1, Left). Bypassing these realistic constraints inherently leads to deployment failures during sim-to-real transfer.

Specifically, the first limitation lies in the reliance on simulator-privileged perceptual information during physical exploration. Many existing environments equip agents with idealized tools that access global object lists [6], [18], [19] or locate targets via oracle category-finding APIs [17], [18]. Since these privileged interfaces are difficult to retrieve in physical environments, policies trained within such setups can face performance discrepancies during real-world deployment. To address this, several recent works [20]–[22] adopt a non-privileged paradigm that restricts agents to standard sensor inputs like RGB images. However, their exploration and planning trajectories often operate primarily at the textual level rather than being tightly grounded in visual observations. Consequently, these methods can still struggle with precise visual disambiguation when encountering visually similar objects or distractors.

The second limitation stems from the assumption of explicit user instructions, leaving the need for dynamic intent alignment less addressed. Current benchmarks [17], [21], [23] typically treat execution as an open-loop process, assuming that the user possesses sufficient knowledge of the environment and issues unambiguous commands. In practical deployment, however, human users often operate under incomplete information, leading to initially ambiguous goals that require active, context-aware interaction to clarify intent. While recent studies [24]–[26] have introduced simulated users to facilitate interaction, their scope is primarily focused on text-based preference elicitation or memorization, leaving the co-optimization of communication and physical mobile manipulation tasks less explored.

To resolve these issues, we present REAL(exploRatory, communicativE, And, deployLable), an agentic framework designed for interactive and deployable open-world mobile manipulation. As illustrated in 1 (Right), REAL establishes a closed-loop visual environment entirely free of oracle perceptual APIs. The agent is restricted from accessing global object lists, target 3D poses, privileged simulator states, or goal locations. Instead, to ground objects, the agent must actively explore the environment and discover objects from raw RGB observations through a deployable toolchain spanning navigation, object detection, and visual prompting for manipulation [27]. In addition, we integrate a simulated user that issues initially ambiguous instructions and provides dynamic, context-aware natural language feedback. This feedback is tightly grounded in the agent’s ongoing exploration results, effectively capturing the information blind spots inherent in real-world human interactions.

Within this environment, we develop an automated, scalable data generation and filtering pipeline to construct diverse exploration and interaction tasks. Utilizing the resulting trajectory data, we align and optimize the REAL agent based on Qwen3-VL-8B-Instruct [28] via a two-stage paradigm: supervised fine-tuning for tool-use alignment, followed by online reinforcement learning to enhance adaptive reasoning and error recovery. To systematically evaluate our agent, we introduce REAL-Bench, a comprehensive benchmark comprising 241 task instances across four distinct families: active exploration, visual distraction, articulated manipulation, and interactive disambiguation. Experimental results show that our trained agent successfully outperforms leading commercial closed-source VLMs, achieving a 56.9% success rate on interactive tasks. Finally, we deploy and evaluate our agent on a physical LIFT2 dual-arm mobile robot. The high-level policy achieves a 78.3% end-to-end success rate over 60 real-world episodes, demonstrating robust zero-shot transferability and validating that our sim-to-real-consistent framework design effectively bridges the reality gap.

In summary, our main contributions are threefold:

1. ****REAL** framework.** We introduce REAL, a sim-to-real-consistent agentic framework without oracle perception, coupled with a simulated user for dynamic human-in-the-loop intent alignment.

2. Training pipeline. We develop an automated task-generation pipeline and train a Qwen3-VL-8B-based agent with dual-stage SFT and online RL, enabling robust tool-use alignment and adaptive replanning.

3. Benchmark and deployment. We construct REAL-Bench with 241 tasks and show that the trained agent outperforms strong commercial baselines in simulation while transferring zero-shot to a physical dual-arm mobile robot with a 78.3% success rate.

2 Related Work↩︎

2.1 Embodied Agent Environments and Benchmarks↩︎

Most embodied agent environments build on foundational simulators [7]–[11], [29]–[31]. The Embodied Agent Interface [6] and ManiTaskGen [19] evaluate LLMs or generate fine-grained tasks, but simplify exploration by providing ground-truth object lists. VisualAgentBench [32], EmbodiedBench [18], and VoTa-Bench [17] incorporate visual inputs yet rely on unrealistic primitives (e.g.,, a “find” tool that teleports objects), hindering real-robot deployment. PARTNR [20] explores rooms via Concept Graphs [33] and EMMOE [21] uses low-level policies without privileged states, but both reference objects through text, causing ambiguity with visual distractors. In contrast, our environment eliminates oracle perceptual APIs and grounds discovered objects in visual space, enabling robust disambiguation and real-world-compatible exploration.

2.2 Vision-Driven Embodied Agents↩︎

Recent work has explored visually grounded embodied agents, ranging from LLM-based exploration agents such as Voyager [34] to VLA policies that directly condition on visual observations, such as OpenVLA [35] and \(\pi_{0.5}\) [36]. In simulation-oriented settings, VoTa-Bench [17] applies dual preference optimization for state prediction and action selection; EMMOE [21] trains a DPO-based planner with lightweight policies [37] for mobile manipulation; ERA [23] distills embodied priors and refines agents through online RL; and OWMM-Agent [38] adapts VLMs to open-world mobile manipulation through simulation-based multimodal agentic data synthesis. However, these methods do not center on oracle-free active exploration and dynamic user interaction for acquiring missing task state. RoboOS [39] and Being-0 [40] both support real-world robot deployment. However, RoboOS does not explicitly train the agentic planning capability, while Being-0 delegates high-level planning to a closed-source foundation-model API. In contrast, REAL trains its high-level VLM agent end-to-end in simulation, where a procedural environment and a data-generation pipeline generate planning-level training data at scale.

Classical task and motion planning (TAMP) integrates symbolic task planning with continuous motion feasibility and state tracking [41], [42]. LLM planners such as SayCan [1] ground language in predefined skills and affordances, while uncertainty-aware or replanning frameworks such as KnowNo [43] and RePLan [44] decide when to ask for help or recover from execution failures. REAL is complementary to this line of work: it does not solve the TAMP problem or model all uncertainty in low-level motion execution. Instead, REAL studies how to train a deployable visual-interactive policy that acquires missing task state through RGB exploration and user dialogue before dispatching physical skills.

2.3 Simulated User Modeling for Embodied Agents↩︎

Multi-agent systems have been widely investigated in embodied tasks to facilitate flexible collaboration. Beyond single-agent configurations, EMOS [45] proposes a heterogeneous multi-robot operating system driven by LLM agents. Similarly, REMAC [46] enables multi-agent long-horizon manipulation through self-reflection and self-evolution. Focusing on communication, CoELA [47] investigates natural language interactions among agents, albeit under the assumption of shared environmental awareness. These works highlight the importance of open-ended agent collaboration in complex environments. TEACh [48] establishes multi-turn dialogue for household tasks, and VL-LN [49] uses simulated users to resolve ambiguities via active communication. For user intent understanding, ADAPT [24] elicits preferences through questions, while FARMER [25] employs theory of mind to infer unstated goals. Our framework goes beyond query generation or habit memorization by enforcing a closed-loop process of active exploration, multi-turn dialogue, and manipulation, simulating realistic scenarios requiring genuine human-agent cooperation.

3 Environment Design↩︎

To facilitate the deployment of vision-driven embodied agents from simulation to the real world, we construct a closed-loop embodied environment (2). Built upon the GRUtopia [50] platform, the proposed environment features high-fidelity rendering and rigid-body physics. At the macro level, the design adheres to two core principles that distinguish it from previous environments that simplify exploration and communication: providing a toolchain free of oracle perceptual APIs and capable of real-world implementation, and integrating a simulated user for natural language interaction. Within this architecture, scene entities are categorized into fixed macroscopic Receptacles (e.g.,, tables, cabinets) and manipulable Objects (e.g.,, cups, books). For a specific scene, the Receptacles remain static across different task instances, whereas the quantities, categories, and spatial placements of manipulable Objects vary.

3.1 Privilege-Free Environment Modeling↩︎

3.1.0.1 Initial Observation.

We assume that the agent is equipped with a long-term 3D occupancy map and semantic segmentation of the receptacles, enabling navigation to a receptacle through path planning. This prior is physically obtainable from a robot’s previous scans or mapping stack. Consequently, at task initialization, the agent receives only a prior list of receptacle IDs and their semantic categories (e.g.,, table_2; see the Embodied Agent block in 2). Despite this structural prior, the agent has no access to global object lists, target 3D poses, simulator object states, or goal locations. It also lacks the fine-grained appearance of receptacles and the distribution of objects upon or within them. To ground target objects, the agent must employ the active exploration toolchain described below to acquire observations.

3.1.0.2 Multi-level Active Exploration.

To enable exploration, we introduce a multi-level active exploration toolchain (see the Exploration Tools block in 2). These tools avoid oracle perceptual APIs at the policy interface and can be transferred to the real world (details are provided in the appendix). The overall process is structured into four sequential actions. The inter-receptacle navigation tool (nav_to) enables movement between different receptacle nodes based on structural priors. Subsequently, the intra-receptacle scanning tool (walk_around) functions as a broad-spectrum scanner driven by open-vocabulary 2D perception models. This tool navigates the agent around a specific receptacle to bypass physical occlusions and detect common object categories from multiple viewpoints. Following this, the object detection tool (show_object_by_category) actively isolates objects of a user-defined category within the current ego-centric view. Ultimately, the target approach and alignment tool (gaze_at) advances the agent toward a selected target to center it within the view of the camera, while accounting for physical distance constraints.

3.1.0.3 Vision-Grounded Interaction.

To maintain spatial memory without relying on privileged 3D coordinates, the detected objects are dynamically back-projected from the 2D segmentation masks into the 3D space. This geometric projection constructs a temporary exploration map that records all discovered object categories and their spatial anchors during the most recent invocation of the exploration tool. Objects recorded in this map are highlighted within the RGB/SoM observation of the agent [51] (see the Observation block in 2). This mechanism bridges perception with physical interaction, enabling the agent to distinguish objects of the same category and parameterize downstream manipulation skills (e.g.,, pick, open; see the Pick/Place/Open/Close controls in the ToolChain block in 2) by referencing the assigned visual IDs.

3.1.0.4 Simulated User for Collaborative Tasks.

To simulate collaborative tasks, we integrate a simulated user server driven by gemini-3-flash [52] directly into the dynamics of the environment (see the Simulated User block in 2). At the top level, this module functions as an interactive supervisor that directs the overall task flow. It provides ambiguous initial instructions, monitors the task progress of the agent remotely, and progressively clarifies objectives by answering queries based on real-time exploration results.

To operationalize this top-level design, the module relies on specific mechanisms for configuration, synchronization, and communication. A task-specific profile is initialized for the simulated user to establish the boundaries of preferences and choices, such as restricting targets to objects currently present in the scene. This configuration ensures evaluation stability and prevents the generation of instructions involving non-existent or unexplored items. Furthermore, the server maintains continuous synchronization with the execution trajectory of the embodied agent. By accessing the interaction history and physical observations of the agent, the system constructs a consistent shared context for multi-turn alignment, accurately replicating a human user with limited perception of the initial scene layout. Finally, a bilateral communication interface facilitates interaction. When physical exploration alone fails to resolve ambiguity, the agent can issue an ask action to query the user (see the Ask control and question flow in 2). Leveraging the shared context and predefined profile, the simulated user subsequently generates natural language guidance to clarify intent and drive the completion of the task.

4 Agent Training↩︎

To minimize online inference overhead and mitigate risks of context explosion or temporal hallucinations during long-horizon deployment, we formulate the decision-making process as a Partially Observable Markov Decision Process (POMDP). Instead of continuously concatenating raw historical observations, our policy relies on a compact aggregated state paired with the current visual input to derive discrete tool actions, maintaining a computationally efficient, localized historical proxy.

Our training pipeline consists of three steps. First, we construct a Markovian state representation and transition pipeline to support efficient data generation (4.1). Second, we use expert demonstrations to align the model with the tool interface and structured reasoning format (4.2). Finally, we optimize the policy with Group Sequence Policy Optimization (GSPO) [53], using exploration and environmental feedback to improve task-solving performance (4.3).

4.1 Problem Formulation and Context Management↩︎

4.1.0.1 Closed-Loop Execution.

We formulate the single-step decision-making process of the embodied agent as a POMDP (see the appendix for the complete tuple definition). Lacking direct access to the underlying environment state \(s_t \in \mathcal{S}\), the agent receives partial observations \(o_t\) through an observation function \(\Omega(o_t \mid s_t)\) extracting perceivable information from the true state. The agent operates in a continuous perception-reasoning-action loop. Given a task instruction \(T\), the VLM policy maps the current observation \(o_t\) to a formatted tool call \(a_t \in \mathcal{A}\), using environment tools free of oracle perceptual APIs. A tool executor validates format and dispatches the call, immediately returning invalid actions as environmental feedback. This architecture decouples high-level decision-making from low-level safety verification and environmental dynamics.

4.1.0.2 Observation and State Aggregation.

To address context explosion and temporal hallucinations inherent in long-horizon tasks, we partition the agent’s observation into an environmental component and a historical component: \[o_t = \langle o^{env}_t, o^{hist}_t \rangle\] The environmental observation \(o^{env}_t\) encapsulates immediate perceptual results (e.g.,, ego-centric visual features, structural world graph changes) and the execution feedback \(e_t\) corresponding to \(a_{t-1}\).

To avoid linear context growth from concatenating full interaction histories, we manage \(o^{hist}_t\) through self-summarization. We define this compressed observation as: \[o^{hist}_t = \langle o^{env}_{t-1}, a_{t-1}, h_{t-1} \rangle\] where \(h_t\) represents a structured historical state maintained by the VLM. Initialized by \(T\), the VLM decomposes the global task into sub-goals. At each step, alongside Chain-of-Thought (CoT) reasoning and tool invocation, the VLM generates a summary \(\sigma_t\) of completed work and an analysis of the active phase \(task_t\).

The history is aggregated as \(h_t = (\Sigma_t, task_t)\), where \(\Sigma_t = [\sigma_0, \sigma_1, \dots, \sigma_t]\), and is updated online via: \[h_t \leftarrow \operatorname{VLM}(o^{env}_t, a_t, h_{t-1})\] Note that all components of \(o^{hist}_t\) (namely \(o^{env}_{t-1}\), \(a_{t-1}\), and \(h_{t-1}\)) are fully determined by information available up to step \(t{-}1\), i.e.,, \(o^{hist}_t\) is \(\mathcal{F}_{t-1}\)-measurable with respect to the natural filtration \(\{\mathcal{F}_t\}_{t \geq 0}\). This ensures that the policy \(\pi_\theta(a_t \mid o_t)\) conditions only on legitimately available information, preserving the causal structure required by the POMDP formulation.

This bounds the per-step prompt to a fixed structure (current observation, previous action, and compressed history) rather than replaying the interaction trace. Crucially, \(\Sigma_t\) records only the actions confirmed as successful by the tool returns \(e_t\). Tracking execution failures solely in \(o^{env}_t\) rather than persistent history provides a deterministic basis for subsequent reasoning and effectively mitigates temporal hallucinations.

4.1.0.3 Scalable Asset Library and Task Instantiation.

Our data engine leverages 7 high-fidelity scenes from GRScenes [50] (11 room types, 100 interactive receptacles) and over 2,500 object instances spanning 639 categories from MesaTask [54]. We represent each scene state as a World Graph (WG), formally \(\text{WG} = \{(r_j, \mathcal{E}_j)\}\), mapping each receptacle \(r_j\) to its contained object instances \(\mathcal{E}_j\). The WG serves as the canonical state representation throughout the framework: it grounds task definitions, enables state-differential reward computation (4.3.0.2), and provides the success criterion for evaluation.

Tasks center on cross-receptacle object rearrangement: we prompt LLMs to generate semantically plausible (source, destination, object) triplets conditioned on the scene layout, which are instantiated as task configurations by specifying an initial WG and a goal WG in the simulator.

To systematically challenge the agent, we introduce distractors at both the object and receptacle levels. Object-level distractors inject visually distinct instances of the same category to require fine-grained discrimination. At the receptacle level, each receptacle is annotated with uniquely identifying descriptions covering functional purpose and visual appearance. LLMs then generate unambiguous natural-language instructions from these annotations. Further details and dataset statistics are provided in the appendix.

4.1.0.4 SFT Data Synthesis.

For each task, a rule-based planner decomposes the objective into an action sequence \(\{a_t\}_{t=1}^L \subset \mathcal{A}\) covering navigation, exploration, and manipulation, and executes it in the simulator. At each step, the observation extractor \(\Omega\) yields \(o^{env}_t\), including ego-centric RGB images with visual IDs and execution feedback \(e_t\); all manipulation operates exclusively on visual IDs from 2D perception. In an automated Social Annotation phase for collaborative trajectories, LLMs rewrite instructions to be deliberately vague, injecting communicative actions (ask) with simulated user responses into trajectories to train the agent to communicate actively under uncertainty. The resulting SFT dataset is \(\mathcal{D}=\{(T^{(i)}, \tau^{(i)})\}_{i=1}^{N}\), where each trajectory \(\tau=\{(o_t, a_t, h_t)\}_{t=1}^{L}\) of length \(L\) records the observation, action, and historical state at each step. Actions and observations come from executed trajectories, while historical states are annotated by gemini-3-pro [52]. See the appendix for annotation details.

4.2 Paradigm Alignment via SFT↩︎

4.2.0.1 Unified Input-Output Paradigm.

We select Qwen3-VL-8B as the backbone VLM to support deployment on computationally constrained robotic systems, implementing a unified input-output structure for both training and inference.

At each time step \(t\), the VLM receives a prompt containing the available tool APIs \(\mathcal{A}\), the task instruction \(T\), and the observation \(o_t = \langle o^{env}_t, o^{hist}_t \rangle\). The model generates internal reasoning (e.g., <think>...</think>) to analyze the visual state and plan, then outputs a JSON specifying the updated historical state \(h_t\) and next tool call \(a_t \in \mathcal{A}\).

4.2.0.2 Tool-Calling Alignment via Behavior Cloning.

In the first stage, we adapt the pre-trained VLM to the tool interface by behavior cloning on \(\mathcal{D}\). Let \(\pi_\theta\) denote the VLM policy. We minimize the negative log-likelihood of expert trajectories: \[\mathcal{L}_{\text{SFT}} = -\sum_{(T,\tau)\in\mathcal{D}}\sum_{t=1}^{L} \log \pi_\theta\!\left(a_t, h_t \mid T, o_t\right),\] where \(o_t\) is the agent’s observation at step \(t\). We freeze the vision encoder to preserve pre-trained visual representations.

4.3 Reinforcement Learning for Skill Optimization↩︎

Deploying the SFT-aligned initial policy \(\pi_{\text{init}}\) (4.2) in closed-loop inference exposes two limitations: (1) distribution shift, where behavior cloning on expert demonstrations does not equip the model to replan or recover from execution failures (e.g.,, occluded targets, invalid API calls); and (2) scarce interactive data, where static datasets cannot cover the diversity of proactive human-robot interaction, causing the policy to overfit to templates rather than querying the user under ambiguity. RL addresses both issues by enabling the agent to explore dynamic environments and actively acquire information from users.

4.3.0.1 Algorithm Selection.

We aim to maximize the expected discounted return over variable-length trajectories under the POMDP formulation (4.1). Accurately modeling a critic for VLM-based RL is computationally prohibitive, making critic-free methods such as Group Relative Policy Optimization (GRPO) a natural choice. However, GRPO assigns sequence-level credit via step-wise importance ratios whose cumulative log-ratios grow in variance with trajectory length, frequently triggering clipping and destabilizing training. We therefore adopt GSPO, which replaces the step-wise ratios with a length-normalized sequence ratio, compressing variance by \(1/L_i^2\) and keeping proximal updates valid during long rollouts. This drives the model toward adaptive replanning and proactive interaction rather than static template matching. Detailed formulations are provided in the appendix.

4.3.0.2 Environment-Grounded Reward Shaping.

Instead of sparse task-completion signals, we design a step-wise reward \(r_t\) grounded in the WG (4.1.0.3). We compute \(\Delta_{task}\), the set of required state changes between the initial and goal WG (e.g.,, picking and placing \(n\) objects yields \(2n\) entries). At each step, the instantaneous difference \(\delta_t = \text{WG}_t - \text{WG}_{t-1}\) is matched against \(\Delta_{task}\); a match yields a positive reward and the item is removed to prevent reward hacking. The physics engine inherently enforces temporal causality (e.g.,, pick-before-place), eliminating the need to manually model temporal dependencies. Articulation and navigation sub-goals are appended to \(\Delta_{task}\) and evaluated online.

Beyond task progress, the agent receives a per-step timing penalty \(-\lambda_{time}\) for efficiency and a severity-scaled penalty for failed actions based on \(e_t\) traceback logs. For tasks involving distractors, we reward the communicative tool ask to resolve ambiguity, subject to a difficulty-conditioned query budget \(B_{\texttt{ask}}\) to prevent over-reliance on the user. By formulating the reward as \(r_t = R(o^{env}_{t-1}, a_t, o^{env}_t)\), the Markov property is preserved and the dense signals align with the GSPO optimization objective (detailed numerical configurations are provided in the appendix).

5 Experiments↩︎

5.1 Benchmark Construction↩︎

To evaluate embodied agents along two key dimensions, active exploration and communicative interaction, we introduce REAL-Bench. Built on the environment and tool interface (3) and the data generation pipeline (4.1), it covers novel scenarios across REAL rooms. REAL-Bench contains 241 task instances in four families: Furniture-Distractor Pick-and-Place (FDP, 72), Furniture & Object-Distractor Pick-and-Place (FODP, 56), Furniture-Distractor Open/Close (FDO, 48), and Simulator-User-Loop (SUL, 65).

FDP and FODP are cluttered mobile manipulation tasks that require moving a target object from a source to a destination receptacle. FODP further adopts an open-vocabulary, full-category distractor setting at both the furniture and object levels, placing greater demands on active exploration and fine-grained visual disambiguation. FDO evaluates articulation-centric manipulation with openable and closeable furniture. These tasks involve highly dynamic state transitions and challenge temporal consistency and precise sequential tool use (e.g.,, interleaving open/close with perception and pick-and-place actions). Finally, SUL introduces instruction ambiguity by design to test whether the agent can proactively query the simulated user via ask before execution.

5.2 Results and Analysis of SFT and RL Training↩︎

We evaluate (i) strong open-source and commercial VLMs under zero-shot prompting, and (ii) the proposed Qwen3-VL-8B agent after tool-interface alignment via SFT and closed-loop improvement via GSPO. 1 reports success rates (higher is better) on the four REAL-Bench task families.

5pt

5.2.0.1 Tool Alignment and Behavior-Cloning Overfitting.

Zero-shot Qwen3-VL-8B fails on manipulation tasks and achieves only 1.5% on interactive scenarios. One epoch of supervised fine-tuning aligns the model with the tool interface, reaching 45.8% on FDP. However, a second epoch exposes the limits of behavior cloning: while execution stability on standard manipulation improves (FDP rises to 65.3%), performance on the distractor-rich FODP split drops from 30.4% to 28.6%. This suggests that strict imitation overfits expert trajectories and weakens robustness under open-vocabulary clutter and visual ambiguity.

5.2.0.2 Reinforcement Learning Restores Generalization.

Applying closed-loop group sequence policy optimization to the single-epoch checkpoint circumvents this overfitting. RL recovers the exploration capability lost during extended supervised training, increasing FODP success to 33.9% while maintaining tool-calling performance comparable to the model more tightly fine-tuned on standard tasks. This suggests that environmental feedback promotes adaptive problem-solving over rote trajectory memorization, helping the agent handle edge cases.

5.2.0.3 Social Reward Improves Proactive Intent Alignment.

The impact of closed-loop optimization is most pronounced in the interaction-heavy SUL split, where the agent reaches 56.9% success and outperforms the strongest proprietary zero-shot baselines. By integrating a difficulty-conditioned query budget and explicit rewards for resolving ambiguity, the training pipeline shifts the agent from passive instruction following to proactive intent alignment. The policy learns when to pause and query the simulator-user, showing that social interaction emerges from end-to-end environmental feedback rather than static prompt engineering.

5.2.0.4 Dynamic Manipulation Reveals a Vision Bottleneck.

Despite the gains from policy optimization, success rates consistently drop in the articulation-heavy FDO split. Operating doors and drawers introduces highly dynamic visual state changes. Because we freeze the vision encoder to preserve pre-trained representations, the agent remains bottlenecked by the base model’s visual grounding limits. This suggests that while closed-loop RL improves action and communication, mastering articulated manipulation may require further scaling on its multimodal abilities.

5.2.0.5 Failure Mode Taxonomy.

We manually annotate 100 episodes from the best SFT+RL checkpoint to diagnose residual errors (3). Of 53 failures, most fall under Object Confusion (25), followed by Key Action Missing (17), Lost Memory (9), and Other Error (2). These results identify cluttered visual grounding, procedural completeness, and long-horizon memory as the main remaining bottlenecks.

5.3 Real-World Deployment via Sim-to-Real Transfer↩︎

To validate the effectiveness and robustness of our training pipeline, we deploy the simulated agent in a real-world household scenario using the Ark LIFT2 mobile manipulator. This direct sim-to-real transfer tests how capabilities learned in simulation map to physical execution across varying environmental complexity.

5.3.0.1 Seamless Transfer Through Standardized Interfaces.

The system uses a Model Context Protocol(MCP) server for tool invocation, keeping the high-level cognitive framework identical between simulation and the physical robot. At each step, agent receives real-time ego-centric observations, combines them with the structured history and immediate past action, and outputs the next discrete tool call. This standardized interface decouples the reasoning algorithm from the environment, allowing the physical world to act as an alternative backend to the simulator without retraining the high-level policy.

5.3.0.2 Hierarchical Execution with Specialized Physical Policies.

While high-level planning is transferred zero-shot from simulation, continuous physical dynamics still require robust motor control. We therefore map the abstract tools used in simulation to dedicated real-world implementations. Specifically, atomic manipulation commands are executed by a fine-tuned \(\pi_{0.5}\) [36] base model. This hierarchy keeps VLM focused on visual grounding, state tracking, and task planning, while the \(\pi_{0.5}\) policy provides stable execution for opening, closing, picking, and placing.

5.3.0.3 Long-Horizon Causal Reasoning in Physical Environments.

As shown in 4, the system successfully executes a multi-stage food preparation task requiring the robot to fetch bread and place it in a microwave. Rather than immediately navigating to the cabinet, the reasoning engine first approaches the table and opens the microwave door. Only after preparing the receptacle does the robot navigate to the secondary location, select the target object, and return to complete placement and closure. This sequence suggests that the structured memory mechanism helps prevent temporal hallucinations, maintain an accurate internal state, and respect causal constraints over long physical executions.

5.3.0.4 Physical Evaluation and Pipeline Analysis.

We further evaluate the pipeline in the physical environment across different spatial layouts. Across 60 real-world episodes, REAL achieves 78.3% end-to-end success with zero unrecoverable system crashes (2). The low-level primitive layer is executable in 85.3% of 600 VLA primitive executions, indicating that the high-level policy transfers without requiring privileged simulator state while still depending on robust physical skills. A more detailed breakdown of these real-world trials, along with primitive-level evaluation and failure analysis, is provided in the appendix.

6 Conclusion and Limitations↩︎

We presented REAL, a closed-loop framework for training exploratory and communicative embodied agents that transfer directly to physical robots. By replacing oracle perceptual APIs with a physically realizable exploration toolchain and integrating an LLM-driven simulated user for dynamic intent alignment, REAL addresses two key oversimplifications in prior work. Training a Qwen3-VL-8B agent via SFT followed by closed-loop GSPO on REAL-Bench(241 tasks, four families) shows that tool-interface alignment is a prerequisite for closing the perception-action loop, while RL uniquely improves interactive disambiguation. Deployment on the Ark LIFT2 dual-arm mobile robot further validates sim-to-real transferability for long-horizon household tasks.

Despite these results, several limitations remain. First, the task space centers on cross-receptacle rearrangement and does not yet encode richer compositional constraints such as temporally ordered sub-tasks [20] or spatial relational goals [19] (e.g.,, “place the cup to the left of the plate”). Second, the simulated user operates within a bounded behavioral envelope—its preferences are constrained to in-scene objects and its responses are anchored to the agent’s trajectory—whereas real users may introduce new goals mid-task or convey intent through implicit cues. Third, receptacles are modeled as monolithic entities (e.g.,, cabinet_1) without part-level distinction (e.g.,, individual shelves), limiting spatial precision for both exploration and manipulation. Incorporating part-level segmentation and hierarchical receptacle modeling would address this gap.

7 Acknowledgments↩︎

This work was supported in part by Shanghai Artificial Intelligence Laboratory, the National Natural Science Foundation of China under Grant 62401367 and the Shanghai Magnolia Talent Program Pujiang Project under Grant No. 25PJA076.

Appendix↩︎

References↩︎