June 30, 2026
Mobile GUI agents increasingly face long-horizon tasks that require reading, updating, and reusing task-relevant data across pages and applications. Existing memory methods treat memory largely as passive storage, where past observations are accumulated and retrieved when needed. Yet retrieving a value does not reveal its current role in the workflow. The agent must still infer from accumulated records whether the value should be used now, has already been used, or must wait for a later dependency. This implicit reconstruction becomes unreliable in long trajectories with similar fields, repeated values, distractors, and outdated states, causing repeated or missed operations. We propose Active Task Driving Memory (ATMem), which shifts GUI-agent memory from passive storage to an actively maintained execution state. ATMem maintains task-relevant information as a continually updated execution state that links each value to its role and current status, enabling action selection based on the current workflow state. We therefore introduce STR-GRPO, an online reinforcement learning method that learns to use ATMem selectively according to its contribution to task completion. STR-GRPO contrasts memory-on and memory-off rollouts to estimate when memory use improves execution, while memory-cost-aware reward discourages costly memory usage that does not improve execution. To evaluate whether agents can complete all in-scope work while avoiding out-of-scope actions over long-horizon execution, we build a challenging mobile benchmark. From a list of near identical entries, agents must act on every entry that satisfies the instruction and reject entries that violate its constraints. We further introduce App-Level Progress and Scope-Aware F1 to measure these two dimensions separately. Experiments on AndroidWorld, MobileWorld, and our benchmark show that ATMem substantially improves long-horizon execution. With an 8B model, ATMem-UI achieves 76.6% success rate on AndroidWorld, outperforming UI-TARS-2-230B by 3.3 percentage points and the same-sized MAI-UI by 5.9 points. Our code and model will be publicly available.
Online mobile agents have made substantial progress in executing user instructions in real-world mobile environments, advancing from single-step grounding to multi-step navigation [1]–[5]. As tasks extend to long-horizon workflows [6], [7], the agent must carry information across pages and applications while reading, verifying, modifying, and submitting it as the task unfolds. In such workflows, the next action often depends on information observed, generated, or modified many steps earlier that is no longer visible on the current screen [3], [4], [8], [9].
A natural response is to equip GUI agents with memory. Existing approaches largely frame memory as a passive record of the past, whether through full interaction histories [10], [11], trajectory summaries [12], [13], free-form notes [14], [15], or retrieval over past traces [16], [17]. Under this view, memory is framed as remembering more or retrieving better. We argue that this view is incomplete because making past information accessible does not necessarily make it useful for execution. As shown in Figure 1 (a), even when an agent retrieves the required value, it may still repeat a completed operation or omit a required one. Record-centric memory preserves what was observed, but does not bind each value to the subgoal it supports or indicate whether the value is pending, ready, or already used. The agent must therefore infer the current task state from accumulated records at every step, including which subgoals have been completed, which ones remain, and which action is currently valid. As workflows span more pages and applications, values become separated from the operations they are meant to guide, making this inference increasingly error-prone.
This raises a basic question that is rarely asked directly: What memory do GUI agents actually need to drive long-horizon execution? Inspired by human cognition, where memory is not merely a passive record of past experience but an active mechanism for maintaining goals, tracking progress, and guiding future actions [18], [19], we argue that GUI-agent memory should be an active task-driving state rather than an archive of past observations. Guided by this insight, we propose Active Task-Driving Memory (ATMem), which is updated during interaction to track task progress and guide subsequent actions. This view is particularly important in GUI workflows where task progress depends on the evolving status of task-relevant data rather than the mere availability of past observations. As shown in Figure 1 (c), 83% of AndroidWorld tasks involve data operations that advance the workflow by finding, verifying, modifying, or submitting task-relevant information. In such workflows, each completed operation changes the execution state and determines what should be done next. For example, verifying that a contact satisfies a constraint can make it eligible for the next operation, while recording one recipe from a webpage can make the next recipe the current target for transfer to another app. Similarly, once a value has been submitted, the agent should mark it as used and avoid repeating the same operation. These evolving data states determine which subgoals have been completed, which entries remain actionable, which constraints still apply, and which action is valid next. Memory for GUI agents should therefore maintain not only the values observed during execution, but also their provenance, current status, associated constraints, and role in task progress, so that memory can directly drive action selection throughout long-horizon execution.
ATMem operationalizes this idea by organizing task-relevant values, their structure, provenance, and current status into an active execution state that tracks completed subgoals, pending operations, and data available for the next action. Unlike a scratchpad or a fixed-state tracker, ATMem is not a chronological note or a hand-engineered schema. It is induced from the task and updated by the agent during interaction. ATMem maintains a hierarchical and extensible task state across app-level files and their underlying data units. Workflow Progress records the current phase and which app-level data files remain to be processed or have been completed. Constraints encode the instruction-defined logic and conditions independently of the schema. Schema defines the minimal task-relevant data units, such as fields, together with their properties. ItemContent stores the observed content and the execution status of individual data instances. Together, these sections identify remaining files, constraint-satisfying instances, and values available for the next operation. After each new observation or operation outcome, the agent updates ATMem and combines it with the current GUI observation and interaction history to select the next action. This closed loop allows ATMem to externalize the agent’s evolving execution state, helping it perceive workflow progress and maintain consistency across screens.
To equip the agent with ATMem, we train it in two stages. Supervised fine-tuning on verified trajectories teaches the agent to construct a valid ATMem structure, update it from new observations, and reference it when predicting actions. Given that this supervision specifies how ATMem should be formed but not whether relying on it improves task completion, we introduce STR-GRPO, a memory-aware online reinforcement learning method. During interaction, STR-GRPO learns an ATMem usage policy from task outcome rewards. It contrasts paired rollouts of the same task with ATMem enabled and disabled to estimate ATMem’s contribution to task completion, while a memory cost-aware reward penalizes use that adds extra steps without improving the outcome. The agent therefore learns both how to maintain ATMem and when to rely on it for task completion.
To evaluate whether and how well agents can maintain progress and task scope over evolving task states, we build an online mobile benchmark for multi-entry tasks with scope constraints. Each task requires the agent to apply the instructed operation to every entry that satisfies the constraints while excluding entries with the same field structure that violate them. The benchmark holds the core operation schema fixed while varying the number of target entries and same-schema distractors. Difficulty therefore increases through coverage and scope control without introducing new operation types. It contains 32 manually designed templates. Using these templates, we construct three benchmark sets of increasing difficulty, each containing instances of all templates. Across these sets, the distractor-to-target ratio increases from 1.39\(\times\) to 3.22\(\times\). Considering that the terminal success rate is binary and hides partial progress, we introduce two complementary metrics. App-Level Progress measures how far the agent advances through the required operations in each application. Scope-Aware F1 measures precision and recall over task-relevant entries while penalizing operations on irrelevant ones. Across standard benchmarks and our benchmark, ATMem and STR-GRPO yield the largest gains on workflows where progress is tightly coupled with evolving data states.
Our contributions are summarized as follows.
We propose Active Task-Driving Memory (ATMem), which represents task-relevant data as a memory actively driving execution rather than an archive of past observations, enabling explicit tracking of data ownership, task role, task constraints, execution status, and workflow progress.
We introduce STR-GRPO, a memory-aware online reinforcement learning method that turns ATMem usage into a learnable policy decision through memory-on and memory-off rollout interventions and memory-cost-aware optimization.
We build a scalable data-centric online mobile-agent benchmark that evaluates cross-page, cross-application, and data-dependent workflows. It requires agents to complete all instruction-relevant operations while filtering same-schema distractors, and introduces App-Level Progress to measure per-app operation completion and Scope-Aware F1 to evaluate target coverage under distractor filtering.
We demonstrate state-of-the-art performance with our proposed agent ATMem-UI. It reaches 76.6% success rate on AndroidWorld with an 8B model, outperforms UI-TARS-2-230B by 3.3 percentage points, and achieves a 36.2% relative improvement over the strongest baseline on MobileWorld.
Mobile GUI Agents. Mobile GUI agents have made rapid progress in recent years [20]–[25]. AutoDroid [26] formulates Android task automation as a multimodal interaction problem, and subsequent systems such as AppAgent [13], AppAgent-v2 [27], MAI-UI [20], and Mobile-Agent-v3/v3.5 [15], [21] extend agent capabilities to more complex smartphone and GUI tasks. In parallel, AndroidWorld [7], AndroidLab [28], and MobileWorld [6] provide realistic online environments with programmatic evaluation. More recent works, such as MobileGUI-RL [29] and MobileRL [30], explore online reinforcement learning to improve long-horizon execution, while AgentProg [31] reduces reliance on raw history replay through programmatic context management and belief-state tracking. Together, these efforts have substantially advanced GUI interaction capabilities, online benchmarks, and long-horizon mobile-agent research.
Memory for GUI and Mobile Agents. Memory in mobile and GUI agents has developed along both experience-oriented and execution-oriented directions [12], [14], [15], [26], [27], [32]–[42]. Experience-oriented methods store app knowledge [27], human-like app memory [26], [32], past trajectories [33], skills [14], [34], or interaction patterns [38] to support knowledge reuse, generalization, and interface adaptation. Execution-oriented methods, such as note-taking [14], anchored intermediate states [37], and program-based context management [12], preserve key information or compact task context [35], [40] from ongoing trajectories to support subsequent actions. While these works highlight the importance of memory for long-horizon GUI execution, they mainly represent memory as reusable experience, recorded notes, retrieved context, or program/interface state. In contrast, our work explicitly models task-relevant data objects, their structures, values, and execution states, and uses them as actionable signals to guide execution.
As shown in Figure 2, our framework consists of four components. We first introduce ATMem, which turns task-relevant data from passive interaction records into structured execution states that agents actively maintain and use to guide actions across interaction steps (Section 3.1). We then describe verified online trajectory synthesis to build high-quality SFT data (Section 3.2). Next, we present STR-GRPO for improving memory-conditioned execution (Section 3.3). Finally, we detail the construction of our DataScope benchmark for evaluating long-horizon tasks (Section 3.4).
From Passive History to Active Task-Driving Memory. A standard mobile GUI agent can be formulated as a history-conditioned action policy. At step \(t\), given the task instruction \(c\), the current GUI observation \(o_t\), and the interaction history \(h_{<t}\), the agent generates a textual response \(y_t\), from which an output parser \(\mathcal{P}\) extracts an executable action \(a_t\) and an intermediate reasoning trace \(\rho_t\): \[y_t \sim \pi_\theta(\cdot \mid c, o_t, h_{<t}), \qquad (\rho_t, a_t) = \mathcal{P}(y_t).\] Although \(h_{<t}\) preserves values observed or produced in previous steps, it stores them as chronological records rather than as a task state organized around the data being processed. The agent must therefore repeatedly infer which item a value refers to, whether it satisfies the task constraints, and whether the associated work has already been completed. As trajectories grow longer and contain similar fields, repeated values, distractors, or outdated states, inferring task progress from accumulated records becomes increasingly unreliable, often leading to repeated or missed operations. This motivates a shift from passive interaction history to memory actively driving execution, where task progress, data eligibility, and remaining operations are explicitly maintained.
Our key insight is that many long-horizon GUI workflows require explicit tracking of the execution status of task-relevant data. We therefore introduce ATMem, an active task-driving memory that organizes task-relevant information as a hierarchical execution state. ATMem operationalizes this shift through a two-level structure that turns memory from a chronological record into an agent-maintained execution state. Formally, ATMem maintains a memory state \(m_t = (\phi_t, E_t)\), where \(\phi_t\) summarizes workflow-level progress and task constraints, and \(E_t = \{e_i\}\) is a dynamic set of task-relevant item entries. Each entry \(e_i\) stores a stable item identifier, structured task-critical content, and an execution status \(s_i \in \{\texttt{remaining}, \texttt{finished}, \texttt{skipped}\}\), indicating whether the item still needs to be processed, has been completed, or should be skipped. ATMem therefore represents not only the available data, but also the item-level state needed to determine remaining work.
Before each model call, the current ATMem state \(m_t\) is inserted into the system prompt as the current task-driving memory. Conditioned on this injected memory, the agent generates a memory-augmented response \(y_t^{\mathrm{M}}\). The parser \(\mathcal{P}\) then extracts the updated task-driving memory \(m_{t+1}\), the intermediate reasoning trace \(\rho_t\), and the executable GUI action \(a_t\) from the model response: \[y_t^{\mathrm{M}} \sim \pi_\theta(\cdot \mid c, o_t, h_{<t}, m_t), \qquad (m_{t+1}, \rho_t, a_t) = \mathcal{P}(y_t^{\mathrm{M}}).\] The extracted \(m_{t+1}\) is then inserted into the next system prompt as the current memory, enabling the agent to carry forward an up-to-date execution record without relying on history re-examination.
Basic Structure Design. As illustrated in Figure 2 (a), ATMem organizes memory into two complementary levels that implement the shift from passive history to active task-driving state. Workflow-level
fields encode where the agent is in the task and what global constraints should guide execution, while schema-level fields define and track the minimal actionable data units on which GUI operations are performed. A minimal actionable data unit is the
smallest semantically complete data object that can be independently operated on during task execution. For example, an expense record comprising expense, amount, and category forms one such unit, whereas any
individual attribute alone is insufficient to constitute an operable target.
Workflow-level fields. Workflow-level fields capture task-level execution progress and global task conditions with a fixed structure shared across tasks. The Phase field partitions execution into two macro-stages,
harvest and execute. During harvest, the agent inspects available sources and populates task-relevant item entries. During execute, the agent performs data-dependent operations over the populated entries.
The phase transitions from harvest to execute once sufficient items have been collected and no sources remain to be inspected. For tasks that require source-level tracking, RemainingFiles and
CompletedFiles maintain a checklist of sources pending inspection and sources already processed, providing the concrete signal for phase transition. When source-level tracking is not required, both fields remain empty. The
Constraints field stores task-derived filtering conditions as global execution rules, determining which collected items are eligible for operation, such as selecting only records that satisfy a specified date range, category, or recipient.
Schema-level fields. Schema-level fields define the minimal actionable data units for a given task and maintain their execution states. Schema.fields specifies the attribute set that constitutes one semantically complete
data unit, such as expense, amount, category in an expense-management task or recipient, subject in an email task. During schema construction, explicit task-specific attributes can be added
when required. Once the relevant attributes are identified, Schema.locked freezes the field set to prevent structural drift across steps. Schema.ItemContent enumerates instantiated data units, each indexed by a stable identifier
and associated with structured field values and an execution status from \(\{\texttt{remaining}, \texttt{finished}, \texttt{skipped}\}\). Here, remaining and finished indicate whether a unit still
awaits processing or has been completed. skipped provides an item-level refinement complementary to Constraints. While Constraints captures global eligibility rules derived from the task instruction,
skipped allows the agent to mark a specific item as inapplicable when richer item-level details become available during execution.
Together, workflow-level fields control task flow and global filtering conditions, while schema-level fields define and track the concrete data units to be operated on. The base ATMem structure provides a shared template across tasks, while task-specific fields can be added to accommodate different workflows, such as date attributes for calendar tasks. We provide the execution prompt in Appendix 6.3.
As shown in Figure 2 (b), we synthesize supervised data through verifier-gated online rollouts in a virtual mobile environment with 20 applications. We construct 120 task templates, each human-designed and verified, to define a controllable and verifiable task space rather than treating tasks as isolated natural-language instructions. Each template consists of two executable components and a set of instantiation guidelines. The initializer constructs the required environment state by injecting task-relevant data and distractors, and the task-specific verifier determines whether the terminal environment state satisfies the intended goal. The guidelines include a goal pattern, descriptions of required data objects and fields, task-instance examples, and suggested expansion directions to support downstream instantiation.
Based on these templates, an LLM-assisted generator instantiates about 1.2K candidate task instances with user-facing instructions and concrete initialization data. The generator leverages template-provided examples and expansion directions to produce valid task variations, while executability and correctness are determined by the initializer and verifier. After removing unreachable, malformed, or invalid instances, we obtain 1.1K executable task instances for rollout collection.
For each executable task instance, we run an online rollout framework that combines an LLM planner [43] with UIIns [44], a UI grounding model that maps predicted operations to executable GUI actions. The planner is instructed to maintain ATMem only when structured memory is needed, such as in workflows involving multiple data items, cross-file information collection, or repeated data-dependent operations. When ATMem is not needed, the memory state is kept empty. At each step, the rollout framework produces a response containing the updated memory state, an intermediate reasoning trace, and an executable GUI action. A trajectory is retained only if its terminal environment state passes the corresponding task verifier. This verifier-gated process yields 21,713 step-level SFT samples. Each sample maps the current execution context \((c,o_t,h_{<t},m_t)\) to the target response \(y_t=(m_{t+1},\rho_t,a_t)\), where \(m_t\) and \(m_{t+1}\) are empty when ATMem is not activated.
Online RL Setup. After SFT on verifier-accepted trajectories, the agent can construct and update ATMem during GUI execution. However, SFT only imitates memory patterns observed in successful rollouts and does not estimate whether ATMem contributes to the current policy’s execution. We therefore introduce Structured Task-driven Reward GRPO (STR-GRPO), an interventional online RL method. During paired rollouts, we intervene on the explicit ATMem channel by enabling or masking memory access while keeping the task initialization unchanged. The resulting rewards provide a relative estimate of ATMem’s execution utility.
Memory-Conditional Intervention. Given a task \(q\), STR-GRPO samples a group of \(N_g\) rollouts from the current policy and assigns each rollout to one of two memory conditions. Let \(z_i \in \{0,1\}\) denote the intervention for rollout \(i\), where \(z_i=1\) enables the explicit ATMem channel and \(z_i=0\) masks it. At step \(t\), the policy input can be formulated as: \[x_t^{(z_i)} = (c, o_t, h_{<t}, \tilde{m}_t^{(z_i)}), \qquad \tilde{m}_t^{(z_i)} = \begin{cases} m_t, & z_i=1,\\ \varnothing, & z_i=0. \end{cases}\]
Here, \(c\) is the task instruction, \(o_t\) is the current observation, and \(h_{<t}\) denotes the standard interaction history. The intervention changes only the explicit memory channel, while previous observations, executed actions, and reasoning traces remain available in both conditions. To reduce the effect of task difficulty, we apply a balanced assignment within each group, with \(N_g/2\) rollouts assigned to each condition.
Reward Design and STR-GRPO Objective. Given the balanced memory intervention above, STR-GRPO assigns each rollout a trajectory-level reward that combines terminal task validation and a memory-active cost: \[R_i = V(s^i_{T_i}) - \alpha z_i \rho_i, \qquad \rho_i = \frac{1}{T_i} \sum_{t=1}^{T_i} \mathbb{I}\left[\ell({m}^i_{t,\mathrm{out}})>0\right].\] Here \(V(s^i_{T_i})\in\{0,1\}\) is the verifier reward for the terminal environment state, \({m}^i_{t,\mathrm{out}}\) denotes the ATMem block generated at step \(t\), and \(\ell(m)=0\) if and only if all fields of the ATMem block are empty. Thus, \(\rho_i\) measures the fraction of memory-active steps, and the gate \(z_i\) restricts the memory cost to memory-ON rollouts, where ATMem is propagated across turns.
For each rollout group \(g\), we compute the group-normalized advantage over the balanced mixture of memory-ON and memory-OFF trajectories \(\hat{A}_i = (R_i - \mu_g) / (\sigma_g + \epsilon_A)\), where \(\mu_g\) and \(\sigma_g\) are the mean and standard deviation of rewards within group \(g\), and \(\epsilon_A\) is a small constant for numerical stability. This provides a shared baseline for both memory conditions within the same task group, so that advantages directly reflect the marginal utility of the ATMem channel.
Overall, the STR-GRPO objective is defined as: \[\mathcal{L}(\theta) = -\mathbb{E}\!\left[\sum_{i,t,k} \min\!\left(r_{t,k}\,\hat{A}_i,\; \mathrm{clip}(r_{t,k},1\pm\varepsilon)\,\hat{A}_i\right)\right], \quad r_{t,k} = \frac{\pi_{\theta}(y^i_{t,k}\mid x^i_t,y^i_{t,<k})}{\pi_{\theta_{\mathrm{old}}}(y^i_{t,k}\mid x^i_t,y^i_{t,<k})}.\] For memory-ON rollouts, the loss covers the ATMem block, reasoning trace, and executable action. For memory-OFF rollouts, ATMem block tokens are masked from the loss, and the loss covers only the reasoning trace and executable action under an empty memory channel. Memory-ON rollouts receive higher relative advantages only when ATMem improves verifier reward enough to offset the memory-active cost, discouraging redundant memory dependence and promoting selective structured memory usage.
Benchmark Overview. We introduce DataScope, an online mobile benchmark for multi-entry tasks with controlled scope difficulty. Each task requires the agent to apply the instructed operations to every entry that satisfies the instruction constraints while excluding structurally matched distractors that share the same field schema or data type. Unlike prior benchmarks that evaluate end-to-end GUI task completion, DataScope controls both target coverage and distractor filtering while holding the underlying operation schema fixed.
As shown in Figure 3, DataScope contains 32 manually designed task templates, each instantiated at three difficulty levels from DC-V1 to DC-V3. The difficulty levels increase the number of target entries and structurally matched distractors without introducing new operation types. The distractor-to-target ratio increases from \(1.39\times\) in DC-V1 to \(3.22\times\) in DC-V3, requiring broader target coverage and more precise distractor filtering. In total, DataScope includes 96 evaluation instances across 14 apps, with 94% of tasks spanning at least two apps.
Task Construction. Each task template is defined by a human-verified workflow specification: \[\Omega = \left( \mathcal{A}, g_0, \mathcal{X}, \mathcal{C}_{\ell}, I_{\Omega}, \mathcal{S}_{\mathrm{exp}}, \mathcal{B}_{\mathrm{val}} \right).\] Here \(\mathcal{A}\) specifies the apps involved in the workflow, and \(g_0\) defines the task objective to be completed by the agent. \(\mathcal{X}\) describes the structure and seed examples of task data, including target entries that must be operated on and confusable distractors that must remain unchanged. \(\mathcal{C}_{\ell}\) defines the data and operation constraints for difficulty level \(\ell\). The initializer \(I_{\Omega}\) writes an instantiated data sample into the corresponding app storage. \(\mathcal{S}_{\mathrm{exp}}\) specifies the terminal-state requirements for successful execution, while \(\mathcal{B}_{\mathrm{val}}\) contains controlled environment states and their expected evaluation scores for verifier calibration.
Our task construction proceeds in three stages. First, the workflow specification \(\Omega\) is manually constructed and verified. Second, an LLM-assisted generator instantiates executable task instances from \(\Omega\). Each instance consists of a task instruction \(c\), task-specific initialization data \(x\), and a terminal-state specification \(y^\star\). The initializer \(I_{\Omega}\) writes \(x\), including target entries and confusable distractors, into the corresponding app storage to construct the initial environment state. The agent must therefore access and manipulate all task-relevant data through real GUI interaction. Third, we generate a functional verifier for each task template. The LLM framework synthesizes the verifier from the initialization function, terminal-state requirements, and validation basis. For calibration, \(\mathcal{B}_{\mathrm{val}}\) provides controlled environment states covering successful completion, missing required operations, over-operation on distractors, missing app-level operations, and incorrect field values. Each state is paired with its expected terminal success, App-Prog., and Scope-F1 scores. A generated verifier is validated against all controlled states. If its outputs do not match the expected scores, the verifier is manually corrected and revalidated until it passes all calibration cases. Further construction details and template examples are provided in Appendix 6.4, 6.6, and 6.5.
App- and Data-Level Metrics. Terminal success equals one only when all task-specific terminal-state requirements are satisfied. Although appropriate for measuring complete execution, it assigns the same zero score to unsuccessful trajectories that terminate at different stages or exhibit different data-operation errors. We therefore introduce two complementary metrics at finer levels of granularity. App-Prog measures the completion of app-level subgoals, while Scope-F1 evaluates the completeness of individual data operations.
For task \(q\), let \(\mathcal{A}_q\) be the set of apps involved, and let \(c_{q,a} \in \{0,1\}\) indicate whether all required operations in app \(a\) are completed. App-Prog. measures the fraction of apps with all required operations satisfied: \[S_{\mathrm{prog}}(q) = \frac{1}{|\mathcal{A}_q|} \sum_{a \in \mathcal{A}_q} c_{q,a}.\] An app contributes only when its complete required subgoal is achieved. App-Prog therefore provides coarse-grained progress awareness across a multi-app workflow and distinguishes trajectories that complete different numbers of required applications. It focuses only on required subgoal completion, while unintended operations are evaluated separately by Scope-F1.
Scope-F1 evaluates execution at the atomic data-unit level. For each app \(a\), let \(\mathcal{D}^{\star}_{q,a}\) denote the set of required atomic data units and \(\hat{\mathcal{D}}_{q,a}\) the set of data units correctly handled by the agent. Each unit is the smallest independently verifiable element of task execution, jointly identified by its data entry, field, and expected value where applicable. A unit is counted as correctly handled only when the final environment state reflects the intended change on the intended entry. We define \(\mathrm{TP}_{q,a} = \left|\hat{\mathcal{D}}_{q,a} \cap \mathcal{D}^{\star}_{q,a}\right|, \quad \mathrm{FP}_{q,a} = \left|\hat{\mathcal{D}}_{q,a} \setminus \mathcal{D}^{\star}_{q,a}\right|, \quad \mathrm{FN}_{q,a} = \left|\mathcal{D}^{\star}_{q,a} \setminus \hat{\mathcal{D}}_{q,a}\right|,\) and compute the per-app F1 as \(F_{q,a} = 2\,\mathrm{TP}_{q,a} \;/\; (2\,\mathrm{TP}_{q,a} + \mathrm{FP}_{q,a} + \mathrm{FN}_{q,a} + \epsilon)\), where \(\epsilon\) is a small constant for numerical stability. Let \(\mathcal{A}^{\mathrm{act}}_q\) denote the set of apps containing at least one data unit modified by the agent. Scope-F1 is macro-averaged over both required and additionally modified apps: \[S_{\mathrm{scope}}(q) = \frac{1}{\left|\mathcal{A}_q \cup \mathcal{A}^{\mathrm{act}}_q\right|} \sum_{a \in \mathcal{A}_q \cup \mathcal{A}^{\mathrm{act}}_q} F_{q,a}.\]
Missing required units are counted as false negatives. Distractor entries, incorrect fields, incorrect values, and changes in unrelated apps are counted as false positives. Including additionally modified apps ensures that data changes outside the required workflow are also penalized.
Terminal success, App-Prog, and Scope-F1 evaluate execution at the task, application, and atomic data-unit levels, respectively. App-Prog indicates how far the agent progresses across the required applications, while Scope-F1 measures whether the required data units are correctly completed without modifying data outside the intended scope.
Training Environment and Infrastructure. We use a virtual Android platform containing 20 applications for both SFT trajectory collection and online RL training. SFT trajectories are retained only after successful task-specific verification. For online RL, we build an asynchronous on-policy training arena with 128 active containerized Android virtual devices distributed across multiple bare-metal servers. Multi-turn rollouts are generated by the latest policy and asynchronously collected to reduce idle time.
Policy optimization is performed on 128 NVIDIA H20 GPUs using hybrid parallelism to support long multimodal trajectories containing GUI observations, reasoning traces, actions, and ATMem states. This infrastructure enables scalable rollout collection and end-to-end optimization for long-horizon mobile interaction. Further implementation details and hyperparameters are provided in Appendix 6.1 and Appendix 6.2.
Evaluation Benchmarks. We evaluate our models on AndroidWorld and MobileWorld, two widely used online mobile-agent benchmarks, together with our proposed DataScope benchmark. AndroidWorld and MobileWorld evaluate general online mobile-agent performance, while DataScope focuses on cross-app workflows that require target data identification, distractor filtering, and stateful data manipulation.
DataScope contains three difficulty levels, instantiated from the same 32 task families by progressively increasing target entries, confusable distractors, and required operations. Each level contains 32 task instances, yielding 96 instances across 14 mobile apps. Overall, 94% of DataScope tasks involve at least two apps.
Baselines. Our 4B and 8B models are initialized from Qwen3-VL-4B and Qwen3-VL-8B, respectively [45]. We compare them with the corresponding Qwen3-VL backbones and representative mobile agents, including GUI-Owl [15], [21], UI-Venus [24], Doubao-1.5-UI-TARS [22], and GELab-Zero [25].
We additionally report a strong teacher-agent reference that uses GPT-5 [43] for high-level planning and UIIns [44] for visual grounding. Because this system relies on a substantially stronger proprietary planner, we treat it as a teacher reference rather than a size-matched baseline.
3pt
Performance on General Online Benchmarks. As shown in Table ¿tbl:tab:androidworld?, ATMem-UI-8B achieves 76.6% SR on AndroidWorld, outperforming all recent 4B/8B agents by a clear margin, with gains of \(+\)5.0 points over GUI-Owl-1.5-Thinking and \(+\)5.9 points over MAI-UI-8B. More tellingly, it surpasses UI-TARS-2-230B by 3.3 points while using roughly \(1/29\) of its parameters. This disproportionate gain suggests that the bottleneck in long-horizon mobile execution is not only model capacity.
The 4B results further illustrate the scale efficiency of structured memory. ATMem-UI-4B achieves 65.5% SR, within 0.4 points of the 72B-scale UI-TARS-SFT and UI-Venus, and exceeds Qwen3-VL-235B-A22B. This comparison suggests that explicit state tracking can compensate for part of the capability gap usually addressed by parameter scaling, especially in long-horizon tasks where failures often arise from losing intermediate task state rather than from single-step perception or action selection alone.
Table [tbl:tab:mobileworld] evaluates cross-environment generalization on MobileWorld, where all environments are unseen during training. ATMem-UI-8B achieves 23.3% SR, outperforming the strongest non-ours baseline, GELab-Zero-4B, by 7.2 points and exceeding the 30B reference, UI-Venus-1.5, by 6.2 points. The 4B variant already surpasses all listed non-ours baselines, including 72B references. This strong transfer suggests that ATMem does not merely memorize training-app interaction patterns; instead, it provides a more portable representation of task progress and task-relevant data state, enabling better generalization to new mobile applications and workflows.
Performance on our benchmark. DataScope directly probes the capabilities ATMem is designed to support, including tracking target data items across sources and maintaining execution scope over long cross-app workflows. As shown in Table 1, existing end-to-end agents struggle substantially on all difficulty levels.
On DC-V1, the strongest end-to-end baseline MAI-UI-8B obtains only 3.1% SR, 8.8% \(S_{\mathrm{prog}}\), and 10.7% \(S_{\mathrm{scope}}\). ATMem-UI-8B improves these to 6.2%, 11.7%, and
15.7%, yielding gains of \(+\)3.1, \(+\)2.9, and \(+\)5.0 points, respectively. The larger gain on \(S_{\mathrm{scope}}\) is
especially informative: it indicates that ATMem improves not only whether the agent makes progress, but also whether it operates on the correct set of required data items. This is precisely the failure mode targeted by ATMem’s explicit
Constraints field and item-level status tracking, which provide a persistent record of which data objects remain eligible, completed, or skipped.
| Method | Params | Data-Scope-V1 | Data-Scope-V2 | Data-Scope-V3 | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 3-5 (lr)6-8 (l)9-11 | SR | \(\boldsymbol{S_{\mathrm{prog}}}\) | \(\boldsymbol{S_{\mathrm{scope}}}\) | SR | \(\boldsymbol{S_{\mathrm{prog}}}\) | \(\boldsymbol{S_{\mathrm{scope}}}\) | SR | \(\boldsymbol{S_{\mathrm{prog}}}\) | \(\boldsymbol{S_{\mathrm{scope}}}\) | |
| GELab-Zero [25] | 4B | 0.0 | 4.2 | 3.9 | 0.0 | 3.6 | 6.4 | 0.0 | 2.6 | 4.4 |
| UI-TARS-1.5 [46] | 7B | 0.0 | 1.6 | 1.7 | 0.0 | 1.6 | 1.8 | 0.0 | 1.6 | 1.5 |
| GUI-Owl [15] | 7B | 0.0 | 3.6 | 5.7 | 0.0 | 1.6 | 1.1 | 0.0 | 1.6 | 1.5 |
| UI-Venus [24] | 7B | 0.0 | 1.6 | 3.8 | 0.0 | 1.6 | 0.6 | 0.0 | 0.0 | 0.7 |
| GUI-Owl-1.5 [21] | 8B | 4.7 | 7.3 | 0.0 | 2.6 | 0.0 | 2.6 | |||
| MAI-UI [20] | 8B | 3.9 | 0.0 | 4.2 | ||||||
| GUI-Owl [15] | 32B | 0.0 | 2.6 | 2.2 | 0.0 | 1.6 | 1.1 | 0.0 | 0.0 | 0.5 |
| UI-Venus-1.5-A3B [47] | 30B | 0.0 | 1.6 | 4.1 | 0.0 | 1.6 | 1.8 | 0.0 | 1.6 | 1.5 |
| ATMem-UI (Ours) | 8B | |||||||||
4.5 pt
| Training Recipe | AndroidWorld | MobileWorld | ||
|---|---|---|---|---|
| 2-3 (l)4-5 | SR (%) | Mem. (%) | SR (%) | Mem. (%) |
| Baseline | 47.6 | – | 9.4 | – |
| SFT | 70.7 | 48.2 | 17.2 | 49.6 |
| \(\Delta\) vs. Baseline | +23.1 | – | +7.8 | – |
| SFT + GRPO | ||||
| \(\Delta\) vs. SFT | +4.1 | -2.5 | +3.5 | -4.9 |
| SFT + STR-GRPO | ||||
| \(\Delta\) vs. SFT + GRPO | +1.8 | -13.8 | +2.6 | -14.6 |
| \(\Delta\) vs. SFT | +5.9 | -16.3 | +6.1 | -19.5 |
r0.58
8.5pt
As difficulty increases from DC-V1 to DC-V3, the distractor-to-target ratio rises from \(1.39\times\) to \(3.22\times\), and all agents’ performance drops sharply, confirming that denser
distractors and tighter data dependencies substantially increase task difficulty. ATMem-UI-8B degrades more slowly than all end-to-end baselines, maintaining the best end-to-end results at every level. On DC-V2, it achieves 6.2% SR, 9.4% \(S_{\mathrm{prog}}\), and 9.8% \(S_{\mathrm{scope}}\), improving over the best non-ours end-to-end result for each metric by 3.1, 5.2, and 2.4 points, respectively. On DC-V3, it achieves 3.1% SR,
6.3% \(S_{\mathrm{prog}}\), and 8.2% \(S_{\mathrm{scope}}\), with corresponding gains of 3.1, 2.6, and 3.2 points. The gains on both \(S_{\mathrm{prog}}\)
and \(S_{\mathrm{scope}}\) show that ATMem improves not only how far the agent progresses, but also how reliably it preserves the correct operation scope under dense distractors. This pattern is consistent with the role of
ATMem’s Constraints field and skipped status, which help maintain item-level eligibility information throughout execution and become increasingly important as distractor density grows.
We further include GPT-5.2+UIIns as a framework-level reference built on a strong proprietary planner with explicit external orchestration. It substantially outperforms all end-to-end agents on DC-V1, achieving 18.8% SR, 35.4% \(S_{\mathrm{prog}}\), and 38.4% \(S_{\mathrm{scope}}\). However, its SR also drops from 18.8% on DC-V1 to 3.1% on DC-V3, matching ATMem-UI-8B at the highest difficulty level. This convergence in task success is revealing: under high distractor density, stronger external orchestration alone is not sufficient to avoid failure, suggesting that fine-grained data discrimination and item-level eligibility judgment become critical bottlenecks. At the same time, the framework retains higher \(S_{\mathrm{prog}}\) and \(S_{\mathrm{scope}}\), indicating better partial execution quality and pointing to richer intermediate state representation as an important direction for end-to-end agents. The consistent gains of ATMem-UI across all difficulty levels demonstrate that structured memory is an effective step in this direction, without relying on external orchestration or proprietary planners.
We conduct an ablation study on Qwen3-VL-8B-Instruct to isolate the contribution of each training stage. In addition to SR, we report Mem., defined as the fraction of tasks in which the agent invokes ATMem at least once during execution, to measure how selectively the model applies structured memory across different tasks.
As shown in Table 2, the baseline achieves 47.6% and 9.4% SR on AndroidWorld and MobileWorld, respectively. After SFT, SR increases substantially by 23.1 and 7.8 points on the two benchmarks, with Mem. reaching 48.2% and 49.6%. To encourage more selective and effective memory invocation, we introduce STR-GRPO, which estimates the marginal utility of ATMem through paired memory-ON and memory-OFF rollouts.
As Table 2 suggests, Standard GRPO further improves SR by 4.1 and 3.5 points, and modestly reduces Mem. by 2.5 and 4.9 points. The reduction is real but limited. Through online interaction, the agent receives indirect feedback that some memory-active trajectories do not perform better than simpler ones, which weakly discourages unnecessary invocations. However, because task-level reward does not distinguish whether a successful trajectory succeeded due to ATMem or despite it, the agent cannot reliably identify which memory invocations are causally useful and which are redundant. As a result, Mem. remains elevated and the agent continues to apply the ATMem structure to tasks even without extra benefit.
STR-GRPO addresses this issue by estimating the marginal utility of ATMem through paired memory-ON and memory-OFF rollouts within each task group. This design provides an explicit signal for whether structured memory improves the verifier outcome, allowing the agent to suppress memory updates that do not contribute to success. Compared with standard GRPO, STR-GRPO reduces Mem. by 13.8 and 14.6 points while further improving SR by 1.8 and 2.6 points. This joint improvement indicates that many SFT-induced ATMem updates are redundant, and that STR-GRPO removes unnecessary invocations while preserving memory use where it is beneficial. Relative to SFT, STR-GRPO reduces Mem. by 16.3 and 19.5 points while improving SR by 5.9 and 6.1 points. These results show that STR-GRPO does not simply discourage memory use, but recalibrates ATMem from a frequently imitated behavior into a more selective, utility-driven mechanism.
Flat Notes vs.ATMem. To isolate the effect of memory structure, we run the same agentic framework on identical tasks, changing only the memory format. One setting uses recording-centric memory[14], while the other uses ATMem. Figure 5 shows a representative task where the agent must delete three specific expense entries from a list containing many same-schema distractors. The recording-centric agent stores the target names as a static list and starts searching for them. However, this memory format does not track which entries have already been deleted. After removing some targets, the agent revisits previously processed regions and eventually reaches the 100-step limit. This is a state-tracking failure. Without item-level completion status, the agent cannot reliably determine which targets remain.
ATMem changes the execution pattern by maintaining explicit status for each target item. The agent instantiates the three expense entries under Schema.ItemContent, assigns each status:remaining, and encodes the target condition
in the Constraints field. After each deletion, the corresponding item is updated to status:finished, so later decisions are conditioned on the remaining unfinished items. The task is completed in 31 steps. This comparison suggests
that structured status tracking converts open-ended search into a bounded checklist, reducing the ambiguity that causes flat-note agents to loop.
Representative Failure on DataScope. Figure 6 shows a representative failure on DataScope. The task requires sending an SMS to two direct phone numbers and all starred contacts, with the message content derived from a local HTML file. MAI-UI-8B correctly adds the first recipient and produces a coherent intermediate thought. However, it then repeatedly attempts to add the second direct number, cycling between the recipient field, the back button, and the compose screen. By Step 60, the second number is still not confirmed, and the starred contacts have not been processed. This failure arises from missing execution-state awareness. The agent has no persistent record that the first recipient has already been added and confirmed. Because the visible UI state is ambiguous, the agent repeatedly re-evaluates the same situation from interaction history alone and fails to advance to the next subtask. This pattern is especially common in DataScope, where multi-recipient and multi-item workflows require tracking which specific data units have already been processed.
ATMem in a Long-Horizon Trajectory. Figure 4 traces ATMem across a successful 58-step trajectory where the agent transfers three recipes from Simple Gallery Pro to the Broccoli recipe app.
During the harvest phase, the agent extracts all recipe fields and stores them as structured entries under Schema.ItemContent, each initialized with status:remaining. When execution begins, the stored content remains available
while the phase field indicates that the agent should start entering recipes into the target app. As each recipe is saved, ATMem updates the corresponding item to status:finished. This gives the agent an explicit record of completed and
pending items, allowing it to move to the next recipe without re-reading previous screenshots or revisiting completed entries. By the end of the trajectory, all three recipes are marked finished, and the agent terminates correctly. This trajectory
illustrates three roles of ATMem in long-horizon execution. During harvest, it stores structured task data that would otherwise need to be recovered from screenshots. During execution, item status provides a checklist for tracking progress. Across item
transitions, the memory state helps preserve the correct operation scope, enabling the agent to complete a long multi-item workflow without losing track of pending actions.
Training Dynamics under Memory Intervention. Figures 7 (a)–(c) show how ATMem usage evolves during STR-GRPO training and why selective memory invocation emerges. Figure 7 (a) reports outcome rewards under memory-ON and memory-OFF conditions throughout training. The memory-ON curve remains consistently higher than the memory-OFF curve, indicating that ATMem provides a positive contribution when it is retained. Importantly, this gap does not disappear as training progresses. Even after the model learns to invoke memory more selectively, the remaining memory-active cases still achieve higher reward under the memory-ON condition. This suggests that STR-GRPO does not improve performance by simply avoiding memory. Instead, it suppresses low-utility memory updates while preserving cases where structured memory remains beneficial.
Figure 7 (b) compares memory length under STR-GRPO and standard GRPO. Under standard GRPO, memory length remains close to its initial level, suggesting that task-level reward alone provides little pressure to reduce memory content. In contrast, STR-GRPO produces a substantial and sustained reduction in memory length. Read together with Figure 7 (a), this reduction is not accompanied by lower outcome reward. The results suggest that STR-GRPO mainly removes memory updates that provide limited marginal benefit, rather than compressing away useful structured state.
Figure 7 (c) directly measures this effect through the ON-minus-OFF reward gap within each task group. The marginal reward gain from ATMem stays positive across most of the training. It is larger in early training, when memory is used more broadly, and stabilizes at a smaller but sustained level after redundant memory calls are pruned. This pattern is consistent with selective invocation. As low-utility memory updates are removed, the remaining ON-OFF gap reflects the utility of ATMem on tasks that still benefit from explicit state tracking.
Failure Mode Analysis on DataScope. Figure 7 (d) breaks down the failure cases of MAI-UI-8B, the strongest end-to-end baseline. Stuck loops are the dominant failure mode at 40.9%, followed by wrong termination (21.9%), step cap (21.4%), and miscellaneous errors, including grounding failures and environment edge cases (15.8%).
The dominance of stuck loops provides a clear diagnosis of why current end-to-end agents struggle on data-scope tasks. Stuck loops occur when the agent cannot determine whether a target data item has already been processed, causing it to repeat the same action or revisit completed sub-tasks. The qualitative example in Figure 6 illustrates this pattern, and the aggregate statistics show that it is a recurring failure mode rather than an isolated case. This is precisely the failure that ATMem is designed to address. By maintaining an explicit execution status for each item, ATMem allows the agent to identify which items remain pending and which have been completed, without relying only on implicit inference from interaction history. Wrong termination and step cap failures reflect different remaining challenges. Wrong termination occurs when the agent declares completion before all required data operations are verified, pointing to limitations in completion checking. Step cap failures occur when the agent remains in a valid but unresolved execution state and fails to converge within the allowed horizon, pointing to limitations in long-horizon planning and recovery. These failures are not fully solved by item-level status tracking alone. They therefore represent the primary remaining challenges for end-to-end agents on data-scope workflows beyond what ATMem directly addresses.
We presented ATMem, an active task-driving memory framework for long-horizon mobile GUI agents. Our central argument is that memory in mobile execution should not be treated as a passive archive of past observations. Instead, task-relevant data should be maintained as an evolving execution state that records what has been collected, what remains pending, which constraints apply, and which operations have already been completed. ATMem operationalizes this view by organizing task data into structured units with ownership, role, constraints, and status, allowing the agent to condition its next action on current workflow progress rather than relying only on raw interaction history. To make this memory behavior efficient, we introduced STR-GRPO. By comparing memory-on/off rollouts and incorporating memory cost into online RL, STR-GRPO teaches the agent not only how to construct ATMem, but also when ATMem is actually useful. This turns memory invocation from an always-active behavior learned through imitation into a utility-driven policy decision. We also introduced DataScope, a long-horizon online benchmark that stresses cross-page and cross-application workflows involving data collection, verification, transfer, update, and reuse. By requiring agents to cover all instruction-relevant targets while filtering same-schema distractors, DataScope exposes failure modes that are difficult to observe from terminal success rate alone. Across AndroidWorld, MobileWorld, and DataScope, ATMem-UI improves long-horizon execution while reducing unnecessary memory usage. Our failure analysis further shows that stuck loops, premature termination, and incomplete operation tracking remain key bottlenecks.
Long-horizon mobile-agent training requires rollouts to be collected through real multi-turn interactions with the environment. This makes a standard synchronous RL pipeline inefficient, since policy updates must wait for many long and heterogeneous
trajectories to finish. The trajectories themselves are also expensive to optimize over: each rollout may concatenate multiple GUI observations, actions, reasoning traces, and ATMem states, resulting in extremely long multimodal contexts that exceed
single-GPU memory limits. Following MAI-UI [20], we build an asynchronous on-policy RL framework based on verl and HybridFlow [48] to support scalable online training.
Asynchronous rollout collection. We implement a customized agent loop that asynchronously dispatches rollout requests to a pool of inference servers hosting the latest policy model. This design reduces GPU idle time caused by long multi-turn environment interactions and improves rollout throughput. The agent loop also supports asynchronous environment interaction and session management, including backup sessions for failure recovery and replacement. On the inference side, request load balancing and prefill caching are used to accelerate generation under long interaction histories. Although rollouts are collected asynchronously, training remains on-policy: each rollout batch is generated by the current policy version and stored with the corresponding behavior log probabilities for policy optimization.
Hybrid parallelism for long trajectories. To optimize policies over ultra-long trajectories, we adopt Megatron-style hybrid parallelism, including tensor parallelism, pipeline parallelism, and context parallelism. This partitions both model computation and long sequence contexts across multiple GPUs, allowing end-to-end policy updates while keeping per-GPU memory usage manageable. We additionally downsample GUI screenshots to half resolution, which reduces visual-token length and training cost without degrading empirical performance.
Overall, this infrastructure enables stable online RL for long-horizon mobile agents by decoupling rollout collection from policy optimization, improving inference throughput, and supporting training over extremely long multimodal trajectories.
We provide the training infrastructure and main hyperparameters in Table 3. For SFT, we train on verified rollout trajectories using 128 NVIDIA H20 GPUs across 16 nodes. Each node contains 8 GPUs and a 96-core CPU with approximately 900GB of memory. The SFT stage takes about 2 hours. We train the language model while freezing the vision encoder, using bfloat16 precision, FlashAttention, gradient checkpointing, Liger kernel acceleration, and DeepSpeed ZeRO-2 offload.
For online RL, we use the same 128-GPU training configuration and train for about 3 days. The mobile environments are deployed on bare-metal servers. In our main run, we use 128 active environment containers for rollout collection and maintain additional backup containers for failure recovery. Each environment server has 96 CPU cores and 384GB of memory. The rollout system sets the maximum response length to 81,920 tokens. The RL actor is initialized from the SFT checkpoint.
The RL system is built on a verl-based multi-turn agent training stack. For generation and policy optimization, we use tensor parallelism with degree 2, context parallelism with degree 2, and pipeline parallelism with degree 1. The rollout
engine uses a GPU memory utilization ratio of 0.55, disables replay, and performs checkpointing and evaluation every epoch.
During evaluation, our agent uses the structured execution prompt in Box [box:agent95execution95prompt]. It constrains each response to include reasoning, a single executable tool call, and an optional ATMem memory update.
## Task. You are a GUI agent. You are given a task goal, previous action history, and screenshots of the current screen. You need to perform the NEXT action to complete the task.
## Output Format. For each step, output exactly THREE blocks in this order:
<thinking>
... return the thinking process
</thinking>
<tool_call>
{"name": "mobile_use", "arguments": <args-json-object>}
</tool_call>
<memory>
{ ...valid JSON dict or {} ... }
</memory>
Rules.
## Action Space. Choose exactly one action per step:
| Action | JSON Format |
|---|---|
open_app |
{"action": "open_app", "app_name": "<Available Apps>"} |
click |
{"action": "click", "coordinate": [x, y]} |
long_press |
{"action": "long_press", "coordinate": [x, y]} |
input_text |
{"action": "input_text", "text": ""} |
scroll |
{"action": "scroll", "direction": "up|down|left|right"} |
drag |
{"action": "drag", "coordinate": [x, y], "direction": "up|down|left|right"} |
navigate_back |
{"action": "navigate_back"} |
navigate_home |
{"action": "navigate_home"} |
wait |
{"action": "wait"} |
status |
{"action": "status", "goal_status": "complete|infeasible"} |
answer |
{"action": "answer", "text": ""} |
## Hard Constraints.
Use answer only when the task can be completed without interacting with the device UI.
Otherwise, use device actions and do not answer directly.
Output goal_status="complete" only after verifying the result on screen.
## When to Use Memory. Use <memory> only when cross-step storage or comparison is needed; otherwise output {}.
## Memory Schema. When used, memory must follow this structure:
{
"phase": "harvest" | "execute",
"context": {},
"schema": {
"fields": []
},
"items": {
"1": {
"content": {},
"status": "remaining" | "finished" | "skipped",
"skipReason": ""
}
}
}
Rules.
## Memory Usage Protocol.
If using memory, start with phase="harvest" and switch to phase="execute" once enough information is collected.
Update an item status to finished in the same step after a successful operation.
| Configuration | SFT | Online RL |
|---|---|---|
| Base model | Qwen3-VL-4B/8B-Instruct | SFT checkpoint |
| Training GPUs | 128 NVIDIA H20 | 128 NVIDIA H20 |
| Node setup | 16 nodes, 8 GPUs/node | 16 nodes, 8 GPUs/node |
| CPU / memory per training node | 96 cores / \(\sim\)900GB | 96 cores / \(\sim\)900GB |
| Training time | \(\sim\)2 hours | \(\sim\)3 days |
| Precision | bfloat16 | bfloat16 |
| Max sequence / response length | 20,768 | 81,920 |
| Vision encoder | Frozen | – |
| Parallelism | ZeRO-2 offload | TP=2, PP=1, CP=2 |
| Batch size | 1 per GPU | global 32, mini-batch 32 |
| Rollout group size | – | 16 |
| Screenshot resolution | max pixels 12,845,056 | \(1600\times720\) |
| Environment servers | – | 3 bare-metal servers |
| Environment server config | – | 96 cores / 384GB each |
| Environment containers | – | 200 total, 128 active |
5pt
This section provides detailed statistics of our Data-Centric benchmark. Each task family is designed as a data-dependent mobile workflow involving one or more applications, where the agent must identify target data, avoid confusable distractors, and perform the required data operations. Here, confusable distractors refer to non-target entries that share the same schema or data type with target entries and are generated with template-level similarity constraints. To construct controlled difficulty levels, we instantiate the same task families across DC-V1, DC-V2, and DC-V3 while progressively increasing the number of target entries, confusable distractors, and required operations. This design keeps the underlying app workflow largely comparable across subsets, while scaling difficulty through data volume and distractor density.
Table 1 reports task-family-level details. For each subset, T/N/O denotes the number of target entries, confusable distractors, and required operations, respectively. The application column lists the apps involved in each workflow using color-coded labels. A “–” indicates that the corresponding task variant is not included in that subset after filtering unreachable or invalid instances.
| Task Family | Apps | #Apps | DC-V1 | DC-V2 | DC-V3 |
|---|---|---|---|---|---|
| Task Family | Apps | #Apps | DC-V1 | DC-V2 | DC-V3 |
| Continued on next page | |||||
| CalendarFocusTracksRetroPlaylist | |||||
| 2 | 0/4/3 | /13/3 | /20/3 | ||
| CalendarMeetingPrepAndNotify | 1 | /13/5 | /43/20 | /100/36 | |
| CalendarTodayAlarmsInClock | |||||
| 2 | 2/7/2 | /10/3 | /17/4 | ||
| CallLogContactTodoMarkorSync | |||||
| 4 | 2/0/2 | /6/3 | /7/3 | ||
| CallLogTopContactsFavorite | |||||
| 2 | 13/1/13 | /14/18 | /29/22 | ||
| ChromeInviteCodeAndSmsSend | |||||
| 2 | 3/4/3 | /7/4 | /18/9 | ||
| ExpensePurgeHousingFoodLogMarkor | |||||
| 2 | 0/0/3 | /0/3 | /0/5 | ||
| FilesClosedProjectContactDelete | |||||
| 2 | 2/4/2 | /12/3 | /35/3 | ||
| FilesDatedReimburseTxtToExpense | |||||
| 2 | 3/0/3 | /0/3 | /0/4 | ||
| FilesPartyMenuRecipesToBroccoli | |||||
| 2 | 2/0/2 | /0/2 | /0/2 | ||
| FilesSetlistExportRetroM3u | |||||
| 2 | 0/3/4 | /6/4 | /9/4 | ||
| FilesWorkEventsToCalendar | |||||
| 2 | 3/6/3 | /10/6 | /17/8 | ||
| FilesWorkMergeToMarkorCsv | |||||
| 2 | 3/3/3 | /3/4 | /8/6 | ||
| GalleryImageListToMarkor | |||||
| 2 | 4/1/4 | /6/4 | /12/6 | ||
| GalleryVlcArtistPlaylistsFromImages | |||||
| 2 | 2/3/2 | /12/3 | /21/4 | ||
| MarkorTaskAssignmentDistribute | 1 | /5/2 | /25/4 | /60/5 | |
| RecipeNutFreePurgeLogMarkor | |||||
| 2 | 6/3/6 | /3/7 | /3/9 | ||
| RetroLightMusicPartyMarkorSms | |||||
| 3 | 2/2/2 | /6/2 | /8/3 | ||
| SmsAgentPhoneToMarkorTxt | |||||
| 2 | 4/10/4 | /7/4 | /30/6 | ||
| SmsAlbumQueueRetroMarkor | |||||
| 3 | 2/8/2 | /13/2 | /21/2 | ||
| SmsColleagueTripReimburseToExpense | |||||
| 2 | 1/4/1 | /4/1 | /15/10 | ||
| SmsDavidBugfixToCalendar | |||||
| 2 | 3/6/3 | /8/4 | /10/6 | ||
| SmsDavidClientsToContacts | |||||
| 2 | 3/7/3 | /18/3 | /48/4 | ||
| SmsDavidTodoToTasks | |||||
| 2 | 1/3/1 | /10/2 | /25/3 | ||
| SmsExpenseAuditAndReport | |||||
| 2 | 0/2/2 | /6/6 | /15/12 | ||
| SmsFamilyRecipeToBroccoli | |||||
| 2 | 2/4/2 | /9/3 | /13/4 | ||
| SmsFriendRecommendationsVlcPlaylists | |||||
| 2 | 2/3/2 | /13/5 | /20/7 | ||
| SmsPotluckRecipeBroccoliCalendar | |||||
| 3 | 4/6/4 | /6/4 | /9/5 | ||
| SmsStockDataToMarkorCsv | |||||
| 2 | 2/5/2 | /12/4 | /27/7 | ||
| SmsWorkPlanToMarkorTodo | |||||
| 3 | 1/3/1 | /10/3 | /25/5 | ||
| SmsWorkoutPlaylistRetroMarkor | |||||
| 3 | 1/3/1 | /9/1 | /20/1 | ||
| VlcPlaylistsToMarkorMdFiles | |||||
| 3 | 2/8/2 | /11/2 | /17/2 | ||
To illustrate our data-centric workflow specification, Listing [lst:workflow_template_example] presents a compact example from one of
our 32 workflow templates. The example is shortened for readability but preserves the main fields used in benchmark construction. The task, goal, and apps fields define the task family, user-facing instruction, and
involved mobile applications. The custom_params_template and initialization_example specify the concrete data injected into the environment, including target files, app-specific records, and confusable distractors. The
expected_output_example field defines the required terminal state after successful execution, while _verification lists the app-level checks used by the verifier. The task_extension_suggestions field further describes
how the same workflow can be instantiated with different data values and scaled to harder variants, such as adding more target entries or distractors. Together, these fields bind the task goal, environment initialization, expected outcome, and automatic
verification into a single executable task specification.
Following AndroidWorld’s [7] app-state-based task construction practice, we implement app-specific initialization and verification functions for each workflow template. For each involved application, the initializer writes the required target data and confusable distractors into the corresponding app storage, such as SQLite-backed records or file-system entries. This ensures that all task-relevant data and distractor data are visible to the agent through normal GUI interaction, rather than being provided only in the instructions.
For each task family, we further implement a task-specific verifier that checks whether the final environment state satisfies the expected outcome. The verifier inspects app-level terminal states, such as created files, modified records, sent messages, calendar entries, contact updates, or untouched distractors, and converts them into rule-level scores. This enables automatic evaluation of both terminal success and partial app-/data-level progress.
To validate verifier reliability, we perform verifier calibration with controlled terminal-state variants. Given the expected result of each task, GPT-5.2 generates multiple perturbed terminal states, including successful completion, missing one or more required data operations, extra operations on non-target data, missing operations in a particular app, and incorrect field values. We then initialize the environment directly with each controlled terminal state, treating it as a simulated post-execution state, and run the corresponding verifier. A verifier is retained only if its score matches the expected score for all controlled variants; otherwise, the task case is returned for manual correction. This calibration process helps ensure that our automatic evaluators correctly distinguish complete success, partial completion, over-operation, and incorrect data manipulation.
To instantiate diverse task variants from each workflow template, we use an LLM-based task generator with a structured system prompt. The prompt constrains generation to the same logical task class as the source template: the involved applications, cross-app workflow, output types, and structural requirements must remain unchanged, while concrete data values and scenario entities can vary. Difficulty is controlled only through the number of target data entries and confusable distractors, rather than by adding new steps or extra instructions. The prompt also requires each generated variant to include initialization data and expected outputs following the source template schema, so that the resulting task can be initialized in the environment and evaluated automatically by the verifier. A shortened version of the system prompt is shown in Box [box:task95generation95prompt].
## Task. Generate feasible GUI automation task variants for an Android long-horizon benchmark.
## Inputs.
requested_num_tasks: exact number of variants to output.
per_variant_difficulty: ordered list of difficulty labels.
full_benchmark_template: complete benchmark JSON for one task family.
Optional trajectory screenshots: UI references only.
## Task-Class Constraints. Generate variants for the same logical task class as the source template.
Keep the same apps, cross-app workflow, and output types.
Scenario wording and concrete values may change, but apps, steps, and structural requirements must not be added or removed.
task_goal and cn_task_goal should remain concise, specific, and agent-actionable.
## Difficulty Model. Difficulty is increased only through target data volume and confusable distractor density, not by adding new steps or extra instructions.
| Difficulty | Operational / Target Data | Noise / Distractor Data |
|---|---|---|
easy |
–2 target sub-groups. | –5 noise items, clearly distinct from targets. |
medium |
–3 target sub-groups; roughly 1.5\(\times\) easy counts. | –8 noise items with structural similarity. |
hard |
–5 target sub-groups; roughly 2\(\times\) easy counts. | –12 highly similar noise items, including near-duplicate names, formats, or partial-match fields. |
very_hard |
–8 target sub-groups; roughly 3\(\times\) easy counts. | –20 highly confusable noise items, including trap entries with one missing or close-but-wrong field. |
## Reference and Ground Truth. When an lh_v1_reference is provided, use it as the authoritative reference for:
task_goal and cn_task_goal phrasing;
expected_output structure, including top-level keys, nesting, and field names;
init_data.custom_params schema.
Only concrete values, counts, and scenario entities may change.
## Expected Output Rules.
Fill expected_output to exactly match what a correct agent should produce for the current variant.
Use the source template’s expected_output_example as the structural reference.
Do not emit rule_validation; it is derived automatically from expected_output.
Do not emit keys starting with _ inside expected_output.
## Initialization Data.
init_data must not be null.
Follow the structural layout of full_benchmark_template.initialization_example.
Ensure unique names, phone numbers, and IDs for all contacts or records.
Strip all _-prefixed keys from emitted initialization data.
## Output Format. Return a single valid JSON object only. Do not include markdown fences, comments, or extra text.
{
"task_type": "<exact task type from source template>",
"variants": [
{
"variant_id": 1,
"difficulty": "easy|medium|hard|very_hard",
"task_goal": "<same task class; values may change>",
"cn_task_goal": "<Chinese task goal>",
"init_data": {
"task_type": "<same as task_type>",
"custom_params": { "<seeded data fields from template>" }
},
"expected_output": { "<ground truth with reference structure>" },
"rationale": "<=120 words describing target/noise counts>"
}
]
}
Listing lst:workflow_template_example: Compact example of a data-centric workflow template. Arrays are shortened for readability while preserving the main template structure.
{
"task": "CalendarMeetingPrepAndNotify",
"goal": "Check next week's meetings in Simple Calendar Pro. For each target meeting, find related pending tasks in Tasks whose notes mention the meeting title, create a Markor agenda file named {meeting_title}_agenda.md, look up attendee phone numbers in Contacts, and send the agenda via Simple SMS Messenger.",
"apps": [
"Simple Calendar Pro",
"Tasks",
"Contacts",
"Markor",
"Simple SMS Messenger"
],
"task_type": "long-horizon",
"_why_hard": [
"Five apps are chained: Calendar -> Tasks -> Markor -> Contacts -> SMS.",
"Target meetings, tasks, and contacts are mixed with confusable distractors.",
"The SMS body must match the corresponding meeting agenda."
],
"task_extension_suggestions": {
"1": "Increase the number of target meetings; each meeting requires an independent agenda file and SMS messages.",
"2": "Increase noise_events, noise_tasks, and noise_contacts.",
"3": "Add attendees without matching contacts to test skipping or fault tolerance.",
"4": "Require strict agenda filename or section formatting."
},
"difficulty_control": {
"easy": {
"target_meetings": 1,
"attendees_per_meeting": 2,
"related_tasks_per_meeting": 2,
"noise_events": 3,
"noise_tasks": 5,
"noise_contacts": 5
},
"medium": {
"target_meetings": "2-3",
"attendees_per_meeting": "2-4",
"related_tasks_per_meeting": "2-5",
"noise_events": 8,
"noise_tasks": 15,
"noise_contacts": 20
},
"hard": {
"target_meetings": "3-4",
"attendees_per_meeting": "3-6",
"related_tasks_per_meeting": "3-8",
"noise_events": 15,
"noise_tasks": 35,
"noise_contacts": 50
}
},
"custom_params_template": {
"calendar_events": {
"target_events": [
{
"title": "Vendor Contract Sync",
"date": "2023-10-24",
"time": "10:00",
"duration_mins": 60,
"location": "Meeting Room 2B",
"description": "Align on contract renewal timeline and action items.",
"attendees": ["Nina Patel", "Omar Reyes"]
}
],
"noise_events": [
{
"title": "Team Lunch",
"date": "2023-10-25",
"time": "12:00",
"duration_mins": 60,
"description": "Lunch at the cafe.",
"attendees": []
}
]
},
"tasks_app_data": {
"target_tasks": [
{
"title": "Draft agenda for vendor sync",
"completed": 0,
"deleted": 0,
"notes": "For Vendor Contract Sync: propose topics and timing."
},
{
"title": "Collect renewal milestones",
"completed": 0,
"deleted": 0,
"notes": "Needed for Vendor Contract Sync."
}
],
"noise_tasks": [
{
"title": "Buy dish soap",
"completed": 0,
"deleted": 0,
"notes": null
}
]
},
"contacts_data": {
"target_contacts": [
{
"name": "Nina Patel",
"number": "+13661100111"
},
{
"name": "Omar Reyes",
"number": "+13661100112"
}
],
"noise_contacts": [
{
"name": "Mia Chen",
"number": "+13661100202"
}
]
},
"output_config": {
"markor_file_template": "{meeting_title}_agenda.md",
"markor_content_format": "# {meeting_title}\n- Date: {date}\n- Time: {time}\n- Location: {location}\n- Attendees: {attendees}\n\n## Related Tasks\n{task_list}",
"sms_format": "Agenda for {meeting_title} ({date} {time}): {task_list}. Location: {location}."
}
},
"initialization_example": {
"task_type": "CalendarMeetingPrepAndNotify",
"task_idx": 0,
"custom_params": {
"calendar_events": "instantiated from custom_params_template.calendar_events",
"tasks_app_data": "instantiated from custom_params_template.tasks_app_data",
"contacts_data": "instantiated from custom_params_template.contacts_data",
"output_config": "instantiated from custom_params_template.output_config"
}
},
"expected_output_example": {
"markor_files": [
{
"file_name": "Vendor_Contract_Sync_agenda.md",
"content": "# Vendor Contract Sync\n- Date: 2023-10-24\n- Time: 10:00\n- Location: Meeting Room 2B\n- Attendees: Nina Patel, Omar Reyes\n\n## Related Tasks\n- Draft agenda for vendor sync\n- Collect renewal milestones\n"
}
],
"sms_sent": [
{
"to": "Nina Patel (+13661100111)",
"for_meeting": "Vendor Contract Sync",
"body": "Agenda for Vendor Contract Sync: Draft agenda for vendor sync; Collect renewal milestones. Location: Meeting Room 2B."
},
{
"to": "Omar Reyes (+13661100112)",
"for_meeting": "Vendor Contract Sync",
"body": "Agenda for Vendor Contract Sync: Draft agenda for vendor sync; Collect renewal milestones. Location: Meeting Room 2B."
}
]
}
}