1 Introduction↩︎

Commercial Digital Adoption Platforms (DAPs) overlay guidance on web systems to help new users operate complex, unfamiliar interfaces. When a user faces an unfamiliar page, the platform highlights the element that matters and shows a short instruction next to it (Figure 1 (a)). This convenience rests on manual labor: vendors locate the target element of every step and write its guide text, and every redesign forces both to be redone. Because one goal often spans several pages and a chain of dependent operations, guidance for a whole system is expensive to author and maintain.

Web agents are a natural way to remove this labor, but the two relevant research lines have developed separately. Agent benchmarks score task completion alone [1]–[4]: none asks the agent to produce, or scores, the instruction a human would need at each step. Guide generation has been studied in the opposite direction, producing guidance for one given page [5] without executing anything, on textual page representations, although multimodal agents show that acting from rendered screens is practical [6]–[8].

We address this gap with MAG, a benchmark for Multimodal Action and Guide generation. To our knowledge, it is the first benchmark in which an agent must both complete a multistep task on a live website and write, at every step, the guide sentence a future user would need at that point (Figure 1 (b)). The agent observes the site through screenshots and acts under one of two grounding schemes: Set-of-Mark selection [9] over numbered elements, or raw pixel coordinates. MAG builds on the six live websites of WebArena [1]: 581 tasks with verified success demonstrations, split into 407 training and 174 test. For 563 of them the demonstrations are reannotated into gold trajectories, 4,760 Set-of-Mark and 5,779 coordinate steps, each step carrying a guide drafted by an LLM and then corrected by human annotators.

We release the full harness around the benchmark: annotation pairing LLM drafts with human correction, supervised and reinforcement training pipelines, live evaluation with functional checkers and an LLM judge, and joint metrics for task success and guide quality. Every successful run yields a verified outcome and a candidate guide (Figure 1 (c)), reducing guide authoring to review. Benchmarking three frontier API models and an open 9B model under this protocol shows the task is far from solved: the best configuration completes 37.4% of the test tasks. It also shows that grounding is a real design choice with no universal answer: Gemini is far stronger with Set-of-Mark selection, GPT-5.5 and Claude show no significant preference, and the trained 9B model improves only under Set-of-Mark grounding (13.2% versus 9.2%).

Finally, we study how far training can push the small model. Supervised finetuning on the gold demonstrations teaches the output format but not the task: with Set-of-Mark grounding it reaches 6.9%, below the 8.1% of the untuned base, because the tuned model learns to declare completion too early. Plain GRPO [10] then stalls: all-fail groups carry no reward variance and yield no gradient, an issue also reported in large scale RL systems [11]. Injecting cached expert trajectories from a frontier model into the groups, in the spirit of off-policy guidance [12], restores the signal: Set-of-Mark success climbs from 6.9% to 13.2% (coordinate grounding does not benefit), guide quality improves as well, and a pass@6 analysis shows new capability rather than sharpened sampling. The remaining gap concentrates on long tasks: every 9B variant stays near zero beyond nine gold steps, where the API models still complete 26 to 34%.

We make three contributions.

• The MAG task and, to our knowledge, the first benchmark to unify multistep web task execution with guide generation over screenshots, with two grounding schemes and a human verified gold guide at every step.

• A full harness from annotation to live evaluation and joint metrics, used to benchmark frontier and open models, exposing model specific grounding preferences and a shared long horizon gap.

• A GRPO recipe augmented with expert trajectories that nearly doubles the Set-of-Mark success rate of a supervised 9B agent (6.9% to 13.2%) while improving its guides, verified with significance tests and a pass@k study.

2 Related Work↩︎

Web agents from text to vision. Web agents have progressed from synthetic platforms [13] to benchmarks that score functionally verified tasks on realistic sites [1]–[3], [14], [15], typically from a textual page abstraction on which strong agents layer prompting or search [16]–[18]. A second line moves to what users see: Set-of-Mark prompting overlays numbered marks on the screenshot [9]; SeeAct, WebVoyager, CogAgent, and Pix2Act act from rendered pages or raw pixels [6]–[8], [19]; dedicated grounding models locate elements from pixels [20], [21]; and benchmarks now span visual web tasks, desktops, and phones [4], [22], [23]. In all of these the agent emits only actions and is judged by completion alone.

Guide generation for web interfaces. Commercial DAPs attach human written guidance to production sites, and keeping it aligned with evolving interfaces is a recognized maintenance burden. Research on automating this is thin. [5] formulate guide generation for a single given page: pick the element a user should interact with and write the matching instruction. Nothing is executed, so the guide is never verified against task success, and assistance stops at one page.

Reinforcement learning for web agents. GRPO [10], [24] made critic free reinforcement learning practical for LLMs; prompted reflection improves agents without weight updates [25]; WebRL and WebAgent-R1 apply online RL to web agents [26], [27]. Two failure modes shape our recipe. A group whose rollouts all fail carries zero advantage, which DAPO counters with dynamic sampling [11]; and on-policy exploration cannot discover what the policy cannot yet do, which LUFFY counters by mixing off-policy expert traces into the groups [12]. Binary rewards from live websites make both problems severe for a 9B agent.

Our work. MAG differs from all three lines at the level of the task: the agent must act and explain in the same step, and both outputs are scored. The coupling is not cosmetic: a guide is trustworthy only if the step it describes advances the task, and live execution is what verifies this. Against agent benchmarks, MAG adds a second supervised output; against single page guide generation, it covers whole tasks on live sites from the visual inputs users see, and it offers, to our knowledge, the first controlled comparison of Set-of-Mark and coordinate grounding on identical tasks. On the training side, we carry off policy guidance to multimodal web RL and show it is the enabling component: without expert traces, GRPO yields no lift here.

3 The MAG Benchmark↩︎

3.1 Task Definition↩︎

An MAG episode is a triple \((q, s_0, \Phi)\): a natural language intent \(q\), an initial page state \(s_0\) on one of six live websites, and a functional checker \(\Phi\) inherited from WebArena. At step \(t\) the agent receives an observation \(o_t\) and the history of its own guides \(g_{<t}\), and must produce an action and a guide sentence jointly: \[(a_t, g_t) = \pi_\theta\!\left(q,\, o_t,\, g_{<t}\right). \label{eq:policy}\tag{1}\] The guide history is the only text state carried across steps: what the agent tells the user is also what it remembers.

Both schemes receive the identical observation: a \(1440\times900\) viewport screenshot \(x_t\) in which interactive elements are marked with numbered boxes, and a candidate menu \(C_t=\{(j,\, e_j,\, v_j)\}_{j=1}^{n_t}\) listing each mark’s element type \(e_j\) and visible text \(v_j\). The menu also lists elements beyond the visible fold, which scroll brings into view, so \(o_t = (x_t, C_t)\) throughout and the input side is held fixed. The two schemes differ only in how the action is grounded. An action is a triple \[a_t = (\alpha_t,\, \rho_t,\, \omega_t),\qquad \alpha_t \in \mathcal{A}, \label{eq:action}\tag{2}\] with a verb \(\alpha_t\) from the seven verb space \(\mathcal{A}=\{\)click, type, select, scroll, press_enter, go_back, finish\(\}\), a grounding argument \(\rho_t\), and a payload \(\omega_t\) (text to type, an option label, a scroll direction, or the final answer). The schemes instantiate \(\rho_t\) differently: \(\rho_t \in \{1,\dots,n_t\}\) picks a mark under Set-of-Mark (SoM) grounding, while under coordinate grounding \(\rho_t\) is a pixel position on the page; \(\rho_t\) is required exactly for the element verbs click, type, and select. Prompt, observation, budget, and scoring are all held identical, so any performance difference between the schemes is attributable to the grounding of the action itself.

The episode ends when the agent emits finish or after \(H=25\) steps, and success is judged functionally on the live site: \[S = \Phi\!\left(s_{T+1},\, \omega_T\right) \in \{0, 1\}, \label{eq:success}\tag{3}\] where \(T\) is the index of the last executed step, \(s_{T+1}\) the final page state, and \(\omega_T\) the answer returned with finish; if the budget expires without finish, \(\omega_T\) is empty, so checkers that require an answer score \(0\) while state based checkers still evaluate the final page. The defining constraint of MAG is the asymmetry between the two outputs: the action may use marks or pixels, but the guide \(g_t\) must be a short instruction a user could follow on the visible page, free of mark ids and coordinates. Every step is solved twice, in machine terms and in human terms, and Section 3.3 scores both.

3.2 Dataset Construction: LLM-Assisted Human Annotation↩︎

Section 3.1 fixes an interface that no existing resource fills: WebArena ships tasks and checkers, but no demonstrations in either grounding form and no guide text at any step. We therefore build MAG on top of the success demonstrations released by OpAgent [28]: trajectories for 581 of WebArena’s 812 tasks, recorded in coordinate form and verified by the source authors, which we convert and enrich in the pipeline of Figure 2. MAG inherits exactly those tasks, so the benchmark leans toward tasks a strong prior agent could complete, and its success rates are not comparable to WebArena leaderboard numbers.

The first stage replays every step on the live sites and saves a Set-of-Mark screenshot of the page as it looked at that step, mapping each recorded coordinate onto the marked candidate it hits. One recorded trajectory thus yields a coordinate view and a SoM view of the same behavior, the duality behind the controlled grounding comparison of Section 5; the views share their tasks, though a step whose target maps to no marked candidate survives only in the coordinate view (Appendix 8). The second stage adds the guides: a full web task can be long and entangled, but writing the instruction for one step, given the screenshot, the task, the action taken, and the guides so far, is squarely within the competence of a frontier LLM, so GPT-5.5 [29] drafts a think rationale and a guide sentence for every step.

Because LLM drafting alone is not trustworthy, two review mechanisms follow. A rule based filter drops steps with replay mismatches, malformed guides, actions outside the seven verb space, or unmapped targets; the survivors then pass a human stage in a review interface purpose built for MAG (Appendix 9), where three annotators step through every guide next to its screenshot and correct it on the spot. Their consistent report: the drafts were largely correct, and few corrections were needed. The result is 563 fully annotated tasks (396 training, 167 test) with a screenshot, both action forms, and a guide at every step, 4,760 SoM and 5,779 coordinate gold steps; the 407/174 split is fixed over the 581 source tasks before capture. Statistics and licensing are in Appendix 8.

3.3 Evaluation Suite↩︎

MAG contributes more than a dataset: it comes with an evaluation suite designed for the dual output of Equation 1 . Task success alone says nothing about whether the produced guides could walk a user through the task; text overlap alone rewards a fluent guide attached to a failed trajectory. The suite therefore scores the task, the guides, and a fused headline that credits guides only on solved tasks.

3.3.0.1 Task success.

SR is judged on the live site by WebArena’s functional checkers (exact URL, page content, and program queries), applied to the final state and answer as in Equation 3 ; 29 of the 174 test tasks need semantic answer comparison and are scored by a fixed LLM judge that sees only the answer and the reference.

3.3.0.2 Guide quality.

For task \(i\), the predicted guides \(\hat{g}^{(i)}\) are joined in step order and compared to the joined gold guides \(g^{\star(i)}\) with four reference metrics of equal weight, \(\mathcal{M} = \{\)BLEU-1, BLEU-2, ROUGE-1, ROUGE-L\(\}\) [30], [31]: \[G_i = \tfrac{1}{4}\!\sum_{m\in\mathcal{M}} m\!\left(\hat{g}^{(i)}, g^{\star(i)}\right)\!. \label{eq:guideq}\tag{4}\]

3.3.0.3 Gated Guide Score.

The headline metric gates guide quality by success: \[\mathrm{GGS}_i = S_i\left(\gamma + (1-\gamma)\, G_i\right), \qquad \gamma = 0.4, \label{eq:ggs}\tag{5}\] averaged over all 174 test tasks (reference guides exist for every test task, including the seven outside the released training corpus). The gate encodes the product requirement: a failed task scores zero regardless of guide fluency, while on a solved task the guides modulate the score between \(\gamma\) and \(1\). By construction \(\gamma\,\mathrm{SR} \le \mathrm{GGS} \le \mathrm{SR}\), and we call the gap \(\mathrm{SR} - \mathrm{GGS}\) the guide tax: the score a system loses to imperfect guides. Because GGS is coupled to SR by design, we always report SR and the ungated \(G\) next to it.

3.3.0.4 Format gate.

OFCR (Output Format Correctness Rate) is the fraction of steps that parse under the output contract, form a valid action in the seven verb space with the required arguments, and keep the guide free of leaked internals: a guide that mentions a mark id or a pixel coordinate fails the gate, since it is useless to a user who sees neither.

3.3.0.5 Step diagnostics and protocol.

For training time analysis we additionally score teacher forced steps with SAA (action correctness against gold) and GACS, a fused step score \(F \cdot A \cdot \sqrt{\mathrm{Faith}\cdot\mathrm{Suff}}\); its gold free Faith and Suff terms double as guide diagnostics during training, while the GRPO reward itself is binary task success (Section 4.3). Every reported run follows one protocol: environment reset and fresh authentication before each sweep, greedy decoding for locally served models (API models use fixed provider settings), the same locked prompt, and the 25 step budget. Formal definitions and the evaluation pseudocode are in Appendix 11.

4 The MAG Agent↩︎

4.1 The MAG Harness↩︎

Everything in this paper, the three API baselines, the SFT corpus, every GRPO rollout, and every reported number, runs through one harness (Figure 3a). Its job is to turn a live website and a vision language model into a loop stable enough to train against and fair enough to compare across models; much of the difficulty of MAG lives here, and we release it in full.

On the input side, each step renders the page in a \(1440\times900\) viewport, marks the interactive elements, and builds the candidate menu of Section 3.1, up to 130 lines of [id] TYPE visible text with elements beyond the fold included. The locked system prompt fixes the output contract: the five JSON keys, the seven verbs with per verb parameter rules, general operating rules that reference no specific site, page, or answer, and the guide rules that ban mark ids and coordinates from user facing text. The user message carries the task, the guide history, the menu, and the screenshot, under budgets of 16k input and 1,024 output tokens. The full prompt and a complete worked step are reproduced in Appendix 10.

On the output side, one tolerant parser is shared by data construction, training, and evaluation: it extracts the first well formed JSON object from the raw completion, surviving code fences and reasoning preambles. The executor maps the seven verbs onto browser primitives on the live page; a step that yields no valid action executes nothing and consumes budget. Because parser and executor are identical everywhere, OFCR measures exactly the gate that training and evaluation apply.

The operational layer is where live web RL usually breaks, and each rule here exists because its absence corrupted an experiment: six replicated environment sets run the rollouts of a group in parallel; every round and every sweep begins by resetting the site containers to their snapshots and re registering, then verifying, every account, since stale logins silently depress success rates; judge calls queue through a gateway when the training host has no API egress; and every constant is frozen in one configuration module that all components import, so no two stages can drift apart.

4.2 Stage 1: SFT for the Contract and the Guide Style↩︎

Stage 1 finetunes the base VLM [32] separately for each grounding form on its gold steps, 3,308 SoM and 4,018 coordinate examples over the 396 annotated training tasks, each example rendered exactly as at inference and supervised with cross entropy on the assistant JSON tokens only (full finetuning; hyperparameters in Appendix 12).

The purpose of this stage is deliberately not task competence; it is twofold. First, the contract. The base model produces a parseable, executable step in only 19% of attempts under SoM grounding and 14% under coordinates, and nothing downstream survives that. After SFT the rate is about 97% for both variants, and every later stage presumes it. Second, the guide register. The gold guides carry the imperative, user facing style that the MAG task demands, and this stage is where that style is learned; the Stage 2 reward never scores guides, yet guide quality persists and improves through RL precisely because SFT anchored it. What SFT does not deliver is success: rates barely move, and the SoM variant even trails its base by learning to declare completion too early (Section 5.2). Competence is the next stage’s job.

4.3 Stage 2: Expert Augmented GRPO↩︎

With a binary reward on a live site, GRPO [10] learns only from groups whose rollouts disagree: every trajectory in an all fail group has zero advantage. At SFT level competence this is the common case: across ten plain GRPO attempts spanning reward shaping, curricula, and penalty terms, no run produced a sustained gain (Appendix 15); dynamic sampling in the style of DAPO [11] does not help, because it assumes solvable prompts exist to be resampled, while here the policy never reaches a first success on most tasks. The bottleneck is capability, not variance reduction.

We therefore import the missing successes from outside the policy, in the spirit of off policy guidance [12]. Before training we roll GPT-5.5 [29] through the same harness on the 407 training tasks twice, two independent passes under identical budgets, and cache only the trajectories the evaluator verifies as successful: 112 and 122 tasks, 139 in union (95 in both). A cached trajectory stores the full step records, so it enters a group exactly like a policy rollout.

For a covered task \(q\), a group joins six on policy rollouts, sampled on the live environments at temperature 1.0 under the 25 step budget, with the cached expert trajectories \(\mathcal{E}_q\) (\(|\mathcal{E}_q| \le 2\)): \[\mathcal{G}_q = \{\tau_1,\dots,\tau_6 \sim \pi_{\theta_{\mathrm{old}}}\} \cup \mathcal{E}_q, \quad R_i = S(\tau_i), \label{eq:group}\tag{6}\] with \(S \in \{0,1\}\) the functional success of Equation 3 . Advantages are centered but not rescaled: \[A_i = R_i - \frac{1}{|\mathcal{G}_q|}\sum_{j} R_j; \label{eq:adv}\tag{7}\] groups with zero reward variance are dropped, and no standard deviation division is applied, so the advantage of a lone success in a failing group keeps its full magnitude. The policy ascends the token level clipped objective [33] \[J(\theta) = \frac{1}{\sum_i |\tau_i|} \sum_{i,t} \min\!\big(\rho_{i,t} A_i,\; \tilde{\rho}_{i,t} A_i\big), \label{eq:grpoobj}\tag{8}\] where \(\rho_{i,t}\) is the importance ratio between \(\pi_\theta\) and \(\pi_{\theta_{\mathrm{old}}}\) on token \(t\) of \(\tau_i\) and \(\tilde{\rho}_{i,t} = \mathrm{clip}(\rho_{i,t},\, 1-\varepsilon_l,\, 1+\varepsilon_h)\) with an asymmetric clip \(\varepsilon_l = 0.2\), \(\varepsilon_h = 0.28\) [11], which throttles positive advantages less; no KL term is applied.

Two details matter. The teacher’s token probabilities are unavailable, so \(\rho\) on expert tokens is computed against the policy’s own round start log probabilities, making expert rows ordinary off policy data with \(A_i > 0\). And round start log probabilities are recomputed once per round with the training stack, not the inference engine, whose numerics differ; the frozen \(\pi_{\theta_{\mathrm{old}}}\) keeps the clipped objective meaningful across a round’s updates.

Training runs ten rounds, expert covered tasks first, so early rounds see expert groups and later rounds continue with plain six rollout groups; each round resets and re authenticates the environments, collects rollouts, merges judge verdicts, and performs one inner epoch of updates (Algorithm 8, Appendix 12). The reward carries no guide term, and yet guide quality improves alongside success (Section 6.0.0.5): the style anchored in Stage 1 rides along with competence.

5 Experiments↩︎

5.1 Setup and Baselines↩︎

We evaluate three frontier API models, GPT-5.5 [29], Gemini 3.5 Flash [34], and Claude Sonnet 4.6 [35], and the open Qwen3.5 9B VLM [32]¹ as base, after SFT, and after GRPO rounds 5 and 10, each under both grounding schemes. Every run follows the protocol of Section 3.3 with the same prompt, parser, budgets, and judge.

We deliberately compare against no earlier WebArena agent. MAG is a new task with a second scored output; its tasks are reannotated and inherited from one agent’s solvable subset (Section 3.2); and its metrics require guides that existing agents do not produce. The evaluation therefore answers how well current models do MAG, not where they rank on WebArena.

5.2 Main Results↩︎

Table 1 reports the full suite; four findings organize what follows.

5.2.0.1 MAG is far from solved.

The best configuration, GPT-5.5 with coordinates, completes 37.4% of the test tasks; the best trained 9B agent reaches 13.2%. Every model also pays a guide tax: even the best GGS (.226) sits far below its own SR.

5.2.0.2 The contract is learnable; competence is not the same thing.

API models satisfy the output contract out of the box (OFCR .88 to .999); the 9B base does not (.19 SoM, .14 coord), and SFT repairs exactly this, .97 or higher on every tuned row (Section 4.2). No evaluated step in any run leaked a mark id or coordinate into a guide: the constraint is easy to satisfy; being right is not.

5.2.0.3 Grounding preference is model specific.

Gemini is far stronger with SoM than with coordinates (+13.8 points, \(p<0.001\)), GPT-5.5 and Claude show no significant preference (\(p=0.69\) and \(p=1.0\)), and the trained 9B agent makes progress only under SoM: 6.9 to 13.2 through GRPO, while its coordinate variant peaks at 9.2 in round 5 and falls back to 8.1 by round 10: grounding has to be measured per model, exactly the comparison MAG enables.

5.2.0.4 Expert augmented GRPO is the only recipe that moves success.

Under SoM it lifts the same weights from 6.9 to 10.9 to 13.2 (+6.3 points over SFT, \(p=0.035\)), nearly doubling success and doubling GGS (.040 to .080), with the reference guide metrics rising alongside although the reward never scores guides; under coordinates it does not, consistent with the grounding finding.

6 Analysis↩︎

6.0.0.1 Statistical significance.

The headline SoM gain, GRPO round 10 over same run SFT, is significant under an exact McNemar test (17 tasks solved only by GRPO versus 6 only by SFT, \(p=0.035\); paired bootstrap \(p=0.023\); 95% CI \([+1.2, +11.5]\) points), and the intermediate steps are monotone though individually within noise. The coordinate GRPO gains are not significant (\(p\ge0.63\)), a finding we report rather than hide. Among API models, Gemini’s SoM advantage is the strongest modality effect (\(p<0.001\)); the gap between GPT-5.5 with coordinates and the best 9B agent is +24.1 points (\(p<0.001\)). Full table and caveats in Appendix 13.

6.0.0.2 Sharpening or learning? A pass@k view.

Sampling six trajectories per task at temperature 1.0 separates two readings of the GRPO gain. If RL only sharpened the SFT distribution, pass@6 would stay flat; instead it rises from 14.4% (SFT) to 21.3% (round 10), +6.9 points (\(p=0.043\), CI \([+1.2, +13.2]\)), alongside pass@1 (4.9 to 12.1). Sampling actually hurts the SFT policy (its greedy 6.9% beats its sampled pass@1 4.9%), while the GRPO policy is sharp around successful sequences (12.1 sampled versus 13.2 greedy). GRPO adds capability, not just concentration (Appendix 14).

6.0.0.3 What GRPO learns.

Solved sets: round 10 solves 17 tasks SFT could not and loses 6, so the lift is new competence rather than variance reduction; the union of its SoM and coordinate successes covers 16.7% of tasks (overlap only 8), an easy routing headroom. Actions: from SFT to round 10, scroll drops from 34.9% to 13.5% of steps, click rises to 57.9%, and go_back reappears (0 to 2.3%): the policy trades timid wandering for decisive element interaction. Horizon: gains concentrate on short and medium tasks (+9.8 and +9.1 points), while beyond nine gold steps every 9B variant stays at or under 2%, where the API models sustain 26 to 34% (Appendix 13). The remaining teacher gap is a long horizon gap, not a per step accuracy gap.

6.0.0.4 Ablation: expert injection.

Ten earlier plain GRPO configurations without expert rows produced no sustained gain (first to last round deltas \(-14.6\) to \(+1.0\) points), while the number of groups carrying reward variance tracks expert coverage almost exactly. Appendix 15 documents the runs and the caveats that make this evidence observational.

6.0.0.5 Guide and success coupling.

The two outputs of the task are not independent. On round 10, guide quality \(G\) on solved episodes is .369 against .258 on failed ones (point biserial \(r=0.24\), \(p=0.006\); \(r=0.19\) after SFT), and the coupling strengthens through RL although the reward never sees a guide. The agent that can do the task describes it better: the premise of unifying the two outputs, and a partial answer to the single reference limitation of \(G\) (Appendix 11).

7 Conclusion↩︎

MAG turns the manual labor behind in app web guidance into a measurable agent task: complete a live multistep task from screenshots and write, at every step, the guide sentence a user would need. We contribute the benchmark with human verified gold guides, the end to end harness, an evaluation suite whose headline metric credits guides only on solved tasks, and a training recipe in which cached expert trajectories restore the signal that plain GRPO lacks. The best frontier configuration completes 37.4%, grounding preferences differ per model, and small models fail on long horizons; we release everything so that progress on MAG is measurable rather than anecdotal.

Limitations↩︎

MAG is a new and deliberately hard task: live multistep websites, screenshot only observation, a 25 step budget, and two jointly scored outputs. Absolute success rates are accordingly low, 37.4% for the strongest frontier configuration and 13.2% for our best 9B agent, and should be read as a measure of the task’s difficulty and headroom rather than of any single method; the benchmark exists to make progress on this gap measurable. Beyond that, the usual caveats of a first release apply. The test set holds 174 tasks, so single digit differences carry roughly \(\pm 2\) point uncertainty, and the GRPO result rests on one seed per grounding scheme and one teacher model. The guide metric is single reference (Appendix 11), and all tasks come from six WebArena sites, so transfer to other interfaces remains to be shown.

Ethics Statement↩︎

All sites are self hosted WebArena sandboxes populated with synthetic content; no live third party service is acted upon and no personal data is collected or processed. Guide annotation and verification were carried out by three members of the research team. API models were accessed under their providers’ terms of service; the released corpus and harness carry the Apache 2.0 license, and released screenshots contain sandbox content derived from Wikipedia and OpenStreetMap, redistributed under their respective terms (Appendix 8). The intended use of MAG is assistive: reducing the authoring cost of in app guidance. We see limited misuse potential beyond generic web automation concerns, which the sandboxed environment does not enable.

8 Dataset Details↩︎

MAG is built from the success demonstrations released by OpAgent [28] under the Apache 2.0 license, covering 581 of WebArena’s 812 tasks, replayed and reannotated as described in Section 3.2. All pages are rendered in a \(1440\times900\) viewport with Set-of-Mark marking; the candidate menu lists each mark as [id] TYPE visible text, with visible text clipped to 80 characters, and includes elements beyond the visible fold, which the scroll verb brings into view. Table 2 breaks the corpus down by site, Table 3 by action verb, and Table 4 shows gold guides spanning sites and verbs.

The two views share the same 563 tasks but differ in step counts for two reasons. The SoM view drops a step when its recorded coordinate maps to no marked candidate: 1,019 of the 4,768 element steps (21.4%) survive only in the coordinate view. In both views every trajectory ends with exactly one finish step that carries the task answer and a templated guide announcing it, synthesized when the recording lacked an explicit final step, which is why the finish count equals the task count. Trajectories average 10.3 steps in the coordinate view and 8.5 in the SoM view (median 6 and maximum 71 in both), and gold guides average 14 words (median 13). Eleven test tasks have gold demonstrations longer than the 25 step budget; the budget is identical for every system, so comparisons are unaffected, though success along the demonstrated path is capped near 94%. The verb press_enter is available to the agent at inference time but absent from the gold demonstrations, which submit forms through visible controls instead. All sites are self hosted WebArena sandboxes populated with synthetic content, so the corpus contains no real user data. The corpus and the harness are released under Apache 2.0, matching the license of the source demonstrations; released screenshots contain sandbox content derived from Wikipedia (CC BY SA) and OpenStreetMap (ODbL), redistributed under those terms.

9 Guide Verification Interface↩︎

Every drafted guide passes through a manual verification stage carried out in the desktop tool shown in Figure 4. The tool walks the annotator through the dataset step by step. For each step it displays the captured screenshot that served as the drafting input and lists the task intent together with the step record: page URL, executed action, and the drafted think. Guides were drafted and reviewed on the initial replay captures (\(1280\times720\), as visible in Figure 4); the released corpus recaptures every step at \(1440\times900\) and realigns the recorded actions. The guide sentence sits in an editable text box, and three keyboard commands cover the entire workflow: accept the guide as written, save an edited version, or flag the step as bad. A progress counter tracks coverage and an index field jumps to any task.

The review concentrates on the guides: the annotator checks that each sentence names the correct target on the current page and reads as a clear, self contained instruction for that step. Since all trajectories originate from verified success demonstrations, action correctness is only spot checked in passing. Most drafts passed review unchanged; typical edits added the location of the target element or simplified wording.

10 Prompts↩︎

This appendix reproduces the locked prompt verbatim. One system prompt (Figure 5) and one user template (Figure 6) serve the API baselines, SFT data construction, GRPO rollouts, and evaluation; only the grounding argument differs between the two variants.

11 Metric Definitions and Evaluation Protocol↩︎

11.0.0.1 Output contract.

At every step the model must emit exactly one JSON object with five keys: think, action_type, selected_candidate_id (replaced by a coordinate pair under coordinate grounding), content, and guide_text. A single locked parser is shared by data construction, training, and evaluation.

11.0.0.2 OFCR.

A step passes the format gate if three conditions hold. (i) Parse: the output parses as the contract above with nonempty think, guide_text, and a recognized verb. (ii) Valid action: the verb is one of the seven in \(\mathcal{A}\); the element verbs click, type, and select carry a grounding argument, and a SoM id must appear in the current candidate menu; type carries text and select an option label. (iii) No leakage: the guide matches none of a fixed regular expression family covering mark references (element 17, candidate 17, [17]) and coordinate pairs ((312, 174)); the full family ships with the harness, and the human pass of Section 3.2 backstops paraphrased leakage. OFCR is the fraction of steps passing all three.

11.0.0.3 Guide quality and GGS.

\(G_i\) (Equation 4 ) joins the predicted guides of task \(i\) in step order and scores them against the joined gold guides with BLEU-1, BLEU-2, ROUGE-1 (unigram token F1), and ROUGE-L, averaged with equal weights. Joining is deliberate: no step alignment exists when the predicted and gold trajectories differ in length, so a step count mismatch simply lowers overlap; the BLEU brevity penalty and F1 based ROUGE bound length gaming; and step level fidelity is measured separately by the teacher forced diagnostics below. Guides that fail OFCR are not additionally zeroed in \(G_i\); the two metrics are reported side by side. A successful run that takes a valid alternative path is penalized by the single reference, so part of the guide tax reflects path divergence rather than guide quality; Section 6.0.0.5 probes this with a guide and success coupling analysis. Corpus GGS averages Equation 5 over all 174 test tasks; \(\gamma=0.4\), \(\tau=120\) px, the content thresholds, and the Suff floor were all fixed before any experiment and never tuned.

11.0.0.4 Teacher forced step diagnostics.

Given a gold step with candidate set \(C\), action correctness decomposes as \(A = \mathrm{Type}\cdot\mathrm{Target}\cdot\mathrm{Content}\). \(\mathrm{Type}\) is exact verb equality. \(\mathrm{Target}\) is exact id equality under SoM; under coordinates it is \(1\) if the predicted point falls inside the gold element’s box and \(e^{-d/\tau}\) otherwise, where \(d\) is the distance to the box edge and \(\tau=120\) px. \(\mathrm{Content}\) compares payloads by token F1 with thresholds \(0.9\) for type and select and \(0.5\) for the finish answer, and is \(1\) for the remaining verbs. Guide faithfulness and sufficiency are \[\begin{align} \mathrm{Faith} &= \tfrac{1}{2}\,\mathrm{Verb} + \tfrac{1}{2}\,\mathrm{Obj},\\ \mathrm{Suff} &= \begin{cases} 0 & \text{if } m < 0.10,\\ m \cdot p_{\mathrm{tie}} & \text{otherwise,} \end{cases} \end{align}\] with \(m = \max_{c \in C} \mathrm{F1}(\mathrm{anchor}(g),\, v_c)\) and \(p_{\mathrm{tie}} = 0.5\) when the top two candidates lie within \(0.10\) of each other and \(1\) otherwise. \(\mathrm{Verb}\) checks the guide’s head verb against the action verb through a small lexicon. \(\mathrm{Obj}\) is the token F1 between the guide’s quoted anchor span, extracted by \(\mathrm{anchor}(g)\) with fallback to the full sentence, and the step’s target string: the element’s visible text for click and select, the typed text for type, the direction for scroll, and the answer for finish. \(\mathrm{Suff}\) asks whether the guide identifies a unique referent on the page: a best match below \(0.10\) names nothing, and a near tie makes the reference ambiguous; whether that referent is the intended one is carried by \(\mathrm{Obj}\). For verbs that target no element, \(\mathrm{Suff}\) is \(1\) with a clear head verb and \(0.5\) otherwise. The fused step score and its aggregate are \[\mathrm{GACS} = F \cdot A \cdot \sqrt{\mathrm{Faith}\cdot\mathrm{Suff}}, \qquad \mathrm{SAA} = \overline{A},\] with \(F\) the step’s OFCR bit. All quantities are deterministic and dependency free, so diagnostic scoring is identical wherever it runs.

11.0.0.5 Evaluation protocol.

Algorithm 7 gives the live protocol, identical under both grounding schemes; only the action’s grounding argument differs. Every model output consumes one step of the budget and is scored by OFCR; an action that names no valid target executes nothing on the page. Locally served models decode greedily; the API model exposes no deterministic mode and is called with fixed settings (low reasoning effort, 1,024 output tokens). The LLM judge (GPT-5.5, fixed prompt) scores only the 29 test tasks whose checker requires semantic comparison of the final answer; it receives the answer and the reference, never the trajectory or the system identity, and the same judge scores every run.

12 Training Details↩︎

Table 5 lists both configurations. The expert cache is built before training: GPT-5.5 runs twice over the 407 training tasks through the harness, at the same budgets as the policy; the two passes solve 112 and 122 tasks respectively, 139 in union and 95 in both, and only evaluator verified successes are cached. Each round of GRPO follows Algorithm 8; the environments are reset and re authenticated at every round boundary, and rollouts for one group run in parallel across six replicated environment sets.

13 Full Results↩︎

13.0.0.1 Run provenance.

All rows of Table 1 are single greedy sweeps over the 174 test tasks (the API rows under fixed provider settings, Appendix 11), scored from saved per step records. OFCR is recomputed for every row from the raw model outputs with the three tier definition; the qwen rows decode the raw completions with the locked parser and recover the candidate menus from the stored prompts.

13.0.0.2 Solved set overlap.

GRPO round 10 (SoM) solves 17 tasks its SFT initialization could not and loses 6; from round 5 to 10 it keeps 15 of 19, loses 4, and gains 8, so acquisition continues through the self rollout phase. Its SoM and coordinate successes overlap on only 8 tasks; the union covers 29 of 174 (16.7%), an immediate headroom for modality routing.

13.0.0.3 Action distribution.

Over all steps of the SoM runs: the base model emits 81.4% unparseable steps (its OFCR); SFT eliminates these (2.3%) but scrolls on 34.9% of steps; GRPO round 10 shifts mass to decisive interaction (click 57.9%, scroll 13.5%, type 16.2%) and rediscovers go_back (2.3%). The GPT-5.5 SoM run distributes 69.5% click, 16.0 type, 5.2 scroll, 4.1 finish, 2.5 press_enter, 1.5 select, and 1.2 go_back.

14 pass@6 Details↩︎

Six trajectories per task are sampled at temperature 1.0, one per environment set (1,044 episodes per model), same harness and judge as the main runs. SFT SoM: greedy 6.9%, sampled pass@1 4.9%, pass@3 10.1%, pass@6 14.4%, 149 never solved tasks. GRPO round 10: greedy 13.2%, pass@1 12.1%, pass@3 17.6%, pass@6 21.3%, 137 never solved. The paired pass@6 gap is \(+6.9\) points (21 tasks only GRPO, 9 only SFT; McNemar \(p=0.043\), bootstrap \(p=0.027\), CI \([+1.2, +13.2]\)). Figure 9 shows the curves: GRPO lifts both the single sample rate and the exploration ceiling, and its sampled pass@1 nearly matches its greedy rate, while SFT loses success under sampling.

15 Plain GRPO Runs without Expert Injection↩︎

Before expert injection we ran ten plain GRPO configurations (G = 6 self rollouts, no expert rows) spanning shaped and binary rewards, curricula, premature finish penalties, and KL variants, for 1 to 13 rounds each. Across the nine runs with at least two logged rounds, the first to last round training batch success deltas were \(-9.4\), \(-7.3\), \(-10.4\), \(-5.2\), \(+24.0^{*}\), \(+1.0\), \(-10.4\), \(-8.3\), and \(-14.6\) points; the starred outlier used an inflated stub judge during a period without API egress, and its clean re evaluation measured 6%. Two caveats make this documentary rather than a controlled ablation: the numbers are training batch statistics on freshly sampled tasks (not the fixed test set), and most of these runs predate the deterministic judge routing. The mechanism is nonetheless visible in the expert augmented runs themselves (Figure 10): reward variance, the precondition for any GRPO gradient, tracks expert coverage almost exactly.

16 Qualitative Case Studies↩︎

Figures 11, 12, and 13 render three complete gold trajectories the way a user would experience them: at every step, the guide sentence appears in a callout anchored on the target element of the real page, and the final step announces the answer. The three tasks are among the shortest in the corpus and cover three sites and three verbs (click, scroll, finish). They illustrate the intended product shape of MAG from Figure 1: the agent’s two outputs per step, an action and a guide, are exactly the assets an in app guidance overlay needs.

References↩︎

---

1. https://huggingface.co/Qwen/Qwen3.5-9B; our SFT and GRPO checkpoints are released with the harness.↩︎