PatchWorld: Gradient-Free Optimization of Executable World Models

Jiaxin Bai\(^{1}\), Yue Guo\(^{2}\), Yifei Dong\(^{3}\), Jiaxuan Xiong\(^{4}\), Tianshi Zheng\(^{3}\), Yixia Li\(^{5}\)
Tianqing Fang\(^{3}\), Yufei Li\(^{3}\), Yisen Gao\(^{3}\), Haoyu Huang\(^{3}\), Zhongwei Xie\(^{3}\)
Hong Ting Tsang\(^{3}\), Zihao Wang\(^{2}\), Lihui Liu\(^{6}\), Jeff Pan\(^{7}\), Yangqiu Song\(^{3}\)
\(^{1}\)Hong Kong Baptist University \(^{2}\)Independent Researcher \(^{3}\)HKUST
\(^{4}\)Beijing Institute of Technology \(^{5}\)Southern University of Science and Technology
\(^{6}\)Wayne State University \(^{7}\)University of Edinburgh
baijiaxin@hkbu.edu.hk ireneyueguo@gmail.com xiongjiaxuan@bit.edu.cn
ydongbl, tzhengad, ylivm, ygaodi, hhuangcp, zxiebk@connect.ust.hk
httsangaj, zwanggc@connect.ust.hk tfangaa, yqsong@cse.ust.hk
liyixia@me.com hw6926@wayne.edu j.z.pan@ed.ac.uk

May 29, 2026

Abstract

Text-agent environments are typically modeled as partially observable Markov decision processes (POMDPs), assuming that the simulator’s latent state and transition dynamics are hidden from the agent. Yet little work has examined whether executable code can be induced to serve as a world model for prediction and planning under partial observability. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair. Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score among evaluated methods, reaching 76.4% macro success in live one-step lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility. This exposes a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.

1 Introduction↩︎

Humans build scientific models by learning compact symbolic rules that explain observations, predict the effects of interventions, and generalize beyond seen data [1]–[3]. In artificial agents, world models play an analogous role by learning latent dynamics for prediction, simulation, planning, and control [4]–[7], with recent systems extending these ideas to embodied and generative environments [8]–[11]. For interactive text agents, such models could capture how actions change an environment and thereby support simulation, diagnosis, and, when useful, planning before trial-and-error. Code-based world models are especially attractive in this setting because they are interpretable, exactly executable, and locally repairable [12], [13]. This paper asks whether they can be realized for text-agent environments, including simulators, interfaces, games, and interactive virtual worlds, where the agent never observes the simulator’s internal state.

The central obstacle is partial observability. Text-agent environments are intrinsically POMDPs, where the simulator maintains latent state while the agent receives only a rendered text observation after each action [14]–[21]. In AlfWorld, a kitchen description omits objects inside closed drawers; in Wordle, the target word is never revealed; in WebShop, backend relevance scores are invisible. Prior text-environment world models often sidestep this hidden state by predicting the next observation from recent history [22], [23] or by distilling trajectory knowledge into parametric planning aids [24]. These approaches can be useful, but they do not necessarily maintain the persistent belief state needed for multi-step simulation or planning.

This hidden state also makes code induction underdetermined. This means that trajectory log can be reproduced by many executable programs, ranging from a lookup table that memorizes every observed transition to a compact stateful program that infers latent variables and generalizes to unseen inputs. Because the trajectories alone cannot distinguish memorization from structure, the learner needs an inductive bias toward programs that explain the data through compact latent state.

We propose PatchWorld to address these. PatchWorld is a gradient-free framework that generates one world model per environment and treats world-model induction as generative optimization. Given logged trajectories, an LLM first synthesizes an executable Python program with an explicit symbolic belief state, transition rules, correction logic, and rendering logic. We then replay the program against the same trajectories, convert prediction failures into concrete counterexamples, and ask the LLM to propose candidate patches. Since PatchWorld performs no gradient updates, improvement comes from discrete search over programs and patches, with a patch accepted only when it improves formal replay fidelity. This makes the loop analogous to counterexample-guided inductive synthesis (CEGIS) [25]. The output is therefore not only a next-token predictor, but also a reusable and inspectable executable hypothesis about environment dynamics whose belief updates and transition rules can be run, tested, and locally repaired.

1.0.0.1 A Pareto frontier, not a single best model.

At a conceptual level, world modeling is pulled between two related but distinct goals. Some work emphasizes faithful reconstruction of future observations [22], [23], while other work emphasizes downstream decisions through action-contrastive predictions [4], [5], [7], [24]. Text environments make this tension especially visible. Under partial observability, a model can recover surface text while offering weak action guidance, or miss exact renderings while preserving useful transition structure (Figure 3). PatchWorld therefore treats text world modeling as a frontier rather than a single optimum. PatchWorld-Simple stays on the purely symbolic side of this frontier, while PatchWorld-Residual adds a human-specified residual-memory bias for exact, unambiguous textual state signatures. This improves reconstruction fidelity without replacing the executable dynamics used for planning. More broadly, these results show that LLMs can act as symbolic optimizers, searching, patching, and improving executable symbolic systems that are competitive with LLM-based world models on both observation prediction and planning utility.

1.0.0.2 Contributions.

(1) We identify a key challenge for code-based text world modeling. Under partial observability, finite logs can support exact replay without revealing the compact latent-state rules needed for generalization (Section 3.0.0.2). (2) We introduce PatchWorld, a gradient-free framework that uses LLMs as symbolic optimizers to synthesize and repair executable belief-state programs; its residual-memory variant treats recurring textual state evidence as a human-specified inductive bias, not an unconstrained cache (Section 4). (3) Across seven AgentGym environments, we find a frontier between observation reconstruction and planning utility. PatchWorld-Residual attains the highest code-based fidelity, PatchWorld-Simple attains the highest code-based live lookahead planning score, and our diagnostics explain where the two goals diverge (Section 5).

2 Related Work↩︎

2.0.0.1 Code-based and symbolic world models.

Recent work represents environment dynamics as executable code or symbolic rules. WorldCoder induces transition models, GIF-MCTS searches over code edits, PoE-World composes programmatic experts, and OneLife learns probabilistic precondition and effect rules [13], [26]–[28]. PDDL and LLM-as-planning-formalizer methods provide formal structure but are less natural for template-heavy text rendering [12], [29], [30]; neuro-symbolic verification similarly motivates executable checks for LLM-generated formal artifacts [31], [32]. PatchWorld targets partial observability with an explicit symbolic belief state, execution-grounded repair, and a tunable split between symbolic dynamics and retrieval-based rendering.

2.0.0.2 Neural and implicit world models for text agents.

An alternative is to train or prompt an LLM to predict the next observation. Word2World [22] fine-tunes an LM as a next-observation predictor; WebEvolver [23] co-evolves agent and world-model LLMs; WKM [24] trains a parametric knowledge model for agent planning; and WALL-E [33], [34] and CoEx [35] maintain rule sets via online interaction. These approaches improve surface prediction or planning, but rely on online interaction or supervised fine-tuning without producing an offline executable belief program. PatchWorld operates offline and gradient-free, and produces an inspectable program whose belief can be examined and locally repaired.

2.0.0.3 Video and embodied world models.

Beyond text, a parallel line of work learns action-conditional world models in pixel or sensor space. Genie [10] induces interactive environments from unlabeled video, and CWM [11] and \(\pi_0\) [9] extend the world-model abstraction to code execution and robot control. These systems share PatchWorld’s action-conditional framing but rely on pixel- or sensor-space pretraining; text-agent observations are short and templated, which both rewards exact tokens and motivates an executable, locally repairable substrate rather than an implicit network.

2.0.0.4 Program synthesis and counterexample-guided repair.

PatchWorld’s induction loop builds on program synthesis from examples [36]–[39] and instantiates counterexample-guided inductive synthesis [25] for sequential POMDP dynamics, and this specification is a set of replayable trajectories, the verifier is an executable validator that reports typed failures, and the synthesizer is an LLM prompted with structured diagnostics. Related logical-hypothesis work studies constraint-based reasoning in knowledge graphs [40], [41], whereas our hypotheses are executable transition programs. Compared with agent reflection and planning-adaptation methods [42]–[46] and LLM code self-repair from execution feedback [47]–[50], PatchWorld addresses a failure mode of hallucinated or weakly checked code [51], [52]: it accepts a patch only when the patch improves formal replay score on the full validation set, preventing the regression problem that plagues unconstrained self-correction.

3 Task Formulation↩︎

We model each environment as a POMDP \((\mathcal{S}, \mathcal{A}, \mathcal{T}, \Omega, \mathcal{O})\) with latent states \(s_t\), text actions \(a_t\), observations \(o_t\), transition kernel \(\mathcal{T}(s_{t+1}\!\mid\!s_t,a_t)\), and emission \(\Omega(o_t\!\mid\!s_t)\). The learner sees neither \(s_t\) nor \(\mathcal{T}\), only offline trajectories \(\mathcal{H}=\{(o_0,a_0,\ldots,a_{T-1},o_T)\}\) and an optional description \(E\).

3.0.0.1 Goal.

Induce executable Python \(c\) that approximates dynamics through a symbolic belief \(\hat{s}_t \in \hat{\mathcal{S}}\) and a predict and correct cycle. The program implements \(f_c\) (predict), \(\rho_c\) (readout), and \(g_c\) (correct). Let \(\bot\) denote the blank belief returned by init_belief; correct_belief then populates it from the initial observation: \[\begin{align} \hat{s}_0 &= g_c(\bot,\, o_0), \tag{1}\\ \tilde{s}_{t+1} &= f_c(\hat{s}_t,\, a_t), \tag{2}\\ \hat{o}_{t+1} &= \rho_c(\tilde{s}_{t+1},\, a_t), \tag{3}\\ \hat{s}_{t+1} &= g_c(\tilde{s}_{t+1},\, o_{t+1}). \tag{4} \end{align}\] The induction objective is to minimize replay loss \[\min_{c}\; \mathbb{E}_{(o_t,a_t,o_{t+1})\sim\mathcal{H}} \bigl[\ell(\hat{o}_{t+1},\, o_{t+1})\bigr], \label{eq:induction95objective}\tag{5}\] where \(\ell\) is a text loss over predicted and observed next observations and \(\hat{s}_t\) comes from running 1 –4 along the prefix. Since \(c\) is discrete, we optimize by search (Section 4); the output is an inspectable program whose beliefs, rules, and render logic can be locally patched.

The formulation also allows \(c\) to contain a constrained residual readout memory for high-confidence surface details, while preserving the predict and correct interface; the constraints act as an inductive bias analogous to weight decay, shaping the search toward simpler explanations (Section 4.4).

3.0.0.2 Feasibility and identification.

Three observations motivate the inductive bias in Section 4. (1) Existence: if \(\mathcal{H}\) has no contradictory history and action pairs, zero loss on Eq. 5 is achievable by a lookup table over prefix-action keys, feasible but non-generalizing. (2) Non-identifiability: perfect replay does not imply correct latent structure; multiple compact programs can agree on logged transitions yet diverge on unseen prefix-action combinations [14]. (3) Inductive bias: PatchWorld biases search via the LLM’s code prior, counterexample-guided repair that rewards consistency across diverse contrastive patterns, and explicit constraints on any residual memory; empirically defensible biases that discourage lookup-table replay while permitting limited memorization of reliable renderer details.

3.0.0.3 Why fidelity and utility can diverge.

Eq. 5 optimizes textual reconstruction of \(o_{t+1}\), but a planner only needs the model to discriminate among candidate actions: \(a^\star=\arg\max_a Q^\star(s,a)\). Let \(\Delta_c(s,a,a')=\rho_c(f_c(s,a),a)-\rho_c(f_c(s,a'),a')\) denote the action-contrastive readout signal. Two models can attain identical replay loss while differing arbitrarily in \(\Delta_c\): any aspect of \(\rho_c\) constant across \((a,a')\) (template phrasing, identifiers, list orderings) costs reconstruction but does not affect \(\Delta_c\), and vice versa. This is the partial-observation analogue of the value-equivalence principle of [53]: models can be planning-equivalent yet diverge on observation likelihood. We therefore expect, and empirically observe (§5.4, Appendix 17), a Pareto frontier between fidelity and utility rather than a single optimum.

Figure 1: Overall framework of PatchWorld for generative optimization of executable world models and their subsequent repair and evaluation.
Table 1: Environment regimes for deterministic code-based prediction.
Regime Envs. Character
Deterministic structure Maze, BabyAI, TextCraft Fixed hidden map/grid/recipe; exploration collapses belief, exact prediction possible.
Irreducibly stochastic Wordle, WebShop Latent choices stay ambiguous; exact text impossible deterministically.
Deterministic dynamics, complex rendering AlfWorld, SciWorld Learnable dynamics, but simulator-specific wording hurts exact match.

None

Figure 2: Core executable interface implemented by each induced program, separating parsing, belief correction, dynamics, and readout..

4 Methodology↩︎

PatchWorld induces standalone Python world models through execution-grounded search (Figure 1): select contrastive trajectory evidence, prompt an LLM to synthesize a complete BaseWorldModel, replay the module to expose counterexamples, and accept only repairs that improve full replay validation. The resulting program maintains belief state, supports planning, and predicts next observations without further LLM calls. Induction uses train; repair uses validation when available, else train; test is held out.

4.1 Executable World-Model Interface↩︎

Each generated module implements the predict and correct interface from Section 3 and Figure 2. Replay corrects belief with the logged observation before each transition; rollout conditions on predicted observations to expose compounding errors. The prompt fixes the environment type but leaves the state representation to the LLM, yielding graph-like beliefs for spatial tasks and record-like state for API-style tasks.

4.2 Evidence Selection via Contrastive Mining↩︎

Since prompt space is limited, PatchWorld selects transitions exposing distinct behaviors. Each transition \(\tau=(o,a,o')\) is grouped by an action signature \(\alpha(a)\) that abstracts instances into typed forms (e.g. take mug 1 from shelf 2 \(\to\) take OBJ from RECEPTACLE) and an outcome signature \(\omega(o')\) distinguishing successful changes, no-op or invalid messages, terminals, and informational responses. Retaining both success-like and failure-like continuations per action type pushes the synthesized program toward conditional rules over memorized strings. Algorithm 5 keeps at most \(k\) examples per \((\alpha,\omega)\) and round-robins across signatures up to \(m\) total. Validation still runs on the full replay set. Pseudocode is given in Appendix 8.

4.3 Counterexample-Guided Repair↩︎

The initial prompt contains \(T\), the environment description, the interface contract, and guidance to emit a complete executable module. The repair loop treats induction as program search with LLM generation as proposal and replay as fitness. Given current program \(c\) and replay set \(B\), \[\begin{align} \hat{o}^{(c)}_{t+1}\! =&\!\rho_c(f_c(\hat{s}_t,a_t),a_t), \nonumber \\ \mathcal{L}(c;B)\! =&\!\tfrac{1}{|B|}\sum_{(o_t,a_t,o_{t+1})\in B}\!\! \ell_{\mathrm{rep}}(\hat{o}^{(c)}_{t+1},o_{t+1}), \end{align}\] where \(\ell_{\mathrm{rep}}\) is the repair-time replay loss: normalized edit distance for textual readouts, plus task-specific parsed-state checks when available. This is distinct from held-out evaluation, which reports Token F1, BLEU-4, and episode success. Validation records typed counterexamples \(\mathcal{E}(c;B)\), grouping failures by execution, parsing, state update, readout structure, or content value; Appendix 20 gives the full taxonomy. Two helpers operate on \(\mathcal{E}\): Diagnose(\(\mathcal{E}\)) summarizes recurring patterns as a compact natural-language brief (e.g., “move predictions drop the walls_visible field whenever the action is blocked”), giving the LLM higher-level guidance than raw cases alone; Prioritize(\(\mathcal{E}\)) ranks counterexamples by severity (execution and parsing errors before semantic mismatches), then by frequency of the underlying signature, so that the 16 examples shown in the prompt cover the most-impactful and most-representative failures rather than the first 16 in iteration order. Each repair prompt contains the current program, the diagnostic, the top-16 counterexamples (input, expected, actual), and the fixed interface guidance; the LLM emits a complete replacement module. We sample \(C\) independently decoded candidates per round and accept at most one: the candidate that improves the lexicographic replay score \(\mathrm{Score}(c;B)\), which orders programs by aggregate failure severity, number of counterexamples, then replay loss. This hill-climbing gate rejects patches that fix displayed counterexamples but regress on broader replay.

Refinement stops when \(\mathcal{E}=\emptyset\), the budget is exhausted, or no candidate improves the score; the accepted program is saved as a Python module (Algorithm 4, Appendix 7; a Maze example is in Appendix 21). Replay may inspect lightweight parser outputs \(\delta_t=\pi_c(o_t)\) for consistency, but next-observation prediction always uses the persistent belief through predict_belief \(\to\) readout_observation; inference is interface-only, with no neural renderer.

4.4 Inductive Bias for Fidelity↩︎

PatchWorld-Simple relies only on the induced symbolic program. Its explicit action updates support planning, but compact rules often miss renderer details such as identifiers, product lists, templates, or long API payloads. PatchWorld-Residual treats these recurring textual details as part of a human-specified belief-state bias rather than as an unconstrained cache. Compact dynamics remain symbolic, while exact and unambiguous textual state signatures may be tracked empirically. The symbolic path remains primary: \(\tilde{s}^{\mathrm{sym}}_{t+1}=f_c(\hat{s}_t,a_t)\), \(\hat{o}^{\mathrm{sym}}_{t+1}=\rho_c(\tilde{s}^{\mathrm{sym}}_{t+1},a_t)\). The residual memory \(\mathcal{K}\) is built only from training transitions, keyed by normalized signatures \(q_i=q(o_i,a_i)\). The signature normalizes whitespace, lower-cases, and substitutes instance-specific entity mentions with their typed forms. For example, in AlfWorld it collapses (‘‘You arrive at safe 1. The safe is closed.’’, open safe 1) and (‘‘You arrive at safe 2.…’’, open safe 2) to a single key when both license the same outcome template, so the cache stores rules over receptacle types rather than instance strings. A key is retained only when its majority next observation has confidence \(\max_{o'} p_{\mathcal{H}_{\mathrm{train}}}(o'\mid q)\geq\tau\). Prediction returns \(\mathrm{Retrieve}_{\mathcal{K}}(q(o_t,a_t))\) on a cache hit and \(\hat{o}^{\mathrm{sym}}_{t+1}\) otherwise; state still flows through predict_belief and correct_belief, so the residual participates in belief tracking rather than patching over it. Two safeguards prevent leakage: the memory is train-only, and conflicting keys are rejected by default (\(\tau=1\)), avoiding hidden-state aliases such as identical Wordle observations with different target words. For WebShop, a deterministic browser fallback handles local navigation when no exact key matches; Appendix 12 isolates the train-key component.

@llll *8cc@ Method & Backbone & LLM Usage & Infer. LLM? & & & & & & & &
(lr)5-6(lr)7-8(lr)9-10(lr)11-12 (lr)13-14(lr)15-16(lr)17-18(lr)19-20 & & & & F1 & BL & F1 & BL & F1 & BL & F1 & BL & F1 & BL & F1 & BL & F1 & BL & F1 & BL

Word2World & Qwen3.5-4B & SFT & Yes & 0.89 & 0.66 & 0.93 & 0.75 & 0.97 & 0.89 & 0.96 & 0.95 & 0.94 & 0.68 & 0.63 & 0.42 & 0.60 & 0.37 & 0.85 & 0.67
(l)2-20 & Qwen3-Coder-480B & ICL & Yes & 0.53 & 0.35 & 0.73 & 0.44 & 0.83 & 0.75 & 0.48 & 0.30 & 0.76 & 0.51 & 0.56 & 0.34 & 0.60 & 0.21 & 0.64 & 0.41
& Mimo-v2.5 & ICL & Yes & 0.55 & 0.33 & 0.81 & 0.54 & 0.83 & 0.75 & 0.45 & 0.25 & 0.76 & 0.50 & 0.52 & 0.25 & 0.51 & 0.24 & 0.63 & 0.41
& DeepSeek-V4-Flash & ICL & Yes & 0.55 & 0.34 & 0.81 & 0.56 & 0.83 & 0.73 & 0.52 & 0.34 & 0.78 & 0.51 & 0.58 & 0.36 & 0.53 & 0.19 & 0.66 & 0.43

& Qwen3-Coder-480B & CodeGen & No & 0.63 & 0.42 & 0.78 & 0.61 & 0.83 & 0.76 & 0.40 & 0.36 & 0.88 & 0.61 & 0.58 & 0.45 & 0.31 & 0.11 & 0.63 & 0.48
& Mimo-v2.5 & CodeGen & No & 0.59 & 0.35 & 0.77 & 0.61 & 0.88 & 0.78 & 0.37 & 0.33 & 0.88 & 0.61 & 0.53 & 0.37 & 0.35 & 0.13 & 0.63 & 0.46
& DeepSeek-V4-Flash & CodeGen & No & 0.40 & 0.21 & 0.70 & 0.54 & 0.73 & 0.61 & 0.41 & 0.33 & 0.60 & 0.35 & 0.58 & 0.45 & 0.55 & 0.22 & 0.57 & 0.39
(l)2-20 & Qwen3-Coder-480B & CodeGen & No & 0.62 & 0.40 & 0.78 & 0.62 & 0.83 & 0.77 & 0.39 & 0.34 & 0.78 & 0.55 & 0.58 & 0.44 & 0.43 & 0.17 & 0.63 & 0.47
& Mimo-v2.5 & CodeGen & No & 0.62 & 0.40 & 0.78 & 0.62 & 0.86 & 0.78 & 0.41 & 0.36 & 0.88 & 0.61 & 0.60 & 0.47 & 0.40 & 0.16 & 0.65 & 0.49
& DeepSeek-V4-Flash & CodeGen & No & 0.37 & 0.14 & 0.73 & 0.54 & 0.76 & 0.61 & 0.21 & 0.21 & 0.50 & 0.41 & 0.47 & 0.33 & 0.55 & 0.22 & 0.51 & 0.35
(l)2-20 & Qwen3-Coder-480B & CodeGen & No & 0.36 & 0.10 & 0.85 & 0.58 & 0.80 & 0.69 & 0.57 & 0.39 & 0.28 & 0.10 & 0.53 & 0.30 & 0.58 & 0.20 & 0.57 & 0.34
& Mimo-v2.5 & CodeGen & No & 0.73 & 0.47 & 0.41 & 0.19 & 0.88 & 0.75 & 0.48 & 0.30 & 0.93 & 0.67 & 0.28 & 0.12 & 0.47 & 0.16 & 0.60 & 0.38
& DeepSeek-V4-Flash & CodeGen & No & 0.48 & 0.21 & 0.77 & 0.51 & 0.90 & 0.80 & 0.22 & 0.04 & 0.84 & 0.61 & 0.28 & 0.10 & 0.69 & 0.40 & 0.60 & 0.38
(l)2-20 & Qwen3-Coder-480B & CodeGen & No & 0.70 & 0.47 & 0.69 & 0.47 & 0.97 & 0.91 & 0.56 & 0.48 & 0.71 & 0.48 & 0.50 & 0.26 & 0.72 & 0.44 & 0.69 & 0.50
& Mimo-v2.5 & CodeGen & No & 0.77 & 0.50 & 0.49 & 0.28 & 0.87 & 0.82 & 0.69 & 0.56 & 0.95 & 0.68 & 0.50 & 0.26 & 0.63 & 0.35 & 0.70 & 0.49
& DeepSeek-V4-Flash & CodeGen & No & 0.57 & 0.29 & 0.80 & 0.56 & 0.98 & 0.93 & 0.56 & 0.46 & 0.91 & 0.66 & 0.50 & 0.26 & 0.60 & 0.34 & 0.70 & 0.50

5 Experiments↩︎

We evaluate PatchWorld on observation fidelity and planning utility. One-step and rollout prediction test future-observation match, while live one-step lookahead tests action choice. The two evaluations favor different variants. PatchWorld-Residual is the best code-based predictor, while PatchWorld-Simple is the best code-based planner.

5.1 Setup↩︎

5.1.0.1 Environments and data.

We evaluate on seven AgentGym tasks [20] from interactive language-game benchmarks [44], [54], [55], including Maze, BabyAI, TextCraft, Wordle, WebShop, AlfWorld, and SciWorld. These tasks cover navigation, hidden or stochastic state, crafting, shopping, and long-horizon manipulation. Trajectories are collected with a Qwen3-Coder-480B-A35B-Instruct ReAct agent1 and split 60/20/20 by instance ID, keeping all rollouts from an instance in the same train, validation, or test split. Train is used for induction and Word2World fine-tuning, validation for hyperparameters and repair feedback, and test only for reporting. For each environment and backbone, we induce one model with fixed repair hyperparameters; the PatchWorld-Residual index is train-only. Per-split counts are in Appendix 9.

5.1.0.2 Baselines.

LLM-Direct predicts with \(k{=}3\) in-context transitions; Word2World [22] fine-tunes an implicit predictor; WorldCoder [13] and PoE-World [27] are program-induction baselines; and ReAct [56] is used only in planning. In live planning, Word2World is queried as a learned next-observation model; unlike the code-based methods, it remains a neural predictor at inference time. All code-induction methods share the same splits, backbone per row, and per-environment LLM budget caps; PatchWorld uses contrastive caps \(k{=}5,m{=}60\). We report Token F1 and BLEU-4 for prediction, and Episode Success Rate for live planning. All LLM calls use Qwen3-Coder-480B unless noted. Additional backbones follow their model cards: Qwen3.5-4B for Word2World,2 Mimo-v2.5,3 and DeepSeek-V4-Flash.4

5.2 Fidelity frontier favors PatchWorld-Residual for observation prediction↩︎

For each \((o_t,a_t,o_{t+1})\in\mathcal{H}_{\text{test}}\), PatchWorld initializes its belief from the prefix and predicts \(\hat{o}_{t+1}=\rho_c(f_c(\hat{s}_t,a_t),a_t)\); baselines use their fitted predictors with no per-instance adaptation.

5.2.0.1 Analysis.

PatchWorld-Simple learns useful state updates, but its rendered text is often noncanonical. With Qwen3-Coder-480B, it reaches 0.57 macro Token F1, below the 0.63 of WorldCoder and PoE-World; the largest gaps are AlfWorld and TextCraft, where exact simulator templates matter. Validator analysis (Appendix 20) confirms that these are mostly readout mismatches, not transition failures.

PatchWorld-Residual fixes this surface-form bottleneck. Across all three backbones, it is the best program-based predictor on macro Token F1 (0.69–0.70) and BLEU-4 (0.49–0.50), about 6 points above WorldCoder and PoE-World. A five-seed check with Qwen3-Coder-480B gives \(0.6942\pm0.0192\) Token F1 and \(0.4991\pm0.0231\) BLEU-4 (Appendix 10), making the gap larger than the observed induction variance. Appendix 12 reports a cache coverage diagnostic: exact train-key residuals cover 33% of held-out transitions on average, and a retrieval-only predictor reaches 0.32 macro Token F1 when uncovered transitions count as zero.

5.3 Belief state reduces rollout drift↩︎

We initialize from ground-truth \(o_0\) and roll out using ground-truth actions but the model’s own predicted observations: \(\hat{s}_{t+1}\!\leftarrow\!g_c(f_c(\hat{s}_t,a_t),\hat{o}_{t+1})\), with \(H{=}\min(T_{\text{episode}},5)\). Token F1 at \(t{=}1\) is computed under episode-level filtering and is not directly comparable to Section 5.2.

Table 2: Macro-average rollout Token F1, unweighted over the sevenenvironments. Bold marks the best code-based score. Per-environment resultsare in Appendix 13.
Method \(t{=}1\) \(t{=}2\) \(t{=}3\) \(t{=}5\)
LLM-Direct 0.64 0.63 0.60 0.59
1-5 WorldCoder 0.63 0.60 0.58 0.57
PoE-World 0.63 0.57 0.53 0.51
PatchWorld-Simple (ours) 0.56 0.50 0.43 0.40
PatchWorld-Residual (ours) 0.69 0.64 0.60 0.58

5.3.0.1 Analysis.

Rollout stresses error propagation, because each step conditions on the model’s own previous output. PatchWorld reduces this drift by mapping text back into structured belief state before the next transition. As a result, PatchWorld-Residual has the best code-based macro score at every horizon, and PatchWorld-Simple remains competitive despite weaker surface text. This pattern supports the mechanism claim that executable belief state matters because it constrains how errors propagate, not only because it improves one-step prediction.

5.4 Planning frontier favors PatchWorld-Simple for code-based utility↩︎

We place each world model inside the same one-step lookahead planner. The planner compares the ReAct default action with up to four diverse candidates, rolls out each candidate with the world model, and reranks them with a shared Qwen selector. Thus the planning agent still uses LLMs for candidate proposal and selection; the isolated comparison is whether the lookahead next-observation predictor is an LLM call, a fine-tuned neural model, or a deterministic executable program. We evaluate up to 200 held-out episodes per environment with a 30-step cap; full protocol details and per-environment values are in Appendix 14 and 15.

Figure 3: Fidelity–utility Pareto frontier. Points compare one-step Token F1 (Table [tab:rq1-onestep]) with live lookahead success (Table 10). PatchWorld-Simple gives the best utility, while PatchWorld-Residual gives the best code-based fidelity. Word2World is more faithful but less useful for planning; WorldCoder and PoE-World are dominated. The ReAct line is the no-predictor baseline. All methods use Qwen3-Coder-480B except Word2World (Qwen3.5-4B SFT).

5.4.0.1 Analysis.

PatchWorld-Simple gives the highest code-based planning result in our evaluation, with 76.4% macro success. It is ahead of PoE-World (69.3%) and WorldCoder (64.4%) by 7.1 and 12.0 pp. It is also competitive with the LLM-based planners ReAct (74.4%) and LLM-Direct (75.8%), showing that a local symbolic transition program can provide useful action-selection signal at zero lookahead-prediction LLM calls (vs.897 tokens/task for LLM-Direct, Table 3). The largest gains are WebShop (+2.5 over ReAct, +12.5 over PoE-World) and TextCraft (+2.0, +25.1), where symbolic updates help the selector avoid infeasible actions. Word2World provides a useful counterpoint. It has the highest one-step fidelity (0.85 F1) but only 63.5% macro success, showing that high surface fidelity does not necessarily translate to action-contrastive signal.

5.4.0.2 Fidelity is not utility.

PatchWorld-Residual improves macro Token F1 (0.69 vs.) but lowers planning success (72.9 vs.), because the planner needs action-relevant contrast about what becomes reachable or feasible. Appendix 17 rules out identical retrieved text as the explanation; AlfWorld remains the main negative case, suggesting one-step lookahead helps only once receptacle-state fidelity crosses a task-dependent threshold. Induced programs are in Appendix 22.

Table 3: Planning-time prediction efficiency. Both rows share the candidategenerator, Qwen reranker, and 30-step cap; only the lookahead predictordiffers. Tokens count lookahead prediction.
Method Pred. tokens/task Macro success
LLM-Direct 63,897 75.8
PatchWorld-Simple 0 76.4

5.5 Ablation and repair diagnostics↩︎

Table 4 removes one component at a time from the full PatchWorld-Residual pipeline (split, backbone, and compute budget fixed). The largest dependency is contrastive mining (\(-0.20\) F1 under uniform sampling): without it the LLM sees redundant successful transitions and under-samples failure-disambiguating ones, so the first program already misses the conditional rules that repair would otherwise patch. Repair and residual memory each cost about \(0.09\) F1 but are non-redundant: the former fixes a brittle initial program, the latter is a fidelity add-on that more repair rounds cannot recover. The validation gate adds only \(0.006\) F1 on average yet keeps search grounded in executable replay, preventing candidates that fix shown counterexamples while regressing on the rest. Variant definitions are in Appendix 18.

Table 4: Component ablation. Avg.F1 is validation Token F1 averaged overthe seven environments; failed runs count as 0.
Variant Avg.F1 \(\Delta\)
Full 0.7262 +0.0000
\(-\)Residual memory 0.6310 \(-\)​0.0953
\(-\)Repair loop 0.6340 \(-\)​0.0922
\(-\)Validation gate 0.7204 \(-\)​0.0058
\(-\)Contrastive mining 0.5287 \(-\)​0.1976

PatchWorld shifts cost from inference to offline induction: validation and repair average 17–28 LLM calls per environment, reducing replay error by 12–79% (Appendix 19).

5.5.0.1 When does each variant win?

Cross-referencing per-environment fidelity (Table [tab:rq1-onestep]) with the residual-coverage diagnostic (Appendix 12), PatchWorld-Residual beats PatchWorld-Simple by the largest margin exactly where the train-key cache is dense and the renderer is templated: AlfWorld (+0.34), TextCraft (+0.43), Wordle (+0.14). The gap collapses on Maze and BabyAI (symbolic transition already covers surface variation) and turns slightly negative on WebShop (low coverage, browser fallback carries the load). Coverage thus acts as a cheap pre-deployment predictor of which variant to prefer.

5.5.0.2 What repair leaves on the table.

Typing post-repair counterexamples (Appendix 20) shows readout/rendering causing 53% of residual errors on average, dominant on AlfWorld, SciWorld, and Wordle, exactly where PatchWorld-Residual closes the gap, while compound transition rules (recipes, multi-attribute filters) drive the residual on TextCraft and WebShop.

6 Conclusion↩︎

PatchWorld turns offline trajectories into executable belief-state programs via counterexample-guided code repair. PatchWorld-Residual attains the highest code-based fidelity in our evaluation (0.69 macro Token F1), while PatchWorld-Simple attains the highest code-based planning utility (76.4% macro success, zero lookahead LLM calls), exposing a Pareto frontier grounded in value equivalence (§3.0.0.3). More broadly, LLMs can serve as symbolic optimizers that produce inspectable, executable, and locally repairable world models rather than models implicit in weights.

Limitations↩︎

This paper asks whether executable code induced from offline text-agent trajectories can support prediction and planning under partial observability. We use a simple one-step lookahead planner to isolate the induced world model rather than a learned controller, value function, or deep search procedure. We also induce one model per environment, so the results do not measure transfer across tasks or domains. Finally, the benchmark is limited to language-based environments, and our interpretability claims rely on inspectable programs and execution diagnostics rather than user studies.

7 PatchWorld Repair Loop Pseudocode↩︎

Algorithm 4 gives the full counterexample-guided search inlined as prose in Section 4.3.

Figure 4: PatchWorld Repair Loop: counterexample-guided patch search

8 Contrastive Transition Selection↩︎

Algorithm 5 gives the selection procedure used to build the prompt evidence set from induction trajectories.

Figure 5: Contrastive transition selection

9 Dataset Statistics↩︎

Table 5 reports per-environment instance, trajectory, and transition counts for the train/val/test splits used throughout Section 5. All methods are fit on the same train split and evaluated on the same test split.

Trajectories are collected with a Qwen3-Coder-480B-A35B-Instruct ReAct agent and split 60/20/20 by instance ID (not by transition or rollout) into train, validation, and test. This keeps repeated rollouts from the same task instance in one split and prevents instance-level overlap between train, validation, and test. The train split is used for both PatchWorld induction and Word2World fine-tuning; the validation split is used for hyperparameter selection and as the source of PatchWorld repair feedback; the test split is held out for all reported metrics. The PatchWorld-Residual index is built only from train transitions to avoid leakage.

Table 5: Dataset statistics per environment.
Train Val Test
2-4(lr)5-7(lr)8-10 Env Inst. Traj. Trans. Inst. Traj. Trans. Inst. Traj. Trans.
AlfWorld 1572 4,716 127,308 524 1,572 42,685 524 1,572 42,702
BabyAI 540 1,620 9,559 180 540 3,942 180 540 3,588
Maze 15 45 400 5 15 135 6 18 179
SciWorld 1355 4,065 87,851 451 1,353 29,757 453 1,359 29,346
TextCraft 284 852 6,701 94 282 1,822 96 288 2,174
WebShop 2478 7,434 63,747 826 2,478 21,070 826 2,478 21,360
Wordle 575 1,725 5,602 191 573 1,855 193 579 1,845

10 Seed Variance for One-Step Prediction↩︎

To estimate induction variability for PatchWorld, we repeat the Qwen3-Coder-480B PatchWorld-Residual pipeline five times with independent seeds, refitting both the program \(c\) and the residual index from scratch on each seed.

Table 6: Five-seed variance for PatchWorld-Residual (Qwen3-Coder-480B) on theheld-out test set. Mean \(\pm\) std over seeds.
Environment Token F1 BLEU-4
AlfWorld \(0.6982 \pm 0.0205\) \(0.4665 \pm 0.0188\)
BabyAI \(0.6878 \pm 0.0247\) \(0.4691 \pm 0.0264\)
Maze \(0.9663 \pm 0.0089\) \(0.9091 \pm 0.0142\)
SciWorld \(0.5618 \pm 0.0291\) \(0.4761 \pm 0.0253\)
TextCraft \(0.7104 \pm 0.0186\) \(0.4805 \pm 0.0172\)
WebShop \(0.4985 \pm 0.0224\) \(0.2603 \pm 0.0179\)
Wordle \(0.7363 \pm 0.0102\) \(0.4324 \pm 0.0176\)
Macro \(\mathbf{0.6942 \pm 0.0192}\) \(\mathbf{0.4991 \pm 0.0231}\)

The macro standard deviation (\(\approx 0.02\) Token F1) is small relative to the gap between PatchWorld-Residual and the strongest non-residual baseline (\(\approx 0.06\) Token F1), supporting the main-text effect-size reading.

11 BERTScore Consistency with Token F1 and BLEU-4↩︎

We re-score the held-out predictions of the five program-based RQ1 configurations with BERTScore [57] (roberta-large [58], English baseline, no re-scaling). The induced programs and residual indices are persisted on disk, so we replay test-time predictions deterministically with no new LLM calls and recover Token F1 / BLEU-4 matching the saved aggregates exactly on 33/35 (cell, env) cells (the remaining two differ by \(\le 0.04\) due to seed-dependent Wordle belief initialization).

Table 7: Macro per-cell results across the seven environments(BERTScore-F1 against gold next observation, averaged over \(\sim\)​100k–160kcovered transitions per cell). Bold marks the best score per column.
Configuration TokF1 BLEU4 BERT-F1
PatchWorld-Simple + Mimo-v2.5 0.471 0.272 0.916
PatchWorld-Simple + Qwen3-Coder-480B 0.635 0.404 0.934
PatchWorld-Residual + DeepSeek-V4-Flash 0.728 0.519 0.948
PatchWorld-Residual + Qwen3-Coder-480B 0.718 0.527 0.948
PatchWorld-Residual + Mimo-v2.5 0.699 0.493 0.950

BERTScore preserves the main-text ordering: all three PatchWorld-Residual variants outperform both PatchWorld-Simple variants (\(0.948\)–\(0.950\) vs.\(0.916\)–\(0.934\)), and within the residual group the three backbones are statistically indistinguishable (\(\Delta\le 0.002\)), consistent with the seed variance in Table 6. Across \(5\!\times\!10^5\) scored transitions, Pearson / Spearman correlation with Token F1 is \(0.93/0.93\) and with BLEU-4 is \(0.83/0.90\) (cell-, (cell, env)-, and transition-level correlations all agree to within \(\pm 0.05\)); the slightly lower BLEU-4 correlation is expected since BLEU-4 saturates whenever any \(n\)-gram precision is zero. The compressed dynamic range is the standard BERTScore caveat: bucketing transitions by Token F1 shows a noise floor near \(0.85\) (\(F_1{=}0\!:\!0.859\), \(F_1{\in}(0,0.25){:}\,0.834\), \(F_1{\in}[0.25,0.5){:}\,0.877\), \(F_1{\in}[0.5,0.75){:}\,0.933\), \(F_1{\ge}0.75{:}\,0.991\)). The only visible cell-level outlier (PatchWorld-Simple+Mimo on Maze, Token F1 \(=\) BLEU-4 \(=0\) but BERTScore \(0.85\)) sits exactly at this floor, not at a hidden success. We therefore keep Token F1 and BLEU-4 as primary fidelity metrics in the main text.

12 Residual Memory Coverage Diagnostic↩︎

To separate the symbolic program from the train-only residual, Table 8 reports how often an exact normalized observation and action key from train appears in the held-out test transitions. We also evaluate a retrieval-only diagnostic: on a cache hit, it returns the memorized majority next observation; on a miss, it returns no prediction, which counts as zero Token F1 in the all-transition score.

Table 8: Exact train-key residual coverage on the held-out test split. Hit F1 isToken F1 conditioned on cache hits; All F1 counts cache misses as zero.
Env. Hit rate Hit F1 All F1
AlfWorld 0.20 0.94 0.18
BabyAI 0.12 0.95 0.12
Maze 0.87 1.00 0.87
SciWorld 0.45 0.99 0.44
TextCraft 0.42 0.99 0.41
WebShop 0.00 0.00 0.00
Wordle 0.26 0.95 0.24
Macro 0.33 0.83 0.32

The residual is therefore high precision but not high coverage. In practice, PatchWorld-Residual improves rendering for recurring, unambiguous surface patterns while still relying on the executable program for most held-out transitions.

13 Per-Environment Rollout Results↩︎

We complement the macro summary in Table 2 with per-environment rollout Token F1. Each cell is computed under episode-level filtering: we keep only episodes long enough for the given horizon, and the \(t{=}1\) column is therefore not directly comparable to Table [tab:rq1-onestep].

Table 9: Rollout Token F1 by environment and horizon. Bold marks the bestcode-based score per cell; PatchWorld variants share theQwen3-Coder-480B backbone.
Env. Method \(t{=}1\) \(t{=}2\) \(t{=}3\) \(t{=}5\)
AlfWorld WorldCoder 0.62 0.59 0.57 0.56
PoE-World 0.62 0.55 0.50 0.46
PatchWorld-Simple 0.36 0.31 0.28 0.27
PatchWorld-Residual 0.69 0.62 0.58 0.57
BabyAI WorldCoder 0.78 0.75 0.74 0.74
PoE-World 0.77 0.71 0.68 0.66
PatchWorld-Simple 0.83 0.77 0.69 0.63
PatchWorld-Residual 0.69 0.65 0.62 0.62
Maze WorldCoder 0.83 0.81 0.79 0.78
PoE-World 0.83 0.78 0.74 0.70
PatchWorld-Simple 0.80 0.74 0.66 0.61
PatchWorld-Residual 0.97 0.94 0.90 0.85
SciWorld WorldCoder 0.40 0.37 0.35 0.34
PoE-World 0.39 0.34 0.31 0.30
PatchWorld-Simple 0.57 0.49 0.41 0.39
PatchWorld-Residual 0.55 0.49 0.44 0.41
TextCraft WorldCoder 0.88 0.85 0.83 0.82
PoE-World 0.78 0.73 0.70 0.69
PatchWorld-Simple 0.28 0.22 0.18 0.15
PatchWorld-Residual 0.71 0.66 0.64 0.63
WebShop WorldCoder 0.58 0.55 0.52 0.50
PoE-World 0.58 0.52 0.46 0.43
PatchWorld-Simple 0.53 0.46 0.38 0.36
PatchWorld-Residual 0.50 0.48 0.46 0.45
Wordle WorldCoder 0.31 0.29 0.28 0.28
PoE-World 0.43 0.38 0.32 0.30
PatchWorld-Simple 0.58 0.49 0.41 0.39
PatchWorld-Residual 0.72 0.65 0.55 0.50

13.0.0.1 Pattern.

PatchWorld-Residual is uniformly best on Maze, AlfWorld, and Wordle, tasks where templated readouts dominate, and matches or leads at long horizons on SciWorld and WebShop. WorldCoder retains an early-step advantage on TextCraft, consistent with the main-text observation that recipe-style deterministic dynamics already favor program induction. PatchWorld-Simple is strong on Maze and BabyAI at short horizons but degrades faster on token metrics. This supports the split between surface fidelity and utility, since its symbolic state remains useful (as Section 5.4 confirms), but its rendered text drifts from simulator templates.

14 Planner Protocol↩︎

For the live planning study, every world model is plugged into the same one-step lookahead planner, identical except for the predictor module:

  1. Belief update. On observation \(o_t\), the world model ingests \((o_{t-1},a_{t-1},o_t)\) and updates its internal state. For PatchWorld, this calls correct_belief; for WorldCoder/PoE-World, the program’s transition; for LLM-Direct, an ICL prompt that conditions on the last \(k{=}3\) transitions.

  2. Candidate generation. A ReAct policy proposes a default action \(a^{\text{default}}\). Up to four additional diverse candidates are drawn from the environment’s exposed action API (deduplicated and capped at eight total candidates, including the default).

  3. Lookahead rollout. For each candidate \(a^{(i)}\), the world model predicts \(\hat{o}_{t+1}^{(i)}\). No multi-step rollout is used in this study; the lookahead depth is exactly one.

  4. Reranking with a gate. A shared Qwen3-Coder-480B selector scores \((o_t, a^{(i)}, \hat{o}_{t+1}^{(i)})\) tuples. The gate falls back to \(a^{\text{default}}\) when (a) no candidate beats the default by a fixed margin or (b) the world model returns an empty / parse-failed prediction. This avoids penalizing strong reactive baselines when the world model adds no signal.

  5. Termination. Episodes terminate on environment success, environment failure, or a 30-step cap, whichever comes first. Each environment uses up to 200 held-out instances.

The selector, candidate cap, and step cap are identical across all rows in Figure 3 and Table 10.

15 Per-Task Planning Results↩︎

Table 10: Episode success rate (%) per environment under the sharedone-step lookahead planner. “Macro” is the unweighted mean over the sevenenvironments. Bold marks the best score per environment.
Method Alf. Baby. Maze Sci. Text. Web. Wor. Macro
ReAct 17.5 86.7 83.3 83.5 93.8 56.0 100.0 74.4
LLM-Direct 14.5 88.9 100.0 85.0 95.8 46.0 100.0 75.8
Word2World 9.6 75.6 83.3 66.7 86.5 22.6 100.0 63.5
WorldCoder 3.5 81.7 66.7 88.5 67.7 43.0 100.0 64.4
PoE-World 3.5 84.4 83.3 93.5 70.8 49.5 100.0 69.3
PatchWorld-Simple 6.0 87.8 100.0 86.5 95.8 58.5 100.0 76.4
PatchWorld-Residual 5.5 86.1 83.3 82.5 96.9 56.0 100.0 72.9

15.0.0.1 Where the gains come from.

PatchWorld-Simple’s macro lead is driven by WebShop (\(+2.5\) over ReAct) and competitive performance on Maze and TextCraft. Its AlfWorld score is below ReAct, mirroring the failure mode discussed in Section 5.4: under partial observability of receptacles, predicted observations omit objects and the reranker overrides correct defaults. PatchWorld-Residual’s lower planning score relative to PatchWorld-Simple motivates the contrast diagnostic in Appendix 17: higher token-level fidelity does not by itself guarantee that the lookahead text encodes the distinctions needed for action selection. Per-environment success rates are computed over up to 200 held-out instances (Maze and Wordle have fewer; see Table 5).

16 Planning Uncertainty↩︎

Table 11 reports macro planning success as a five-run average with standard deviation. We use the same unweighted macro estimator as Table 10.

Table 11: Macro episode success as five-run mean \(\pm\) standard deviation.
Method Macro success (%)
ReAct \(74.4 \pm 2.3\)
LLM-Direct \(75.8 \pm 0.8\)
Word2World \(63.5 \pm 1.2\)
WorldCoder \(64.4 \pm 2.9\)
PoE-World \(69.3 \pm 2.4\)
PatchWorld-Simple \(\mathbf{76.4 \pm 0.8}\)
PatchWorld-Residual \(72.9 \pm 2.3\)

The code-baseline gaps are larger than these standard deviations, while the difference between PatchWorld-Simple and LLM-Direct is small. We therefore treat the main planning conclusion as strongest for program-based world models and as parity with LLM-based lookahead.

17 Cross-Action Contrast Diagnostics↩︎

The planning result in Section 5.4 raises a mechanistic question: does the residual hurt planning because it collapses the planner’s candidate predictions into identical strings, or because its higher-fidelity text is not the right kind of contrast for action selection? We can test the first hypothesis directly from saved RQ3 planning reports without rerunning any environment. For each world-model lookahead step with candidate predictions \(\{\hat{o}_{t+1}^{(i)}\}_{i=1}^{n}\), we normalize whitespace and case, discard empty or parse-failed predictions, and compute three quantities: (i) the percentage of steps with fewer than two distinct usable predictions, (ii) the mean number of distinct predictions, and (iii) the mean pairwise SequenceMatcher distance \(1-\mathrm{ratio}(\hat{o}^{(i)},\hat{o}^{(j)})\) over candidate pairs, with long strings truncated to 500 normalized characters for speed. Larger values for (ii) and (iii) indicate more raw textual contrast across candidates.

Table 12: Cross-action contrast measured from saved Qwen3-Coder-480B RQ3planning logs across the seven environments. “Identical” is the percentageof planning steps with fewer than two distinct usable predicted observations.
Model Identical (%) Unique preds. Pairwise dist.
PatchWorld-Simple 6.4 4.73 0.42
PatchWorld-Residual 5.7 4.87 0.44

The result rules out the simplest collapse story. PatchWorld-Residual does not produce more fully identical candidate observations than PatchWorld-Simple, and it is marginally higher on both distinct predictions and pairwise textual distance. Thus the planning drop in Table 10 is not evidence that retrieval usually returns the exact same string for all candidates. Instead, the diagnostic clarifies what the Pareto frontier means. Token-level fidelity rewards the residual for reproducing simulator language; raw pairwise distance confirms that the residual still produces diverse strings; planning utility requires a stricter property, namely that those differences encode the counterfactual consequences of the candidate actions. The symbolic core can be worse at rendering but better aligned with this decision-relevant contrast. We therefore use the frontier not as a claim that retrieval always collapses text, but as evidence that world-model evaluation for agents needs a decision-facing axis in addition to observation-fidelity metrics.

18 Component Ablation Details↩︎

We ablate one component at a time from the full PatchWorld-Residual pipeline, holding split, backbone, and budget fixed. The variants in Table 4 are: \(-\)Residual memory, which reduces the model to PatchWorld-Simple; \(-\)Repair loop, which sets \(R{=}0\); \(-\)Validation gate, which accepts patches without the improves-replay rule; \(-\)Contrastive mining, which uses uniform sampling.

The largest drop comes from removing contrastive mining, which reduces validation F1 by 0.20 because the prompt loses coverage of rare action and outcome cases. Repair and residual memory each add about 0.09 F1, but on different axes: repair improves both fidelity and planning, whereas residual memory mainly improves fidelity and can hurt planning (Section 5.4). The validation gate provides a smaller gain in this aggregate table, but it keeps the search anchored to executable replay.

19 Detailed Induction Cost and Replay-Error Reduction↩︎

Table 13 gives the per-environment LLM-call, token, and repair-round counts behind the macro summary in Section [sec:finding-cost]. Table 14 reports the corresponding per-environment validation replay-error counts before and after the PatchWorld Repair Loop.

Three patterns are consistent with the main-text claim that the repair loop optimizes the formal replay objective and that the magnitude of reduction is gated by initial synthesis quality. First, within each backbone, longer horizons (SciWorld, AlfWorld) consume both more calls and more rounds. Second, \(\Delta\)Err is large and stable for Qwen3-Coder-480B and Mimo-v2.5 (\(\geq 65\%\) on every environment) but small for DeepSeek-V4-Flash, whose initial programs are already low-coverage and where additional repair rounds therefore have less to fix without a structural rewrite. Third, no environment under any backbone reaches exact convergence within \(R{=}15\), consistent with the validator’s strict round-trip criterion and the readout-dominated error mix in Table 15.

Table 13: Induction cost for PatchWorld. The world-modeling stage is moreexpensive than a single program-generation pass because the LLM repeatedlyreasons over validation failures and emits full candidate patches (averaging17.0–28.4 calls and 300k–503k tokens per environment). “Rounds” is attemptedrepair iterations before convergence, budget (\(R{=}15\)), or hill-climbingrejection. Token counts are rounded to thousands; wall-clock time is omittedbecause it is dominated by external API latency and rate limits.
Backbone Env Calls In tok. Out tok. Rounds Conv.?
Qwen3-Coder-480B AlfWorld 25 245.1k 97.1k 6 No
Qwen3-Coder-480B BabyAI 29 446.5k 97.0k 7 No
Qwen3-Coder-480B Maze 17 128.3k 37.6k 4 No
Qwen3-Coder-480B SciWorld 29 219.6k 126.2k 7 No
Qwen3-Coder-480B TextCraft 21 153.6k 84.3k 5 No
Qwen3-Coder-480B WebShop 45 1002.0k 181.2k 11 No
Qwen3-Coder-480B Wordle 33 227.5k 115.3k 8 No
2-7 Qwen3-Coder-480B Avg. 28.4 346.1k 105.5k 6.9 No
Mimo-v2.5 AlfWorld 25 230.8k 256.2k 6 No
Mimo-v2.5 BabyAI 17 331.0k 173.8k 4 No
Mimo-v2.5 Maze 17 78.5k 99.2k 4 No
Mimo-v2.5 SciWorld 29 377.7k 366.7k 7 No
Mimo-v2.5 TextCraft 45 303.6k 390.2k 11 No
Mimo-v2.5 WebShop 13 297.6k 130.2k 3 No
Mimo-v2.5 Wordle 29 209.0k 277.3k 7 No
2-7 Mimo-v2.5 Avg. 25.0 261.2k 241.9k 6.0 No
DeepSeek-V4-Flash AlfWorld 25 208.8k 98.1k 6 No
DeepSeek-V4-Flash BabyAI 13 304.7k 58.1k 3 No
DeepSeek-V4-Flash Maze 17 134.5k 39.5k 4 No
DeepSeek-V4-Flash SciWorld 17 207.3k 84.8k 4 No
DeepSeek-V4-Flash TextCraft 13 94.1k 37.0k 3 No
DeepSeek-V4-Flash WebShop 21 571.5k 79.5k 5 No
DeepSeek-V4-Flash Wordle 13 112.0k 69.3k 3 No
2-7 DeepSeek-V4-Flash Avg. 17.0 233.3k 66.6k 4.0 No
Table 14: PatchWorld Repair Loop optimization effect. Replay errors are measured oninduction-time validation trajectories before and after accepted candidatepatches; reduction is relative to the initial generated program.
Backbone Env Init. err. Final err. \(\Delta\) err. Red.
Qwen3-Coder-480B AlfWorld 7866 1059 6807 86.5%
Qwen3-Coder-480B BabyAI 2102 951 1151 54.8%
Qwen3-Coder-480B Maze 507 305 202 39.8%
Qwen3-Coder-480B SciWorld 3472 24 3448 99.3%
Qwen3-Coder-480B TextCraft 2028 404 1624 80.1%
Qwen3-Coder-480B WebShop 1060 552 508 47.9%
Qwen3-Coder-480B Wordle 475 446 29 6.1%
2-6 Qwen3-Coder-480B Total 17510 3741 13769 78.6%
Mimo-v2.5 AlfWorld 5720 1430 4290 75.0%
Mimo-v2.5 BabyAI 1564 518 1046 66.9%
Mimo-v2.5 Maze 508 20 488 96.1%
Mimo-v2.5 SciWorld 6113 1788 4325 70.8%
Mimo-v2.5 TextCraft 334 162 172 51.5%
Mimo-v2.5 WebShop 460 460 0 0.0%
Mimo-v2.5 Wordle 462 10 452 97.8%
2-6 Mimo-v2.5 Total 15161 4388 10773 71.1%
DeepSeek-V4-Flash AlfWorld 723 675 48 6.6%
DeepSeek-V4-Flash BabyAI 479 479 0 0.0%
DeepSeek-V4-Flash Maze 318 142 176 55.3%
DeepSeek-V4-Flash SciWorld 2665 2200 465 17.4%
DeepSeek-V4-Flash TextCraft 164 164 0 0.0%
DeepSeek-V4-Flash WebShop 1266 1232 34 2.7%
DeepSeek-V4-Flash Wordle 517 517 0 0.0%
2-6 DeepSeek-V4-Flash Total 6132 5409 723 11.8%

20 Validator Error Analysis↩︎

The validator records, for every replay failure, a typed error tag derived from which check fired: parser failure, transition mismatch, readout/render mismatch, or unhandled action. We aggregate these tags over the PatchWorld-Residual (Qwen3-Coder-480B) post-repair validation log.

Table 15: Distribution of post-repair validation errors by typed cluster.Rows sum to 100% within each environment. “Unhandled” covers actionsthe program rejects or for which it raises.
Env. Parser Transition Readout Unhandled
AlfWorld 6% 18% 63% 13%
BabyAI 4% 31% 49% 16%
Maze 3% 22% 60% 15%
SciWorld 9% 20% 58% 13%
TextCraft 5% 47% 32% 16%
WebShop 7% 44% 36% 13%
Wordle 3% 12% 75% 10%
Macro 5% 28% 53% 14%

20.0.0.1 Implication.

Readout/rendering accounts for over half of post-repair errors on average and dominates AlfWorld, SciWorld, and Wordle, which is precisely the regime where PatchWorld-Simple can have correct task-relevant beliefs while underperforming on Token F1 and BLEU-4 in Section 5.2, and where the residual closes the gap by overriding the symbolic readout with a memorized template. Belief and transition errors are concentrated in TextCraft and WebShop, where compound rules (recipes, multi-attribute filtering) stress the synthesized program’s expressivity. Two complementary directions follow: (i) environment-specific renderer templates that a program can fill in rather than re-emit, and (ii) state-update invariants enforced by the validator beyond exact textual match.

21 Qualitative PatchWorld repair example↩︎

Figure 6 shows a representative accepted patch from Maze. The initial program parsed any observation containing the token Success as terminal. This was wrong because the first Maze observation contains an instructional example before the current state. The validator surfaced a concrete counterexample: after move right, the ground truth moved the agent from \((1,2)\) to \((1,3)\) and reported a wall above, while the program predicted terminal Success or dropped the wall set. The accepted patch tightened terminal parsing and preserved wall information for blocked moves, reducing the round’s validation error count from 127 to 109.

None

Figure 6: Qualitative PatchWorld repair example. The validator turns a failed Maze transition into a localized code change..

22 Induced world models by environment↩︎

Figures [fig:wm-alfworld]–[fig:wm-wordle] show the complete Python programs induced by PatchWorld for each evaluation environment. Each program implements the shared BaseWorldModel interface: parse_observation, init_belief, correct_belief, predict_belief, readout_observation, and extract_valid_action_forms. The source files are also included in the code release under generated_world_models/.

23 Use of AI Assistants↩︎

We used AI-based tools for grammar checking and copyediting.

24 Risks↩︎

The main risk is that an induced executable model can appear interpretable while still encoding wrong rules, missing hidden state, or gaps inherited from limited trajectories. If used for prediction or planning without checks, such errors can lead to confident but incorrect decisions. Generated models should therefore be sandboxed, reviewed, and treated as diagnostic or assistive tools in higher-stakes settings, not as trusted simulators or decision makers.

from abductworld.worldmodel_base import BaseWorldModel
import re
from collections import defaultdict

class AlfworldWorldModel(BaseWorldModel):
    def __init__(self):
        super().__init__()
        self.receptacles = set()
        self.objects = set()
        self.object_locations = defaultdict(dict)  # object_id -> {receptacle_id: status}
        self.receptacle_states = defaultdict(str)  # receptacle_id -> state (open/closed)
        self.inventory = set()
        self.current_location = None
        self.valid_actions = {
            "go to": ["go to <receptacle>"],
            "take": ["take <object> from <receptacle>"],
            "move": ["move <object> to <receptacle>"],
            "examine": ["examine <object>"],
            "look": ["look"],
            "inventory": ["inventory"],
            "open": ["open <receptacle>"],
            "close": ["close <receptacle>"],
            "use": ["use <object>"],
            "heat": ["heat <object> with <receptacle>"],
            "clean": ["clean <object> with <receptacle>"]
        }

    def parse_observation(self, obs_text: str) -> dict:
        """Parse observation text into structured data"""
        parsed = {
            "location": None,
            "objects": [],
            "receptacles": [],
            "inventory": [],
            "message": obs_text.strip()
        }
        
        # Extract location
        location_match = re.search(r"You are in the middle of a room\. Looking quickly around you, you see (.+)$", obs_text)
        if location_match:
            items = location_match.group(1).split(", ")
            for item in items:
                if " " in item:
                    name, num = item.rsplit(" ", 1)
                    parsed["receptacles"].append(f"{name} {num}")
        else:
            # Check for desk-specific observation
            desk_match = re.search(r"You arrive at ([^\.]+)\. On the ([^,]+), you see (.+)", obs_text)
            if desk_match:
                parsed["location"] = desk_match.group(1)
                objects_str = desk_match.group(3)
                if objects_str != "nothing":
                    object_list = objects_str.split(", ")
                    for obj in object_list:
                        if " " in obj:
                            # Handle cases like "a alarmclock 1" or "alarmclock 1"
                            parts = obj.replace("a ", "").strip().split(" ")
                            if len(parts) >= 2:
                                obj_name = " ".join(parts[:-1])
                                obj_id = parts[-1]
                                parsed["objects"].append(f"{obj_name} {obj_id}")
        
        # Check for inventory
        if "You are not carrying anything" in obs_text:
            parsed["inventory"] = []
        elif "You are carrying:" in obs_text:
            inv_match = re.search(r"You are carrying: (.+)", obs_text)
            if inv_match:
                items = inv_match.group(1).split(", ")
                parsed["inventory"] = [item.strip() for item in items]
                
        return parsed

    def init_belief(self):
        """Initialize belief state with latent support"""
        return {
            "location": None,
            "receptacles": set(),
            "objects": set(),
            "object_locations": defaultdict(dict),
            "receptacle_states": defaultdict(str),
            "inventory": set(),
            "last_action": None,
            # Adding latent belief fields to satisfy validator
            "latent_variables": {},
            "facts": set(),
            "hypotheses": [],
            "frontier": set(),
            "hidden_state": {}
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Update belief state based on observation"""
        belief = belief_prior.copy()
        # Preserve latent fields
        if "latent_variables" not in belief:
            belief["latent_variables"] = {}
        if "facts" not in belief:
            belief["facts"] = set()
        if "hypotheses" not in belief:
            belief["hypotheses"] = []
        if "frontier" not in belief:
            belief["frontier"] = set()
        if "hidden_state" not in belief:
            belief["hidden_state"] = {}
            
        parsed = self.parse_observation(obs_text)
        
        # Update location
        if parsed["location"]:
            belief["location"] = parsed["location"]
            
        # Update inventory
        belief["inventory"] = set(parsed["inventory"])
        
        # Update object locations from "On the X, you see..." patterns
        on_pattern = r"On the ([^,]+), you see (.+)"
        on_matches = re.findall(on_pattern, obs_text)
        for receptacle, objects_str in on_matches:
            if objects_str != "nothing":
                object_list = objects_str.split(", ")
                for obj in object_list:
                    obj_clean = obj.replace("a ", "").strip()
                    if " " in obj_clean:
                        belief["object_locations"][obj_clean][receptacle] = "on"
                        belief["objects"].add(obj_clean)
        
        # Update receptacle states
        if "is open" in obs_text:
            open_matches = re.findall(r"The ([^ ]+ [0-9]+) is open", obs_text)
            for receptacle in open_matches:
                belief["receptacle_states"][receptacle] = "open"
                
        if "is closed" in obs_text:
            closed_matches = re.findall(r"The ([^ ]+ [0-9]+) is closed", obs_text)
            for receptacle in closed_matches:
                belief["receptacle_states"][receptacle] = "closed"
                
        return belief

    def predict_belief(self, belief, action: str):
        """Predict next belief state given action"""
        # Create a deep copy that preserves all fields including latent ones
        next_belief = {}
        for key, value in belief.items():
            if isinstance(value, (set, list)):
                next_belief[key] = type(value)(value)
            elif isinstance(value, dict):
                next_belief[key] = value.copy()
            else:
                next_belief[key] = value
        
        next_belief["last_action"] = action
        
        # Parse action
        action = action.strip().lower()
        
        if action.startswith("go to "):
            location = action[6:]  # Remove "go to "
            next_belief["location"] = location
            
        elif action.startswith("take "):
            # take <object> from <receptacle>
            match = re.match(r"take (.+) from (.+)", action)
            if match:
                obj, receptacle = match.groups()
                # Remove from receptacle location
                if obj in next_belief["object_locations"]:
                    next_belief["object_locations"][obj].pop(receptacle, None)
                # Add to inventory
                next_belief["inventory"].add(obj)
                
        elif action.startswith("move "):
            # move <object> to <receptacle>
            match = re.match(r"move (.+) to (.+)", action)
            if match:
                obj, receptacle = match.groups()
                # Remove from inventory if present
                next_belief["inventory"].discard(obj)
                # Add to receptacle location
                next_belief["object_locations"][obj][receptacle] = "on"
                
        elif action.startswith("open "):
            receptacle = action[5:]  # Remove "open "
            next_belief["receptacle_states"][receptacle] = "open"
            
        elif action.startswith("close "):
            receptacle = action[6:]  # Remove "close "
            next_belief["receptacle_states"][receptacle] = "closed"
            
        return next_belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Generate observation text from belief state"""
        # For simplicity, return a generic success message for now
        # In a full implementation, this would reconstruct the observation
        # based on the belief state and action taken
        if belief["last_action"]:
            action_type = belief["last_action"].split()[0]
            if action_type in ["take", "move", "open", "close"]:
                return f"You {belief['last_action']}."
            elif action_type == "go":
                return f"You arrive at {belief['location']}."
            elif action_type == "examine":
                return "Nothing happens."
            elif action_type == "look":
                return "Nothing happens."
            elif action_type == "inventory":
                if belief["inventory"]:
                    items = ", ".join(sorted(belief["inventory"]))
                    return f"You are carrying: {items}."
                else:
                    return "You are not carrying anything."
        
        return "Nothing happens."

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Return valid action templates"""
        return self.valid_actions
from abductworld.worldmodel_base import BaseWorldModel
import re
from collections import defaultdict

class BabyaiWorldModel(BaseWorldModel):
    def parse_observation(self, obs_text: str) -> dict:
        """Parse observation text into structured data"""
        result = {
            'objects': [],
            'carrying': None,
            'facing': None,
            'goal': None,
            'walls': True
        }
        
        # Extract goal if present
        goal_match = re.search(r'Your goal: (.+)', obs_text)
        if goal_match:
            result['goal'] = goal_match.group(1)
        
        # Extract carrying information
        carrying_match = re.search(r'You are carrying (?:a|an|the) ([^.]+)', obs_text)
        if carrying_match:
            result['carrying'] = carrying_match.group(1)
        elif 'You are not carrying anything' in obs_text:
            result['carrying'] = None
            
        # Extract facing information
        facing_match = re.search(r'You are facing (?:a|an|the) ([^.]+)', obs_text)
        if facing_match:
            result['facing'] = facing_match.group(1)
        elif (facing_wall := re.search(r'You are facing a wall(?: (\d+) steps away)?', obs_text)):
            steps = facing_wall.group(1) if facing_wall.group(1) else "unknown"
            result['facing'] = f"wall {steps} steps away"
        elif 'You are facing a wall' in obs_text:
            result['facing'] = "wall"
            
        # Extract objects - more robust pattern
        if "In front of you in this room, you can see several objects:" in obs_text:
            object_section = obs_text.split("In front of you in this room, you can see several objects:")[1]
            if "You are facing" in object_section:
                object_section = object_section.split("You are facing")[0]
            
            # Handle the case where objects are listed
            object_lines = object_section.strip()
            
            # Pattern to match objects like: "There is a red box 1 1 steps in front of you and 2 steps to your left."
            object_pattern = r'There is (?:a|an) ([^,]+?) (\d+)(?: right in front of you )?(?:(\d+) steps away|(\d+) steps in front of you)?(?: and (\d+) steps to your (left|right))?[,.]'
            
            for match in re.finditer(object_pattern, object_lines):
                obj_name = match.group(1).strip()
                obj_id = match.group(2)
                
                # Determine front steps
                front_steps = "0"
                if match.group(3):  # steps away
                    front_steps = match.group(3)
                elif match.group(4):  # steps in front of you
                    front_steps = match.group(4)
                elif 'right in front of you' in match.group(0):
                    front_steps = "0"
                
                # Determine lateral position
                lateral_steps = 0
                lateral_dir = 'center'
                if match.group(5) and match.group(6):  # Has lateral position
                    lateral_steps = int(match.group(5))
                    lateral_dir = match.group(6)
                
                full_name = f"{obj_name} {obj_id}"
                result['objects'].append({
                    'name': obj_name,
                    'id': obj_id,
                    'full_name': full_name,
                    'front_steps': int(front_steps),
                    'lateral_steps': lateral_steps,
                    'lateral_dir': lateral_dir
                })
        
        return result

    def init_belief(self):
        """Initialize belief state with latent support"""
        return {
            'objects': [],
            'carrying': None,
            'facing': None,
            'position': (0, 0),  # (front, lateral) coordinates
            'orientation': 0,    # 0=north, 1=east, 2=south, 3=west
            'goal': None,
            'latent_variables': {},  # Add latent support
            'facts': set(),         # Track known facts
            'hypotheses': {},       # Track hypotheses about hidden state
            'frontier': set(),      # Track frontier of exploration
            'hidden_state': {}      # General hidden state tracking
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Correct belief state based on observation"""
        parsed = self.parse_observation(obs_text)
        belief = belief_prior.copy()
        
        # Deep copy lists and dicts
        belief['objects'] = [obj.copy() for obj in parsed['objects']]
        belief['carrying'] = parsed['carrying']
        belief['facing'] = parsed['facing']
        belief['goal'] = parsed['goal']
        
        # Maintain latent variables
        if 'latent_variables' not in belief:
            belief['latent_variables'] = {}
        if 'facts' not in belief:
            belief['facts'] = set()
        if 'hypotheses' not in belief:
            belief['hypotheses'] = {}
        if 'frontier' not in belief:
            belief['frontier'] = set()
        if 'hidden_state' not in belief:
            belief['hidden_state'] = {}
        
        return belief

    def predict_belief(self, belief, action: str):
        """Predict next belief state given action"""
        new_belief = {}
        for k, v in belief.items():
            if isinstance(v, list):
                new_belief[k] = v.copy()
            elif isinstance(v, set):
                new_belief[k] = v.copy()
            elif isinstance(v, dict):
                new_belief[k] = v.copy()
            else:
                new_belief[k] = v
        
        action_lower = action.lower().strip()
        
        if action_lower.startswith('turn left'):
            new_belief['orientation'] = (new_belief['orientation'] - 1) % 4
        elif action_lower.startswith('turn right'):
            new_belief['orientation'] = (new_belief['orientation'] + 1) % 4
        elif action_lower.startswith('move forward'):
            # Movement doesn't change object positions in this environment
            pass
        elif action_lower.startswith('pickup'):
            # Remove picked up object from scene
            obj_name_match = re.search(r'pickup ([^,]+)$', action_lower)
            if obj_name_match:
                obj_name = obj_name_match.group(1)
                new_belief['objects'] = [obj for obj in new_belief['objects'] if obj['full_name'] != obj_name]
                new_belief['carrying'] = obj_name
        elif action_lower.startswith('drop'):
            # Drop currently carried object in front of agent
            if new_belief['carrying']:
                # Check if there's already an object right in front
                front_objects = [obj for obj in new_belief['objects'] if obj['front_steps'] == 0 and obj['lateral_steps'] == 0]
                if not front_objects:  # Only drop if no object is already in front
                    # Add the dropped object to the front of the agent
                    dropped_obj = {
                        'name': new_belief['carrying'].split(' ')[0],  # Extract object type
                        'id': new_belief['carrying'].split(' ')[1] if len(new_belief['carrying'].split(' ')) > 1 else '1',
                        'full_name': new_belief['carrying'],
                        'front_steps': 0,  # Dropped right in front
                        'lateral_steps': 0,
                        'lateral_dir': 'center'
                    }
                    new_belief['objects'].append(dropped_obj)
                new_belief['carrying'] = None
        elif action_lower.startswith('go to'):
            # Navigation action - moves agent to object location
            obj_name_match = re.search(r'go to ([^,]+)$', action_lower)
            if obj_name_match:
                target_obj_name = obj_name_match.group(1)
                # Find target object
                target_obj = None
                for obj in new_belief['objects']:
                    if obj['full_name'] == target_obj_name:
                        target_obj = obj
                        break
                if target_obj:
                    # Agent moves to object position - object is now in front
                    new_belief['objects'] = [obj for obj in new_belief['objects'] if obj['full_name'] != target_obj_name]
                    target_obj['front_steps'] = 0
                    target_obj['lateral_steps'] = 0
                    target_obj['lateral_dir'] = 'center'
                    new_belief['objects'].append(target_obj)
        elif action_lower.startswith('toggle and go through') or action_lower.startswith('go through'):
            # Door traversal - removes door from scene
            door_match = re.search(r'(?:toggle and go through|go through) ([^,]+)$', action_lower)
            if door_match:
                door_name = door_match.group(1)
                new_belief['objects'] = [obj for obj in new_belief['objects'] if obj['full_name'] != door_name]
        
        return new_belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Generate observation text from belief state"""
        # Handle check available actions specially
        if action.lower().strip() == "check available actions":
            return self._generate_available_actions_response(belief)
        
        lines = []
        
        if belief['goal']:
            lines.append(f"Your goal: {belief['goal']}")
            
        if belief['objects']:
            lines.append("In front of you in this room, you can see several objects:")
            for obj in belief['objects']:
                if obj['lateral_steps'] == 0 and (obj['lateral_dir'] == 'center' or obj['lateral_dir'] == ''):
                    if obj['front_steps'] == 0:
                        lines.append(f"There is a {obj['name']} {obj['id']} right in front of you.")
                    else:
                        lines.append(f"There is a {obj['name']} {obj['id']} {obj['front_steps']} steps in front of you.")
                else:
                    direction = obj['lateral_dir']
                    lines.append(f"There is a {obj['name']} {obj['id']} {obj['front_steps']} steps in front of you and {obj['lateral_steps']} steps to your {direction}.")
        else:
            lines.append("In front of you in this room, you can see several objects: The room has walls around you.")
            
        if belief['facing']:
            lines.append(f"You are facing {belief['facing']}.")
        else:
            lines.append("You are facing a wall.")
            
        if belief['carrying']:
            lines.append(f"You are carrying {belief['carrying']}.")
        else:
            lines.append("You are not carrying anything.")
            
        return " ".join(lines)

    def _generate_available_actions_response(self, belief) -> str:
        """Generate response for 'check available actions' command"""
        # This would normally extract available actions from the belief state
        # For now, we'll return a placeholder that indicates the action was processed
        return "You can take the following actions: turn left, turn right, move forward, check available actions"

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Extract valid action templates"""
        return {
            "go to": ["go to <object>"],
            "pickup": ["pickup <object>"],
            "drop": ["drop"],
            "toggle": ["toggle", "toggle and go through <door>"],
            "go through": ["go through <door>"],
            "move": ["move forward"],
            "turn left": ["turn left"],
            "turn right": ["turn right"],
            "check": ["check available actions"]
        }
from abductworld.worldmodel_base import BaseWorldModel
import re

class MazeWorldModel(BaseWorldModel):
    def parse_observation(self, obs_text: str) -> dict:
        """Parse observation text into structured data."""
        # Extract goal position
        goal_match = re.search(r"The goal is at position (\d+), (\d+)", obs_text)
        if not goal_match:
            return None
        goal_x, goal_y = int(goal_match.group(1)), int(goal_match.group(2))
        
        # Extract current position
        pos_match = re.search(r"Your current position is at position (\d+), (\d+)", obs_text)
        if not pos_match:
            return None
        pos_x, pos_y = int(pos_match.group(1)), int(pos_match.group(2))
        
        # Extract wall information
        walls = []
        if "wall above" in obs_text or "walls above" in obs_text or "above you" in obs_text:
            walls.append("up")
        if "wall below" in obs_text or "walls below" in obs_text or "below you" in obs_text:
            walls.append("down")
        if "wall to your left" in obs_text or "walls to your left" in obs_text or "to your left" in obs_text:
            walls.append("left")
        if "wall to your right" in obs_text or "walls to your right" in obs_text or "to your right" in obs_text:
            walls.append("right")
            
        return {
            "goal": (goal_x, goal_y),
            "position": (pos_x, pos_y),
            "walls": walls
        }

    def init_belief(self):
        """Initialize belief state."""
        return {
            "goal": None,
            "position": None,
            "walls": []
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Update belief state with new observation."""
        parsed = self.parse_observation(obs_text)
        if parsed is None:
            return belief_prior
            
        # Update belief with parsed information
        belief = belief_prior.copy()
        if parsed["goal"] is not None:
            belief["goal"] = parsed["goal"]
        if parsed["position"] is not None:
            belief["position"] = parsed["position"]
        if parsed["walls"] is not None:
            belief["walls"] = parsed["walls"]
            
        return belief

    def predict_belief(self, belief, action: str):
        """Predict next belief state given action."""
        # Create a copy of current belief
        next_belief = belief.copy()
        
        # Parse action
        action = action.strip().lower()
        
        # Update position based on action if it's a valid move
        if next_belief["position"] is not None:
            x, y = next_belief["position"]
            
            # Apply movement (based on examples: right increases y, down increases x)
            # Fix: when moving down, x should increase; when moving up, x should decrease
            if action == "move up" and "up" not in next_belief["walls"]:
                x -= 1
            elif action == "move down" and "down" not in next_belief["walls"]:
                x += 1
            elif action == "move left" and "left" not in next_belief["walls"]:
                y -= 1
            elif action == "move right" and "right" not in next_belief["walls"]:
                y += 1
                
            next_belief["position"] = (x, y)
            
        return next_belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Convert belief state back to observation text."""
        if belief["goal"] is None or belief["position"] is None:
            return ""
            
        goal_x, goal_y = belief["goal"]
        pos_x, pos_y = belief["position"]
        
        # Start building the observation text
        obs_text = f"The goal is at position {goal_x}, {goal_y}. Your current position is at position {pos_x}, {pos_y}."
        
        # Add wall information
        walls = belief["walls"]
        if walls:
            wall_descriptions = []
            # Maintain deterministic ordering
            if "up" in walls:
                wall_descriptions.append("above you")
            if "down" in walls:
                wall_descriptions.append("below you")
            if "left" in walls:
                wall_descriptions.append("to your left")
            if "right" in walls:
                wall_descriptions.append("to your right")
                
            if len(wall_descriptions) == 1:
                obs_text += f" There is a wall {wall_descriptions[0]}."
            else:
                obs_text += f" There are walls {', '.join(wall_descriptions)}."
        else:
            obs_text += "."
                
        return obs_text

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Return valid action forms for this domain."""
        return {
            "move <X>": ["move up", "move down", "move left", "move right"]
        }
from abductworld.worldmodel_base import BaseWorldModel
import re

class SciworldWorldModel(BaseWorldModel):
    def parse_observation(self, obs_text: str) -> dict:
        """
        Parse observation text into structured data.
        """
        # Basic parsing - extract key elements
        parsed = {
            "raw_text": obs_text,
            "room": "",
            "objects": [],
            "inventory": [],
            "actions": [],
            "task": "",
            "room_contents": {},
            "object_states": {},
            "container_contents": {}
        }
        
        # Extract room name
        room_match = re.search(r"This room is called the ([^.]+)\.", obs_text)
        if room_match:
            parsed["room"] = room_match.group(1).strip()
            
        # Extract task description
        task_match = re.search(r"Task description:\s*([^.]+(?:\.[^.]+)*)", obs_text)
        if task_match:
            parsed["task"] = task_match.group(1)
            
        # Extract inventory
        inv_match = re.search(r"In your inventory, you see:\s*((?:\s*.+)+?)(?=\n\n|\Z)", obs_text)
        if inv_match:
            items = inv_match.group(1).strip().split("\n")
            parsed["inventory"] = [item.strip() for item in items if item.strip() and item.strip() != "nothing"]
        elif "In your inventory, you see:" in obs_text and "nothing" in obs_text:
            parsed["inventory"] = []
            
        # Extract room contents
        contents_match = re.search(r"In it, you see:((?:\n\t.+(?:\n\t\t.+)*)+)", obs_text)
        if contents_match:
            contents_text = contents_match.group(1)
            # Parse objects and their states/contents
            lines = contents_text.strip().split('\n')
            current_parent = None
            for line in lines:
                if line.startswith('\t\t'):
                    # This is content of a container
                    if current_parent:
                        content = line.strip()
                        if current_parent not in parsed["container_contents"]:
                            parsed["container_contents"][current_parent] = []
                        parsed["container_contents"][current_parent].append(content)
                elif line.startswith('\t'):
                    # This is an object
                    obj_desc = line.strip()
                    obj_name = obj_desc.split('.')[0] if '.' in obj_desc else obj_desc
                    parsed["objects"].append(obj_name)
                    current_parent = obj_name
                    
                    # Check for states like open/closed, on/off
                    if 'open' in obj_desc:
                        parsed["object_states"][obj_name] = 'open'
                    elif 'closed' in obj_desc:
                        parsed["object_states"][obj_name] = 'closed'
                    elif 'on' in obj_desc:
                        parsed["object_states"][obj_name] = 'on'
                    elif 'off' in obj_desc:
                        parsed["object_states"][obj_name] = 'off'
                        
        return parsed

    def init_belief(self):
        """
        Initialize belief state with latent support.
        """
        return {
            "room": "unknown",
            "objects": {},
            "inventory": [],
            "task": "",
            "room_contents": {},
            "object_states": {},
            "container_contents": {},
            "latent_variables": {
                "door_states": {},
                "container_states": {},
                "object_locations": {}
            },
            "facts": set(),
            "hypotheses": [],
            "frontier": set(),
            "hidden_state": {}
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """
        Update belief state based on new observation.
        """
        parsed = self.parse_observation(obs_text)
        belief = belief_prior.copy()
        
        if parsed["room"]:
            # Fix room name normalization issue
            room_name = parsed["room"].strip()
            if room_name.startswith("the "):
                room_name = room_name[4:]
            elif room_name.startswith("LOC "):
                room_name = room_name[4:]
            belief["room"] = room_name
            
        if parsed["task"]:
            belief["task"] = parsed["task"]
            
        if parsed["inventory"]:
            belief["inventory"] = parsed["inventory"].copy()
        elif "In your inventory, you see:" in obs_text and "nothing" in obs_text:
            belief["inventory"] = []
        # Only clear inventory if explicitly stated as empty, otherwise preserve it
            
        # Update room contents
        if parsed["objects"]:
            belief["room_contents"] = {obj: True for obj in parsed["objects"]}
            
        # Update object states
        belief["object_states"].update(parsed["object_states"])
        
        # Update container contents
        belief["container_contents"].update(parsed["container_contents"])
        
        # Maintain latent variables
        if "latent_variables" not in belief:
            belief["latent_variables"] = {
                "door_states": {},
                "container_states": {},
                "object_locations": {}
            }
            
        if "facts" not in belief:
            belief["facts"] = set()
            
        if "hypotheses" not in belief:
            belief["hypotheses"] = []
            
        if "frontier" not in belief:
            belief["frontier"] = set()
            
        if "hidden_state" not in belief:
            belief["hidden_state"] = {}
            
        return belief

    def predict_belief(self, belief, action: str):
        """
        Predict next belief state given current belief and action.
        """
        belief_next = belief.copy()
        
        # Handle specific actions that change state
        if action.startswith("open "):
            obj_name = action[5:]  # Remove "open "
            if obj_name not in belief_next["object_states"]:
                belief_next["object_states"][obj_name] = "open"
            else:
                belief_next["object_states"][obj_name] = "open"
                
        elif action.startswith("pick up "):
            obj_name = action[8:]  # Remove "pick up "
            if obj_name not in belief_next["inventory"]:
                belief_next["inventory"].append(obj_name)
            # Remove from room contents if present
            if obj_name in belief_next.get("room_contents", {}):
                del belief_next["room_contents"][obj_name]
                
        elif action.startswith("put down "):
            obj_name = action[9:]  # Remove "put down "
            if obj_name in belief_next["inventory"]:
                belief_next["inventory"].remove(obj_name)
                
        elif action.startswith("go to "):
            room_name = action[6:]  # Remove "go to "
            # Fix room name normalization
            if room_name.startswith("the "):
                room_name = room_name[4:]
            elif room_name.startswith("LOC "):
                room_name = room_name[4:]
            belief_next["room"] = room_name
            
        elif action.startswith("examine ") or action.startswith("look at "):
            # These actions can change the current room context
            location = action[8:] if action.startswith("examine ") else action[8:]  # Remove "examine " or "look at "
            # If it's a room name, update the current room
            rooms = ["kitchen", "hallway", "greenhouse", "bathroom", "living room", "bedroom", "workshop", "art studio", "foundry"]
            # Normalize location name
            loc_clean = location
            if loc_clean.startswith("the "):
                loc_clean = loc_clean[4:]
            if loc_clean in rooms:
                belief_next["room"] = loc_clean
                
        # Ensure all required latent fields exist
        required_fields = ["latent_variables", "facts", "hypotheses", "frontier", "hidden_state"]
        for field in required_fields:
            if field not in belief_next:
                if field == "latent_variables":
                    belief_next[field] = {
                        "door_states": {},
                        "container_states": {},
                        "object_locations": {}
                    }
                elif field == "facts":
                    belief_next[field] = set()
                elif field == "hypotheses":
                    belief_next[field] = []
                elif field == "frontier":
                    belief_next[field] = set()
                elif field == "hidden_state":
                    belief_next[field] = {}
                    
        return belief_next

    def readout_observation(self, belief, action: str = "") -> str:
        """
        Generate observation text from belief state.
        """
        if action.startswith("look around") or action == "look around":
            # Generate detailed room description
            room_name = belief.get('room', 'unknown')
            obs_lines = [f"This room is called the {room_name}."]
            
            if belief.get("room_contents") or belief.get("inventory"):
                obs_lines.append("In it, you see:")
                
                # Add room objects
                for obj_name in belief.get("room_contents", {}):
                    obj_line = f"\t{obj_name}"
                    if obj_name in belief.get("object_states", {}):
                        state = belief["object_states"][obj_name]
                        if state in ["open", "closed"]:
                            obj_line += f". The {obj_name} is {state}."
                        elif state in ["on", "off"]:
                            obj_line += f", which is turned {state}."
                    
                    # Check for container contents
                    if obj_name in belief.get("container_contents", {}):
                        contents = belief["container_contents"][obj_name]
                        if contents:
                            obj_line += f". On the {obj_name} is: {', '.join(contents)}."
                        else:
                            obj_line += f". On the {obj_name} is: nothing."
                    elif any(keyword in obj_name for keyword in ["drawer", "cupboard", "fridge", "oven", "freezer"]):
                        obj_line += ". The door is closed."
                        
                    obs_lines.append(obj_line)
                
                # Always add the agent to room contents when looking around
                obs_lines.append("\tthe agent")
                    
            return "\n".join(obs_lines)
            
        elif action.startswith("inventory") or action == "inventory":
            if belief.get("inventory"):
                items = "\n\t".join(belief["inventory"])
                return f"In your inventory, you see:\n\t{items}"
            else:
                return "In your inventory, you see:\n\tnothing"
                
        elif action.startswith("task") or action == "task":
            if belief.get("task"):
                return f"Task description:\n{belief['task']}"
            else:
                return "No task specified."
                
        elif action.startswith("open "):
            obj_name = action[5:]  # Remove "open "
            return f"The {obj_name} is now open."
            
        elif action.startswith("look in "):
            container_name = action[8:]  # Remove "look in "
            if container_name in belief.get("container_contents", {}):
                contents = belief["container_contents"][container_name]
                if contents:
                    content_list = "\n\t".join(contents)
                    return f"Inside the {container_name} is: \n\t{content_list}"
                else:
                    return f"Inside the {container_name} is: \n\tnothing"
            else:
                return f"Inside the {container_name} is: \n\tnothing"
                
        elif action.startswith("pick up "):
            obj_name = action[8:]  # Remove "pick up "
            return f"You move the {obj_name} to the inventory."
            
        elif action.startswith("focus on "):
            obj_name = action[9:]  # Remove "focus on "
            return f"You focus on the {obj_name}."
            
        elif action.startswith("look at "):
            obj_name = action[8:]  # Remove "look at "
            # Try to find object description in belief
            if obj_name in belief.get("object_states", {}):
                state = belief["object_states"][obj_name]
                if "door" in obj_name:
                    return f"A door to the {obj_name.replace(' door', '')} (that is {state})"
                else:
                    return f"a {obj_name}, which is turned {state}."
            elif obj_name in ["kitchen", "hallway", "greenhouse", "bathroom", "living room", "bedroom", "workshop", "art studio", "foundry"]:
                # Looking at a room
                return f"This room is called the {obj_name}."
            else:
                return f"a {obj_name}"
                
        elif action.startswith("examine "):
            obj_name = action[8:]  # Remove "examine "
            if obj_name in ["kitchen", "hallway", "greenhouse", "bathroom", "living room", "bedroom", "workshop", "art studio", "foundry"]:
                # Examining a room - return basic room description
                return f"This room is called the {obj_name}."
            else:
                return f"a {obj_name}"
                
        elif action.startswith("go to "):
            room_name = action[6:]  # Remove "go to "
            # Normalize room name
            if room_name.startswith("the "):
                room_name = room_name[4:]
            return f"You move to the {room_name}."
            
        elif action == "wait" or action == "wait1":
            return "You decide to wait for 10 iterations."
            
        elif action == "":
            # Default response for ambiguous situations
            return "Ambiguous request: Please enter the number for the action you intended (or blank to cancel):"
            
        else:
            # For unhandled actions, return a generic but non-empty response
            return f"You perform the action: {action}"

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """
        Return valid action templates.
        """
        return {
            "look at": ["look at <object>"],
            "examine": ["examine <object>"],
            "focus on": ["focus on <object>"],
            "pick up": ["pick up <object>"],
            "put down": ["put down <object>"],
            "move": ["move <object> to <location>"],
            "go to": ["go to <location>"],
            "open": ["open <object>"],
            "close": ["close <object>"],
            "activate": ["activate <object>"],
            "deactivate": ["deactivate <object>"],
            "connect": ["connect <object> to <object>"],
            "disconnect": ["disconnect <object>"],
            "use": ["use <object> on <object>"],
            "pour": ["pour <object> in <object>"],
            "dunk": ["dunk <object> in <object>"],
            "mix": ["mix <object>"],
            "read": ["read <object>"],
            "inventory": ["inventory"],
            "task": ["task"],
            "look around": ["look around"],
            "wait": ["wait"],
            "wait1": ["wait1"],
            "look in": ["look in <object>"]
        }
from abductworld.worldmodel_base import BaseWorldModel
from collections import defaultdict
import re

class TextcraftWorldModel(BaseWorldModel):
    def parse_observation(self, obs_text: str) -> dict:
        """Parse observation text into structured data."""
        obs = {
            "inventory": {},
            "crafting_recipes": [],
            "goal": None,
            "message": None,
            "goal_item": None,
            "last_action_outcome": None
        }
        
        lines = obs_text.strip().split('\n')
        for line in lines:
            line = line.strip()
            if line.startswith("Inventory:"):
                if "You are not carrying anything" in line:
                    obs["inventory"] = {}
                else:
                    # Parse inventory items like [bricks] (1) [brick] (6)
                    items = re.findall(r'$([^$]+)$$(\d+)', line)
                    for item, count in items:
                        obs["inventory"][item.strip()] = int(count)
            elif line.startswith("Crafting commands:"):
                continue
            elif line.startswith("craft "):
                obs["crafting_recipes"].append(line)
            elif line.startswith("Goal:"):
                goal_text = line.replace("Goal:", "").strip()
                obs["goal"] = goal_text
                # Extract the goal item (e.g., "minecraft:stone_pickaxe")
                goal_match = re.search(r'minecraft:([a-z_]+)', goal_text)
                if goal_match:
                    obs["goal_item"] = goal_match.group(1)
            elif line.startswith("Got "):
                obs["message"] = line
                obs["last_action_outcome"] = "got"
            elif line.startswith("Crafted "):
                obs["message"] = line
                obs["last_action_outcome"] = "crafted"
            elif line.startswith("Could not execute") or "Could not find" in line:
                obs["message"] = line
                if "find enough items" in line:
                    obs["last_action_outcome"] = "not_enough_items"
                elif "find a valid recipe" in line:
                    obs["last_action_outcome"] = "invalid_recipe"
                else:
                    obs["last_action_outcome"] = "not_found"
        return obs

    def init_belief(self):
        """Initialize belief state with latent support."""
        return {
            "inventory": defaultdict(int),
            "recipes": set(),
            "crafting_recipes": [],
            "goal_item": None,
            "last_action": None,
            "last_obs": None,
            "last_action_outcome": None,
            "latent_variables": {
                "obtainable_items": set(),
                "known_unavailable_items": set(),
                "recipe_graph": {}
            }
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Update belief state based on observation."""
        obs = self.parse_observation(obs_text)
        belief = belief_prior.copy()
        
        # Update inventory
        if obs["inventory"]:
            belief["inventory"] = defaultdict(int, obs["inventory"])
        
        # Add new recipes
        for recipe in obs["crafting_recipes"]:
            belief["recipes"].add(recipe)
            belief["crafting_recipes"].append(recipe)
            
        # Update goal
        if obs["goal_item"]:
            belief["goal_item"] = obs["goal_item"]
            
        # Update outcome
        if obs["last_action_outcome"]:
            belief["last_action_outcome"] = obs["last_action_outcome"]
            
        # Update latent variables based on message
        if obs["message"]:
            if "Got " in obs["message"]:
                item_match = re.search(r'Got (\d+) ([a-z_ ]+)', obs["message"])
                if item_match:
                    _, item = item_match.groups()
                    belief["latent_variables"]["obtainable_items"].add(item)
            elif "Could not find" in obs["message"] and "enough items" not in obs["message"]:
                item_match = re.search(r'Could not find ([a-z_ ]+)', obs["message"])
                if item_match:
                    item = item_match.group(1)
                    belief["latent_variables"]["known_unavailable_items"].add(item)
        
        belief["last_obs"] = obs
        return belief

    def predict_belief(self, belief, action: str):
        """Predict next belief state given action."""
        next_belief = {
            "inventory": belief["inventory"].copy(),
            "recipes": belief["recipes"].copy(),
            "crafting_recipes": belief["crafting_recipes"].copy(),
            "goal_item": belief["goal_item"],
            "last_action": action,
            "last_obs": belief["last_obs"],
            "last_action_outcome": None,
            "latent_variables": {
                "obtainable_items": belief["latent_variables"]["obtainable_items"].copy(),
                "known_unavailable_items": belief["latent_variables"]["known_unavailable_items"].copy(),
                "recipe_graph": belief["latent_variables"]["recipe_graph"].copy()
            }
        }
        
        action = action.strip()
        
        # Handle crafting actions
        craft_match = re.match(r'craft (\d+) ([^(]+) using (.+)', action)
        if craft_match:
            quantity, item, ingredients_str = craft_match.groups()
            quantity = int(quantity)
            item = item.strip()
            
            # Parse ingredients
            ingredients = {}
            parts = ingredients_str.split(', ')
            for part in parts:
                ing_match = re.search(r'(\d+) (.+)', part.strip())
                if ing_match:
                    ing_qty, ing_name = ing_match.groups()
                    ingredients[ing_name] = int(ing_qty)
            
            # Check if we have enough ingredients
            can_craft = True
            for ing_name, ing_qty in ingredients.items():
                if next_belief["inventory"][ing_name] < ing_qty:
                    can_craft = False
                    break
            
            # If we can craft, update inventory
            if can_craft:
                # Deduct ingredients
                for ing_name, ing_qty in ingredients.items():
                    next_belief["inventory"][ing_name] -= ing_qty
                
                # Add crafted item
                next_belief["inventory"][item] += quantity
                next_belief["last_action_outcome"] = "crafted"
                next_belief["latent_variables"]["obtainable_items"].add(item)
            else:
                next_belief["last_action_outcome"] = "not_enough_items"
                
        # Handle get actions
        get_match = re.match(r'get (\d+) (.+)', action)
        if get_match:
            quantity, item = get_match.groups()
            quantity = int(quantity)
            # Check if item is known to be unavailable
            if item in next_belief["latent_variables"]["known_unavailable_items"]:
                next_belief["last_action_outcome"] = "not_found"
            else:
                next_belief["inventory"][item] += quantity
                next_belief["last_action_outcome"] = "got"
                next_belief["latent_variables"]["obtainable_items"].add(item)
            
        # Handle inventory actions
        if action == "inventory":
            next_belief["last_action_outcome"] = "inventory"
            
        return next_belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Generate observation text from belief state."""
        if belief["last_action_outcome"] == "got":
            get_match = re.match(r'get (\d+) (.+)', belief["last_action"])
            if get_match:
                quantity, item = get_match.groups()
                return f"Got {quantity} {item}"
        elif belief["last_action_outcome"] == "crafted":
            craft_match = re.match(r'craft (\d+) ([^(]+) using (.+)', belief["last_action"])
            if craft_match:
                quantity, item, _ = craft_match.groups()
                return f"Crafted {quantity} minecraft:{item}"
        elif belief["last_action_outcome"] == "not_enough_items":
            return f"Could not find enough items to craft {belief['last_action'].split('craft ')[1].split(' using')[0]}"
        elif belief["last_action_outcome"] == "not_found":
            get_match = re.match(r'get (\d+) (.+)', belief["last_action"])
            if get_match:
                _, item = get_match.groups()
                return f"Could not find {item}"
        elif belief["last_action_outcome"] == "invalid_recipe":
            return "Could not find a valid recipe"
        elif belief["last_action_outcome"] == "inventory":
            if not belief["inventory"]:
                return "Inventory: You are not carrying anything."
            else:
                items = []
                for item, count in belief["inventory"].items():
                    if count > 0:
                        items.append(f"[{item}] ({count})")
                return "Inventory: " + " ".join(items)
        
        # Default fallback
        if not belief["inventory"]:
            return "Inventory: You are not carrying anything."
        else:
            items = []
            for item, count in belief["inventory"].items():
                if count > 0:
                    items.append(f"[{item}] ({count})")
            return "Inventory: " + " ".join(items)

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Return valid action templates."""
        return {
            "craft": ["craft <quantity> <item> using <ingredients>"],
            "get": ["get <quantity> <item>"],
            "inventory": ["inventory"],
            "i": ["I <statement>"],
            "im": ["I'm <statement>"]
        }
from abductworld.worldmodel_base import BaseWorldModel
import re

class WebshopWorldModel(BaseWorldModel):
    def parse_observation(self, obs_text: str) -> dict:
        """Parse the observation text into a structured dictionary."""
        parts = [part.strip() for part in obs_text.split(" [SEP] ")]
        obs_dict = {
            "type": "",
            "instruction": "",
            "navigation": [],
            "items": [],
            "product_info": {},
            "page_info": {}
        }
        
        i = 0
        if not parts:
            return obs_dict
            
        # Check if this is a results page or product page by looking for instruction first
        if parts[0] == "Instruction:" or (len(parts) > 1 and parts[1] == "Instruction:"):
            obs_dict["type"] = "instruction"
            # Find instruction
            instr_idx = parts.index("Instruction:") + 1 if "Instruction:" in parts else 1
            if instr_idx < len(parts):
                obs_dict["instruction"] = parts[instr_idx]
                i = instr_idx + 1
            
            page_number = 1
            while i < len(parts):
                if parts[i] == "Back to Search":
                    obs_dict["navigation"].append("Back to Search")
                    i += 1
                elif parts[i] == "< Prev":
                    obs_dict["navigation"].append("< Prev")
                    i += 1
                elif parts[i] == "Next >":
                    obs_dict["navigation"].append("Next >")
                    i += 1
                elif parts[i].startswith("Page "):
                    obs_dict["navigation"].append(parts[i])
                    # Extract page number
                    page_match = re.search(r"Page (\d+)", parts[i])
                    if page_match:
                        page_number = int(page_match.group(1))
                    i += 1
                elif parts[i] == "Search":
                    obs_dict["navigation"].append("Search")
                    i += 1
                elif re.match(r"B\d+", parts[i]):  # ASIN
                    asin = parts[i]
                    i += 1
                    if i < len(parts):
                        title = parts[i]
                        i += 1
                        item = {
                            "asin": asin,
                            "title": title
                        }
                        if i < len(parts) and (parts[i].startswith("$") or "to $" in parts[i]):
                            item["price"] = parts[i]
                            i += 1
                        obs_dict["items"].append(item)
                elif parts[i] == "Price:":
                    i += 1
                    if i < len(parts):
                        obs_dict["product_info"]["price"] = parts[i]
                        i += 1
                elif parts[i] in ["Rating:", "Description", "Features", "Reviews", "Buy Now"]:
                    obs_dict["product_info"][parts[i].lower().replace(" ", "_").replace(":", "")] = True
                    i += 1
                elif parts[i] in ["size", "color", "fit_type", "style", "item_shape"]:
                    attr_name = parts[i]
                    i += 1
                    options = []
                    while i < len(parts) and not (parts[i] in ["Back to Search", "< Prev", "Next >", 
                                                              "Page 1 (Total results: 50)", 
                                                              "Page 2 (Total results: 50)",
                                                              "Page 3 (Total results: 50)",
                                                              "Page 4 (Total results: 50)", 
                                                              "Search"] or 
                                                  re.match(r"B\d+", parts[i]) or
                                                  parts[i] == "Price:" or
                                                  parts[i] in ["Rating:", "Description", "Features", "Reviews", "Buy Now"]):
                        options.append(parts[i])
                        i += 1
                    obs_dict["product_info"][attr_name] = options
                else:
                    # Product description text or other content
                    if "product_description" not in obs_dict["product_info"]:
                        obs_dict["product_info"]["product_description"] = []
                    obs_dict["product_info"]["product_description"].append(parts[i])
                    i += 1
            obs_dict["page_info"]["page_number"] = page_number
        elif parts[0] == "WebShop":
            # Initial search page
            obs_dict["type"] = "webshop"
            i = 1
            if i < len(parts) and parts[i] == "Instruction:":
                i += 1
                if i < len(parts):
                    obs_dict["instruction"] = parts[i]
                    i += 1
            if i < len(parts) and parts[i] == "Search":
                obs_dict["navigation"].append("Search")
        elif parts[0] == "Thank you for shopping with us!":
            obs_dict["type"] = "purchase_complete"
            obs_dict["message"] = parts[0]
        
        return obs_dict

    def init_belief(self):
        """Initialize the belief state with latent support."""
        return {
            "page_type": "initial",
            "search_query": "",
            "instruction": "",
            "items": [],
            "selected_item": None,
            "product_attributes": {},
            "navigation_stack": [],
            # Adding latent belief fields as required by the environment
            "latent_variables": {
                "current_page": "initial",
                "selected_filters": {},
                "product_details": {},
                "search_history": [],
                "purchase_state": "not_started"
            },
            "facts": set(),
            "hypotheses": {},
            "frontier": [],
            "hidden_state": {}
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Correct the belief state based on the observation."""
        obs = self.parse_observation(obs_text)
        belief = belief_prior.copy()
        
        # Preserve latent belief structure
        if "latent_variables" not in belief:
            belief["latent_variables"] = {
                "current_page": "initial",
                "selected_filters": {},
                "product_details": {},
                "search_history": [],
                "purchase_state": "not_started"
            }
        if "facts" not in belief:
            belief["facts"] = set()
        if "hypotheses" not in belief:
            belief["hypotheses"] = {}
        if "frontier" not in belief:
            belief["frontier"] = []
        if "hidden_state" not in belief:
            belief["hidden_state"] = {}
        
        if obs["type"] == "webshop":
            belief["page_type"] = "initial"
            belief["instruction"] = obs["instruction"]
            belief["latent_variables"]["current_page"] = "initial"
            belief["items"] = []
            belief["selected_item"] = None
            belief["product_attributes"] = {}
        elif obs["type"] == "instruction":
            belief["instruction"] = obs["instruction"]
            if "Back to Search" in obs["navigation"] and len(obs["navigation"]) == 1 and not obs["items"] and not obs["product_info"]:
                # This is back to search from product page - should show results
                belief["page_type"] = "results"
                belief["latent_variables"]["current_page"] = "results"
                belief["items"] = []
            elif "Back to Search" in obs["navigation"] and obs["items"]:
                # This is a results page with Back to Search
                belief["page_type"] = "results"
                belief["items"] = obs["items"]
                belief["latent_variables"]["current_page"] = "results"
            elif any("Page" in nav for nav in obs["navigation"]):
                # This is a results page
                belief["page_type"] = "results"
                belief["items"] = obs["items"]
                belief["latent_variables"]["current_page"] = "results"
            elif "Search" in obs["navigation"] and not obs["items"] and not obs["product_info"]:
                # This is the initial page but reached via back navigation
                belief["page_type"] = "initial"
                belief["latent_variables"]["current_page"] = "initial"
                belief["items"] = []
                belief["selected_item"] = None
                belief["product_attributes"] = {}
            elif obs["product_info"] and not obs["items"]:
                # This is a product page
                belief["page_type"] = "product"
                belief["latent_variables"]["current_page"] = "product"
                belief["product_attributes"] = obs["product_info"]
                belief["latent_variables"]["product_details"] = obs["product_info"]
            else:
                # Default to results page when in doubt
                belief["page_type"] = "results"
                belief["items"] = obs["items"]
                belief["latent_variables"]["current_page"] = "results"
        elif obs["type"] == "purchase_complete":
            belief["page_type"] = "purchase_complete"
            belief["latent_variables"]["current_page"] = "purchase_complete"
            belief["latent_variables"]["purchase_state"] = "completed"
            
        return belief

    def predict_belief(self, belief, action: str):
        """Predict the next belief state given an action."""
        next_belief = belief.copy()
        
        # Ensure latent belief structure is maintained
        if "latent_variables" not in next_belief:
            next_belief["latent_variables"] = {
                "current_page": "initial",
                "selected_filters": {},
                "product_details": {},
                "search_history": [],
                "purchase_state": "not_started"
            }
        if "facts" not in next_belief:
            next_belief["facts"] = set()
        if "hypotheses" not in next_belief:
            next_belief["hypotheses"] = {}
        if "frontier" not in next_belief:
            next_belief["frontier"] = []
        if "hidden_state" not in next_belief:
            next_belief["hidden_state"] = {}
        
        if action.startswith("search["):
            query = action[7:-1]  # Remove search[ and ]
            next_belief["page_type"] = "results"
            next_belief["search_query"] = query
            next_belief["items"] = []
            next_belief["selected_item"] = None
            next_belief["navigation_stack"] = ["search"]
            next_belief["latent_variables"]["current_page"] = "results"
            next_belief["latent_variables"]["search_history"].append(query)
            next_belief["latent_variables"]["selected_filters"] = {}
            next_belief["product_attributes"] = {}
        elif action.startswith("click[") and "B" in action and "Back to Search" not in action and "Next >" not in action and "Prev" not in action and "Buy Now" not in action:
            # Click on an item
            asin = action[6:-1]  # Remove click[ and ]
            next_belief["page_type"] = "product"
            next_belief["selected_item"] = asin
            next_belief["navigation_stack"] = next_belief.get("navigation_stack", []) + ["item_click"]
            next_belief["latent_variables"]["current_page"] = "product"
            if "latent_variables" not in next_belief:
                next_belief["latent_variables"] = {}
            if "product_details" not in next_belief["latent_variables"]:
                next_belief["latent_variables"]["product_details"] = {}
            next_belief["latent_variables"]["product_details"]["asin"] = asin
        elif action == "click[Back to Search]" or action == "Back to Search":
            next_belief["page_type"] = "results"  # Show results page, not initial page
            next_belief["navigation_stack"] = []
            next_belief["latent_variables"]["current_page"] = "results"
            next_belief["items"] = []
            next_belief["selected_item"] = None
            next_belief["product_attributes"] = {}
        elif action == "click[< Prev]":
            if next_belief.get("navigation_stack"):
                prev_action = next_belief["navigation_stack"].pop()
                if prev_action == "item_click":
                    next_belief["page_type"] = "results"
                    next_belief["selected_item"] = None
                    next_belief["latent_variables"]["current_page"] = "results"
                elif prev_action == "next_page":
                    next_belief["page_type"] = "results"
                    next_belief["latent_variables"]["current_page"] = "results"
        elif action == "click[Next >]":
            next_belief["page_type"] = "results"
            next_belief["navigation_stack"] = next_belief.get("navigation_stack", []) + ["next_page"]
            next_belief["latent_variables"]["current_page"] = "results"
        elif action == "click[Buy Now]":
            next_belief["page_type"] = "purchase_complete"
            next_belief["navigation_stack"] = next_belief.get("navigation_stack", []) + ["buy_now"]
            next_belief["latent_variables"]["current_page"] = "purchase_complete"
            next_belief["latent_variables"]["purchase_state"] = "completed"
        elif action.startswith("click[") and any(opt in action.lower() for opt in ["size", "color", "fit_type", "style", "item_shape"]):
            # Clicking on an attribute option
            option = action[6:-1]  # Remove click[ and ]
            next_belief["navigation_stack"] = next_belief.get("navigation_stack", []) + [f"option_{option}"]
            # Update latent variables with selected filters
            if "color" in action.lower():
                if "selected_filters" not in next_belief["latent_variables"]:
                    next_belief["latent_variables"]["selected_filters"] = {}
                next_belief["latent_variables"]["selected_filters"]["color"] = option
            elif "size" in action.lower():
                if "selected_filters" not in next_belief["latent_variables"]:
                    next_belief["latent_variables"]["selected_filters"] = {}
                next_belief["latent_variables"]["selected_filters"]["size"] = option
            elif "style" in action.lower():
                if "selected_filters" not in next_belief["latent_variables"]:
                    next_belief["latent_variables"]["selected_filters"] = {}
                next_belief["latent_variables"]["selected_filters"]["style"] = option
            
        return next_belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Generate an observation string from the belief state."""
        # Ensure belief has latent support
        if "latent_variables" not in belief:
            belief["latent_variables"] = {
                "current_page": belief.get("page_type", "initial"),
                "selected_filters": {},
                "product_details": {},
                "search_history": [],
                "purchase_state": "not_started"
            }
            
        if belief["page_type"] == "initial":
            return f"WebShop [SEP] Instruction: [SEP] {belief['instruction']} [SEP] Search"
        elif belief["page_type"] == "results":
            obs_parts = [f"Instruction: [SEP] {belief['instruction']} [SEP] Back to Search"]
            if belief["items"]:
                # Add page navigation
                page_nav = ["Page 1 (Total results: 50)"]
                if "next_page" in belief.get("navigation_stack", []):
                    page_nav.append("Next >")
                elif len(belief["navigation_stack"]) > 0 and belief["navigation_stack"][-1] == "next_page":
                    page_nav = ["Page 2 (Total results: 50)", "< Prev", "Next >"]
                elif action == "click[Next >]":
                    page_nav = ["Page 2 (Total results: 50)", "< Prev", "Next >"]
                elif action == "click[< Prev]":
                    page_nav = ["Page 1 (Total results: 50)", "Next >"]
                obs_parts.extend(page_nav)
                
                # Add items (limit to first 10 to match typical webshop behavior)
                for item in belief["items"][:10]:
                    if "price" in item:
                        obs_parts.append(f"{item['asin']} [SEP] {item['title']} [SEP] {item['price']}")
                    else:
                        obs_parts.append(f"{item['asin']} [SEP] {item['title']}")
            else:
                obs_parts.append("Search")
            return " [SEP] ".join(obs_parts)
        elif belief["page_type"] == "product":
            obs_parts = [f"Instruction: [SEP] {belief['instruction']} [SEP] Back to Search"]
            
            # Add navigation based on how we got here
            if belief.get("navigation_stack") and "item_click" in belief["navigation_stack"]:
                obs_parts.append("< Prev")
            
            # Add product attributes if available
            product_attrs = belief.get("product_attributes", {})
            for attr, values in product_attrs.items():
                if attr in ["size", "color", "fit_type", "style", "item_shape"]:
                    obs_parts.append(attr)
                    if isinstance(values, list):
                        obs_parts.extend(values)
            
            # Add product info
            if "price" in product_attrs:
                obs_parts.append(f"Price: {product_attrs['price']}")
            
            # Add standard product page elements
            obs_parts.extend(["Rating: N.A.", "Description", "Features", "Reviews", "Buy Now"])
            
            return " [SEP] ".join(obs_parts)
        elif belief["page_type"] == "purchase_complete":
            return "Thank you for shopping with us! [SEP] Your code: [SEP] None [SEP] (Paste it in your MTurk interface.) [SEP] Purchased [SEP] asin [SEP] B09P8D2Q1Q [SEP] options [SEP] {} [SEP] attrs [SEP] None [SEP] category [SEP] None [SEP] query [SEP] None [SEP] product category [SEP] None [SEP] Target [SEP] asin [SEP] options [SEP] attrs [SEP] price upper [SEP] instuction text [SEP] category [SEP] product category [SEP] query [SEP] Goal [SEP] None [SEP] Reward [SEP] Your score (min 0.0, max 1.0) [SEP] 1.0 [SEP] Reward Details [SEP] None"
        else:
            # Fallback for unknown page types - try to reconstruct from available info
            if belief.get("instruction"):
                return f"Instruction: [SEP] {belief['instruction']} [SEP] Back to Search [SEP] Search"
            else:
                return "WebShop [SEP] Instruction: [SEP]  [SEP] Search"

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Extract valid action forms for this environment."""
        return {
            "search": ["search[<query>]"],
            "click": [
                "click[<asin>]",
                "click[Back to Search]",
                "click[< Prev]",
                "click[Next >]",
                "click[Buy Now]",
                "click[Description]",
                "click[Features]",
                "click[Reviews]",
                "click[<option>]",  # For size/color options
                "click[Search]"
            ]
        }
from abductworld.worldmodel_base import BaseWorldModel
from collections import Counter
import re

class WordleWorldModel(BaseWorldModel):
    def __init__(self):
        # We'll use a larger set of valid 5-letter words
        self.valid_words = {
            "solar", "proxy", "ultra", "prick", "shire", 
            "could", "gifts", "spint", "frown", "throb",
            "moral", "world", "about", "other", "which",
            "their", "there", "would", "these", "first",
            "never", "after", "where", "great", "place",
            "every", "house", "night", "point", "water",
            "money", "story", "young", "month", "south",
            "party", "today", "right", "child", "until",
            "level", "times", "often", "always", "power",
            "since", "given", "taken", "known", "woman",
            "least", "light", "voice", "whole", "thing",
            "major", "third", "white", "heart", "later",
            "force", "among", "early", "study", "human",
            "black", "death", "sense", "value", "carry",
            "table", "green", "cause", "short", "field",
            "paper", "space", "under", "total", "event",
            "order", "round", "means", "works", "front",
            "blood", "quite", "class", "bring", "small",
            "large", "sound", "write", "offer", "ready",
            "press", "music", "clear", "moved", "words",
            "frame", "trove", "shore", "spare",
            "cynic", "panic", "whack", "tacit", "those", 
            "shake"
        }
        self.feedback_pattern = re.compile(r'^[byg]( [byg]){4}$')
        self.word_pattern = re.compile(r'^[a-z]( [a-z]){4}$')

    def parse_observation(self, obs_text: str) -> dict:
        """Parse observation text into structured data."""
        obs_text = obs_text.strip()
        if obs_text == "invalid word":
            return {"type": "invalid", "webshop_page_type": "invalid", "webshop_goal_completed": False}
        elif self.feedback_pattern.match(obs_text):
            return {"type": "feedback", "value": obs_text.split(), "webshop_page_type": "feedback", "webshop_goal_completed": False}
        elif obs_text.startswith("Welcome to the game of Wordle"):
            return {"type": "welcome", "webshop_page_type": "welcome", "webshop_goal_completed": False}
        else:
            return {"type": "unknown", "webshop_page_type": "unknown", "webshop_goal_completed": False}

    def init_belief(self):
        """Initialize belief state: all possible target words."""
        # In a real implementation, we would randomly select a target word
        # For now, we'll use a fixed word for consistency in testing
        return {
            "possible_words": self.valid_words.copy(),
            "guess_history": [],
            "feedback_history": [],
            "target_word": "frame"  # Fixed target for consistent testing
        }

    def correct_belief(self, belief_prior, obs_text: str):
        """Update belief based on observation."""
        parsed = self.parse_observation(obs_text)
        belief = belief_prior.copy()
        
        if parsed["type"] == "feedback":
            feedback = parsed["value"]
            last_guess = belief["guess_history"][-1] if belief["guess_history"] else None
            
            if last_guess:
                # Filter possible words based on feedback
                belief["possible_words"] = self._filter_words(
                    belief["possible_words"], 
                    last_guess.replace(" ", ""), 
                    feedback
                )
                belief["feedback_history"].append(feedback)
                
        return belief

    def predict_belief(self, belief, action: str):
        """Predict next belief state given an action."""
        belief = belief.copy()
        action = action.strip().lower()
        
        # Check if action is a valid 5-letter word
        if " " in action and len(action.split()) == 5 and all(c.isalpha() for c in action.split()):
            word = action.replace(" ", "")
            # Accept any 5-letter word for prediction, not just those in valid_words
            belief["guess_history"].append(action)
        elif action.isalpha() and len(action) == 5:
            # Accept any 5-letter word for prediction, not just those in valid_words
            belief["guess_history"].append(" ".join(list(action)))
        
        return belief

    def readout_observation(self, belief, action: str = "") -> str:
        """Generate observation based on belief and action."""
        action = action.strip().lower()
        
        # Handle invalid word format
        if " " in action and len(action.split()) == 5:
            word = action.replace(" ", "")
        elif action.isalpha() and len(action) == 5:
            word = action
        else:
            return "invalid word"
            
        # Generate feedback based on the target word - don't reject valid 5-letter words
        if "target_word" in belief:
            target = belief["target_word"]
            feedback = self._generate_feedback(word, target)
            return " ".join(feedback)
        else:
            # Fallback if no target word is set - generate reasonable feedback
            # For replay purposes, we need to generate consistent feedback
            # Let's use a simple pattern based on the word
            feedback = ['b'] * 5
            for i, char in enumerate(word[:5]):
                if i < len(word) and i < 5:
                    # Simple heuristic: make some positions green/yellow for variety
                    if ord(char) % 3 == 0:
                        feedback[i] = 'g'
                    elif ord(char) % 3 == 1:
                        feedback[i] = 'y'
            return " ".join(feedback)

    def extract_valid_action_forms(self) -> dict[str, list[str]]:
        """Define valid action formats."""
        return {
            "<5-letter-guess>": [
                "a b c d e",
                "f g h i j",
                "k l m n o",
                "p q r s t",
                "u v w x y",
                "z a b c d"
            ],
            "WORDLE_INVALID": [
                "invalid word"
            ],
            "WORDLE_FEEDBACK": [
                "b b b b b",
                "g g g g g",
                "y y y y y",
                "b g y b g",
                "g b y g b"
            ]
        }
    
    def _filter_words(self, word_list, guess, feedback):
        """Filter possible words based on guess and feedback."""
        filtered = set()
        guess_chars = list(guess)
        feedback_chars = feedback
        
        # Count characters in guess for handling duplicates
        guess_counts = Counter(guess_chars)
        
        for word in word_list:
            word_chars = list(word)
            # Create a copy of guess counts to track used letters
            available_chars = guess_counts.copy()
            valid = True
            
            # First pass: process 'g' (green) - correct position
            for i in range(5):
                if feedback_chars[i] == 'g':
                    if word_chars[i] != guess_chars[i]:
                        valid = False
                        break
                    available_chars[guess_chars[i]] -= 1
                    
            if not valid:
                continue
                
            # Second pass: process 'y' (yellow) and 'b' (black)
            for i in range(5):
                if feedback_chars[i] == 'g':
                    continue  # Already handled
                elif feedback_chars[i] == 'y':
                    # Letter is in word but not in this position
                    if word_chars[i] == guess_chars[i] or guess_chars[i] not in word_chars:
                        valid = False
                        break
                    if available_chars[guess_chars[i]] <= 0:
                        valid = False
                        break
                    available_chars[guess_chars[i]] -= 1
                elif feedback_chars[i] == 'b':
                    # Letter is not in word, or has been accounted for
                    if guess_chars[i] in word_chars and available_chars[guess_chars[i]] > 0:
                        valid = False
                        break
                        
            if valid:
                filtered.add(word)
                
        return filtered
    
    def _generate_feedback(self, guess: str, target: str) -> list[str]:
        """Generate feedback for a guess compared to target word."""
        feedback = ['b'] * 5  # Default to black
        target_chars = list(target)
        guess_chars = list(guess)
        
        # Track character counts for handling duplicates
        target_counts = Counter(target_chars)
        used_chars = {char: 0 for char in target_counts}
        
        # First pass: mark greens (correct position)
        for i in range(5):
            if guess_chars[i] == target_chars[i]:
                feedback[i] = 'g'
                used_chars[guess_chars[i]] += 1
        
        # Second pass: mark yellows (correct letter, wrong position)
        for i in range(5):
            if feedback[i] != 'g':  # Not already marked green
                char = guess_chars[i]
                if char in target_counts and used_chars[char] < target_counts[char]:
                    feedback[i] = 'y'
                    used_chars[char] += 1
        
        return feedback

References↩︎

[1]
P. Langley, Scientific discovery: Computational explorations of the creative processes. MIT press, 1987.
[2]
M. Schmidt and H. Lipson, “Distilling free-form natural laws from experimental data,” science, vol. 324, no. 5923, pp. 81–85, 2009.
[3]
T. Zheng et al., “NewtonBench: Benchmarking generalizable scientific law discovery in LLM agents,” in The fourteenth international conference on learning representations, 2026, [Online]. Available: https://openreview.net/forum?id=Gk6umqW74m.
[4]
D. Ha and J. Schmidhuber, “World models,” CoRR, vol. abs/1803.10122, 2018, [Online]. Available: http://arxiv.org/abs/1803.10122.
[5]
D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” in International Conference on Learning Representations, 2019.
[6]
N. R. Ke et al., “Modeling the long term future in model-based reinforcement learning,” in International conference on learning representations, 2019.
[7]
Y. LeCun and Courant, “A path towards autonomous machine intelligence version 0.9.2, 2022-06-27,” 2022.
[8]
A. Brohan et al., “RT-2: Vision-language-action models transfer web knowledge to robotic control,” in arXiv (Cornell University), 2023.
[9]
K. Black et al., “\(\pi_0\): A vision-language-action flow model for general robot control,” Robotics: Science and Systems XXI, 2025.
[10]
J. Bruce et al., “Genie: Generative interactive environments,” in arXiv (Cornell University), 2024.
[11]
F. C. team et al., “CWM: An open-weights LLM for research on code generation with world models,” CoRR, vol. abs/2510.02387, 2025, doi: 10.48550/ARXIV.2510.02387.
[12]
D. McDermott et al., “PDDL-the planning domain definition language,” 1998, [Online]. Available: https://api.semanticscholar.org/CorpusID:59656859.
[13]
H. Tang, D. Key, and K. Ellis, “WorldCoder, a model-based LLM agent: Building world models by writing code and interacting with the environment,” in Advances in neural information processing systems 38: Annual conference on neural information processing systems 2024, NeurIPS 2024, vancouver, BC, canada, december 10 - 15, 2024, 2024, [Online]. Available: http://papers.nips.cc/paper\_files/paper/2024/hash/820c61a0cd419163ccbd2c33b268816e-Abstract-Conference.html.
[14]
L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” Artificial intelligence, vol. 101, no. 1–2, pp. 99–134, 1998.
[15]
M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents (extended abstract),” in Proceedings of the twenty-fourth international joint conference on artificial intelligence, IJCAI 2015, buenos aires, argentina, july 25-31, 2015, 2015, pp. 4148–4152, [Online]. Available: http://ijcai.org/Abstract/15/585.
[16]
M.-A. Côté et al., “TextWorld: A learning environment for text-based games,” in Computer games - 7th workshop, CGW 2018, held in conjunction with the 27th international conference on artificial intelligence, IJCAI 2018, stockholm, sweden, july 13, 2018, revised selected papers, 2018, vol. 1017, pp. 41–75, doi: 10.1007/978-3-030-24337-1\_3.
[17]
M. Shridhar, X. Yuan, M.-A. Côté, Y. Bisk, A. Trischler, and M. J. Hausknecht, “ALFWorld: Aligning text and embodied environments for interactive learning,” in 9th international conference on learning representations, ICLR 2021, virtual event, austria, may 3-7, 2021, 2021, [Online]. Available: https://openreview.net/forum?id=0IOX0YcCdTn.
[18]
R. Wang, P. Jansen, M.-A. Côté, and P. Ammanabrolu, “ScienceWorld: Is your agent smarter than a 5th grader?” in Proceedings of the 2022 conference on empirical methods in natural language processing, Dec. 2022, pp. 11279–11298, doi: 10.18653/v1/2022.emnlp-main.775.
[19]
S. Yao, H. Chen, J. Yang, and K. R. Narasimhan, “WebShop: Towards scalable real-world web interaction with grounded language agents,” in Advances in neural information processing systems, 2022, [Online]. Available: https://openreview.net/forum?id=R9KnuFlvnU.
[20]
Z. Xi et al., “AgentGym: Evaluating and training large language model-based agents across diverse environments,” in Proceedings of the 63rd annual meeting of the association for computational linguistics (volume 1: Long papers), Jul. 2025, pp. 27914–27961, doi: 10.18653/v1/2025.acl-long.1355.
[21]
T. Xu et al., “CRAB: Cross-environment agent benchmark for multimodal language model agents,” in Findings of the association for computational linguistics: ACL 2025, Jul. 2025, pp. 21607–21647, doi: 10.18653/v1/2025.findings-acl.1113.
[22]
Y. Li et al., “From word to world: Can large language models be implicit text-based world models?” 2025, [Online]. Available: https://arxiv.org/abs/2512.18832.
[23]
T. Fang et al., “WebEvolver: Enhancing web agent self-improvement with co-evolving world model,” in Proceedings of the 2025 conference on empirical methods in natural language processing, Nov. 2025, pp. 8959–8975, doi: 10.18653/v1/2025.emnlp-main.454.
[24]
S. Qiao et al., “Agent planning with world knowledge model,” in Advances in neural information processing systems, 2024, [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/hash/d032263772946dd5026e7f3cd22bce5b-Abstract-Conference.html.
[25]
A. Solar-Lezama, L. Tancau, R. Bodik, S. Seshia, and V. Saraswat, “Combinatorial sketching for finite programs,” in Proceedings of the 12th international conference on architectural support for programming languages and operating systems, 2006, pp. 404–415, doi: 10.1145/1168857.1168907.
[26]
N. Dainese, M. Merler, M. Alakuijala, and P. Marttinen, “Generating code world models with large language models guided by monte carlo tree search,” in The thirty-eighth annual conference on neural information processing systems, 2024, [Online]. Available: https://openreview.net/forum?id=9SpWvX9ykp.
[27]
W. T. Piriyakulkij, Y. Liang, H. Tang, A. Weller, M. Kryven, and K. Ellis, “PoE-world: Compositional world modeling with products of programmatic experts,” in The thirty-ninth annual conference on neural information processing systems, 2025, [Online]. Available: https://openreview.net/forum?id=obwRcksFZw.
[28]
Z. Khan, A. Prasad, E. Stengel-Eskin, J. Cho, and M. Bansal, “One life to learn: Inferring symbolic world models for stochastic environments from unguided exploration,” in The fourteenth international conference on learning representations, 2026, [Online]. Available: https://openreview.net/forum?id=UQ36IrVCw2.
[29]
M. Tantakoun, C. Muise, and X. Zhu, “LLMs as planning formalizers: A survey for leveraging large language models to construct automated planning models,” in Findings of the association for computational linguistics: ACL 2025, Jul. 2025, pp. 25167–25188, doi: 10.18653/v1/2025.findings-acl.1291.
[30]
M. Hu et al., “Text2World: Benchmarking large language models for symbolic world model generation,” in Findings of the association for computational linguistics: ACL 2025, Jul. 2025, pp. 26043–26066, doi: 10.18653/v1/2025.findings-acl.1337.
[31]
X. Quan, M. Valentino, D. Carvalho, D. Dalal, and A. Freitas, “PEIRCE: Unifying material and formal reasoning via LLM-driven neuro-symbolic refinement,” in Proceedings of the 63rd annual meeting of the association for computational linguistics (volume 3: System demonstrations), Jul. 2025, pp. 11–21, doi: 10.18653/v1/2025.acl-demo.2.
[32]
Y. Su et al., “Neuro-symbolic verification on instruction following of LLMs.” 2026, [Online]. Available: https://arxiv.org/abs/2601.17789.
[33]
S. Zhou et al., “WALL-E: World alignment by rule learning improves world model-based LLM agents,” CoRR, vol. abs/2410.07484, 2024, doi: 10.48550/ARXIV.2410.07484.
[34]
S. Zhou et al., “WALL-E 2.0: World alignment by NeuroSymbolic learning improves world model-based LLM agents,” CoRR, vol. abs/2504.15785, 2025, doi: 10.48550/ARXIV.2504.15785.
[35]
M. Kim and S. Hwang, “CoEx – co-evolving world-model and exploration,” in Findings of the association for computational linguistics: EMNLP 2025, Nov. 2025, pp. 21629–21651, doi: 10.18653/v1/2025.findings-emnlp.1179.
[36]
S. Gulwani, O. Polozov, and R. Singh, “Program synthesis,” Found. Trends Program. Lang., vol. 4, no. 1–2, pp. 1–119, Jul. 2017, doi: 10.1561/2500000010.
[37]
K. Ellis et al., “Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning,” in Proceedings of the 42nd acm sigplan international conference on programming language design and implementation, 2021, pp. 835–850.
[38]
R. Wang, E. Zelikman, G. Poesia, Y. Pu, N. Haber, and N. Goodman, “Hypothesis search: Inductive reasoning with language models,” in International conference on learning representations, 2024, vol. 2024, pp. 38993–39014.
[39]
A. Wei et al., “CodeARC: Benchmarking reasoning capabilities of LLM agents for inductive program synthesis.” 2025, [Online]. Available: https://arxiv.org/abs/2503.23145.
[40]
J. Bai, X. Liu, W. Wang, C. Luo, and Y. Song, “Complex query answering on eventuality knowledge graph with implicit logical constraints,” in Advances in neural information processing systems 36: Annual conference on neural information processing systems 2023, NeurIPS 2023, new orleans, LA, USA, december 10 - 16, 2023, 2023, [Online]. Available: http://papers.nips.cc/paper\_files/paper/2023/hash/6174c67b136621f3f2e4a6b1d3286f6b-Abstract-Conference.html.
[41]
J. Bai, Y. Wang, T. Zheng, Y. Guo, X. Liu, and Y. Song, “Advancing abductive reasoning in knowledge graphs through complex logical hypothesis generation,” in Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), ACL 2024, bangkok, thailand, august 11-16, 2024, 2024, pp. 1312–1329, doi: 10.18653/V1/2024.ACL-LONG.72.
[42]
N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” in Advances in neural information processing systems 36: Annual conference on neural information processing systems 2023, NeurIPS 2023, new orleans, LA, USA, december 10 - 16, 2023, 2023, [Online]. Available: http://papers.nips.cc/paper\_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html.
[43]
G. Wang et al., “Voyager: An open-ended embodied agent with large language models,” Trans. Mach. Learn. Res., vol. 2024, 2024, [Online]. Available: https://openreview.net/forum?id=ehfRiF0R3a.
[44]
A. Prasad et al., “ADaPT: As-needed decomposition and planning with language models,” in Findings of the association for computational linguistics: NAACL 2024, Jun. 2024, pp. 4226–4252, doi: 10.18653/v1/2024.findings-naacl.264.
[45]
S. Yuan, Z. Chen, Z. Xi, J. Ye, Z. Du, and J. Chen, “Agent-r: Training language model agents to reflect via iterative self-training.” 2025, [Online]. Available: https://arxiv.org/abs/2501.11425.
[46]
J. Kim et al., “ReflAct: World-grounded decision making in LLM agents via goal-state reflection,” in Proceedings of the 2025 conference on empirical methods in natural language processing, Nov. 2025, pp. 33433–33465, doi: 10.18653/v1/2025.emnlp-main.1697.
[47]
T. T. Quoc, D. H. Minh, T. Q. Thanh, and A. Nguyen-Duc, “An empirical study on self-correcting large language models for data science code generation,” CoRR, vol. abs/2408.15658, 2024, doi: 10.48550/ARXIV.2408.15658.
[48]
R. Kamoi, Y. Zhang, N. Zhang, J. Han, and R. Zhang, “When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 1417–1440, 2024, doi: 10.1162/tacl_a_00713.
[49]
N. Jiang et al., “LeDex: Training LLMs to better self-debug and explain code,” in Advances in neural information processing systems, 2024, [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/hash/3ea832724870c700f0a03c665572e2a9-Abstract-Conference.html.
[50]
Md. A. Islam, M. E. Ali, and M. R. Parvez, “CodeSim: Multi-agent code generation and problem solving through simulation-driven planning and debugging,” in Findings of the association for computational linguistics: NAACL 2025, Apr. 2025, pp. 5128–5154, doi: 10.18653/v1/2025.findings-naacl.285.
[51]
V. Agarwal, Y. Pei, S. Alamir, and X. Liu, “CodeMirage: Hallucinations in code generated by large language models,” CoRR, vol. abs/2408.08333, 2024, doi: 10.48550/ARXIV.2408.08333.
[52]
A. Councilman et al., “Towards formal verification of LLM-generated code from natural language prompts,” CoRR, vol. abs/2507.13290, 2025, doi: 10.48550/ARXIV.2507.13290.
[53]
C. Grimm, A. Barreto, S. Singh, and D. Silver, “The value equivalence principle for model-based reinforcement learning,” Advances in neural information processing systems, vol. 33, pp. 5541–5552, 2020.
[54]
M. Chevalier-Boisvert et al., “BabyAI: First steps towards grounded language learning with a human in the loop,” in International conference on learning representations, 2019, [Online]. Available: https://openreview.net/forum?id=rJeXCo0cYX.
[55]
M. Abdulhai et al., “LMRL gym: Benchmarks for multi-turn reinforcement learning with language models,” in Forty-second international conference on machine learning, 2025, [Online]. Available: https://openreview.net/forum?id=hmGhP5DO2W.
[56]
S. Yao et al., “ReAct: Synergizing reasoning and acting in language models,” in The eleventh international conference on learning representations, 2023, [Online]. Available: https://openreview.net/forum?id=WE_vluYUL-X.
[57]
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating text generation with BERT,” in International conference on learning representations, 2020.
[58]
Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.

  1. https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct.↩︎

  2. https://huggingface.co/Qwen/Qwen3.5-4B.↩︎

  3. https://huggingface.co/XiaomiMiMo/MiMo-V2.5.↩︎

  4. https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash.↩︎

Subjects

Computation and Language (cs.CL)

Artificial Intelligence (cs.AI)

Updated on Academus

2026/06/02 02:06 UTC