1 Introduction↩︎

Robots today execute a rich catalog of low-level skills: quadrupeds walk, trot, climb stairs, and balance, while humanoids add grasping and manipulation on top; many platforms ship turnkey APIs for locomotion, speech, and perception. These skills solve basic motion and action well, but they do not tell a robot what to do when a human asks for something new, the scene changes unexpectedly, or several goals compete for attention. The hard problem has shifted from how a robot body moves to how an embodied system decides, and hard-coded task logic or per-task trained policies do not scale to the open world.

LLMs offer a path to close the decision-making gap: pretrained on large-scale data, they supply the common sense to interpret a situation and compose the robot’s existing skills into a plan [@chen2025typefly; @codeaspolicies2022]. But using an LLM to drive an embodied agent raises three system challenges that persist even as models become faster and more capable.

(i) First, LLM inference is too slow for real-time reaction. There are systems that place a language or vision–language model in the perception–action loop for high-level reasoning to achieve impressive open-world generality [@ahn2022saycan; @huang2023voxposer; @yenamandra2023homerobot; @liu2024ok; @shah2025bumble]. But they operate far below real-time, with long pauses between consecutive actions, so demonstrations are commonly shown at increased playback speed. Small models respond fast but plan poorly, while state-of-the-art models are capable but substantial, adding seconds of latency. This latency–quality tradeoff is fundamental, and neither extreme alone suffices for real-time control. (ii) Second, real-world embodied agents rarely run one task in isolation. It may be patrolling an area while communicating with a nearby human, and still needs to watch for hazards. Arbitrating the robot’s shared actuators among concurrent tasks while honoring their priorities and preempting cleanly are classic OS problems that the LLM literature has largely sidestepped. (iii) Third, one-shot natural-language instructions cover only goals. Embodied agents also need reactive rules (e.g., “back up if anything show abruptly within a near space (\(0.2\)m)”) that must fire far faster than an LLM call. Making natural language a first-class medium for both goals and fast reaction rules, authored by non-programmers without a code editor, remains an open question.

To tackle these challenges, we hypothesize that LLM-based control of an embodied agent should run as a continuously executing, asynchronous runtime* rather than a request/response loop around the LLM.* In effect, it works like a small OS for the agent. We build TypeGo, a prototype that realizes this runtime design. TypeGo runs four concurrent agent loops at different timescales over a Skill Kernel that mediates the body’s actuators. TypeGo is orthogonal to work that accelerates a single LLM call. The design instead aims to hide LLM calls from the critical path through speculation and tiered dispatch, so any per-call speedup composes directly with its architecture.

This paper develops three mechanisms that make the runtime hypothesis concrete:

• Multi-cadence asynchronous planning (§6). TypeGo runs its planning layers as independent loops at distinct levels of abstraction: a fast reflex layer (S0), an action streamer (S1), slower task-decomposer (S2) and scheduler (S3). Specifically, the S1 features speculative skill streaming, which overlaps planning with skill execution and stores the upcoming skill calls in a bounded queue. As a result, the LLM planning latency is hided behind ongoing motion while bounded look-ahead preserves per-step adaptivity with dynamic environment.

• OS-style runtime with semantic scheduling (§5). Each task creates a process and assigns it a process control block (PCB). A Skill Kernel arbitrates the categorized robot’s resources for processes. S3 schedules these processes semantically rather than by fixed numeric priority. When two processes compete for the same resource, S3 decides which wins based on the scene and prompt guidelines. Whether a preempted process then resumes depends on its origin: a user task does not resume by default, whereas an S3-initiated task resumes automatically.

• Natural language as a first-class programming medium (§4). Users author prescriptions, including natural-language tasks that specify goals and reflexes that specify reactive rules. Tasks are supplied online to the S1–S3 planners, while reflexes compile into Python condition/action functions that S0 evaluates at high frequency. Together, they let non-programmers author both deliberate and reactive behavior without writing code.

We report an early implementation of TypeGo(§7) and use a prototype deployment on a Unitree Go2 quadruped to validate the design direction in single- and concurrent-task scenarios (§8). In our current task suite, TypeGo cuts per-step delay by 50% over step-by-step planning and TTFA by 73% over monolithic planning, while adding modest coordination overhead: about 200 ms for S3 scheduling and 50 ms for S0 reflex handling.

2 Related Work↩︎

Hierarchical robot control. TypeGo’s planner hierarchy is inspired by the classical three-layer architecture [@gat1998three]: where a deliberator forms a plan, a sequencer turns it into actions, and a controller runs primitives. TypeGo modernizes this template in three ways. First, it drives the deliberator (S2) and sequencer (S1) with LLMs rather than symbolic planners. Second, it adds two layers the classical template lacks: a reflex layer (S0) for low-latency reaction and a task scheduler (S3) for multi-task arbitration. Third, all layers run asynchronously, so the sequencer emits actions without blocking on the deliberator. Inspired by dual-process theory [@kahneman2011thinking], some recent planners pair a fast “System 1” with a slow “System 2” and route between them [@saha2025system1x; @ziabari2025reasoning; @zhu2024language; @wang2025hierarchicalreasoningmodel]. Such two-tier designs activate only one tier at a time and switch between modes. In contrast, TypeGo runs its four loops concurrently across timescales (S0–S3), rather than as alternatives selected per request.

LLM-based robot planning. LLMs are widely used to turn instructions into robot plans [@zeng2023llm; @jeong2024survey]. Existing systems are either monolithic planners that prompt once for a full plan (Code-as-Policies [@codeaspolicies2022], TypeFly [@chen2025typefly]) or step-by-step planners that query per action (SayCan [@ahn2022saycan], Inner Monologue [@huang2022inner], ReAct [@yao2023react]). A monolithic plan delays the first action and cannot recover when the scene changes; a step-by-step planner reacts but stalls the robot between actions while it rebuilds its prompt. TypeGo explores a middle point: asynchronous LLM loops across levels of abstraction, plus a speculative planner (S1) that aims to hide planning latency at runtime. Orthogonally, end-to-end vision–language–action models map perception directly to low-level control [@brohan2023rt2]; such a policy can sit inside a TypeGo skill, while TypeGo addresses concurrency, preemption, and latency-hiding above the policy level.

Multi-agent LLM systems. Prior multi-agent work arranges agents in chains or graphs: some refine an instruction top-down [@prakash2023llm; @kienle2025lodge], while Mixture-of-Agents [@wang2024mixture] combines outputs from multiple proposer models through an aggregator. In these designs agents run sequentially, causing execution time for a single request to grow with the number of agents. TypeGo instead runs each planner in its own loop, coordinated through synchronization and scheduling rather than sequential dependency, keeping planning latency low.

Operating systems for LLM agents. A recent line of work borrows OS structure for software agents. AIOS [@mei2024aios] adds an “LLM kernel” that schedules and manages agent requests, and MemGPT [@packer2023memgpt] treats the context window like virtual memory, paging information in and out. TypeGo shares their OS framing but targets a fundamentally different workload: a multi-tasking embodied agent, whose resources include physical actuators with exclusivity and safety constraints rather than context tokens or API calls.

3 System Overview↩︎

TypeGo realizes the runtime thesis (§1) as a stack of asynchronous planning loops (S0 through S3) over a Skill Kernel that mediates the embodied agent’s physical actuators (1).

The runtime borrows its structure from OS design (1), which we use as a guiding analogy rather than flavor: most components have an OS counterpart, and we are explicit about where it deliberately departs. A user-authored task is a static, natural-language job specification, analogous to an OS program; the runtime realizes each task as a process with its own process control block (PCB). The Skill Kernel plays the role of the OS kernel, arbitrating the robot body’s typed actuator subsystems (its devices) among concurrent processes; reflexes act as interrupt handlers with priority preemption; and the speculative S1 step queue resembles a write-behind look-ahead buffer that decouples planning latency from execution.

6pt

3.0.0.1 Where the analogy departs.

The OS framing is productive precisely because an embodied agent breaks several assumptions a classical OS takes for granted, and these departures are where the research problems lie. First, resources are physical and semantic at once: preempting an actuator has real-world, sometimes unsafe, consequences, so clean preemption is a correctness property enforced by bounded-interruption skills. Second, scheduling requires understanding meaning: S3 decides which process runs from an LLM’s reading of the current observation and behavioral guidelines, a semantic policy a CPU scheduler never needs; additionally, whether a preempted process later resumes depends on the source of the preemption rather than a numeric priority (§5.2). Third, programs are natural language: prescriptions are ambiguous and are compiled or interpreted at runtime by LLMs rather than ahead of time (§4). Fourth, correctness is probabilistic: any planning layer can be wrong, so the runtime must contain and recover from bad outputs rather than assume correct programs. We return to these departures throughout.

The three mechanisms behind this runtime map to the sections that follow: speculative skill streaming overlaps planning with execution to hide LLM latency (§6); the Skill Kernel and S3 scheduler together multiplex concurrent tasks, running non-contending processes in parallel and preempting the rest cleanly (§5); and prescriptions make natural language a first-class programming medium, routed to the online planners or compiled into S0 interrupt handlers. Users and developers interact with TypeGo through three programming abstractions: prescription, observation, and skills.

4 Programming Abstractions↩︎

TypeGo exposes three programming abstractions: user writes prescriptions, and system developers supply observations and skills.

A prescription is TypeGo’s natural-language programming interface: it lets a user shape how the embodied agent behaves without writing code. The key design point is that TypeGo routes each prescription to the layer whose latency matches its purpose. TypeGo supports two kinds. Tasks are high-level objectives (e.g., “patrol the hallway to find a sports ball”) consumed online by the S1–S3 planners; each task spawns its own process, and multiple tasks may coexist. Reflexes are NL condition–action pairs that an LLM compiles offline into a pair of Python functions over authored skills and observation accessors; S0 evaluates them at high frequency for low-latency reactions (e.g., “when anything is closer than \(0.2\) m in front, step back \(0.5\) m”). S0 has the highest priority in TypeGo: a firing reflex interrupts any in-progress process holding contended subsystems and resumes it on completion.

@robot_skill("name", desc="...", subsystem=SubSystem.MOVEMENT)
def name(arg: type, [pause_evt], [stop_evt]) -> SkillReturn:
    # skill logic; check events for preemption

An observation is a structured, continuously updated representation of the embodied agent’s environment and internal state, grounded in the robot body’s sensors and exposed to all planners. It is a collection of fields, each with a developer-specified semantic role, data source, post-processing, and serialization. For example, an object_detections field binds to the front RGB camera, applies YOLO, and serializes detections as JSON for planner prompts.

Skills are callable primitives that implement concrete operations on the robot body; developers author and verify every skill (Listing [code:skill95interface]). Three principles guide their design. Synchronous: a skill blocks until it returns, matching ordinary function-call semantics, though TypeGo invokes it asynchronously through the Skill Kernel(§5.1). Interruptible or atomic: interruptible skills weave pause_evt and stop_evt into their control loop and stop within a bounded time \(t_s\), returning a SkillReturn that distinguishes success, failure, and interruption; a few hardware-constrained skills (e.g., stand) are atomic. This bounded-interruption guarantee is what lets TypeGo recover from failures and switch tasks with bounded delay. Authored, not synthesized: a skill’s control flow is fixed by the developer and never synthesized at runtime by TypeGo’s agents, though it may call learned models internally, keeping core operations predictable and verifiable.

Most skill interfaces are platform-agnostic, so TypeGo’s planners operate largely independently of the underlying hardware. Composite skills may encapsulate an entire optimized control procedure in a single invocation, provided they honor preemption within \(t_s\).

5 Runtime Abstractions↩︎

TypeGo’s runtime is structured like a small operating system for an embodied agent. Two abstractions carry the OS analogy: the Skill Kernel (§5.1), the resource manager that arbitrates typed actuator subsystems of the robot body among concurrent processes, and the process (§5.2), a schedulable, stateful unit that handles one user task. Together they meet two design objectives: (1) decouple skill design from scheduling, giving developers an extensible interface for adding new capabilities without touching the core runtime; and (2) enable safe concurrent and interruptible execution across multiple tasks, so that processes with disjoint exclusive subsystems run in parallel while conflicting ones are serialized or preempted.

5.1 Skill Kernel↩︎

The Skill Kernel is TypeGo’s resource manager. Like an OS kernel mediating device access among processes, it mediates a typed registry of actuator subsystems among the processes. It enforces two properties: across processes, two skill calls cannot conflict over a shared physical resource; within a process, a new skill call may override an in-flight one. The kernel arbitrates by acquiring subsystems on behalf of skills, the analogue of driver calls.

Subsystems. A subsystem abstracts a physical resource of the robot body that a skill consumes during execution. Each is configured along two axes: exclusive_per_process (only one process may hold it at a time) and parallel (multiple invocations may run concurrently). Developers can add a new subsystem (e.g., a manipulator arm or a display) without modifying the scheduler or existing skills. TypeGo ships three: MOVEMENT (exclusive, serial) for locomotion and gestures, which rejects a second requester to prevent conflicting motion; SOUND (shared, serial) for audio output, which serializes invocations in a queue; and DEFAULT (shared, parallel) for skills needing no exclusive hardware.

Execution interface. When a skill such as move_forward is invoked, its declared subsystem (MOVEMENT) is marked held by the calling process, blocking other processes that need the same exclusive subsystem. Processes use the non-blocking execute interface, which returns immediately with a unique id and a flag indicating whether the skill was scheduled or rejected due to a conflict; the caller queries progress via the id.

5.2 Process Abstraction↩︎

A process is TypeGo’s runtime unit of multitasking, modeled on an OS process. The runtime realizes each task by spawning a process with a private process control block (PCB) which holds the process ID; the task text (user- or S3-generated); the S2/S1 plan (subtask list and index, global events, S1 step queue); the skill-execution history; the lifecycle status (In-Progress, Paused, Success, Failed); and the robot skill set.

Life cycle. For every incoming task, S3 instantiates a PCB and a dedicated S1/S2 pair. While In-Progress, S2 decomposes the task and S1 streams skill calls; each call attempts to acquire its required subsystems, and any failure moves the process to Paused, where both loops block until S3 reschedules it. The process ends in a terminal state when it completes or is stopped by S3 or the user.

Process types. TypeGo distinguishes processes by source rather than numeric priority: user processes carry explicit instructions; reactive processes are spawned by S3 when standing behavioral guidelines fire; and a single idle process runs default behavior whenever nothing else is active.

Scheduling: concurrency. Two processes run simultaneously only when the exclusive subsystems they hold are disjoint; otherwise, one is Paused until the other releases its subsystems. A process retains its subsystems until it pauses or terminates; it keeps MOVEMENT reserved even after move_forward finishes, because a following take_pic may depend on the physical body staying still.

5.2.0.1 Scheduling: source-dependent resumption.

TypeGo encodes two post-preemption behaviors that a single numeric priority would conflate. Interrupt-and-return fires when a transient event (e.g., a person walks into view) triggers a brief reactive detour: the new process takes over, completes, and the displaced process auto-resumes. Replace-without-return fires when a new user instruction supersedes the old goal for good: the previous process is stopped and never auto-resumed. Flat priority schemes tell the runtime which process wins but not whether to stash the loser for later or discard it. S3 derives the distinction from the preemption’s source: a user process replaces-without-return (S3 stops every running user and reactive process, none auto-resumed unless the user’s task explicitly asks for it); a reactive process always interrupts-and-returns (it preempts only user or idle processes, and S3 auto-resumes the predecessor when it terminates); the idle process never preempts.

6 Hierarchical Collaborative Agents↩︎

TypeGo pairs rapid response with deliberation through four asynchronous agents, from S3 (slowest, most abstract) down to S0 (fastest, most concrete). The names generalize the System 1 / System 2 dichotomy [@kahneman2011thinking] to four cadences, but, unlike that two-tier view, our tiers run concurrently rather than as alternatives selected per request. S3 and S0 are global across processes; S2 and S1 are instantiated per process. Each runs its own independent loop.

6.0.0.1 System 3: process scheduler

S3 manages processes from two inputs: explicit user tasks and behavioral guidelines embedded in its prompt. The second lets TypeGo act autonomously. For example, under the guideline “you may pause a non-emergency task to engage a nearby person,” S3 pauses “find a sports ball” and starts “play with the person” when it detects someone. User-initiated processes outrank S3-created ones; S3’s concurrency and preemption policy is detailed in §5.2.

6.0.0.2 System 2: task decomposer

S2 decomposes a task into (1) an ordered list of subtasks, each a natural-language description of what to achieve plus a success criterion, which S1 consumes one at a time; and (2) a set of global events, condition/response pairs that S1 monitors as reactive interrupts during execution (Listing [lst:s2-plan]). After the initial decomposition, S2 periodically (every 2 s in our implementation) re-inspects the action history and observation and decides whether to keep or update the decomposition.

# S2 output for "Find an apple and report to the user. Take a picture whenever you see a ball".
{"operation": "update",
  "subtasks": [
    {"desc": "Find the apple", "criteria": "Found, or nowhere left to search"},
    {"desc": "Report to the user", "criteria": "See and speak to user"}],
  "global_evt": [{"cond": "see a ball", "resp": "take a picture"}]}

6.0.0.3 System 1: action planner

S1 turns the current subtask into a stream of skill calls (Listing [lst:s1-plan]) via two concurrent threads: a streamer that generates steps and an executor that runs them, both able to call an LLM. Rather than emit all steps at once, the streamer generates a few per LLM call into a bounded queue (size three in our case), while the executor dequeues the oldest; each dequeue triggers one enqueue, so generation overlaps execution. The stream-interpretation idea itself follows TypeFly [@chen2025typefly]; TypeGo’s contribution is to embed it in a multi-cadence runtime, where S2 continuously revises the subtask that S1 streams against and S3 can preempt the whole loop, rather than running it as a single open-loop interpreter. Each step holds one or more mutually exclusive branches pairing a condition (a predicate over observation or a short NL expression) with a skill call; the executor invokes the matching branch. The executor also watches S2’s global events: when one fires, it interrupts the in-flight skill, records the event on the PCB, and the streamer handles it on its next cycle.

// In-progress skill: stand_up(), Subtask: Find the apple.
{"steps": [
    {"branches": [{"action": "search('apple')", "cond": "always"}]},
    {"branches": [
      {"action": "approach('apple')", "cond": "visible('apple')"},
      {"action": "goto_waypoint(3)", "cond": "always"}
    ]}
  ],
  "subtask_complete": false}

6.0.0.4 System 0: reflexive layer

S0 evaluates a list of condition functions and invokes the matching action function; both are plain Python that an LLM compiles from reflexes offline, so no LLM sits on the reaction path. S0 outranks every task-driven process: a firing reflex pauses every process holding a contended subsystem and resumes each on completion. Reflexes themselves preempt one another by hard priority.

7 Implementation↩︎

Our current prototype is implemented in Python and packaged as a ROS2 node, with three parts: the TypeGo runtime (planning and execution), a lightweight platform-specific SDK that exposes skills and sensor data over ROS2, and a pool of AI services (LLMs and perception models). S0 runs at 100 Hz, S2 and S3 at 0.5 Hz, and S1 is event-driven.

Platform SDK and case study (Kalos). The ROS2 SDK is TypeGo’s data and control provider for the robot body: it publishes sensor data (RGB images, LiDAR scans, state estimates) and forwards control commands over TCP, and integrates SLAM and navigation with an indexed waypoint service. Our initial deployment, Kalos, runs TypeGo on a Unitree Go2 and registers skills in four categories: basic locomotion (move_forward, rotate, nav), etc; postural and expressive actions (stand, sit, nod); high-level composites (search, approach, follow, goto_waypoint, patrol); and system utilities (take_pic, speak, query_memory). TypeGo is not tied to the Go2: it runs on platforms as modest as the Petoi quadruped [@opencat_quadruped_robot] or a wheeled robot, requiring only basic locomotion and continuous perception.

AI services and deployment. An edge server (with an RTX 4090) runs an HTTP gateway that forwards LLM calls to the cloud and dispatches perception to a pool of stateless gRPC workers: object detection (YOLO [@redmon2016yolo]), open-vocabulary detection (OmDet [@zhao2024omdet]), image–text embedding (CLIP [@ilharco_gabriel_2021_5143773]), and TTS. Our deployment uses two off-robot machines over Wi-Fi: a laptop for the runtime and a GPU edge server, but the stack can be consolidated onto a single on-robot edge device (e.g., an NVIDIA Jetson) for self-contained mobile operation.

8 Preliminary Evaluation↩︎

We use a small prototype task suite to validate three claims from §1: effectiveness (can the prototype complete composite, long-horizon, perception-grounded tasks end-to-end, and degrade gracefully when it cannot?); responsiveness (does layered planning reduce per-step latency relative to a re-plan-every-step controller while retaining more adaptivity than a monolithic planner?); and concurrency (can the runtime admit concurrent tasks or reflexes, how fast is preemption, and at what cost?). The goal is not to exhaustively benchmark robot planning, but to test whether the proposed OS-style abstractions behave as intended on a real embodiment: effectiveness probes the end-to-end runtime, responsiveness probes multi-cadence planning with speculative streaming, and concurrency probes the Skill Kernel together with scheduling.

8.0.0.1 Baselines and setup.

Both baselines share TypeGo’s skill library and use minimal-reasoning prompts. TypeGo and both baselines drive every planning layer with GPT-5.4; the sole exception is TypeGo’s S1P fast-response path, which uses Groq-hosted GPT-OSS-120B (§6.0.0.3). ReAct [@yao2023react] emits one reasoning step plus one skill call per step from the instruction, current observation, and prior trace; it has no decomposition/execution split, speculation, reflex, or scheduler, so it cannot plan its next action until the current skill returns. Plan-and-execute (PAE) prompts once for a complete ordered plan, executed open-loop. We run three task categories (2), with 10 trials per task.

8.1 Effectiveness and Responsiveness↩︎

7pt

3 reports success and three latency metrics: TTFA (input to first action), total (normalized end-to-end time), and per-step (wait between skills), plus normalized token usage. In this prototype task suite, TypeGo attains the lowest TTFA (40% below ReAct, 73% below PAE) and the lowest completion time, and cuts per-step delay by 50% relative to ReAct. Two design choices drive this. First, S1P emits one immediate skill call at process creation, overlapping the first action with the higher-level planners’ deliberation and pulling reaction latency under one second; this action matches the eventual plan in roughly 60% of cases but consistently produces visible feedback within the first second. Second, the speculative step queue keeps upcoming skill calls ready, removing the per-step planning stall that dominates ReAct’s latency. The cost is token usage: TypeGo’s multiple asynchronous loops consume substantially more tokens than ReAct (one loop) or PAE (one initial call), an explicit responsiveness–cost tradeoff we leave to future runtime-policy optimization.

8.1.0.1 Failure cases.

T3 has a blocked path: TypeGo and ReAct both recognize the failure and report it after a few attempts, while PAE has no closed-loop feedback and stalls indefinitely (0/10). This supports the need for reactive re-planning, but also shows why the prototype should be tested on richer failure modes. T4 is the only task where PAE wins: its global plan commits to going to the hallway before searching, whereas TypeGo and ReAct greedily inspect chairs already in view. It is a known bias of step-by-step planning toward locally promising actions [@wang2026reasoningfailsplanplanningcentric; @xu2026faraheadllmsplan] and an open design point for the S1/S2 split.

8.2 Concurrency and Scheduling↩︎

Each concurrency task injects a second activity during T5, exercising a different scheduling path and, with it, a different facet of the abstractions. T6 and T8 both follow the interrupt-and-return path (one triggered by a user instruction, one by an S0 reflex) and qualitatively confirm that the displaced process is paused and correctly resumed afterward, while T7 confirms that processes holding disjoint subsystems run in parallel. Since neither baseline can admit a second activity mid-execution, we report only TypeGo’s scheduling overhead—the runtime beyond running the activities independently: their summed standalone durations when they contend and serialize (T6), or the base alone when the add-on runs in parallel (T7). Overhead tracks subsystem contention (4). In T6, the injected instruction contends with T5 for the MOVEMENT subsystem, so S3 serializes the two, pausing and resuming T5; of the 1.42 s total, roughly 1.2 s is S1 regenerating its step queue on resume, leaving only a small remainder as true scheduling overhead. T7’s memory query uses no movement subsystem and runs in parallel (0.15 s), and T8’s S0 reflex preempts T5 for a single obstacle-avoidance action before immediately returning (0.05 s). These early results suggest that runtime overhead is tied to subsystem contention, not merely to the presence of a second activity.

9 Future Work: Toward a Full Agent OS↩︎

TypeGo is an early step toward an operating system for embodied agents, not a finished OS. Our prototype focuses on three primitives that were immediately necessary for validating the idea on a physical embodiment: concurrent processes, resource arbitration over shared actuators, and multi-cadence planning. A complete agent OS needs additional primitives that make long-lived, adaptive agents manageable across hours, days, and deployments.

9.0.0.1 Memory as an OS primitive.

Memory remains the most important area for future work. Today, TypeGo includes only a simple memory service: the runtime samples observations and execution history, stores them in a retrieval-accessible database, and exposes retrieval through skills such as query_memory. This works, but it treats memory as an application service rather than an OS abstraction. In an agent OS, memory should be a first-class managed resource, analogous to a file system or virtual memory: processes should name, share, isolate, evict, checkpoint, and recover state through a common interface.

We envision a memory hierarchy mirroring the planners’ cadence, so each layer reads state at the abstraction and frequency it can afford: M0, raw structured observations (detections, pose, timestamps) consumed by the fast S0/S1 loops; M1, natural-language observations distilled from M0 (“at noon, a ball was seen near the kitchen”); M2, episode summaries that compress M1 and human interactions for S2; and M3+, long-term knowledge such as habits and regularities accumulated across tasks and deployments for S3’s strategic decisions. Lower tiers update at high frequency near perception; higher tiers are refined by active consolidation, asynchronous background processes that group, summarize, and promote entries without stalling real-time control. Crucially, rather than exposing memory only through an explicit query API, the runtime itself would choose which tier to inject into each planner’s context, or even into its KV cache, so memory continuously shapes planning the way virtual memory backs computation.

This raises OS questions that are distinct from ordinary vector-store design. What is the unit of memory: a raw sensor frame, an event, an executed skill, a failed plan, a user preference, or a distilled episode? Which memories are process-private, which are shared across processes, and which survive process termination? How should the runtime arbitrate memory bandwidth and storage budget among competing agents? What consistency guarantees should hold when S1, S2, S3, and S0 read or write different views of the same world state? When should observations be evicted from memory, given that semantically obsolete data may remain safety-relevant? These questions are central to the OS vision because planning, scheduling, and resource control all depend on what the agent can remember.

9.0.0.2 Richer resource and isolation models.

We evaluated TypeGo on a quadruped robot, whose main exclusive resource is locomotion. Other embodiments, such as humanoids and mobile manipulators, will require richer subsystem declarations: arms, hands, gaze, torso posture, displays, tools, private data streams, and external APIs. The Skill Kernel mechanism already supports new subsystems, but future systems should expose stronger isolation policies: which processes may see which observations, which tools may externalize data, and which skills may run concurrently under safety constraints. This moves the Skill Kernel closer to an OS protection boundary, not just a scheduler for actuators.

9.0.0.3 Cost-aware runtime policy.

The current prototype deliberately spends tokens to buy responsiveness. A production agent OS should make this tradeoff explicit and dynamic: S2 monitoring could become event-triggered, S1 could skip re-generation when the queue and observation are unchanged, and the use of smaller distilled models. More generally, the scheduler should consider not only actuator contention but also LLM budget, model latency, network availability, and risk.