1 Introduction↩︎

Large language model (LLM) agents are now expected to provide help proactively rather than waiting for instructions, motivating recent work on proactive desktop assistants, context-aware agents, and mobile agents [1]–[5]. The hard problem in proactive assistance is not task execution but the two decisions that precede it: when to intervene at all, and what to ground the intervention on. Both have been recognised since early work on proactive and context-aware agents [6]–[8].

A proactive assistant maps these two decisions plus the user-facing execution onto three responsibilities: an always-on trigger that decides when to interrupt, a context provider that selects which entities accompany the intervention, and a downstream agent that produces the user-facing action. Current systems instantiate all three with language-model-shaped components [5], [9]–[11]. Figure 1 contrasts the fragmented design with the unified architecture we develop in the rest of the paper.

Two problems follow.

Expensive and slow triggering. A trigger that runs continuously becomes the most expensive component if it is itself a large model. The demo released by [1] attempts a new offer every \(15\) seconds, i.e.\(240\) trigger decisions per hour, each paying full LLM cost regardless of whether a task is generated. Recent work has highlighted the need for real-time intervention [3], [5], [11]; LLM latency sits directly on the critical path.

Ungrounded, decoupled suggestions. When a proactive system does fire, the resulting suggestion often fails to name the specific artifact the user is working on: on busy session contexts the agent often defaults to the session topic or the active application rather than the entity actually in focus. The trigger and the context module are typically separate stages [1], [2], [5], [12], so the wake-up signal and the evidence supporting it can disagree.

These two problems share a root: language-model-shaped components are inefficient for the always-on path. The trigger and the context provider run continuously on every event, where a strong reasoning model can certainly do the job but a small specialised model handles it at orders of magnitude lower per-event cost. And the signal they run on is not natively text: when a user clicks a file, switches an application, or types a query, the system logs a structured event (actor, verb, object, timestamp) and renders it through a fixed template (e.g., "The user opened ‘email_filter.py’ in Visual Studio Code."); the desktop and mobile benchmarks release these activity streams in that rendered textual form for downstream consumption [1], [3]. The structure is the data; the text is a serialisation. The same entities recur across events (the same file edited at \(t{=}3\) and \(t{=}17\)) and each event touches multiple entities at once, so flattening to a feature row loses the cross-event entity linkage and flattening to a token sequence loses the event-entity many-to-many structure; the native representation is a heterogeneous temporal graph. Rendering the structure as text and asking an LLM to recover it is a round-trip the system never had to take. Once the always-on signal is treated as graph updates rather than text, the wake-up decision becomes a node classification, the entity selection becomes another node classification on the same graph, and the language model is reserved for what it is good at: turning a small structured handoff into a fluent user-facing sentence.

We construct a heterogeneous temporal interaction graph from user activity and use a small temporal-graph-learning (TGL) model as the encoder. A trigger head over event nodes produces a wake-up probability (addressing the expensive-triggering problem) and a routing head over entity nodes produces a per-entity relevance score (addressing the ungrounded-suggestions problem), both read off one forward pass and trained jointly. The downstream language agent is invoked only when the trigger head clears its threshold and consumes the cleared routing entities as structured context; the graph trigger is small enough to run on-device alongside the privacy-sensitive activity stream it consumes (file names, URLs, search queries), a footprint not reached by the LLM-as-trigger configurations measured in Section 5.4.

Our contributions are threefold:

• We formulate proactive triggering and context selection as joint node-level prediction over a heterogeneous temporal event-entity graph rather than as a language-modelling problem, enabling a lightweight always-on controller before downstream LLM generation.

• We instantiate this design with a per-event trigger head and a per-entity routing head sitting on a shared TGL backbone, trained jointly against an anchor-routing objective so that the two decisions are produced from the same hidden state in one forward pass.

• On the ProactiveAgent benchmark, experiments across \(14\) language-agent backbones show that TGL improves F1 on every backbone (range \(+3.1\) to \(+46.0\), mean \(+16.7\)). Across four trigger families, the same checkpoint supplies the strongest trigger AUCs, the most stable deployed threshold, and a lightweight always-on footprint: \(11.13\) ms per event on a GPU server and \(13.99\) ms on a consumer laptop, \(\sim\!4\)–\(7\times\) and \(\sim\!12\)–\(83\times\) faster than every single-forward LLM-as-trigger configuration we tested across the \(0.6\)–\(8\) B Qwen3 range.

2 Related Work↩︎

Proactive assistance. The idea of proactive assistance predates LLM agents and has progressed through situation-aware assistive systems and human-centered interaction studies into LLM-era trigger-and-task formulations. Recent benchmarks formalize proactive task suggestion as a per-event fire-or-skip decision on both desktop [1] and mobile [3] platforms, with broader proactive-agent formulations following [11]. We isolate the always-on trigger and keep intervention decisions timely, cheap, and behaviorally selective.

Memory, personalization, and realistic evaluation. A second body of work studies how agents accumulate user context (memory-augmented agents, retrieval-augmented user models) [13], [14], but these efforts focus on what happens after the system has already decided to engage. Complementary human-computer interaction work establishes that interruption timing and attentional cost materially affect user experience [15], [16], and recent web, GUI, and mobile benchmarks provide realistic substrates for autonomous agents [17], [18]. Together, these two lines establish the pre-invocation policy as the load-bearing step: the wake-up decision must be both personalized and interruption-aware before any downstream worker is called.

Temporal graph learning. The temporal-graph literature provides the inductive bias most aligned with the proactive-trigger setting. Continuous-time dynamic graph models such as TGAT and TGN capture evolving interactions without collapsing them into static snapshots [19], [20], building on standard graph encoders [21]. Our TGL encoder stacks relation-aware GATv2 layers [22] with a Jumping Knowledge head [23] over the heterogeneous temporal graph; the wake-up and routing heads are two node-classification readouts off the same encoded graph.

The full citation list and broader discussion are in Appendix 15.

3 Problem Formulation↩︎

We consider a proactive assistant that observes a timestamped event stream \(\mathcal{E} = \{e_1, e_2, \ldots, e_t, \ldots\}\), where each event records a user or environment observation such as an application switch, file edit, browser visit, query, or other interaction trace. At each decision time \(t\), the system must answer two coupled questions: (i) should the expensive worker agent be invoked now; and (ii) if so, which contextual entities should be routed forward as evidence?

Both decisions are produced from the observed history up to \(t\), represented as a heterogeneous temporal graph \(G_t = (V_t, E_t)\) over event nodes and the entities they touch.

The supervision signal contains a label per event and a label per entity. For each event node \(e_t\) a binary trigger label \(y^{\mathrm{trig}}_t \in \{0, 1\}\) records whether the assistant should fire. For each entity node \(v\) a binary routing label \(y^{\mathrm{rout}}_v \in \{0, 1\}\) records whether \(v\) is task-relevant evidence at the current decision. We jointly model the two questions as two readouts off a shared encoder of \(G_t\): a trigger head over event nodes produces \(\hat{y}^{\mathrm{trig}}_t\), and a routing head over entity nodes produces \(\hat{y}^{\mathrm{rout}}_v\). The two heads are trained against a single combined objective and read off the same hidden state in one forward pass.

4 Method↩︎

We model proactive activity as a heterogeneous temporal graph \(G_t = (V_t, E_t)\). Nodes represent the events themselves and the semantic entities the user touches at each event (files, applications, URLs, search queries, and lightweight type nodes that group entities by extension, topic, domain, or query language). Edges encode has_entity relations from an event to its extracted entities and the reverse belongs_to relation, two directional temporal edges between consecutive events (forward \(t \to t{+}1\), past-informs-future; backward \(t{+}1 \to t\), future-informs-past), and self loops. The graph is built deterministically from the event stream by a rule-based extractor; construction details (entity vocabulary, type-augmentation rules, edge channels) are deferred to Appendix 8.

4.1 Joint Trigger and Routing as Two Heads↩︎

A single TGL backbone consumes the per-session graph and produces two outputs in one forward pass: (i) a per-event trigger probability \(p_{\mathrm{trig}}(t)\) from a trigger head applied only to event nodes; and (ii) a per-entity routing score \(s_{\mathrm{rout}}(v)\) from a routing head applied only to entity nodes.

The two heads share the same TGL body, so they consume the same hidden state and are trained jointly. This is the architectural sense in which triggering and routing remain unified: not as a single classifier (the trigger is binary over events while the routing is binary over entities), but as two readouts off one shared graph encoding, optimized together against one loss. The encoder stacks three relation-aware GATv2 layers [22] with a Jumping Knowledge head [23] that concatenates the input projection and every layer; node features combine a frozen text-encoder embedding of the surface label with a learned type embedding and, for event nodes, a time-gap embedding. Layer dimensions, head counts, and the exact projection sizes are in Appendix 11.

At test time, the trigger fires when \(p_{\mathrm{trig}}(t)\) clears the natural sigmoid decision boundary, and the highest-scoring routed entities are forwarded to the downstream agent as structured context; when the trigger is below threshold no downstream-agent call is issued. The trigger threshold, routed-list cap, and causal-masking protocol applied at inference are held fixed across every backbone (Appendix 11).

4.2 Anchor Routing Labels↩︎

The trigger and routing heads are trained on event- and entity-level labels derived from the released ProactiveAgent training split. Appendix 8 gives the deterministic labeling procedure and train/test isolation checks; at evaluation time the heads read only the held-out event graph.

4.3 Downstream Agent↩︎

Our TGL model is a node classifier: its output is a scored list of graph nodes, not a natural-language suggestion. A downstream language agent turns the routing list into a user-facing action and runs only when the trigger fires. We instantiate this agent with a frozen instruction-tuned chat model that consumes the routing list as structured context and emits a JSON object whose proposed-task string is the proactive suggestion. Prompt design and few-shot construction are described in Appendix 9; the same prompt is used across all backbones.

5 Experiments↩︎

5.1 Setup↩︎

The main results follow the protocol of [1] on the desktop ProactiveAgent benchmark; transfer evidence on the mobile FingerTip-20K benchmark [3] is reported in Appendix 7. The vanilla baseline is the originating paper’s published vanilla prompt protocol against the same judge; + Ours feeds the same downstream agent TGL-routed structured context through our prompt. The judge is the released ProactiveAgent reward model, trained to score proactive task suggestions [1].

One TGL checkpoint, one trigger threshold, and one prompt template serve every backbone. We evaluate \(14\) instruction-following backbones spanning four families (open-weight, OpenAI, Anthropic, DeepSeek) as the downstream language agent; two fine-tuned proactive baselines from [1] appear for reference. Backbone list, model architecture, prompt, serving setup, and graph-construction rules are deferred to Appendices 8, 9, and 11.

5.2 Main Results↩︎

2.2pt

Table 1 compares each downstream language agent against the vanilla protocol from [1].

Top-line numbers. Relative to the ProactiveAgent vanilla downstream-agent runs in Table 1, TGL improves F1 on every one of the \(14\) backbones (range \(+3.1\) to \(+46.0\), mean \(+16.7\)); no backbone regresses. The lift extends across the full backbone spectrum, from open-weight \(4\)–\(8\)B models (Qwen3-8B \(+46.0\), Qwen3-4B \(+28.2\), LLaMA-3.1-8B \(+17.0\)) through mid-tier APIs (DeepSeek-V4-Pro \(+26.6\), Gemma-3-12B-it \(+19.3\), Claude-Sonnet-4.6 \(+17.6\)) to the strongest closed reasoning models (Claude-Opus-4.7 \(+5.4\), GPT-5.4 \(+3.1\)). A single TGL checkpoint at a single fixed threshold delivers the gain without per-backbone tuning of the trigger, the routing, or the prompt. The trigger and routing decisions are produced from the graph alone and consumed identically by every backbone, so the lift transfers cleanly between architectures, parameter scales, and serving regimes.

Beating downstream-LLM fine-tuning with a small graph-encoder add-on. [1] release two fine-tuned proactive baselines that adapt LLaMA-3.1-8B and Qwen2-7B end-to-end on the ProactiveAgent training set, reaching F1 \(=\) 66.25 and 66.47. The cleanest within-architecture comparison is Qwen2-7B: vanilla F1 \(=\) 60.74, proactively fine-tuned \(=\) 66.47, + Ours \(=\) 70.68. TGL prompting beats parameter-level proactive adaptation on the same backbone.

One calibrated signal serves every backbone. TGL delivers a single wake-up decision and a concrete routing anchor that every downstream agent consumes the same way; one checkpoint at the deployed threshold aligns all \(14\) backbones to \(R \geq 96\)%, \(P \geq 50\)%. The vanilla configurations drift in opposite directions: some over-fire on essentially every event (Qwen3-8B at \(R = 99\)%, \(P = 15\)%; Qwen3-4B, GPT-4o-mini, Gemma-3-12B-it at \(R = 91\)–\(100\)%, \(P = 24\)–\(43\)%), others under-fire (DeepSeek-V4-Pro vanilla \(R = 52\)%, Claude-Sonnet-4.6 \(R = 71\)%). The trigger absorbs the over-firers’ false-alarm wedge; the routing field gives the under-firers a concrete anchor that pulls them into action.

The lift concentrates where vanilla self-regulation drifts furthest. Backbones whose vanilla configuration sits furthest from the calibrated operating point, over-firing (Qwen3-x, Gemma-3-12B-it) or under-firing (DeepSeek-V4-Pro at vanilla \(R = 52\)%), gain \(+19\) to \(+46\) F1; both heads of TGL carry their full weight. Configurations whose vanilla self-regulation is already close to that operating point (GPT-5.4 vanilla F1 \(=\) 73.5, Claude-Opus-4.7 vanilla F1 \(=\) 74.4) gain \(+3.1\) to \(+5.4\), driven by the routing field reaching the agent on events where it would otherwise have stayed quiet.

The cost-sensitive deployment regime shows the largest lift. The high-gain class includes the cheaper backbones (open-weight \(4\)–\(8\)B models, mini-class APIs, DeepSeek-V4-Flash), the setting where per-event trigger cost matters most. The open-weight subset is additionally favoured by privacy: a desktop proactive agent’s activity log (file names, URLs, search queries) is sensitive (Limitations), and on-device serving keeps it local. The strongest reasoning configurations (GPT-5.4, Claude-Opus-4.7) self-regulate close to the calibrated operating point already; their per-call latency (Appendix 10.4) also sits outside the user-facing intervention budget targeted by timely proactive assistance [3], [5], [11].

5.3 Ablation Study↩︎

Figure 3 disentangles the contributions of the trigger and the routing list with three controls.

Does the trigger matter? Disabling the trigger and forcing the downstream agent to fire on every event while still feeding it the TGL routing list (No trigger bars in Figure 3) trails Ours by a mean of \(+3.2\) F1, positive on every one of the \(14\) backbones (range \(+2.2\) to \(+5.3\)): by abstaining on events where the trigger head is below threshold, the trigger suppresses the false-alarm wedge that always-firing pays for. The gap is largest on backbones that over-fire when always firing (Claude-Sonnet-4.6, Qwen3-4B), and smallest on the strong reasoning models that already self-regulate.

Does the routing content matter? Replacing the routing entities with random in-distribution entities at the same cardinality (drawn from a held-out pool of entity surface labels of the same types the production routing list contains), while keeping the trigger and prompt fixed (Rand bars in Figure 3), trails Ours by a mean of \(+8.6\) F1 on every one of the \(14\) backbones (range \(+1.1\) to \(+20.7\)). The gap is largest on LLaMA-3.1-8B-Instruct (\(+20.7\)) and Qwen3-8B (\(+18.1\)), which lean most heavily on the routing scaffold, and on Qwen2-7B-Instruct (\(+14.6\)). The downstream agent uses the identity of the routing entities, not just the presence of a routing field.

Is it just the prompt? The + Ours prompt bundles TGL-specific instructions (“treat the first routing entry as the anchor”, “quote its surface text”) with TGL-agnostic style directives (“single sentence, max \(150\) characters”, “lead with what the user receives”). The No TGL bars in Figure 3 report a matched no-graph prompt-only control: a single-page prompt that keeps the style directives, removes every TGL-specific instruction and the three routing few-shots, omits the routing field, and asks the language agent to self-gate by emitting an empty proposed-task string. This matched prompt-only control isolates the prompt surface; the Rand bars above isolate the routing-content variable on its own. The mean \(\Delta\) vs Ours is \(+12.7\) F1, positive on every backbone (range \(+2.8\) to \(+26.1\)). The trigger and the routing content together carry the lift; the dedicated trigger contributes a uniform fire/skip decision independent of the downstream backbone, whereas prompt-only self-gating depends on each LLM’s own gating calibration, and on this panel no backbone’s self-gating reaches the dedicated trigger’s F1.

5.4 Which Data Structure Best Suits the Proactive Trigger?↩︎

4pt

Our central claim is that the always-on path is a graph problem rather than a language problem. Testing this claim requires triggers from the non-graph regimes a practitioner would otherwise reach for. We compare against the conventional baseline in each non-graph family: (1) Rule, the canonical zero-parameter dictionary; (2) Tabular, standard logistic regression (LR) and histogram-based gradient-boosted-tree (HGB) classifiers on hand-crafted features, the convention for binary event prediction; (3) Textual, the standard BERT-style recipe of an MLP head on the same text encoder TGL uses, in both frozen and end-to-end fine-tuned variants (BGE-frozen + MLP and BGE-FT + MLP); and (4) LLM-as-trigger, with two output protocols. The first is the standard chat-LLM-as-classifier setup: yes/no logit readout (the MCQ-evaluation convention) over two Qwen3 sizes (\(0.6\) B and \(8\) B), each fine-tuned on the same supervision every other trained trigger sees. The second keeps the ProactiveAgent full-generation recipe at the \(0.6\) B scale and derives fire/skip from whether the generated proposed_task field is empty. Each baseline shares the same training supervision as TGL; the routing list, prompt, and downstream agents are held fixed, so each comparison isolates the trigger. Architectural and training details for all four families are in Appendix 10.2.

Gold-label structure. The released gold labels split the test set into three classes: must-fire (\(\sim 24\%\)) where a non-null proactive task is required, can-skip (\(\sim 51\%\)) where the no-suggestion response is acceptable but a suggestion may also be acceptable, and a must-skip subset of can-skip (\(\sim 14\%\)) where every alternative suggestion is rejected. The three AUC columns in Table 2 report ranking quality under different positive-class definitions: m-fire (positives are the must-fire class), c-skip (positives are every event outside the can-skip class), and m-skip (positives are every event outside the must-skip subset). A higher AUC means the trigger pushes the positive set higher in its score distribution, so all three columns reward the same orientation: fire when fire is required, abstain when not.

TGL leads every trigger AUC. TGL reaches \(0.738\) AUC on must-fire ranking, \(+7.0\) pp above the strongest LLM trigger (Qwen3-0.6B at \(0.668\)). The two skip-side columns show the same pattern: TGL at \(0.658\) on can-skip and \(0.639\) on must-skip, \(+4.0\) and \(+4.2\) pp above the next best (Qwen3-8B at \(0.618\) and \(0.597\)). The AUC lead comes with lower latency: TGL runs at \(11.13\) ms per event versus \(40.4\)–\(1156.8\) ms for the three LLM-as-trigger configurations. Scaling LLM-as-trigger by \({\sim}14\times\) (\(0.6\) B \(\to\) \(8\) B) does not lift the must-fire AUC (\(0.668 \to 0.644\)), and the full-SFT recipe on the same \(0.6\) B base lands at \(0.505\), indistinguishable from chance.

Generation-derived triggering pays decoding cost. Table 2 separates the LLM family from the output protocol. The two LoRA Qwen3 rows answer the fire/skip question with one forward pass and a yes/no logit read. The ProactiveAgent full-SFT row keeps the proactive-generation recipe intact: it generates a full proposed-task string for each event and converts the result to a trigger score by thresholding whether the field is empty. Under that protocol, latency is \(3{,}927\) ms on server and \(11{,}966\) ms on laptop (\({\sim}50\)–\(100\times\) the single-forward LLM rows on server alone), while the trigger AUCs are \(0.505 / 0.508 / 0.485\), indistinguishable from chance on the three binary readings of the gold labels. The pattern supports the deployed division of labour in TGL: the always-on controller makes a fast binary decision, and full downstream LLM generation is reserved for events that pass the trigger.

Well-calibrated probabilities serve every backbone with one threshold. The Brier and ECE columns in Table 2 split the panel cleanly: TGL, Rule, and the Tabular triggers produce probabilities that track the label (Brier \(0.27\)–\(0.32\), ECE \(0.20\)–\(0.33\)); the BGE-flavoured Textual baselines and the LLM triggers collapse to the score extremes (Brier \(0.58\)–\(0.66\), ECE \(0.59\)–\(0.66\), roughly \(2\times\) worse), and their probability at the deployed threshold carries little information. The per-event probability distributions in Figure 4 (Appendix 10.2) show the mechanism: \(94\)–\(98\%\) of test events pin to \([0, 0.05) \cup [0.95, 1]\) for the LLM and BGE-FT rows, while TGL produces a continuous distribution centred on the deployed threshold. The deployment consequence appears in Trigger std: for each backbone we compute the oracle threshold that would maximise the downstream score with test-set access, and report the standard deviation across the \(14\) evaluation backbones. Trigger std measures how much the per-backbone oracle drifts across the panel; large drift means committing to one global threshold leaves performance on the table for many backbones, since the per-backbone oracle is not available at deployment. TGL’s Trigger std is \(0.035\), the smallest of all nine rows, so the deployed threshold is near-optimal for every backbone; non-graph triggers spread widely (\(0.14\)–\(0.26\) for LLM, \(\geq 0.36\) for BGE-flavoured Textual), so a deployment of those triggers needs a per-backbone calibration step, an operational complexity TGL avoids.

5.5 Efficiency↩︎

The motivation for replacing an LLM-shaped trigger with a small graph network is that the trigger sits on the always-on path: it runs on every event regardless of whether a task is ultimately generated. We measure latency and resource use on a single NVIDIA RTX A6000 (48 GB); the cross-trigger latency comparison lives in the trigger architecture ablation of Section 5.4 (Table 2), which evaluates four non-graph trigger families (Rule, Tabular, Textual, LLM) on the same test set; all rows except the generation-based ProactiveAgent full-SFT baseline use a unified single-forward protocol.

Per-event latency. TGL’s \(11.13\) ms forward is dominated by BGE text encoding (\({\sim}7\)–\(8\) ms for the event text plus a single fresh entity batched together); the GAT layers add \({\sim}3\) ms. Across the two single-forward LLM-as-trigger configurations in Table 2 (Qwen3-0.6B and Qwen3-8B), per-event latency is \(40.4\) and \(78.6\) ms; TGL is \(\sim\!4\)–\(7\times\) faster than each. The generation-based full-SFT row (Proactive-Qwen3-0.6B) decodes a mean of \({\sim}139\) task tokens per event and costs \(3{,}927\) ms on server, \({\sim}353\times\) TGL. On the public ProactiveAgent demo polling rate (one decision every \(15\) seconds, \(5{,}760\) events per day), this is \({\approx}6.5\) GPU-hours per device-year for TGL versus \({\approx}24\)–\(46\) GPU-hours per device-year for the LLM-as-trigger range.

Consumer-laptop deployment. On a consumer laptop with \(36\) GB memory (full hardware and runtime spec in Appendix 11), TGL runs at \(13.99\) ms per event, within a few ms of its server measurement. The two LLM-as-trigger configurations transfer at substantially higher per-event latency: Qwen3-0.6B reaches \(162\) ms and Qwen3-8B reaches \({\sim}1.16\) s, \(\sim\!12\times\) and \(\sim\!83\times\) slower than TGL. The small-model trigger class (Tabular, Textual, Graph) all stay within \(14\) ms on this laptop.

Memory footprint. TGL carries \(1.16\) M trainable parameters plus a \(109\) M frozen BGE encoder. Resident weights occupy \(\sim\!220\) MiB at BF16; per-session streaming inference peaks at \(\sim\!267\) MiB. An \(8\)B-class LLM trigger, by contrast, requires \(\sim\!16\) GB of VRAM at BF16 just to remain resident (Qwen3-8B’s \(8.19\) B parameters in Table 6), \(\sim\!75\times\) TGL’s resident footprint on the same hardware class. Even the smallest LLM trigger we tested (Qwen3-0.6B at \(\sim\!1.5\) GB BF16) is \(\sim\!7\times\) TGL’s footprint with substantially worse calibration (Section 5.4). TGL’s sub-gigabyte footprint fits inside consumer-class memory budgets (laptop GPUs, integrated graphics); an \(8\) B LLM-as-trigger plus a \(7\) B downstream agent largely saturates an A6000 once inference activations and KV cache are counted, and a consumer laptop that also has to run the user’s everyday applications has no headroom for either.

6 Conclusion↩︎

We presented a TGL-based trigger plus routing for proactive agents, in which a small graph model trained on activity-event labels jointly drives the wake-up decision and selects the entities forwarded as supporting context. The trigger and the routing head share a single backbone and a single forward pass, so the wake-up signal and the structured evidence delivered to the downstream language agent are produced from the same hidden state. The same TGL checkpoint serves every language-agent backend and improves F1; per-event trigger latency is \(11.13\) ms, \(\sim\!4\)–\(7\times\) faster than every LLM-as-trigger configuration we tested. More broadly, the study suggests a simple design principle for proactive agents: keep a lightweight temporal model always on, and reserve full LLM reasoning for moments that survive its trigger.

Limitations↩︎

We focus on the trigger and routing heads. We follow the offline RM-judge protocol of [1] and [3]; subjective user experience under deployment is not part of this evaluation.

On ProactiveAgent, the released activity stream is text-serialised, so our implementation reconstructs platform entities with the deterministic extractor in Appendix 8. In a deployment that owns the data pipeline, the same graph nodes come directly from system identifiers such as file names, application names, URL domains, and file extensions.

Always-on activity logs are privacy-sensitive: file names, URLs, search queries, application switches, and (on FingerTip) demographic profile fields all flow through the trigger and routing heads. The graph-shaped architecture proposed here keeps these signals within the user’s trust boundary by enabling end-to-end on-device deployment (Section 5.5); in the trigger comparison of Section 5.4, the LLM-as-trigger rows require substantially larger resident memory under the same always-on setting. Production deployment additionally requires standard data minimisation, sensitive-entity filtering, opt-out, and retention controls. Fairness analysis across the FingerTip demographic subgroups (gender, age, occupation, location, device) is required before any are used as routing features in a deployed system.

Ethics Statement↩︎

This work uses publicly released benchmarks (ProactiveAgent [1] and FingerTip-20K [3]) under their original terms. We introduce no new data collection and do not redistribute the underlying activity logs. The contribution of this paper is methodological: a more efficient and better-calibrated trigger and routing architecture for proactive agents. The upstream privacy properties of the benchmarks are inherited from their releases. Privacy, fairness, and deployment considerations specific to a real-world always-on activity-monitoring system are discussed in the Limitations section.

Use of AI Assistance↩︎

Large language models were used as a general-purpose writing and engineering toolkit during the preparation of this manuscript: drafting and editing prose, and debugging code. All AI-generated text and code that appears in the final manuscript was reviewed and audited by the authors. The technical contributions, experimental design, and analysis presented in this paper are the authors’ own.

7 Auxiliary Study: TGL on Mobile Proactive Agent↩︎

This appendix evaluates the TGL graph-encoder backbone on mobile proactive task suggestion (FingerTip-20K [3]) as broader evidence that a typed temporal-graph encoder transfers across proactive-agent settings. FingerTip formalises proactive task suggestion as a single-intent prediction per episode; we train the same TGL GNN backbone from scratch for \(40\) epochs on FingerTip’s own training split and use the query-node semantic vector as a retrieval prior over training intents, paired with a same-app coherence filter (Appendices 8.2 and 9.2). The downstream agent is Qwen2.5-VL-7B-Instruct, the same backbone weights as the paper’s LoRA fine-tuning experiments.

7.1 Evaluation Protocol↩︎

We adopt the same evaluation as FingerTip-20K. The proactive task-suggestion track formalises the task as \(I = f(U, T, S, I_{\mathrm{history}}, O)\), where the agent predicts a Chinese natural-language intent \(I\) from the user profile \(U\), the timestamp \(T\), the scenario (location category) \(S\), the user’s last \(20\) historical intents \(I_{\mathrm{history}}\) drawn from the global usage log, and \(t\) initial screenshots \(O\) from the current episode (\(t \in \{0,1,2,3\}\) is the official task-difficulty parameter). We evaluate on the public Test-suggestion split of \(1{,}000\) episodes and report the same two metrics as the paper: \(\mathrm{Sim}_1 = (S_1 + S_2)/2\), where \(S_1\) is the paraphrase-multilingual-``MiniLM-L12-v2 cosine and \(S_2\) is the Levenshtein similarity, both normalised to \([0,1]\); and \(\mathrm{SR}_1\), a DeepSeek-V3 binary equivalence judgment between predicted and target intent strings. The downstream agent is Qwen2.5-VL-7B-Instruct served via vLLM, using the same backbone weights as the paper’s LoRA fine-tuning experiments.

7.2 Results↩︎

4pt

Table 3 compares against the published numbers from [3]. TGL plus Qwen2.5-VL-7B at \(t{=}2\) exceeds the paper’s strongest LoRA fine-tune on both metrics (\(\mathrm{SR}_1 = 26.5\%\) vs.\(26.0\%\), \(\mathrm{Sim}_1 = 0.61\) vs.\(0.55\)). Against the paper’s GPT-4.1 vanilla baseline at the same screenshot budget (\(t{=}3\), full history), we improve \(\mathrm{Sim}_1\) by \(+0.04\) and \(\mathrm{SR}_1\) by \(+12\) percentage points using only the last \(20\) history events. The comparison uses the same Qwen2.5-VL-7B weights as the paper’s LoRA setup, so the gain is from the typed session-graph routing rather than from a stronger language agent.

8 Per-Session Graph Construction↩︎

8.0.0.1 Principled construction rule.

The graph nodes our TGL backbone consumes correspond to entities the operating system or mobile platform already exposes through its standard APIs: applications from the OS app registry / installed-app list, files and their extensions and mime types from the filesystem, URLs from the browser’s history layer, search queries from query logs, and per-user demographic context from the user-profile store. The natural implementation of the entity-extraction layer is therefore a thin schema mapping over these native APIs, independent of platform-specific surface vocabulary. We use regex parsing in the public-benchmark implementation because both releases provide already-rendered activity strings (the storage format their data pipelines emit); the regex layer recovers from the rendering exactly what the platform internally already had as structured data.

8.0.0.2 Same recipe, two instantiations.

Both benchmarks share the same general design: anchor nodes (event nodes for desktop, a single query node for FingerTip), entity nodes (apps, files, URLs, queries, etc.), shared type nodes carrying coarse class-level information, a temporal chain linking consecutive anchors in chronological order, self-loops with learned relation embeddings, and a frozen text-encoder providing surface-label embeddings concatenated with a learned \(32\)-d type embedding and a \(32\)-d log-time-gap MLP. The same three-layer TGL GNN backbone consumes both graphs (Appendix 11). The two benchmarks instantiate this recipe with different per-platform schemas (entity vocabulary, edge channels) and different head shapes matched to each protocol’s prediction target. The two subsections below give the per-dataset instantiation details.

8.0.0.3 Why rules suffice.

Activity-log events are templated by construction: a real desktop activity logger records each user action as a structured (actor, verb, object) tuple and renders it through a small template grammar (“The user opened ‘{file}’ in {app}.”, “The user searched for ‘{query}’ on {site}.”); a mobile usage logger emits structured tuples wrapped in a fixed template. Users do not write generic prose descriptions of their own clicks; the underlying object is already an event-session knowledge graph serialised to text for storage, and the extraction step reverses the rendering. Standard NER and entity-typing pipelines are designed for genuine natural-language sources (Wikipedia, news) and tend to inject false positives by hallucinating entities the template never emitted, so we rely on regex and categorical rules: the same structured data that produced the rendering recovers it cleanly. The same reasoning applies one level up to our central claim: the always-on trigger-and-routing signal lives natively in graph form, which motivates a graph encoder for the controller path.

8.1 Instantiation: Desktop Activity Logger (ProactiveAgent)↩︎

The rule-based recipe by which we turn a desktop activity stream into the heterogeneous temporal graph consumed by TGL is fully deterministic; nothing in it queries an LLM or a reward model.

Inputs↩︎

For each session we receive a sequence of timestamped event strings produced by the user-activity logger. Examples: “The user opened ‘email_filter.py’ in Visual Studio Code.”, “The user searched for ‘js fade-in’ on Bing.”. Sessions are processed independently; there are no event-to-entity edges across sessions. Metadata-only events (lines starting with “# Assistant Available Operations” or containing “Operation(name=”) are filtered out before extraction.

Entity Extraction↩︎

The extractor reads each event against the platform’s standard entity ontologies and emits one node per match, in five entity classes:

(i) app: match against the desktop’s installed-application registry. We use the surface-label vocabulary that ProactiveAgent’s activity logger emits in its training stream (eight distinct labels: Visual Studio Code / VSCode / Code.exe, IntelliJ IDEA, Outlook, Jupyter Notebook, Terminal, Browser / Chrome / Bing / Google, Email Client, IDE); a deployment on a real desktop would read this set from the operating system’s installed-application registry directly. Surface aliasing collapses to a fixed canonical id (e.g., Visual Studio Code / VSCode / Code.exe \(\to\) app:vscode) over each app’s known aliases.

(ii) file: tokens whose extension is registered in a standard file-type database (e.g., the freedesktop shared-mime-info database, or its Windows / macOS equivalent) become file nodes. The recognised extensions populate one type-node augmentation, file_ext:<ext>; a coarse semantic super-category derived from the extension populates a second, file_topic:<topic>, over five buckets (code, writing, data, slides, other). On ProactiveAgent’s training stream twenty-five extensions surface across these buckets; the count is observed, not designed.

(iii) url: any https?:// substring becomes a url node; the url_domain:<domain> augmentation extracts the registered top-level domain (a coarse www.-stripped last-two-parts heuristic; production deployments would resolve this against the Mozilla Public Suffix List for full eTLD+1).

(iv) query: substrings following the search-verb templates the logger emits (searched for "...", typed "..."); the query_lang:<lang> augmentation reads the dominant Unicode script block of the query text (cn, en, mixed, other).

(v) artifact: quoted strings preceded by document-class nouns from a small fixed list (file, document, report, spreadsheet, article, tab, summary, email, draft); deduplicated against (ii) so document "report.docx" does not double-emit.

Of these, (i)–(iv) read from external ontologies (OS application registry, MIME database, top-level-domain conventions, Unicode); (v) is a templated grammar match against the logger’s own surface format. None requires inspecting the test split. For (ii)–(v) the node id is <type>:<sha1(label.lower())[:16]>; this hashing collapses the same surface label across sessions to one node.

The full extension-to-bucket mapping for the file_topic augmentation, the canonical extension list per bucket, and the surface-alias-to-canonical mapping for apps are released with the code; the construction principle (read the platform-known ontology, project to its coarse classes) is what ports across deployments.

Anchor Selection and Per-Event Writes↩︎

For each event we choose one anchor entity: the first app-typed entity if any, otherwise the first extracted entity. Each event then writes (i) an event node with id event:<sha1(sample_id::event_id)>; (ii) a self-loop on the event node; (iii) a next_event edge from the previous event node when the event is not the first in the session; (iv) a has_entity edge from the event node to each extracted entity (after type augmentation); (v) a primary_entity edge from the event node to its anchor; and (vi) a next_interaction edge from the previous event’s anchor to the current anchor when both exist and differ, the entity-level temporal backbone.

What the Graph Network Actually Sees↩︎

The on-disk CSVs above retain semantically-typed edges (opened, edited, primary_entity, etc.) for human-readable provenance. At training and inference time we collapse the schema to five edge channels actually consumed by the GNN: has_entity (event \(\to\) entity), belongs_to (entity \(\to\) event, the reverse direction added at session-graph build time), forward (\(t \to t{+}1\), past-informs-future) and backward (\(t{+}1 \to t\), future-informs-past) temporal edges between consecutive event nodes, and self loops. Training uses both temporal directions so each event node aggregates richer session context; at test time we drop the backward temporal edges so inference is strictly causal: the trigger and routing readouts at event \(t\) depend only on events \(t' \le t\). Per-session graphs are capped at \(64\) events; each entity node carries a \(\Delta t\) of \(0\) while each event node carries the seconds since the previous event in that session.

Anchor Routing Labels↩︎

Each session in the released ProactiveAgent training set [1] comes with a (history, current_observation, proposed_task) triple where proposed_task is either a non-null task suggestion or null. We derive supervision as follows.

Trigger label on the event node: \(1\) iff the assistant proposed a non-null task at this event, else \(0\). The benchmark also marks acceptance in-line via the substring “Is Accepted: True”, which forces label \(1\) on the corresponding event even when the assistant turn is null at that turn.

Routing label on entity nodes. Routing supervision is derived by applying the same deterministic extractor to the proposed-task text and matching the extracted surface forms back to entities in the event graph. This keeps the routing target inside the activity graph: the proposed-task string supplies the supervision signal, while the model input remains the event graph and its entity nodes. Events without a matched graph entity are not used for routing supervision; the trigger head still trains on their event labels.

The comparison uses the same released training-supervision source across the learned trigger rows and the proactive-finetuning baselines, while evaluation inputs are the held-out event histories.

Train/Test Isolation↩︎

ProactiveAgent’s released split is disjoint at the session level: the test events come from held-out sessions that are not present in the training set. Two classes of entities recur across the split. Category-level entities are properties of the platform rather than of any session: applications like VSCode, Outlook, and Bing, file-extension type nodes like file_ext:py or file_topic:writing, and similarly fixed surface labels. These appear in essentially every coding or writing session, and their recurrence across the split is expected. Session-specific entities appear only in one session and not in any other: the specific file names the user is editing (aw_tools.py, brandResearch.md), the specific search queries typed, the per-project artefacts, and the proposed task strings. These do not transfer across the train/test split by construction, and they are what the routing head must actually identify on a previously unseen session. We additionally verified by event-text matching that there is zero direct overlap of event-text strings between our training set and the test events.

8.2 Instantiation: Mobile Usage Logger (FingerTip)↩︎

On mobile, the platform exposes its event stream as already-structured categorical tuples (app, intent class, scenario, hour), so the schema-mapping is direct: there is no rendered text to parse. The app vocabulary is the Android package-manager namespace (e.g., com.tencent.mm), surfaced through a Chinese app-name lookup table mined from the training-set target intents (so the text encoder reads 微信 rather than the opaque package id); the intent-class vocabulary is the FingerTip taxonomy of forty-two classes [3]; the scenario and hour vocabularies come directly from FingerTip’s release. The per-user demographic context FingerTip exposes (gender, age, occupation, location, device) is added as an explicit user_profile node. Each test or training episode produces one heterogeneous typed graph with \(7\) node types and \(9\) edge relations.

Node Types↩︎

(i) query, exactly one per graph, with text concatenating “时间 {time} 场景 {scenario} 用户画像 {profile_text}” (literally “time / scenario / user-profile” wrapped around the episode’s metadata); this is the prediction target node and never contains the target intent or app. (ii) history_event, one per past intent in the last \(20\) episodes from total.csv, with text concatenating “时间 {t} 场景 {s} App {app} 类 {intent_class} 意图 {intent_text}” (“time / scenario / app / class / intent” for that past episode). (iii) app, one per unique app in history, deduplicated within the graph; the surface text uses a Chinese app-name table mined offline from training-set target intents (for example, com.tencent.mm \(\to\) 微信 (WeChat); com.huawei.deskclock \(\to\) 闹钟 (Clock)), so the text encoder sees a natural-language app name rather than the opaque package string. (iv) intent_class, one per unique intent class in history (the FingerTip taxonomy has \(42\) classes such as 闹钟/计时 (alarm/timer), 视频 (video), 商品搜索与购物 (shopping)). (v) scenario, one per unique scenario string (location categories such as 住所 (home), 办公场所 (workplace), 交通 (transit)). (vi) hour, one per unique hour-of-day, bucketed into five Chinese-readable bands: 凌晨 (late-night) \(0\)–\(5\), 早上 (morning) \(6\)–\(11\), 中午 (noon) \(12\)–\(13\), 下午 (afternoon) \(14\)–\(17\), 晚上 (evening) \(18\)–\(23\). (vii) user_profile, exactly one per graph, text drawn directly from user_profile.csv (gender, age, occupation, location, device).

Edge Relations↩︎

Five has_* relations connect the query and history-event nodes to their entities: has_app, has_intent, has_scenario, has_hour, has_profile. The query node is connected to the user-profile node, the current scenario node, and the current hour node, but not to any app or intent_class node, since those are the prediction target. Each history-event node is connected to its app, intent class, scenario, and hour. Forward (\(t \to t{+}1\)) and backward (\(t{+}1 \to t\)) temporal edges link consecutive history events in chronological order, plus a tail link from the most recent history event to the query node so information from the latest event reaches the query at GNN depth \(1\). Every node carries a self self-loop with its own learned relation embedding. All inter-node edges are added in both directions. (FingerTip is single-intent prediction per episode and there is no later event to leak from, so the bidirectional temporal chain is used unchanged at inference.)

Heads and Training↩︎

The query node’s final representation drives the prediction. Three heads sit on it: a \(462\)-class app classifier, a \(42\)-class intent-class classifier, and a \(1024\)-d semantic-alignment head whose target is the BGE embedding of the target intent text; only the semantic head is consumed at inference, the two classifier heads exist as auxiliary multi-task losses that regularise the representation. The text encoder is BAAI/bge-large-zh-v1.5 (\(1024\)d) for Chinese text. The model is trained for \(40\) epochs on FingerTip’s \(16{,}003\)-episode training split with AdamW (learning rate \(3 \times 10^{-4}\), weight decay \(10^{-4}\)), batch \(32\) graphs. Total trainable parameters are \(\approx 2.5\) M. The desktop trigger head and per-entity routing head are not used here, since the FingerTip protocol asks for a single intent prediction per episode rather than a per-event fire-or-skip decision.

9 Downstream Language Agent: Design Details↩︎

9.0.0.1 Shared design.

A frozen instruction-tuned chat model receives TGL’s structured output together with a small bundle of session context, is shown a handful of hand-written few-shot demonstrations, and emits a JSON object whose proposed-task string is the proactive suggestion. Decoding is greedy (temperature \(0\)) with structured-output JSON enforcement and a small parsing fallback chain. None of this depends on which benchmark we run on or which downstream backbone we plug in: the same design serves all \(14\) downstream backbones in Table 1.

9.0.0.2 Per-platform instantiation.

What the agent receives reflects what each platform’s data pipeline emits. ProactiveAgent’s desktop pipeline emits rendered event text (recovered to entities via Appendix 8); the agent reads the rolling session history, the current observation, and the routing list. FingerTip’s mobile pipeline emits structured episode tuples plus JPEG screenshots; the agent reads the user-profile, scenario, hour, last-\(20\) history, retrieval candidates, and screenshots. The input shape on each platform is the platform’s released data shape, not a per-dataset choice.

9.0.0.3 Per-benchmark protocol conformance.

The output format follows each benchmark’s released specification. ProactiveAgent prescribes an English task string; FingerTip prescribes a short Chinese intent imperative of \(8\)–\(25\) characters. The output schema, length cap, decoding cap, and wrapper-rejection rules in the two subsections below are dictated directly by these prescriptions: benchmark-conformance work common to any agent evaluated under the released protocols. The two subsections below give the per-benchmark instantiation.

9.1 Instantiation: Desktop Agent (ProactiveAgent)↩︎

The desktop agent instantiates the shared design on ProactiveAgent’s released data shape. The protocol-conformance layer on this benchmark is light: only an output-length ceiling and a banned-keyword set. The per-event input bundles the rolling session history, the current observation, and the routing list; the agent itself is interchangeable with any other consumer of the structured routing output (a planner, a tool router, or a UI component) and we report its design here for reproducibility.

Inputs and Output Schema↩︎

For each event the agent receives three inputs: (i) the rolling session history, the last five raw event texts in the current session (appended at every event regardless of whether the agent fires, so non-fire events still contribute context to subsequent fire events); (ii) the current observation, the event text at step \(t\) describing what the user is doing right now; and (iii) the routed nodes, the highest-scoring entities from the TGL routing head with their scores, e.g.[{"node": "email_filter.py", "score": 0.78}, ...], sorted by descending \(s_{\mathrm{rout}}\).

The routing nodes are the primary structural signal: without them, the agent would have to infer the relevant artifact purely from the event text and would default to the active application or session topic. The agent is constrained to emit a JSON object with a single proposed-task string plus five auxiliary bookkeeping fields (session theme, purpose, thoughts, response, intent); only the proposed-task string is consumed downstream by the F1 pipeline; the rest are logged for analysis. The proposed task is allowed to be empty (treated as no-fire downstream); the bookkeeping fields require non-empty strings. The schema is enforced through structured-output decoding when the served backend supports it.

System Prompt↩︎

In every prompt and prompt example shown in this appendix and in Appendix 12, the token “TGN” is used as the in-prompt nickname for our TGL network. The system prompt has three components. (i) Role description: the model is told it is a proactive assistant that receives the rolling session history, the current observation, and a routing list of graph-identified topics most relevant to the current activity. (ii) Routing-anchor task-formulation rules, four directive lines: treat the first entry of the routing list as the routing anchor and use its surface text as the artifact the user is working with; quote the anchor text inline using single quotes (e.g.’email_filter.py’), preserving any extension or domain marker verbatim; lead with what the user receives (a summary, an extract, a draft, a comparison, a checklist, a function added to the anchor) rather than mirroring the user’s last verb back as the task verb; and never invent an artifact that is not in the routing list. The task is constrained to a single sentence of at most \(150\) characters ending in a single period (a conservative ceiling above the released-demonstration length distribution), and is not allowed to mention the words “TGN”, “routing”, “graph”, or “scores”. (iii) Three hand-written few-shot examples, each a JSON input bundling the session history, current observation, and routing list, paired with a JSON output. The examples cover a coding session with a quoted file anchor, a research session whose routing is intentionally empty (teaching the model that empty routing means lean harder on session context, not stay silent), and a writing session with a markdown file anchor.

The downstream agent prompt is held fixed across all backbones in Table 1 and across all ablation cells in Table 4. The only thing that changes between ablation cells is what is fed in the routing list (full TGL routing, empty, broad-random, or routing-only without the trigger) and whether the trigger is forced on at every event. The banned-keyword list (TGN, routing, graph, scores) rejects internal mechanism vocabulary that should not surface in a user-facing task.

Decoding↩︎

We use temperature \(0\) and a \(1{,}536\)-token output cap (generous over the protocol’s target-string length distribution). JSON output is enforced through structured-output schemas where the backend supports it, and through a strict-format system instruction otherwise; on parsing failure the driver retries first with json_object mode and then with no constraint, with a regex-based {...} fallback that extracts the first JSON object from the response body. If all paths fail, the raw response is forwarded to the benchmark judge unchanged. No model-specific prompt rewriting is performed across backbones; the same prompt content is sent to every backend.

9.2 Instantiation: Mobile Agent (FingerTip)↩︎

The mobile agent instantiates the same shared design on FingerTip’s released data shape. The protocol-conformance layer is heavier here: the protocol prescribes Chinese intent imperatives of \(8\)–\(25\) characters scored under an edit-cosine plus binary judge, so the output schema, length cap, and wrapper-rejection rules are dictated by that protocol. Three FingerTip protocol properties shape the agent’s input. First, the protocol’s scoring rule rewards short imperatives, so the schema is a Chinese-only imperative of \(8\) to \(25\) characters; the desktop variant’s English “response” / “intent” scaffolding is dropped. Second, FingerTip releases episode screenshots, so the agent is multimodal (Qwen2.5-VL-7B-Instruct) and the user payload is augmented with up to \(t\) JPEG screenshots. Third, FingerTip is single-intent prediction per episode (no trigger to apply at inference), so TGL’s output is consumed as a retrieval signal: the candidates exposed to the downstream agent are concrete training-set intent strings retrieved by the semantic head, paired with a confidence summary and a same-app coherence filter.

Candidate Retrieval↩︎

At test time we run TGL once per episode and read off the query node’s \(1024\)-d \(L_2\)-normalised semantic vector. Train semantics are pre-computed once over all \(16{,}003\) training episodes. We compute the \(1{,}000 \times 16{,}003\) cosine matrix and take the top six training neighbours per test episode by raw semantic similarity, a count large enough that the downstream same-app filter still has candidates to drop. Duplicate target-intent strings are removed; no re-ranking by the auxiliary classifier heads, no handcrafted context bonuses, and no truncation past raw semantic rank.

Same-App Coherence Filter↩︎

The retrieved candidates then pass through a same-app filter that keeps only those whose app matches the top-\(1\) candidate’s app. The motivation: the top-\(2\) by raw cosine sometimes belong to different apps (e.g.top-\(1\) is a WeChat episode, top-\(2\) is Meituan), and surfacing app-disagreeing candidates to the downstream agent tends to confuse it. The filter outputs at most two same-app candidates and a confidence label of high, medium, or low: high when the top-\(1\) cosine is \(\geq 0.78\) and at least two same-app candidates survive, low when the top-\(1\) cosine falls below \(0\), medium otherwise. The \(0.78\) threshold is fixed before evaluation and is not tuned on the test set. The low bucket is part of the fixed routing-confidence rule; it is not triggered on the FingerTip test set. On the FingerTip test set the resulting distribution is \(56\%\) high, \(44\%\) medium, \(0\%\) low.

Prompt and Decoding↩︎

The user payload bundles the user profile, time, scenario, the last \(20\) previous intents, the routing-confidence label, and the routed candidates (each carrying an app, an intent class, and an intent text). At task levels \(t \geq 1\) the first \(t\) episode screenshots are loaded as JPEG bytes, base64-encoded, and prepended to the user message as image content blocks. The system prompt (in Chinese, since the target intent strings are Chinese) instructs the model to read the screenshots first, then the previous intents, then the candidates as a soft prior whose trust is gated by the routing-confidence label: high-confidence candidates may be copied as-is; medium-confidence candidates are advisory but the predicted app must agree with the screenshots and history; low-confidence candidates are ignored entirely. The prompt requires a JSON output containing the predicted app, the predicted intent class, and a proposed-task string of \(8\) to \(25\) Chinese characters containing the predicted app name, plus session-theme and thoughts bookkeeping fields. Four hand-written few-shot demonstrations precede the test payload: three demonstrate the high-confidence case where the model copies a same-app candidate (an alarm-setting episode anchored on 闹钟 (Clock), a video-watching episode anchored on 哔哩哔哩 (Bilibili), a food-delivery episode anchored on 美团 (Meituan)), and one demonstrates a low-confidence override case where the candidates point to apps not in recent history and the model is expected to ignore the candidates and synthesise an answer from history alone (the override demonstration is anchored on 支付宝 (Alipay) drawn from the user’s repeated morning routine). The override demonstration specifies the low-confidence branch of the trust-gating rule.

Decoding uses temperature \(0\) and a \(320\)-token output cap (generous over the protocol’s \(8\)–\(25\)-char target distribution). The output JSON is parsed with a small fallback chain that strips Markdown code fences, regex-extracts the first {...} block, and rejects responses where the proposed task is empty, exceeds \(60\) characters (a generous upper cutoff above the protocol’s target distribution before we fall back to the top-\(1\) candidate), starts with a banned wrapper prefix (用户 / 时间： / 地点： / 意图：, generic “user / time: / location: / intent:” headers that the protocol scores as wrapper noise instead of an imperative), or matches a wrapper pattern of the form “用户 …意图是 …” (“user …intent is …”). On any parse failure the prediction falls back to the top-\(1\) candidate’s target intent text, so every test episode receives a parseable prediction. In practice \(\geq 99\%\) of outputs parse cleanly at temperature \(0\). The wrapper-rejection rules target two general LLM input/output hygiene failures. (i) Input-payload field-label echo: the rejected prefixes 用户 / 时间： / 地点： / 意图： are exactly the labels we feed into the user payload, and any output beginning with these labels is structurally an echo of the payload header instead of a task imperative. (ii) Meta-description: patterns of the form “用户 …意图是 …” ascribe an intent in the third person, while the protocol prescribes the imperative form. Both rules are general format hygiene applied at parse time.

10 Additional Experiments↩︎

10.1 Anatomy Ablation: Full Numbers↩︎

4pt

Table 4 reports the absolute F1 numbers behind Figure 3: per-backbone F1 under each control (No TGL, No trigger, Rand. routes) alongside its \(\Delta\) vs Ours, plus Ours itself. The figure visualises the \(\Delta\) columns; this table reports the absolute F1 columns with per-cell mean\({\pm}\)std over \(n{=}3\) runs.

10.1.0.1 Broad-random sampling pool.

For the Rand.routes control, we replace the TGL routing list with random in-distribution entities at the deployed cardinality, sampled uniformly from the training-set entity vocabulary (\(354\) distinct surface labels covering the same node types as production routing: file, app, query, URL, type nodes), formatted as {"node": label, "score": 0.5} so the payload shape matches production. The sampling is event-deterministic, so re-runs are reproducible.

10.2 Trigger-Architecture Baselines↩︎

Implementation details for the eight non-graph trigger baselines compared against TGL in Section 5.4 (Table 2). All baselines share the same training supervision (anchor-event labels) as TGL, and the routing list, prompt, and downstream agents are held fixed at evaluation, so each row in Table 2 isolates the trigger.

10.2.0.1 Rule.

A zero-parameter dictionary that maps each event’s verb to its empirical fire rate observed in the training-set anchor labels. At inference the trigger probability is the lookup value; the trigger threshold sweeps that value.

10.2.0.2 Tabular (LR, HGB).

Logistic regression and HistGradientBoosting (sklearn defaults) fitted on a \(65\)-dimensional hand-crafted feature vector per event: verb identity (one-hot), foreground-application identity (one-hot), per-entity-type counts (file, app, query, URL, type nodes touched in the current event), and time-since-previous-event in the session.

10.2.0.3 Textual (BGE-frozen + MLP, BGE-FT + MLP).

A two-layer MLP head sits on the [CLS] representation of the last five session events, joined by [SEP] and encoded by a BAAI/bge-base-en-v1.5 encoder. The frozen variant freezes the encoder and trains only the MLP head (\(0.2\) M parameters), isolating the encoder’s pre-trained representation; the FT variant fine-tunes the encoder end-to-end (\(109\) M trainable). Both share TGL’s training supervision and positive-class weighting.

10.2.0.4 LLM-as-trigger (Qwen3-0.6B, Qwen3-8B).

A chat-tuned LLM is asked one binary yes/no question per event with the rolling session history and current observation as user payload, and the trigger probability is read off the next-token logit as \(P(\mathrm{yes})/(P(\mathrm{yes})+P(\mathrm{no}))\) summed over case-variant token IDs (\(\{\)yes, Yes, YES\(\}\) vs.\(\{\)no, No, NO\(\}\)). Qwen3 chat models default to reasoning mode and emit a <think> token first (probability \({\approx}1\) on every event without bypass); we append the empty thinking block <think>\(\backslash\)n\(\backslash\)n</think>\(\backslash\)n\(\backslash\)n after the chat-template prefix to expose the answer logit. LoRA fine-tuning uses \(r{=}16\), \(\alpha{=}32\), dropout \(0\), target modules \(\{q,k,v,o\}\)_proj, \(2\) epochs at lr \(10^{-4}\) with warmup ratio \(0.03\) in BF16 at effective batch \(32\). To prevent the model from learning “always say yes” under the dominant-positive label distribution, we balance the LLM-trigger training data \(1{:}1\) between fire and no-fire events. Loss is cross-entropy on the assistant’s first token only. At inference we use the merged-LoRA HuggingFace transformers eager mode (peft.merge_and_unload(), batch \(1\), torch.cuda.synchronize() before/after each forward) for the latency measurement; this is the deployment-realistic regime since merging the adapter recovers base-model latency to within \(\pm 3\) ms.

10.2.0.5 ProactiveAgent full-SFT replication (Proactive-Qwen3-0.6B).

A full-SFT (no LoRA) replication of the ProactiveAgent release recipe at the \(0.6\) B scale: Qwen/Qwen3-0.6B (\(596\) M parameters, every weight trainable, embeddings included), \(7{,}245\) multi-turn conversations from the paper’s released agent_trainset.json, AdamW at lr \(10^{-5}\) with cosine schedule and warmup ratio \(0.1\), effective batch \(32\) (per-device \(1 \times\) grad-accum \(32\), matching the paper’s \(8\) GPU \(\times\) per-device \(1 \times\) grad-accum \(4\)), \(10\) epochs, sequence length \(4{,}096\), BF16, max grad norm \(1.0\), assistant-tokens-only cross-entropy loss. Wall-clock \(520\) min on a single NVIDIA RTX A6000 (\(48\) GB). At inference the per-event protocol matches the released-checkpoint protocol of [1]: the system prompt is the paper’s Role/Task/Format/Rules block, the per-event user payload is the rolling session history plus the current observation as a JSON {Instructions, Observation} object, and the trigger is derived from the generated proposed_task field (non-empty \(\Rightarrow\) prob \(1.0\), empty \(\Rightarrow\) \(0.0\)), averaged over \(3\) inference seeds. Sampling follows the Qwen3 non-thinking defaults (\(T{=}0.7\), \(\mathrm{top\_p}{=}0.8\), \(\mathrm{top\_k}{=}20\), \(\mathrm{max\_new\_tokens}{=}512\), mean \(139\) new tokens per event). This row is the only LLM trigger in Table 2 that derives its decision from a full task-text generation rather than a single-token logit read; its Latency columns therefore measure decoding cost, not a single forward pass.

10.2.0.6 Score distributions.

Figure 4 plots the per-event \(P(\mathrm{fire})\) histogram produced by each of the eight triggers on the ProactiveAgent test set. TGL’s continuous distribution centred on \([0.55, 0.85]\) contrasts with the LLM and BGE-FT collapse onto the score extremes; the same pattern explains the Trigger std column in Table 2.

10.3 Drop-In TGL Trigger on Fine-Tuned Downstream Agents↩︎

TGL’s trigger composes as a drop-in pre-filter on existing fine-tuned proactive systems. We apply it offline to the two fine-tuned baselines from [1], LLaMA-3.1-8B-Proactive and Qwen2-7B-Proactive: events whose trigger probability falls below the deployed threshold are forced to task = null; everything else (their published outputs, prompts, and decoding) is unchanged.

Figure 5 plots F1 versus the applied trigger threshold, with a dashed line at the model’s published F1 from Table 1 (\(66.25\) for LLaMA-3.1-8B-Proactive, \(66.47\) for Qwen2-7B-Proactive). At our deployed threshold, both checkpoints exceed their published F1: LLaMA-3.1-8B-Proactive reaches \(67.66\) (\(+1.4\) F1) with \(25\%\) fewer LLM calls, and Qwen2-7B-Proactive reaches \(67.38\) (\(+0.9\) F1) with \(21\%\) fewer LLM calls. Both stay above the published line across the entire \(0.4\)–\(0.6\) range.

10.4 Per-Backbone Downstream-Agent Latency↩︎

4pt

Table 5 reports the wall-clock cost of one downstream-agent call for each of the \(14\) backbones, observed end-to-end under our evaluation serving setup (open-weight on an A6000 vLLM; OpenAI / Anthropic / DeepSeek over their respective official APIs). These are not intrinsic model latencies (API queueing and load have large effects), but they are the real per-event cost a deployed system pays. Even the cheapest API observation (\(1503\) ms for GPT-5.4-mini) is \(\approx 135\times\) TGL’s trigger forward.

11 Implementation Details↩︎

This appendix lists the architectural and training details deferred from Section 4 and Section 5.

11.1 Evaluation Backbones↩︎

Fourteen instruction-following backbones serve as the downstream language agent in the panel evaluations of Table 1 and the ablations of Tables 4 and 2:

• Open-weight (5), served via vLLM on a single A6000: Qwen2-7B-Instruct, LLaMA-3.1-8B-Instruct, Gemma-3-12B-it, Qwen3-4B, Qwen3-8B.

• OpenAI (4), served via the OpenAI API: GPT-4o-mini, GPT-4o, GPT-5.4-mini, GPT-5.4.

• Anthropic (3), served via the Anthropic API: Claude-Haiku-4.5, Claude-Sonnet-4.6, Claude-Opus-4.7.

• DeepSeek (2), served via the DeepSeek API: DeepSeek-V4-Flash, DeepSeek-V4-Pro.

Two fine-tuned proactive baselines from [1] (LLaMA-3.1-8B-Proactive and Qwen2-7B-Proactive) appear for reference in Table 1 and Figure 5; both are open-weight, served via the same vLLM setup.

11.1.0.1 RM judge.

All ProactiveAgent precision, recall, accuracy, false-alarm, and F1 numbers are computed with the released reward-model judge from [1]. The originating paper trains this RM on human-annotated event–prediction judgments and reports above-\(90\) agreement with held-out human annotations in its reward-model assessment (\(91.80\) F1 and \(91.67\) accuracy).

11.1.0.2 Run aggregation.

Each cell in Tables 1 and 4 aggregates \(n{=}3\) runs: \(3\) vendor-default decoding runs for API backbones, and \(3\) recommended-sampling runs (seeds \(\{0, 1, 100\}\)) for open-weight backbones. Each trigger-architecture metric in Table 2 (AUC, Brier, ECE, Trigger std) is similarly averaged over \(3\) training seeds of the corresponding trigger model.

2pt

11.2 Node Featurisation↩︎

Every node is described by three features. (i) text_emb: a 768-dimensional sentence embedding from a frozen BAAI/bge-base-en-v1.5 encoder, applied to the node’s text (the full event surface string for event nodes; the entity surface label for entity nodes; the type string for type-augmented nodes such as "file_ext:py"); the encoder is frozen and forwarded once per minibatch with no gradients flowing into it. (ii) type_emb: a 32-dimensional learned embedding indexed by node type (event, file, file_ext, file_topic, app, query, query_lang, url, url_domain, artifact). (iii) dt_emb: a 32-dimensional learned MLP applied to \(\log(1 + s)\) where \(s\) is the seconds elapsed since the previous event in the session (\(0\) for the first event), for event nodes; zero for entity nodes.

The three are concatenated into an 832-dimensional vector and projected to the GNN hidden dimension via Linear(\(768 + 32 + 32 \to 256\)).

11.3 GNN Body↩︎

A stack of three identical relation-aware GATv2 [22] layers (implemented as torch_geometric.nn.GATv2Conv) with: (i) \(4\) attention heads, head dimension \(256/4 = 64\), concatenated; (ii) a \(32\)-dimensional edge embedding (one entry per edge relation type) fed into each layer’s attention; (iii) no self-loops added inside the GNN layer; instead we provide explicit self edges so the attention sees the same edge-type embedding for self-attention as for other relations; and (iv) a residual + LayerNorm + GELU + dropout \(0.2\) block around each layer, \(x \leftarrow \text{LN}(x + \text{Drop}(\text{GELU}(\text{GATv2}(x, e, \text{edge\_emb}))))\).

11.4 Jumping Knowledge Head↩︎

Following [23], we keep the output of the input projection plus every GNN layer: \[h_{\text{jk}} = \text{concat}(\text{layer}_0, \text{layer}_1, \text{layer}_2, \text{layer}_3),\] of dimension \(256 \times 4 = 1024\). This lets the routing head lean on shallow features (near-raw text identity for the entity node) while the trigger head can exploit the deepest layer’s full session context.

11.5 Heads↩︎

Two parallel MLPs, each Linear(\(1024 \to 256\)) \(\to\) GELU \(\to\) Dropout(\(0.2\)) \(\to\) Linear(\(256 \to 1\)): the trigger head is applied only to event nodes and outputs the trigger logit; the routing head is applied only to entity nodes and outputs the per-entity relevance logit. Total trainable parameters: \(1.16\) M.

11.6 Training↩︎

Each batch produces one trigger logit per event and one routing logit per entity. We mask both outputs to nodes that have a label and use BCE with logits: \[\begin{align} \mathcal{L}_{\text{trig}} &= \text{BCE}(\text{logit}_{\text{trig}},\, y_{\text{trig}};\, w_{+} = 0.15), \\ \mathcal{L}_{\text{rout}} &= \text{BCE}(\text{logit}_{\text{rout}},\, y_{\text{rout}}), \\ \mathcal{L} &= 0.3\,\mathcal{L}_{\text{trig}} + 1.0\,\mathcal{L}_{\text{rout}}. \end{align}\]

Notes. A label of \(-1\) flags an unlabelled node and is excluded from the loss. The trigger positive-class weight \(w_{+} = 0.15\) down-weights the dominant positive class in the training set. The relative loss weighting is fixed at \(0.3\) for triggering and \(1.0\) for routing.

Optimisation. AdamW with learning rate \(3 \times 10^{-4}\), weight decay \(10^{-4}\), gradient clipping at global norm \(1.0\). Batch: 32 sessions per gradient step (the batch is one big disjoint Batch.from_data_list). We train for exactly two epochs and use the epoch-\(2\) checkpoint; the trigger threshold and routing top-\(k\) are not tuned.

Hardware and runtime. All measurements run on a host with \(8\) NVIDIA RTX A6000 GPUs (\(48\) GB each), \(2 \times\) Intel Xeon Gold 6254 CPUs (\(72\) threads at \(3.10\) GHz), and \(251\) GiB RAM. Training and per-event latency benchmarks use a single A6000. One epoch over the training set is \(\approx 140\) s wall-time, dominated by the BGE forward; training to \(2\) epochs takes about \(5\) minutes total.

11.7 Causal Inference at Test Time↩︎

Training uses both forward and backward temporal edges so each event node aggregates richer session context. TGL inference is strictly causal: at event \(t\), message-passing aggregates only events \(t' \le t\). We drop the backward temporal edges and zero the input features and hidden-state contributions of future event nodes (\(t' > t\)) and entities introduced at \(t' > t\). The trigger and routing readouts at event \(t\) therefore depend only on the prefix of the session up to \(t\).

11.8 Inference Hyperparameters↩︎

Trigger threshold \(\tau_{\mathrm{trig}} = 0.5\) (the sigmoid decision boundary); routed-list cap \(k = 20\). These hyperparameters are held fixed across all backbones and ablation cells.

11.9 Latency Measurement Protocol↩︎

The Latency columns of Table 2 are measured under two hardware regimes. Server runs on a single NVIDIA A6000; the Rule, LR, and HGB rows run on the host’s dual Intel Xeon Gold 6254 CPUs. Local runs on an Apple M3 Pro laptop with \(36\) GB unified memory: Textual, LLM, and Graph rows execute under Metal Performance Shaders (MPS); Rule, LR, and HGB run on the M3 Pro CPU. Transformer inference uses auto padding under streaming-serve protocol (events arrive one at a time, the encoder is called per event), so there is no padding/truncation distinction across rows. Each cell is the mean of the last \(3\) of \(4\) timed forward passes; the first pass is discarded to absorb MPS compilation. No row uses compilation, quantization, or batching across events; the TGL row uses the cached streaming protocol in Section 11.10, reusing recurring entity embeddings while encoding the event text and genuinely fresh entity strings per event. The Qwen3 LoRA rows on Local report the mean across the three yes/no LoRA seeds, identical recipe to the server-side measurement. The Proactive-Qwen3-0.6B (full SFT) row is the one exception to single-forward measurement: deriving its binary trigger requires generating the full proposed-task text and then thresholding on whether the generation is non-empty, so each event triggers a full decoding pass (mean \(139\) new tokens at \(T{=}0.7\), \(\mathrm{top\_p}{=}0.8\), \(\mathrm{top\_k}{=}20\), \(\mathrm{max\_new}{=}512\)). Its latency cells therefore report full decoding cost rather than a single forward pass: \(3{,}927\) ms on server and \(11{,}966\) ms on local, averaged across \(3\) inference seeds.

11.10 Cached Streaming Serving↩︎

The latency reported in Table 2 for TGL (\(11.13\) ms server, \(13.99\) ms local) uses cached streaming serving under HuggingFace transformers + PyG inference. Recurring entity embeddings are reused, while the event text and genuinely fresh entity strings are encoded per event. This is the serving policy enabled by the graph representation, not a model-level acceleration such as compilation, quantization, or cross-event batching.

In actual deployment, TGL’s graph-shaped input admits a serving-time optimization that sequence-based triggers (Textual, LLM) cannot access: per-entity embedding caching. The BGE encoder is the dominant forward cost (\(\sim\!7\)–\(8\) of the \(11.13\) ms), and most entity nodes a typical session touches are drawn from a small recurring vocabulary (applications, file extensions, file topics, query languages, URL domains). Caching these embeddings reduces per-event BGE work to encoding only the event text plus any genuinely fresh entity strings (specific filenames, queries, URLs).

6pt

The TGL latency in Table 2 uses this cached streaming protocol: recurring entity embeddings are reused, while the event text and genuinely fresh entity strings are encoded per event. The measurement still uses batch-\(1\) streaming inference with no compilation, quantization, or cross-event batching.

12 Verbatim Prompts↩︎

This appendix reproduces the system prompts and few-shot demonstrations used by the downstream agent on each benchmark, exactly as sent to the served model. Section 9 describes the structure; this appendix gives the literal text. Each demonstration is sent as one user message followed by one assistant message, with both encoded as JSON strings.

12.1 Desktop ProactiveAgent↩︎

System prompt↩︎

You are a proactive assistant. You are shown:
1. session_history: the last few user activities in this session (oldest first)
2. current_observation: what the user is doing right now
3. tgn_routing_nodes: graph-identified topics most relevant to the current activity

Your task: infer the user's current TASK THEME from the session history,
then generate ONE short, specific, actionable suggestion grounded in that theme.

Rules:
- Use session_history to identify what the user is working on (the session theme).
- If tgn_routing_nodes contains specific items (files, topics), anchor your task
  to one of them.
- The task must be a single sentence, concrete, max 150 characters.
- Reference the session theme explicitly (e.g., the programming language,
  document topic, tool).
- Do NOT give a generic suggestion like 'open the last file' or 'check recent
  activity'.
- Do NOT mention TGN, routing, graph, or scores.

TGN-grounded task formulation:
- The first entry of tgn_routing_nodes is the routing anchor: the entity the
  graph identified as the user's current focus. Build the task around delivering
  something concrete about that anchor; quote its surface text inline
  (e.g. 'email_filter.py') and preserve any file extension or domain marker
  verbatim.
- Lead with what the user RECEIVES, not what they should DO next. Because the
  routing anchor has already located the entity, do not mirror the user's last
  observed verb (open / search / browse) back as the task. Instead state the
  deliverable: a summary, an extract, a draft, a comparison, a checklist, a
  function added to the anchor.
- End the task with a single period. No question mark, no exclamation, no
  trailing offer or hedge: the routing score already encodes confidence, the
  surface text should match.

Output ONLY JSON: {"task":"string","session_theme":"string","purpose":"string",
"thoughts":"string","response":"string","intent":"string"}.

Demonstration 1: Python email validation (code session, file anchor)↩︎

User payload:

{
  "session_history": [
    "The user navigated to the DeepSeek chat application.",
    "The user typed 'dianz youjian dizhi jiaoyan4 python' in DeepSeek chat.",
    "The user typed 'emiacaps_lockemailshift_filter.py' and opened email_filter.py."
  ],
  "current_observation": "The user moved the mouse within the 'email_filter.py' file in Visual Studio Code.",
  "tgn_routing_nodes": [
    {"node": "email_filter.py", "score": 0.46},
    {"node": "VSCode",          "score": 0.37}
  ]
}

Assistant output:

{
  "session_theme": "Python email address validation",
  "task": "Add a regex-based email validation function to email_filter.py.",
  "purpose": "Session shows user is building email validation in Python -- a concrete next step.",
  "thoughts": "Session history + routing node email_filter.py = clear scope. The DeepSeek query reveals the goal.",
  "response": "I can write an email validation function for your current Python file.",
  "intent": "code_generation_email_validation"
}

This demonstration teaches the model to anchor on a specific file from the routing list (matching the rule that the surface text be quoted inline), to infer the session theme from a noisy mixed-language search query, and to produce a deliverable-style task (“Add …function”) rather than mirroring the user’s last verb.

Demonstration 2: Go versus Python research (empty routing)↩︎

User payload:

{
  "session_history": [
    "The user typed 'go language' in DeepSeek chat.",
    "The user clicked on Google search results for 'go language'.",
    "The user opened 'All releases - The Go Programming Language' on the Go website.",
    "The user browsed 'Go language | runoob' and the Go official documentation."
  ],
  "current_observation": "The user navigated Google search results for 'comparison of go and python' and opened a Tencent Cloud article.",
  "tgn_routing_nodes": []
}

Assistant output:

{
  "session_theme": "Learning Go programming language (comparison with Python)",
  "task": "Summarize the key differences between Go and Python for the user's use case based on the Tencent Cloud article.",
  "purpose": "User is actively evaluating Go vs Python -- a targeted comparison summary accelerates the decision.",
  "thoughts": "Session clearly shows Go learning; current page is a direct Go vs Python comparison article.",
  "response": "I can summarize the Go vs Python tradeoffs from the article you're reading.",
  "intent": "research_summary_go_vs_python"
}

This demonstration deliberately shows an empty tgn_routing_nodes list. The assistant still produces a non-null task by anchoring on the session-history theme, specifying the prompt behavior for events where routing is empty but session context remains actionable.

Demonstration 3: Public-transit research writing (markdown anchor)↩︎

User payload:

{
  "session_history": [
    "The user edited a markdown file 'search.md' in project 'PublicTrafficImprovement' in VSCode.",
    "The user browsed Google search results about improving public transportation.",
    "The user returned to Google to search 'How can digital payment enhance traffic'."
  ],
  "current_observation": "The user switched back to Visual Studio Code and typed insights related to digital payment systems improving public transportation.",
  "tgn_routing_nodes": [
    {"node": "search.md", "score": 0.44},
    {"node": "VSCode",    "score": 0.37}
  ]
}

Assistant output:

{
  "session_theme": "Research: digital payments improving public transportation",
  "task": "Draft a concise paragraph on how digital payments reduce boarding time in public transit for search.md.",
  "purpose": "User is compiling research insights into search.md -- a concrete writing suggestion advances the document.",
  "thoughts": "Session theme (public transit + digital payments) + routing to search.md = clear writing task.",
  "response": "I can draft that paragraph about digital payment impact on public transit for your document.",
  "intent": "research_writing_public_transit"
}

This demonstration covers the writing half of the testset and exhibits the same pattern as Demonstration 1 (anchor on a specific file, deliverable-style task) in a non-code domain.

No-TGL Prompt Variant (Style Only)↩︎

The No TGL ablation in Table 4 replaces the production system prompt with the single-page variant below. It retains the style directives (single sentence, \(150\)-character cap, lead-with-deliverable, do-not-invent-artifacts) and removes every reference to tgn_routing_nodes, the four-line routing-anchor block, and all three few-shot demonstrations. The trigger is disabled; the agent self-gates by emitting task = "".

You are a proactive assistant. You are shown:
1. session_history: the last few user activities in this session, oldest first.
2. current_observation: what the user is doing right now.

Your task: infer the user's current TASK THEME from the session history,
then generate ONE short, specific, actionable suggestion grounded in that theme.

Rules:
- Use session_history to identify what the user is working on.
- The task must be a single sentence, concrete, max 150 characters.
- Reference the session theme explicitly.
- Do NOT give a generic suggestion like "open the last file" or "check recent activity".
- Do NOT invent files, URLs, pages, or topics not present in the input.
- Lead with what the user RECEIVES, not what they should DO next.
- End non-empty tasks with a single period.
- If the current moment does not support a useful, specific suggestion, set task to "".

Output ONLY JSON: {"task":"string","session_theme":"string","purpose":"string",
"thoughts":"string","response":"string","intent":"string"}.

12.2 FingerTip Mobile Proactive Agent↩︎

The FingerTip prompt is in Chinese because the target intent strings are Chinese; we reproduce it verbatim. Chinese characters are typeset with the xeCJK package.

System prompt↩︎

你是一个安卓助手，根据用户画像、时间、场景、最近意图、（如果有）当前屏幕截图，
以及 TGN 时序图谱召回的候选意图，预测用户当前一句简短的中文意图。

输入字段说明：
- previous_intents: 该用户最近的意图（按时间升序）。是用户长期偏好的强信号。
- screenshot: 当前打开的 App 与界面，是**最强证据**（如果有）。
- tgn_candidates: 图谱召回的候选意图（按相关度排序）。**软先验，可信度由
  tgn_confidence 给出**：
  - high: 候选可信，强烈建议参考；可能直接 copy 一句作为 task。
  - medium: 候选有用但需谨慎；选 App 应跟截图/历史一致。
  - low: 候选不可信，**忽略 tgn_candidates**，只看截图与历史。
- 候选 App 跟截图/历史所示 App 不一致时，**永远以截图/历史为准**，可以独立生成意图。

输出严格 JSON：
{
  "session_theme": "20 字内总结用户最近在做什么 (例: '早晨重复设闹钟'、'晚上看哔哩哔哩')",
  "thoughts": "20 字内说明你怎么定下 task 的",
  "app": "用户当前使用的 App 中文名（要跟截图/历史/候选三者最强证据一致）",
  "intent_class": "用户行为大类（如 闹钟/计时、视频、商品搜索与购物、信息查询...）",
  "task": "一句 8-25 个汉字的中文祈使句，以动词或 App 名开头，包含 app 字段中的
          App 名，禁止 用户/时间：/地点：/意图： 前缀，禁止日期/星期/画像信息"
}

只输出 JSON，不要 markdown 包裹、不要前后说明。

Demonstration 1: alarm clock (high confidence, copy top-1)↩︎

User payload:

{
  "user_profile": "女, 28, 上班族, 北京市, 已婚, 华为",
  "time": "20250410_073000",
  "scenario": "住所",
  "previous_intents": [
    "时间：20250408_073000，地点：住所，意图：在时钟开启早晨7点45分的闹钟",
    "时间：20250409_073000，地点：住所，意图：在时钟开启早晨7点45分的闹钟"
  ],
  "tgn_confidence": "high",
  "tgn_candidates": [
    {"app": "时钟", "intent_class": "闹钟/计时", "intent": "在时钟设置明天7点的闹钟"},
    {"app": "时钟", "intent_class": "闹钟/计时", "intent": "在时钟开启排序第一的闹钟"}
  ]
}

Assistant output:

{
  "session_theme": "早晨重复设7点45闹钟",
  "thoughts": "TGN高+历史明确，沿用昨天的设法",
  "app": "时钟",
  "intent_class": "闹钟/计时",
  "task": "在时钟开启早晨7点45的闹钟"
}

What this episode looks like. The user is a 28-year-old female office worker living in Beijing, married, on a Huawei phone. The current episode is at 7:30 AM in her home (scenario = 住所). Her two most recent intents, both at home and at the same morning hour on the prior two days, opened the Clock app (时钟) and set a 7:45 AM alarm. TGL’s retrieval returns two Clock-app candidates with tgn_confidence = "high"; the top-1 candidate is a slightly off-target “set tomorrow’s 7:00 alarm” but on the correct app, and the top-2 is “turn on the first alarm in Clock”.

What the assistant does. The assistant emits a session theme of “morning routine of repeatedly setting a 7:45 alarm”, a thoughts string “TGL high + history clear, follow yesterday’s setup”, and a task that is essentially yesterday’s behaviour: “turn on the morning 7:45 alarm in Clock”. The lesson is: when tgn_confidence is high and the recent history defines a clear recurring pattern, the candidate’s app should be trusted but the specific time/imperative can be drawn from history rather than copied verbatim.

Demonstration 2: video watching (high confidence, copy top-1)↩︎

User payload:

{
  "user_profile": "男, 22, 学生, 上海市, 未婚, 小米",
  "time": "20250415_220000",
  "scenario": "住所",
  "previous_intents": [
    "时间：20250414_213000，地点：住所，意图：打开哔哩哔哩查看首页推荐第一个视频",
    "时间：20250414_220000，地点：住所，意图：在哔哩哔哩搜索英雄联盟比赛集锦"
  ],
  "tgn_confidence": "high",
  "tgn_candidates": [
    {"app": "哔哩哔哩", "intent_class": "视频", "intent": "在哔哩哔哩观看LPL最新比赛回放"},
    {"app": "哔哩哔哩", "intent_class": "视频", "intent": "打开哔哩哔哩观看推荐视频第一条"}
  ]
}

Assistant output:

{
  "session_theme": "晚上看哔哩哔哩LOL比赛",
  "thoughts": "TGN高+历史一致，copy LPL候选",
  "app": "哔哩哔哩",
  "intent_class": "视频",
  "task": "在哔哩哔哩观看LPL最新比赛回放"
}

What this episode looks like. The user is a 22-year-old male student in Shanghai, single, on a Xiaomi phone. The current episode is at 10:00 PM at home. His two most recent intents (the previous evening, at home) opened Bilibili (哔哩哔哩): first to view the top home-feed recommendation, then to search for League of Legends match highlights. TGL’s retrieval returns two Bilibili-app candidates with tgn_confidence = "high": the top-1 is the more specific “watch the latest LPL match replay in Bilibili”, and the top-2 is the more generic “open Bilibili and continue watching the first recommended”.

What the assistant does. The assistant emits a session theme of “evening Bilibili LoL viewing” and copies the more specific top-1 candidate verbatim as the task. The lesson is: when both candidates agree on the same app and the recent history is consistent with the more specific candidate’s content, copy the top-1 rather than generalise. This is the canonical “tgn_confidence = high, top-1 looks right, take it” case.

Demonstration 3: takeout ordering (high confidence, session arc)↩︎

User payload:

{
  "user_profile": "女, 32, 上班族, 杭州市, 已婚, 华为",
  "time": "20250420_125500",
  "scenario": "办公场所",
  "previous_intents": [
    "时间：20250420_120000，地点：办公场所，意图：打开美团搜索附近午餐外卖",
    "时间：20250420_121500，地点：办公场所，意图：在美团领取新人优惠券"
  ],
  "tgn_confidence": "high",
  "tgn_candidates": [
    {"app": "美团", "intent_class": "点餐/外卖", "intent": "在美团下单附近的午餐外卖"},
    {"app": "美团", "intent_class": "点餐/外卖", "intent": "在美团查看已下单外卖的配送状态"}
  ]
}

Assistant output:

{
  "session_theme": "中午办公室点外卖",
  "thoughts": "TGN高+刚领券，下一步下单",
  "app": "美团",
  "intent_class": "点餐/外卖",
  "task": "在美团下单附近的午餐外卖"
}

What this episode looks like. The user is a 32-year-old female office worker in Hangzhou, married, on a Huawei phone. The current episode is at 12:55 PM at her workplace (scenario = 办公场所). Her two most recent intents are from earlier the same day, both on Meituan (美团): first “search nearby lunch takeout”, then “claim a new-user coupon”. TGL’s retrieval returns two Meituan-app candidates with tgn_confidence = "high": the top-1 is “place an order for a nearby lunch takeout in Meituan”, and the top-2 is “check delivery status of an already-placed order”.

What the assistant does. The assistant emits a session theme of “ordering office lunch takeout”, a thoughts string “TGL high + just claimed coupon, next step is order”, and the task “place an order for a nearby lunch takeout in Meituan”. The lesson is more subtle than Demonstration 2: when the recent history exhibits a session arc (browse \(\to\) coupon \(\to\) order), the candidate that continues the arc is preferred over a same-app candidate whose verb points to a different next step. The model is taught to use the history’s directionality, not just its app frequency.

Demonstration 4: override (low confidence, ignore candidates)↩︎

User payload:

{
  "user_profile": "男, 26, 上班族, 深圳市, 未婚, 苹果",
  "time": "20250422_080000",
  "scenario": "住所",
  "previous_intents": [
    "时间：20250420_080000，地点：住所，意图：在支付宝蚂蚁森林收取能量",
    "时间：20250421_080000，地点：住所，意图：在支付宝蚂蚁森林收取能量"
  ],
  "tgn_confidence": "low",
  "tgn_candidates": [
    {"app": "微博", "intent_class": "信息查询", "intent": "打开微博查看热搜"},
    {"app": "天气", "intent_class": "天气", "intent": "打开天气查看明天天气"}
  ]
}

Assistant output:

{
  "session_theme": "早晨支付宝蚂蚁森林收能量",
  "thoughts": "TGN低且与历史不符，按历史延续",
  "app": "支付宝",
  "intent_class": "金融/生活服务",
  "task": "在支付宝蚂蚁森林收取能量"
}

What this episode looks like. The user is a 26-year-old male office worker in Shenzhen, single, on an iPhone. The current episode is at 8:00 AM at home. His two most recent intents, both at the same morning hour on the prior two days, are identical: “harvest green energy in Alipay’s Ant Forest” (支付宝蚂蚁森林收取能量). TGL’s retrieval returns tgn_confidence = "low" with two candidates that point to different apps not seen in the history at all: “open Weibo and view trending” and “open Weather and check tomorrow’s weather”.

What the assistant does. The assistant ignores the candidate list entirely. It emits a session theme of “morning Alipay Ant Forest energy harvest”, a thoughts string “TGL low and disagrees with history, follow history”, and a task that extrapolates directly from the recent two days: “harvest energy in Alipay Ant Forest”. The lesson is the override rule: when tgn_confidence is low and the candidates name apps that do not appear in the recent history, the language agent should disregard TGL and synthesise from history alone. Including this demonstration is what teaches the model to actually exercise the low-confidence branch of the trust-gating rule rather than always copying a candidate.

13 Case Study↩︎

We pick three test events where all three pipelines (vanilla, Ours, broad-random) fire and produce different tasks for the same observation. Each case shows the routing context the downstream agent received under Ours, and the RM verdict for every variant. The downstream agent is Qwen2-7B-Instruct under the deployed configuration; broad-random replaces the top-\(20\) routing entries with draws from the training entity pool (Section 5.3).

Case 1: coding session, romanised-Chinese query. TGL routes to [query_lang:en, ’dianz youjian dizhi jiaoyan4 python’]. Vanilla emits “Translate the message from Chinese to English.” (rejected as too vague). Ours quotes the query verbatim and ties it to the working file: “Translate ‘dianz youjian dizhi jiaoyan4 python’ into English for context in the ‘aw_tools.py’ file.” (accepted). Broad-random keeps aw_tools.py (carried over from session history) but loses the query anchor: “Extract key points from the ‘aw_tools.py’ file and compile them into a concise summary.” (rejected). The query anchor is what makes the suggestion specific enough to accept.

Case 2: writing session, multi-route file anchor. TGL’s top-ranked routing entries for this event include the file brandResearch.md along with the type markers file_topic:writing and file_ext:md that reinforce the document type, plus the host VSCode. Ours grounds on the file: “Review and refine the structure of ‘brandResearch.md’ for better readability.” (accepted). Vanilla misreads the activity as research discovery: “Search for key metrics and indicators used in measuring the sustainability of brands and provide them to the user for reference.” (rejected). Broad-random does mention the file (carried over from session context) but proposes content the user is not working on: “Add a section on audience engagement techniques to the ‘brandResearch.md’ file.” (rejected).

Case 3: sparse routing, host-app only. TGL routes to [VSCode] only and defers the artifact to session history. Ours produces a deliverable aligned with the user’s note-taking activity: “Extract and summarize Katla’s product pricing details for your Markdown notes.” (accepted). Vanilla overshoots into research/positioning: “Offer to compare the pricing of Katla’s products with similar items from other eco-friendly brands, providing insights on positioning.” (rejected). Broad-random fixates on a comparison table the RM judges too narrow: “Add a table comparing prices of Katla’s eco-friendly products in your Markdown notes.” (rejected). Even when TGL surfaces only a host app, the routing scaffold steers the downstream agent into deliverable-style output rather than generic research framing.

In all three cases the broad-random control preserves the routing cardinality and the trigger but injects entities from unrelated training sessions. The downstream agent faithfully tries to incorporate those entities, producing tasks that name files or topics the user never touched, which the RM rejects.

14 Discussion↩︎

14.0.0.1 Resource and latency implications.

Our TGL trigger runs in \(11.13\) ms per event on a single A6000, \(\sim\!4\)–\(7\times\) faster than the single-forward LLM-as-trigger configurations we tested (Qwen3-0.6B and Qwen3-8B; Table 2); the generation-based ProactiveAgent full-SFT baseline in the same table costs another two orders of magnitude on top. On the demo polling rate of [1], this is the difference between \({\sim}6.5\) GPU-hours per device-year and \({\sim}24\)–\(46\) GPU-hours per device-year. The full LLM stack remains, reserved for moments that survive the trigger.

14.0.0.2 Why GNN trigger latency scales smoothly from server to consumer hardware.

The LoRA LLM-as-trigger rows in Table 2 read the trigger off a single forward pass over the rolling session history — a transformer prefill regime whose arithmetic intensity makes it compute-bound on dense matrix multiplications and squarely on the side of the inference roofline that GPUs are built to accelerate [24]. GNN message-passing is the opposite shape: TGL’s GAT layers scatter/gather over a per-session graph capped at \(64\) events, per-kernel work is small, and the SpMM-style aggregation that dominates GNN runtime is repeatedly characterised as the binding bottleneck on GPU, with workload imbalance, irregular memory access, and kernel-launch overhead leaving GPUs substantially underutilised [25]–[27]. The empirical consequence appears in the move from server to laptop: TGL widens by only \({\sim}25\%\) (\(11.13 \to 13.99\) ms) because its sparse aggregation never had a large GPU advantage on the server to lose, whereas the LoRA LLM rows widen \({\sim}4\)–\(14\times\) (\(40.4 \to 162.3\) ms for \(0.6\) B; \(78.6 \to 1156.8\) ms for \(8\) B) because their dense-matmul prefill was substantially GPU-bound. Both architecture families still run on the laptop; the difference is how steeply the latency curve responds to losing data-center GPU bandwidth. TGL’s graceful degradation comes from being on the underutilised-GPU side of that curve to begin with, so the same checkpoint serves the trigger budget on both ends of the hardware spectrum without retraining, quantisation, or distillation.

14.0.0.3 Shared hidden state.

The conceptual implication is that the trigger and the context should share state. Once the routing scores are produced by the same backbone that also produces the trigger probability, the two decisions cannot drift apart in their evidence: the trigger fires on one head’s output, the routing list is the other head’s output, and both are read off the same hidden state in one forward pass. The random-routing control (Section 5.3) confirms this is more than aesthetic: replacing the routing list with random in-distribution entities at the same cardinality reduces F1 on every one of the \(14\) backbones, by a mean of \(8.6\) points, even when prompt and trigger are held constant. The control preserves routing-cardinality and draws from the same training entity pool, so the active variable is the event-specific entity selection.

14.0.0.4 Drop-in compatibility.

A second practical implication is that TGL‘s trigger is composable with existing fine-tuned proactive systems. Appendix 10.3 shows that applying our trigger offline to released fine-tuned downstream agents’ outputs improves their F1 while saving \(\geq 21\%\) of LLM calls (Figure 5). No retraining, no prompt change, no routing-node injection is required for this to work; the trigger is a pure pre-filter.

14.0.0.5 Personalization path.

The same decomposition gives a direct user-level adaptation path. A deployment can initialize from a generic checkpoint and update the small trigger/routing model from ordinary user feedback: accepted suggestions mark fire events and expose task text for anchor extraction, while dismissed suggestions mark over-fire events. The downstream language agent can remain fixed; user-specific state lives in the lightweight graph model that already runs on device.

15 Extended Related Work↩︎

Proactive assistance. The idea of proactive assistance predates LLM agents. Earlier work studied proactive behavior in situation-aware agents, personal assistive systems, context-aware services, plan-recognition assistants, and proactive recommenders [6]–[8], [28], [29]. More recent human-computer interaction work shifted attention to user expectations, interaction design, and human-centered conversational proactivity [9], [30]–[32]. In the LLM era, proactivity has become a concrete modeling target through proactive desktop and computer-use agents [1], open-world sensory agents [2], [33], proactive mobile and GUI agents [3], [4], proactive dialogue and evaluation benchmarks [11], [34], and broader proactive-agent formulations [5], [12], [35], [36]. Domain-specific systems for programming and cooperative gameplay further show the breadth of the space [10], [37]. We isolate the always-on trigger and keep intervention decisions timely, cheap, and behaviorally selective.

Memory, personalization, and realistic evaluation. A second body of work studies how agents accumulate user-specific knowledge and personalize behavior over time. This includes personalization for LLMs [38]–[40], retrieval-augmented or profile-based user modeling [14], [41]–[43], and memory-augmented agents that combine reasoning, acting, and experience accumulation [13], [44]–[48]. These efforts are important because proactive assistance is inseparable from user context, but most of them focus on what happens after the system has already decided to engage. A related evaluation literature studies the cost of interruption and the realism of computer-use environments. Prior human-computer interaction work shows that interruption timing, attentional utility, task resumption burden, and repeated alerts all affect user experience and productivity [15], [16], [49]–[54]. In parallel, benchmarks for web, GUI, and mobile interaction have provided increasingly realistic substrates for autonomous agents [17], [18], [55]–[59]. Together, these two lines establish the pre-invocation policy as the load-bearing step: the wake-up decision must be both personalized and interruption-aware before any downstream worker is called.

Temporal graph learning. The temporal-graph literature provides the inductive bias most aligned with the proactive-trigger setting. Continuous-time and dynamic graph models such as TGAT and TGN capture evolving interactions without collapsing them into static snapshots [19], [20]. Related work on dynamic knowledge and temporal reasoning further explored recurrent, evolutionary, and neighborhood-based graph updates [60]–[64]. More generally, graph encoders established practical ways to model relational structure and node-local message passing [21], [65]–[67]. Our TGL encoder stacks relation-aware GATv2 layers [22] with a Jumping Knowledge head [23] over the heterogeneous temporal graph; the wake-up and routing heads are two node-classification readouts off the same encoded graph.

References↩︎