1 Introduction↩︎

Text-driven human motion generation aims to synthesize human motion trajectories from natural language descriptions. Most existing models treat motions as short and self-contained clips rather than continuous and socially coherent behaviors [1], [2]. However, in realistic settings, individuals may act alone, engage with others, or transition between solo and interactive behaviors [3]–[5]. The ability to generate such dynamic long-horizon behaviors is essential for applications including animation, virtual agents, robotics, and immersive environments.

Despite recent progress in human–human interaction generation, existing methods are typically designed to produce fixed-length motion clips in a single pass [1]. This design conflicts with realistic social behavior, where interactions emerge, evolve, and dissolve without fixed temporal boundaries. Moreover, most approaches are tailored to a fixed agent configuration, modeling either solo motion or interaction using separate frameworks [3]–[5]. As a result, they struggle to support solo–social transitions in a temporally incremental setting, often causing boundary discontinuities or relational drift over long horizons.

Generating socially coherent motion in an incremental setting is challenging because the model must preserve temporal coherence for each individual while maintaining consistent inter-person relationships over extended horizons [6], [7]. Supporting streams that evolve between solo and two-person interaction further requires the model to accommodate changes in active agents and social coupling within a unified generative process.

To address these challenges, we propose ARMS (Anchor–Relational Motion Streaming), a causal autoregressive diffusion framework for temporally incremental human motion and interaction synthesis under changing one-/two-person social configurations. ARMS is built on a novel dynamics-asymmetric representation that unifies solo and interactive motions. The key is a partner-referenced relative-translation term that controls when inter-person coupling is represented. The term is inactive for solo motion and for the Anchor stream, but instantiated as the partner-relative displacement for the Relational stream during interaction. This design enables unbounded causal extension while avoiding global-coordinate drift, and preserves stable inter-person geometry when interactions emerge.

To enable streaming generation, motion sequences are first encoded into a temporally compressed latent space. Crucially, the same causal temporal autoencoder encodes all individual motions, both in solo scenes and as components of interactions, under the proposed Anchor–Relational representation. Building on a causal autoregressive diffusion backbone, we introduce mode-aware relational gating masks that realize appropriate causal relational attention in both solo and interaction modes. Dynamic history conditioning further improves long-horizon consistency by attending to variable-length past context. Our contributions are:

• We formulate temporally incremental motion synthesis under one-/two-person social configuration changes, unifying solo behavior and human–human interaction within a single streaming process that supports seamless solo–social transitions.

• We propose a dynamics-asymmetric Anchor–Relational motion representation with a partner-referenced relative-translation term, enabling configuration-agnostic learning from both solo and interaction data while using a shared causal temporal autoencoder for individual motion encoding.

• Building on an autoregressive diffusion backbone, we instantiate the denoiser as a causal relational transformer and introduce mode-aware relational gating masks with dynamic history conditioning, enabling incremental generation while preserving temporal coherence and stable inter-person geometry.

2 Related Work↩︎

Human motion generation has been widely studied under various conditioning modalities, including action labels [8], [9], audio signals [10]–[14], scene context [3], [5], [15]–[17], and natural language [18]–[24]. We focus on text conditioning for its flexibility and accessibility [3], [22], [25].

Human–Human Interaction Generation. Early motion generation primarily focused on single-person dynamics without social interactions. Building upon a pretrained single-person diffusion model, ComMDM [26] extends it to two-person generation by learning a lightweight communication module, while RIG [27] and InterGen [6] introduce cooperative diffusion denoisers with two shared-weight Transformer branches. To enhance interaction diversity, in2IN [28] and MixerMDM [29] augment interaction modeling with additional single-person motion data, while Text2Interact [30] introduces a scalable synthesis-by-composition pipeline with fine-grained word-level conditioning. TIMotion [31] explicitly structures interaction dynamics along a temporal sequence to better capture evolving inter-person dependencies. Beyond continuous diffusion, InterMask [32] formulates interactions in a discrete 2D VQ space with masked modeling, and SocialGen [33] aligns motion and language tokens to extend interaction modeling toward multi-human social scenarios.

Despite these advances, many interaction models remain optimized for fixed-length synthesis from a static prompt, and do not support incremental extension. Moreover, most interaction methods assume a fixed two-person setting and do not address solo–social mode switching in evolving streams, where agents may approach, interact, and disengage without restarting generation. This leaves open the problem of unified streaming motion synthesis that preserves both per-person temporal coherence and stable inter-person geometry across transitions.

Long-horizon Motion Generation. Recent works extend motion generation beyond isolated short clips toward long-horizon or sequential synthesis [17], [34]–[39]. Several methods perform autoregressive diffusion for real-time control, generating motion incrementally from past states, including CAMDM [40], A-MDM [41], and PRIMAL [39]. CLoSD [17] and DART [38] formulate motion as autoregressive segments, while InfiniDreamer [36] explores arbitrarily long motion synthesis via segment refinement and score distillation. MotionStreamer [37] operates in a continuous causal latent space with a temporally causal autoencoder and strictly causal masking. Most long-horizon autoregressive methods are developed for single-person motion generation [19]. Recent interaction extensions such as Interact2Ar [42] and HINT [43] extend diffusion-based interaction models to the autoregressive setting via sliding-window generation. Interact2Ar incorporates a memory mechanism to reuse past motion features, while HINT adopts hierarchical conditioning in a canonicalized latent space, which can introduce discontinuities across segments.

Overall, existing approaches either generate human–human interactions within fixed-length clips, or extend motion synthesis to long horizons primarily for single-person scenarios. Autoregressive interaction extensions typically still assume a fixed agent configuration and focus on maintaining continuity within that regime. Consequently, streaming generation under changing one-/two-person social configurations remains largely unexplored.

Interactive Motion Representation. Motion representation is critical for both motion quality and long-horizon stability in human motion generation. Most single-person motion generation methods adopt canonicalized representations derived from HumanML3D [19], expressing motion in a root-centered coordinate system with frame-to-frame relative updates. Such incremental formulations simplify learning and naturally support temporally extensible generation [17], [37]. Several interaction models extend this paradigm by introducing initial relative transformations between agents [26], [33], but accumulated incremental updates can introduce spatial drift that disrupts inter-person alignment [6]. To avoid this issue, InterGen [6] models motion in global coordinates to explicitly preserve spatial relations between agents, which improves alignment but constrains motion generation to the coordinate distributions observed during training and limits their compatibility with open-ended or temporally extensible synthesis. HINT [43] mitigates drift by generating short motion segments in a canonicalized latent space, but segment-wise canonicalization introduces discontinuities that cause jittery transitions. DuetGen [12] instead represents two-person motion as a unified long feature sequence that jointly encodes canonicalized motion and relational features, but is limited to fixed two-person scenarios and increases dimensionality.

These designs highlight a fundamental tension in interaction representation: maintaining stable inter-person spatial relations while enabling temporally extensible generation. A shared representation that supports causal encoding and autoregressive rollout, preserves interaction geometry, and enables seamless solo–social transitions within a unified streaming process remains underexplored.

3 Method↩︎

3.1 Problem Formulation↩︎

As shown in 1, we address the task of streaming motion generation under changing one-/two-person social configurations, where textual instructions arrive incrementally, with one text remaining active until replaced by a new one. Let \(\mathcal{T} = \{\tau_1, \tau_2, \dots\}\) denote the ordered instruction stream, and \(\beta(t)\) the index of the instruction active at timestep \(t\), inducing change points that partition the timeline into variable-length intervals. At each timestep \(t\), the model generates the motion states of all active agents: \[\{\mathbf{x}_t^{(i)}\}_{i=1}^{N_t},\] where \(N_t \in \{1, 2\}\) denotes the number of active agents at time \(t\). We denote the generated motion history before \(t\) as \[\mathcal{H}_{<t} = \{ \mathbf{x}_{t'}^{(i)} \mid t' < t,;i = 1,\dots,N_{t'} \},\] where the number of active agents in the history may vary over time. The objective is therefore to learn a causal generative model that produces motion incrementally, \[p\big( \{\mathbf{x}_t^{(i)}\}_{i=1}^{N_t} \mid \mathcal{H}_{<t}, \tau_{\beta(t)}, N_t \big),\] while preserving long-horizon temporal coherence within each individual and maintaining relational consistency whenever interactions are present.

3.2 Motion Representation with Asymmetric Dynamics↩︎

Streaming motion generation commonly adopts canonicalized incremental features that enable temporally extensible synthesis, but these representations can accumulate spatial drift over long horizons. In contrast, interaction models often operate in global coordinates to preserve inter-person alignment, at the cost of restricting generation to coordinate distributions observed during training. To balance these trade-offs, inspired by [6], [12], we adopt a dynamics-asymmetric representation that separates incremental temporal dynamics from relational dynamics within a shared motion state. The key is a partner-referenced relative-translation term that controls when inter-person coupling is represented: it is set to zero for solo motion (and for the Anchor stream), and instantiated as the relative displacement to the partner for the Relational stream during interaction. This design preserves temporally extensible causal rollout while maintaining stable inter-person spatial geometry whenever interactions are present.

Per-frame state definition. We adopt a coordinate system with axes defined as \(x\)-left, \(y\)-up, and \(z\)-forward. At each timestep \(t\), the motion state of an agent is defined as \[\mathbf{x}_t = \left[ \mathbf{r}_t^{yaw}, \mathbf{v}_t^{xz}, \mathbf{\Delta}_t^{xz}, \mathbf{p}_t^{local}, \dot{\mathbf{p}}_t^{local}, \mathbf{q}_t, \mathbf{c}_t \right],\] where \(\mathbf{r}_t^{yaw} = [\sin(\theta_t), \cos(\theta_t)] \in \mathbb{R}^2\) encodes the root yaw orientation to stabilize long-horizon integration and avoid angular discontinuities. The root linear velocity \(\mathbf{v}_t^{xz} \in \mathbb{R}^2\) and the relational displacement \(\mathbf{\Delta}_t^{xz} \in \mathbb{R}^2\) are defined on the world-frame \(xz\)-plane. Joint positions \(\mathbf{p}_t^{local} \in \mathbb{R}^{J\times 3}\) and velocities \(\dot{\mathbf{p}}_t^{local} \in \mathbb{R}^{J\times 3}\) are expressed in the root-local frame. Joint rotations \(\mathbf{q}_t \in \mathbb{R}^{J\times 6}\) adopt the continuous 6D rotation representation [44] for all \(J\) non-root joints. Foot-contact indicators \(\mathbf{c}_t \in \mathbb{R}^{4}\) encode binary contact states for four predefined foot joints.

Asymmetric motion dynamics. Although both agents share the same state formulation, the two motion dynamics are instantiated asymmetrically during generation. We designate one agent as the Anchor Agent (A), which models canonical temporal dynamics, and the other as the Relational Agent (R), which explicitly encodes relational alignment. For the Anchor branch, the relational displacement is fixed: \[\mathbf{\Delta}_t^{(A)} = \mathbf{0},\] which ensures that the Anchor motion is modeled independently of inter-agent spatial relations and remains compatible with solo motion generation. The global root position \(\mathbf{P}_t^{(A)}\) is obtained via integration of \(\mathbf{v}_t^{xz}\) over time. The Relational branch encodes interaction dynamics. It stores the root translation offset \(\mathbf{\Delta}_t^{(R)}\) with respect to the Anchor agent. Its global root position is reconstructed as \[\mathbf{P}_t^{(R)} = \mathbf{P}_t^{(A)} + \mathbf{\Delta}_t^{(R)}.\] Restricting global integration to a single branch prevents independent integration errors that cause drifting global trajectories and unstable inter-agent alignment. Both agents retain identical state space, which enables architectural symmetry and parameter sharing. Notably, when the relational displacement is inactive, the Anchor branch alone reduces to canonical incremental motion modeling, making the formulation naturally compatible with solo motion generation.

3.3 Causal Temporal AutoEncoder↩︎

Long-horizon autoregressive generation in high-dimensional motion space is computationally costly and prone to drift. Following the causal compression paradigm introduced in [37], we adopt a causal variational autoencoder to compress motion into a temporally causal latent space, where each latent depends only on past frames and supports incremental decoding during streaming generation. The model is a temporal variational autoencoder composed of 1D causal convolution and residual blocks in both encoder and decoder. Given a motion sequence \({\mathbf{x}_t}_{t=1}^{T}\) for a single agent, the encoder produces temporally downsampled Gaussian parameters \({\boldsymbol{\mu}_k, \boldsymbol{\sigma}_k^2}_{k=1}^{T/l}\), where \(l\) is the temporal downsampling factor. Latent variables \(\mathbf{z}_k \in \mathbb{R}^d\) are sampled via reparameterization, forming a continuous causal latent sequence \(\mathbf{z}_{1:T/l}\). For latent timestep \(k\), we denote by \(\mathbf{z}_{\le k}\) the causal latent prefix available up to that timestep. The decoder reconstructs motion from the causal latent prefix \(\mathbf{z}_{\le k}\).

For interaction modeling, the same autoencoder is applied independently to each agent branch with shared weights. The motion of each individual is encoded identically whether it appears in a solo sequence or as part of an interaction under our Anchor–Relational representation. This yields a unified latent space while preserving independent temporal encoding for different agents.

3.4 Causal Relational Diffusion↩︎

Let \(\mathbf{z}_k^{(A)} \in \mathbb{R}^{d}\) and \(\mathbf{z}_k^{(R)} \in \mathbb{R}^{d}\) denote the causal latents at latent timestep \(k\) for the Anchor and Relational agent, respectively. For a window of length \(L\) latent steps per agent, where \(L\) denotes the maximum context size, we form a single ordered latent stream by concatenating the two agent streams into a single sequence: \[\mathbf{z}_{1:2L} = \big[\mathbf{z}^{(A)}_{1:L}\,;\,\mathbf{z}^{(R)}_{1:L}\big].\]

Segment-wise causal noising in latent space. Following diffusion forcing [45], we adopt a temporally structured noise schedule such that earlier latent segments are assigned lower noise levels than later ones. This creates a progressive certainty gradient along the temporal axis, encouraging committed refinement of past frames while preserving flexibility for future ones. Instead of assigning independent noise levels for every latent timestep, we partition the latent timeline into contiguous non-overlapping segments of size \(S\), where \(S\) denotes the number of latent steps in each segment. All latent steps within the same segment share a common noise level. Let \(s(k)=\lceil k/S\rceil\) denote the segment index for timestep \(k\), and let \(\alpha_{s}\in[0,1]\) denote the noise level associated with segment \(s\). The noised latent at timestep \(k\) is constructed as \[\tilde{\mathbf{z}}_{k} = \alpha_{s(k)}\,\mathbf{z}_{k} + (1-\alpha_{s(k)})\,\boldsymbol{\epsilon}_{k}, \qquad \boldsymbol{\epsilon}_{k}\sim\mathcal{N}(\mathbf{0},\mathbf{I}). \label{eq:segmentwise95noising}\tag{1}\] Segment-wise noise sharing enforces joint refinement over short temporal segments rather than isolated latent steps. This increases the effective temporal context at each diffusion stage and improves local motion consistency. For the two-agent latent stream \(\mathbf{z}_{1:2L}\), segments corresponding to the same timestep of each agent share identical noise levels. This symmetric assignment ensures that the Anchor and Relational branches are refined at the same temporal stage, which stabilizes interaction dynamics during diffusion.

Relational diffusion denoiser. The denoiser is built upon the causal diffusion architecture proposed in [46]. As shown in 2, the noised latent stream \(\tilde{\mathbf{z}}_{1:2L}\) is processed by a stack of transformer layers that jointly model temporal dynamics and inter-agent relations under a segment-wise causal attention mechanism. Noise levels are first embedded and injected into each transformer layer through timestep-conditioned adaptive layer normalization (AdaLN) [47]. Self-attention is then applied with rotary positional encoding (RoPE) [48] along the temporal dimension to preserve ordering under incremental extension. To enforce temporal causality and relational reasoning, we apply a segment-wise causal relational attention mask to the self-attention logits.

Let \(q,r \in \{1,\dots,2L\}\) index query and key positions in the concatenated latent stream. For any position \(u\), we denote by \(a(u)\in\{A,R\}\) its agent identity and by \(k(u)\in\{1,\dots,L\}\) its per-agent latent timestep. Timesteps are grouped into segments of size \(S\) with segment index \[s(u)=\left\lceil \frac{k(u)}{S}\right\rceil.\] The relational attention mask is defined as \[M_{qr} = \begin{cases} 0, & s(r)\le s(q)\;\wedge\;\Gamma\!\big(a(q),a(r)\big)=1, \\ -\infty, & \text{otherwise}\,. \end{cases}\] This formulation ensures that each query attends only to its own, past segments, and optionally cross-agent segments, preventing future leakage. \(\Gamma(\cdot,\cdot)\) is a mode gate that controls cross-agent visibility and is defined as \[\Gamma(a(q),a(r))= \begin{cases} 1, & \text{interaction mode}, \\ 1[a(q)=a(r)=A], & \text{solo mode}. \end{cases}\] In solo mode, only the Anchor tokens remain visible, while Relational tokens are ignored. After the self-attention, text instructions are encoded using a frozen DistilBERT encoder [49], producing token embeddings that are linearly projected to the model dimension and incorporated via cross-attention in every transformer layer. This design allows latent vectors to condition on the active textual instruction while maintaining temporal causality in the motion stream. The overall denoiser predicts velocity-like residuals [50] for all latent steps jointly: \[\mathbf{v}_\theta = \mathcal{F}_\theta \!\left( \tilde{\mathbf{z}}_{1:2L}, \phi(s), \mathbf{c}, M \right),\] where \(\phi(s)\) denotes embedded segment noise levels, \(\mathbf{c}\) the text features, and \(M\) the mode-aware relational gating mask.

Inference. At inference time, we generate motions by progressively refining latent segments under a temporally offset noise schedule [45], [46]. Let \(K\) denote the number of diffusion refinement iterations and \(\delta\) the refinement offset between adjacent segments. Segment \(i+1\) begins denoising \(\delta\) refinement steps after segment \(i\). Under this schedule, earlier segments become deterministic sooner while later segments remain partially noisy and continue to be refined. This overlapping denoising avoids hard segment boundaries and improves efficiency compared to strictly sequential autoregressive refinement. For continuous generation, we adopt variable-length history conditioning during inference. Under the same prompt, the full context window \(L\) is preserved to maintain long-range temporal coherence. When the textual instruction changes, we retain only a short history \(H\) of previously generated latents while replacing the conditioning text embedding \(\mathbf{c}\) for future segments. This reduced history provides sufficient motion context for transition while allowing the model to adapt smoothly to the new instruction. Agent configurations can also change dynamically during inference. When generating solo motion, the Relational branch is masked through \(\Gamma(\cdot,\cdot)\), reducing the model to a single-agent generator. When transitioning to interaction, the Relational branch is activated and initialized either from noise or from encoded relative dynamics derived from existing motion. Unless otherwise stated, we initialize the Relational branch from noise at interaction onset. Details of the exact inference procedure are provided in the supplementary material.

4 Experiments↩︎

4.1 Experimental Setup↩︎

Dataset. We use HumanML3D [19] for single-person motion and InterHuman [6] for two-person interactions. We adopt a 272-dimensional representation variant of the HumanML3D dataset used in [37] for better compatibility with InterHuman. It contains 26,846 motion sequences derived from the AMASS [51] dataset, each paired with three textual descriptions. InterHuman consists of 7,779 two-person interaction sequences with 23,337 textual descriptions. Since HumanML3D and InterHuman adopt different motion representations, we convert both datasets into our shared Anchor–Relational representation for training. This conversion is deterministic and invertible. For evaluation, generated motions are converted back to the native representation of each dataset before computing metrics, strictly following standard evaluation protocols. We train a unified model jointly on HumanML3D and InterHuman for cross-scenario streaming motion generation, while training dataset-specific models on InterHuman for fair comparison in interaction generation evaluation.

Following prior work [32], we additionally report results on InterX [52] to assess cross-skeleton generalization. InterX follows the SMPL-X [53] representation and contains 11,388 motion sequences, each paired with three textual descriptions. To ensure compatibility with our model, we convert the InterX motions into the same anchor–relational representation used for training.

Implementation details. The causal temporal encoder downsamples the motion sequences by a factor of \(4\), producing latent tokens with dimension \(64\). The motion generation model is implemented as a causal relational diffusion transformer with \(8\) layers, a hidden dimension of \(512\), \(4\) attention heads, a maximum context length of \(L=75\), and a segment size of \(S = 5\). Text conditions are extracted using a frozen DistilBERT encoder. We train the denoiser with a batch size of \(64\) and a maximum sequence length of \(300\) frames, i.e., 75 latent tokens, for \(500\) epochs. Optimization uses AdamW with learning rate \(2\times 10^{-4}\), betas \((0.9,0.99)\), and weight decay \(1\times 10^{-5}\). We employ flow matching [50], [54], [55] as the ODE sampler in causal diffusion forcing. During inference, we perform \(K=50\) denoising steps, and the refinement offset \(\delta\) is set to \(5\).

4.2 Quantitative Evaluation on Interaction Generation↩︎

We evaluate human–human interaction generation capability of ARMS under the standard benchmark setting. We follow evaluation protocols in text-to-motion generation using the pretrained text–motion retrieval evaluator adopted in prior work [6], [32]. We report: (1) R-Precision, the retrieval accuracy of the ground-truth motion given a text query; (2) MM Dist, the average embedding distance between generated motions and their paired captions; (3) FID, the Fréchet distance between feature distributions of generated and real motions; and (4) Diversity, the average pairwise distance among generated motions. All methods are evaluated under identical settings as prior work for fair comparison.

We evaluate our model under two inference regimes. Full-window generation performs diffusion over the entire motion window in a single denoising process. Streaming generation adopts our causal segment-wise refinement schedule and incrementally synthesizes motions. Table 1 reports quantitative results on the InterHuman test set. Our full-window generation variant achieves the best overall performance among all compared methods, indicating superior text–motion alignment and realism. In the streaming generation setting, ARMS maintains strong quality (FID \(4.44\), MM Dist \(3.78\)), with lower R-Precision due to compounding uncertainty over long incremental rollouts. Importantly, streaming generation enables long-duration motion synthesis beyond fixed-length clips while maintaining comparable realism and alignment quality.

Table 2 presents results on InterX. Our streaming generation model outperforms other models on all metrics. Notably, the improved R precision and diversity demonstrates the strong text–motion alignment and rich motion variation. These results indicate that our formulation generalizes effectively across different skeletal definitions, without being specialized to a specific joint configuration.

4.3 Motion Streaming: Qualitative Evaluation↩︎

We present qualitative results on long-duration interaction synthesis and cross-scenario generalization. 3 compares our method with the adapted InterMask. In extended sequences, InterMask often exhibits unnatural spatial shifts during interaction transition. In contrast, our method achieves smooth transitions between sub-actions, maintains consistent relative positioning, and preserves semantically aligned interaction evolution over time, demonstrating improved stability in long-duration interaction synthesis. 4 and 5 further illustrate results of three cross-agent scenarios: (1) a solo motion transitions into a human–human interaction; (2) an interaction evolves into another; and (3) an interaction resolves back into a single-person motion. In all cases, motions are generated continuously under incremental text updates. The model produces smooth transitions between different scenarios within a single framework, while maintaining temporal coherence and spatial consistency.

4.4 Motion Streaming: Quantitative Evaluation↩︎

We adopt the jerk-based metrics introduced in FlowMDM [35]. Jerk is defined as the time derivative of acceleration, which is sensitive to abrupt motion changes. Peak Jerk (PJ) measures the maximum instantaneous jerk magnitude across all joints, capturing extreme discontinuities, while Area Under the Jerk (AUJ) accumulates deviations from the average jerk over time, reflecting persistent smoothness irregularities. For transition evaluation, PJ and AUJ are computed within a 2-second temporal window centered at the behavior transition point. We evaluate two long-horizon streaming scenarios: (1) continuous interaction generation and (2) transitions between solo motion and interaction. Since no standardized benchmark exists for these tasks, we construct evaluation sequences from the InterHuman and HumanML3D test sets.

For interaction streaming, we randomly sample 64 groups of interaction descriptions and generate sequences composed of eight consecutive interaction segments using streaming inference. To analyze both transition quality and semantic fidelity, we report jerk-based transition metrics together with standard motion generation metrics computed on the individual interaction subsequences. For comparison, we adapt the state-of-the-art method InterMask [32] to support history-conditioned generation by feeding previously generated frames as unmasked prefix tokens. As shown in Table 3, InterMask with inpainting achieves the best R@Top3 and FID on the interaction subsequences. However, we observe that this behavior arises because the model effectively starts a new motion conditioned on the next prompt rather than generating a smooth transition from the preceding motion. In contrast, our streaming model explicitly enforces temporal continuity across segments. As a result, the generated motion gradually adapts to satisfy the new textual condition, which slightly reduces semantic alignment on the isolated subsequences but produces substantially smoother long-horizon motion. In addition, the initial interaction state of each sequence in our generation is conditioned on the previously generated sequence, whereas InterMask initializes each segment from a newly generated starting pose that may exhibit foot sliding. These results show that our method significantly reduces motion discontinuities compared with the InterMask baseline, achieving lower PJ (0.071 vs. 0.265) and AUJ (2.925 vs. 22.440).

For solo–interaction transitions, we randomly construct 64 sequences consisting of a single-person motion, followed by an interaction motion, and another single-person motion. We adapt and retrain InterMask on HumanML3D and InterHuman datasets to support both solo and duo generation by masking the second agent before interaction onset and activating it during the duo phase. As shown in Table 3, the adapted InterMask baseline exhibits large discontinuities at the solo-to-interaction boundary, with PJ/AUJ of 3.131/7.444, while ARMS remains close to ground truth with 0.077/1.070. This gap reflects the difficulty of abruptly synthesizing a globally consistent two-person configuration after suppressing the second agent, whereas ARMS keeps the Anchor stream causally continuous and introduces the second agent through the Relational branch.

4.5 Ablation Study↩︎

We conduct ablation studies to analyze the contribution of key design components and the sensitivity to critical hyperparameters. Results are summarized in Table 4. Replacing our Anchor–Relational representation with InterGen’s non-canonical formulation [6] slightly improves interaction alignment, but significantly degrades transition smoothness, confirming that globally anchored modeling weakens long-horizon stability. Removing the autoregressive streaming mechanism achieves the best single-window interaction metrics, yet increases transition jerk. Setting segment size \(S=1\), i.e., disabling joint latent refinement within segments, substantially degrades interaction realism and increases jerk, indicating insufficient temporal coupling. Moderate segment sizes achieve the best balance, whereas larger segments slightly reduce transition smoothness, suggesting that coarse refinement weakens local adaptability. Varying latent dimension \(d\) shows that smaller latent spaces (e.g., \(d=32\)) improve interaction alignment but amplify jerk. Increasing diffusion sampling steps \(K\) improves transition stability and realism, while a moderate inter-segment offset achieves the lowest jerk, validating the effectiveness of the temporal refinement schedule. Finally, the refinement offset \(\delta\), which controls the temporal delay between denoising adjacent segments during streaming inference, achieves the best trade-off between interaction quality and transition smoothness at a moderate value.

5 Conclusion↩︎

In this work, we present ARMS, an Anchor–Relational causal diffusion framework for streaming solo–social motion generation. ARMS unifies solo motion and human–human interaction within a single incremental generative process through dynamics-asymmetric motion modeling, decoupling temporal progression from inter-person relational alignment. Our causal relational diffusion model supports temporal refinement, variable-length history conditioning, and seamless transitions across one-/two-person social configurations. Experiments demonstrate strong generation performance and substantially improved transition smoothness and social coherence in long-horizon streaming, suggesting a scalable pathway toward more flexible and realistic socially aware motion synthesis.

Limitations. Although ARMS unifies solo and human–human interaction generation within a single streaming framework, several limitations remain. First, the current formulation does not explicitly condition on surrounding agents or environmental context during solo motion generation, so crowd-level and scene-level collision avoidance are not directly addressed. Second, this work focuses on up to two-person interaction scenarios and assumes that social mode is activated within the interaction-distance range covered by the training data. If agents begin far outside this range, solo locomotion should first bring them into a plausible interaction range before activating social coupling. While the relational formulation can be extended to variable numbers of agents through pairwise modeling [43], [57], scaling to multi-agent settings requires explicit reasoning over higher-order social structures. Third, ARMS currently uses hard mode-aware relational gating to preserve strict causal consistency between solo and interaction modes. While effective, this binary switching may still introduce abrupt changes in challenging transition cases. Gradually blending relational attention across mode switches could further improve transition smoothness.

Acknowledgements↩︎

This work was done during an internship at LY Corporation and was also supported by JST BOOST JPMJBS2423.

6 Implementation Details↩︎

6.1 Training Details↩︎

6.1.1 Temporal Autoencoders↩︎

We train a causal temporal variational autoencoder to compress motion sequences into a continuous latent space. All datasets are converted to our anchor-relational motion representation. Specifically, we extract the yaw angle from the root orientation and the \(xz\) translation from the root trajectory while keeping the remaining joint features unchanged. The encoder and decoder are implemented using causal 1D convolution and residual blocks. The encoder progressively downsamples the temporal dimension by a factor of \(4\), producing latent tokens with dimension \(d=64\). The model is trained on motion windows of \(64\) frames with a stride of \(10\) frames using the AdamW optimizer with a learning rate of \(2\times10^{-4}\) and \(\beta=(0.9,0.99)\). We use a batch size of \(256\) and apply linear learning-rate warmup followed by step decay at \(70\%\) and \(85\%\) of the total training iterations. The training objective consists of latent reconstruction loss, root reconstruction loss, and KL regularization, following prior work [37]. Training is performed on a single NVIDIA RTX 5090 GPU. Training takes approximately one hour on InterHuman and around two hours when jointly training on HumanML3D and InterHuman.

6.1.2 Causal Relational Diffusion↩︎

The motion diffusion model is implemented as a causal relational transformer operating in the latent space produced by the temporal autoencoder. We use a transformer with \(8\) layers, hidden dimension \(512\), feed-forward dimension \(1024\), and \(4\) attention heads. Text conditions are encoded using a frozen DistilBERT encoder and projected to the transformer dimension. During training, motion sequences are first encoded into temporally downsampled latent tokens using the pretrained causal autoencoder. We train the diffusion model with a flow-matching formulation using velocity prediction. Given a clean latent trajectory \(\mathbf{z}_k\), we first construct a noised latent trajectory by perturbing each latent token with segment-wise noise, \[\tilde{\mathbf{z}}_{k} = \alpha_{s(k)}\,\mathbf{z}_{k} + (1-\alpha_{s(k)})\,\boldsymbol{\epsilon}_{k}, \qquad \boldsymbol{\epsilon}_{k}\sim\mathcal{N}(\mathbf{0},\mathbf{I}), \label{eq:add95noise}\tag{2}\] where \(\alpha_{s(k)}\) is determined by the noise level of the segment containing timestep \(k\). The transformer then predicts the target velocity field \(\mathbf{u}_k\) along the linear transport path from the noised latent trajectory. The training objective is the mean squared error between the predicted and target velocity fields, \[\mathcal{L}_{\mathrm{fm}} = \frac{1}{|\mathcal{M}|} \sum_{(b,k)\in\mathcal{M}} \mathrm{mean}_{d} \left[ \left\| \hat{\mathbf{u}}_{b,k,d} - \mathbf{u}_{b,k,d} \right\|_2^2 \right],\] where \(\mathcal{M}\) denotes the set of valid non-padding latent timesteps and the mean is taken over the latent feature dimension. In practice, we compute the squared error over the latent dimension for each latent timestep and average it over all valid latent tokens in the batch. Training is performed with the AdamW optimizer using a learning rate of \(2\times10^{-4}\), \(\beta=(0.9,0.99)\), and weight decay \(10^{-5}\). We use a batch size of \(64\) and apply linear learning-rate warmup followed by step decay at \(50\%\), \(70\%\), and \(85\%\) of the total training iterations. Classifier-free guidance is implemented by randomly dropping text conditions with probability \(0.1\) during training. Latent sequences are partitioned into fixed-size segments during training, with \(5\) latent steps per segment in the latent timeline. All experiments are trained on a single RTX 5090 GPU. Training takes approximately \(3\) hours on InterHuman and around \(7\) hours on HumanML3D.

6.2 Inference Details↩︎

At inference time, motion is generated by progressively refining latent segments under a temporally structured noise schedule. Given a latent state \(\mathbf{z}^{(i)}\) at noise level \(\alpha^{(i)}\), where \(i\in \{1, \dots, K\}\) denotes the denoising step, the diffusion model predicts a velocity field that updates the latent according to \[\mathbf{z}^{(i+1)} = \mathbf{z}^{(i)} + \big(\alpha^{(i+1)} - \alpha^{(i)}\big) \,\mathbf{v}_\theta(\mathbf{z}^{(i)},\alpha^{(i)},\mathbf{c}),\] where \(\mathbf{v}_\theta\) is the velocity predicted by the causal relational transformer conditioned on text embedding \(\mathbf{c}\). Classifier-free guidance is applied during inference by combining conditional and unconditional predictions with guidance scale \(2.5\).

6.2.1 Prompt-conditioned streaming generation↩︎

For long-horizon generation under a fixed textual instruction, motion is generated using a segment-based sliding window in the latent space. The latent timeline is divided into contiguous segments, each containing a fixed number of latent steps. At each iteration, a new noisy segment is appended to the current sequence while previously generated segments remain within the context window. To efficiently refine long sequences, we employ a staggered scheduling that assigns different noise levels to different segments [45], [46]. Earlier segments receive lower noise levels while later segments remain noisier. This staggered scheduling allows multiple segments to be refined jointly during the same denoising process while preserving causal temporal ordering. Consequently, earlier segments gradually converge to deterministic motion while later segments remain partially noisy and continue to be refined. This design enables arbitrarily long motion sequences to be generated without restarting the diffusion process. When textual instructions change during generation, the new prompt replaces the conditioning embedding for future segments while previously generated motion remains fixed as context. To preserve temporal continuity, we adopt a resampling strategy used in RePaint [58], where a short history of previously generated latent tokens is retained and injected into the denoising process. Specifically, the model first predicts a clean latent estimate \[\hat{\mathbf{z}}_0 = \mathbf{z}^{(i)} + (1-\alpha^{(i)}) \,\mathbf{v}_\theta(\mathbf{z}^{(i)},\alpha^{(i)},\mathbf{c}),\] after which the historical context tokens are restored at their corresponding temporal positions. The latent sequence is then re-noised to the next scheduled noise level before continuing the denoising process. The incremental generation process is summarized in 6 and 7.

6.2.2 Agent configuration transitions↩︎

Our framework supports dynamic transitions between solo motion and human–human interaction during generation. Let \(\mathbf{z}^{(A)}\) and \(\mathbf{z}^{(R)}\) denote the latent sequences of the anchor and relational agents, respectively. During diffusion refinement, each active branch is updated by the velocity predictor \[\mathbf{z}^{(j)}_{i+1} = \mathbf{z}^{(j)}_{i} + (\alpha_{i+1}-\alpha_i)\, \mathbf{v}_\theta^{(j)}(\mathbf{z}^{(A)}_{i},\mathbf{z}^{(R)}_{i},\alpha_i,\mathbf{c}), \qquad j\in\{A,R\},\] with cross-agent visibility controlled by the attention mask. When a second agent appears, the relational branch is activated while the anchor branch is preserved from the previously generated solo motion. The new relational latent can be initialized from noise, \[\mathbf{z}^{(R)}_0 \sim \mathcal{N}(0,\mathbf{I}),\] or from an encoded motion context when such initialization is available. Cross-agent attention is then enabled so that the relational branch can align with the historical anchor states \(\mathbf{z}^{(A)}_{H}\): \[\mathbf{v}_\theta^{(R)} = \mathbf{v}_\theta^{(R)}(\mathbf{z}^{(R)},\mathbf{z}^{(A)}, \mathbf{z}^{(A)}_H,\alpha,\mathbf{c}).\] This allows the newly introduced agent to align with the existing motion. When the interaction ends, cross-agent attention is disabled and each branch can continue independently as a solo motion stream. If the former anchor agent continues, its latent trajectory is refined using the same streaming generation process as 7, conditioned on the existing anchor latent history. Likewise, the former relational branch can be reassigned as the new anchor stream and continued using its own latent history under the same streaming generation procedure. In both cases, generation proceeds under the solo-mode attention mask. This role reassignment allows either participant in the interaction to continue as an individual motion stream without restarting generation. The overall transition procedure is summarized in 8.

6.3 Practical Usage and Scope↩︎

ARMS assumes a shared text condition within one model instance. When two agents need to perform simultaneous but independent solo motions before interacting, a practical usage is to run two independent ARMS streams in parallel, each with its own prompt, duration, and timing. Once a shared interaction prompt begins, the streams can be merged by activating the relational branch and continuing generation under the interaction-mode mask.

The Anchor–Relational representation is trained on the interaction-distance distribution present in the data. If agents start far outside this range, directly activating social mode can make the relational offset out-of-distribution. In this case, the intended procedure is to first generate solo locomotion that brings the agents into a plausible interaction range and then activate social mode.

ARMS uses hard mode-aware relational gating to preserve strict causal consistency between solo and interaction modes. For user control, the generation mode can be specified explicitly. As an optional automatic alternative, we train a lightweight binary MLP classifier on frozen DistilBERT text embeddings to predict whether the prompt describes solo motion or two-person interaction. The classifier mean-pools valid token embeddings and applies a MLP, achieving 99.9% mode classification accuracy.

6.4 On the InterX Dataset↩︎

We additionally evaluated our method on the InterX dataset, a large-scale human–human interaction benchmark based on the SMPL-X representation [53]. InterX contains 11,388 motion sequences, each paired with three textual descriptions. Compared with InterHuman, InterX uses the SMPL-X skeleton with 54 body, hand, and face joints plus the root joint, resulting in a significantly higher-dimensional motion representation. To ensure compatibility with our model, we convert the InterX motions into the same anchor–relational representation used for training. This conversion follows the same preprocessing procedure as for the HumanML3D and InterHuman datasets.

7 More Experimental Results↩︎

7.1 On the HumanML3D Dataset↩︎

We evaluate our method on the HumanML3D dataset using the 272-dim motion representation proposed in MotionStreamer [37]. This representation is convertible with the anchor branch of our representation and allows direct comparison with prior streaming text-to-motion models. Quantitative results on the HumanML3D test set are reported in 5. Our streaming generation model achieves the best performance across most evaluation metrics, significantly improving retrieval accuracy and motion quality compared with existing methods.

7.2 Long-horizon Drift Analysis↩︎

To directly evaluate long-horizon global drift, we generate a 5-min solo motion using the prompt “a person walks straight forward.” We compare ARMS with MotionStreamer [37] using two trajectory-level metrics. Mean Lateral Deviation (MLD) measures the average absolute deviation from the expected forward direction, where lower is better. Forward Efficiency (FE) measures the ratio between net forward progress and total traveled distance, where higher is better and the maximum value is 1.

As shown in 6, ARMS substantially reduces lateral drift and maintains forward progress over the 5-min rollout. Because interaction generation in ARMS builds on the anchor trajectory and relational displacement, stable solo anchor motion also supports stable anchor-based interaction rollout.

7.3 Close-contact Penetration Analysis↩︎

We further evaluate fine-grained interaction quality on a close-contact subset with contact frequency approximately 60%. We report average Penetration Depth (PD) and Penetration Frame Rate (PFR). PD measures the average inter-body penetration depth in centimeters, while PFR measures the percentage of frames with penetration; lower values are better for both metrics.

As shown in 7, ARMS achieves the lowest penetration depth and lower penetration frame rate than InterGen, while InterMask obtains a slightly lower penetration frame rate. Because contact precision is affected by the body model, body shape, mesh resolution, and the use of explicit physics constraints, we report these results as reference evidence for close-contact interaction quality. ARMS does not include a physics-aware or contact-refinement module, so dedicated contact refinement remains a complementary direction for future work.

7.4 Inference Efficiency↩︎

We evaluate the inference efficiency of our framework and compare it with representative streaming and interaction motion generation methods. Following prior work [37], we report the latency required to generate the first frame and the total time required to generate a 10-second motion sequence. For streaming models, the first-frame latency measures the time needed to produce the first output frame after receiving a prompt, whereas for full-window models it corresponds to the time required to generate the entire sequence.

As shown in ¿tbl:table:inference95time?, MotionStreamer [37] achieves the lowest first-frame latency for single-person motion generation but requires substantially longer time to generate long motion sequences. In contrast, our model with autoregressive streaming has a slightly higher first-frame latency, but generates long motions substantially faster due to the staggered noise scheduling strategy that allows later segments to be refined while earlier ones converge. For interaction generation, InterMask [32] and TIMotion [31] generate the entire motion sequence in a single pass, leading to higher overall latency. Our non-streaming configuration (denoted as full-window generation) generates the entire sequence within a single denoising window, similar to InterMask and TIMotion, and therefore exhibits nearly constant generation time independent of sequence length. Notably, it is faster than both InterMask and TIMotion for interaction generation. In contrast, the streaming configurations generate motion incrementally using a segment-based schedule, and the inference time therefore scales with the sequence length. With a reduced number of sampling steps (\(K=20\)), the model achieves a lower first-frame latency (0.067 s) and faster overall sequence generation. Larger segments (e.g., \(S=10\)) allow more frames to be refined jointly during each denoising step, further reducing the total generation time for long sequences compared with smaller segments. Importantly, our framework supports transitions between solo motion and interaction within the same generation process. The transition latency reported in ¿tbl:table:inference95time? shows that the model can switch between agent configurations with minimal overhead, enabling efficient streaming generation in dynamic social scenarios.

8 Qualitative Results↩︎

In addition to the quantitative evaluations, we provide further qualitative demonstrations on the project website. Since motion is inherently temporal, static images cannot fully capture the dynamics of the generated sequences. We therefore present the qualitative results in video format. The videos illustrate streaming generation across various scenarios, including incremental generation of solo motions, human–human interactions, transitions between solo and interaction behaviors, and comparisons with other methods.

9 Representation↩︎

Different interaction motion generation methods adopt distinct motion representations, which influence spatial consistency and interaction quality. To clarify these differences, we summarize the design of representative representations in 8 and provide qualitative comparisons on the project website.

Notation: \(\mathbf{p}\) denotes joint positions. \(\dot{\mathbf{p}}\) denotes joint velocities. \(\mathbf{q}\) represents joint rotations. \(\mathbf{v}\) denotes root velocities, including yaw angular velocity \(\mathbf{v}^{yaw}\) and planar velocity \(\mathbf{v}^{xz}\). \(\mathbf{r}^{yaw}\) denotes root yaw rotation. \(\mathbf{c}\) denotes foot contact indicators. \(\Delta^{xz}\) represents planar relative displacement between agents.

References↩︎