1 Introduction↩︎

Text-conditioned 3D human motion generators [1]–[5] now produce diverse, semantically faithful motion at competitive quality. In production workflows, however, a generated motion is seldom the end of the pipeline: animators sketch a root path the character must follow, character-interaction designers fix end-effector contacts to known props, and embodied-AI planners hand the policy a trajectory the agent must track. A generator may produce a plausible “walk casually while waving” when given the prompt alone, yet have the feet drift a dozen centimetres off the sketched path or the hand miss the prop the user marked—a small text prompt edit is no help. The unmet need is a generator that is at once text-faithful and precisely controllable along extrinsic geometric constraints—and that delivers both with a control adapter whose trainable parameters and architectural footprint are a small fraction of the underlying text-to-motion backbone. In this paper we study joint trajectory control: forcing user-specified joint positions—a single end-effector keyframe, a root path, or a multi-joint trajectory—to match given values while the prompt continues to dictate gait, style, and intent. Existing adapters typically reduce to two trained patterns: duplicate-branch adapters preserve attention access but are heavy, while test-time-optimization approaches sidestep retraining by adding a per-sample inference-time refinement stage.

1.0.0.1 Existing methods and a shared architectural mismatch.

Two families dominate prior trained trajectory control on text-to-motion generators, sitting at opposite extremes on a parameters-versus-inference-cost axis. ControlNet-style adapters [6]–[9] train a zero-initialized replica of the backbone connected layer-by-layer to the frozen trunk: this restores per-layer access to self-attention and keeps text alignment intact, but the duplicated branch grows the trainable budget to a sizable fraction of the backbone, scaling roughly with the backbone subset that the branch duplicates [7]. A separate family performs test-time optimization in latent code or sample noise without a trained side-adapter—TLControl [10] refines part-VQ codes via L-BFGS, DNO [11] optimizes diffusion noise—avoiding the parameter cost but moving the per-sample compute cost to inference. We read these two families as instances of an underlying pattern: control signal, motion representation, and attention pathway are architecturally mismatched—ControlNet restores per-layer attention access at the cost of duplication, while sample- or code-space optimization avoids duplication at the cost of per-sample inference compute. Neither family recovers per-layer attention access at a small fraction of either cost.

1.0.0.2 Approach.

We propose KV-Control, a coupled three-component design in which a part-tokenized motion representation, a backbone layout, and an attention-side controller are jointly engineered for parameter-efficient, per-layer trajectory control on a frozen backbone. The three components share a single design question: at every self-attention layer, can each anatomical part-frame token be made attention-addressable, so a control signal is injected at exactly that site rather than routed through a global pose token? PartVQ learns a data-driven \(Q\)-part anatomical token layout (here \(Q\!=\!6\)); T-Concat unpacks these per-part codebooks along the sequence so each (frame, part) pair is a distinct attention slot with per-position text cross-attention preserved; the K/V injection module attaches a control-conditioned key/value pair at each self-attention layer through a near-identity low-rank projection, leaving the motion-token query stream and frozen backbone unchanged. The trio is mutually reinforcing—PartVQ supplies anatomy-aligned tokens, T-Concat exposes them as attention sites, and the K/V module reads from those sites with a 1.5  mechanism (10.5  including the shared 9  trajectory encoder)—and the coupling is empirically necessary (§4.2). PartVQ and T-Concat are pretrained once and held frozen during the control-adaptation stage that is the focus of this paper. Figure 1 visualizes the inputs, the trainable adapter, the frozen backbone, and the K/V injection in a single diagram.

1.0.0.3 Contributions.

(1) Part-addressable motion substrate: we introduce a PartVQ \(+\) T-Concat substrate that factorizes motion into data-driven anatomical part codebooks and unpacks them along the sequence axis, making each frame–part token an explicit attention-addressable site while preserving per-position text cross-attention. (2) Attention-side trajectory control formulation (§3.3): we cast joint trajectory control for frozen masked motion transformers as memory retrieval inside self-attention, where continuous joint constraints are encoded as part- and time-addressed K/V memories rather than injected through global pose tokens or optimized only at the output side. (3) Parameter-efficient K/V injection mechanism: we develop KV-Control, a per-layer near-identity low-rank K/V branch with learnable control-column attention bias, adding 1.5  trainable injection parameters while leaving the motion-token query stream, FFN, text cross-attention pathway, and frozen backbone weights unchanged. Under the MaskControl protocol the system reaches 0.40 cm pelvis error and 0.71 cm multi-joint with 1.5  mechanism parameters — about \(26{\times}\) fewer than a same-backbone duplicated-branch sanity check at the matched M2 protocol (Tables 2, 4).

2 Related Work↩︎

2.0.0.1 Text-to-motion generation.

Masked motion-token models [1], [2], [12], [13] tokenize motion via VQ-VAE and predict masked tokens, building on the bidirectional masked-token paradigm introduced for images by MaskGIT [14]; discrete-autoregressive variants [15] predict tokens causally instead. Diffusion models [3]–[5], [16], [17] diffuse continuous motion features. Recent work pushes these families along orthogonal axes: state-space generators [18], lightweight architectures [19], multi-action discrete diffusion [20], part-coordinating synthesis [21], and motion-language LLMs [22]. The two families reach comparable text-alignment quality; we adopt the masked-transformer family for faster inference and cheaper adapters. Classical controllable character animation [23]–[25] pursued the same runtime-controllability goal in pre-neural settings.

2.0.0.2 Motion representation for control.

A single global VQ-VAE codebook [15] (or a hierarchical residual variant [1]) packs whole-body motion into a monolithic token sequence; TLControl [10] factors geometry across manually defined per-part VQs; ACMDM [17] argues for absolute coordinates as inherently control-ready. We follow the part-aware direction but derive the partition from data via lagged cross-correlation clustering with a kinematic-chain integrity prior.

2.0.0.3 Controllable text-to-motion: a control-interface taxonomy.

We organize prior trajectory-controlled methods by where the control signal enters the generator. (i) Sample-, noise-, or code-space inference-time optimization: GMD [26] and PriorMDM [27] steer the iterative sampling trajectory in score- or sample-space; DNO [11] optimizes diffusion noise latents; TLControl [10] refines part-VQ codes via L-BFGS. CondMDI [28] trains a keyframe-conditional in-betweening diffusion model. MotionLCM [29] accelerates inference orthogonally to control. (ii) ControlNet-style duplicated branches: OmniControl [7], MaskControl/ControlMM [8], and InterControl [9] train a zero-initialized replica of the backbone connected layer-by-layer to the frozen trunk, restoring per-layer access at a substantial fraction of the backbone parameter budget; MaskControl additionally applies logit-side refinement at inference. (iii) Semantic body-part editing: fine-grained spatio-temporal editing [30] and pose-code editing [31] target semantic body-part control rather than spatial trajectory targets. (iv) Attention-side memory injection (ours): KV-Control places the trajectory signal as part- and time-addressed key/value memory inside self-attention, between input conditioning and output logits, leaving the motion-token query stream and frozen backbone weights unchanged. Inference-time optimization [8], [10] is treated as an optional precision/compute knob (M2/M3 in §4.1).

2.0.0.4 Parameter-efficient adapters.

KV-Control relates to three families introduced for NLP and image domains: adapter modules [32] (added bottleneck layers; we instead inject K/V tokens without inserting new layers), LoRA [33] (low-rank weight updates; we leave all backbone weights untouched), prefix tuning [34] (fixed learnable prefixes; ours are dynamically conditioned on per-frame trajectory), and IP-Adapter [35] (decoupled cross-attention adding an image-prompt path to text-to-image diffusion; we fuse into existing self-attention without inserting additional transformer blocks or duplicating the trunk).

3 Method↩︎

Human motion has structured anatomical degrees of freedom rather than an undifferentiated token structure. Part trajectories—pelvis, torso, arms, legs, hands, feet—are coupled but distinct, and external trajectory targets are typically localized to a small subset of those parts. The token layout therefore directly affects how easily a control signal can reach and modulate the relevant degrees of freedom. We exploit this by exposing part-level structure as sequence positions rather than hiding it inside the channel dimension, and by routing the control signal into the attention memory rather than the motion-token query stream.

Concretely, this paper develops three tightly coupled components that together turn anatomical structure into per-layer attention-addressable control (Figure 1). PartVQ (§3.1) factors body geometry across \(Q\) per-part codebooks; T-Concat (§3.2) then unpacks the \(Q\) codebooks along the sequence axis so that each anatomical part-frame token becomes a distinct attention-addressable site while per-position text cross-attention is preserved; KV-Control (§3.3) injects a control-conditioned key/value pair at each self-attention layer of the (now-frozen) backbone, reading exactly from the sites the layout exposes. PartVQ and T-Concat are pretrained once on text-to-motion data and held frozen during the control-adaptation stage that is the focus of this paper. §3.4 describes the KV-Control training objective and a matched inference-time refinement adapted from prior work.

3.1 Anatomy-Aware Part Tokenization (PartVQ)↩︎

3.1.0.1 Partition derivation.

Per-joint activation \(a_j(t)\) is the \(\ell_2\) norm of joint \(j\)’s relative-to-parent HumanML3D feature, standardized across the corpus. Pairwise joint similarity is the absolute maximum-lag normalized cross-correlation \(s_{jk}\) over a small lag window; joints are clustered hierarchically on \(1-s_{jk}\) and a kinematic-chain integrity post-step reassigns each predefined limb chain (left/right arm/leg) to its majority label so no chain splits. Algorithm 2 summarizes the full procedure; the resulting six-part partition used throughout this paper is listed in the supplement.

3.1.0.2 VQ-VAE architecture and objective.

A 1D-conv encoder maps each part’s motion window to latent tokens quantized by per-part EMA-reset vector quantizers [36]; a shared decoder operates on the channel-concatenated quantized latents. Training combines smooth-L1 reconstruction, VQ commitment, and an auxiliary joint-position smooth-L1 loss for direct kinematic supervision.

3.2 T-Concat Backbone↩︎

3.2.0.1 Motivation.

A channel-concatenated layout (which we contrast against in §4.2) packs the \(Q\) part codebooks into a single channel vector per timestep, leaving downstream attention layers to recover per-part factors from a shared channel embedding rather than from distinct sequence positions. T-Concat instead unpacks the codebooks along the sequence axis, making each anatomical part-codebook factor an attention-addressable site and giving the K/V branch in §3.3 per-part landing positions to read from.

3.2.0.2 Layout.

T-Concat unpacks the \(Q\) codebooks of length \(T_{\mathrm{tok}}\) along the sequence axis, producing \(S\,=\,T_{\mathrm{tok}}\!\cdot\!Q\) tokens of fixed dimension \(d_{\text{model}}\) (versus a \(T_{\mathrm{tok}}\)-token channel-concatenated layout used by global-codebook masked-transformer baselines, which we contrast against in §4.2). The \(Q\)-fold sequence-length increase is the price of preserved per-quantizer token identity and a one-to-one ratio between motion tokens and text-conditioning sites.

3.2.0.3 Position encoding.

Tokens at position \((t,q)\) in the unpacked sequence receive an additive embedding \(\mathbf{p}_{t,q}\,=\,\mathbf{e}^{(\text{time})}_t\,+\,\mathbf{e}^{(\text{quant})}_q\), decoupling temporal order from quantizer index without a multiplicative embedding table.

3.2.0.4 Text pathway and regularization.

A frozen CLIP [37] text encoder feeds a small TransformerEncoder adapter trained jointly with the backbone. Its outputs feed gated text cross-attention blocks inserted after self-attention layers at interval \(I\) (we use \(I\,=\,1\) in the main configuration so every self-attention layer is followed by cross-attention). Each cross-attention block has a learnable scalar gate so its contribution starts near identity. The backbone is a post-norm Transformer trained with masked-token prediction; we additionally apply a hybrid 2D masking regularizer (a fraction of samples receive per-quantizer independent masks). Classifier-free-guidance dropout [38] enables CFG at inference.

3.3 KV-Control↩︎

3.3.0.1 Motivation.

Given the addressable per-part token layout above, the remaining design question is how the trajectory signal should enter the frozen generator. We supply it as additional attention memory: at each self-attention layer, control-conditioned pseudo-tokens are appended to the key/value sets, and the adapter’s role reduces to learning what control memory to add at each layer. The mechanism, the rationale for choosing the K/V side over the Q side, and the per-layer near-identity initialization that lets the controller start as a no-op are described next.

3.3.0.2 Mechanism.

KV-Control attaches the control signal as a small set of additional attention sites at the key/value side of every self-attention layer. The motion-token queries and the backbone’s forward pass are unchanged except for a low-rank computation off the side and an additive attention-bias term. At initialization, the adapter implements a near-identity mapping on the frozen backbone—the control columns participate in the softmax denominator but contribute negligible mass under the strongly negative initial attention bias—and the original frozen backbone is recovered exactly only in the limit \(b_i\!\to\!-\infty\) (or, equivalently, when the control branch is masked out at inference). Non-trivial control behavior then emerges as the bias and low-rank weights move away from this near-identity initialization during training.

3.3.0.3 Why the key/value side and not the query side.

The choice is deliberate. A query-side intervention changes the motion-token query stream that the backbone’s text cross-attention reads as per-position grounding, so any disturbance propagates into how text attends to motion. A key/value-side intervention instead leaves the motion-token query sequence pointwise unchanged: the control branch contributes additional entries motion queries may consult, gated to start near zero by the \(b_i\) initialization. Concretely, the motion-token query input \(\mathbf{Q}_i\) defined below is identical to the frozen backbone’s, so the dense per-position text cross-attention that the backbone was pretrained with is preserved by construction. This asymmetric treatment of \(\mathbf{Q}\) vs \(\widetilde{\mathbf{K}},\widetilde{\mathbf{V}}\) makes K/V injection parameter-efficient and empirically effective under the PartVQ\(+\)T-Concat substrate and protocols evaluated here.

3.3.0.4 Control encoder.

At each sampling step \(s\) the controller decodes the current token state \(\mathbf{z}_s\) via the frozen PartVQ decoder under stop-gradient, producing \(\mathbf{p}^{\text{pred}}_s\,=\,\mathrm{FK}(\mathrm{sg}(D_{\mathrm{VQ}}(\mathrm{safe}(\mathbf{z}_s))))\footnote{\mathrm{safe}(\mathbf{z}_s) replaces mask or out-of-codebook ids with a valid zero code before the frozen decoder.}\) (out-of-codebook ids are replaced by \(0\)). \(\mathbf{z}_s\) is the iterative unmask state at inference and the BERT-style random-masked GT state at training (the same state consumed by the masked-prediction objective), so the training residual \(\mathbf{p}^{\text{tgt}}-\mathbf{p}^{\text{pred}}_s\) is non-trivial. The control input \(\mathbf{c}_s=[\mathbf{p}^{\text{tgt}}-\mathbf{p}^{\text{pred}}_s;\,\mathbf{p}^{\text{tgt}}]\odot\mathbf{m}\) combines target and residual, gated by the joint–frame mask \(\mathbf{m}\). A 1D-conv stack downsamples to \(T_{\mathrm{tok}}\) tokens of dimension \(d_{\text{model}}\). \(\mathbf{p}^{\text{pred}}_s\) is read off the same pass as the transformer, so no extra frozen-base forward is needed (Table 1 wall-clock includes this).

3.3.0.5 Per-layer K/V injection.

Each self-attention layer \(i\) is augmented as follows. Let \(\mathbf{X}_i\,\in\,\mathbb{R}^{B\times S\times d_{\text{model}}}\) be the layer’s motion-token input. The shared \(\mathbf{f}_{\text{ctrl}}\) is mapped through a layer-specific zero-initialized low-rank down-projection followed by two parallel up-projections, \[\mathbf{h}_i\,=\,\mathbf{f}_{\text{ctrl}}\,\mathbf{W}^{\text{down}}_i,\quad \mathbf{C}^{\text{K}}_i\,=\,\mathbf{h}_i\,\mathbf{W}^{\text{K}}_i,\quad \mathbf{C}^{\text{V}}_i\,=\,\mathbf{h}_i\,\mathbf{W}^{\text{V}}_i,\] with \(\mathbf{W}^{\text{down}}_i\,\in\,\mathbb{R}^{d_{\text{model}}\times r}\) initialized to zero and \(\mathbf{W}^{\text{K}}_i,\mathbf{W}^{\text{V}}_i\,\in\,\mathbb{R}^{r\times d_{\text{model}}}\) randomly initialized. Augmented key/value input sequences are formed by concatenation, \(\widetilde{\mathbf{K}}_i^{\text{in}}\,=\,[\,\mathbf{X}_i;\,\mathbf{C}^{\text{K}}_i\,]\) and \(\widetilde{\mathbf{V}}_i^{\text{in}}\,=\,[\,\mathbf{X}_i;\,\mathbf{C}^{\text{V}}_i\,]\) of length \(S\,+\,T_{\mathrm{tok}}\). The query input remains \(\mathbf{X}_i\), unchanged. The attention output of layer \(i\) is then the standard \[\mathrm{Attn}_i\,=\,\mathrm{softmax}\!\Bigl(\tfrac{\mathbf{Q}_i\widetilde{\mathbf{K}}_i^{\!\top}}{\sqrt{d_h}}\,+\,\mathbf{B}_i\Bigr)\,\widetilde{\mathbf{V}}_i,\] with \(\mathbf{Q}_i\,=\,\mathbf{X}_i\mathbf{W}^{\text{Q}}_i\), \(\widetilde{\mathbf{K}}_i\,=\,\widetilde{\mathbf{K}}_i^{\text{in}}\mathbf{W}^{\text{K},\text{attn}}_i\), \(\widetilde{\mathbf{V}}_i\,=\,\widetilde{\mathbf{V}}_i^{\text{in}}\mathbf{W}^{\text{V},\text{attn}}_i\) formed from the frozen backbone projections, and \(\mathbf{B}_i\,\in\,\mathbb{R}^{S\times(S+T_{\mathrm{tok}})}\) a learnable additive bias that is zero on motion-token columns and a single learnable scalar \(b_i\) on the control-token columns, initialized to a strongly negative value so the controller starts near identity at step zero. This double near-identity initialization lets the adapter learn its own layer-wise influence curve from training without a hand-tuned schedule.

3.4 Frozen-Backbone Training and Inference Refinement↩︎

3.4.0.1 Frozen-backbone discipline.

During control adaptation, the PartVQ tokenizer, CLIP text adapter, and T-Concat backbone are pretrained and held fixed; their parameters are never updated by the control objective. The forward pass still traverses the frozen token embeddings, transformer self-attention layers, gated text cross-attention, PartVQ decoder, and the forward-kinematics path that turns codebook embeddings into joint positions, so the optimizer sees the full computation graph and can route gradients through it. The trainable budget, however, is limited to the control encoder, the per-layer K/V down/up projections, and the control-column attention bias scalars. Concretely, the masked-token cross entropy (\(\mathcal{L}_{\mathrm{CE}}\)) and the trajectory loss (\(\mathcal{L}_{\mathrm{traj}}\)) below supervise the controlled output estimate \(\hat{\mathbf{x}}^{\text{out}}_0\) produced by the K/V-augmented transformer; they do not supervise the detached current-state estimate \(\mathbf{p}^{\text{pred}}_s\) that the controller consumes when building \(\mathbf{c}_s\) in §3.3.

3.4.0.2 Adapter training objective.

KV-Control is trained with AdamW. The objective combines a masked cross-entropy term on motion tokens with a forward-kinematics trajectory loss on the controlled joints, \[\mathcal{L}_{\mathrm{KV}} = \lambda_{\mathrm{CE}}\mathcal{L}_{\mathrm{CE}} + \lambda_{\mathrm{traj}}\mathcal{L}_{\mathrm{traj}},\] where \(\mathcal{L}_{\mathrm{CE}}\) uses the masked-token loss formulation of MoMask [1] on the per-part codebooks, and \[\mathcal{L}_{\mathrm{traj}} = \|\mathbf{m}\odot(\mathrm{FK}(\hat{\mathbf{x}}^{\text{out}}_0) -\mathbf{p}^{\text{tgt}})\|_1\] applies FK supervision to the predicted clean-token estimates \(\hat{\mathbf{x}}^{\text{out}}_0\). Because the predictions are discrete codebook tokens, we evaluate \(\mathcal{L}_{\mathrm{traj}}\) through a soft codebook relaxation: token logits define an expectation over each per-part codebook, which is decoded by the frozen PartVQ decoder before FK. This differentiable path also supports the optional inference-time refinement used in M1/M2/M3, where logits or post-sampling token embeddings are optimized while model parameters remain fixed; the exact schedules are reported in Table 1.

4 Experiments↩︎

4.1 Implementation Details↩︎

We evaluate on HumanML3D [39] under the MaskControl protocol [8] (CFG \(3.25\), \(T\,=\,10\) steps, seed \(3407\), batch \(32\)). PartVQ uses \(Q\,=\,6\) parts (codebook \(128\), code dim \(128\)). The T-Concat backbone is a \(20\)-layer post-norm Transformer (\(d_{\text{model}}\,=\,384\), \(8\) heads, sequence \(S\,=\,294\) tokens) with frozen CLIP ViT-B/32. KV-Control uses per-layer rank \(r\,=\,64\), \(\mathbf{W}^{\text{down}}_i\) zero-init, control bias \(b_i\,=\,-5\), contributing \(1{,}474{,}560\) K/V parameters; together with the shared trajectory encoder the full control-side trainable budget is 10.5 . Inference-time refinement is adopted verbatim from prior work [8], [10] and exposes two stages—Stage 1 optimizes motion-token logits during sampling, and Stage 2 optimizes token embeddings post-sampling, both with AdamW (lr \(6{\times}10^{-2}\), \(\beta\,=\,(0.5, 0.9)\)). Four operating-point configurations are summarized in Table 1: M0 (pure feed-forward, no test-time optimization) measures the trained adapter’s standalone control ability; M1 adds Stage-1 logit optimization only; M2 follows the MaskControl-matched (MC), iteration-matched refinement schedule; M3 is the strongest-refinement setting and is the operating point used for the headline numbers. Because M1 and M3 share dynamic Stage-1, M1\(\!\rightarrow\!\)M3 isolates adding Stage-2; M2 and M3 share Stage-2, so M2\(\!\rightarrow\!\)M3 isolates the static-to-dynamic Stage-1 switch. Per-stage schedule diagnostics and the data-driven PartVQ part listing are in the supplemental material.

3pt

4.2 Backbone-Layout and Mechanism Ablations↩︎

The backbone-layout ablation isolates how PartVQ codebooks are presented to the transformer. T-Concat unpacks \(Q\,=\,6\) codebooks along the sequence axis (\(S\,=\,294\)); C-Concat channel-stacks them into one \(768\)-channel frame token (\(S\,=\,49\)). Both backbones share the trunk of §4.1, are trained from scratch under matched protocols, then frozen before attaching an identical KV-Control adapter.

4.2.0.1 Findings.

Table 2 rules out three alternative readings of the K/V-side gap. Backbone quality: T-Concat and C-Concat match on unconditional FID and Top-3, so the post-adapter divergence is not attributable to the trunk. Refinement protocol: T-Concat tracks \(\sim\!2.25\!\times\) better at M3 and \(\sim\!1.98\!\times\) better at M2, with Top-3 within \(0.01\) of C-Concat across both protocols. Parameter budget (Table 3): a same-substrate duplicated-branch with \(26\!\times\) our K/V mechanism budget still reaches only \(1.24\,\)cm at M2, versus \(0.51\,\)cm for K/V-Control—added capacity supplies reach, not anatomical addressability.

4.2.0.2 Why layout matters.

T-Concat’s advantage over C-Concat (Table 2) stems from a representational property: each (frame, codebook) pair occupies a distinct attention slot, so a low-rank K/V residual at a query position can selectively modulate one anatomical sub-token. C-Concat collapses all six codebooks into a single frame-shared hidden state: the adapter sees \(49\) query slots rather than \(294\), and any K/V residual must steer six entangled codebooks through one latent. This places representation-layout co-design at the centre of building parameter-efficient controllable text-to-motion adapters.

3pt

6pt

4.3 Trajectory Control↩︎

The same K/V-Control adapter applies to two trajectory-control settings on HumanML3D: single-joint (pelvis only) and multi-joint (random subset of up to six joints from \(\{\)pelvis, head, \(\ell\)/\(r\) foot, \(\ell\)/\(r\) wrist\(\}\), sampled per training example). Table 4 reports prior-work rows verbatim from [8] as benchmark context, and our M1/M2/M3 rows as re-runs on the KV-Control substrate under the operating points in Table 1; same-backbone mechanism evidence is isolated in Table 2.

4.3.0.1 Findings.

On single-joint pelvis, M3 reaches \(0.40\,\)cm (FID \(0.065\), Top-3 \(0.799\)); M1 (\(0.89\,\)cm) uses dynamic Stage-1 without Stage-2; M0 (\(29.88\,\)cm) is pure feed-forward, vs MaskControl’s \(40.41\,\)cm at its matched no-optimization configuration [8]. The all-joints multi-joint M3 row reaches \(0.71\,\)cm at \(17\!\times\!\) fewer mechanism parameters; the gap to MaskControl’s \(0.72\,\)cm prior baseline is within the benchmark’s reporting precision, so we interpret it as comparable tracking accuracy rather than a significant improvement.

3pt

4.3.0.2 Per-joint decomposition.

Table 6 reports the multi-joint adapter with the controlled set fixed to one anatomical joint at a time. Aggregate single-joint position error is \(0.41\,\)cm (FID \(0.086\), Top-3 \(0.795\)); per-joint position error ranges from \(0.31\,\)cm (left wrist) to \(0.69\,\)cm (pelvis). The foot rows (left foot FID \(0.117\), right foot FID \(0.102\)) breach the \(0.10\) FID gate—a reminder that the average single-joint FID should not be read as uniform per-joint gate satisfaction.

4.3.0.3 Extended diagnostics.

Table 5 adds matching-score (Match., lower better) and intra-set Diversity (Div., real-data \(\sim\!9.5\)) to the operating-point progression, plus a refinement-only baseline (frozen T-Concat, no K/V adapter). Stripping the K/V adapter and running the identical M2/M3 schedules drives position error to near zero but collapses FID to \(\sim\!104\) and Top-3 to \(\sim\!0.12\)—the adapter, not the refinement loop, supplies the structural conditioning that keeps refinement inside the text-conditioned motion manifold.

3pt

3pt

4.4 Qualitative Results↩︎

Figs. 3–4 probe trajectory control on OOD letter-shaped targets and contrast unconditional vs. in-distribution multi-joint control; Fig. 5 shows additional patterns. Per-letter videos and multi-joint stress demos are in the supplement. The letter trajectories test long-horizon root-path tracking with sharp turns and self-crossing curves outside the training prompt distribution; the same prompt threads visibly distinct paths whose gait, foot contact, and limb posture remain text-conditioned. The unconditional vs.controlled contrast shows that the K/V adapter complements a competent frozen generator: for the same prompt, motion semantics persist with or without trajectory control—only the spatial path changes. The multi-joint demos stress simultaneous pelvis and end-effector constraints where text intent and geometric targets compete; failures concentrate at foot rows under dense per-frame contact constraints, consistent with the foot-FID breach analysed in the supplement.

5 Conclusion↩︎

We presented KV-Control, a parameter-efficient K/V adapter (1.5  mechanism, 10.5  total) for trajectory control on a co-designed PartVQ \(+\) T-Concat substrate. Under the inherited refinement protocol, K/V reaches 0.40 cm pelvis error (M3) and 0.71 cm multi-joint. At the MaskControl-matched M2 setting, a same-backbone duplicated-branch sanity check needs \(39.3\,\)M mechanism parameters to reach \(1.24\,\)cm, versus \(0.51\,\)cm for our \(1.5\,\)M mechanism (\(\sim\!26.2{\times}\) fewer).

Several directions remain open. Substrate transfer: porting K/V to other discrete-latent motion backbones (autoregressive, diffusion-based, hybrid masked-AR) beyond HumanML3D’s masked-transformer family. Denser constraints: dense per-frame contact and multi-joint targets stress Stage-2 toward over-pinning; tighter regularization would broaden the effective regime. Cross-skeleton adaptation: the PartVQ partition is single-skeleton, and partition-aware re-attachment could lift the adapter to other morphologies.

References↩︎