Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation
January 01, 1970
Abstract
Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text. Motion is inherently temporal, especially in compositional and long-duration sequences that require semantic consistency across multiple action segments and smooth kinematic transitions throughout the trajectory. We posit that the initial noise is central to this consistency: within the Gaussian noise space, certain instances, winning noise tickets, carry latent structure that biases denoising toward particular motion semantics, even under null prompts. We propose WInning Noise Retrieval and Optimization (WINRO), a training-free, model-agnostic framework that improves text–motion alignment by selecting and refining such tickets before diffusion sampling. WINRO maps random noises to motion features generated under null prompts, retrieves the best-aligned noise for a given text, and refines it via a KL-regularized objective that reduces the residual semantic gap while preserving the Gaussian prior. An optional LoRA-based adapter amortizes this refinement into a single forward pass. WINRO consistently improves text–motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining, improves temporal robustness on the MTT benchmark, and generalizes to applications such as motion stylization and spatial constraint satisfaction.
1 Introduction↩︎
Human motion generation is a fundamental task in computer graphics, robotics, and virtual reality, where the goal is to synthesize realistic, semantically coherent motion sequences from high-level inputs. In particular, text-to-motion (T2M) diffusion models translate natural language descriptions into motion trajectories, enabling intuitive workflows for animation and embodied AI systems [1]–[7]. Despite their success, diffusion-based T2M models often suffer from semantic drift, a challenge particularly pronounced in temporally structured, long-horizon generation, where the text specifies multiple action segments and the generated trajectory must realize them with consistent semantics and smooth kinematic transitions. This limits their applicability in scenarios demanding fine-grained control over style or body-part behavior.
We posit that within the Gaussian noise space used to initialize diffusion-based motion generators, there exist specific noise instances, termed winning noise tickets, that consistently bias the denoising trajectory toward temporally coherent motions with particular semantics or styles. This notion builds on the extension of the Lottery Ticket Hypothesis from network pruning [8] to diffusion models [9]. Empirically, certain noise samples produce coherent, meaningful motions even under null prompts, while others yield implausible or text-misaligned results even when conditioned on descriptive text (Fig. 1). Thus, the initial noise is not mere randomness, but acts as a latent semantic prior that shapes the generative trajectory. While similar observations have been made for image diffusion [9], the effect is especially impactful in motion generation, where training data is orders of magnitude scarcer (approximately 15k samples vs.billions of images), making models more sensitive to the choice of initial noise.
Prior works that optimize initial noise in motion diffusion do so to satisfy physical or geometric constraints [10], [11]. Because their objectives are defined over predefined spatial targets rather than the text prompt itself, the optimized noise may fulfill the physical goal while still producing motion that drifts from the intended textual meaning. We address this gap by directly optimizing noise against text–motion semantic similarity, and ask: Can we identify and refine the winning noise tickets that best embody the semantics of a given text prompt?
To address this, we introduce WInning Noise Retrieval and Optimization (WINRO), a training-free and model-agnostic framework that improves text–motion alignment by discovering and refining winning noise tickets before the diffusion process. Selecting a semantically aligned noise requires associating each noise sample with a descriptor that (i) captures motion semantics and (ii) is comparable with text. We achieve this by constructing a noise dictionary. We sample a random Gaussian noise and use a frozen diffusion model to generate a motion under a null prompt. A pretrained text–motion retrieval model then encodes the motion into a feature vector in a shared text–motion embedding space, which we store with the noise as a dictionary entry. Given an input text prompt, WINRO retrieves the noise whose motion feature is closest to the text embedding in this space, providing a semantically favorable initialization. The retrieved noise is then refined through a KL-regularized objective that minimizes the residual text–motion discrepancy while preserving the Gaussian prior. Optionally, a trained LoRA-based Noise Refiner can amortize this iterative refinement into a single forward pass. The key contributions are:
We identify the Winning Noise Ticket in motion diffusion, showing that certain initial noises inherently bias denoising trajectories toward specific motion semantics.
We propose WINRO, a training-free framework for motion-specific semantic noise retrieval and refinement, featuring a null-prompt motion dictionary, frame-adjusted retrieval across variable durations, and KL-regularized optimization in the text–motion embedding space.
We optionally extend WINRO with a LoRA-based Noise Refiner that, once trained, amortizes test-time optimization into a single forward pass without updating the base model weights.
We demonstrate consistent improvements in text–motion fidelity across diffusion backbones on HumanML3D, improved long-horizon temporal robustness on the MTT benchmark, and generalization to motion stylization and spatial constraint satisfaction.
2 Related Work↩︎
2.0.0.1 Motion Generation Models.
Advances in motion-capture and video-based reconstruction [12]–[14] have driven interest in generating realistic human motion. Early methods focused on motion prediction or interpolation [15], [16], later extending to action-conditioned synthesis [17], [18]. More recently, text-to-motion (T2M) diffusion models [3], [6], [19], [20] have emerged as powerful tools for natural-language-driven motion generation. Notably, MotionDiffuse [21], MDM [5], MotionLCM [22], and the skeleton-aware latent diffusion model SALAD [23] achieve strong realism but treat initial noise as an unstructured random seed, leaving its semantic role unexplored.
2.0.0.2 Motion-Text Retrieval.
Motion-text retrieval models [24]–[26] learn aligned multimodal spaces for semantic comparison. Retrieval-augmented generation (RAG) frameworks such as ReMoDiffuse [27] retrieve motion exemplars to condition the denoising process, improving fidelity through external references [28], [29]. Rather than retrieving exemplar motions, our method retrieves semantically meaningful initial noises whose induced null-prompt motions align with the input text, making it complementary to exemplar-based retrieval.
2.0.0.3 Initial Noise Selection and Optimization.
Recent studies show that the initial noise in diffusion models affects generated outcomes [30], [31], motivating noise optimization [32]–[35] and selection [9], [36]. The Lottery Ticket Hypothesis in Denoising [9] suggests that certain “winning” noise patterns bias denoising toward specific semantics. In motion generation, such biases are consequential, as the initial noise can affect temporal semantic consistency. Accordingly, in the motion domain, works such as DNO [10], ProgMoGen [11], and PINO [37] optimize noise via physical or geometric constraints. In this research, we target text–motion semantic alignment, retrieving noise from a precomputed dictionary and refining it against the text prompt.
3 Winning Noise Tickets for Motion Diffusion↩︎
3.1 Diffusion-Based Motion Generation↩︎
We build upon diffusion-based text-to-motion (T2M) models, which generate motion by progressively denoising Gaussian noise. A motion sequence \(\mathbf{x}\in \mathbb{R}^{F \times D}\) has \(F\) frames and \(D{=}263\) features [19]. The forward process gradually corrupts clean motion \(\mathbf{x}_0\) into noise \(\mathbf{x}_T\) over \(T\) steps: \[q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) = \mathcal{N}(\mathbf{x}_t; \sqrt{\alpha_t}\mathbf{x}_{t-1}, (1-\alpha_t)\mathbf{I})\,,\] where \(\alpha_t\) is the noise schedule. With a sufficiently long schedule, we approximate \(\mathbf{x}_T \!\sim\! \mathcal{N}(\mathbf{0}, \mathbf{I})\). The denoising network \(G_\theta\) learns to invert this process: \[\mathbf{x}_0 = G_\theta(\mathbf{x}_T, c) = [f_0 \circ f_1 \cdots \circ f_T](\mathbf{x}_T, c)\,, \label{eq:generation}\tag{1}\] where \(f_t\) is the \(t\)-th denoising update and \(c\) is a conditioning input such as text. Models like MDM [5] operate in the motion domain, while latent variants (, MLD [2]) denoise compact motion latents. With deterministic samplers (, DDIM [38]), there is a one-to-one mapping between the initial noise \(\mathbf{x}_T\) and the generated motion \(\mathbf{x}_0\), making the initial noise a controllable factor.
3.2 Theoretical Perspective: Why Noise Encodes Motion Semantics↩︎
Under deterministic sampling, diffusion defines a continuous map from initial noise to a motion trajectory, so nearby noises tend to produce similar motions. Composed with a retrieval encoder, the resulting text–motion alignment score varies smoothly over the noise space; hence high-alignment initial noise form regions rather than isolated points, enabling both retrieval and local refinement.
3.2.0.1 Setup.
Let \(\mathcal{F}_\text{Motion}\) and \(\mathcal{F}_\text{Text}\) be the motion–text retrieval encoders. For deterministic sampling, define the cosine alignment score \[S_c(\mathbf{x}_T)= \frac{ \mathcal{F}_\text{Motion}(G_\theta(\mathbf{x}_T,c))^\top \mathcal{F}_\text{Text}(c) }{ \lVert \mathcal{F}_\text{Motion}(G_\theta(\mathbf{x}_T,c))\rVert\, \lVert \mathcal{F}_\text{Text}(c)\rVert }\in[-1,1]\,.\] We use the null score \(\tilde{S}_c(\mathbf{x}_T)\) to describe intrinsic semantics under unconditional generation, defined by setting the diffusion condition to \(c{=}\varnothing\) while keeping the query text embedding \(\mathcal{F}_\text{Text}(c)\). We call \(\mathbf{x}_T\) a winning noise ticket for \(c\) (level \(\gamma\)) if \(\tilde{S}_c(\mathbf{x}_T)\ge\gamma\).
3.2.0.2 Smoothness implies semantic regions.
If the deterministic sampler and \(\mathcal{F}_\text{Motion}\) are locally Lipschitz, then \(\tilde{S}_c\) is locally Lipschitz in \(\mathbf{x}_T\). Therefore, whenever \(\tilde{S}_c(\mathbf{x}_T^\star)\ge \gamma\), there exists a radius \(r>0\) such that all \(\mathbf{x}_T\) with \(\lVert \mathbf{x}_T-\mathbf{x}_T^\star\rVert\le r\) also satisfy \(\tilde{S}_c(\mathbf{x}_T)\ge \gamma/2\). This locality motivates (i) best-of-\(N\) dictionary retrieval and (ii) small, prior-preserving refinements (Sec. 4).
3.3 Evidence of Winning Noise Tickets↩︎
To examine whether initial noise influences motion semantics, we analyze T2M generation using MDM [5] and a text–motion retrieval model TMR [24], measuring text–motion similarity as cosine similarity in the shared embedding space.
Figure 2: \(z\)-score heatmaps of action-specific tendencies under four fixed noise seeds. For each seed and prompt group, we generate ten motions with varied prompts, compute the mean TMR similarity per category, and apply per-seed \(z\)-score normalization.. a — Action category, b — Action direction, c — Action parts
Figure 3: Similarity-score histograms for the fixed prompts “the person walks/runs/jumps.” For each prompt, we compute TMR similarity scores over 1,000 initial noise seeds.. a — “walk”, b — “run”, c — “jump”
3.3.0.1 Bias Patterns Under Fixed Noise.
We fix four random noise seeds and measure text–motion similarity across prompt groups targeting action categories, directions, and body parts. Fig. 2 shows clear seed-specific preferences, especially for direction and body-part movement (Fig. 2 (b), 2 (c)), indicating distinct semantic regions in the noise space.
3.3.0.2 Alignment Variability Under Noise Variation.
Conversely, fixing three prompts and varying the noise over 1,000 seeds reveals prompt-dependent sensitivity (Fig. 3): some prompts have only a small subset of highly compatible seeds, while others show broader variability. This confirms that prompt-specific winning noise tickets exist, motivating retrieval and refinement.
We build on these observations in Sec. 4 by retrieving promising seeds from a null-prompt dictionary (Sec. 4.1) and refining them with a KL-regularized objective that preserves the Gaussian prior (Sec. 4.2).
4 Winning Noise Retrieval and Optimization↩︎
We propose to exploit this semantic bias of initial noise with two core stages: (i) retrieving a semantically favorable noise seed from a null-prompt noise dictionary built in a text–motion embedding space, yielding WIN-FAR, a training-free and real-time variant, and (ii) optionally refining that seed by optimizing a KL-regularized text–motion similarity objective, yielding the full WINRO pipeline, which remains training-free but is iterative and therefore suited for offline or quality-critical use. Under deterministic sampling, the alignment varies smoothly with \(\mathbf{x}_T\) (Sec. 3.2), so retrieval provides a good initialization for local refinement. Both stages are training-free and model-agnostic.
4.1 Stage 1: Winning Noise Ticket Retrieval↩︎
We build a noise dictionary \(\mathcal{D}\) using a frozen diffusion model \(G_\theta\) and a pretrained motion–text retrieval model (Fig. 4). This construction is a one-time offline cost for each backbone, after which the dictionary is reused across all queries. Per-query inference requires only a retrieval lookup and optional refinement, without additional diffusion runs for dictionary construction. For each entry \(i\in[1,\cdots,N]\), we sample \(\mathbf{x}^i_T\!\sim\!\mathcal{N}(\mathbf{0},\mathbf{I})\), generate a null-prompt motion \(\mathbf{x}^i_0=G_\theta(\mathbf{x}^i_T,\varnothing)\), and store its retrieval feature \(\mathbf{z}^i_l=\mathcal{F}_\text{Motion}(\mathbf{x}^i_0)\): \[\mathcal{D}_{reg}=\{(\mathbf{z}^i_l,\mathbf{x}^i_T)\}_{i=1}^N\,.\] To support retrieval across varying target durations, we introduce frame-adjusted retrieval (FAR). Specifically, we encode truncated motions at multiple lengths \(l'\!\in[m,M]\) and store their length-specific retrieval features: \[\mathcal{D}_{adj}=\{(\mathbf{z}^i_{l'},\mathbf{x}^i_T)\mid i\in[1,N],\,l'\in[m,M]\}\,.\]
4.1.0.1 Why retrieval works.
Let \(\tilde{S}_c(\mathbf{x}_T)\) denote the null-prompt alignment score used for retrieval (Sec. 3.2). If \(p_{c,\gamma}=\mathbb{P}(\tilde{S}_c(\mathbf{x}_T)\ge\gamma)\), then for \(N\) independent samples, \(\mathbb{P}(\max_{i\in[N]}\tilde{S}_c(\mathbf{x}_T^{(i)})\ge\gamma)=1-(1-p_{c,\gamma})^N\). Thus, the probability of sampling a winning region increases with dictionary size \(N\), but with diminishing returns. Moreover, null-prompt retrieval can transfer to text prompts when conditioning acts as a limited perturbation in the retrieval space, consistent with classifier-free guidance mixing unconditional and conditional predictions. The detailed proof is provided in the supplementary materials.
At retrieval time, we compute the cosine similarity between each dictionary feature and the text embedding: \[s(\mathbf{z}, c) = \frac{\mathbf{z}\cdot \mathcal{F}_\text{Text}(c)}{\lVert \mathbf{z}\rVert \lVert \mathcal{F}_\text{Text}(c) \rVert}\,,\] where \(\mathcal{F}_\text{Text}\) is the text encoder of the retrieval model. We rank all entries by similarity and define the top-\(k\) candidate set: \[\mathcal{N}_k(c) = \operatorname*{Top\text{-}k}_{(\mathbf{z}^i_l, \mathbf{x}^i_T) \in \mathcal{D}}\, s(\mathbf{z}^i_l, c)\,. \label{eq:retrieval}\tag{2}\] where \(\mathcal{D}\in\{\mathcal{D}_{\mathrm{reg}},\mathcal{D}_{\mathrm{adj}}\}\). The retrieved feature \(\mathbf{z}^{\dagger}\) is then sampled uniformly from \(\mathcal{N}_k(c)\). When \(k{=}1\), this reduces to deterministic argmax retrieval. Along with the provided text prompt \(c\), the initial noise \(\mathbf{x}^{\dagger}_T\) corresponding to \(\mathbf{z}^{\dagger}\) is used to generate the final motion (Fig. 5). We denote retrieval using \(\mathcal{D}_{reg}\) as “Retrieval” and using \(\mathcal{D}_{adj}\) as “Frame-adjusted retrieval (FAR)”.
4.1.0.2 Noise perturbation.
To avoid over-concentrating on a small subset of seeds while preserving semantics, we perturb the retrieved noise with small Gaussian jitter: \[\mathbf{x}'_T = \mathbf{x}^{\dagger}_T + \boldsymbol{\eta}, \quad \boldsymbol{\eta} \sim \mathcal{N}(\mathbf{0}, \sigma^2 \mathbf{I})\,, \label{eq:perturbation}\tag{3}\] where \(\sigma\) controls the stochasticity.
4.2 Stage 2: Winning Noise Ticket Refinement↩︎
We refine the retrieved noise \(\mathbf{x}^{\dagger}_T\) by maximizing text–motion similarity (Fig. 6): \[\mathcal{L}_{sim} = 1 - s(\mathcal{F}_\text{Motion}(\mathbf{x}_0), c) \,.\] As in retrieval, \(\mathbf{x}_0\) can be truncated to the target length before encoding. Directly optimizing this similarity loss can move the noise outside the Gaussian typical set (e.g., by drifting its empirical mean/variance), where the diffusion model is poorly calibrated and motion quality/diversity may degrade. We therefore add a KL-divergence term that penalizes deviations from the standard normal: \[\mathcal{L}_{KL} = D_{KL}\!\left(\mathcal{N}(\mu(\mathbf{x}_T), \sigma^2(\mathbf{x}_T)) \| \mathcal{N}(0, 1)\right)\,,\] where \(\mu(\mathbf{x}_T)\) and \(\sigma^2(\mathbf{x}_T)\) are the empirical mean and variance of the noise tensor. The full objective is: \[\mathcal{L} = \mathcal{L}_{sim} + \lambda \mathcal{L}_{KL}\,, \label{eq:optimization}\tag{4}\] where \(\lambda\) is the balancing parameter. We freeze the motion diffusion model and optimize only the initial noise to obtain \(\mathbf{x}^*_T\) aligned with the user-provided text.
4.3 LoRA-Based Noise Refiner↩︎
Iterative refinement runs diffusion sampling at every optimization step, which is costly (see AITS in Table 2). As an amortized alternative, we train a Low-Rank Adaptation (LoRA) [39] module that predicts a noise correction in a single forward pass, following [35]. Unlike the retrieval and optimization stages, the LoRA refiner requires an additional training step and is therefore not training-free, but it enables real-time inference: \[\hat{\mathbf{x}}_T = \mathbf{x}_T + R_\phi(\mathbf{x}_T)\,.\] Here, \(R_\phi\) predicts a residual correction so that \(\hat{\mathbf{x}}_T\) approximates the refined noise produced by retrieval and iterative optimization. Only the LoRA weights \(\phi\) are updated while all diffusion parameters \(\theta\) remain frozen. During training, \(\mathbf{x}_T\) is sampled from the standard Gaussian prior rather than from the retrieval dictionary, so no explicit noise-to-noise supervision pairs are required. The refiner minimizes a loss analogous to Eq. 4 : \[\begin{align} \mathcal{L}_{LoRA} &= 1 - \cos\!\left( \mathcal{F}_\text{Motion}(G_\theta(\hat{\mathbf{x}}_T, c)), \mathcal{F}_\text{Text}(c) \right) + \lambda \lVert R_\phi(\mathbf{x}_T) \rVert^2_2, \end{align}\] where the second term regularizes the correction magnitude, helping the corrected noise maintain proximity to the Gaussian prior. At inference, the refiner can be applied either to random noise or to the retrieved \(\mathbf{x}^{\dagger}_T\) from Stage 1.
5 Experiments↩︎
In this section, we first verify the intrinsic semantics of winning noise tickets through null-prompt generation (Sec. 5.1), then evaluate the full WINRO pipeline on text-conditioned benchmarks (Sec. 5.2). We further examine compositional and long-duration generation (Sec. 5.3) and demonstrate task adaptability to motion stylization and controllable generation (Sec. 5.4). Experimental details are provided in the supplementary material (Sec. B).
5.1 Intrinsic Semantics of Winning Noise Tickets↩︎
5.1.0.1 Generation with Null Prompt.
To isolate the effect of noise selection from text conditioning, we first retrieve winning noise tickets using text embeddings but generate motions under a null prompt, so that any observed semantic alignment must originate from the noise itself. Following Section 4.1, we construct a noise dictionary containing 10,000 randomly sampled noise–motion pairs, where each feature is computed by encoding a null-prompt motion. For efficiency, truncated samples are created every 4 frames, resulting in motions of lengths \(40, 44, \ldots, 196\) for Frame-Adjusted Retrieval (FAR). All noises are drawn from a standard Gaussian distribution, and no ground-truth motion data is used.
We conduct experiments on the HumanML3D dataset [19]. Two representative diffusion backbones are adopted: MDM-50step [5] and MotionLCM [22]. The pretrained TMR model [24] is used to embed both motions and texts into a shared 256-dimensional feature space for retrieval. Unless otherwise noted, we use top-1 retrieval (\(k{=}1\)). Details are provided in the supplementary material.
Following prior works [2], [5], we report FID, R-Precision, Diversity, Multimodal Distance (MMDist), MultiModality, and additionally include AITS. FID measures the distributional distance between generated and real motions in a learned feature space, reflecting overall motion quality. R-Precision and MMDist evaluate text–motion alignment, while Diversity measures variation across generated motions. MultiModality measures variation among motions generated from the same text; see Section C. AITS (Average Inference Time per Sentence) reports batch-size-one wall-clock inference time in seconds, excluding data/model loading and the one-time noise dictionary construction. All reported experiments use a single NVIDIA H100 GPU, except where explicitly stated.
| Method | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Diversity \(\rightarrow\) | MMDist \(\downarrow\) |
|---|---|---|---|---|---|---|
| GT | \(0.002^{\scriptstyle\pm.000}\) | \(0.511^{\scriptstyle\pm.002}\) | \(0.704^{\scriptstyle\pm.003}\) | \(0.798^{\scriptstyle\pm.002}\) | \(9.458^{\scriptstyle\pm.088}\) | \(2.970^{\scriptstyle\pm.007}\) |
| MDM-50step | \(4.956^{\scriptstyle\pm.150}\) | \(0.037^{\scriptstyle\pm.002}\) | \(0.070^{\scriptstyle\pm.002}\) | \(0.102^{\scriptstyle\pm.004}\) | \(8.101^{\scriptstyle\pm.087}\) | \(8.932^{\scriptstyle\pm.039}\) |
| w/ WIN-R | \(4.500^{\scriptstyle\pm.228}\) | \(0.250^{\scriptstyle\pm.006}\) | \(0.385^{\scriptstyle\pm.009}\) | \(0.474^{\scriptstyle\pm.008}\) | \(8.482^{\scriptstyle\pm.068}\) | \(5.540^{\scriptstyle\pm.057}\) |
| w/ WIN-FAR | \(\mathbf{4.458}^{\scriptstyle\pm.200}\) | \(\mathbf{0.274}^{\scriptstyle\pm.006}\) | \(\mathbf{0.415}^{\scriptstyle\pm.006}\) | \(\mathbf{0.507}^{\scriptstyle\pm.007}\) | \(\mathbf{8.547}^{\scriptstyle\pm.059}\) | \(\mathbf{5.325}^{\scriptstyle\pm.046}\) |
| MotionLCM | \(8.460^{\scriptstyle\pm.072}\) | \(0.034^{\scriptstyle\pm.001}\) | \(0.067^{\scriptstyle\pm.002}\) | \(0.099^{\scriptstyle\pm.002}\) | \(6.646^{\scriptstyle\pm.064}\) | \(8.313^{\scriptstyle\pm.010}\) |
| w/ WIN-R | \(1.681^{\scriptstyle\pm.071}\) | \(0.404^{\scriptstyle\pm.002}\) | \(0.597^{\scriptstyle\pm.004}\) | \(0.710^{\scriptstyle\pm.003}\) | \(8.386^{\scriptstyle\pm.029}\) | \(3.613^{\scriptstyle\pm.020}\) |
| w/ WIN-FAR | \(\mathbf{1.569}^{\scriptstyle\pm.071}\) | \(\mathbf{0.424}^{\scriptstyle\pm.003}\) | \(\mathbf{0.617}^{\scriptstyle\pm.004}\) | \(\mathbf{0.728}^{\scriptstyle\pm.004}\) | \(\mathbf{8.424}^{\scriptstyle\pm.039}\) | \(\mathbf{3.485}^{\scriptstyle\pm.021}\) |
1pt
As shown in Table 1, retrieved winning noise tickets enable both MDM-50step and MotionLCM to produce semantically aligned motions even under null prompts, where baselines yield near-zero alignment. This validates the Winning Noise Ticket Hypothesis (Sec. 3) by showing that semantic information can be partially encoded in the initial noise itself, independent of text conditioning.
5.2 Generation with Refined Winning Noise Tickets↩︎
We next examine whether this effect carries over to text-conditioned generation with the full WINRO pipeline. The retrieved noise \(\mathbf{x}_T^{\dagger}\) from the FAR dictionary is optimized using the KL-regularized objective (Eq. 4 ), yielding a refined noise \(\mathbf{x}_T^{*}\). The optimization is performed at inference time without modifying model parameters. Details are in the supplementary material. Alternatively, a LoRA Noise Refiner can approximate this refinement in a single feed-forward pass; when combined with retrieval (WIN-RLoRA), it amortizes the iterative optimization while preserving the benefit of a semantically aligned initialization.
| Method | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) | AITS \(\downarrow\) |
|---|---|---|---|---|---|---|---|
| GT | \(0.002^{\scriptstyle\pm.000}\) | \(0.511^{\scriptstyle\pm.002}\) | \(0.704^{\scriptstyle\pm.003}\) | \(0.798^{\scriptstyle\pm.002}\) | \(9.458^{\scriptstyle\pm.088}\) | \(2.970^{\scriptstyle\pm.007}\) | - |
| MDM-50step | \(0.438^{\scriptstyle\pm.009}\) | \(0.469^{\scriptstyle\pm.002}\) | \(0.661^{\scriptstyle\pm.002}\) | \(0.763^{\scriptstyle\pm.002}\) | \(10.025^{\scriptstyle\pm.066}\) | \(3.258^{\scriptstyle\pm.009}\) | 0.265 |
| w/ WIN-R | \(0.250^{\scriptstyle\pm.007}\) | \(0.505^{\scriptstyle\pm.002}\) | \(0.702^{\scriptstyle\pm.003}\) | \(0.799^{\scriptstyle\pm.003}\) | \(10.303^{\scriptstyle\pm.048}\) | \(3.047^{\scriptstyle\pm.012}\) | – |
| w/ WIN-FAR | \(0.248^{\scriptstyle\pm.009}\) | \(0.512^{\scriptstyle\pm.003}\) | \(0.706^{\scriptstyle\pm.003}\) | \(0.804^{\scriptstyle\pm.002}\) | \(10.295^{\scriptstyle\pm.033}\) | \(3.013^{\scriptstyle\pm.011}\) | 0.268 |
| w/ WINRO | \(0.115^{\scriptstyle\pm.002}\) | \(0.539^{\scriptstyle\pm.002}\) | \(0.727^{\scriptstyle\pm.004}\) | \(0.817^{\scriptstyle\pm.002}\) | \(9.657^{\scriptstyle\pm.026}\) | \(2.883^{\scriptstyle\pm.010}\) | 81.408 |
| w/ WIN-RLoRA | \(\mathbf{0.102}^{\scriptstyle\pm.002}\) | \(\mathbf{0.546}^{\scriptstyle\pm.003}\) | \(\mathbf{0.741}^{\scriptstyle\pm.003}\) | \(\mathbf{0.832}^{\scriptstyle\pm.002}\) | \(\mathbf{9.577}^{\scriptstyle\pm.025}\) | \(\mathbf{2.846}^{\scriptstyle\pm.007}\) | 0.280 |
| MotionLCM | \(0.093^{\scriptstyle\pm.004}\) | \(0.548^{\scriptstyle\pm.003}\) | \(0.741^{\scriptstyle\pm.003}\) | \(0.835^{\scriptstyle\pm.002}\) | \(9.566^{\scriptstyle\pm.069}\) | \(2.763^{\scriptstyle\pm.008}\) | 0.029 |
| w/ WIN-R | \(0.091^{\scriptstyle\pm.004}\) | \(0.566^{\scriptstyle\pm.003}\) | \(0.762^{\scriptstyle\pm.002}\) | \(0.852^{\scriptstyle\pm.002}\) | \(9.424^{\scriptstyle\pm.028}\) | \(2.699^{\scriptstyle\pm.007}\) | – |
| w/ WIN-FAR | \(0.083^{\scriptstyle\pm.003}\) | \(0.572^{\scriptstyle\pm.003}\) | \(0.767^{\scriptstyle\pm.002}\) | \(0.855^{\scriptstyle\pm.001}\) | \(\mathbf{9.474}^{\scriptstyle\pm.028}\) | \(2.674^{\scriptstyle\pm.006}\) | 0.033 |
| w/ WINRO | \(\mathbf{0.072}^{\scriptstyle\pm.002}\) | \(\mathbf{0.580}^{\scriptstyle\pm.002}\) | \(\mathbf{0.775}^{\scriptstyle\pm.002}\) | \(\mathbf{0.861}^{\scriptstyle\pm.001}\) | \(9.242^{\scriptstyle\pm.026}\) | \(\mathbf{2.609}^{\scriptstyle\pm.005}\) | 6.870 |
| w/ WIN-RLoRA | \(0.083^{\scriptstyle\pm.002}\) | \(\mathbf{0.580}^{\scriptstyle\pm.002}\) | \(0.772^{\scriptstyle\pm.001}\) | \(0.860^{\scriptstyle\pm.001}\) | \(9.550^{\scriptstyle\pm.032}\) | \(2.649^{\scriptstyle\pm.005}\) | 0.043 |
1pt
Table 2 summarizes the text-conditioned generation results. Each stage of WINRO yields consistent improvements across both diffusion backbones. Plain retrieval (WIN-R) already improves text alignment over the baselines, reducing the FID for MDM-50step from \(0.438\) to \(0.250\). Frame-adjusted retrieval with perturbation (WIN-FAR) further improves upon WIN-R (FID \(0.248\), R-Top1 \(0.512\)); the perturbation (Eq. 3 ) restores local diversity around the retrieved noise, mitigating the distribution shift caused by discrete selection. Subsequent refinement (WINRO) further strengthens alignment: for MDM-50step, the FID decreases to \(0.115\) and R-Top1 rises to \(0.539\); for MotionLCM, WINRO achieves the best R-Top1 of \(0.580\) and MMDist of \(2.609\). Since the optimization objective already incorporates KL regularization (Eq. 4 ), we omit additional perturbation during refinement. LoRA-based refinement with retrieval and perturbation (WIN-RLoRA) provides an amortized alternative to full WINRO refinement, which is iterative and suited for offline or quality-critical use. Although WIN-RLoRA requires an additional training step and is therefore not training-free, it avoids iterative test-time optimization. It further improves over WINRO on MDM-50step and remains competitive on MotionLCM, while reducing AITS from \(81.408\) to \(0.280\) and from \(6.870\) to \(0.043\), respectively.
Note that the retrieval model (TMR [24]) used for dictionary construction and refinement is independent of the evaluation model used for the HumanML3D metrics. We further verify that improvements are not specific to the choice of retrieval backbone by repeating the experiment with an alternative encoder; details are provided in the supplementary material. The noise dictionary is constructed once offline per backbone: approximately 4 minutes for MDM-50step and 2 minutes for MotionLCM on a single A100 GPU, after which per-query inference requires only retrieval (negligible) and optional refinement. Additional analyses and ablations are provided in the supplementary material.
5.2.0.1 Qualitative Results.
Fig. 7 compares motions generated with random noise, retrieved noise (WIN-FAR), and refined noise (WINRO) using MDM-50step.
In the first example, the prompt demands a clockwise walk. Random noise produces a generic walking motion with no clear trajectory, while the retrieved noise captures a partial arc but eventually turns in the wrong direction. In contrast, WINRO generates a smooth clockwise arc that closely matches the intended motion. In the second example, the prompt requires the person to bend down and crawl to the left. Random noise yields a crawling motion that ends in an unintended standing pose, and the retrieved noise produces only slight leftward motion. With WINRO refinement, the model is able to generate a consistent leftward crawl as instructed.
These qualitative results illustrate the complementary roles of retrieval and refinement: retrieval selects a noise seed with an inherent bias toward the desired behavior, while refinement further adjusts it to satisfy the prompt more faithfully. Additional examples are in the supplementary material.
5.3 Compositional and Long-Duration Generation.↩︎
Practical motion generation often requires composing multiple actions within a single sequence: actions may be arranged in temporal order (, “walk forward, then sit down”) or performed simultaneously on different body parts (, “wave the right hand while walking”). The multi-track timeline (MTT) formulation [40] unifies these settings by assigning each text prompt to a temporal interval on parallel tracks, allowing overlapping and sequential actions to be specified in a single structured input. Generating faithful motions from such timelines is challenging because the model must maintain semantic alignment across all intervals, especially as the total duration grows. WINRO naturally fits this setting, since it can also be applied to multiple interval-level text constraints.
5.3.0.1 Method.
We evaluate on the MTT benchmark [40] under two settings: (1) single-text compositional generation (\(\le\)196 frames) using MDM-50step, where the full WINRO pipeline is applied, and (2) multi-track long-duration generation using STMC [40], where only the optimization stage is used because constructing a noise dictionary from variable-length null-prompt outputs is computationally impractical at extended durations. For the multi-track setting, the similarity loss is defined as the average text–motion similarity over timeline intervals. Each motion subsequence is encoded and compared with its corresponding text prompt, enabling the optimization to maintain semantic consistency across segments.
5.3.0.2 Results.
As shown in Table 3, both configurations yield consistent improvements, with particularly large gains in long-duration generation (R@1: \(33.6 \to 62.1\)). These results suggest that winning noise tickets carry semantic priors that remain effective even in extended and compositional settings, where semantic drift is otherwise difficult to control. Fig. 8 shows qualitative examples: STMC alone suffers from semantic drift in later segments, whereas WIN-O maintains faithful alignment with each text prompt throughout the timeline.
| Input type | Per-crop semantic correctness | Realism | ||||||
| Method | #tracks | #crops | R@1 \(\uparrow\) | R@3 \(\uparrow\) | M2T \(\uparrow\) | M2M \(\uparrow\) | FID \(\downarrow\) | Trans. \(\downarrow\) |
| Ground Truth | – | – | 55.0 | 73.3 | 0.748 | 1.000 | 0.000 | 1.5 |
| MDM-50step | Single | Single | 12.2 | 24.6 | 0.571 | 0.561 | 0.543 | 2.2 |
| w/ WINRO | Single | Single | 20.7 | 38.1 | 0.661 | 0.597 | 0.492 | 2.1 |
| w/ STMC [40] | Multi | Multi | 33.6 | 55.3 | 0.687 | 0.656 | 0.480 | 1.7 |
| w/ STMC + WIN-O | ” | ” | 62.1 | 79.1 | 0.807 | 0.702 | 0.452 | 1.4 |
3pt
![Figure 8: Comparison of motions generated by STMC [40] with and without initial noise refinement on multi-track long-duration generation. STMC produces motions that are not faithful to the text prompt of each segment (, missing throw or hop actions), while refining the initial noise improves alignment.](/article/2607.06843/img/14_stmc_qualitative_v2.png "fig:")
5.4 Task Adaptability↩︎
As noted in Sec. 4, the retrieval and optimization stages are modular: the dictionary can be replaced with a domain-specific one or omitted, and the optimization objective can incorporate additional loss terms. We demonstrate this flexibility on two tasks.
5.4.1 Motion Stylization.↩︎
Motion stylization aims to generate motions that follow a given text prompt while exhibiting a particular movement style (, performing a “walking” motion in a “proud” or “depressed” manner), guided by a reference style motion.
![Figure 9: Overview of WINRO for motion stylization. The generic noise dictionary is replaced with a style-specific one built from 100STYLE motions [41] via DDIM inversion. The optimization jointly minimizes semantic loss \mathcal{L}_\text{sim}, style loss \mathcal{L}_\text{style}, and KL regularization term \lambda\mathcal{L}_\text{KL}.](/article/2607.06843/img/11_stylization_v3.png "fig:")
5.4.1.1 Method.
We extend WINRO by constructing a style-specific noise dictionary \(\mathcal{D}_s\): we apply DDIM inversion to motions from the 100STYLE dataset [41], so that each entry inherently encodes stylistic attributes. Given a text prompt, we retrieve the best-matching entry from \(\mathcal{D}_s\) using the same text-based retrieval as in Eq. 2 , then refine the retrieved noise by jointly optimizing the semantic loss \(\mathcal{L}_{sim}\) (Eq. 4 ) and a style loss \(\mathcal{L}_{style} = 1 - \cos(\mathcal{F}_\text{Style}(\mathbf{x}^{\dagger}_0), \mathcal{F}_\text{Style}(\mathbf{x}_s))\), where \(\mathbf{x}_s\) is the reference style motion. We compare WIN-O (optimization only from random noise) and WINRO (retrieval from \(\mathcal{D}_s\) + optimization). Fig. 9 illustrates the pipeline. Since WINRO is model-agnostic, it can be applied to any diffusion-based stylization backbone. We adopt SMooDi [42] for its public availability.
| Method | FID \(\downarrow\) | Foot Skating \(\downarrow\) | MMDist \(\downarrow\) | R-Top3 \(\uparrow\) | Diversity \(\rightarrow\) | SRA \(\uparrow\) |
|---|---|---|---|---|---|---|
| SMooDi [42] | 1.609 | 0.124 | 4.477 | 0.571 | 9.235 | 72.418 |
| w/ WIN-O | 0.551 | 0.104 | 3.550 | 0.717 | 9.071 | 68.029 |
| w/ WINRO | 1.066 | 0.129 | 4.055 | 0.633 | 7.766 | 75.818 |
5.4.1.2 Results.
Table 4 shows that both variants improve upon the SMooDi backbone without modifying its parameters. WIN-O, starting from random noise, optimizes freely toward text alignment and achieves the best FID (\(0.551\) vs.\(1.609\)), MMDist, and R@3. WINRO, starting from the style-specific dictionary, begins closer to the target style; accordingly, it attains the highest Style Recognition Accuracy (SRA) (\(75.82\) vs.\(72.42\)), confirming stronger style adherence, while text-alignment metrics remain competitive. The two variants thus offer complementary trade-offs: WIN-O prioritizes text fidelity, while WINRO prioritizes style preservation—both operating solely at the noise level.
5.4.1.3 Qualitative Results
We qualitatively compare stylized motions in Fig. 10. In the first example, both SMooDi and our method reproduce the “Rocket” style, but SMoodi’s output drifts semantically, producing a wandering motion instead. Our framework selects a more suitable initial noise, yielding a coherent “jogging in place” motion that respects both style and text. A similar trend is observed in the second example, where the baseline captures style but loses semantic context, while our method maintains alignment between content and style.
5.4.2 Controllable Motion Generation.↩︎
Beyond text semantics, practical applications often require motions to satisfy explicit spatial constraints, such as reaching a specific location or avoiding obstacles. The optimization objective can incorporate such geometric or physical constraints from prior works [10], [11]. Concretely, we augment Eq. 4 with a constraint term \(\beta\,\mathcal{L}_{\mathrm{ctrl}}\) (, the waypoint/target-reaching objective of ProgMoGen [11]). As shown in Fig. 11, this combined objective finds an initial noise that satisfies both textual semantics and external spatial constraints, such as reaching target locations.
![Figure 11: Comparison of motions generated using the initial noise optimized by ProgMoGen [11] and the refined noise produced by the proposed WINRO method. WINRO produces motions that better satisfy both textual and spatial constraints.](/article/2607.06843/img/13_control_v3.png "fig:")
6 Limitations↩︎
Since WINRO does not modify the base model, it cannot address systematic failure modes of the diffusion backbone and is limited to guiding generation within its learned manifold. The iterative refinement stage incurs substantial inference overhead, as each optimization step requires a full diffusion generation pass; the LoRA-based refiner mitigates this for latency-sensitive applications, but requires an additional training step. Finally, the current motion-text retrieval model treats motions holistically, making it difficult to pinpoint fine-grained temporal properties in noise patterns. Investigating retrieval models with temporal localization is a promising direction for future work.
7 Conclusion↩︎
We present WINRO, a training-free and model-agnostic framework that exploits the semantic structure of the initial noise space to improve diffusion-based text-to-motion generation. By retrieving and refining winning noise tickets, WINRO consistently improves text–motion alignment across MDM and MotionLCM without retraining the base model. WINRO also improves temporal robustness on the MTT benchmark. Its modular design generalizes to motion stylization and spatial constraint satisfaction, demonstrating that principled noise selection is a versatile and complementary tool for controllable motion synthesis.
Acknowledgements↩︎
This study was carried out using the TSUBAME4.0 supercomputer at Institute of Science Tokyo.
Supplementary Material
Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation
8 Theoretical Perspective on Winning Noise Tickets↩︎
We address the question why certain noises encode motion semantics through the deterministic (DDIM/ODE) view of diffusion sampling. The key observation is that deterministic sampling defines a flow map from initial noise to a motion trajectory; under mild smoothness assumptions this map is stable, so high semantic alignment cannot occur at isolated points but instead occupies neighborhoods (, semantic regions) in the noise space. For motion, where denoising couples all frames, such neighborhoods naturally correspond to coherent action-level trajectories rather than per-frame artifacts. This, in turn, justifies (i) dictionary search over many seeds and (ii) KL-regularized refinement to preserve the Gaussian prior.
8.0.0.1 Setup.
For a deterministic sampler, let \(\mathbf{x}_0 = G_\theta(\mathbf{x}_T,c)\) denote the generated motion from initial noise \(\mathbf{x}_T\in\mathbb{R}^{d}\) (\(d{=}FD\)) and text condition \(c\). Let \(\mathcal{F}_\text{Motion}\) and \(\mathcal{F}_\text{Text}\) be a motion–text retrieval model’s encoders, and define the normalized embeddings \[\Phi_c(\mathbf{x}_T)=\frac{\mathcal{F}_\text{Motion}(G_\theta(\mathbf{x}_T,c))}{\lVert \mathcal{F}_\text{Motion}(G_\theta(\mathbf{x}_T,c))\rVert},\quad \tau(c)=\frac{\mathcal{F}_\text{Text}(c)}{\lVert \mathcal{F}_\text{Text}(c)\rVert}\,.\] We consider the cosine alignment score \[S_c(\mathbf{x}_T)=\Phi_c(\mathbf{x}_T)^\top \tau(c)\in[-1,1]\,.\] In particular, our retrieval dictionary (Sec. 4.1) uses the null score \(\tilde{S}_c(\mathbf{x}_T)=\Phi_{\varnothing}(\mathbf{x}_T)^\top\tau(c)\), which sets only the diffusion condition to \(\varnothing\) while retaining the text query \(c\).
Definition 1 (Winning region). For a prompt \(c\) and threshold \(\gamma\), define the super-level set \(\mathcal{W}_{c,\gamma}=\{\mathbf{x}_T:\tilde{S}_c(\mathbf{x}_T)\ge\gamma\}\). We call \(\mathbf{x}_T\in\mathcal{W}_{c,\gamma}\) a winning noise ticket (at level \(\gamma\)) for \(c\).
Lemma 1 (Stability of deterministic sampling). View DDIM sampling as solving a probability-flow ODE \(\dot{\mathbf{x}}=f_\theta(\mathbf{x},t,c)\) from \(t{=}1\) to \(t{=}0\) with terminal condition \(\mathbf{x}(1)=\mathbf{x}_T\) and output \(\mathbf{x}(0)=G_\theta(\mathbf{x}_T,c)\). If \(f_\theta(\cdot,t,c)\) is \(L\)-Lipschitz in \(\mathbf{x}\) for all \(t\in[0,1]\), then for any \(\mathbf{x}_T,\mathbf{x}'_T\), \[\lVert G_\theta(\mathbf{x}_T,c)-G_\theta(\mathbf{x}'_T,c)\rVert \le e^{L}\lVert \mathbf{x}_T-\mathbf{x}'_T\rVert\,.\]
Proof. Let \(\mathbf{x}(t)\) and \(\mathbf{x}'(t)\) be the ODE solutions with terminal values \(\mathbf{x}_T\) and \(\mathbf{x}'_T\), and define \(\Delta(t)=\mathbf{x}(t)-\mathbf{x}'(t)\). By Lipschitzness, \(\frac{d}{dt}\lVert \Delta(t)\rVert \le \lVert \dot{\Delta}(t)\rVert \le L\lVert \Delta(t)\rVert\). Grönwall’s inequality gives \(\lVert \Delta(0)\rVert \le e^{L}\lVert \Delta(1)\rVert\), yielding the claim. 0◻ ◻
Assume \(\mathcal{F}_\text{Motion}\) is \(L_F\)-Lipschitz and its output norm is bounded away from \(0\) on the null-prompt generations of interest, so that \(\Phi_{\varnothing}\) is well-defined and locally Lipschitz. Then \(\tilde{S}_c\) is locally Lipschitz in \(\mathbf{x}_T\). In particular, if \(\tilde{S}_c(\mathbf{x}_T^\star)\ge \gamma\) for some \(\mathbf{x}_T^\star\), there exists \(r>0\) such that every \(\mathbf{x}_T\) in the ball \(\{\lVert \mathbf{x}_T-\mathbf{x}_T^\star\rVert\le r\}\) satisfies \(\tilde{S}_c(\mathbf{x}_T)\ge \gamma/2\).
Proof. By Lem. 1 (with \(c{=}\varnothing\)) and Lipschitzness of \(\mathcal{F}_\text{Motion}\), the unnormalized feature map \(\mathbf{x}_T\mapsto \mathcal{F}_\text{Motion}(G_\theta(\mathbf{x}_T,\varnothing))\) is locally Lipschitz. Since normalization is locally Lipschitz away from \(0\), \(\Phi_{\varnothing}\) is locally Lipschitz, and \[\big|\tilde{S}_c(\mathbf{x}_T)-\tilde{S}_c(\mathbf{x}'_T)\big| =\big|(\Phi_{\varnothing}(\mathbf{x}_T)-\Phi_{\varnothing}(\mathbf{x}'_T))^\top\tau(c)\big| \le \lVert \Phi_{\varnothing}(\mathbf{x}_T)-\Phi_{\varnothing}(\mathbf{x}'_T)\rVert\,.\] Choosing \(r\) small enough so that \(\big|\tilde{S}_c(\mathbf{x}_T)-\tilde{S}_c(\mathbf{x}_T^\star)\big|\le \gamma/2\) for all \(\lVert \mathbf{x}_T-\mathbf{x}_T^\star\rVert\le r\) yields the result. 0◻ ◻
Prop. [prop:ticket95region] offers a concrete mechanism: once a seed yields a semantically aligned null-prompt motion, nearby seeds tend to yield similar motions and thus inherit the semantic bias, which matches our empirical locality observations.
Let \(\{\mathbf{x}_T^{(i)}\}_{i=1}^N\) be i.i.d.samples from \(\mathcal{N}(\mathbf{0},\mathbf{I})\). Define \(p_{c,\gamma}=\mathbb{P}(\mathbf{x}_T\in\mathcal{W}_{c,\gamma})\). Then the probability that at least one sampled seed is a winning ticket is \[\mathbb{P}\!\left(\max_{i\in[N]} \tilde{S}_c(\mathbf{x}_T^{(i)})\ge \gamma\right) = 1-(1-p_{c,\gamma})^N\,,\] which increases with \(N\) but saturates as \(N\) grows.
Proof. By independence, the event that no seed lands in \(\mathcal{W}_{c,\gamma}\) has probability \((1-p_{c,\gamma})^N\); take the complement. 0◻ ◻
Assume the conditioning-induced feature shift is bounded in the retrieval space: there exists \(\delta(c)\ge 0\) such that for all \(\mathbf{x}_T\), \(\lVert \Phi_c(\mathbf{x}_T)-\Phi_{\varnothing}(\mathbf{x}_T)\rVert \le \delta(c)\). Then for all \(\mathbf{x}_T\), the conditional score satisfies \[\big|S_c(\mathbf{x}_T)-\tilde{S}_c(\mathbf{x}_T)\big| \le \delta(c)\,.\] Consequently, any seed with \(\tilde{S}_c(\mathbf{x}_T)\ge \gamma+\delta(c)\) also satisfies \(S_c(\mathbf{x}_T)\ge \gamma\).
Proof. Since \(\tau(c)\) is unit-norm, \[\begin{align} |S_c(\mathbf{x}_T)-\tilde{S}_c(\mathbf{x}_T)| &=|(\Phi_c(\mathbf{x}_T)-\Phi_{\varnothing}(\mathbf{x}_T))^\top\tau(c)| \\ &\le \lVert \Phi_c(\mathbf{x}_T)-\Phi_{\varnothing}(\mathbf{x}_T)\rVert \le \delta(c)\,. \end{align}\] The second claim follows immediately. 0◻ ◻
Prop. [prop:null95to95text] explains why our dictionary, built from null-prompt generations, can still yield strong initializations for text-conditioned sampling: if conditioning acts as a bounded perturbation in the retrieval embedding space, then high intrinsic (null) alignment implies high conditional alignment. This assumption is also consistent with classifier-free guidance, where each denoising update explicitly mixes conditional and unconditional predictions, making the null trajectory a persistent component of the final motion.
Consider maximizing alignment under a moment-based Gaussian prior constraint: \[\max_{\mathbf{x}_T}\;S_c(\mathbf{x}_T)\quad \text{s.t.}\quad D_{KL}\!\left(\mathcal{N}(\mu(\mathbf{x}_T), \sigma^2(\mathbf{x}_T)) \| \mathcal{N}(0, 1)\right)\le \varepsilon,\] where \(\mu(\mathbf{x}_T)\) and \(\sigma^2(\mathbf{x}_T)\) are the empirical mean and variance of the noise tensor. The objective in Eq. 4 is the Lagrangian relaxation of this constrained problem (for some multiplier \(\lambda\)), encouraging alignment improvement while keeping the optimized seed close to the Gaussian prior.
Proof. This follows from standard Lagrangian relaxation: introducing \(\lambda\ge 0\) yields the unconstrained objective \(-S_c(\mathbf{x}_T)+\lambda\,D_{KL}(\cdot)\) up to an additive constant, which matches Eq. 4 after converting to a minimization problem. 0◻ ◻
Let \(\mu\) and \(\sigma^2\) be the empirical mean and variance of a noise tensor. Then \[D_{KL}\!\left(\mathcal{N}(\mu, \sigma^2) \| \mathcal{N}(0, 1)\right) = \frac{1}{2}\!\left(\mu^2 + \sigma^2 - 1 - \log \sigma^2\right) \ge \frac{1}{2}\mu^2 + \frac{(\sigma^2-1)^2}{4(1+\sigma^2)}\,,\] so a small KL penalty forces both \(\mu\!\approx\!0\) and \(\sigma^2\!\approx\!1\).
Proof. The closed form is the standard KL divergence between univariate Gaussians. For the bound, it suffices to show that for all \(x>0\), \[x-1-\log x \ge \frac{(x-1)^2}{2(1+x)}\,.\] Let \(\phi(x)=x-1-\log x-\frac{(x-1)^2}{2(1+x)}\). We have \(\phi(1)=\phi'(1)=0\) and \[\phi''(x)=\frac{1}{x^2}-\frac{4}{(1+x)^3} =\frac{x^3-x^2+3x+1}{x^2(1+x)^3}\ge 0\,,\] so \(\phi\) is convex and minimized at \(x{=}1\), implying \(\phi(x)\ge 0\). Substituting \(x=\sigma^2\) completes the proof. 0◻ ◻
9 Implementation Details↩︎
9.1 Base Models and Sampling↩︎
We adopt MDM-50step [5] and MotionLCM-V2 [22] as base generators. For MDM, we use DDIM sampling with 50 steps for both text and null prompts. For MotionLCM, we adopt the 1-step inference setting. Classifier-free guidance is applied in the same manner as in the original implementations.
9.2 Noise Dictionary and Retrieval↩︎
To construct the noise dictionary used in Stage 1, we sample 10,000 noise tensors from the standard Gaussian distribution. To reduce the retrieval cost, we create truncated versions of each noise at 4-frame intervals, and use motions truncated to \(40, 44, \dots, 196\) frames for Frame-Adjusted Retrieval (FAR). For motion–text retrieval in both stages, we use TMR [24], which encodes motion and text tokens into 256-dimensional feature vectors. All experiments use the official pretrained TMR weights.
9.3 Optimization Setup↩︎
Given the retrieved initial noise, we optimize it using the objective described in the main paper. The balancing weight \(\lambda\) scales with the dimensionality of the noise space. On HumanML3D, we set \(\lambda{=}5000\) for MDM, which operates in the data space, and \(\lambda{=}250\) for MotionLCM, which uses a low-dimensional latent space. For the MTT benchmark, we increase the MDM weight to \(\lambda{=}50{,}000\) to account for the longer generated sequences and correspondingly higher-dimensional noise. We use the Adam optimizer with a learning rate of \(0.01\) for MDM and \(0.05\) for MotionLCM, and perform up to 100 gradient steps per sample. The optimization accepts either random Gaussian noise (WIN-O; Sec. 10) or a seed retrieved by WIN-FAR (WINRO; Table 2) as input.
9.4 LoRA-Based Noise Refiner↩︎
The LoRA-based noise refiner amortizes the test-time optimization of WINRO into a single forward pass. Given an initial noise sample \(\mathbf{x}_T\), the refiner \(R_\phi\) predicts a residual \(\Delta\mathbf{x}_T\) so that the refined seed \(\hat{\mathbf{x}}_T = \mathbf{x}_T + \Delta\mathbf{x}_T\) lies in a region of noise space that yields higher text–motion alignment.
9.4.0.1 Architecture.
We construct \(R_\phi\) by attaching rank-64 LoRA modules [39] with \(\alpha{=}64\) (scaling factor \(\alpha/r{=}1\)) to a copy of the pretrained generator \(G_\theta\). Adapters are injected into all query, key, value, and output projections in every attention layer, all feed-forward layers, and the final output projection head. To ensure that the network output is purely the learned residual \(\Delta\mathbf{x}_T\) rather than a denoised prediction, we zero out the base weights and biases of the output head before training. Combined with the standard zero-initialization of the LoRA output matrices [39], this guarantees \(R_\phi(\mathbf{x}_T) = \mathbf{0}\) at initialization, so that training begins from the identity mapping in noise space.
9.4.0.2 Delta computation.
The refiner performs a single forward pass of the LoRA-augmented denoiser at a fixed anchor timestep \(t_0\), set to the first step of the diffusion schedule (i.e.the maximum noise level): \[\Delta\mathbf{x}_T = R_\phi(\mathbf{x}_T,\, t_0,\, c), \label{eq:delta}\tag{5}\] where \(c\) is the text conditioning. No reverse-diffusion iterations are executed; the single-pass output is directly used as the residual.
9.4.0.3 Training.
We freeze the base generator \(G_\theta\) and optimize only the LoRA parameters \(\phi\) end to end. Each training iteration proceeds as follows: (i) sample \(\mathbf{x}_T \sim \mathcal{N}(\mathbf{0}, \mathbf{I})\); (ii) compute \(\Delta\mathbf{x}_T\) via Eq. 5 ; (iii) run the full reverse-diffusion process of the frozen \(G_\theta\) from \(\hat{\mathbf{x}}_T = \mathbf{x}_T + \Delta\mathbf{x}_T\) to obtain a generated motion \(\mathbf{x}_0\); (iv) evaluate text–motion alignment via TMR [24]. Gradients flow from the TMR similarity through the entire generation pipeline back to the LoRA parameters. Because \(\mathbf{x}_T\) is drawn from the same Gaussian prior that the retrieval dictionary is built from, no explicit supervision pairs \((\mathbf{x}_T^\dagger, \mathbf{x}_T^*)\) are required.
The training objective is: \[\mathcal{L}(\phi) = 1 - s(\mathcal{F}_\text{Motion}(\mathbf{x}_0),\, c) + \lambda \,\lVert \Delta\mathbf{x}_T \rVert_2^2, \label{eq:lora95loss}\tag{6}\] where \(s(\cdot, c)\) is the cosine similarity defined in the main paper and \(\lambda{=}0.5\) controls the strength of the \(\ell_2\) regularization on the residual, following Eyring et al. [35].
Table 5 summarizes the training hyperparameters. Both configurations use AdamW with learning rate \(1 \times 10^{-4}\) and gradient clipping at norm \(1.0\), trained for 25 epochs on the HumanML3D training split.
| MDM | MotionLCM | |
|---|---|---|
| LoRA rank \(r\) | 64 | 64 |
| LoRA \(\alpha\) | 64 | 64 |
| Batch size | 32 | 64 |
| Learning rate | \(10^{-4}\) | \(10^{-4}\) |
| Regularization \(\lambda\) | 0.5 | 0.5 |
| Gradient clip norm | 1.0 | 1.0 |
| Training Epochs | 25 | 25 |
9.4.0.4 Inference.
At test time, the refiner accepts either random Gaussian noise or the output of WIN-FAR retrieval as input, since both are drawn from the same prior. When applied to random seeds without retrieval (WIN-LoRA; see Table ¿tbl:tab:winro95full95variants?), the refiner moves each seed toward a winning-ticket region while preserving per-prompt diversity. In the WIN-RLoRA rows of Table 2, we first retrieve a seed via WIN-FAR and then apply the refiner, which further improves text–motion alignment.
9.5 Motion Stylization Setup↩︎
As illustrated in Fig. 9 (Sec. 5.4), the stylization pipeline extends the base WINRO framework. We construct the style-specific noise dictionary \(\mathcal{D}_s\) by applying DDIM inversion to motions from each style category of the 100STYLE dataset [41]. Because these entries originate from style motions rather than random Gaussian samples, they inherently encode stylistic attributes but offer limited text-semantic coverage. Given a text prompt, we retrieve the best-matching entry from \(\mathcal{D}_s\) using the same text-based retrieval as in the base framework (Eq. 2 ). The retrieved noise is then refined by jointly optimizing the semantic loss \(\mathcal{L}_{sim}\) and a style loss that measures cosine distance in the style feature space between the generated motion and the reference style motion.
We adopt SMooDi [42], built on MLD [2], as the stylization backbone. In addition to the standard metrics, we report Style Recognition Accuracy (SRA), which measures how well the generated motion preserves the target style, and Foot Skating Rate, which quantifies physically implausible foot sliding artifacts.
9.6 WINRO Pipeline Pseudocode↩︎
Algorithm 12 summarizes the complete WINRO pipeline. The dictionary is constructed once offline per backbone; at inference, only retrieval and optional refinement are executed. The refinement loop backpropagates through the entire deterministic denoising chain of \(G_\theta\); this is feasible because \(G_\theta\) is frozen and all operations are differentiable under DDIM sampling.
10 Extended Variant Comparison↩︎
Table ¿tbl:tab:winro95full95variants? extends Table 2 with additional WINRO variants on HumanML3D, including optimization from random noise without retrieval (WIN-O), the LoRA-based noise refiner applied to random seeds without retrieval (WIN-LoRA; Sec. 9.4), and stochastic sampling strategies. Each row is characterized by three design choices: Retrieval—initial noise selection from the noise dictionary (Sec. 4.1): random Gaussian ( – ), Top-1 retrieval, or Top-\(k\) (\(k{=}150\)) retrieval; Refine—noise refinement: none ( – ), KL-regularized optimization (Optim; Sec. 4.2), or the LoRA-based noise refiner (LoRA; Sec. 9.4); Perturbation—Gaussian perturbation of the final noise (\(\sigma{=}0.5\); Eq. 3 ). Top-\(k\) retrieval and perturbation are stochastic strategies that recover per-prompt diversity; the main paper uses Top-1 retrieval with perturbation for WIN-FAR and omits perturbation for WINRO where KL regularization already preserves the prior. Shaded rows correspond to configurations reported in Table 2.
Table ¿tbl:tab:winro95full95variants? additionally includes MultiModality (MModality) [19], which measures the diversity of motions generated from the same text prompt; higher values indicate greater per-prompt variety. Deterministic configurations that produce a single output per prompt have undefined MModality, shown as “–”. Results are averaged over 20 evaluation runs with 95% confidence intervals.
| Retr. | Refine | Pert. | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) | MModality \(\uparrow\) |
|---|---|---|---|---|---|---|---|---|---|
| – | – | – | \(0.438^{\scriptstyle\pm.009}\) | \(0.469^{\scriptstyle\pm.002}\) | \(0.661^{\scriptstyle\pm.002}\) | \(0.763^{\scriptstyle\pm.002}\) | \(10.025^{\scriptstyle\pm.066}\) | \(3.258^{\scriptstyle\pm.009}\) | \(2.169^{\scriptstyle\pm.050}\) |
| Top-1 | – | \(✔\) | \(0.248^{\scriptstyle\pm.009}\) | \(0.512^{\scriptstyle\pm.003}\) | \(0.706^{\scriptstyle\pm.003}\) | \(0.804^{\scriptstyle\pm.002}\) | \(10.295^{\scriptstyle\pm.033}\) | \(3.013^{\scriptstyle\pm.011}\) | \(1.422^{\scriptstyle\pm.028}\) |
| Top-\(k\) | – | \(✔\) | \(0.190^{\scriptstyle\pm.003}\) | \(0.493^{\scriptstyle\pm.003}\) | \(0.690^{\scriptstyle\pm.002}\) | \(0.789^{\scriptstyle\pm.002}\) | \(9.652^{\scriptstyle\pm.035}\) | \(3.104^{\scriptstyle\pm.006}\) | \(\mathbf{2.302}^{\scriptstyle\pm.017}\) |
| 1-10 – | Optim | – | \(0.106^{\scriptstyle\pm.004}\) | \(0.528^{\scriptstyle\pm.003}\) | \(0.718^{\scriptstyle\pm.002}\) | \(0.808^{\scriptstyle\pm.002}\) | \(9.638^{\scriptstyle\pm.041}\) | \(2.935^{\scriptstyle\pm.006}\) | – |
| Top-1 | Optim | – | \(0.115^{\scriptstyle\pm.002}\) | \(0.539^{\scriptstyle\pm.002}\) | \(0.727^{\scriptstyle\pm.004}\) | \(0.817^{\scriptstyle\pm.002}\) | \(9.657^{\scriptstyle\pm.026}\) | \(2.883^{\scriptstyle\pm.010}\) | – |
| Top-1 | Optim | \(✔\) | \(\mathbf{0.093}^{\scriptstyle\pm.004}\) | \(0.529^{\scriptstyle\pm.002}\) | \(0.714^{\scriptstyle\pm.005}\) | \(0.810^{\scriptstyle\pm.002}\) | \(9.418^{\scriptstyle\pm.029}\) | \(2.933^{\scriptstyle\pm.009}\) | \(1.171^{\scriptstyle\pm.039}\) |
| 1-10 – | LoRA | – | \(0.237^{\scriptstyle\pm.006}\) | \(0.533^{\scriptstyle\pm.003}\) | \(0.728^{\scriptstyle\pm.003}\) | \(0.822^{\scriptstyle\pm.002}\) | \(9.885^{\scriptstyle\pm.060}\) | \(2.931^{\scriptstyle\pm.008}\) | \(1.620^{\scriptstyle\pm.043}\) |
| Top-1 | LoRA | – | \(0.238^{\scriptstyle\pm.005}\) | \(\mathbf{0.550}^{\scriptstyle\pm.002}\) | \(\mathbf{0.744}^{\scriptstyle\pm.003}\) | \(\mathbf{0.834}^{\scriptstyle\pm.002}\) | \(10.303^{\scriptstyle\pm.022}\) | \(\mathbf{2.846}^{\scriptstyle\pm.006}\) | – |
| Top-1 | LoRA | \(✔\) | \(0.102^{\scriptstyle\pm.002}\) | \(0.546^{\scriptstyle\pm.003}\) | \(0.741^{\scriptstyle\pm.003}\) | \(0.832^{\scriptstyle\pm.002}\) | \(9.577^{\scriptstyle\pm.025}\) | \(\mathbf{2.846}^{\scriptstyle\pm.007}\) | \(1.097^{\scriptstyle\pm.017}\) |
| Top-\(k\) | LoRA | \(✔\) | \(0.098^{\scriptstyle\pm.002}\) | \(0.535^{\scriptstyle\pm.002}\) | \(0.732^{\scriptstyle\pm.002}\) | \(0.823^{\scriptstyle\pm.002}\) | \(\mathbf{9.528}^{\scriptstyle\pm.023}\) | \(2.897^{\scriptstyle\pm.006}\) | \(1.826^{\scriptstyle\pm.016}\) |
| Retr. | Refine | Pert. | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) | MModality \(\uparrow\) |
|---|---|---|---|---|---|---|---|---|---|
| – | – | – | \(0.093^{\scriptstyle\pm.004}\) | \(0.548^{\scriptstyle\pm.003}\) | \(0.741^{\scriptstyle\pm.003}\) | \(0.835^{\scriptstyle\pm.002}\) | \(9.566^{\scriptstyle\pm.069}\) | \(2.763^{\scriptstyle\pm.008}\) | \(\mathbf{1.799}^{\scriptstyle\pm.062}\) |
| Top-1 | – | \(✔\) | \(0.083^{\scriptstyle\pm.003}\) | \(0.572^{\scriptstyle\pm.003}\) | \(0.767^{\scriptstyle\pm.002}\) | \(0.855^{\scriptstyle\pm.001}\) | \(9.474^{\scriptstyle\pm.028}\) | \(2.674^{\scriptstyle\pm.006}\) | \(1.051^{\scriptstyle\pm.040}\) |
| Top-\(k\) | – | \(✔\) | \(0.085^{\scriptstyle\pm.003}\) | \(0.552^{\scriptstyle\pm.002}\) | \(0.751^{\scriptstyle\pm.002}\) | \(0.842^{\scriptstyle\pm.002}\) | \(9.405^{\scriptstyle\pm.024}\) | \(2.724^{\scriptstyle\pm.005}\) | \(1.741^{\scriptstyle\pm.060}\) |
| 1-10 – | Optim | – | \(0.066^{\scriptstyle\pm.003}\) | \(0.573^{\scriptstyle\pm.003}\) | \(0.764^{\scriptstyle\pm.002}\) | \(0.855^{\scriptstyle\pm.002}\) | \(9.176^{\scriptstyle\pm.022}\) | \(2.628^{\scriptstyle\pm.004}\) | – |
| Top-1 | Optim | – | \(0.072^{\scriptstyle\pm.002}\) | \(\mathbf{0.580}^{\scriptstyle\pm.002}\) | \(\mathbf{0.775}^{\scriptstyle\pm.002}\) | \(\mathbf{0.861}^{\scriptstyle\pm.001}\) | \(9.242^{\scriptstyle\pm.026}\) | \(\mathbf{2.609}^{\scriptstyle\pm.005}\) | – |
| Top-1 | Optim | \(✔\) | \(\mathbf{0.061}^{\scriptstyle\pm.002}\) | \(0.569^{\scriptstyle\pm.002}\) | \(0.767^{\scriptstyle\pm.002}\) | \(0.857^{\scriptstyle\pm.002}\) | \(9.447^{\scriptstyle\pm.026}\) | \(2.641^{\scriptstyle\pm.005}\) | \(1.108^{\scriptstyle\pm.045}\) |
| 1-10 – | LoRA | – | \(0.103^{\scriptstyle\pm.005}\) | \(0.559^{\scriptstyle\pm.002}\) | \(0.754^{\scriptstyle\pm.002}\) | \(0.845^{\scriptstyle\pm.003}\) | \(9.637^{\scriptstyle\pm.092}\) | \(2.714^{\scriptstyle\pm.008}\) | \(1.603^{\scriptstyle\pm.066}\) |
| Top-1 | LoRA | – | \(0.114^{\scriptstyle\pm.003}\) | \(0.575^{\scriptstyle\pm.002}\) | \(0.774^{\scriptstyle\pm.001}\) | \(\mathbf{0.861}^{\scriptstyle\pm.002}\) | \(10.076^{\scriptstyle\pm.018}\) | \(2.640^{\scriptstyle\pm.005}\) | – |
| Top-1 | LoRA | \(✔\) | \(0.083^{\scriptstyle\pm.002}\) | \(\mathbf{0.580}^{\scriptstyle\pm.002}\) | \(0.772^{\scriptstyle\pm.001}\) | \(0.860^{\scriptstyle\pm.001}\) | \(\mathbf{9.550}^{\scriptstyle\pm.032}\) | \(2.649^{\scriptstyle\pm.005}\) | \(0.975^{\scriptstyle\pm.0042}\) |
| Top-\(k\) | LoRA | \(✔\) | \(0.078^{\scriptstyle\pm.002}\) | \(0.562^{\scriptstyle\pm.002}\) | \(0.760^{\scriptstyle\pm.002}\) | \(0.849^{\scriptstyle\pm.002}\) | \(\mathbf{9.518}^{\scriptstyle\pm.027}\) | \(2.681^{\scriptstyle\pm.004}\) | \(1.617^{\scriptstyle\pm.068}\) |
2pt MDM-50step
MotionLCM
10.0.0.1 WIN-O: optimization without retrieval.
WIN-O applies the test-time optimization objective (Eq. 4 ) directly to random Gaussian noise, bypassing the retrieval stage. This already yields substantial gains over the base model (e.g., MDM FID: \(0.438 \to 0.106\)). Adding Top-1 retrieval as initialization (WINRO) further improves R-Precision (MDM R-Top1: \(0.528 \to 0.539\); MotionLCM: \(0.573 \to 0.580\)), consistent with the smoothness argument in Sec. 3.2: a retrieved seed lies closer to the target region, benefiting local refinement.
10.0.0.2 WIN-LoRA: LoRA refiner on random seeds.
WIN-LoRA applies the LoRA refiner (Sec. 9.4) to random noise without retrieval, achieving competitive alignment while preserving non-trivial MModality, since different random seeds receive different corrections. Compared with WIN-RLoRA (Table 2), which combines Top-1 retrieval with the refiner, WIN-LoRA trades alignment (e.g., MDM R-Top1: \(0.533\) vs.\(0.550\)) for per-prompt diversity.
10.0.0.3 Stochastic sampling.
Adding perturbation to WIN-RLoRA recovers MModality while substantially reducing FID (e.g., MDM: \(0.238 \to 0.102\)). Top-\(k\) retrieval provides complementary diversity by sampling from a broader candidate set.
11 Ablation Studies↩︎
11.1 Noise Dictionary Size↩︎
Table 6 ablates the noise dictionary size for WIN-FAR on MotionLCM, with Gaussian perturbation (\(\sigma{=}0.5\)) applied after retrieval. Results are averaged over 20 evaluation runs with 95% confidence intervals. Increasing the dictionary size steadily improves R-Precision and MMDist, indicating that a larger pool of candidate noises enables better text–motion matching. FID improves from 100 to 10k entries and plateaus beyond that, suggesting diminishing returns from further expansion. We use 10k as the default dictionary size throughout the paper, as it offers a favorable balance between alignment gains and dictionary construction cost.
| Method | Size | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) |
|---|---|---|---|---|---|---|---|
| MotionLCM | – | \(0.093^{\scriptstyle\pm.004}\) | \(0.548^{\scriptstyle\pm.003}\) | \(0.741^{\scriptstyle\pm.003}\) | \(0.835^{\scriptstyle\pm.002}\) | \(9.566^{\scriptstyle\pm.069}\) | \(2.763^{\scriptstyle\pm.008}\) |
| w/ WIN-FAR | 100 | \(0.094^{\scriptstyle\pm.006}\) | \(0.559^{\scriptstyle\pm.002}\) | \(0.754^{\scriptstyle\pm.002}\) | \(0.845^{\scriptstyle\pm.002}\) | \(9.333^{\scriptstyle\pm.019}\) | \(2.730^{\scriptstyle\pm.009}\) |
| w/ WIN-FAR | 1k | \(0.084^{\scriptstyle\pm.003}\) | \(0.566^{\scriptstyle\pm.003}\) | \(0.761^{\scriptstyle\pm.002}\) | \(0.851^{\scriptstyle\pm.002}\) | \(9.404^{\scriptstyle\pm.030}\) | \(2.696^{\scriptstyle\pm.005}\) |
| w/ WIN-FAR | 10k | \(0.083^{\scriptstyle\pm.003}\) | \(0.572^{\scriptstyle\pm.003}\) | \(0.767^{\scriptstyle\pm.002}\) | \(0.855^{\scriptstyle\pm.001}\) | \(9.474^{\scriptstyle\pm.028}\) | \(2.674^{\scriptstyle\pm.006}\) |
| w/ WIN-FAR | 100k | \(0.085^{\scriptstyle\pm.003}\) | \(0.575^{\scriptstyle\pm.001}\) | \(0.769^{\scriptstyle\pm.002}\) | \(0.857^{\scriptstyle\pm.002}\) | \(9.490^{\scriptstyle\pm.029}\) | \(2.659^{\scriptstyle\pm.005}\) |
11.2 Optimization Hyperparameters↩︎
Table 7 ablates the refinement hyperparameters on MotionLCM, starting from the same WIN-FAR retrieval result. The KL regularization weight \(\lambda\) has the largest impact: without regularization (\(\lambda{=}0\)), the optimized noise departs from the Gaussian prior, degrading FID to \(0.136\) despite comparable alignment. This confirms that the KL term in Eq. 4 is essential for preserving generation quality during refinement. \(\lambda{=}250\) yields the best balance between alignment and realism. In contrast, performance is robust to the learning rate across an order of magnitude (\(0.01\)–\(0.1\)), and cosine similarity consistently outperforms Euclidean distance. Based on these results, we adopt cosine similarity, \(\lambda{=}250\), and lr\(=\)0.05 as defaults.
| Method | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) |
|---|---|---|---|---|---|---|
| Default (\(\lambda{=}250\), lr\({=}0.05\), cos.) | 0.066 | 0.580 | 0.769 | 0.856 | 9.189 | 2.617 |
| \(\lambda=0\) | 0.136 | 0.549 | 0.741 | 0.837 | 9.833 | 2.717 |
| \(\lambda=2.5\) | 0.088 | 0.563 | 0.756 | 0.847 | 9.046 | 2.682 |
| \(\lambda=25\) | 0.070 | 0.567 | 0.762 | 0.850 | 9.108 | 2.657 |
| \(\lambda=2500\) | 0.083 | 0.577 | 0.774 | 0.863 | 9.242 | 2.616 |
| lr\(=\)0.005 | 0.084 | 0.572 | 0.774 | 0.864 | 9.248 | 2.616 |
| lr\(=\)0.5 | 0.073 | 0.575 | 0.768 | 0.854 | 9.242 | 2.624 |
| Euclidean | 0.095 | 0.563 | 0.767 | 0.851 | 9.255 | 2.671 |
11.3 Retrieval Backbone↩︎
In the main experiments we use TMR [24] as the retrieval model. To check whether our framework depends on a particular retrieval backbone, we also replace TMR with PartTMR [28] and repeat the experiments on MotionLCM. All other components, including the diffusion model, optimization setup, and evaluation protocol, are kept unchanged.
| Method | Backbone | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) |
|---|---|---|---|---|---|---|---|
| MotionLCM | – | \(0.093^{\scriptstyle\pm.004}\) | \(0.548^{\scriptstyle\pm.003}\) | \(0.741^{\scriptstyle\pm.003}\) | \(0.835^{\scriptstyle\pm.002}\) | \(9.566^{\scriptstyle\pm.069}\) | \(2.763^{\scriptstyle\pm.008}\) |
| w/ WIN-FAR | PartTMR | \(0.086^{\scriptstyle\pm.003}\) | \(0.569^{\scriptstyle\pm.002}\) | \(0.765^{\scriptstyle\pm.002}\) | \(0.854^{\scriptstyle\pm.002}\) | \(9.450^{\scriptstyle\pm.019}\) | \(2.679^{\scriptstyle\pm.005}\) |
| w/ WINRO | PartTMR | \(0.070^{\scriptstyle\pm.002}\) | \(0.579^{\scriptstyle\pm.002}\) | \(0.777^{\scriptstyle\pm.002}\) | \(0.861^{\scriptstyle\pm.002}\) | \(9.210^{\scriptstyle\pm.017}\) | \(2.601^{\scriptstyle\pm.005}\) |
| w/ WIN-LoRA | PartTMR | \(0.103^{\scriptstyle\pm.003}\) | \(0.563^{\scriptstyle\pm.004}\) | \(0.761^{\scriptstyle\pm.003}\) | \(0.851^{\scriptstyle\pm.002}\) | \(9.587^{\scriptstyle\pm.066}\) | \(2.699^{\scriptstyle\pm.007}\) |
As shown in Table 8, the PartTMR-based variant shows the same qualitative behavior as the original TMR-based one. This indicates that our method does not rely on a specific retrieval backbone and can work with different motion–text retrieval models as long as they provide a shared feature space for text and motion.
11.4 LoRA Rank and Injection Ablation↩︎
Table 9 ablates the LoRA rank \(r\) and injection configuration of the WIN-LoRA noise refiner on MotionLCM. Performance is robust across ranks from 16 to 128, with the default setting (\(r{=}64\), full injection) offering a good balance. Removing attention or feed-forward injections slightly reduces alignment, confirming that both pathways contribute to the learned noise correction.
| Factor | Variant | FID\(\downarrow\) | R-Top1\(\uparrow\) | R-Top2\(\uparrow\) | R-Top3\(\uparrow\) | MMDist\(\downarrow\) | Div.\(\rightarrow\) | \(\Delta\)R-Top1 | |
|---|---|---|---|---|---|---|---|---|---|
| WIN-LoRA | full, r64 | 0.103 | 0.559 | 0.754 | 0.845 | 2.714 | 9.637 | – | |
| Rank | r16 | 0.097 | 0.555 | 0.750 | 0.842 | 2.721 | 9.577 | \(-\)0.004 | |
| Rank | r32 | 0.104 | 0.553 | 0.752 | 0.843 | 2.721 | 9.623 | \(-\)0.006 | |
| Rank | r128 | 0.109 | 0.560 | 0.757 | 0.847 | 2.697 | 9.658 | +0.001 | |
| Injection | w/o Attn | 0.112 | 0.558 | 0.755 | 0.844 | 2.714 | 9.705 | \(-\)0.001 | |
| Injection | w/o Attn, FFN | 0.097 | 0.553 | 0.751 | 0.842 | 2.727 | 9.701 | \(-\)0.006 |
1.8pt
12 Additional Generalization Experiments↩︎
12.1 KIT-ML Dataset↩︎
To verify that WINRO generalizes beyond HumanML3D, we evaluate on KIT-ML [7] using MDM-50step with the same noise dictionary. Table 10 shows that both retrieval and refinement yield consistent improvements, confirming that winning noise tickets are not specific to a single dataset.
| Method | FID \(\downarrow\) | R-Top1 \(\uparrow\) | R-Top2 \(\uparrow\) | R-Top3 \(\uparrow\) | Div. \(\rightarrow\) | MMDist \(\downarrow\) |
|---|---|---|---|---|---|---|
| MDM-50step | \(0.382_{\pm0.050}\) | \(0.443_{\pm0.007}\) | \(0.658_{\pm0.008}\) | \(0.777_{\pm0.009}\) | \(11.364_{\pm0.129}\) | \(2.733_{\pm0.026}\) |
| w/ WIN-FAR | \(0.366_{\pm0.018}\) | \(0.451_{\pm0.007}\) | \(\mathbf{0.681}_{\pm0.006}\) | \(0.789_{\pm0.004}\) | \(11.246_{\pm0.029}\) | \(2.714_{\pm0.014}\) |
| w/ WINRO | \(\mathbf{0.347}_{\pm0.025}\) | \(\mathbf{0.454}_{\pm0.011}\) | \(0.678_{\pm0.013}\) | \(\mathbf{0.792}_{\pm0.013}\) | \(11.403_{\pm0.157}\) | \(\mathbf{2.654}_{\pm0.028}\) |
12.2 Rare-Prompt Analysis↩︎
To assess whether WINRO also benefits rare or difficult prompts, we split the HumanML3D test set by CLIP-based prompt rareness following ReMoDiffuse [27]: each caption is scored by \(r(p){=}1{-}\max_i \cos(E_\text{CLIP}(p), E_\text{CLIP}(t_i))\) over training captions \(t_i\), which is independent of TMR. Table 11 reports MMDist on the rarest 20% and tail 5% subsets. WINRO provides larger improvements on rarer prompts, suggesting that noise retrieval and refinement are particularly beneficial when the base model has fewer relevant training examples to draw upon. Note that this remains an in-distribution evaluation; similar to other optimization methods, WINRO cannot create motions outside the frozen backbone’s learned support.
| Method | Rare-20% | Tail-5% |
|---|---|---|
| MDM-50step | 4.06 | 4.17 |
| w/ WIN-FAR | 3.79 | 3.82 |
| w/ WINRO | 3.34 | 3.39 |
13 Additional Qualitative Examples↩︎
We provide additional qualitative examples that complement the quantitative results and show how different initial noise settings affect the generated motions.
Fig. 13 presents additional text-to-motion examples extending Fig. 7, comparing motions generated from random noise, retrieved noise (WIN-FAR), and optimized noise (WINRO) under MDM-50step. Retrieved noise already produces motions closer to the text descriptions, and refinement further improves faithfulness.
![Figure 14: Additional compositional generation results on the multi-track timeline benchmark. STMC [40] often fails to realize individual action segments (, missing jump, punch, or hop), while applying our noise refinement (w/ Ours) improves faithfulness to each segment’s text prompt.](/article/2607.06843/img/15_stmc_additional.png "fig:")
Fig. 14 shows additional compositional generation examples using STMC [40] with and without our noise refinement, extending the comparison in Fig. 8. Across four multi-track timelines with overlapping and sequential action prompts, STMC frequently drops or truncates individual actions, whereas applying our method consistently improves segment-level faithfulness.
For animated visualizations of all results, we strongly encourage readers to view the supplementary video and the interactive motion viewer included in the supplementary materials.