Abstract

Generating realistic synthetic sequential data is critical in real-world applications across operations research, finance, healthcare, energy systems, and scientific computing, where time-indexed observations are used for prediction, simulation, risk assessment, and data-driven decision-making. While diffusion models have achieved remarkable success in generating static data, their direct extensions to sequential settings often fail to capture temporal dependence and information structure. Designing diffusion models that can simulate sequential data in an adapted manner, and hence without anticipation of future information, therefore remains an open challenge.

In this work, we propose a sequential forward–backward diffusion framework for adapted time series generation. Our approach progressively injects and removes noise along the sequence, conditioning on the previously generated history to ensure adaptiveness. A novel score-matching objective is introduced for efficient parallel training. We derive rigorous statistical guarantees under a generic framework, then establish score approximation, score estimation, and distribution estimation results with ReLU networks serving as a concrete instance. Empirically, we validate our method on synthetic data, including ARMA models and Gaussian processes, and demonstrate its effectiveness in constructing mean-variance optimal portfolios. Our code is available at https://github.com/yinbinhan/adapted_diffusion_model.

1 Introduction↩︎

Generating realistic synthetic sequential data is important for many real-world applications across operations research, financial markets, healthcare, energy systems, and scientific computing, where time-indexed observations are used for prediction, simulation, risk assessment, and data-driven decision-making. At the same time, this task remains challenging in complex sequential settings, where the data may exhibit temporal dependence, nonstationarity, nonlinear dynamics, and multiple sources of uncertainty. Beyond matching the high-dimensional joint law of the observed process, a sequential generator must also respect the information flow under which the data are observed: each generated value should depend only on the past and present information available at that stage, and not on future observations. This adapted (i.e., non-anticipative) structure is a basic modeling requirement in domains where the timing of information affects prediction or decision-making. For example, in stress testing, synthetic data generation requires not only capturing temporal dependence, but also preserving adaptiveness with respect to the prescribed information flow [1]–[3]; in portfolio optimization, trading decisions are updated dynamically based only on currently available signals [4], [5]; and in simulation-based policy learning, synthetic data are used to train or evaluate decision rules that themselves operate sequentially [6]–[9]. Under these circumstances, preserving the temporal direction of information is as essential as maintaining marginal and joint laws.

Transformer-based autoregressive models [10] provide a natural framework for sequential data generation. However, their empirical success has been largely confined to discrete modalities, most notably next-token generation in natural language, and extending them to continuous settings remains challenging and fragile [11], [12]. Existing approaches often rely on discretization or quantization, which can introduce non-negligible approximation error in continuous spaces [13]–[15]. In contrast, diffusion models have emerged as a flexible and powerful paradigm for modeling and sampling from continuous distributions in high-dimensional spaces [16]–[18], with strong empirical performance and increasing theoretical support [19]–[23]. Despite these advances, most diffusion-based methods are designed for static data or settings where the entire sample is treated as a single object. A direct extension to sequential data—jointly noising and denoising the full trajectory—can match the joint distribution, but does not enforce the underlying information flow. In particular, it does not guarantee that each generated value depends only on past information, and thus may violate the causal ordering inherent in sequential systems.

This gap motivates the central question of our paper:

How can we unlock diffusion models for sequential data generation, while preserving the adaptiveness and retaining theoretical guarantees?

We address this question by introducing a sequential forward–backward diffusion framework, Adaptive Diffusion for Sequential Data Generation (AD-Seq), that aligns the diffusion dynamics with the temporal information flow. The key idea is to generate the sample path through conditional reverse diffusion steps, such that each generated coordinate depends only on the previously generated history. This construction preserves the adapted structure of sequential data while retaining the flexibility of diffusion-based generation.

Beyond full time series generation, this conditional perspective also suggests a broader use of AD-Seq as a generator of admissible scenarios conditional on available information. Given a realized history, the model can generate multiple future continuations that respect the same information flow, which can then be used in downstream tasks such as statistical inference, predictive control, and risk evaluation. Although we focus on time series in this paper, the same principle can be applied to whenever generated data must respect certain causal structure, including panel data, network data, and causal inference.

1.0.0.1 Our work and contributions.

Our contributions are three-fold.

(i) A new diffusion model framework for adapted sequential data generation. We introduce a novel Adaptive Diffusion for Sequential Data Generation (AD-Seq) framework for generating time series that are adapted to an information flow. Unlike classical diffusion models that operate on the full time series as a single object, our construction generates each coordinate sequentially, conditioning only on the previously generated history. This yields a generative model that preserves the conditional distributions of the data while enforcing adaptiveness by design. At a conceptual level, this provides a principled way to incorporate filtration constraints into diffusion models. On the algorithmic side, we develop a novel score-matching objective that decouples the learning problem across time steps, enabling parallel training of the score functions and ensuring scalability to high-dimensional sequences. Beyond synthetic data generation, this framework opens the door to a broad range of downstream applications, including multi-step prediction, predictive decision-making, and statistical inference under information-flow constraints.

(ii) Statistical learning theory for the AD-Seq framework. We develop a general statistical learning theory for our proposed AD-Seq framework, including score approximation, score estimation, and distribution estimation guarantees. Our analysis is network architecture-agnostic and can incorporate generic approximation and covering number assumptions. A key technical challenge is to mitigate the growing length of the history for long-horizon generation. We introduce a history truncation mechanism based on the temporal dependence decay in the sequence. This yields the following finite-sample total-variation bound for the conditional law generated by AD-Seq: \[\tilde{\mathcal{O}}(n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}}).\] Here, \(n\) is the sample size and \(\beta\) is the distribution smoothness parameter denoted in Assumption 4. The rate depends on the effective history length \(d_{\rm trunc}\), rather than the full horizon length, showing that long-horizon generation can remain statistically tractable when temporal dependence decays; see for example the exponential and polynomial decays in Propositions 6–7. Moreover, despite the additional adaptiveness constraint, this rate matches the best-known rates for static diffusion models.

(iii) Empirical validation for temporal structure learning and decision-making. We implement the proposed framework using transformer architectures with causal masking and evaluate it on both synthetic and real data. The empirical results indicate that the method captures meaningful aspects of the sequential dependence structure, rather than merely reproducing static marginals. On ARMA models, it accurately recovers the autocorrelation structure, including in small-sample regimes, and produces synthetic samples whose estimated autocorrelation functions are more accurate than those obtained directly from the empirical training data. On Gaussian processes, it reconstructs the covariance matrix with the correct decay pattern, showing that the model captures the dependence structure of the underlying process. We further evaluate performance in a downstream decision problem, namely mean–variance portfolio optimization on the S&P 500. In this experiment, policies trained using diffusion-generated samples attain the highest Sharpe ratios among several benchmark methods. These findings suggest that respecting temporal dependence and information flow in sequential data generation can lead to measurable improvements in both statistical fidelity and downstream decision quality.

Our work is related to several emerging research directions.

First, a variety of generative modeling frameworks have been developed for time series synthesis. Prior to the emergence of diffusion models, most approaches were based on the GAN framework [3], [24]–[34] while others adopted the VAE paradigm [1], [35]–[41]. Diffusion models have gained recent attentions for time series generation [42]–[48]. In parallel, methods based on Schrödinger Bridges have also been explored for generative modeling of time series [2], [49]–[51].

Our work is also closely related to the recent advances of diffusion models. A line of research focused on the convergence theory of diffusion models [52]–[74]. In parallel, several studies analyzed the statistical properties of score matching, establishing statistical error rates and sample complexity bounds [70], [74]–[81]. However, none of the prior work addressed the statistical estimation theory of diffusion models for sequential data, except [82], which investigated diffusion transformers restricted to (discrete) Gaussian process data.

Compared with the existing literature, our work is the first to study diffusion models for generic sequential data with rigorous statistical guarantees. More fundamentally, the proposed framework is designed to preserve not only marginal or joint path distributions, but also the temporal dependence and information structure inherent in the underlying data-generating process. In particular, the adaptiveness guarantee established in this work makes diffusion models applicable to settings where respecting the information flow is essential, and thereby extends their scope beyond data generation to tasks such as multi-step prediction, predictive decision-making, and statistical inference under information flow constraints.

1.0.0.3 Notations.

Let \((\Omega, \mathcal{F}, \mathbb{P}, (\mathcal{F}_h)_{h=1}^H)\) be a filtered probability space. Denote \((W_t^h)_{h=1}^H\) and \((\bar{W}_t^h)_{h=1}^H\) be \(2H\) independent one-dimensional Brownian motions. For a vector \(\mathbf{X}= (X^1, \dots, X^H) \in \mathbb{R}^H\), we denote \(\mathbf{X}^{[\ell:h]} \mathrel{\vcenter{:}}= (X^\ell, \dots, X^h)\) and \(\mathbf{X}^{[\ell:h)} \mathrel{\vcenter{:}}= (X^\ell, \dots, X^{h-1})\). Given two random variables \(X\) and \(Y\), we denote \(\relax_X[\cdot]\) as the expectation over \(X\) and denote \(\relax_{X\vert Y}[\cdot]\) as the conditional expectation of \(X\) given \(Y\). Let \(\mathcal{N}(\mu, \sigma^2)\) be one-dimensional Gaussian distribution with mean \(\mu\) and variance \(\sigma^2\). We refer the notation \(X \sim p\) as random variable \(X\) with probability density \(p\). Let \(\sigma(\mathbf{X})\) be the \(\sigma\)-algebra generated by the random vector \(\mathbf{X}\). For two \(\sigma\)-algebra \(\mathcal{A}\) and \(\mathcal{B}\), \(\mathcal{A} \vee \mathcal{B}\) denotes the smallest \(\sigma\)-algebra containing \(\mathcal{A} \cup \mathcal{B}\). Finally, \(\left \|\cdot\right \|\) is the Euclidean norm. We use the notation \(X\stackrel{d}{=}Y\) to mean two random variables \(X\) and \(Y\) have the same distribution. Define the set \([p] = \left\lbrace 1, \dots, p \right\rbrace\) for \(p \in \mathbb{Z}_+\).

2 Problem Setup↩︎

Our objective is to design a diffusion model that generates adapted time series whose law matches the underlying data distribution, with respect to a specified (and meaningful) filtration. For notational convenience, we present the development in the one-dimensional case: let \(\mathbf{X}_0 = (X_0^h)_{h=1}^H \in \mathbb{R}^H\) denote a length-\(H\) time series in discrete-time and adapted to the filtration \((\mathcal{F}_h)_{h=1}^H\), i.e., \(X_0^h\in \mathcal{F}_h\). The methodology extends straightforwardly to higher dimensions. In addition, denote \(\mu_0^{[1:H]}\in \mathcal{P}(\mathbb{R}^H)\) as the distribution of \(\mathbf{X}_0\).

2.0.0.1 Preliminaries on diffusion models.

In the classical setting [16]–[18], diffusion models include a forward process that gradually add noise to data and a time-reverse process that gradually remove noise to generate new data. Mathematically, the forward process is often chosen to be an \(H\)-dimensional Ornstein-Uhlenbeck (OU) process defined on \(\mathbb{R}^H\): \[\label{eq:diffusion} {\rm d}\mathbf{X}'_t = - \frac{1}{2}g(t)\mathbf{X}'_t{\rm d}t + \sqrt{g(t)}{\rm d}{\boldsymbol{W}}_t', \quad \mathbf{X}_0' \sim \mu_0^{[1:H]},\tag{1}\] where \(\{{\boldsymbol{W}}'_t\}_{0 \leq t\leq T}\) is a \(H\)-dimensional Brownian motion and \(g: \mathbb{R}\to \mathbb{R}_+\) is a positive-valued function. The time-reversed SDE of 1 is given by \[\label{eq:reversed95diff} {\rm d}\overline{\boldsymbol{X}}'_t = \frac{1}{2}g(T-t)\Big[\bar \mathbf{X}'_t + \nabla_x \log p_{T-t} (\overline{\boldsymbol{X}}_t')\Big]{\rm d}t + \sqrt{g(T-t)}{\rm d}\overline{\boldsymbol{W}}'_t,\tag{2}\] where \(\{\overline{\boldsymbol{W}}'_t\}_{0 \leq t\leq T}\) is another independent \(H\)-dimensional Brownian motion and \(p_t: \mathbb{R}^H \to \mathbb{R}\) represents the marginal density of \(\mathbf{X}_t'\) at time \(t\) in the forward process 1 .

Such a framework is appropriate for generating static data or fully predictable sequences (e.g., videos), where no filtration-based adaptiveness constraint is imposed. Even when the forward OU process is driven by independent Gaussian noise across coordinates, the corresponding backward dynamics generally fail to preserve the information flow of the data, since the score term \(\nabla \log p_{T-t}(\cdot)\) depends on the joint density and thus introduces cross-coordinate dependence. As a result, the value at a given time step may implicitly depend on future components of the sequence. Consequently, adaptiveness with respect to a prescribed filtration is not enforced, and the generated data may violate the intended information flow or temporal structure. This limitation motivates us to develop the AD-Seq framework, which adds and removes noise progressively along the time index, conditioning only on previously generated history, and thereby preserves the prescribed information flow by construction.

2.0.0.2 Adaptive Sequential Diffusion Model.

For each coordinate/timestamp \(h\) \((1\leq h \leq H)\), we define a pair of forward and time-reversed one-dimensional SDEs, driven by OU processes. In the construction below, we specialize to the constant noise schedule \(g(\cdot)\equiv 1\).

For \(h = 1\), we consider the following one-dimensional forward process on \(\mathbb{R}\): \[\begin{align} \label{eq:forward-1} {\rm d}X_t^1 = -\tfrac{1}{2}X_t^1 {\rm d}t + {\rm d}W_t^1, \;\; X_0^1 \sim \phi^1(0, \cdot), \end{align}\tag{3}\] where \(\phi^1(0, \cdot): \mathbb{R}\to \mathbb{R}\) is the probability density of \(X_0^1\). To generate new samples, we consider another stochastic process on \(\mathbb{R}\) for sampling defined as 3 : \[\begin{align} \label{eq:backward-1} {\rm d}\bar{X}_t^1 = \Big[\tfrac{1}{2}\bar{X}_t^1 + \partial_x\log \phi^1(T-t, \bar{X}_t^1)\Big]{\rm d}t + {\rm d}\bar{W}_t^1, \;\; \bar{X}_0^1 \sim \phi^1 (T, \cdot), \end{align}\tag{4}\] where \(\phi^1(t, \cdot): \mathbb{R}\to \mathbb{R}\) is the probability density of \(X_t^1\) and \(\partial_x\log \phi^1(t, \cdot): \mathbb{R}\to \mathbb{R}\) denotes its score function. Under regularity conditions, the forward process 3 admits a strong solution and thus the score function is well-defined. For each \(h > 1\), we construct a forward process on \(\mathbb{R}\) iteratively by conditioning on the history: \[\begin{align} \label{eq:forward-h} {\rm d}X_t^h = -\frac{1}{2}X_t^h {\rm d}t + {\rm d}W_t^h, \;\; X_0^h \sim \phi_{\bar{\mathbf{X}}_T^{[1:h)}}^h(0, \cdot), \end{align}\tag{5}\] where for any \(\mathbf{z}\in\mathbb{R}^{h-1}\) and \(t\in[0,T]\), \(\phi_{\mathbf{z}}^h(t,\cdot):\mathbb{R}\to\mathbb{R}\) denotes the conditional density function of \(X^h_{t}|\mathbf{X}^{[1:h)}_0=\mathbf{z}\). Compared to 3 , the forward process 5 is initialized from a conditional probability distribution, incorporating the temporal dependence inherent in the time series. Similar to the sampling process 4 , we define another stochastic process on \(\mathbb{R}\) for sampling: \[\begin{align} \label{eq:backward-h} \textstyle {\rm d}\bar{X}_t^h = \Big[\frac{1}{2}\bar{X}_t^h + \partial_x \log \phi_{\bar{\mathbf{X}}_T^{[1:h)}}^h(T-t, \bar{X}_t^h)\Big]{\rm d}t + {\rm d}\bar{W}_t^h, \;\; \bar{X}_0^h \sim \phi_{\bar{\mathbf{X}}_T^{[1:h)}}^h(T, \cdot), \end{align}\tag{6}\] with \(\partial_x\log{\phi^h_{\bar{\mathbf{X}}_{T}^{[1:h)}}}(T-t,\bar X^h_t)=\partial_x\log{\phi^h_{\mathbf{z}}(T-t,x)}\biggl|_{\mathbf{z}=\bar{\mathbf{X}}^{[1:h)}_T,\;x=\bar X^h_t}\). Here the sampling process 6 incorporates previously generated trajectories \(\bar{\mathbf{X}}_T^{[1:h)}\), ensuring that the synthesized series \(\bar{\mathbf{X}}_T = (\bar{X}_T^h)_{h=1}^H\) is generated sequentially. The following proposition verifies the key distributional consistency property of this construction: with the exact conditional score and the corresponding terminal marginal initialization, the sampling processes 4 and 6 recover the desired marginal and conditional laws.

Proposition 1 (Property of the sampling processes). Let \((\bar{X}_T^h)_{h=1}^H\) follow 3 6 . Then the following distributional property holds: \[\begin{align} \bar{X}_T^1 \stackrel{d}{=} X_0^1, \;\; \text{and} \;\; (\bar{X}_T^h\vert \bar{\mathbf{X}}_T^{[1:h)}=\mathbf{x})\stackrel{d}{=} (X_0^h \vert \mathbf{X}_0^{[1:h)}=\mathbf{x}),\;\;\forall h>1,\;\mathbf{x}\in \mathbb{R}^{h-1}. \end{align}\] Consequently, we have \((\bar{X}_T^h)_{h=1}^H \stackrel{d}{=} (X_0^h)_{h=1}^H\).

This law-matching property justifies referring to the sampling processes 4 and 6 as the time-reversed SDEs associated with the conditional forward diffusions.

Proof. We only prove the property when \(h > 1\) as the case where \(h = 1\) follows directly from [83], [84]. From the forward process 5 , we know that \(\phi_\mathbf{z}^h(t, \cdot)\) satisfies the Fokker-Planck equation (in the weak sense): \[\begin{align} \label{eq:fk-transition} \frac{\partial}{\partial t}\phi_\mathbf{z}^h(t, x) = \frac{1}{2}\partial_{xx} \phi_\mathbf{z}^h(t, x) + \frac{1}{2}\partial_x \Big(\phi_\mathbf{z}^h(t, x) x \Big), \end{align}\tag{7}\] where we recall that \(\phi^h_\mathbf{z}(0, \cdot)\) is the probability density of \(X_0^h\) given the history \(\mathbf{X}_0^{[1:h)} = \mathbf{z}\).

Denote \(\psi_\mathbf{z}^h(t, x) \mathrel{\vcenter{:}}= \phi_\mathbf{z}^h(T-t, x)\) by the time reversed probability density. Differentiate \(\psi_\mathbf{z}^h(t, x)\) w.r.t. \(t\) to have \[\begin{align} \label{eq:FK-reversal} \frac{\partial}{\partial t}\psi_\mathbf{z}^h(t, x) & = - \frac{\partial}{\partial s} \phi_\mathbf{z}^h(s, x)\vert_{s = T - t} \nonumber \\ & = -\frac{1}{2}\partial_{xx} \phi_\mathbf{z}^h(T-t, x) - \frac{1}{2}\partial_x \Big( \phi_\mathbf{z}^h(T-t, x) x \Big) \nonumber \\ & = -\frac{1}{2}\partial_{xx} \psi_\mathbf{z}^h(t, x) - \frac{1}{2}\partial_x \Big( \psi_\mathbf{z}^h(t, x) x \Big). \end{align}\tag{8}\] Define the function \(\bar{b}^h: [0, T] \times \mathbb{R}\times \mathbb{R}^{h-1} \to \mathbb{R}\) as \[\begin{align} \bar{b}^{h}(t, x, \mathbf{z}) \mathrel{\vcenter{:}}= \frac{\partial_x \psi_\mathbf{z}^h(t, x)}{\psi_\mathbf{z}^h(t, x)} + \frac{1}{2}x= \partial_x \log \psi_\mathbf{z}^h(t, x) + \frac{1}{2}x. \end{align}\] Note that \[\begin{align} \partial_{xx} \psi_\mathbf{z}^h(t, x) + \frac{1}{2}\partial_x\Big( \psi_\mathbf{z}^h(t, x)x\Big) & = \partial_x \Big(\partial_x\psi_\mathbf{z}^h(t, x) + \frac{1}{2}\psi_\mathbf{z}^h(t, x)x\Big)\\ & = \partial_x \left( \psi_\mathbf{z}^h(t, x) \Big(\partial_x \log \psi_\mathbf{z}^h(t, x) + \frac{1}{2}x\Big) \right) \\ & = \partial_x \Big(\psi_\mathbf{z}^h(t, x) \bar{b}^{h}(t, x, \mathbf{z})\Big) \end{align}\] As a consequence, it follows from 8 that \[\begin{align} \label{eq:backward-FP} \frac{\partial}{\partial t}\psi_\mathbf{z}^h(t, x) = \frac{1}{2}\partial_{xx} \psi_\mathbf{z}^h(t, x) - \partial_x \Big(\psi_\mathbf{z}^h(t, x) \bar{b}^h(t, x, \mathbf{z})\Big). \end{align}\tag{9}\] This is exactly the Fokker-Planck equation of the SDE 6 . ◻

3 Algorithm design and adaptiveness property↩︎

In practice, the time-reversed SDEs 4 and 6 cannot be directly used to generate samples, as both the score functions \(\partial_x \log \phi_{\bar{\mathbf{X}}_T^{[1:h)}}^h(t,\cdot)\) and the initial distributions \(\phi_{\bar{\mathbf{X}}_T^{[1:h)}}^h(T, \cdot)\) are unknown. To address this issue, we follow common practice to replace the initial distribution by the standard Gaussian distribution and replace the ground-truth score \(\partial_x \log \phi^h_\mathbf{z}(t, x)\) by a score estimator \(\hat{s}^h(t, x, \mathbf{z}): [0, T] \times \mathbb{R}\times \mathbb{R}^{h-1} \to \mathbb{R}\). By convention, we set \(\mathbf{z}= \varnothing\) when \(h = 1\) and thus \(\hat{s}^1(t, x, \varnothing) = \hat{s}^1(t, x)\). With these modifications, we obtain an approximation of the time-reversed process, which is practically implementable: \[\begin{align} \label{eq:backward-approx} {\rm d}\tilde{X}_t^h = \Big[\tilde{X}_t^h/2 + \hat{s}^h(T-t, \tilde{X}_t^h, \tilde{X}_T^{[1:h)})\Big]{\rm d}t + {\rm d}\bar{W}_t^h, \;\; \tilde{X}_0^h \sim \mathcal{N}(0, 1). \end{align}\tag{10}\] Using 10 , we propose Algorithm 1 for sequential data sampling.⁵ For ease of exposition, Algorithm 1 is written in continuous time for simplicity while implemented with a time discretization.

Figure 1: Sampling algorithm for sequential data

3.0.0.1 Score matching.

To estimate the score function, a natural choice is to minimize the weighted quadratic loss for all coordinates: \[\begin{align} \label{eq:score-matching-naive} \min_{s^h\in \mathcal{S}^h}\;\int_{t_0}^T \frac{1}{T- t_0}\relax_{\bar{\mathbf{X}}^{[1:h)}_T}\relax_{X_t^h \vert \bar{\mathbf{X}}^{[1:h)}_T}\big[\big|s^h(t, X_t^h, \bar{\mathbf{X}}^{[1:h)}_T) - \partial_x \log \phi^h_{\bar{\mathbf{X}}^{[1:h)}_T}(t, X_t^h)\big|^2\big]{\rm d}t, \end{align}\tag{11}\] where \(\mathcal{S}^h\) is a function class (often neural networks) for coordinate \(h\). Here, \(t_0\) is an early-stopping time to prevent the blow-up of score functions, which is commonly adopted in practice [17], [80], [85]. As \(\bar{\mathbf{X}}_T^{[1:h)}\) has the same distribution as \(\mathbf{X}_0^{[1:h)}\) for each coordinate \(h\), one can equivalently minimize the alternative: \[\begin{align} \label{eq:score-matching-equiv} \min_{s^h\in \mathcal{S}^h}\;\int_{t_0}^T \frac{1}{T- t_0}\relax_{\mathbf{X}_0^{[1:h)}}\relax_{X_t^h \vert \mathbf{X}_0^{[1:h)}}\big[\big|s^h(t, X_t^h, \mathbf{X}_0^{[1:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[1:h)}}(t, X_t^h)\big|^2\big]{\rm d}t. \end{align}\tag{12}\] As shown by [86], rather than minimizing the integral above, we can minimize an equivalent denoising score matching objective which shares the same minimizer as 12 : \[\begin{align} \label{eq:dsm-naive} \min_{s^h\in \mathcal{S}^h}\;\int_{t_0}^T \frac{1}{T- t_0}\relax_{\mathbf{X}_0^{[1:h)}}\relax_{X_0^h \vert \mathbf{X}_0^{[1:h)}}\relax_{X_t^h \vert X_0^h}\big[\big|s^h(t, X_t^h, \mathbf{X}_0^{[1:h)}) - \sigma_t^{-2}(\alpha_t X_0^h - X_t^h)\big|^2\big]{\rm d}t, \end{align}\tag{13}\] with \(\alpha_t = e^{-t/2}\) and \(\sigma_t^2 = 1 - e^{-t}\). Furthermore, we denote \(\mathbf{s}= (s^1, \dots, s^H)\), where the score \(s^h\) takes current state \(X_t^h\) and the history \(\mathbf{X}_0^{[1:h)}\) as input. Using this vector notation, we rewrite 13 as \[\begin{align} \label{eq:dsm-final} \min_{\mathbf{s}\in \mathcal{S}}\;\int_{t_0}^T \frac{1}{T- t_0}\relax_{\mathbf{X}_0}\relax_{\mathbf{X}_t\vert \mathbf{X}_0}\Big[\Big\|\mathbf{s}(t, \mathbf{X}_t, \mathbf{X}_0) - \sigma_t^{-2}(\alpha_t \mathbf{X}_0 - \mathbf{X}_t)\Big\|^2\Big]{\rm d}t. \end{align}\tag{14}\] Here, the function class \(\mathcal{S}\) is an aggregation of all function classes at each coordinate \(h\), i.e., \(\mathcal{S}= \mathcal{S}^1 \times \mathcal{S}^2 \times \cdots \times\mathcal{S}^H\). We remark that the objective 14 allows for training score functions in parallel in contrast to the sequential sampling in Algorithm 1. In fact, 14 can be implemented in a parallel fashion for each timestamp \(h\in[H]\), without knowledge of information at prior timestamps.

In practice, we approximate 14 by its empirical version using available data points. This full-history objective becomes statistically less efficient for later coordinates: as \(h\) increases, the input dimension of \(s^h\) grows with the conditioning history. In many financial applications, however, dependence on remote coordinates decays with lag [87], [88]. We therefore condition the score on a recent-history window. Specifically, given \(n\) i.i.d. sample trajectories \(\mathcal{D}\mathrel{\vcenter{:}}= \left\lbrace \mathbf{x}_i \right\rbrace_{i = 1}^n\) with \(\mathbf{x}_i\in \mathbb{R}^H\), we sample \(X_t^h\) given \(X_0^h = x_i^h\) from \(\mathcal{N}(\alpha_t x_i^h, \sigma_t^2)\). We focus on the truncation window from coordinate \(k\) to \(h\) and denote the associated loss function by \[\begin{align} \label{eq:loss-lh} \ell^{[k:h]}(\mathbf{x}_i^{[k:h]}; s) = \int_{t_0}^T \frac{1}{T - t_0}\relax_{X_t^h\sim \mathcal{N}(\alpha_t x_i^h, \sigma_t^2)}\big[\big|s(t, X_t^h, \mathbf{x}_i^{[k:h)}) - \sigma_t^{-2}(\alpha_t x_i^h - X_t^h)\big|^2\big]{\rm d}t. \end{align}\tag{15}\] Here the truncation window can be properly chosen to balance the estimation efficiency and the loss of historical information; see Section 4.4. Let \(\hat{s}^h\) be a minimizer to the empirical score matching risk \(\widehat{\mathcal{L}}^{[k:h]}(s) := \frac{1}{n}\sum_{i = 1}^n \ell^{[k:h]}(\mathbf{x}_i^{[k:h]}; s)\).

3.0.0.2 Adaptiveness property.

We now establish the adaptiveness property of the time series generated by Algorithm 1. This property is not automatic for vanilla diffusion models. Specifically, when a vanilla diffusion model is applied to the full time series jointly, the reverse-time score \(\nabla \log p_t(x^1,\ldots,x^H)\) generally depends on all time indices, so the update of an earlier coordinate may use information from later coordinates. Such dependence breaks the intended information structure of a sequential system. By contrast, Algorithm 1 generates coordinates sequentially and uses only the previously generated history when sampling the next coordinate.

Assumption 1. For each coordinate \(h\), the score estimator \(\hat{s}^h\) is trained such that \(\hat{s}^h\in \mathcal{F}_h\vee \sigma(\tilde{U}^h)\), in which \(\tilde{U}^h\) is an independent random variable defined on the same probability space.

Here, \(\tilde{U}^h\) can be viewed as exogenous randomness occurred in training (e.g., adopting a stochastic optimization algorithm). Assume the probability space \((\Omega, \mathcal{F},\mathbb{P})\) is rich enough to define Brownian motions \(\bar W_t^h\), random variables \(\tilde{U}^h\), and standard Gaussian random variables \(\tilde{X}^h_0\sim \mathcal{N}(0, 1)\), \(h\in\{1,\dots,H\}\). The next proposition constructs a filtration such that the generated sequential data is adapted.

Proposition 2 (Adaptiveness). Suppose Assumption 1 holds. In addition, assume \(\bar W_t^h\), \(\tilde{U}^h\), and \(\tilde{X}^h_0\sim \mathcal{N}(0, 1)\), \(h\in\{1,\dots,H\}\) are independent.⁶ Then the output \((\tilde{X}_T^h)_{h=1}^H\) from Algorithm 1 is adapted to the enlarged filtration \(\tilde{\mathcal{H}}\mathrel{\vcenter{:}}=(\tilde{\mathcal{H}}_h)_{h=1}^H\) defined as, for \(1\leq h \leq H\) \[\begin{align} \tilde{\mathcal{H}}_1 &=& \sigma(\tilde{X}^1_0) \vee \sigma(\{\bar{W}_t^1\}_{t=0}^T)\vee \sigma(\tilde{U}^1)\vee \mathcal{F}_1,\\ \tilde{\mathcal{H}}_{h} &=& \tilde{\mathcal{H}}_{h-1} \vee \sigma(\tilde{X}^h_0) \vee \sigma(\{\bar{W}_t^h\}_{t=0}^T) \vee \sigma(\tilde{U}^h)\vee \mathcal{F}_h. \end{align}\]

To interpret Proposition 2, recall that the enlarged filtration is obtained by adjoining to the data filtration the auxiliary randomness used in training and sampling. Concretely, it is the filtration generated by (i) the data filtration \((\mathcal{F}_h)_{h\ge 0}\), (ii) the training randomization \(\sigma(\tilde{U}^h)\), (iii) the (random) initialization \(\sigma(\tilde{X}_0^h)\), and (iv) the driving Brownian motion \(\sigma(\bar W^h_t:0\le t\le T)\). Equivalently, \(\tilde{\mathcal{H}}\) is the smallest \(\sigma\)-field (and the associated smallest filtration, upon taking the natural time-indexed version) with respect to which all random objects appearing in the training procedure and in the sampling dynamics are measurable.

4 Distribution Estimation Theory↩︎

In this section, we present our main distribution estimation results for AD-Seq. To ease the presentation, we consider utilizing the continuous-time backward processes 4 and 6 for distribution estimation. In practice, a proper discretization is applied to generate samples, whose deviation from the continuous-time backward process can be controlled by the step size of the discretization; see [52], which requires a Lipschitz score, and [57], which removes this assumption via stochastic localization.

The key step is decomposing the joint distribution learning to a series of conditional distribution learning given the sequential conditional nature of our framework. For learning one-step conditional distributions, we leverage recent advances in conditional diffusion models [70], [81], [89]. Recall the ground-truth joint distribution is \(\mu_0^{[1:H]}\) and we denote the learned joint distribution of \(\tilde{\mathbf{X}}_{T-t_0}^{[1:H]}\) in Algorithm 1 by \(\hat{\mu}_{t_0}^{[1:H]}\). In addition, let \(\mu^h_0[\mathbf{z}]\) be the ground-truth distribution of \(X^h_0 \vert \mathbf{X}_0^{[1:h)} = \mathbf{z}\) and \(\hat{\mu}_{t_0}^h[\mathbf{z}]\) be the learned distribution of \(\tilde{X}_{T-t_0}^h \vert \tilde{\mathbf{X}}_{T-t_0}^{[1:h)} = \mathbf{z}\). The chain rule of TV distance implies that \[\begin{align} {\rm TV}(\mu_0^{[1:H]}, \hat{\mu}_{t_0}^{[1:H]}) \leq \sum_{h = 1}^H \relax_{\mathbf{X}_0^{[1:h)} \sim \mu_0^{[1:h)}}\left[ {\rm TV}\left( \mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}] \right) \right]. \label{eq:tv-chain} \end{align}\tag{16}\] Inspired by 16 , it suffices to bound the one-step conditional distribution for each \(h \in [H]\) in TV distance. The truncated nature of loss function \(\ell^{[k:h]}(\cdot)\) implies \(\hat{\mu}^h_{t_0}[\mathbf{z}^{[1:h)}] = \hat{\mu}^h_{t_0}[\mathbf{z}^{[k:h)}]\) for any \(\mathbf{z}^{[1:h)} \in \mathbb{R}^{h-1}\). Moreover, triangle inequality implies \[\begin{align} \relax_{\mathbf{X}_0^{[1:h)}}\left[ {\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}]) \right] & \leq \underbrace{\relax_{\mathbf{X}_0^{[k:h)}}\left[ {\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[k:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[k:h)}]) \right]}_{\text{learning error}} \nonumber \\ & \qquad + \underbrace{\relax_{\mathbf{X}_0^{[1:h)}}\left[ {\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \mu_0^{h}[\mathbf{X}_0^{[k:h)}]) \right]}_{\text{truncation error}}. \label{eq:tv-triangle} \end{align}\tag{17}\] Here, we overload the notation \(\mu^h_0[\mathbf{z}^{[k:h)}]\) as the distribution of \(X_0^h \vert \mathbf{X}_0^{[k:h)} = \mathbf{z}^{[k:h)}\) for any \(\mathbf{z}\in \mathbb{R}^{h - k}\). We make the following assumption on the truncation error.

Assumption 2. Denote \(d_{\rm trunc} = h-k+1\) as the truncated sequence length. Fix \(h \in [H]\) and considering \(k < h\), it holds that \[\begin{align} \relax_{\mathbf{X}_0^{[1:h)}}\left[ {\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \mu_0^{h}[\mathbf{X}_0^{[k:h)}]) \right] \leq u(d_{\rm trunc}), \end{align}\] where \(u(\cdot)\) only depends on \(d_{\rm trunc}\) and is monotonically decreasing.

This assumption imposes a quantitative decay-of-dependence property in expectation: the expected total variation distance between the conditional law given the full history and that given only the most recent \(d_{\rm trunc}\) observations is bounded by \(u(d_{\rm trunc})\). As \(u\) decreases in \(d_{\rm trunc}\), the dependence on the remote past vanishes progressively. Such an assumption is justified in many financial applications [87], [88]. In particular, our numerical experiments in Section 5.2 show that log returns of the S&P 500 index are close to Markovian [88], [90] Thus, conditioning on a short history is nearly as informative as conditioning on the full past. The decay rate of \(u(\cdot)\) plays an important role in our theoretical analysis. We will explicitly discuss how different decay regimes influence the choice of \(k\) in Section 4.4.

It remains to bound the learning-error term in 17 .

To proceed, we decompose it informally into three sources: the bias from early stopping, the mismatch in the initial distribution, and the error from score estimation. The first source comes from stopping the time-reverse SDE at time \(T-t_0\) rather than evolving it all the way to time \(T\). The second arises because the sampling process is initialized from the standard normal distribution instead of the exact forward marginal at time \(T\). The third is due to replacing the true conditional score along the sampling trajectory by its learned approximation \(\hat{s}^h\). This decomposition is discussed in more detail in Section 4.3.

We then reduce the score-estimation contribution to a statistical learning problem, consisting of an approximation component and a finite-sample statistical error component. Section 4.1 controls the former by showing that the neural network class contains a candidate whose discrepancy from the target conditional score is bounded at a prescribed rate. Section 4.2 controls the latter by proving that the empirical score-matching minimizer attains small population score-matching risk. Combining these two estimates yields a bound on the score-estimation contribution, and hence on the learning error and the overall distribution estimation error.

4.1 Score Approximation↩︎

We devote this subsection to conditional score approximation theory. For ease of exposition, we overload \(\phi^h_\mathbf{z}(0, \cdot)\) as the conditional density of \(X_0^h\) given \(\mathbf{X}_0^{[k:h)} = \mathbf{z}\) when the conditional information is clear from the context. We make the following assumption on each conditional distribution \(X_0^h \vert \mathbf{X}_0^{[k:h)}\).

Assumption 3 (Conditional sub-Gaussianity). There are constants \(C_1^1,C_2^1>0\) such that the marginal density \(\phi^1(0,\cdot)\) satisfies \[\begin{align} \phi^1(0,x) \leq C_1^1\exp\left( -C_2^1\left |x\right |^2/2 \right) \end{align}\] for all \(x\in\mathbb{R}\). For each \(h \in [H]\) and \(k\in [h-1]\), there exist two positive constants \(C_1^h, C_2^h > 0\) and a vector \(\mathbf{v}^h \in \mathbb{R}^{h - k}\) such that \[\begin{align} \phi_{\mathbf{z}}^h(0,x) \leq C_1^h\exp\left( -C_2^h \left |x - (\mathbf{v}^h)^\top \mathbf{z}\right |^2/2 \right) \end{align}\] for all \(x \in \mathbb{R}\) and \(\mathbf{z}\in \mathbb{R}^{h - k}\).

Throughout the analysis, we omit the dependence of \(C_1, C_2\), and \(\mathbf{v}\) on the indices \(k\) and \(h\) for ease of exposition. Assumption 3 imposes sub-Gaussian condition on the marginal probability density \(\phi^1(0, \cdot)\) and all conditional probability densities. Proposition 8 in Appendix 6 verifies that every nondegenerate multivariate Gaussian distribution satisfies Assumption 3. Notably, Assumption 3 does not impose boundedness on the conditioning variable \(\mathbf{z}\), making it well suited for financial settings in which auxiliary time series or unbounded factors often serve as conditioning inputs [88], [91], [92]. This unboundedness of the conditioning variable places our setting beyond the framework of [81]. We address this new challenge by leveraging an additional truncation step; see Theorem 3. As a consequence of Assumption 3, the following lemma shows that all joint distributions of interest are also sub-Gaussian.

Lemma 1. Under Assumption 3, the probability density \(p^{[1:h]}(\cdot)\) of \(\mathbf{X}_0^{[1:h]}\) is sub-Gaussian for each \(h \in [H]\), i.e., there are positive constants \(C_3, C_4 > 0\) such that \[p^{[1:h]}(\mathbf{x}) \leq C_3\exp\left( -C_4\left \|\mathbf{x}\right \|^2/2 \right).\]

Similar to constants \(C_1\) and \(C_2\), the constants \(C_3\) and \(C_4\) are also \(h\)-dependent and we omit the dependence when it is clear from the context.

Proof. We prove the lemma by induction on \(h\). When \(h = 1\), the claim is satisfied with \(C_3 = C_1\) and \(C_4 = C_2\). Assume the inductive hypothesis holds for \(p^{[1:h-1]}(\cdot)\). Set \(C_4' = \min\left\lbrace C_4, C_2 \right\rbrace\). For every \(\mathbf{x}\in \mathbb{R}^h\), it holds that \[\begin{align} p^{[1:h]}(\mathbf{x}) &= p^{[1:h-1]}(\mathbf{x}^{[1:h)})\phi^h_{\mathbf{x}^{[1:h)}}(0, x^h) \nonumber \\ &\leq C_3C_1 \exp\left( -\frac{C_4}{2}\left \|\mathbf{x}^{[1:h)}\right \|^2 \right)\exp\left( -\frac{C_2}{2}\left |x^h - \mathbf{v}^\top \mathbf{x}^{[1:h)}\right |^2 \right) \nonumber \\ &\leq C_3C_1 \exp\left( -\frac{C_4'}{2}\left( \left \|\mathbf{x}^{[1:h)}\right \|^2 + \left |x^h - \mathbf{v}^\top \mathbf{x}^{[1:h)}\right |^2 \right) \right). \label{eq:joint-bound-1} \end{align}\tag{18}\] Define \(\mathbf{M}\in \mathbb{R}^{h \times h}\) as \[\begin{align} \mathbf{M}= \begin{pmatrix} \mathbf{v}\mathbf{v}^\top + \mathbf{I}_{h-1} & -\mathbf{v}\\ -\mathbf{v}^\top & 1 \end{pmatrix}. \end{align}\] We further bound 18 with \[\begin{align} p^{[1:h]}(\mathbf{x}) \leq C_3C_1 \exp\left( -\frac{C_4'}{2} \mathbf{x}^\top \mathbf{M}\mathbf{x} \right). \end{align}\] Here, the matrix \(\mathbf{M}\) is positive definite as long as the Schur complement of the top-left block \(\mathbf{S}\mathrel{\vcenter{:}}= \mathbf{v}\mathbf{v}^\top + \mathbf{I}_{h-1}\) is positive definite. To see this, the Woodbury identity implies that \[\begin{align} \mathbf{M}/\mathbf{S}\mathrel{\vcenter{:}}= 1 - \mathbf{v}^\top\left( \mathbf{v}\mathbf{v}^\top + \mathbf{I}_{h-1} \right)^{-1}\mathbf{v}= 1 - \mathbf{v}^\top\left( \mathbf{I}_{h-1} - \frac{\mathbf{v}\mathbf{v}^\top}{1 + \left \|\mathbf{v}\right \|^2} \right)\mathbf{v}= \frac{1}{1 + \left \|\mathbf{v}\right \|^2}>0. \end{align}\] Combining 18 with \(\mathbf{x}^\top\mathbf{M}\mathbf{x}\geq \lambda_{\min}(\mathbf{M})\left \|\mathbf{x}\right \|^2\) yields \[\begin{align} p^{[1:h]}(\mathbf{x}) \leq C_3C_1\exp\left( -\frac{C_4'\lambda_{\min}(\mathbf{M})}{2}\left \|\mathbf{x}\right \|^2 \right), \end{align}\] which completes the proof. ◻

By applying the marginalization technique, the sub-Gaussian property naturally holds for all joint distributions over any interval \([k,h]\).

Corollary 1. Under Assumption 3, the probability density \(p^{[k:h]}(\cdot)\) of \(\mathbf{X}_0^{[k:h]}\) is sub-Gaussian, i.e., there are constants \(C_5, C_6 > 0\) such that \[p^{[k:h]}(\mathbf{x}) \leq C_5\exp\left( -C_6\left \|\mathbf{x}\right \|^2/2 \right).\] In particular, probability density \(p^h(\cdot)\) of \(X_0^h\) is sub-Gaussian.

In additional to the light-tail property of each conditional distribution, we need the following assumption on the smoothness. Denote \(\mathcal{H}^\beta(\Omega, B)\) by the set of all \(\beta\)-Hölder smooth functions over the domain \(\Omega\) with Hölder norm bounded by \(B\); see [81].

Assumption 4 (Hölder smoothness). The marginal density at \(h = 1\) and all conditional densities are Hölder smooth in the sense that \(\phi^1(0, \cdot) \in\mathcal{H}^\beta(\mathbb{R}, B)\) and \(\phi_{\mathbf{z}}^h(0, \cdot)\in\mathcal{H}^\beta(\mathbb{R}^{h-k+1}, B)\) for all \(1 \leq k < h \leq H\) and \(\mathbf{z}\in \mathbb{R}^{h-k}\).

We are ready to establish score approximation theory. To this end, we rewrite the score function as \(\partial_x \log \phi_\mathbf{z}^h(t, x) = \frac{\partial_x \phi_\mathbf{z}^h(t, x)}{\phi_\mathbf{z}^h(t, x)}\). Following the ideas in [81], we first approximate \(\partial_x \phi_\mathbf{z}^h(t, x)\) and \(\phi_\mathbf{z}^h(t, x)\) with diffused local polynomials. The next result provides polynomial approximations for both the denominator \(\phi_\mathbf{z}^h(t,x)\) and the numerator \(\sigma_t\partial_x\phi_\mathbf{z}^h(t,x)\). Later, we combine these approximations to form a clipped ratio \(f_3\) in 19 , which is then approximated by a neural network.

Lemma 2 (Diffused local polynomial approximation). Suppose Assumptions 3 and 4 hold. For a sufficiently large integer \(N>0\), there exist two polynomials \(f_1(x, \mathbf{z}, t)\) and \(f_2(x, \mathbf{z}, t)\), and a positive constant \(R_1 \asymp \sqrt{\log N}\) such that \[\begin{align} & \left |f_1(x, \mathbf{z}, t) - \phi^h_{\mathbf{z}}(t, x) \right | \lesssim BN^{-\beta}\log^{\frac{\beta + 1}{2}}N, \label{eq:poly-1} \\ & \left |f_2(x, \mathbf{z}, t) - \sigma_t\partial_x \phi_{\mathbf{z}}^h(t, x)\right | \lesssim BN^{-\beta}\log^{\frac{\beta + 2}{2}}N, \label{eq:poly-2} \end{align}\] {#eq: sublabel=eq:eq:poly-1,eq:eq:poly-2} for any \(x \in \mathbb{R}, \mathbf{z}\in [-R_1, R_1]^{h-k}\) and \(t>0\).

The proof adapts the strategy of [81], with an additional truncation step to handle the unbounded conditioning variable. This truncation argument is established in Lemma 5 in Appendix 6. The remaining calculations follow [81].

Proof. We first approximate the density \(\phi^h_{\mathbf{z}}(t, x)\) by an integral evaluated over a bonded domain. Denote the integral by \[\begin{align} \tilde{f}_1(x, \mathbf{z}, t) = \frac{1}{\sigma_t(2\pi)^{1/2}}\int_{\bar{B}} \phi_{\mathbf{z}}^h(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y, \end{align}\] where we define \[\begin{align} \bar{B} \mathrel{\vcenter{:}}= \left[ \frac{x - \sigma_t R_2}{\alpha_t}, \frac{x + \sigma_t R_2}{\alpha_t} \right] \bigcap \left[ \mathbf{v}^\top \mathbf{z}- R_3, \mathbf{v}^\top \mathbf{z}+ R_3 \right], \end{align}\] with \(\mathbf{v}\) as defined in Assumption 3. Later, we will choose \(R_2, R_3 \asymp \sqrt{\beta\log N}\) for some large enough positive integer \(N\). Consequently, Lemma 5 in Section 6 implies that \[\begin{align} \left |\tilde{f}_1(x, \mathbf{z}, t) - \phi^h_{\mathbf{z}}(t, x)\right | \leq N^{-\beta}, \;\; \forall x \in \mathbb{R}, \mathbf{z}\in \mathbb{R}^{h-k}. \end{align}\] Compared to the steps in proving [81], we only need to modify the function \(f\) in (A.24) of [81]. To proceed, we take \[\begin{align} f(y, \mathbf{z}, t) = \phi^h_{2R_1(\mathbf{z}- {\boldsymbol{1}}/2)}(0, 2R_1(y - 1/2) + \mathbf{v}^\top \mathbf{z}), \;\; y \in [0, 1], \mathbf{z}\in [0, 1]^{h - k}, t > 0, \end{align}\] where \({\boldsymbol{1}} \in \mathbb{R}^{h-k}\) is the all one vector and \(R_1 \asymp \sqrt{\beta \log N}\). We calculate the Hölder norm of \(f\) as follows. Let \(g(\cdot)\) be an arbitrary \(\beta\)-Hölder smooth function. If \(\left \|g\right \|_{\mathcal{H}^\beta(\mathbb{R}^d)} \leq B\), then for any \(\mathbf{A}\in \mathbb{R}^{d \times d}\) and \(\mathbf{b}\in \mathbb{R}^d\), it is straightforward to check by definition that \[\begin{align} \left \|g(\mathbf{A}\mathbf{x}+ \mathbf{b})\right \|_{\mathcal{H}^\beta(\mathbb{R}^d)} \leq \left \|\mathbf{A}\right \|^\beta \left \|g\right \|_{\mathcal{H}^\beta(\mathbb{R}^d)} \leq B\left \|\mathbf{A}\right \|^\beta, \;\; \forall \; \mathbf{x}\in \mathbb{R}^d. \end{align}\] Consider the linear transformation \[\begin{align} \begin{pmatrix} y \\ \mathbf{z} \end{pmatrix} \mapsto \underbrace{\begin{pmatrix} 2R_1 & \mathbf{v}^\top \\ \boldsymbol{0} & 2R_1 \mathbf{I}_{h-k} \end{pmatrix}}_\mathbf{A} \begin{pmatrix} y \\ \mathbf{z} \end{pmatrix} \underbrace{- R_1 \begin{pmatrix} 1 \\ \boldsymbol{1} \end{pmatrix}}_\mathbf{b}. \end{align}\] Since the operator norm of \(A\) is bounded with \[\begin{align} \left \|\mathbf{A}\right \| \leq \left \|\mathbf{A}\right \|_F = \sqrt{(h-k)(2R_1)^2 + \left \|\mathbf{v}\right \|^2} \lesssim R_1, \end{align}\] Assumption 4 implies that \(\left \|f\right \|_{\mathcal{H}^\beta} \lesssim BR_1^\beta\). By applying the construction of function \(q(\cdot)\) in (A.26) of [81] with substitution \(\mathbf{x}= y\) and \(\mathbf{y}= \mathbf{z}\), we have \[\begin{align} \left |f(y, \mathbf{z}, t) - q(y, \mathbf{z}, t)\right | \lesssim B\frac{R_1^\beta d_{\rm trunc}^s}{s!N^\beta}. \end{align}\] Finally, we construct a diffused local polynomial as in (A.37) of [81] and follow the same calculation in [81] to deduce \[\begin{align} \left |f_1(x, \mathbf{z}, t) - \phi^h_{\mathbf{z}}(t, x) \right | \lesssim BN^{-\beta}\log^{\frac{\beta + 1}{2}}N, \;\; \forall x \in \mathbb{R}, \mathbf{z}\in [-R_1, R_1]^{h-k}, t>0. \end{align}\] ◻

Let \(\epsilon_{\rm low}> 0\) be a threshold that will be chosen later and fix a large enough positive constant \(C_7 >0\). With the two polynomials defined as in Lemma 2, we construct a score approximator as \[\begin{align} f_3 = \min\left( \frac{f_2}{\sigma_t f_{1, \rm clip}}, \frac{C_7}{\sigma_t^2}\left( R_4 + 1 \right) \right), \label{eq:f953} \end{align}\tag{19}\] where \(f_{1, \rm clip} \mathrel{\vcenter{:}}= \max(f_1, \epsilon_{\rm low})\) and \(R_4 \asymp \sqrt{\log N}\). This construction guarantees that the denominator is bounded away from zero while the range of \(f_3\) is also bounded. The next result establish score approximation with \(f_3\). The proof follows the same calculations as in [81].

Lemma 3. Suppose Assumptions 3 and 4 hold. Let \(f_3\) be defined as in 19 . For sufficiently large integer \(N>0\), it holds that \[\begin{align} \left |f_3(x, \mathbf{z}, t) - \partial_x \log\phi_{\mathbf{z}}^h(t, x)\right | \lesssim \frac{B}{\sigma_t^2 \phi_{\mathbf{z}}^h(t, x)}N^{-\beta}\log^{\frac{\beta+2}{2}}N. \end{align}\] for any \(x \in \mathbb{R}, \mathbf{z}\in [-R_1, R_1]^{h-k}\) and \(t>0\).

With Lemma 3, we will complete the proof of score approximation as long as \(f_3\) can be approximated by a neural network. We make the following assumption on the neural network class \(\mathcal{S}^h\).

Assumption 5. Let \(N > 0\) be a given integer. There exists a neural network \(f^{\rm net}(\cdot) \in \mathcal{S}^h\) such that \(\left \|f^{\rm net}\right \|_{\infty} \lesssim \frac{\sqrt{\log N}}{\sigma_t^2}\) and moreover \[\begin{align} \left |f^{\rm net}(x, \mathbf{z}, t) - f_3(x, \mathbf{z}, t)\right | \lesssim N^{-\beta}, \; \forall x \in [-R_1, R_1], \mathbf{z}\in [-R_1, R_1]^{h-k}, t\in [t_0, T]. \end{align}\]

Since \(f_3\) is Lipschtiz continuous over the compact domain, many universal approximation results [93], [94] guarantee the existence of neural networks as required in Assumption 1. In Section 4.4, we will justify this assumption using ReLU networks as a concrete example. We are ready to state the main result in this subsection.

Theorem 3 (Score approximation). Suppose Assumptions 3, 4, 5 hold. Let \(N > 0\) be a sufficiently large integer and we set early-stop time \(t_0 = N^{-C_\sigma}\) and terminal time \(T = C_\alpha \log N\) for some positive constants \(C_\sigma\) and \(C_\alpha\). There is a neural network \(s^\star \in \mathcal{S}^h\) such that \[\begin{align} \label{eq:univ-approx} \relax\left[ \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \lesssim \frac{B}{\sigma_t^4}N^{-\beta}(\log N)^{2 + \beta/2}, \;\; \forall t \in [t_0, T]. \end{align}\qquad{(1)}\]

Theorem 3 establishes a score approximation result over the time interval \([t_0, T]\). It will serve as a building block for the subsequent score estimation theory. Compared with prior work [81], the main technical difference is the introduction of an additional truncation step. This is necessary because the conditional information \(\mathbf{X}_0^{[k:h)}\) is not assumed to be bounded.

Proof. Let \(s^\star(t, x, \mathbf{z}) = f^{\rm net}(x, \mathbf{z}, t)\) given in Assumption 5. It follows from Lemma 2 that \[\begin{align} \left |s^\star(x, \mathbf{z}, t) - \partial_x \log\phi_{\mathbf{z}}(t, x)\right | \lesssim \frac{B}{\sigma_t^2 \phi_{\mathbf{z}}(t, x)}N^{-\beta}\log^{\frac{\beta+2}{2}}N. \label{eq:approx32trans} \end{align}\tag{20}\] when \(x, \mathbf{z}, t\) are bounded as in Assumption 5.

Consider the following decomposition \[\begin{align} & \relax\left[ \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \\ & \leq \underbrace{\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right]}_{(E_1)} + \\ & \qquad + \underbrace{\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | > R_5 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right]}_{(E_2)} \\ & \qquad + \underbrace{\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h}- \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | \leq R_5 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right]}_{(E_3)}, \end{align}\] where \(R_1\) and \(R_5\) will be chosen later.

We begin with the error term \((E_1)\). Note that \[\begin{align} (E_1) \leq 2\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)})\right |^2 \right] + 2\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |\partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right]. \end{align}\] As \(\left |s^\star(x, \mathbf{z}, t)\right | \lesssim \frac{\sqrt{\log N}}{\sigma_t^2}\) by Assumption 1, we have \[\begin{align} \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)})\right |^2 \right] & \lesssim \frac{\log N}{\sigma_t^4} \int_{\left \|\mathbf{z}\right \|_\infty > R_1}p^{[k:h)}( \mathbf{z}) {\rm d}\mathbf{z}\nonumber \\ & \lesssim \frac{\log N}{\sigma_t^4}\int_{\left \|\mathbf{z}\right \|_\infty > R_1}\exp\left( -C_5\left \|\mathbf{z}\right \|^2/2 \right){\rm d}\mathbf{z}\nonumber \\ & \lesssim \frac{\log N}{\sigma_t^4} (R_1)^{-1}\exp\left( -C_5R_1^2/2 \right),\label{eq:E1-1} \end{align}\tag{21}\] where \(C_5\) is given in Corollary 1.

Combining 21 and Lemma 7, and choosing \(R_1 \geq \sqrt{\frac{4\beta}{\min(C_5, C_6)} \log N}\), we bound the term \((E_1)\) with \[\begin{align} (E_1) \lesssim \frac{\log N}{\sigma_t^4} (R_1)^{-1}\exp\left( -C_5R_1^2/2 \right) + \frac{1}{\sigma_t^4} \exp(-C_6 R_1^2/2) \lesssim \frac{\sqrt{\log N}}{\sigma_t^4}N^{-2\beta}. \label{eq:E1-bound} \end{align}\tag{22}\]

Next, we bound \((E_2)\) with \[\begin{align} (E_2) & \leq 2\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | > R_5 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) \right |^2 \right]\nonumber \\ & \qquad + 2\relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h}- \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | > R_5 \right\rbrace\left |\partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \label{eq:E2-1} \end{align}\tag{23}\] Lemma 8 implies \[\begin{align} & \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | > R_5 \right\rbrace\left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) \right |^2 \right] \nonumber \\ & \lesssim \frac{\log N}{\sigma_t^4}\int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}\int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_5} \phi^h_{\mathbf{z}}(t, x) {\rm d}x p^{[k:h)}( \mathbf{z}) {\rm d}\mathbf{z}\nonumber \\ & \lesssim \frac{\log N}{\sigma_t^4}(R_5)^{-1}\exp\left( -C_2' R_5^2/2 \right), \label{eq:E2-2} \end{align}\tag{24}\] and moreover \[\begin{align} & \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h}- \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | > R_5 \right\rbrace\left |\partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \nonumber \\ & = \int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}\int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_5} \phi^h_{\mathbf{z}}(t, x)\left |\partial_x \log \phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x p^{[k:h)}(\mathbf{z}) {\rm d}\mathbf{z}\nonumber \\ & \lesssim \frac{1}{\sigma_t^4}R_5\exp\left( -C_2'R_5^2/2 \right). \label{eq:E2-3} \end{align}\tag{25}\] With 23 , 24 and 25 , we choose \(R_5 \geq \sqrt{\frac{4\beta}{C_2'}\log N}\) to obtain an upper bound for \((E_2)\): \[\begin{align} (E_2) & \lesssim {\sigma_t^{-4}}(R_5)^{-1}\exp\left( -C_2'R_5^2/2 \right) + \frac{1}{\sigma_t^4}R_5\exp\left( -C_2'R_5^2/2 \right) \nonumber \\ & \lesssim \frac{\log N}{\sigma_t^4}\cdot \frac{N^{-2\beta}}{\sqrt{\log N}} + \frac{\sqrt{\log N}}{\sigma_t^4}N^{-2\beta} \lesssim \frac{\sqrt{\log N}}{\sigma_t^4}N^{-2\beta}. \label{eq:E2-bound} \end{align}\tag{26}\]

We further bound \((E_3) \leq (E_4) + (E_5)\) with \[\begin{align} & (E_4) \mathrel{\vcenter{:}}= \relax\bigg [1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | \leq R_5, \left |\phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right | < \epsilon_{\rm low} \right\rbrace \\ & \qquad\qquad\qquad\qquad\cdot \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\bigg] \\ & (E_5) \mathrel{\vcenter{:}}= \relax\bigg [1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | \leq R_5, \left |\phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right | \geq \epsilon_{\rm low} \right\rbrace \\ & \qquad\qquad\qquad\qquad\cdot \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\bigg] \end{align}\] We bound \((E_4)\) with \[\begin{align} (E_4) & \leq 2\relax\bigg[1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | \leq R_5, \left |\phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right | < \epsilon_{\rm low} \right\rbrace \\ &\times \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)})\right |^2\bigg] \\ & \quad + 2\relax\bigg[1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty \leq R_1, \left |X_t^{h} - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right | \leq R_5, \left |\phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right | < \epsilon_{\rm low} \right\rbrace \\ &\times \left |\partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\bigg], \end{align}\] where \(\epsilon_{\rm low} > 0\) will be chosen later. Lemma 9 implies the following upper bound for \((E_4)\): \[\begin{align} (E_4) \lesssim \int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}\left( \frac{\log N}{\sigma_t^4} R_5 \epsilon_{\rm low} + \frac{\epsilon_{\rm low}}{\sigma_t^4} R_5^3 \right) p^{[k:h)}(\mathbf{z}) {\rm d}\mathbf{z}\lesssim \frac{\epsilon_{\rm low}}{\sigma_t^4} \cdot (\log N)^{3/2}.\label{eq:E4-bound} \end{align}\tag{27}\] By the approximation guarantee 20 , for any fixed \(\mathbf{z}\in [-R_1, R_1]^{h-k}\) we deduce that \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |\leq R_5}1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | \geq \epsilon_{\rm low} \right\rbrace \left |s^\star(x, \mathbf{z}, t) - \partial_x \log\phi^h_{\mathbf{z}}(t, x)\right |^2\phi^h_{\mathbf{z}}(t, x) {\rm d}x \\ & \lesssim \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |\leq R_5}1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | \geq \epsilon_{\rm low} \right\rbrace\frac{B^2}{\sigma_t^4 \phi^h_{\mathbf{z}}(t, x)}N^{-2\beta}\log^{\beta+2}N {\rm d}x \\ & \lesssim \frac{B^2 \log^{\beta + 2}N}{\sigma_t^4\epsilon_{\rm low}}N^{-2\beta}R_5. \end{align}\] Consequently, we obtain an upper bound for \((E_5)\): \[\begin{align} (E_5) \lesssim \frac{B^2 \log^{\beta + 2}N}{\sigma_t^4\epsilon_{\rm low}}N^{-2\beta}R_5 \int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}p^{[k:h)}(\mathbf{z}) {\rm d}\mathbf{z}\leq \frac{B^2 \log^{5/2 + \beta}N}{\sigma_t^4\epsilon_{\rm low}}N^{-2\beta}. \label{eq:E5-bound} \end{align}\tag{28}\] Combining 22 , 26 , 27 and 28 , we have \[\begin{align} \relax\left[ \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \lesssim \frac{\sqrt{\log N}}{\sigma_t^4}N^{-2\beta} + \frac{\epsilon_{\rm low}(\log N )^{3/2}}{\sigma_t^4} + \frac{B^2 \log^{5/2 + \beta}N}{\sigma_t^4\epsilon_{\rm low}}N^{-2\beta}. \label{eq:L2-approx-1} \end{align}\tag{29}\] Substitution \(\epsilon_{\rm low} = B N^{-\beta}\log^{\frac{1+\beta}{2}}N\) back into 29 , we conclude \[\begin{align} \relax\left[ \left |s^\star(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \lesssim \frac{B}{\sigma_t^4}N^{-\beta}(\log N)^{2 + \beta/2}, \;\; \forall t \in [t_0, T]. \end{align}\] ◻

4.2 Score Estimation↩︎

In this section, we present the score estimation result for one-step conditional diffusion models. Define the risk function as \[\begin{align} \mathcal{R}^{[k:h]}(s) = \int_{t_0}^T \frac{1}{T - t_0} \relax_{\mathbf{X}_0^{[k:h)}}\relax_{X_0^{h} | \mathbf{X}_0^{[k: h)}}\relax_{X_t^h\vert X_0^h}\left[ \left |s(t, X_t^h, \mathbf{X}_0^{[k:h)}) - \partial_x \log \phi_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right]{\rm d}t. \label{eq:risk} \end{align}\tag{30}\] Our goal is to bound expected risk \(\relax_\mathcal{D}[\mathcal{R}^{[k:h]}(\hat{s})]\), where we recall that \(\hat{s}^h\) is a minimizer to the empirical score matching risk \(\widehat{\mathcal{L}}^{[k:h]}(s) = \frac{1}{n}\sum_{i = 1}^n \ell^{[k:h]}(\mathbf{x}_i^{[k:h]}; s)\) with \(\ell^{[k:h]}(\cdot; s)\) defined in 15 . In the subsequent analysis, we suppress the superscript \([k:h]\) when the context is clear. Recall the dataset \(\mathcal{D}= \left\lbrace \mathbf{x}_i \right\rbrace_{i = 1}^n\).

One key step towards the score estimation theory is bounding the logarithmic covering number of a function class over a compact domain. To this end, it is convenient to recenter the empirical by defining \[\begin{align} \ell_0(x, \mathbf{z}; s) = \ell(x, \mathbf{z}; s) - \ell(x, \mathbf{z}; \bar{s}), \;\; \text{with} \;\; \bar{s}(t, x, \mathbf{z}) = \partial_x \log \phi_\mathbf{z}^h(t, x). \label{eq:centered32loss} \end{align}\tag{31}\] Moreover, we define a truncated loss associated with \(\ell(\cdot; s)\) as \[\begin{align} \ell^{\rm trunc}(x, \mathbf{z}; s) = \ell(x, \mathbf{z}; s)1\left\lbrace \left |x - \mathbf{v}^\top \mathbf{z}\right | \leq R_3, \left \|\mathbf{z}\right \|_\infty \leq R_1 \right\rbrace. \label{eq:truncated32loss} \end{align}\tag{32}\] The truncated centered loss \(\ell_0^{\rm trunc}(\cdot; s)\) is defined in a similar way. Let \(\mathcal{G}= \left\lbrace \ell_0^{\rm trunc}(\cdot; s): s \in \mathcal{S}^h \right\rbrace\) be a loss function class induced by the neural network class \(\mathcal{S}^h\). We make the following definition on the network complexity.

Definition 1 (Network Complexity). We denote the the logarithmic covering number \(\log(\mathscr{N}(\delta))\) as Network-Complexity.

We are ready to state the main result of this subsection.

Theorem 4 (Score estimation). Suppose Assumptions 3 and 4 hold and let \(\mathcal{S}^h\) be a neural network class satisfying Assumptions 5. Then it holds that \[\begin{align} \relax_{\mathcal{D}}[\mathcal{R}(\hat{s})] \lesssim \frac{\sqrt{\log N}}{t_0}N^{-\beta} + \frac{1}{t_0}N^{-\beta}(\log N)^{2+ \beta/2} + \frac{\log N}{nt_0}\cdot \texttt{Network-Complexity} + \delta. \end{align}\]

Theorem 4 bounds the score estimation error by four error sources: the first term is the truncation error from restricting the unbounded conditioning variable to a bounded domain; the second term is the score approximation error inherited from Theorem 3; the third term is the statistical error over a prescribed network class; and the fourth term \(\delta\) is the covering resolution of \(\mathcal{G}\). We emphasize that the analysis is agnostic to the choice of network class: any class satisfying Assumption 5 can be substituted into Theorem 4. Section 4.4 instantiates the ReLU networks to obtain a concrete rate.

Proof. We follow the proof strategy as in [74] and only mention the differences here. With \(\ell_0(\cdot; s)\) in 31 and the dataset \(\mathcal{D}\), we define centered empirical risk as \[\begin{align} \widehat{\mathcal{L}}_0(s) \mathrel{\vcenter{:}}= \frac{1}{n}\sum_{i=1}^n \ell_0(x_i^h, \mathbf{x}_i^{[k:h)}; s) \end{align}\] Note that \(\widehat{\mathcal{L}}_0(s)\) and \(\widehat{\mathcal{L}}(s)\) share the same minimizer \(\hat{s}\). One key observation is that population centered risk is exactly the risk we would like to minimize, i.e., \[\begin{align} \mathcal{L}_0(s) \mathrel{\vcenter{:}}= \relax_{\mathbf{X}_0^{[k:h]}}\left[ \ell_0(X_{0}^h, \mathbf{X}_{0}^{[k:h)}, s) \right] = \mathcal{R}(s). \label{eq:L0-R} \end{align}\tag{33}\] We define truncated versions \(\mathcal{L}_0^{\rm trunc}\) and \(\widehat{\mathcal{L}}_0^{\rm trunc}\) using \(\ell_0^{\rm trunc}(\cdot; s)\) accordingly.

Consider the following decomposition \[\begin{align} \relax_\mathcal{D}[\mathcal{R}(\hat{s})] = \underbrace{\relax_\mathcal{D}[\mathcal{R}(s^\star)]}_{\rm (I)} + \underbrace{\relax_\mathcal{D}[\mathcal{R}(\hat{s}) - \mathcal{R}(s^\star)]}_{\rm (II)}. \end{align}\] Here, \(s^\star\) is the constructed score approximator given in Theorem 3 with approximation error: \[\begin{align} {\rm (I)} \lesssim \frac{1}{T - t_0}\int_{t_0}^T \frac{B}{\sigma_t^4}N^{-\beta}(\log N)^{2 + \beta/2} {\rm d}t \lesssim \frac{1}{t_0}N^{-\beta}(\log N)^{2+ \beta/2}, \label{eq:score-est-1} \end{align}\tag{34}\] where we apply Lemma D.1 in [81] to bound the integral \(\frac{1}{T-t_0}\int_{t_0}^T \sigma_t^{-4}{\rm d}t = \mathcal{O}(1/t_0)\). We proceed with the calculation as in [74]. By applying the fact that \(\hat{s}\) is the minimizer of \(\widehat{\mathcal{L}}_0\), we further bound \({\rm (II)}\) with \[\begin{align} {\rm (II)} & \leq \underbrace{\relax_\mathcal{D}[\mathcal{L}_0(\hat{s}) - \widehat{\mathcal{L}}_0(\hat{s})]}_{\rm (II-A)}. \end{align}\]

4.2.0.1 Bounding term (II-A).

This part is an extension from a bounded domain to an unbounded domain with truncation argument. The key difference compared to [74] is the truncation error and the covering number. Specifically, we first have \[\begin{align} \text{(II-A)} \leq \underbrace{\relax_{\mathcal{D}, \bar{\mathcal{D}}}\left[ \sup_{s\in \mathcal{S}}\frac{1}{n}\sum_{i=1}^n[\ell_0(\bar{x}_{i}^h, \bar{\mathbf{x}}_{i}^{[k:h)}, s), \ell_0(x_{i}^h, \mathbf{x}_{i}^{[k:h)}, s)] - a\mathcal{R}(s) \right]}_{(\spadesuit)} + a\relax_\mathcal{D}[\mathcal{R}(\hat{s})], \label{eq:score-est-3} \end{align}\tag{35}\] where \(a > 0\) is a parameter to control bias-variance trade-off commonly used in the literature. Here, \(\bar{\mathcal{D}} \mathrel{\vcenter{:}}= \left\lbrace \bar{\mathbf{x}}_{i}^{[k:h]} \right\rbrace_{i=1}^n\) is an independent copy of samples following the same distribution as \(\mathcal{D}\). We use the covering number to bound \((\spadesuit)\). We construct a covering of \(\mathcal{G}\) in \(L^\infty\)-norm as follows. Let \(\delta > 0\) be given. We select a collection of score networks \(s_j, j = 1, \dots, \mathcal{N}(\delta)\) such that for any \(s \in \mathcal{S}\) there is an index \(j\) with \[\left \|\ell_0^{\rm trunc}(\cdot; s) - \ell_0^{\rm trunc}(\cdot; s_j)\right \|_\infty = \left \|\ell^{\rm trunc}(\cdot; s) - \ell^{\rm trunc}(\cdot; s_j)\right \|_\infty \leq \delta.\] Recall that \(B_{2} = \left[ \mathbf{v}^\top \mathbf{z}- R_3, \mathbf{v}^\top \mathbf{z}+ R_3 \right]\). Note that for each \(s \in \mathcal{S}\) (\(s\) can depend on \(\mathbf{X}_0^{[k:h]}\)) we have \[\begin{align} & (T - t_0)\relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}; s) - \ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}; s)\right | \right] \\ & = \int_{t_0}^T \int_{\left \|\mathbf{z}\right \|_\infty > R_1}\int_{\mathbb{R}}\relax_{X_t^h\vert X_0^h=y}\left[ \left |s(t, X_t^h, \mathbf{z}) - \frac{\alpha_t y - X_t^h}{\sigma_t^2}\right |^2 \right]\phi_{\mathbf{z}}^h(0, y)p^{[k:h)}(\mathbf{z}){\rm d}y {\rm d}\mathbf{z}{\rm d}t \\ & \qquad + \int_{t_0}^T \int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}\int_{\mathbb{R}\setminus B_2}\relax_{X_t^h\vert X_0^h=y}\left[ \left |s(t, X_t^h, \mathbf{z}) - \frac{\alpha_t y - X_t^h}{\sigma_t^2}\right |^2 \right]\phi^h_{\mathbf{z}}(0, y)p^{[k:h)}(\mathbf{z}){\rm d}y {\rm d}\mathbf{z}{\rm d}t\\ & \leq 2 \int_{t_0}^T \int_{\left \|\mathbf{z}\right \|_\infty > R_1}\int_{\mathbb{R}}\relax_{X_t^h\vert X_0^h=y}\left[ \left |s(t, X_t^h, \mathbf{z})\right |^2 + \left |\frac{\alpha_t y - X_t^h}{\sigma_t^2}\right |^2 \right]\phi^h_{\mathbf{z}}(0, y)p^{[k:h)}(\mathbf{z}){\rm d}y {\rm d}\mathbf{z}{\rm d}t \\ & \qquad + 2 \int_{t_0}^T \int_{\left \|\mathbf{z}\right \|_\infty \leq R_1}\int_{\mathbb{R}\setminus B_2}\relax_{X_t^h\vert X_0^h=y}\left[ \left |s(t, X_t^h, \mathbf{z})\right |^2 + \left |\frac{\alpha_t y - X_t^h}{\sigma_t^2}\right |^2 \right]\phi^h_{\mathbf{z}}(0, y)p^{[k:h)}(\mathbf{z}){\rm d}y {\rm d}\mathbf{z}{\rm d}t. \end{align}\] Since \(\phi_\mathbf{z}^h(0, \cdot)\) and \(p^{[k:h)}(\cdot)\) are both sub-Gaussian (Assumption 3 and Corrolary 1), we deduce that \[\begin{align} & (T - t_0)\relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}; s) - \ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}; s)\right | \right] \\ & \leq 2\int_{t_0}^T \int_{\left \|\mathbf{z}\right \|_\infty > R_1} \left( \frac{\log N}{\sigma_t^4} + \frac{1}{\sigma_t^2} \right) \exp(-C_6 \left \|\mathbf{z}\right \|^2/2) {\rm d}\mathbf{z}{\rm d}t \\ & \qquad + 2\int_{t_0}^T \int \int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | > R_3} \left( \frac{\log N}{\sigma_t^4} + \frac{1}{\sigma_t^2} \right) \exp(-C_2 \left |y - \mathbf{v}^\top \mathbf{z}\right |^2/2) {\rm d}y {\rm d}\mathbf{z}{\rm d}t\\ & \leq 2(1+\log N) \big( \exp(-C_6 R_1^2/2)(R_1)^{-1} + \exp(-C_2 R_3^2/2)(R_3)^{-1} \big)\int_{t_0}^T\frac{1}{\sigma_t^4}{\rm d}t \end{align}\] We take \(R_1 \geq \sqrt{\frac{2\beta}{C_6}\log N}\) and \(R_3 \geq \sqrt{\frac{2\beta}{C_2}\log N}\) to deduce \[\begin{align} \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}; s) - \ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}; s)\right | \right] \lesssim \frac{\sqrt{\log N}}{t_0}N^{-\beta}. \label{eq:l-trunc-diff} \end{align}\tag{36}\]

For any \(s \in \mathcal{S}^h\) and its close representation \(s_j\) in terms of \(\ell_0^{\rm trunc}\), it follow from 36 that \[\begin{align} & \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell_0(X_0^h, \mathbf{X}_0^{[k:h)}, s) - \ell_0(X_0^h, \mathbf{X}_0^{[k:h)}, s_j)\right | \right] = \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}, s) - \ell(X_0^h, \mathbf{X}_0^{[k:h)}, s_j)\right | \right] \\ & \leq \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}, s) - \ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}, s)\right | \right] + \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}, s) - \ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}, s_j)\right | \right] \\ & \qquad + \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell^{\rm trunc}(X_0^h, \mathbf{X}_0^{[k:h)}, s_j) - \ell(X_0^h, \mathbf{X}_0^{[k:h)}, s_j)\right | \right] \\ & \lesssim \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta. \end{align}\] Furthermore, Eq. 33 implies \[\begin{align} \left |\mathcal{R}(s) - \mathcal{R}(s_j)\right | = \left |\mathcal{L}_0(s) - \mathcal{L}_0(s_j)\right | & = \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell_0(X_0^h, \mathbf{X}_0^{[k:h)}, s) - \ell_0(X_0^h, \mathbf{X}_0^{[k:h)}, s_j)\right | \right] \\ & \leq \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta. \end{align}\] Consequently, Eq. 35 becomes \[\begin{align} (\spadesuit) \leq \relax_{\mathcal{D}, \bar{\mathcal{D}}}\left[ \max_{j}\frac{1}{n}\sum_{i=1}^n[\ell_0(\bar{X}_{0, i}^h, \bar{X}_{0, i}^{[k:h)}, s_j)- \ell_0(X_{0, i}^h, X_{0, i}^{[k:h)}, s_j)] - a\mathcal{R}(s_j) \right] + (a+2)\left( \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta \right). \end{align}\]

To proceed, we prove a modified version of Lemma C.1 in [74] over an unbounded domain. Note that for any fixed \(s \in \mathcal{S}^h\), it holds that \[\begin{align} \ell(y, \mathbf{z}; s) & \leq \frac{2}{T- t_0}\int_{t_0}^T\relax_{X_t^h\vert X_0^h=y}\left[ \left |s(t, X_t^h, \mathbf{z})\right |^2 + \left |\frac{\alpha_t y - X_t^h}{\sigma_t^2}\right |^2 \right]{\rm d}t \\ & \leq \frac{2}{T- t_0}\int_{t_0}^T\relax_{X_t^h\vert X_0^h=y}\left[ \frac{\log N}{\sigma_t^4} + \frac{1}{\sigma_t^2} \right]{\rm d}t \\ & \leq \frac{2(1+ \log N)}{T-t_0}\int_{t_0}^T \frac{1}{\sigma_t^4}{\rm d}t \leq \frac{4(1+ \log N)}{t_0}, \end{align}\] Similarly, we have \[\begin{align} \left |\ell(y, \mathbf{z}; s) - \ell(y, \mathbf{z}; s^\star) \right |^2 \leq \frac{16(1+\log N)}{t_0(T-t_0)}\int_{t_0}^T\relax_{X_t^h\vert X_0^h=y}\left[ \left |(s(t, x, \mathbf{z}) - s^\star(t, x, \mathbf{z}))\right |^2 \right].\label{eq:l-diff} \end{align}\tag{37}\] Taking expectation over 37 to obtain \[\begin{align} & \relax_{\mathbf{X}_0^{[k:h]}}\left[ \left |\ell(X_0^h, \mathbf{X}_0^{[k:h)}; s) - \ell(X_0^h, \mathbf{X}_0^{[k:h)}; s^\star) \right |^2 \right] \\ & \leq \frac{16(1+\log N)}{t_0(T-t_0)}\int_{t_0}^T \relax\left[ \left |(s(t, x, \mathbf{z}) - s^\star(t, x, \mathbf{z}))\right |^2 \right]{\rm d}t = \frac{16(1 + \log N)}{t_0}\mathcal{R}(s). \end{align}\]

Denote \(h_i(s) = \ell_0(\bar{x}_{i}^h, \bar{\mathbf{x}}_{i}^{[k:h)}, s_j) - \ell_0(x_{ i}^h, \mathbf{x}_{i}^{[k:h)}, s_j)\) for each fixed \(s \in \mathcal{S}^h\). Let \(C_\ell \mathrel{\vcenter{:}}= 16(1+\log N)/t_0\). We have \[\begin{align} {\rm Var}[h_i(s)] \leq C_\ell \mathcal{R}(s), \;\; \text{and} \;\; \left |h_i(s)\right | \leq C_\ell. \end{align}\] The analysis in [74] implies \[\begin{align} (\spadesuit) \leq \frac{1}{\lambda}\log \mathscr{N}(\delta) + (a+2)\left( \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta \right). \end{align}\] with \(\lambda = \frac{6an}{(2a+3)C_\ell}\). Hence, we apply Definition 1 to deduce \[\begin{align} (\spadesuit) \lesssim \frac{(2a+3)C_\ell}{6an}\cdot \log \mathscr{N}(\delta) + (a+2)\left( \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta \right). \label{eq:score-est-4} \end{align}\tag{38}\] Combining 34 , 35 and 38 , we have \[\begin{align} \relax_\mathcal{D}[\mathcal{R}(\hat{s})] - a\relax_\mathcal{D}[\mathcal{R}(\hat{s})] & \lesssim \frac{1}{t_0}N^{-\beta}(\log N)^{2+ \beta/2} + \frac{(2a+3)C_\ell}{6an}\cdot \log \mathscr{N}(\delta) \\ & \qquad + (a+2)\left( \frac{2\sqrt{\log N}}{t_0}N^{-\beta} + \delta \right). \end{align}\] We pick \(a = 1/2\) and deduce that \[\begin{align} \relax_\mathcal{D}[\mathcal{R}(\hat{s})] & \lesssim \frac{\sqrt{\log N}}{t_0}N^{-\beta} + \frac{1}{t_0}N^{-\beta}(\log N)^{2+ \beta/2} + \frac{\log N}{nt_0}\cdot \log \mathscr{N}(\delta) + \delta. \end{align}\] Therefore, we complete the proof. ◻

4.3 Distribution Estimation↩︎

In this subsection, we apply the score estimation theory from the previous subsection to obtain one-step and joint distribution estimation theory. Recall \(d_{\rm trunc} = h - k + 1\).

Theorem 5 (Distribution estimation). Let \(h \in [H]\) and \(k \in [h-1]\), and assume that \[\relax_{\mathbf{X}_0^{[k:h)}}\bigl[{\rm KL}(\mu_0^h[\mathbf{X}_0^{[k:h)}]\,\|\,\mathcal{N}(0, 1))\bigr] < \infty.\] Under Assumptions 2, 3, 4, and 5, it holds that \[\begin{align} \relax_{\mathcal{D}} [\relax_{\mathbf{X}_0^{[1:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}])]] \lesssim \sqrt{t_0}(\log(1/t_0))^{\frac{d_{\rm trunc}}{2}} + \exp(-T) + \sqrt{T\relax_{\mathcal{D}}[\mathcal{R}(\hat{s})]} + u(d_{\rm trunc}). \end{align}\]

Theorem 5 translates the score estimation result in Theorem 4 into a distribution estimation guarantee. The first term is the early-stopping error from terminating the sampling process at \(T-t_0\in(0,T)\), and the second is the initialization error from starting the sampling process at the standard Gaussian rather than the true forward marginal at time \(T\). The third term propagates the score estimation error through the sampling process, with \(\relax_{\mathcal{D}}[\mathcal{R}(\hat{s})]\) controlled by Theorem 4. The last term, \(u(d_{\rm trunc})\), is the truncation error, controlled by Assumption 2.

Proof. The proof largely follows from [81]. We first check that condition (D.20) in [81] holds. Note that for any \(s \in \mathcal{S}^h\) and \(\mathbf{z}\in \mathbb{R}^{h - k}\), Assumption 1, Lemmas 4 and 6 imply that \[\begin{align} & \int \phi^h_{\mathbf{z}}(t, x) \left |s(t, x, \mathbf{z}) - \partial_x \log\phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x \\ & \lesssim \int \phi^h_{\mathbf{z}}(t, x)\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2 + \log N}{\sigma_t^4} {\rm d}x \\ & \lesssim \int \exp\left( -\frac{C_2'}{2}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2 \right) \frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2 + \log N}{\sigma_t^4} {\rm d}x \\ & = \int \exp\left( -\frac{C_2'}{2}u^2 \right) \frac{u^2 + \log N}{\sigma_t^4} {\rm d}u \lesssim \frac{1}{\sigma_t^4}. \end{align}\]

We also prove an analog to [81] to bound the early stopping error. Let \(\mathbf{z}\in \mathbb{R}^{h-k}\) be given. Recall in Lemma 5 we define \(B_2 = \left[ \mathbf{v}^\top \mathbf{z}- R_3, \mathbf{v}^\top \mathbf{z}+ R_3 \right]\) and furthermore we denote \(B_3 \mathrel{\vcenter{:}}= \left[ \alpha_t \mathbf{v}^\top \mathbf{z}- R_6, \alpha_t \mathbf{v}^\top \mathbf{z}+ R_6 \right]\). Then we first decompose the TV distance as \[\begin{align} {\rm TV}(\mu_0^{h}[\mathbf{z}], \mu_{t_0}^{h}[\mathbf{z}]) & = \frac{1}{2}\int_{(B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | {\rm d}x \nonumber \\ & \qquad + \frac{1}{2}\int_{\mathbb{R}\setminus (B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | {\rm d}x. \label{eq:early32stop32decomp} \end{align}\tag{39}\] For the second term in 39 , Assumption 3 and Lemma 8 imply \[\begin{align} \int_{\mathbb{R}\setminus (B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | {\rm d}x & \leq \int_{\mathbb{R}\setminus B_2 } \phi^h_{\mathbf{z}}(0, x) {\rm d}x + \int_{\mathbb{R}\setminus B_3} \phi^h_{\mathbf{z}}(t, x) {\rm d}x \nonumber\\ & \lesssim (R_3)^{-1}\exp(-C_2R_3^2/2) + (R_6)^{-1}\exp(-C_2'R_6^2/2) \lesssim \epsilon. \label{eq:tv-1} \end{align}\tag{40}\] as long as we take \(R_3, R_6 \asymp \sqrt{\log(1/\epsilon)}\). Take expectation of \(\mathbf{X}_0^{[k:h)}\) to have \[\begin{align} \relax_{\mathbf{X}_0^{[k:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[k:h)}], \mu_{t_0}^{h}[\mathbf{X}_0^{[k:h)}])] \lesssim \int \int_{(B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | p^{[k:h)}( \mathbf{z}) {\rm d}x {\rm d}\mathbf{z}+ \epsilon. \end{align}\] We consider a region \(B_4 \mathrel{\vcenter{:}}= \left\lbrace \mathbf{z}:\left \|\mathbf{z}\right \|_\infty \leq R_1 \right\rbrace\). Then we have \[\begin{align} \int_{\mathbb{R}\setminus B_4} \int_{(B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | p^{[k:h)}( \mathbf{z}) {\rm d}x {\rm d}\mathbf{z}\lesssim \epsilon\sqrt{\log(1/\epsilon)}, \label{eq:tv-4} \end{align}\tag{41}\] as long as we choose \(R_1 \asymp \sqrt{\log(1/\epsilon)}\).

To bound the first term in 39 , we let \(K_t(x, y) = \frac{1}{\sigma_t(2\pi)^{1/2}}\exp\big(-(2\sigma_t^2)^{-1}\left |x - \alpha_t y\right |^2\big)\) be an auxiliary kernel. Recall \(B_{1} = \left[ \frac{x - \sigma_t R_2}{\alpha_t}, \frac{x + \sigma_t R_2}{\alpha_t} \right]\). Since \(\int_\mathbb{R}K_t(x, y) {\rm d}y = (\alpha_t)^{-1}\), we have \[\begin{align} \phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x) & = \alpha_t\int_\mathbb{R}\phi^h_{\mathbf{z}}(0, x) K_t(x, y) {\rm d}y - \int_\mathbb{R}\phi^h_{\mathbf{z}}(0, y) K_t(x, y) {\rm d}y \nonumber \\ & = \int_\mathbb{R}(\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(0, y)) \alpha_t K_t(x, y) {\rm d}y + (\alpha_t - 1)\phi^h_{\mathbf{z}}(t, x) \nonumber \\ & = \int_{B_1} (\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(0, y)) \alpha_t K_t(x, y) {\rm d}y + (\alpha_t - 1)\phi^h_{\mathbf{z}}(t, x) \nonumber \\ & \qquad + \int_{\mathbb{R}\setminus B_1} (\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(0, y)) \alpha_t K_t(x, y) {\rm d}y. \label{eq:phi-0-diff} \end{align}\tag{42}\] Since \(\phi^h_{\mathbf{z}}(0, x) \leq C_1\) for any \(x\) by Assumption 3, we deduce that \[\begin{align} \int_{\mathbb{R}\setminus B_1} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(0, y)\right | \alpha_t K_t(x, y) {\rm d}y \lesssim \int_{\left |u\right | > R_2}\exp(-u^2/2){\rm d}u \lesssim (R_2)^{-1}\exp(-R_2^2/2). \end{align}\] We choose \(R_2 \asymp \sqrt{\log(1/\epsilon)}\) to make the integral less than \(\epsilon\). To handle the integral over \(B_1\), we apply Assumption 4 to obtain \[\begin{align} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(0, y)\right | \leq B\left |x - y\right | \leq \frac{B}{\alpha_t}\left |\alpha_t y - x\right | + \frac{B(1 - \alpha_t)}{\alpha_t}\left |x\right | \leq \frac{B\sigma_t}{\alpha_t}R_2 + \frac{B(1 - \alpha_t)}{\alpha_t}\left |x\right |. \end{align}\] Thus, Eq. 42 becomes \[\begin{align} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | \lesssim \frac{\sigma_t}{\alpha_t}R_2 + \frac{1-\alpha_t}{\alpha_t}\left |x\right | + (\alpha_t - 1)\phi^h_{\mathbf{z}}(t, x) + \epsilon. \end{align}\] Then, integration \(x\) over \(B_2 \cup B_3\) gives us \[\begin{align} \int_{(B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | {\rm d}x & \lesssim \frac{\sigma_t}{\alpha_t}R_1 + \frac{1-\alpha_t}{\alpha_t}\int_{B_2 \cup B_3}\left |x\right | {\rm d}x + \int_{B_2 \cup B_3} (\alpha_t - 1)\phi^h_{\mathbf{z}}(t, x) + \epsilon {\rm d}x \nonumber \\ & \lesssim \frac{\sigma_t}{\alpha_t}R_1 + \frac{1-\alpha_t}{\alpha_t}\int_{B_2 \cup B_3}\left |x\right | {\rm d}x + (\alpha_t - 1) + \epsilon\sqrt{\log(1/\epsilon)}. \label{eq:tv-2} \end{align}\tag{43}\] Consequently, the integration over \(B_1\) with respect to the density \(p^{[k:h)}(\cdot)\) leads to \[\begin{align} & \int_{B_4}\int_{(B_2 \cup B_3)} \left |\phi^h_{\mathbf{z}}(0, x) - \phi^h_{\mathbf{z}}(t, x)\right | p^{[k:h)}(\mathbf{z}) {\rm d}x {\rm d}\mathbf{z}\nonumber \\ & \lesssim \left( \frac{\sigma_t}{\alpha_t}R_1 + \frac{1-\alpha_t}{\alpha_t}\log(1/\epsilon) + (\alpha_t - 1) + \epsilon\sqrt{\log(1/\epsilon)} \right)(\sqrt{\log(1/\epsilon)})^{h-k},\label{eq:tv-3} \end{align}\tag{44}\] where we use the fact that \(\left |x\right | \lesssim \sqrt{\log(1/\epsilon)}\) over \(B_4 \times (B_2\cup B_3)\) and \(\left |B_2\cup B_3\right | \lesssim \sqrt{\log(1/\epsilon)}\). Combining 40 , 41 , and 44 , to obtain \[\begin{align} & \relax_{X_0^{[k:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[k:h)}], \mu_{t_0}^{h}[\mathbf{X}_0^{[k:h)}])] \\ & \lesssim \left( \frac{\sigma_t}{\alpha_t}\sqrt{\log(1/\epsilon)} + \frac{1-\alpha_t}{\alpha_t}\log(1/\epsilon) + (\alpha_t - 1) + \epsilon\sqrt{\log(1/\epsilon)} \right)(\sqrt{\log(1/\epsilon)})^{h-k} + \epsilon\sqrt{\log(1/\epsilon)} + \epsilon. \end{align}\] Note that \(\frac{\sigma_{t}}{\alpha_{t}} = \mathcal{O}(\sqrt{t})\), \(\frac{1-\alpha_t}{\alpha_t} = \mathcal{O}(t)\), \(\alpha_t - 1 = \mathcal{O}(t)\) as \(t \to 0\). We take \(\epsilon = t_0\) to conclude \[\begin{align} \relax_{\mathbf{X}_0^{[k:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[k:h)}], \mu_{t_0}^{h}[\mathbf{X}_0^{[k:h)}])] = \mathcal{O}\left( \sqrt{t_0}(\log(1/t_0))^{\frac{d_{\rm trunc}}{2}} \right). \end{align}\]

Finally, we apply the same analysis as in [81] but replacing [81] with Theorem 4. Thus, we obtain \[\begin{align} \relax_{\mathcal{D}} [\relax_{\mathbf{X}_0^{[k:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[k:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[k:h)}])]] \lesssim \sqrt{t_0}(\log(1/t_0))^{\frac{h-k+1}{2}} + \exp(-T) + \sqrt{T\relax_{\mathcal{D}}[\mathcal{R}(\hat{s})]}, \end{align}\] where we use the assumption that \(\relax_{\mathbf{X}_0^{[k:h)}}[{\rm KL}(\mu_0^h[\mathbf{X}_0^{[k:h)}]\| \mathcal{N}(0, 1))] < \infty\). Therefore, we complete the proof by applying 17 . ◻

4.4 Example: ReLU Network↩︎

In this subsection, we specialize the function class to ReLU neural networks by leveraging existing results in the literature. This leads to explicit score estimation and distribution estimation guarantees.

Specifically, let \(\mathcal{S}^h\) be the ReLU network class defined as in [81]. Consequently, Eq. (A.15) in [81] guarantees the existence of a ReLU network satisfying Assumption 5 and moreover, the logarithmic \(\delta\)-covering number is bounded with \[\begin{align} \log \mathcal{N}(\delta) \lesssim N^{d_{\rm trunc}}\log^9 N(\log^8 N + \log^2 N\log \log N + \log (\delta^{-1})). \label{eq:covering-relu} \end{align}\tag{45}\] With 45 , we obtain a concrete score estimation and distribution estimation results under the ReLU parameterization.

Corollary 2 (Score estimation with ReLU networks). Suppose Assumptions 3 and 4 hold. By taking the network size parameter \(N = n^{\frac{1}{d_{\rm trunc} + \beta}}\), early stopping threshold \(t_0 = N^{-C_\sigma} < 1\) and terminal horizon \(T = C_\alpha\log n\), then it holds that \[\begin{align} \relax_{\mathcal{D}}[\mathcal{R}(\hat{s})] = \mathcal{O}\Big(\frac{1}{t_0}n^{-\frac{\beta}{d_{\rm trunc} + \beta}} (\log n)^{\max\left\lbrace 2 + \beta/2, 18 \right\rbrace}\Big). \end{align}\]

Proof. We substitute 45 into Theorem 4 and choose \(N\), \(t_0\) and \(T\) as stated to obtain the result. ◻

Corollary 3 (Distribution estimation with ReLU networks). Let \(h \in [H]\) and \(k \in [h-1]\) be given indices. Assume that \(\relax_{\mathbf{X}_0^{[k:h)}}[{\rm KL}(\mu_0^h[\mathbf{X}_0^{[k:h)}]\| \mathcal{N}(0, 1))] < \infty\). Under Assumptions 2 and 3, by taking the early-stopping time \(t_0 = n^{-\frac{\beta}{2(d_{\rm trunc} + \beta)}}\) and terminal time \(T = \frac{\beta}{4(d_{\rm trunc} + \beta)}\log n\), it holds that, \[\begin{align} \relax_{\mathcal{D}} [\relax_{\mathbf{X}_0^{[1:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}])]] = \mathcal{O}\left( n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}}(\log n)^{\max\left\lbrace d_{\rm trunc}, 3 + \beta/2, 19 \right\rbrace/2} + u(d_{\rm trunc}) \right). \end{align}\]

Proof. We combine Theorem 5 and Corollary 1 to have \[\begin{align} & \relax_{\mathcal{D}} [\relax_{\mathbf{X}_0^{[1:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}])]] \\ & \lesssim \sqrt{t_0}(\log(1/t_0))^{\frac{d_{\rm trunc}}{2}} + \exp(-T) + \sqrt{\frac{T}{t_0}n^{-\frac{\beta}{d_{\rm trunc} + \beta}} (\log n)^{\max\left\lbrace 2 + \beta/2, 18 \right\rbrace}} + u(d_{\rm trunc}). \end{align}\] We take \(T = \frac{\beta}{4(d_{\rm trunc} + \beta)}\log n\), \(t_0 = n^{-\frac{\beta}{2(d_{\rm trunc} + \beta)}}\) to deduce that \[\begin{align} & \relax_{\mathcal{D}} [\relax_{\mathbf{X}_0^{[1:h)}}[{\rm TV}(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}])]] \\ & \lesssim n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}}(\log n)^{\frac{d_{\rm trunc}}{2}} + n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}} + \sqrt{\log n}\Big(\frac{1}{t_0}n^{-\frac{\beta}{d_{\rm trunc}+\beta}} (\log n)^{\max\left\lbrace 2 + \beta/2, 18 \right\rbrace}\Big)^{1/2} + u(d_{\rm trunc}) \\ & = \mathcal{O}\left( n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}}(\log n)^{\max\left\lbrace d_{\rm trunc}, 3 + \beta/2, 19 \right\rbrace/2} + u(d_{\rm trunc}) \right). \end{align}\] Therefore, we complete the proof. ◻

As an intermediate consequence, we establish a joint distribution estimation result following from Corollary 1 and the chain rule 16 .

Corollary 4. Suppose the assumptions in Corollary 1 hold for all \(h \in [H]\), it holds that \[\begin{align} \relax_{\mathcal{D}}[{\rm TV}(\mu_0^{[1:H]}, \hat{\mu}_{t_0}^{[1:H]})] = \mathcal{O}\left( \sum_{h = 1}^H n^{-\frac{\beta}{4(d_{\rm trunc} + \beta)}}(\log n)^{\max\left\lbrace d_{\rm trunc}, 3 + \beta/2, 19 \right\rbrace/2} + u(d_{\rm trunc}) \right), \end{align}\] where \(d_{\rm trunc} = h - k(h) + 1\) and \(k(h)\) is the optimal index chosen for each \(h\).

4.4.0.1 Selection of \(d_{\rm trunc}\).

We now discuss how to choose the truncation length \(d_{\rm trunc}\) to balance the two competing terms in Corollary 3: the statistical learning error, which grows with \(d_{\rm trunc}\), and the truncation error \(u(d_{\rm trunc})\), which decreases with \(d_{\rm trunc}\). We state the resulting rates for two canonical decay regimes.

Proposition 6 (Exponential decay). Suppose there are constants \(C_u, \kappa > 0\) such that \(u(d) \leq C_u(e^{-\kappa d} - e^{-\kappa h})\) for all \(1 \leq d \leq h\). If we take \[\begin{align} \label{eq:dtrunc-exp} d_{\rm trunc} &= \min\left\{ h, \left\lceil \Delta_n \right\rceil \right\}, \; \text{with} \;\Delta_n \mathrel{\vcenter{:}}= \frac{-\beta+\sqrt{\beta^2+(\beta/\kappa)\log n}}{2}, \end{align}\qquad{(2)}\] then under the assumptions of Corollary 3, it holds that \[\begin{align} \relax_{\mathcal{D}} \left[\relax_{\mathbf{X}_0^{[1:h)}}\left[{\rm TV}\left(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}]\right)\right]\right] = \begin{cases} \mathcal{O}\left( (\log n)^{\max\left\lbrace d_{\rm trunc}, 3 + \beta/2, 19 \right\rbrace/2}e^{-\kappa d_{\rm trunc}} \right), & h>\Delta_n \\ \mathcal{O}\left( n^{-\frac{\beta}{4(h + \beta)}}(\log n)^{\max\left\lbrace h, 3 + \beta/2, 19 \right\rbrace/2} \right), & h \leq \Delta_n. \end{cases} \end{align}\]

For a fixed horizon \(h\), Proposition 6 provides a truncation rule when \(u(\cdot)\) follows an exponential decay. For timestamps such that \(h>\Delta_n\), the rule selects a truncated history length of order \(\Theta(\sqrt{\log n})\). Since the information from a small dataset is limited, truncating to the most recent timestamps reduces the input dimension and thus improving the statistical learning error. As \(n\) grows, the rule increases \(d_{\rm trunc}\) until \(d_{\rm trunc}=h\), at which point the bound reduces to the statistical learning error with dimension \(h\).

Proof. By definition, \(\Delta_n\) satisfies the quadratic equation \[\begin{align} \frac{\beta\log n}{4(\Delta_n+\beta)} = \kappa \Delta_n. \label{eq:quadratic} \end{align}\tag{46}\] Taking exponentials on both sides gives us \[\begin{align} n^{-\frac{\beta}{4(\Delta_n+\beta)}} = e^{-\kappa \Delta_n}. \end{align}\] When \(d_{\rm trunc} < h\), we have \(d_{\rm trunc} = \left\lceil\Delta_n\right\rceil\) and thus \(\Delta_n \leq d_{\rm trunc} \leq \Delta_n + 1\). Consequently, the following bounds hold \[\begin{align} e^{-\kappa d_{\rm trunc}}\leq e^{-\kappa \Delta_n} = n^{-\frac{\beta}{4(\Delta_n+\beta)}} \leq n^{-\frac{\beta}{4(d_{\rm trunc}+\beta)}},\label{eq:exp-lower} \end{align}\tag{47}\] and 46 implies \[\begin{align} n^{-\frac{\beta}{4(d_{\rm trunc}+\beta)}} & \leq n^{-\frac{\beta}{4(\Delta_n + 1 +\beta)}} = \exp\left\lbrace -\frac{\beta \log n}{4(\Delta_n + 1 + \beta)} \right\rbrace = e^{-\kappa d_{\rm trunc}}\exp\left\lbrace \kappa d_{\rm trunc}-\kappa \Delta_n\frac{\Delta_n + \beta}{\Delta_n + 1 + \beta} \right\rbrace \nonumber \\ & \leq e^{-\kappa d_{\rm trunc}}\exp\left\lbrace \kappa (\Delta_n + 1)-\kappa \Delta_n\frac{\Delta_n + \beta}{\Delta_n + 1 + \beta} \right\rbrace = e^{-\kappa d_{\rm trunc}}\exp\left\lbrace \kappa\left( 1 + \frac{\Delta_n}{\Delta_n + 1 + \beta} \right) \right\rbrace \nonumber\\ & \leq e^{2\kappa}e^{-\kappa d_{\rm trunc}}. \label{eq:exp-upper} \end{align}\tag{48}\] Moreover, it holds that \[\begin{align} e^{-\kappa d_{\rm trunc}} \geq e^{-\kappa d_{\rm trunc}} - e^{-\kappa h} = e^{-\kappa d_{\rm runc}}(1 - e^{-\kappa(h-d_{\rm trunc})}) \geq (1 - e^{-\kappa})e^{-\kappa d_{\rm trunc}}. \label{eq:exp} \end{align}\tag{49}\] Therefore, we combine 47 , 48 and 49 to conclude that \[\begin{align} e^{-\kappa d_{\rm trunc}} - e^{-\kappa h} \leq n^{-\frac{\beta}{4(d_{\rm trunc}+\beta)}} \leq \frac{e^{2\kappa}}{1 - e^{-\kappa}} (e^{-\kappa d_{\rm trunc}} - e^{-\kappa h}). \end{align}\] This completes the proof together with \(d_{\rm trunc} = h\) case. ◻

Invoke the Lambert W function \(W:(0,\infty)\to(0,\infty)\) as the inverse of the mapping \(x\mapsto xe^x\), i.e., \(W(z)e^{W(z)}=z\) for all \(z>0\). We have the following result for the polynomial decay case.

Proposition 7 (Polynomial decay). Suppose there are constants \(C_u, \kappa > 0\) such that \(u(d) \leq C_u(d^{-\kappa} - h^{-\kappa})\) for all \(1 \leq d \leq h\). If we take \[\begin{align} \label{eq:dtrunc-poly} d_{\rm trunc} = \min\left\lbrace h, \left\lceil \Delta_n' \right\rceil \right\rbrace, \;\text{with}\; \Delta_n' \mathrel{\vcenter{:}}= \frac{{\beta \log n}/{4\kappa}}{W({\beta \log n}/{4\kappa})}, \end{align}\qquad{(3)}\] then under the assumptions of Corollary 3, it holds that \[\begin{align} \relax_{\mathcal{D}} \left[\relax_{\mathbf{X}_0^{[1:h)}}\left[{\rm TV}\left(\mu_0^{h}[\mathbf{X}_0^{[1:h)}], \hat{\mu}_{t_0}^{h}[\mathbf{X}_0^{[1:h)}]\right)\right]\right] = \begin{cases} \mathcal{O}\left( (\log n)^{\max\left\lbrace d_{\rm trunc}, 3 + \beta/2, 19 \right\rbrace/2}\left( d_{\rm trunc} \right)^{-\kappa} \right), & h>\Delta_n' \\ \mathcal{O}\left( n^{-\frac{\beta}{4(h + \beta)}}(\log n)^{\max\left\lbrace h, 3 + \beta/2, 19 \right\rbrace/2} \right), & h\leq \Delta_n'. \end{cases} \end{align}\]

For polynomial decay, it holds that \(W(\beta \log /4\kappa)=\Theta(\log\log n)\) in ?? . Consequently, the truncation rule selects a history length of order \(\Theta(\log n/\log\log n)\) before it reaches the horizon \(h\). Compared with the exponential-decay case, polynomial decay requires a larger truncation window, since the impact from the remote history decays more slowly.

Proof. Denote \(A_n \mathrel{\vcenter{:}}= \beta\log n /4\kappa\). Then we have \(\Delta_n'=A_n/W(A_n)\). The definition of the function \(W\) gives \[\begin{align} \label{eq:poly-W-balance} \Delta_n' = \frac{W(A_n)e^{W(A_n)}}{W(A_n)} = e^{W(A_n)}, \;\text{and} \;\Delta_n' \log \Delta_n' = e^{W(A_n)}W(A_n) = A_n. \end{align}\tag{50}\] Consider the case when \(d_{\rm trunc} < h\). Then we have \(d_{\rm trunc} = \lceil \Delta_n' \rceil \leq \Delta_n' + 1\). It follows from 50 that \[\begin{align} n^{-\frac{\beta}{4(d_{\rm trunc}+\beta)}} &= \exp\left\lbrace -\kappa\frac{A_n}{d_{\rm trunc}+\beta} \right\rbrace \leq \exp\left\lbrace -\kappa\frac{A_n}{\Delta_n'+1+\beta} \right\rbrace \nonumber\\ &= \exp\left\lbrace -\kappa\frac{\Delta_n'\log \Delta_n'}{\Delta_n'+1+\beta} \right\rbrace = (\Delta_n')^{-\kappa}\exp\left\lbrace \kappa\frac{(1+\beta)\log \Delta_n'}{\Delta_n'+1+\beta} \right\rbrace \nonumber\\ &\leq \exp\left\lbrace \frac{\kappa(1 + \beta)}{e} \right\rbrace (\Delta_n')^{-\kappa}, \label {eq:poly-stat-bound} \end{align}\] where we use the fact that \(\frac{\log x}{x + 1 + \beta} \leq \frac{\log x}{x} \leq 1/e\) for all \(x \geq 1\). Moreover, since \(d_{\rm trunc}=\left\lceil \Delta_n'\right\rceil\geq \Delta_n'\), we have \[\begin{align} d_{\rm trunc}^{-\kappa}-h^{-\kappa} \leq d_{\rm trunc}^{-\kappa}\leq (\Delta_n')^{-\kappa}. \label{eq:poly-trunc-bound} \end{align}\tag{51}\] This completes the proof together with \(d_{\rm trunc} = h\) case. ◻

5 Empirical Analysis↩︎

In this section, we numerically evaluate the performance of our AD-Seq framework proposed in Section 2 through extensive experiments. We first test the model on synthetic data generated by ARMA models and Gaussian processes. Specifically, we evaluate numerically its effectiveness in terms of recovering both the ground-truth distribution and the underlying temporal structures. Next, we apply our AD-Seq framework to real-world data and evaluate its performance in constructing mean-variance optimal portfolios. In implementation, we use a single transformer to parameterize the score functions across all timestamps. The causal mask ensures that the score at each timestamp depends only on the past history, while the objective 14 allows us to train the coordinatewise scores in parallel.

5.1 Synthetic Data↩︎

In this subsection, we evaluate the AD-Seq framework for learning ARMA models and Gaussian processes, two benchmark synthetic data generators.

5.1.0.1 ARMA models.

We first train a AD-Seq model on data generated by an ARMA(2, 2) with autoregressive coefficients \((\varphi_1, \varphi_2) = (0.9, -0.01)\) and moving average coefficients \((\theta_1, \theta_2) = (0.1, 0.1)\). The diffusion models are trained with a fixed sequence length \(H = 500\) and a variety of sample size \(n \in \left\lbrace 2^8, 2^9, 2^{10}, 2^{11} \right\rbrace\). For each trained model, we generate 1024 samples for evaluation.

We assess the accuracy of the learned temporal dependence through the lens of autocorrelation function (ACF). Let \(\lambda_i\) denote the ground-truth ACF of the ARMA model, and \(\left\lbrace \lambda_i^{\rm Diff} \right\rbrace_{i=1}^K\) and \(\left\lbrace \lambda_i^{\rm Emp} \right\rbrace_{i=1}^K\) denote the top-\(K\) lags of ACF calculated using diffusion-generated samples and empirical training samples, respectively. We introduce and then compare the following \(\ell_1\) relative errors in ACF between the diffusion model generated and the empirical training samples, \[\begin{align} {\rm Diff~RE}_1 \mathrel{\vcenter{:}}= \frac{\sum_{i=1}^K\left |\lambda_i^{\rm Diff} - \lambda_i\right |}{\sum_{i=1}^K\left | \lambda_i\right |}, \quad {\rm Emp~RE}_1 \mathrel{\vcenter{:}}= \frac{\sum_{i=1}^K\left |\lambda_i^{\rm Emp} - \lambda_i\right |}{\sum_{i=1}^K\left | \lambda_i\right |}. \end{align}\]

Table 1 reports the errors in estimating top-40 lags of an ARMA model, with each experiment repeated five times to ensure robustness. Figure 2 highlights the advantage of diffusion generated samples in estimating ACFs across different sample sizes. Notably, as shown in Figure 2 (b), the diffusion samples (green) accurately recover ground-truth ACFs (blue) for lags beyond 30 while empirical training samples are unable to recover ACFs. This capacity is particularly important for financial applications as high-quality historical data is often scarce, making precise ACF estimation a significant challenge.

Table 1: Relative error of the estimated top-40 ACFs of ARMA models for varying training dataset size (standard deviations in parentheses).
ARMA Models with Varying Train Dataset Size (\(H = 500\))
\(n\)	Diff RE\(_1\)	Emp RE\(_1\)
\(2^{8}=256\)	(\(\pm\) 0.0266)	(\(\pm\) 0.0309)
\(2^{9}=512\)	(\(\pm\) 0.0484)	(\(\pm\) 0.0123)
\(2^{10}=1024\)	(\(\pm\) 0.0373)	(\(\pm\) 0.0067)
\(2^{11}=2048\)	(\(\pm\) 0.1446)	(\(\pm\) 0.0045)

5.1.0.2 Gaussian processes.

Next, we evaluate the performance of AD-Seq model in modeling synthetic Gaussian process (GP) sequence of length \(H = 32\). Specifically, following the experimental setups in [82], we define the target distribution as \(\mathbf{X}_0 = (X_0^1, \dots, X_0^H)^\top \sim \mathcal{N}({\boldsymbol{\mu}}, {\boldsymbol{\Sigma}})\). The mean vector is set to zero, \({\boldsymbol{\mu}}= [\mu_1, \dots, \mu_H]^\top = \mathbf{0}\) and the covariance matrix is defined as \({\boldsymbol{\Sigma}}= \sigma^2{\boldsymbol{\Gamma}}\) where \(\sigma^2 > 0\) and \({\boldsymbol{\Gamma}}\in \mathbb{R}^{H \times H}\). The structure of the covariance is determined by the kernel function \(\gamma(\cdot)\) such that the (\(i, j\))-th entry of \({\boldsymbol{\Gamma}}\) is \(\Gamma_{ij} = \gamma(i, j)\). Similar to [82], an exponential kernel function is chosen, i.e., \(\gamma(i, j) = \exp(-\left |i - j\right |^\nu/\ell)\), with hyperparameters \(\ell = 4\) and \(\nu = 0.8\).

To evaluate the model, we use \(n = 2048\) samples from the ground-truth GP for training. After training, we generate another batch of \(2048\) samples to estimate the empirical covariance. As illustrated in Figure 3, the covariance structure of the diffusion-generated samples successfully recovers the characteristic decay pattern of the ground truth. This alignment demonstrates that our sequential diffusion models are capable of accurately capturing the joint distribution and the underlying temporal correlations of the sequential data.

Figure 3: Left: ground-truth covariance matrix used for training data generation; Middle: covariance of training dataset; Right: covariance of samples generated by our diffusion models. Sequential diffusion model learns spatial-temporal dependence.

5.2 Mean-Variance Portfolio Optimization↩︎

In this subsection, we study a mean-variance portfolio optimization problem which was studied by [7] and [95]. Specifically, an agent invests in the S&P 500 index and a risk-free asset with interest rate \(r = 2\%\) for half a year with \(H = 128\) trading days. The problem is to find a self-financing strategy \({\boldsymbol{\theta}}= \left\lbrace \theta^h \right\rbrace_{h=1}^H\) where \(\theta^h \in [0, 1]\) denotes the proportion of wealth invested in the risky asset at time \(h\) that minimizes the terminal wealth \(X^{H, \theta}\) for a given target wealth level \(z\), i.e., \[\begin{align} \min_{{\boldsymbol{\theta}}}\;\; {\rm Var}(X^{H, \theta}) \;\; {\rm s.t.}\;\; \relax[X^{H, \theta}] = z. \label{eq:mean-var} \end{align}\tag{52}\]

Table 2: Performance of different portfolio strategies with and without synthetic paths (standard deviations in parentheses).
Target Level	Policy	SDE Paths	Mean	Variance	Sharpe Ratio \(\uparrow\)
\(z = 1.10\)	Plug-in	Split	1.1759 (\(\pm\) 0.0025)	0.1045 (\(\pm\) 0.0024)	0.5442 (\(\pm\) 0.0081)
		Split + Synthetic	1.1138 (\(\pm\) 0.0037)	0.0399 (\(\pm\) 0.0025)	0.5702 (\(\pm\) 0.0083)
		Split + Ours	1.1685 (\(\pm\) 0.0024)	0.0952 (\(\pm\) 0.0022)	0.5462 (\(\pm\) 0.0081)
	RL	Split	1.0932 (\(\pm\) 0.0016)	0.0113 (\(\pm\) 0.0011)	0.8779 (\(\pm\) 0.0394)
		Bootstrap	1.0934 (\(\pm\) 0.0010)	0.0105 (\(\pm\) 0.0010)	0.9121 (\(\pm\) 0.0450)
		Split + Synthetic	1.0853 (\(\pm\) 0.0014)	0.0074 (\(\pm\) 0.0010)	0.9930 (\(\pm\) 0.0586)
		Split + Ours	1.0732 (\(\pm\) 0.0008)	0.0054 (\(\pm\) 0.0006)	0.9968 (\(\pm\) 0.0589)
\(z = 1.20\)	Plug-in	Split	1.3517 (\(\pm\) 0.0050)	0.4179 (\(\pm\) 0.0094)	0.5442 (\(\pm\) 0.0081)
		Split + Synthetic	1.2262 (\(\pm\) 0.0066)	0.1572 (\(\pm\) 0.0097)	0.5708 (\(\pm\) 0.0087)
		Split + Ours	1.3370 (\(\pm\) 0.0048)	0.3807 (\(\pm\) 0.0087)	0.5462 (\(\pm\) 0.0081)
	RL	Split	1.1861 (\(\pm\) 0.0032)	0.0460 (\(\pm\) 0.0045)	0.8695 (\(\pm\) 0.0376)
		Bootstrap	1.1865 (\(\pm\) 0.0016)	0.0425 (\(\pm\) 0.0043)	0.9068 (\(\pm\) 0.0435)
		Split + Synthetic	1.1706 (\(\pm\) 0.0028)	0.0302 (\(\pm\) 0.0040)	0.9866 (\(\pm\) 0.0574)
		Split + Ours	1.1464 (\(\pm\) 0.0016)	0.0218 (\(\pm\) 0.0025)	0.9965 (\(\pm\) 0.0591)

Following [7], we focus on two approaches:

Plug-in policy: A classical method that estimates coefficients of a GBM model and plug in the estimators into the expression of the optimal deterministic policy of 52 .
RL policy: A model-free reinforcement learning (RL) approach proposed in [6] which requires a market data simulator.

Furthermore, we consider four data sources for obtaining each policy:

Split: We split daily return of S&P 500 data from 1990-2009 into 40 half-year trajectories (called split paths). These paths are used to train our sequential diffusion models.
Split + Synthetic: In addition to 40 split paths, we augment 40 synthetic paths generated by approach in [7] and develop investment policies.
Split + Ours: Compared to Split + Synthetic, we replace the synthetic paths with 40 trajectories generated by our sequential diffusion model.
Bootstrap: For RL policies, we also construct investment polices using 40 bootstrap paths as in [6].

5.2.0.1 Main results.

We test the all the above policies on S&P 500 daily return data from 2010 to 2019 with bootstrap. As shown in Table 2, including diffusion generated samples do not improve plug-in policies. This is aligned with the observations in [7] as the data paths do not necessarily satisfy the GBM assumption. On the other hand, similar to Split + Synthetic, our diffusion generated samples also significantly improve the performance of RL policies. Notably, our approach achieves the highest Sharpe ratio and outperforms the method in [7]. This observation consolidates the belief that our diffusion models learn the underlying temporal structure of financial data in an adapted sequential manner. We also remark that the S&P 500 daily return has strongly Markovian behaviors with only short-range dependence on the past. This explains why our method performs comparably to [7] in this setting. However, for financial time series exhibiting longer historical dependence and more complex temporal structure, we expect our adapted sequential diffusion approach will demonstrate clearer advantages.

6 Omitted Proofs in Section 4 ↩︎

We devote this section to a few omitted proofs in Section 4.

Proposition 8 (Multivariate Gaussian). If \(\mathbf{X}_0\sim\mathcal{N}({\boldsymbol{\mu}},{\boldsymbol{\Sigma}})\) with positive definite covariance matrix \({\boldsymbol{\Sigma}}\), then Assumption 3 holds.

Proof. The marginal density of \(X_0^1\) is Gaussian and thus we focus on \(h > 1\) cases. Let \(h\in\{2,\ldots,H\}\) and \(k\in\{1,\ldots,h-1\}\) be fixed. Let \({\boldsymbol{\mu}}^{[k:h]} \in \mathbb{R}^{h-k+1}\) be the mean of \(\mathbf{X}_0^{[k:h]}\) and \({\boldsymbol{\Sigma}}^{[k:h]} \in \mathbb{R}^{(h-k+1) \times (h-k+1)}\) be the covariance matrix of \(\mathbf{X}_0^{[k:h]}\). We further write \[\begin{align} {\boldsymbol{\Sigma}}^{[k:h]} = \begin{pmatrix} {\boldsymbol{\Sigma}}_{11} & {\boldsymbol{\Sigma}}_{12} \\ {\boldsymbol{\Sigma}}_{21} & {\boldsymbol{\Sigma}}_{22} \end{pmatrix} \end{align}\] where \({\boldsymbol{\Sigma}}_{11} \in \mathbb{R}^{(h-k) \times (h-k)}\) is the covariance matrix of \(\mathbf{X}_0^{[k:h-1]}\), \({\boldsymbol{\Sigma}}_{22} \in \mathbb{R}\) is the variance of \(X_0^{h}\), \({\boldsymbol{\Sigma}}_{12} \in \mathbb{R}^{h-k}\) is the covariance between \(\mathbf{X}_0^{[k:h-1]}\) and \(X_0^{h}\) and \({\boldsymbol{\Sigma}}_{22} = {\boldsymbol{\Sigma}}_{12}^\top\). Since \({\boldsymbol{\Sigma}}\) is positive definite, the principal submatrix \({\boldsymbol{\Sigma}}^{[k:h]}\) is positive definite. It follows that \[\begin{align} X_0^h \mid \mathbf{X}_0^{[k:h-1]}=\mathbf{z}\sim \mathcal{N}\left( \mu^h + {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}(\mathbf{z}-{\boldsymbol{\mu}}^{[k:h-1]}), {\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12} \right). \end{align}\] Consequently, the conditional probability density satisfies \[\begin{align} \phi_{\mathbf{z}}^h(0,x) &= \frac{1}{(2\pi({\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12}))^{1/2}}\exp\left( -\frac{\left |x-\mu^h - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}(\mathbf{z}-{\boldsymbol{\mu}}^{[k:h-1]})\right |^2}{2{\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12}} \right) \\ &\leq \frac{\exp\left( \left |\mu^h - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\mu}}^{[k:h-1]}\right |^2/(2({\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12})) \right)}{(2\pi({\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12}))^{1/2}}\exp\left( -\frac{\left |x-{\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}\mathbf{z}\right |^2}{4({\boldsymbol{\Sigma}}_{22} - {\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{\Sigma}}_{12})} \right). \end{align}\] This completes the proof. ◻

Lemma 4. Under Assumption 3, the following inequalities hold: \[\begin{align} \phi^h_{\mathbf{z}}(t, x) \leq \frac{C_1}{\left( C_2\sigma_t^2 + \alpha_t^2 \right)^{1/2}}\exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2 \right), \label{eq:density-t-history-upper} \end{align}\qquad{(4)}\] and \[\begin{align} \phi^h_{\mathbf{z}}(t, x) \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\exp\left( -\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2+ 1}{\sigma_t^2} \right). \label{eq:density-t-history-lower} \end{align}\qquad{(5)}\]

Proof. Assumption 3 implies \[\begin{align} \phi^h_{\mathbf{z}}(t, x) & = \frac{1}{(2\pi \sigma_t^2)^{1/2}}\int \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \\ & \leq \frac{C_1}{(2\pi \sigma_t^2)^{1/2}}\int \exp\left( -C_2\left |y - \mathbf{v}^\top \mathbf{z}\right |^2/2 \right)\exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y. \end{align}\] Completing the square to have \[\begin{align} & -\frac{C_2\left |y - \mathbf{v}^\top \mathbf{z}\right |^2}{2} -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \\ & = y^2\left( -\frac{C_2}{2} - \frac{\alpha_t^2}{2\sigma_t^2} \right) + y\left( C_2 \mathbf{v}^\top \mathbf{z}+ \frac{\alpha_t x}{\sigma_t^2} \right) -\frac{C_2}{2}\mathbf{z}^\top v\mathbf{v}^\top \mathbf{z}- \frac{(x)^2}{2\sigma_t^2} \\ & = -\frac{C_2 \sigma_t^2 + \alpha_t^2}{2\sigma_t^2}\left( y - \frac{\sigma_t^2}{C_2\sigma_t^2 + \alpha_t^2}\left( C_2 \mathbf{v}^\top \mathbf{z}+ \frac{\alpha_t x}{\sigma_t^2} \right) \right)^2 -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2. \end{align}\] Consequently, we have \[\begin{align} \phi^h_{\mathbf{z}}(t, x) & \leq \frac{C_1}{(2\pi \sigma_t^2)^{1/2}}\exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2 \right) \\ & \times \int\exp\left( -\frac{C_2 \sigma_t^2 + \alpha_t^2}{2\sigma_t^2}\left( y - \frac{\sigma_t^2}{C_2\sigma_t^2 + \alpha_t^2}\left( C_2 \mathbf{v}^\top \mathbf{z}+ \frac{\alpha_t x}{\sigma_t^2} \right) \right)^2 \right) {\rm d}y \\ & = \frac{C_1}{\left( C_2\sigma_t^2 + \alpha_t^2 \right)^{1/2}}\exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2 \right), \end{align}\] which completes the upper bound.

For the lower bound, we have \[\begin{align} \phi^h_{\mathbf{z}}(t, x) & = \frac{1}{(2\pi \sigma_t^2)^{1/2}}\int \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \\ & \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \\ & \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t (y - \mathbf{v}^\top \mathbf{z}) - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2}{2\sigma_t^2} \right){\rm d}y \\ & \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\exp\left( -\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2}{\sigma_t^2} \right)\int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R}\phi^h_{\mathbf{z}}(0, y)\exp\left( -\frac{\alpha_t^2\left |y - \mathbf{v}^\top \mathbf{z}\right |^2}{\sigma_t^2} \right) {\rm d}y \\ & \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\exp\left( -\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2}{\sigma_t^2} \right)\exp\left( -\frac{\alpha_t^2 R^2}{\sigma_t^2} \right)\int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R}\phi^h_{\mathbf{z}}(0, y) {\rm d}y. \end{align}\] Moreover, Assumption 3 implies \[\begin{align} \int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R}\phi^h_{\mathbf{z}}(0, y) {\rm d}y & = 1 - \int_{\left |y - \mathbf{v}^\top \mathbf{z}\right | > R}\phi^h_{\mathbf{z}}(0, y) {\rm d}y \\ & \geq 1 - C_1 \int_{\left |z\right | > R}\exp(-C_2 z^2/2) {\rm d}z \\ & \geq 1 - C_2' R^{-1}\exp(-C_2R^2/2), \end{align}\] where \(C_2' = \frac{2\pi^{1/2}}{C_2\Gamma(3/2)}\) [Lemma 16][80]. Finally, we choose \(R = 1\) to conclude that \[\begin{align} \phi^h_{\mathbf{z}}(t, x) \geq \frac{1}{(2\pi \sigma_t^2)^{1/2}}\exp\left( -\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2+ 1}{\sigma_t^2} \right). \end{align}\] ◻

Lemma 5. Under Assumption 3, for any \(\nu \in \mathbb{Z}_+\) and \(\epsilon > 0\), there exist constants \(R_2\) and \(R_3\) satisfying \(R_2 \gtrsim \max(\sqrt{\log(1/\epsilon)}, \sqrt{\nu})\) and \(R_3 \gtrsim \sqrt{\log(1/\epsilon)}\), such that for any \(x \in \mathbb{R}\) and \(\mathbf{z}\in \mathbb{R}^{h-\ell}\) the following holds: \[\begin{align} \frac{1}{\sigma_t(2\pi)^{1/2}}\int_{\mathbb{R}\setminus \bar{B}} \left |\frac{\alpha_t y - x}{\sigma_t} \right |^\nu\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \leq \epsilon, \end{align}\] where \[\begin{align} \bar{B} = \left[ \frac{x - \sigma_t R_2}{\alpha_t}, \frac{x + \sigma_t R_2}{\alpha_t} \right] \bigcap \left[ \mathbf{v}^\top \mathbf{z}- R_3, \mathbf{v}^\top \mathbf{z}+ R_3 \right], \label{eq:truncation32region} \end{align}\qquad{(6)}\]

Proof. We consider two truncation regions: \[\begin{align} &B_{1} \mathrel{\vcenter{:}}= \left\lbrace y \in \mathbb{R}: \left |\frac{\alpha_t y - x}{\sigma_t}\right | \leq R_2 \right\rbrace = \left[ \frac{x - \sigma_t R_2}{\alpha_t}, \frac{x + \sigma_t R_2}{\alpha_t} \right],\\ & B_{2} \mathrel{\vcenter{:}}= \left\lbrace y \in \mathbb{R}: \left |y - \mathbf{v}^\top \mathbf{z}\right | \leq R_3 \right\rbrace = \left[ \mathbf{v}^\top \mathbf{z}- R_3, \mathbf{v}^\top \mathbf{z}+ R_3 \right]. \end{align}\] It follows \(\bar{B} = B_{1} \cap B_{2}\). We bound the integral with \[\begin{align} & \frac{1}{\sigma_t(2\pi)^{1/2}}\int_{\mathbb{R}\setminus \bar{B}} \left |\left( \frac{\alpha_t y - x}{\sigma_t} \right)^\nu \right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \nonumber \\ & \leq \underbrace{\frac{1}{\sigma_t(2\pi)^{1/2}}\int_{\mathbb{R}\setminus B_{1}} \left |\left( \frac{\alpha_t y - x}{\sigma_t} \right)^\nu \right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y}_{(A_1)} \nonumber \\ & \qquad + \underbrace{\frac{1}{\sigma_t(2\pi)^{1/2}}\int_{\mathbb{R}\setminus B_{2}} \left |\left( \frac{\alpha_t y - x}{\sigma_t} \right)^\nu \right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y}_{(A_2)}. \label{eq:split32int} \end{align}\tag{53}\] Over each region, we bound the integral in two ways. Note that for all \(R_2 > \sqrt{\nu}\), monotonicity implies \[\begin{align} \left |\left( \frac{\alpha_t y - x}{\sigma_t} \right)^\nu \right | \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right) \leq R_2^\nu \exp(-R_2^2/2). \end{align}\] Thus, we bound the term \((A_1)\) with \[\begin{align} (A_1) \leq \frac{R_2^\nu \exp(-R_2^2/2)}{\sigma_t(2\pi)^{1/2}}\int_{\mathbb{R}\setminus B_{1}}\phi^h_{\mathbf{z}}(0, y){\rm d}y \lesssim \frac{R_2^\nu \exp(-R_2^2/2)}{\sigma_t}. \label{eq:a1-1} \end{align}\tag{54}\] On the other hand, change of variable \(w = \alpha_t y - x/\sigma_t\) leads to \[\begin{align} (A_1) \leq \frac{C_1}{\alpha_t(2\pi)^{1/2}}\int_{\left |w\right | > R_2}w^\nu \exp(-w^2/2) {\rm d}w \lesssim \frac{R_2^{\nu - 1}\exp(-R_2^2 / 2)}{\alpha_t}. \label{eq:a1-2} \end{align}\tag{55}\] Combining 54 and 55 , we deduce that \[\begin{align} (A_1) \lesssim R_2^{\nu} \exp(-R_2^2/2). \label{eq:a1-3} \end{align}\tag{56}\] Similarly, we have the following two bounds for \((A_2)\): \[\begin{align} (A_2) & \leq \frac{C_1}{\sigma_t(2\pi)^{1/2}}\exp(-C_2 R_3^2/2)\int_{\mathbb{R}\setminus B_{2}}\left |\left( \frac{\alpha_t y - x}{\sigma_t} \right)^\nu \right | \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \nonumber \\ & \leq \frac{C_1}{\alpha_t(2\pi)^{1/2}}\exp(-C_2 R_3^2/2) \int_\mathbb{R}w^\nu \exp(-w^2/2) {\rm d}w \lesssim \frac{1}{\alpha_t}\exp(-C_2 R_3^2/2), \label{eq:a2-1} \end{align}\tag{57}\] and \[\begin{align} (A_2) \leq \frac{C_1}{\sigma_t (2\pi)^{1/2}}\left( \frac{v}{e} \right)^{v/2}\int_{\mathbb{R}\setminus B_{2}}\exp(-C_2\left |y - \mathbf{v}^\top \mathbf{z}\right |^2/2) {\rm d}y \lesssim \frac{1}{\sigma_t}\exp(-C_2 R_3^2/2). \label{eq:a2-2} \end{align}\tag{58}\] Combining 57 and 58 leads to \[\begin{align} (A_2) \lesssim \exp(-C_2 R_3^2/2). \label{eq:a2-3} \end{align}\tag{59}\] Consequently, adding up 56 and 59 gives us \[\begin{align} (A_1) + (A_2) \lesssim (R_2)^\nu \exp(-R_2^2/2) + \exp(-C_2 R_3^2/2) \lesssim \epsilon, \end{align}\] as long as \(R_2, R_3 \gtrsim \sqrt{\log(1/\epsilon)}\). ◻

Lemma 6. Under Assumption 3, the following inequality holds: \[\begin{align} \left |\partial_x \log\phi^h_{\mathbf{z}}(t, x)\right | \lesssim \frac{1}{\sigma_t^2}\left( \left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | + 1 \right). \end{align}\]

Proof. We write the score function as \[\begin{align} \partial_x \log\phi^h_{\mathbf{z}}(t, x) = \frac{\int \frac{\alpha_t y - x}{\sigma_t^2}\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y }{\int \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y}. \label{eq:score-ratio} \end{align}\tag{60}\] By taking \(\nu \in \left\lbrace 0, 1 \right\rbrace\) in Lemma 5, for any \(\epsilon > 0\) to be chosen later, there are sufficiently large constants \(R_1\) and \(R_4\) such that \[\begin{align} & \int_{\mathbb{R}\setminus B} \left |\frac{\alpha_t y - x}{\sigma_t^2}\right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \leq \epsilon, \tag{61} \\ & \int_{\mathbb{R}\setminus B} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \leq \epsilon. \tag{62} \end{align}\] where \(B\) is defined as in ?? . Lemma 4 implies \(\phi^h_{\mathbf{z}}(t, x) \geq 2\epsilon\) with \(\epsilon \asymp \frac{1}{\sigma_t}\exp\left( -\frac{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |^2 + 1}{\sigma_t^2} \right)\). It follows from 62 that \[\begin{align} \int_{B} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y \geq \epsilon. \label{eq:clip-int-inner-0} \end{align}\tag{63}\] Combining 60 –63 , we obtain \[\begin{align} \left |\partial_x \log\phi^h_{\mathbf{z}}(t, x)\right | & \leq \frac{\int_{\mathbb{R}} \left |\frac{\alpha_t y - x}{\sigma_t^2}\right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y }{\int_{B} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y} \\ & \leq \frac{\int_{B} \left |\frac{\alpha_t y - x}{\sigma_t^2}\right |\phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y }{\int_{B} \phi^h_{\mathbf{z}}(0, y) \exp\left( -\frac{\left |x - \alpha_t y\right |^2}{2\sigma_t^2} \right){\rm d}y} + \frac{\epsilon}{\epsilon} \leq \frac{R_2}{\sigma_t} + 1. \end{align}\] Let \(R_2 \asymp \sqrt{\log(1/\epsilon)}\). We conclude that \[\begin{align} \left |\partial_x \log\phi^h_{\mathbf{z}}(t, x)\right | \lesssim \frac{1}{\sigma_t^2}\left( \left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | + 1 \right). \end{align}\] ◻

Lemma 7. Suppose Assumption 3 holds. There is a constant \(C_3 > 0\), such that for any \(R_1>0\) and \(t>0\) we have \[\begin{align} \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |\partial_x \log \phi^h_{\mathbf{z}}(t, X_t^h)\right |^2 \right] \lesssim \frac{1}{\sigma_t^4} \exp(-C_6 R_1^2/2). \label{eq:E1-2} \end{align}\qquad{(7)}\]

Proof. Given \(\mathbf{z}\), Lemma 6 leads to \[\begin{align} \relax\left[ \left |\partial_x \log\phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\big\vert \mathbf{X}_0^{[k:h)} \right] \lesssim \frac{1}{\sigma_t^4}\left( 1 + \relax\left[ \left |X_t^h - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right |^2\big\vert \mathbf{X}_0^{[k:h)} \right] \right). \label{eq:score-tail-1} \end{align}\tag{64}\] With ?? and change of variable \(z = x - \alpha_t \mathbf{v}^\top \mathbf{z}\), we bound the conditional expectation with \[\begin{align} \relax\left[ \left |X_t^h - \alpha_t \mathbf{v}^\top \mathbf{X}_0^{[k:h)}\right |^2\big\vert \mathbf{X}_0^{[k:h)} \right] & \leq \frac{C_1}{\left( C_2\sigma_t^2 + \alpha_t^2 \right)^{1/2}} \int z^2 \exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}z^2 \right) {\rm d}z \\ & \leq \frac{2C_1}{C_2}(C_2\sigma_t^2 + \alpha_t^2)^{1/2} \lesssim 1. \end{align}\] Consequently, Eq. 64 becomes \[\begin{align} \relax\left[ \left |\partial_x \log\phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\big\vert \mathbf{X}_0^{[k:h)} \right] \lesssim \frac{1}{\sigma_t^4}. \end{align}\] Taking expectation over \(\mathbf{X}_0^{[k:h)}\), we conclude that \[\begin{align} & \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\left |\partial_x \log \phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2 \right] \\ & = \relax\left[ 1\left\lbrace \left \|\mathbf{X}_0^{[k:h)}\right \|_\infty > R_1 \right\rbrace\relax\left[ \left |\partial_x \log\phi^h_{\mathbf{X}_0^{[k:h)}}(t, X_t^h)\right |^2\big\vert \mathbf{X}_0^{[k:h)} \right] \right] \lesssim \frac{1}{\sigma_t^4} \exp(-C_6 R_1^2/2), \end{align}\] for some constant \(C_6 > 0\) as in Corollary 1. ◻

Lemma 8. Suppose Assumption 3 holds. For any \(R_2>0\), \(\mathbf{z}\in \mathbb{R}^{h-\ell}\), \(t>0\), we have \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_2} \phi^h_{\mathbf{z}}(t, x) {\rm d}x \lesssim (R_2)^{-1}\exp\left( -C_2'R_2^2/2 \right),\\ & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_2} \phi^h_{\mathbf{z}}(t, x)\left |\partial_x \log \phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x \lesssim \frac{1}{\sigma_t^4}R_2\exp\left( -C_2'R_2^2/2 \right), \end{align}\] where \(C_2' = \min_{t>0}\frac{C_2}{C_2\sigma_t^2 + \alpha_t^2} = \frac{C_2}{\max(C_2, 1)}\).

Proof. Lemma 4 implies \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_2} \phi^h_{\mathbf{z}}(t, x) {\rm d}x \\ & \leq \frac{C_1}{\left( C_2\sigma_t^2 + \alpha_t^2 \right)^{1/2}}\int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_2}\exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}(x - \alpha_t \mathbf{v}^\top \mathbf{z})^2 \right) {\rm d}x \\ & = \frac{C_1}{\left( C_2\sigma_t^2 + \alpha_t^2 \right)^{1/2}}\int_{\left |u\right |> R_2}\exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}u^2 \right) {\rm d}u \\ & \leq \frac{2\sqrt{\pi}C_1(C_2\sigma_t^2 + \alpha_t^2)^{1/2}}{C_2\Gamma(3/2)R_2}\exp\left( -\frac{C_2 R_2^2}{2(C_2\sigma_t^2 + \alpha_t^2)} \right) \lesssim (R_2)^{-1}\exp\left( -C_2'R_2^2/2 \right), \end{align}\] where we invoke [80] in the second inequality with \(d = 1\) and we set \(C_2' = \min_{t>0}\frac{C_2}{C_2\sigma_t^2 + \alpha_t^2}\). Similarly, Lemma 6 implies \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right |> R_2} \phi^h_{\mathbf{z}}(t, x)\left |\partial_x \log \phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x \\ & \lesssim \frac{C_1}{\sigma_t^4(C_2\sigma_t^2 + \alpha_t^2)^{1/2}}\int_{\left |z\right | > R_2}(1 + z^2) \exp\left( -\frac{C_2}{2(C_2\sigma_t^2 + \alpha_t^2)}z^2 \right){\rm d}z \\ & \leq \frac{C_1}{\sigma_t^4(C_2\sigma_t^2 + \alpha_t^2)^{1/2}}\frac{C_2\sigma_t^2 + \alpha_t^2}{C_2}(R_2^{-1} + R_2)\exp\left( -\frac{C_2 R_2^2}{2(C_2\sigma_t^2 + \alpha_t^2)} \right) \\ & \lesssim \frac{1}{\sigma_t^4}R_2 \exp\left( -\frac{C_2 R_2^2}{2(C_2\sigma_t^2 + \alpha_t^2)} \right) \lesssim\frac{1}{\sigma_t^4}R_2\exp\left( -C_2'R_2^2/2 \right). \end{align}\] ◻

Lemma 9. Suppose Assumption 3 holds. For any \(R_6, \epsilon_{\rm low} > 0\), \(\mathbf{z}\in \mathbb{R}^{h - k}\) and \(t>0\), we have \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6}1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | < \epsilon_{\rm low} \right\rbrace \phi^h_{\mathbf{z}}(t, x) {\rm d}x \lesssim R_6 \epsilon_{\rm low}, \\ & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6} 1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | < \epsilon_{\rm low} \right\rbrace \phi^h_{\mathbf{z}}(t, x)\left |\partial_x \log \phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x \lesssim \frac{\epsilon_{\rm low}}{\sigma_t^4}R_6^3. \end{align}\]

Proof. Direct calculation implies \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6}1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | < \epsilon_{\rm low} \right\rbrace \phi^h_{\mathbf{z}}(t, x) {\rm d}x \\ & \leq \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6} \epsilon_{\rm low} {\rm d}x = 2R_6\epsilon_{\rm low}. \end{align}\] Similarly, Lemma 6 implies \[\begin{align} & \int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6} 1\left\lbrace \left |\phi^h_{\mathbf{z}}(t, x)\right | < \epsilon_{\rm low} \right\rbrace \phi^h_{\mathbf{z}}(t, x)\left |\partial_x \log \phi^h_{\mathbf{z}}(t, x)\right |^2 {\rm d}x \\ & \lesssim \frac{\epsilon_{\rm low}}{\sigma_t^4}\int_{\left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | \leq R_6}\left( \left |x - \alpha_t \mathbf{v}^\top \mathbf{z}\right | + 1 \right)^2 {\rm d}x \lesssim \frac{\epsilon_{\rm low}}{\sigma_t^4}R_6^3. \end{align}\] ◻

References↩︎

[1]

Beatrice Acciaio, Stephan Eckstein, and Songyan Hou. Time-causal VAE: Robust financial time series generator. SIAM Journal on Financial Mathematics, 17 (1): 245–279, 2026.

[2]

Alexandre Alouadi, Baptiste Barreau, Laurent Carlier, and Huyên Pham. Robust time series generation via Schrödinger Bridge: a comprehensive evaluation. In Proceedings of the 6th ACM International Conference on AI in Finance, ICAIF 2025, Singapore, November 15–18, 2025, pages 906–914. ACM, 2025.

[3]

Rama Cont, Mihai Cucuringu, Renyuan Xu, and Chao Zhang. Tail-GAN: Learning to simulate tail risk scenarios. Management Science, 72 (4): 2917–2936, 2025.

[4]

Xun Yu Zhou and Duan Li. . Applied Mathematics & Optimization, 42 (1): 19–33, 2000.

[5]

Gerry Tsoukalas, Jiang Wang, and Kay Giesecke. Dynamic portfolio execution. Management Science, 65 (5): 2015–2040, 2019.

[6]

Yanwei Jia and Xun Yu Zhou. q-learning in continuous time. Journal of Machine Learning Research, 24 (161): 1–61, 2023.

[7]

Xuefeng Gao, Jiale Zha, and Xun Yu Zhou. Data-driven generative simulation of SDEs using diffusion models. In NeurIPS 2025 Workshop MLxOR: Mathematical Foundations and Operational Integration of Machine Learning for Uncertainty-Aware Decision-Making, 2025.

[8]

Andrea Coletta, Matteo Prata, Michele Conti, Emanuele Mercanti, Novella Bartolini, Aymeric Moulin, Svitlana Vyetrenko, and Tucker Balch. Towards realistic market simulations: a generative adversarial networks approach. In Proceedings of the Second ACM International Conference on AI in Finance, pages 1–9, 2021.

[9]

Yilie Huang, Yanwei Jia, and Xun Yu Zhou. Mean–variance portfolio selection by continuous-time reinforcement learning: Algorithms, regret analysis, and empirical study. arXiv preprint arXiv:2412.16175, 2024.

[10]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.

[11]

Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021.

[12]

Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems, 34: 1273–1286, 2021.

[13]

Qiyang Li, Aravind Jain, and Pieter Abbeel. . In James Cussens and Kun Zhang, editors, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pages 1210–1219. PMLR, 2022.

[14]

Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11523–11532, 2022.

[15]

Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. In Advances in Neural Information Processing Systems, volume 37, pages 56424–56445, 2024.

[16]

Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.

[17]

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.

[18]

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33: 6840–6851, 2020.

[19]

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in neural information processing systems, 34: 8780–8794, 2021.

[20]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.

[21]

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. OpenAI technical report, 2023.

[22]

Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 620 (7976): 1089–1100, 2023.

[23]

Emiel Hoogeboom, Vıctor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3D. In International conference on machine learning, pages 8867–8887. PMLR, 2022.

[24]

Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv:1411.1784, 2014.

[25]

Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional GANs. arXiv preprint arXiv:1706.02633, 2017.

[26]

Rao Fu, Jie Chen, Shutian Zeng, Yiping Zhuang, and Agus Sudjianto. Time series simulation by conditional generative adversarial net. International Journal of Neural Networks and Advanced Applications, 7: 25–38, 2020.

[27]

Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. Time-series generative adversarial networks. Advances in Neural Information Processing Systems, 32: 5508–5518, 2019.

[28]

Adriano Koshiyama, Nick Firoozye, and Philip Treleaven. Generative adversarial networks for financial trading strategies fine-tuning and combination. Quantitative Finance, 21 (5): 797–813, 2021.

[29]

Shujian Liao, Hao Ni, Marc Sabate-Vidales, Lukasz Szpruch, Magnus Wiese, and Baoren Xiao. Sig-Wasserstein GANs for conditional time series generation. Mathematical Finance, 34 (2): 622–670, 2024.

[30]

Junyi Li, Xintong Wang, Yaoyang Lin, Arunesh Sinha, and Michael Wellman. Generating realistic stock market order streams. AAAI Conference on Artificial Intelligence, 34 (01): 727–734, 2020.

[31]

Milena Vuletić, Felix Prenzel, and Mihai Cucuringu. . Quantitative Finance, 24 (2): 175–199, 2024.

[32]

Milena Vuletić and Rama Cont. : A generative model for arbitrage-free implied volatility surfaces. Applied Mathematical Finance, 31 (4): 203–238, 2025.

[33]

Magnus Wiese, Robert Knobloch, Ralf Korn, and Peter Kretschmer. Quant GANs: Deep generation of financial time series. Quantitative Finance, 20 (9): 1419–1440, 2020.

[34]

Hang Lou, Siran Li, and Hao Ni. : generating sequential data via the characteristic function of measures on the path space. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 39755–39781. Curran Associates, Inc., 2023.

[35]

Hans Buehler, Blanka Horvath, Terry Lyons, Imanol Perez Arribas, and Ben Wood. Generating financial markets with signatures. Available at SSRN 3657366, 2020.

[36]

Hans Bühler, Blanka Horvath, Terry Lyons, Imanol Perez Arribas, and Ben Wood. A data-driven market simulator for small data environments. In Dan Crisan, Ilya Chevyrev, Thomas Cass, James Foster, Christian Litterer, and Cristopher Salvi, editors, Stochastic Analysis and Applications 2025: In Honour of Terry Lyons, pages 273–310. Springer Nature Switzerland, 2026.

[37]

Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. : A variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:2111.08095, 2021.

[38]

Magnus Wiese, Ben Wood, Alexandre Pachoud, Ralf Korn, Hans Buehler, Phillip Murray, and Lianjun Bai. Multi-asset spot and option market simulation. arXiv preprint arXiv:2112.06823, 2021.

[39]

Borui Cai, Shuiqiao Yang, Longxiang Gao, and Yong Xiang. Hybrid variational autoencoder for time series forecasting. Knowledge-Based Systems, 281: 111079, 2023.

[40]

Yuansan Liu, Sudanthi Wijewickrema, Ang Li, Christofer Bester, Stephen O’Leary, and James Bailey. Time-transformer: Integrating local and global features for better time series generation. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 325–333. SIAM, 2024.

[41]

Hongbin Huang, Minghua Chen, and Xiao Qiao. Generative learning for financial time series with irregular and scale-invariant patterns. In The Twelfth International Conference on Learning Representations, 2024.

[42]

Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning (ICML), 2021.

[43]

Haksoo Lim, Minjung Kim, Sewon Park, and Noseong Park. Regular time-series generation using SGM. arXiv preprint arXiv:2301.08518, 2023.

[44]

Haksoo Lim, Jaehoon Lee, Sewon Park, Minjung Kim, and Noseong Park. : Regular and irregular time-series generation using score-based generative models. arXiv preprint arXiv:2511.21335, 2025.

[45]

Xinyu Yuan and Yan Qiao. Diffusion-TS: Interpretable diffusion for general time series generation. In The Twelfth International Conference on Learning Representations, 2024.

[46]

Ilan Naiman, Nimrod Berman, Itai Pemper, Idan Arbiv, Gal Fadlon, and Omri Azencot. Utilizing image transforms and diffusion models for generative modeling of short and long time series. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 121699–121730. Curran Associates, Inc., 2024.

[47]

Yiyuan Yang, Ming Jin, Haomin Wen, Chaoli Zhang, Yuxuan Liang, Lintao Ma, Yi Wang, Chenghao Liu, Bin Yang, Zenglin Xu, Shirui Pan, and Qingsong Wen. A survey on diffusion models for time series and spatio-temporal data. ACM Computing Surveys, 58 (8): 1–39, 2026.

[48]

Ahmad Aghapour, Erhan Bayraktar, and Fengyi Yuan. Solving dynamic portfolio selection problems via score-based diffusion models. arXiv preprint arXiv:2507.09916, 2025.

[49]

Gefei Wang, Yuling Jiao, Qian Xu, Yang Wang, and Can Yang. Deep generative learning via Schrödinger bridge. In International Conference on Machine Learning (ICML), 2021.

[50]

Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion Schrödinger bridge with applications to score-based generative modeling. In Advances in Neural Information Processing Systems, 2021.

[51]

Mohamed Hamdouche, Pierre Henry-Labordere, and Huyên Pham. Generative modeling for time series via Schrödinger bridge. arXiv preprint arXiv:2304.05093, 2023.

[52]

Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023.

[53]

Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Processing Systems, 35: 22870–22882, 2022.

[54]

Xingchao Liu, Lemeng Wu, Mao Ye, and qiang liu. Let us build bridges: Understanding and extending diffusion generative models. In NeurIPS 2022 Workshop on Score-Based Methods, 2022.

[55]

Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory, pages 946–985. PMLR, 2023.

[56]

Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning, pages 4735–4763. PMLR, 2023.

[57]

Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly \(d\)-linear convergence bounds for diffusion models via stochastic localization. In The Twelfth International Conference on Learning Representations, 2024.

[58]

Sitan Chen, Giannis Daras, and Alexandros G Dimakis. Restoration-degradation beyond linear diffusions: A non-asymptotic analysis for DDIM-type samplers. In International Conference on Machine Learning, pages 4462–4484. PMLR, 2023.

[59]

Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Representations, 2024.

[60]

Francesco Pedrotti, Jan Maas, and Marco Mondelli. Improved convergence of score-based diffusion models via prediction-correction. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.

[61]

Xiuyuan Cheng, Jianfeng Lu, Yixin Tan, and Yao Xie. Convergence of flow-based generative models via proximal gradient descent in Wasserstein space. IEEE Transactions on Information Theory, 70 (11): 8087–8106, 2024.

[62]

Daniel Zhengyu Huang, Jiaoyang Huang, and Zhengjiang Lin. Convergence analysis of probability flow ODE for score-based generative models. IEEE Transactions on Information Theory, 71 (6): 4581–4601, 2025.

[63]

Yuchen Liang, Peizhong Ju, Yingbin Liang, and Ness Shroff. Broadening target distributions for accelerated diffusion models via a novel analysis approach. In The Thirteenth International Conference on Learning Representations, 2025.

[64]

Gen Li and Yuling Yan. convergence theory for diffusion probabilistic models under minimal assumptions. Journal of Machine Learning Research, 26 (292): 1–55, 2025.

[65]

Wenpin Tang and Hanyang Zhao. Contractive diffusion probabilistic models. arXiv preprint arXiv:2401.13115, 2024.

[66]

Runjia Li, Qiwei Di, and Quanquan Gu. Unified convergence analysis for score-based diffusion models with deterministic samplers. In The Thirteenth International Conference on Learning Representations, 2025.

[67]

Yinuo Ren, Haoxuan Chen, Grant M. Rotskoff, and Lexing Ying. How discrete and continuous diffusion meet: Comprehensive analysis of discrete diffusion models via a stochastic integral framework. In The Thirteenth International Conference on Learning Representations, 2025.

[68]

Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow ODEs of diffusion models in Wasserstein distances. In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, volume 258 of Proceedings of Machine Learning Research, pages 1009–1017. PMLR, 2025.

[69]

Marta Gentiloni-Silveri and Antonio Ocello. Beyond log-concavity and score regularity: Improved convergence bounds for score-based generative models in W2-distance. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 55631–55656. PMLR, 2025.

[70]

Jerry Yao-Chieh Hu, Weimin Wu, Yi-Chen Lee, Yu-Chao Huang, Minshuo Chen, and Han Liu. On statistical rates of conditional diffusion transformers: Approximation, estimation and minimax optimality. In The Thirteenth International Conference on Learning Representations, 2025.

[71]

Jiadong Liang, Zhihan Huang, and Yuxin Chen. Low-dimensional adaptation of diffusion models: Convergence in total variation. In Proceedings of Thirty Eighth Conference on Learning Theory, volume 291 of Proceedings of Machine Learning Research, pages 3723–3729. PMLR, 2025.

[72]

Georg A Gottwald, Shuigen Liu, Youssef Marzouk, Sebastian Reich, and Xin T Tong. Localized diffusion models. arXiv preprint arXiv:2505.04417, 2025.

[73]

Song Mei and Yuchen Wu. Deep networks as denoising algorithms: Sample-efficient learning of diffusion models in high-dimensional graphical models. IEEE Transactions on Information Theory, 71 (4): 2930–2954, 2025.

[74]

Zixuan Zhang, Kaixuan Huang, Tuo Zhao, Mengdi Wang, and Minshuo Chen. Diffusion model for manifold data: Score decomposition, curvature, and statistical complexity. arXiv preprint arXiv:2603.20645, 2026.

[75]

Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26517–26582. PMLR, 2023.

[76]

Andre Wibisono, Yihong Wu, and Kaylee Yingxi Yang. Optimal score estimation via empirical Bayes smoothing. In Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pages 4958–4991. PMLR, 2024.

[77]

Kaihong Zhang, Heqi Yin, Feng Liang, and Jingbo Liu. Minimax optimality of score-based diffusion models: Beyond the density lower bound assumptions. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 60134–60178. PMLR, 2024.

[78]

Yinbin Han, Meisam Razaviyayn, and Renyuan Xu. Neural network-based score estimation in diffusion models: Optimization and generalization. In The Twelfth International Conference on Learning Representations, 2024.

[79]

Zehao Dou, Subhodh Kotekal, Zhehao Xu, and Harrison H Zhou. From optimal score matching to optimal sampling. arXiv preprint arXiv:2409.07032, 2024.

[80]

Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In International Conference on Machine Learning, pages 4672–4712. PMLR, 2023.

[81]

Hengyu Fu, Zhuoran Yang, Mengdi Wang, and Minshuo Chen. Unveil conditional diffusion models with classifier-free guidance: A sharp statistical theory. arXiv preprint arXiv:2403.11968, 2024.

[82]

Hengyu Fu, Zehao Dou, Jiawei Guo, Mengdi Wang, and Minshuo Chen. Diffusion transformer captures spatial-temporal dependencies: A theory for gaussian process data. In The Thirteenth International Conference on Learning Representations, 2025.

[83]

Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 (3): 313–326, 1982.

[84]

Ulrich G Haussmann and Etienne Pardoux. Time reversal of diffusions. The Annals of Probability, pages 1188–1205, 1986.

[85]

Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171, 2021.

[86]

Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23 (7): 1661–1674, 2011.

[87]

Andrew W Lo. Long-term memory in stock market prices. Econometrica: Journal of the Econometric Society, pages 1279–1313, 1991.

[88]

Rama Cont. Empirical properties of asset returns: stylized facts and statistical issues. Quantitative finance, 1 (2): 223, 2001.

[89]

Rong Tang, Lizhen Lin, and Yun Yang. Conditional diffusion models are minimax-optimal and manifold-adaptive for conditional distribution estimation. In The Thirteenth International Conference on Learning Representations, 2025.

[90]

Ian Martin. On the autocorrelation of the stock market. Journal of Financial Econometrics, 19 (1): 39–52, 2021.

[91]

Benoit Mandelbrot et al. The variation of certain speculative prices. Journal of business, 36 (4): 394, 1963.

[92]

Parameswaran Gopikrishnan, Vasiliki Plerou, Xavier Gabaix, and H Eugene Stanley. Statistical properties of share volume traded in financial markets. Physical review e, 62 (4): R4493, 2000.

[93]

Francis Bach. Breaking the curse of dimensionality with convex neural networks. The Journal of Machine Learning Research, 18 (1): 629–681, 2017.

[94]

Tokio Kajitsuka and Issei Sato. Are transformers with one layer self-attention using low-rank weight matrices universal approximators? In The Twelfth International Conference on Learning Representations, 2024.

[95]

Haoran Wang and Xun Yu Zhou. Continuous-time mean–variance portfolio selection: A reinforcement learning framework. Mathematical Finance, 30 (4): 1273–1308, 2020.

Department of Applied Mathematics and Statistics, Data Science and AI Institute, and Mathematical Institute for Data Science, Johns Hopkins University. Email: hycao@jhu.edu↩︎
Department of Industrial Engineering and Management Sciences, Northwestern University. Email: minshuo.chen@northwestern.edu↩︎
Department Management Science and Engineering, Stanford University. Email: yinbinha@stanford.edu↩︎
Department Management Science and Engineering, Stanford University. Email: renyuanxu@stanford.edu↩︎
The displayed score input uses the full generated history for notational simplicity. The same design permits replacing \(\mathbf{x}^{[1:h)}\) by a truncated recent-history window \(\mathbf{x}^{[k:h)}\) in \(\hat{s}^h\), as in the empirical loss below, without changing the sequential sampling nature.↩︎
This independence assumption simplifies the statement; our construction accommodates dependence across coordinates.↩︎

Diffusion Models for Adaptive Sequential Data Generation

Abstract

1 Introduction↩︎

1.0.0.1 Our work and contributions.

1.0.0.3 Notations.

2 Problem Setup↩︎

2.0.0.1 Preliminaries on diffusion models.

2.0.0.2 Adaptive Sequential Diffusion Model.

3 Algorithm design and adaptiveness property↩︎

3.0.0.1 Score matching.

3.0.0.2 Adaptiveness property.

4 Distribution Estimation Theory↩︎

4.1 Score Approximation↩︎

4.2 Score Estimation↩︎

4.2.0.1 Bounding term (II-A).

4.3 Distribution Estimation↩︎

4.4 Example: ReLU Network↩︎

4.4.0.1 Selection of \(d_{\rm trunc}\).

5 Empirical Analysis↩︎

5.1 Synthetic Data↩︎

5.1.0.1 ARMA models.

5.1.0.2 Gaussian processes.

5.2 Mean-Variance Portfolio Optimization↩︎

5.2.0.1 Main results.

6 Omitted Proofs in Section 4 ↩︎

References↩︎

Subjects

Updated on Academus

Diffusion Models for Adaptive Sequential Data Generation

Abstract

1 Introduction↩︎

1.0.0.1 Our work and contributions.

1.0.0.2 Related literature.

1.0.0.3 Notations.

2 Problem Setup↩︎

2.0.0.1 Preliminaries on diffusion models.

2.0.0.2 Adaptive Sequential Diffusion Model.

3 Algorithm design and adaptiveness property↩︎

3.0.0.1 Score matching.

3.0.0.2 Adaptiveness property.

4 Distribution Estimation Theory↩︎

4.1 Score Approximation↩︎

4.2 Score Estimation↩︎

4.2.0.1 Bounding term (II-A).

4.3 Distribution Estimation↩︎

4.4 Example: ReLU Network↩︎

4.4.0.1 Selection of \(d_{\rm trunc}\).

5 Empirical Analysis↩︎

5.1 Synthetic Data↩︎

5.1.0.1 ARMA models.

5.1.0.2 Gaussian processes.

5.2 Mean-Variance Portfolio Optimization↩︎

5.2.0.1 Main results.

6 Omitted Proofs in Section 4↩︎

References↩︎

Subjects

Updated on Academus

6 Omitted Proofs in Section 4 ↩︎