1 Introduction↩︎

Dynamical systems have emerged as a powerful framework for generative modeling [3], [4]. Modern approaches rely on SDEs [5]–[7], ODEs [8]–[10], and flow maps [11], achieving substantial progress in vision [12], [13], audio [14], and scientific domains [15], [16]. A wide range of inductive biases has been explored for diffusion models [17]–[22], but comparable biases have been less studied for flow-based models. One example is spectral bias [18].

Many natural datasets exhibit power spectra that approximately follow \(1/f^\gamma\), with \(0 < \gamma \le 2\) [23]–[29]. More generally, Figure 1 shows that even when spectra do not follow a power law, they typically decay with increasing frequency. In contrast, diffusion models and flow matching (FM) commonly inject white noise [30], which can be misaligned with the spectral structure of the data [18], [31]. To better match the data’s spectral content, we introduce Low-Pass Flow Matching (LP-FM), a variant of FM based on an operator-modulated interpolant. LP-FM induces a time-dependent spectral bias; for small \(t\) the path preserves the source spectrum, while as \(t \to 1\) it transitions to a frequency-decaying bias. Empirically, we show that the proposed bias preserves or improves sample quality, can accelerate training, and reduces sampling cost across datasets, including a scientific imaging benchmark.

2 Related Work↩︎

Recent works analyze generative dynamics through a spectral lens. [32] frame generative processes as physical evolutions satisfying precise spectral properties, while [33], [34] recast the noising process as heat-equation-based blurring. A growing line of research focuses on spectral data properties [30], [35] to optimize noise schedules [36] or to favor specific Fourier components by introducing autocorrelated noise [18], [31], [37]. Others have introduced generative models that operate directly in general functional spaces [38]–[40]. Complementary to this, architectural interventions [41]–[44] counteract the inherent spectral bias of the underlying network to better synthesize high- and low-frequency components during generation.

We defer a detailed comparison with [18] and [31] to Appendix 7.1.

3 Background↩︎

3.1 Flow Matching↩︎

Let \(p\) and \(q\) be probability distributions on \(\mathbb{R}^d\), where \(p\) is a simple source distribution and \(q\) is the (unknown) data distribution. FM learns a time-varying vector field \(u : [0,1] \times \mathbb{R}^d \to \mathbb{R}^d\) that defines the ODE \[\frac{d}{dt}\phi_t(x) = u_t\left(\phi_t(x)\right), \qquad \phi_0(x)=x. \label{eq:flowode}\tag{1}\] Setting \(p_0 = p\), the ODE induces the probability path \(p_t \mathrel{\vcenter{:}}= [\phi_t]_{\#}(p_0)\). The goal is to learn \(u_t\) such that the terminal distribution \(p_1\) matches the data distribution \(q\).

In general, \(u_t(x)\) is intractable. Instead, [9], [45] show it is possible to learn \(u_t(x)\) by introducing a conditioning variable \(z\) and minimizing the conditional flow matching (CFM) objective \[\mathcal{L}_{\text{CFM}}(\theta) \mathrel{\vcenter{:}}= \mathbb{E}_{t, \rho(z),p_t(x|z)} \left\|v_\theta(t;x) - u_t(x|z)\right\|^2,\] where \(\rho\) is the distribution over \(z\), and \(p_t(x|z)\), \(u_t(x|z)\) denote the conditional probability path and its associated vector field. Setting \(z=x_1\), with \(\rho(z)=q(x_1)\), recovers FM [9], while setting \(z=(x_0, x_1)\), with \(\rho(z) = p(x_0)q(x_1)\), yields CFM [45].

3.2 Power Spectra of Diffusion Models↩︎

Consider a random field \(x\in\mathbb{R}^{\mathcal{G}}\) on a grid \(\mathcal{G}\) with \(|\mathcal{G}|=d\), e.g., images with \(d=H\times W\). Let \(\hat{x}({\boldsymbol{k}})=\mathcal{F}\{x\}({\boldsymbol{k}})\) denote its discrete Fourier transform (DFT). The power spectral density (PSD) is \[S_x({\boldsymbol{k}}) \mathrel{\vcenter{:}}= \mathbb{E}\!\left[|\hat{x}({\boldsymbol{k}})|^2\right].\] Assuming \(x_0\) and \(x_1\) independent, a standard interpolant for diffusion models and FM is \[x_t = \alpha_t\,x_1 + \sigma_t\,x_0, \qquad x_0 \sim \mathcal{N}(0,I_d),\;\; x_1\sim q. \label{eq:standard95interpolant}\tag{2}\] Consequently, the interpolant’s PSD is [18], [30] \[S_{x_t}({\boldsymbol{k}}) = \alpha_t^2 S_{x_1}({\boldsymbol{k}}) + \sigma_t^2,\] i.e., the injected noise is additive and constant across frequencies. In the following sections we assume that \(x_0\) and \(x_1\) are spectrally uncorrelated, i.e. \(\mathbb{E}[\hat{x}_0({\boldsymbol{k}})\,\overline{\hat{x}_1({\boldsymbol{k}})}]=0\) for all \({\boldsymbol{k}}\), where \(\overline{(\cdot)}\) denotes complex conjugation.

4 Frequency Biased Interpolant for Flow Matching↩︎

We generalize Equation 2 by replacing the scalar noise scaling (\(\sigma_t\)) with a time-dependent linear shift-invariant (LSI) operator. Let \(\mathcal{L}_t\) be a bounded LSI on \(\ell^2(\mathcal{G})\). We define the interpolant \[x_t \mathrel{\vcenter{:}}= t x_1 + \mathcal{L}_t x_0. \label{eq:int}\tag{3}\] We design \(\mathcal{L}_t = (1-t)\mathcal{L}_t'\) such that, informally, \(\mathcal{L}_t' \to \mathcal{I}\) as \(t\to 0\) (where \(\mathcal{I}\) is the identity operator), preserving the source bias early in the path, and \(\mathcal{L}_t'\to \mathcal{L}\) as \(t\to 1\) for a target bias \(\mathcal{L}\).

Proofs of this proposition and the subsequent one are given in Appendix 8. Following [9], we recover the conditional vector field \(u_t(x|x_1)\) from the interpolant in Equation 3 .

Finally, following [45], we can condition on \(z= (x_0, x_1)\) and obtain the CFM form of the conditional vector field (which we always use in our experiments): \[u_t(x|z) = x_1 + \dot{\mathcal{L}}_t x_0 = x_1 +\mathcal{F}^{-1}\left\{\left(\partial_t L_t({\boldsymbol{k}})\right)\hat{x}_0({\boldsymbol{k}})\right\}.\]

Rotation-Invariant Parameterization. For image-like data we use isotropic filters, i.e.\(L_t({\boldsymbol{k}})=L_t(k)\), with radial frequency \(k \mathrel{\vcenter{:}}= \sqrt{\|{\boldsymbol{k}}\|^2}\). We consider two low-pass parameterizations: rational low-pass (RLP), \[|L_t({\boldsymbol{k}})|^2 = \frac{(1-t)^2}{\left(1 + \frac{k^2}{k_0^2}\right)^{\gamma_1t}},\] and Gaussian low-pass (GLP), \[|L_t({\boldsymbol{k}})|^2 = (1-t)^2 e^{-\gamma_1t\|{\boldsymbol{k}}\|^2},\] where \(k_0>0\) sets the cutoff scale and \(\gamma_1>0\) controls the strength of the spectral bias. When used within CFM, we refer to the resulting methods as RLP-CFM and GLP-CFM, respectively.

5 Experiments↩︎

We evaluate RLP-CFM and GLP-CFM against FM, CFM, and variance-preserving flow matching (VP-FM; [9]) for unconditional image generation. We consider three datasets: Dead Leaves [2], CIFAR-10, and Galaxy10 DECaLS [46]. Dead Leaves provides a controlled setting with an approximately \(1/f^\gamma\) spectrum (cf.Figure 1 and the dashed reference in Figure 2), CIFAR-10 is a standard natural-image benchmark, and Galaxy10 represents a real-world scientific imaging task. We report the Fréchet Inception Distance (FID) [47] and the number of function evaluations (NFE) required for sampling. FID is computed with Clean-Fid [48] using legacy_tensorflow settings and training-set Inception statistics. For sampling, we compare several ODE solvers: Euler and adaptive Dopri5 [49] and Tsit5 [50]. Our implementation follows the training protocol of [45]. We use the U-Net backbone of [51] with exponential moving average (EMA), dropout, and gradient-norm clipping, and optimize with either Adam [52] or AdamW [53]. Full hyperparameters are reported in Appendix 9. Qualitative comparisons between real and generated samples are deferred to Appendix 10.

Dead Leaves. We generate \(50\)k \(1{\times}64{\times}64\) images by placing randomly oriented squares uniformly at random, following [2] to obtain an approximately \(1/f^\gamma\) spectrum. We compare CFM and RLP-CFM; results are in Figure 2. RLP-CFM achieves substantially lower FID and reaches its best performance earlier: it achieves roughly half the FID of CFM and converges in \(\sim\)​1350 epochs, while CFM continues to improve up to the end of training. The PSD evolution (right panel) suggests that RLP-CFM recovers higher-frequency content earlier along the path and matches the target spectral shape more closely around \(t\approx 0.75\).

CIFAR-10 & Galaxy10 DECaLS. Table 1 summarizes the results (Galaxy10: \(16\)k RGB images at \(256{\times}256\)). With Euler solver, baselines are strongest, suggesting that a spectral bias may be less beneficial under a fixed-step discretization¹. Under adaptive solvers, however, RLP-CFM becomes competitive: on CIFAR-10 it preserves sample quality while reducing NFE, and on Galaxy10 it improves FID while substantially reducing NFE (up to \(1.65\times\) vs.CFM and \(2.16\times\) vs.VP-FM). In contrast, GLP-CFM underperforms RLP-CFM on CIFAR-10 and we were unable to obtain stable training on Galaxy10. Overall, these results indicate that a spectral bias can be most useful when paired with adaptive integration.

6.5pt

6 Conclusion↩︎

We introduced Low-Pass Flow Matching, a simple operator-modulated interpolant that induces a time-varying spectral bias in Flow Matching. Experiments suggest the bias is most useful with adaptive ODE solvers, improving or preserving sample quality while reducing sampling cost. Future work will evaluate RLP-CFM on additional scientific datasets.

7 Extendend Related Work↩︎

Inductive biases in generative modeling. Generative models often rely on inductive biases to tailor their sampling dynamics to specific data structures. Notable examples include physics-inspired biases, such as Lévy processes for heavy-tailed data [19], [22] and Poissonian flows [20], [21], that have been found to be a valid alternative to diffusion models. Another deeply explored area is geometric bias; [54], [55] developed equivariant continuous normalizing flows, while [15], [17] demonstrate the necessity of symmetry-preserving models for biological data. Beyond symmetry, a broad body of work covers alternative inductive biases in generative dynamics, including methods that design probability path or flows with low-curvature trajectories [9], [10], [45].

7.1 Comparison with EqualSNR and Cloud Diffusion↩︎

We compare LP-FM with EqualSNR and Cloud Diffusion, two methods that modify diffusion models by using autocorrelated noise.

Cloud Diffusion [31] considers noise spectra of the form \(S_{\text{noise}}(k)\propto 1/k\), which diverge as \(k\to 0\) and may therefore lead to numerical instability near the DC component; since no experiments are reported, we limit our discussion to methodological considerations.

EqualSNR [18] instead sets \(S_{\text{noise}}({\boldsymbol{k}})\propto S_1({\boldsymbol{k}})\) to align the forward noising spectrum with the data spectrum, motivated by Gaussian assumptions in the DDPM reverse process [5]. In contrast, FM does not require the same reverse-time Gaussianity; consequently, it is a priori unclear whether this rationale transfers. This motivates studying spectral bias in FM in its own right. Although EqualSNR is motivated by DDPM assumptions, it is evaluated with DDIM sampling [6], which is closely related to FM with Euler solver [56]. Empirically, we do not observe benefits from introducing a spectral bias under Euler (except on the toy Dead Leaves), whereas gains emerge when FM is paired with adaptive ODE solvers. Therefore, the benefits we report arise under sampling conditions substantially different from DDIM. There are also methodological differences: EqualSNR operates in Fourier space, whereas we work in pixel space.

Finally, our approach induces a time-varying bias: \(\mathcal{L}_t\) is designed to preserve the source spectral structure early along the path while progressively imposing a low-pass bias near the data distribution. In contrast, EqualSNR and Cloud Diffusion prescribe a noise spectrum intended to match the data, but do not explicitly preserve the spectral bias of the source distribution. Overall, our results highlight that (i) the solver choice can be critical for realizing the benefits of spectral bias in FM, and (ii) time-varying operator modulation provides a way to incorporate spectral structure while remaining compatible with nontrivial source distributions.

8 Proofs↩︎

This appendix collects proofs of the propositions in Section 4. For convenience, we restate each proposition before its proof. We also recall a few operator-theoretic and Fourier-analytic facts used throughout.

8.1 Preliminaries: LSI operators on a finite grid↩︎

Let \(\mathcal{G}\) be a finite periodic grid, and equip \(\mathbb{C}^{\mathcal{G}}\) with the standard \(\ell^2\) inner product \(\langle x,y\rangle \mathrel{\vcenter{:}}= \sum_{g\in\mathcal{G}} x(g)\,\overline{y(g)}\) and norm \(\|x\|_2^2=\langle x,x\rangle\). We denote by \(\hat{x}({\boldsymbol{k}})=\mathcal{F}\{x\}({\boldsymbol{k}})\) the DFT of \(x\). We use the convention that the DFT is unitary so that Parseval’s identity holds: \[\|x\|_2^2 = \sum_{{\boldsymbol{k}}} |\hat{x}({\boldsymbol{k}})|^2.\]

8.1.0.1 Bounded linear operators.

A linear map \(\mathcal{L}:\ell^2(\mathcal{G})\to \ell^2(\mathcal{G})\) is bounded if \[\|\mathcal{L}\|_{\mathrm{op}} \mathrel{\vcenter{:}}= \sup_{\|x\|_2=1}\|\mathcal{L} x\|_2 < \infty.\] On a finite-dimensional space every linear operator is bounded.

8.1.0.2 Shift-invariant (LSI) operators and frequency response.

An operator \(\mathcal{L}\) is shift-invariant if it commutes with grid shifts. Equivalently, \(\mathcal{L}\) is a circular convolution with some kernel \(h \in\mathbb{R}^{\mathcal{G}}\): \[(\mathcal{L} x)(g) = (h * x)(g).\] Such operators diagonalize in the Fourier domain: there exists a complex-valued function \(L({\boldsymbol{k}})\) (the frequency response) such that for all \({\boldsymbol{k}}\), \[\label{eq:operator95frequency} \mathcal{F}\left\{\mathcal{L} x\right\}({\boldsymbol{k}}) = L({\boldsymbol{k}})\,\hat{x}({\boldsymbol{k}})\tag{4}\] For a time-indexed family \(\{\mathcal{L}_t\}_{t\in[0,1]}\) we write \(L_t({\boldsymbol{k}})\) for its response.

8.1.0.3 Invertibility.

An LSI operator \(\mathcal{L}\) is invertible on \(\ell^2(\mathcal{G})\) if and only if \(L({\boldsymbol{k}})\neq 0\) for all \({\boldsymbol{k}}\). In that case the inverse is also LSI and satisfies \(\mathcal{F}\left\{\mathcal{L}^{-1} x\right\}({\boldsymbol{k}}) = \frac{1}{L({\boldsymbol{k}})}\,\hat{x}({\boldsymbol{k}})\).

8.1.0.4 Spectral (un)correlatedness.

For random fields \(x_0,x_1\in\mathbb{R}^{\mathcal{G}}\) we say they are spectrally uncorrelated if \[\mathbb{E}\!\left[\hat{x}_0({\boldsymbol{k}})\,\overline{\hat{x}_1({\boldsymbol{k}})}\right]=0 \qquad \text{for all }{\boldsymbol{k}}.\]

8.2 Proof of Proposition 1↩︎

8.3 Proof of Proposition 2↩︎

9 Experimental Settings↩︎

4pt

4pt

10 Generated images↩︎

References↩︎