1 Introduction↩︎

Many, if not most, natural and engineered systems, from chemical and molecular processes to climate, ecosystems, brain activity, or stock markets, are naturally described as dynamical systems (DS) [3]–[9]. Reconstructing these DS (DSR) in a data-driven way from just time series observations is of fundamental interest to all sciences, which seek tractable models that enable mechanistic insight into the underlying dynamical processes. DSR, hence, aims for generative surrogate models of the underlying dynamics which not only forecast a few steps ahead, but—importantly—reproduce long-term statistical properties of the DS in question, such as its attractor geometry (ergodic distribution) and power spectrum [3], [10]–[16]. While traditionally DSR involved custom-training ML/AI models on each particular dataset [12], [13], [17]–[19], inspired by the success of in-context learning in large language and time-series foundation models (FMs) [20]–[22], a new line of research explored whether zero-shot generalization to previously unseen DS is also feasible [1], [23].

However, as for most if not all FMs, it is currently utterly unclear how SOTA DSR FMs like DynaMix [1] achieve their zero-shot out-of-domain generalization, and what the essential architectural requirements for this are. Insight into how AI models solve their problems is of general interest, but is particularly important in scientific and medical domains where the focus is on mechanistic understanding [3], [24]. Here we tackle this question in the context of DSR by reverse-engineering the pretrained DynaMix model. We produce a series of systematic ablations and simplifications that largely retain, or even improve on, the model’s zero-shot DSR performance. We ultimately arrive at an algebraically minimal form consisting just of a linear combination of the model’s previous state, the nearest neighbor from the context signal, and its immediate temporal successor. A natural symmetry requirement on the context points further condenses the model into a two-parameter form. We show that this recursive map, DynaBase, can generate point, cyclic, and chaotic attractors, and allows for closed-form one-step least-squares fitting to time series data. Theoretical and empirical analysis of DynaBase trained directly for DSR reveals that optimal zero-shot solutions essentially lie along a 1-parameter curve with truly chaotic dynamics at one end and ‘context parroting’ of a time series snippet at the other, a strategy recently observed in transformer-based time series FMs like Chronos [2], [25]. Our observations are in line with a recent literature that demonstrates that tiny ML/AI models, many orders of magnitude below the parameter load of modern FMs, often achieve comparable, or even widely superior performance, on a range of tasks from cognitive and behavioral modeling [26] to challenging reasoning problems [27], [28].

In a nutshell, our main results are:

• Minimal zero-shot DSR model: Through iterative simplification of DynaMix [1] we create a minimal 2-parameter map, DynaBase, which exhibits competitive zero-shot DSR performance at essentially zero training costs.

• Mechanistic insight: The model’s two coefficients \((\alpha,\beta)\) parametrize a continuous spectrum of dynamical behaviors, with optimal DSR solutions essentially driven to a 1-parameter subspace. We show that at one end of this subspace DynaBase produces strictly chaotic dynamics, mimicking chaotic DS provided in-context, while at the other it yields the exact context-parroting algorithm of [2].

• Impact of training strategy: We further demonstrate that training using either a standard ahead-prediction MSE loss vs. a specific DSR loss yields fundamentally different solutions: Training for ahead-prediction often collapses to simple context parroting and hence does not recover the true dynamics, especially for chaotic systems, which is only achieved by DSR-specific training.

2 Related work↩︎

2.0.0.1 Dynamical systems reconstruction (DSR)

DSR aims to learn generative models from time series data that capture the underlying system’s dynamical long-term behavior, i.e. its long-term or “climate” statistics and state space geometry [3], [11], [13], [14], [18], [29]. Custom-trained models for this purpose build on library-based methods like SINDy [30], [31], reservoir computing [13], [19], [32]–[34], neural ODEs [35]–[37], Koopman operators [38]–[43], or on various types of RNNs [10], [17], [18], [44]–[49]. More important than the particular architecture appears to be the training strategy: sparse or generalized teacher forcing [18], [29] are control-theoretic training techniques that enable long trajectory roll-outs even for chaotic DS, while specific regularization methods based on a system’s invariant measures have been designed to enforce agreement in long-term statistics [13], [50], [51]. All of these methods, however, require purpose-training on each new system, and out-of-domain generalization remains a key challenge [12].

2.0.0.2 Zero-shot DSR and time-series foundation models (FMs)

Multi-environment [52], [53], hyper-network-guided [54], or hierarchical [55] approaches have been used to train DSR models on many DS simultaneously, but all of these still require fine-tuning on new systems or parameter configurations not seen in training. More recently, inspired by the success of in-context inference in LLMs [56]–[58], a line of general-purpose time series (TS) FMs pretrained on large corpora of real and synthetic data has emerged, which forecast TS from a short observed context snippet zero-shot. Most of these, like Chronos [20], [21], TimesFM [59], or Panda [23], are based on transformers, with a few exceptions that engage RNNs like the xLSTM [60] or Mamba [61] as their core. All of these, however, lack the ability to reproduce a system’s long-term behavior as required in DSR [1], [12], [16]. They mostly either converge to just fixed points in the limit, or cyclically repeat the context [2], [25], and thus fail to capture DS with inherently chaotic dynamics [1]. DynaMix [1] closes this gap by pretraining, based on DSR-specific training techniques [29], a mixture of AL-RNN [10] experts, weighted by a context-dependent gating network. It delivers competitive short-term forecasts and SOTA long-term performance, while being relatively lightweight with \(\sim 10^4\) parameters.

2.0.0.3 Mechanisms of in-context learning

What is at the root of in-context learning has been intensely studied in the past few years [56]–[58], and hypotheses range from transformers implementing gradient-descent in-context [62]–[65] to mechanisms that more or less boil down to a simple remixing of their memory contents [66]–[68]. For DSR FMs, the analogous problem remains largely open but—at the same time—appears more tractable due to the more constrained and mathematically better defined [12] nature of the question. Recent studies using Chronos suggested that TS FMs may long-term forecast DS by the simple mechanism of context parroting [25], a well known pattern in LLMs [69]–[71]. Based on this phenomenon, [2] explicitly design a ‘context parroting algorithm’ which takes the last \(D\) values of the context as a delay-embedded query, retrieves the best-matching motif elsewhere in the context, and copies the subsequent fragment as forecast. They show that this strategy works surprisingly well and actually outperforms most TS FMs. But context parroting, as exhibited by Chronos or conceptualized in this form, can only reproduce discrete-cyclic activity and thus inherently fails to capture defining signatures of chaotic systems, like positive Lyapunov exponents and rather broad power spectra [1], [16]. A superficially similar principle is employed in classical nonlinear forecasting algorithms that retrieve from a library (history) locally neighboring trajectories whose next time step predictions are either just averaged or based on which a locally linear model is fit [72], [73]. Crucially, in contrast to mere context parroting, this neighborhood may change, however, with each prediction step taken. Through iterative reduction of DynaMix we arrive at a mathematical form that contains these classes of algorithms as special cases.

3 Methods↩︎

3.1 Background: DynaMix↩︎

DynaMix [1] is a DSR foundation model based on a mixture-of-experts architecture, using \(j=1 \dots J\) almost-linear RNNs (AL-RNNs) [10] as experts, \[\label{eq:alrnn} \boldsymbol{z}^j_{t+1}= F^\text{exp}_j(\boldsymbol{z}_t)=\boldsymbol{A}_j \boldsymbol{z}_t + \boldsymbol{W}_j \Phi^*(\boldsymbol{z}_t)+\boldsymbol{h}_j\;,\tag{1}\] where \(\boldsymbol{z}^j_t\in\mathbb{R}^M\) is an \(M\)-dimensional latent state, \(\Phi^*(\boldsymbol{z}_{t}) := \left[ \boldsymbol{z}_{1:M-P,t},\;\mathrm{ReLU}(\boldsymbol{z}_{M-P+1:M,t})\right]^\top\) is the activation function with \(M-P\) linear units and \(P\) ReLUs, and \(\boldsymbol{A}_j\in \mathrm{diag}(\mathbb{R}^M)\), \(\boldsymbol{W}_j\in\mathbb{R}^{M\times M}\), and \(\boldsymbol{h}_j\in\mathbb{R}^{M}\) are learnable parameters. The different experts are combined into an overall next state prediction by \(\boldsymbol{z}_{t+1}=\sum_{j=1}^J w^{exp}_{j,t}\cdot\boldsymbol{z}_{t+1}^j\), where the weights \(w^{exp}_{j,t} \in (0,1)\) are determined by a gating network through \[\label{eq:exp95weights} \boldsymbol{w}^{exp}_t(\boldsymbol{z}_t,\tilde{\boldsymbol{C}}\boldsymbol{w}^{att}_t)=\sigma\left(\frac{\text{MLP}(\tilde{\boldsymbol{C}}{\boldsymbol{w}^{att}_t},\boldsymbol{z}_t)}{\tau_{\text{exp}}}\right)\in\mathbb{R}^J\;,\tag{2}\] where \(\tau_{\text{exp}}\) is a temperature parameter and \(\tilde{\boldsymbol{C}}=\text{CNN}(\boldsymbol{C})\) are temporal features obtained from the provided context signal \(\boldsymbol{C}=\{\boldsymbol{c}_t\}\in\mathbb{R}^{N\times T_C}\) through a CNN. Attention weights \(\boldsymbol{w}^{att}_t\) at each time step \(t\) are in turn computed based on a distance between current latent state \(\boldsymbol{z}_{t}\) and the context \(\boldsymbol{C}\) as \[\label{eq:attn95weights} \boldsymbol{w}^{att}_t(\boldsymbol{z}_t,\boldsymbol{C}) = \sigma\left( \frac{\left| \boldsymbol{C} - \left( \boldsymbol{Dz}_t + \boldsymbol{\epsilon} \right)\boldsymbol{1}_{T_C}^\top \right|^\top\boldsymbol{1}_N}{\tau_{\text{att}}} \right) \in \mathbb{R}^{T_C}\;,\tag{3}\] where \(\tau_{\text{att}}\) is another temperature, \(\boldsymbol{D}\in\mathbb{R}^{N\times M}\) a learnable matrix, \(\boldsymbol{\epsilon}\sim\mathcal{N}(0,\boldsymbol{\Sigma})\) exploration noise, and \(\boldsymbol{1}_{\{T_C,N\}}\) are column vectors of ones of length \(T_C, N\), respectively. In [1], DynaMix was trained on \(\approx 6 \times 10^5\) trajectories from \(34\) \(3d\) DS in the cyclic or chaotic regime using control-theoretic training techniques that ensure DSR criteria are met [18], [29]. After pretraining, given any context time series snippet \(\boldsymbol{C}\), DynaMix is charged with forecasting the long-term trajectory evolution of the underlying DS without any retraining or fine-tuning.

3.2 Reducing DynaMix↩︎

At a high level, DynaMix can be compactly expressed as a single RNN of the following form \[\boldsymbol{z}_{t+1} = F_\text{DynaMix}(\boldsymbol{z}_t, \boldsymbol{C}) := \sum_j F_j^\text{exp}(\boldsymbol{z}_t)\cdot \boldsymbol{w}^{exp}_t(\boldsymbol{z}_t,\tilde{\boldsymbol{C}}\boldsymbol{w}^{att}_t)\;, \label{eq:update-rule-dynamix}\tag{4}\] with (see [1]) \[\tilde{\boldsymbol{C}}{\boldsymbol{w}^{att}_t} = \sum_{i=1}^{T_C-1} \text{CNN}(\boldsymbol{C})_i\cdot \boldsymbol{w}^\text{att}_i(\boldsymbol{z}_t,\boldsymbol{C})\;. \label{eq:attention-weights-dynamix}\tag{5}\] Note that eq. 4 is a weighted sum of piecewise-linear systems \(F_j^\text{exp}\), where, however, the weights \(\boldsymbol{w}^{exp}_t(\boldsymbol{z}_t,\boldsymbol{C})\) themselves are nonlinear functions of the current state and the context. As already shown in [1], the MLP in eq. 2 may be replaced by a linear layer w/o significant loss in performance, while in Appx. 6.3 we show the output nonlinearity \(\sigma\) (softmax) can as well be linearized over the relevant range. Further, the almost-linear term \(\Phi^*(\boldsymbol{z}_t)\) in eq. 1 can be made fully linear (Appx. 6.3), leaving \(\tilde{\boldsymbol{C}}=\text{CNN}(\boldsymbol{C})\) and \(\boldsymbol{w}^{att}_t(\boldsymbol{z}_t, \boldsymbol{C})\) as the only nonlinearities: \[\label{eq:dynamix95linear} \boldsymbol{z}_{t+1}=F_\text{Linear}(\boldsymbol{z}_t,\boldsymbol{C})=\boldsymbol{A}\boldsymbol{z}_t+\boldsymbol{B}(\text{CNN}(\boldsymbol{C}){\boldsymbol{w}^{att}_t}(\boldsymbol{z}_t, \boldsymbol{C}))+ \boldsymbol{h}\tag{6}\] We next observe that for \(\tau_\text{att} \to 0\), \(\boldsymbol{w}^{att}_t(\boldsymbol{z}_t, \boldsymbol{C})\) essentially just boils down to a nearest neighbor selector, \(\tau(\boldsymbol{z}_t)=\mathop{\mathrm{arg\,min}}_{s}\left\lVert\boldsymbol{c}_s-\boldsymbol{z}_t\right\rVert_2\). Further assuming a CNN kernel size of \(2\) and noticing that for a single linear CNN layer (as used in DynaMix) the CNN operations can be combined with \(\boldsymbol{B}\) in eq. 6 , we obtain \[\begin{align} f_\text{Linear}(\boldsymbol{z}_t,\boldsymbol{C})&=\boldsymbol{A}z_t+\boldsymbol{B}^{(1)}\boldsymbol{c}_{\tau(\boldsymbol{z}_t)}+\boldsymbol{B}^{(2)} \boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}+\tilde{\boldsymbol{h}} \tag{7} \\ \tau(\boldsymbol{z}_t)&=\mathop{\mathrm{arg\,min}}_{s}\left\lVert\boldsymbol{c}_s-\boldsymbol{z}_t\right\rVert_2 \tag{8} \end{align}\] where \(\boldsymbol{A},\boldsymbol{B}^{(1)},\boldsymbol{B}^{(2)} \in \mathbb{R}^{M\times M}\) and \(\tilde{\boldsymbol{h}}\in\mathbb{R}^M\). We call this form a recursive affine nearest-neighbor (NN) map.

3.3 DynaBase↩︎

In DSR, two vector fields that differ only by a rigid coordinate transformation describe the same underlying flow and therefore yield dynamically equivalent reconstructions [74]. Recognizing this invariance, we can state the following result:

Exploiting Thm. 1, we obtain a minimal model, which we call DynaBase, as \[\boldsymbol{z}_{t+1} = f_{\alpha,\beta}(\boldsymbol{z}_t) = \alpha\, \boldsymbol{z}_t + \beta\, \boldsymbol{c}_{\tau(\boldsymbol{z}_t)} + \gamma\, \boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1},\qquad \tau(\boldsymbol{z}_t)=\arg\min_{s}\left\lVert\boldsymbol{c}_s-\boldsymbol{z}_t\right\rVert_2, \label{eq:dynabase}\tag{9}\] with \(\alpha,\beta,\gamma\in\mathbb{R}\) and \(\alpha+\beta+\gamma=1\). Thus, \(\gamma=1-\alpha-\beta\) leaves \((\alpha,\beta)\in\mathbb{R}^2\) as the only free parameters of the minimal model (formalized in Thm. 1). If the data were noise-free, a natural self-consistency condition would be given by \[\boldsymbol{c}_{i+1} = f_{\alpha,\beta}(\boldsymbol{c}_i)\;\forall i \in [1,T_C-1] \;,\] which guarantees that whenever \(\boldsymbol{z}_t\) coincides with a context point \(\boldsymbol{c}_i\), update eq. 9 maps \(\boldsymbol{z}_t\mapsto \boldsymbol{c}_{i+1}\) exactly. Using this reasonable assumption, the affine NN map could be simplified further by imposing \(\alpha = -\beta \implies \gamma=1,\) such that the whole construction depends only on a single parameter: \[\boldsymbol{z}_{t+1}=f_\alpha(\boldsymbol{z}_t)= \alpha\,(\boldsymbol{z}_t - \boldsymbol{c}_{\tau(\boldsymbol{z}_t)})+\boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1} \;, \label{eq:self-consistent-one-param}\tag{10}\] where the local rate of con- or divergence is determined solely by \(\alpha\).

3.4 Training↩︎

For training DynaBase, we assume we are given \(N\)-dimensional training data \(\boldsymbol{X}\in\mathbb{R}^{N\times T}\), of which we define the first \(T_C\) time steps as the context signal \(\boldsymbol{C}\in\mathbb{R}^{N\times T_C}\). Given the model’s simplicity, it allows for straightforward training, where we assume \(\boldsymbol{z}_t=\boldsymbol{x}_t\) (i.e., the dynamics is directly defined in the observation space). Specifically, we explore the two following approaches:

3.4.0.1 Linear regression

As DynaBase is linear in parameters, its one-step-ahead prediction error loss is convex and we can optimize it by least-squares regression in one step using the reparameterized form \[\boldsymbol{z}_{t+1} - \boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1} = \alpha\,(\boldsymbol{z}_t - \boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}) + \beta\,(\boldsymbol{c}_{\tau(\boldsymbol{z}_t)}-\boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}). \label{eq:linear-regression}\tag{11}\] Given a training sequence, nearest neighbors \((\boldsymbol{c}_{\tau(\boldsymbol{x}_t)},\boldsymbol{c}_{\tau(\boldsymbol{x}_t)+1})\;\forall\boldsymbol{x}_i\in\boldsymbol{X}\) are simply precomputed based on eq. 8 , and ordinary-least-squares is used to solve for \((\hat{\alpha},\hat{\beta})\) in closed form. No backpropagation or teacher forcing is required; fitting is embarrassingly cheap compared to conventional FM pretraining.

3.4.0.2 Grid search

However, linear regression only optimizes for short term forecasts, and hence might not be optimal for DSR problems which require capturing long-term behavior. We therefore investigate a second strategy, where the loss is given through a long-term reconstruction measure [29], [75] \[\label{eq:KL95loss} \text{Loss}(\boldsymbol{X},\boldsymbol{\hat{X}})= D_{\mathrm{stsp}}(p_{\mathrm{true}}(\boldsymbol{X})\|p_{\mathrm{pred}}(\boldsymbol{\hat{X}})) \;,\;\tag{12}\] i.e. the Kullback-Leibler divergence between the true and model-generated trajectory distributions based on long term rollouts \(\boldsymbol{\hat{X}}\) of DynaBase. \(D_{\mathrm{stsp}}\) is a common measure in the DSR literature to assess the agreement in true and reconstructed attractor geometries [11], [18], [23], [25], [29], [75], [76]. Since the model has only 2 trainable parameters, searching for an optimal solution by grid search is fairly straightforward and inexpensive.

4 Results↩︎

4.1 Theoretical results↩︎

We now establish a theoretical characterization of the forecasting properties of DynaBase.

4.1.0.1 Structural characterization

By Thm. 1, DynaBase is a unique nearest-neighbor affine family compatible with the symmetry assumptions in Sec. 3.3. We use this structural result as the starting point for the two main mechanism-level consequences below.

Theorem 2 shows that the context-parroting baseline of [2] is recovered for free from DynaBase by simply setting \((\alpha,\beta)=(0,0)\) and delay-embedding the context. Any DynaBase model with \(\alpha\neq 0\) is therefore strictly more expressive than parroting in the sense that a larger class of DS can be embedded.

Given that the map \(f_\alpha\) is piecewise-affine, we can also derive an exact continuous-time analogue [77] \(\boldsymbol{\zeta}(t)\), such that \(\boldsymbol{\zeta}(kh)=\boldsymbol{z}_k \forall k \in \mathbb{Z}, h \in \mathbb{R}\), by integrating the equations \[\dv{\boldsymbol{\zeta}}{t} = \frac{\log\alpha}{h}\boldsymbol{\zeta}-\frac{1}{h}\frac{\log{\alpha}}{1-\alpha}(\boldsymbol{c}_{\tau(\zeta)+1}-\alpha \boldsymbol{c}_{\tau(\zeta)})\label{eq:continuous-limit}\tag{13}\] where we denote the discrete timestep size of \(f_\alpha\) as \(h\) and \(\frac{\log\alpha}{1-\alpha}\rvert_{\alpha=1}\) is defined by its continuous extension. Notably, this limit does not exist for \(\alpha = 0\) (context parroting), and is only defined in the complex number space for \(\alpha < 0\). For \(\alpha = 0\), the map collapses onto \(\boldsymbol{C}\) at every step, i.e. hops between a discrete set of points, which is not reproducible by a continuous flow unless \(\boldsymbol{C}\) contains exactly one (fixed) point. Importantly, context parroting is therefore an inherently discrete special case of DynaBase, defined on a measure-zero set, which cannot reproduce true limit cycles or chaotic behavior. A detailed derivation is given in Appx. 6.6.0.1.

4.1.0.2 Dynamical characterization

We now characterize the dynamical properties of the self-consistent DynaBase – specifically how the parameter \(\alpha\) governs the transition from trivial to genuinely chaotic behavior.

The case \(\alpha=0\) is therefore dynamically trivial: all orbits eventually collapse to \(\boldsymbol{C}\); context parroting is thus structurally incapable of producing chaos.

The theorem thus establishes that \(\alpha\in[0,1]\) is sufficient for the orbit to remain confined to – or track – the context sequence, as distances to \(\boldsymbol{C}\) are non-increasing. For \(\alpha>1\), the map will diverge almost everywhere. If the arrangement of Voronoi cells through the context \(\boldsymbol{C}\) is, however, such that its dynamics still remains bounded, it will almost surely follow a chaotic attractor [78], [79]. An example is discussed in the proof to Thm. 4 and illustrated in Fig. 2b.

4.2 Empirical forecasting comparison↩︎

We evaluate DynaBase on the same DSR test set as in [1], reporting the same three complementary measures, two of them specific to DSR (see Appx. 6.2 for definitions): the disagreement \(D_{\mathrm{stsp}}\) in state space geometry, the long-term temporal disagreement \(D_H\), and the short-term mean absolute scaled error (MASE). To evaluate the long-term statistics \(D_{\mathrm{stsp}}\) and \(D_H\), each system is forecast for \(T=10{,}000\) time steps, following [1] and in agreement more generally with the DSR literature [13], [16], [18] (note that point forecasts become meaningless for chaotic systems beyond a few Lyapunov times). As introduced in Sec. 3.4, we fit DynaBase by either (i) closed-form one-step linear regression, optimizing the one-step MSE in eq. 11 , or (ii) directly for DSR over the grid of parameter values. As baselines we include context parroting as reported in [2], [25], DynaMix [1] and Chronos [20], [21] as FMs, and three custom-trained DSR baselines (AL-RNN [10], reservoir computers [80], and Neural ODEs [35]) trained directly on the context signal provided to the FMs.

As shown in Fig. 1, DynaBase generally performs best, with the variant optimized for DSR (ii) better than the one trained for ahead-prediction (i) on the long-term measure \(D_\text{stsp}\) (while the differences in MASE and \(D_H\) are not statistically significant, Wilcoxon signed-rank test, \(p>0.1\)). However, for longer (but not too long) prediction horizons, optimizing for ahead-prediction achieves lower MASE than DSR-optimization (Table 2). This difference illustrates that, not unexpectedly, the choice of training objective determines the model’s forecasting strategy: a one-step MSE penalizes any local deviations from the ground truth, and hence tends to produce smaller \(\alpha\) values enabling the map to track the immediate next step tightly, yielding better short-term predictions. In contrast, the grid optimization strategy (ii) directly targets the invariant attractor geometry, outperforming (i) on DSR. In both cases, however, training DynaBase with its few parameters is extremely cheap and rapid.

The distribution of estimated \(\hat{\alpha}\) values across the test systems in Fig. 2a makes this point explicit. The two training objectives lead to qualitatively different parameter regimes: linear regression on the 1-step MSE places the map mostly into the parroting/ semi-cyclic regime with \(\alpha<1\), while optimizing directly on \(D_{\mathrm{stsp}}\), in contrast, produces mostly chaotic behavior (in accordance with the ground-truth DS) with \(\alpha>1\). This implies that only DSR-based training has correctly inferred the chaotic ground truth systems, as further confirmed by a high correlation (\(r \approx0.90\)) between max. Lyapunov exponents of the GT DS and the DynaBase map when fitted by DSR, but not when fitted by least-squares (\(r \approx0.18\)), Fig. 11. Moreover, the loss landscape reveals an elongated valley along the \(\alpha=-\beta\) axis (Fig. 2b), providing empirical support for the self-consistency condition in eq. 10 , which reduces DynaBase to a one-parameter map. The bifurcation graph in Fig. 2c further illustrates on a synthetic toy example the cyclic behavior for \(\alpha<1\) and emergence of chaos for \(\alpha>1\).

Another interesting observation is that for DSR-specific training, \(\hat{\alpha}\) tightly concentrates on a single value near \(\alpha\approx1.01\), across the diverse set of DS used. This suggests that, at least on this benchmark, the \(\alpha\)-parameter may be fixed a priori. We test this by estimating \(\alpha\) on a separate training set (similar to the one used for training DynaMix in [1]), obtaining \(\alpha=1.006\) close to the median in Fig. 2a, which leads to a parameter-free zero-shot forecaster that requires no DS-specific fitting. Applying this to the context signals from the test set, we still achieve performance comparable to the custom-trained models (see Fig. 1, and Figs. 6-9 for zero-shot reconstructions).

4.3 Loss landscape tracks dynamics↩︎

To examine how DynaBase’ parameters map onto the ground truth system’s dynamical regime, we study the DSR loss landscape on a representative chaotic, cyclic and fixed point system (Fig. 3). For these different dynamical regimes, the loss landscape exhibits distinct global minima which correspond to different dynamical mechanisms:

• Context parroting \(\alpha=-\beta=0\): the map reduces to \(\boldsymbol{z}_{t+1}=\boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}\), i.e.the state snaps onto the successor of its nearest context neighbor. For a delay-embedded context signal, this exactly reproduces the context-parroting algorithm of [2] (see Thm. 2).

• Flow mimicry \(\alpha=-\beta=1\): the map reduces to \(\boldsymbol{z}_{t+1}-\boldsymbol{z}_t = \boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}-\boldsymbol{c}_{\tau(\boldsymbol{z}_t)}\), a forward-Euler step driven by the local finite difference \(\dot{\boldsymbol{c}}\approx(\boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}-\boldsymbol{c}_{\tau(\boldsymbol{z}_t)})/h\) of the underlying vector field as sampled from the context. This yields a ‘zero-friction’ direction in state space (neither con- nor divergence), as along the limit cycle (\(\lambda_\text{max}=0\)) of a continuous-time DS.

• Chaos \(\alpha>1\): One obtains local divergence almost everywhere and, if bounded, chaotic trajectories are produced (cf. Fig. 12 for Lorenz-63 reconstructions along a bifurcation diagram).

The nearest-neighbor lookup \(\tau(\cdot)\) induces a Voronoi tessellation of the state space (Fig. 4a). Inside each cell \(V_s=\{\boldsymbol{z}:\tau(\boldsymbol{z})=s\}\), DynaBase is a strictly affine map \(\boldsymbol{z}_{t+1}=\alpha\, \boldsymbol{z}_t + \tilde{\boldsymbol{c}}_s\), where \(\tilde{\boldsymbol{c}}_s=\beta \boldsymbol{c}_s+\gamma \boldsymbol{c}_{s+1}\) is fixed by the cell, hence belongs to the class of well-studied switching piecewise-linear (PL) systems [81], [82]. Within each cell \(V_s\), the model’s Jacobian is simply given by \(\alpha\boldsymbol{I}\), i.e. a single scalar \(\alpha\) controls the local con- or divergence almost everywhere in state space. With \(\hat{\alpha}\approx 1.01\) just slightly above \(1\) (Fig. 2), DynaBase diverges almost everywhere but still stays sufficiently close to the cell centers to create a chaotic attractor, despite not having a nonlinear activation function (in line with Thm. 3 & Thm. 4). For context parroting with \(\alpha=0\) (Thm. 2), on the other hand, DynaBase snaps onto a cyclic point that perfectly traces out the empirically observed context trajectory. However, a cyclic point would result in this case even if the observed trajectory snippet comes from a chaotic system (Fig. 13).

Pure context parroting also prevents generalization to new initial conditions of an observed DS, a feature of DynaMix demonstrated in [1]. This is illustrated for the Selkov DS in Fig. 4b: For \(\alpha=0\), DynaBase strictly follows the context points and thus fails to map the vector field anywhere outside the context. On the other hand, pure context mimicry, \(f_1\), while reproducing the \(\lambda_\text{max}=0\) direction on a limit cycle, fails to capture the convergence when started off the stable cycle, since the finite difference \(\boldsymbol{c}_{\tau(\boldsymbol{z}_t)+1}-\boldsymbol{c}_{\tau(\boldsymbol{z}_t)}\) is always evaluated at the nearest context point. Fitting \(\hat{\alpha}\) via eq. 12 on a trajectory bit that contains a transient, however, yields a better approximation of the surrounding vector field outside the observed context while still reproducing the cycle (Fig. 4b).

5 Conclusion↩︎

We reduce DynaMix, a mixture-of-experts DSR foundation model, into DynaBase, a two-parameter nearest-neighbor affine recurrence that can be further reduced to a \(1d\) family of maps by imposing a consistency condition. Despite this minimal form, variants of DynaBase match or exceed substantially larger architectures on both long-horizon DSR measures and short-term predictions, while remaining computationally extremely cheap with either training via 1-step closed-form least squares or grid search on a DSR objective.

Analysis of this minimal map and its training provides a number of interesting insights into the minimal requirements TS and DSR FMs may need to achieve zero-shot DS forecasts. First, \(\alpha = 0\) exactly reproduces context parroting as observed in [2], [25], but this mechanism is inherently unable to reproduce chaotic or even true limit cycle behavior. Second, for \(\alpha > 0\) this affine nearest-neighbor form admits the whole spectrum from convergence to fixed points or cycles to strictly chaotic activity for \(\alpha > 1\), thus being sufficiently expressive to approximate a large range of DS. Third, for \(\alpha \approx 1.01\) the map can even operate in zero-shot mode, at least on a common DSR benchmark set, exposing the profound role the context alone plays in structuring the forecast. Fourth, optimizing for ahead-predictions biases the map toward convergence to fixed or cyclic points, even when the underlying DS is chaotic, explaining why this training objective is suboptimal if DSR is the goal [13], [18], [29], [51]. More generally, with DynaBase we have established a formal framework that enables to study mechanisms of DSR in FMs and beyond in detail, creating an instrument for mathematical analysis of context-based forecasting and DSR methods.

5.0.0.1 Limitations

DynaBase is a minimal 2-parameter model. Although it highlights the fact that complex architectures with tens of thousands to many millions of parameters may not be needed for DSR and time series forecasting, the assumptions on which its reduction is based come at a cost in terms of the diversity of DS and vector field topologies that can be learned, as well as the accuracy with which geometrical properties of the vector field can be captured (as apparent from the remaining mismatch in Fig. 4b). Also, we so far did not explicitly consider the impact of noise (but see Fig. 14, where noise degrades but does not preclude DSR). Currently, DynaBase may be viewed more as a research tool that allows for detailed mathematical analysis of context-based DSR FMs. A more in-depth study of DynaBase, e.g. its DS approximation capabilities or the computation of the full Lyapunov spectrum, which requires the concept of a saltation operator [83], [84], is still pending.

Acknowledgements↩︎

This work was supported by individual grants Du 354/15-1 (# 502196519) and Du 354/18-1 (# 567025973) from the German Research Foundation (DFG), and by the German Ministry for Research, Astronautics, and Technology (BMFTR) through NAILIt (“Neuro-Inspired AI for Learning & Inference in Non-Stationary Environments”, # 01GQ2509A).

6 Appendix↩︎

6.1 Methodological details↩︎

6.1.0.1 Training details

Training of the DynaBase parameters \((\alpha, \beta)\) uses a sampled training sequence \(\boldsymbol{X}_{\text{train}}\in\mathbb{R}^{N\times T}\) drawn from a DS, where the first \(T_C\) steps serve as context signal \(\boldsymbol{C}\). For the experiments in Fig. 1, we set \(T = 2000\) and \(T_C = 1000\). At evaluation time, we supplied the training sequence as context alongside the estimated parameter values \((\alpha, \beta)\), and assessed performance on a sequence unseen during training and context construction (using the self-consistent DynaBase \(f_\alpha\) as defined in eq. 10 ). This ensures that DynaBase receives exactly the same context and training sequence information as all other comparison methods.

To fit DynaBase via linear regression, we compute \((\hat{\alpha},\hat{\beta})\) in closed form via ordinary least-squares. Nearest neighbors \(\tau(\cdot)\) are precomputed once via \(\ell^2\)-distances over \(\boldsymbol{C}\) and the training sequence \(\boldsymbol{X}\). To cast this as a standard linear regression problem, we define the response vector \(\boldsymbol{b}\) and design matrix \(\boldsymbol{A}\) based on eq. 9 , for each training sample \(i\) \[\begin{align} \boldsymbol{b}_i &= \boldsymbol{z}_{i+1} - \boldsymbol{c}_{\tau(\boldsymbol{z}_i)+1}, \\ \boldsymbol{A}_i &= \Bigl[\;\boldsymbol{z}_i - \boldsymbol{c}_{\tau(\boldsymbol{z}_i)+1}\;,\;\boldsymbol{c}_{\tau(\boldsymbol{z}_i)} - \boldsymbol{c}_{\tau(\boldsymbol{z}_i)+1}\;\Bigr]. \end{align}\] The regression then takes the form \(\boldsymbol{b} = \boldsymbol{A}[\alpha, \beta]^\top\), and the coefficients are obtained by the standard least-squares solution \[[\hat{\alpha}, \hat{\beta}]^\top = (\boldsymbol{A}^\top \boldsymbol{A})^{-1}\boldsymbol{A}^\top \boldsymbol{b}.\] In the self-consistent setting of eq. 10 , the constraint \(\beta=-\alpha\) reduces the two-parameter fit to a single unknown scalar, simplifying \(\boldsymbol{A}_i = \boldsymbol{z}_i - \boldsymbol{c}_{\tau(\boldsymbol{z}_i)}\).

To train parameters via grid search, we sweep across \(\alpha \in [0,1.2]\) with step size \(\Delta\alpha=0.002\) along the self-consistency line \(\beta=-\alpha\). For each candidate \(\alpha\), DynaBase is rolled out for \(T=10{,}000\) steps using the context \(\boldsymbol{C}\) and scored by the KL-divergence loss given in eq. 12 between predicted trajectories \(\hat{\boldsymbol{X}}\) and the training data. The global minimizer \(\hat{\alpha}\) is returned.

Training and evaluation of DynaBase was performed on a single CPU (AMD EPYC 9655 96-Core).

6.1.0.2 Comparison methods

We compare DynaBase against three groups of baselines, all evaluated on the same \(54\) test DS and the same input data of length \(T=2000\) as in [1].

Zero-shot foundation models: DynaMix [1], the DSR FM that DynaBase is reduced from (mixture-of-experts of AL-RNNs with context-dependent gating; pretrained on \(\sim\!6\times 10^5\) trajectories from \(34\) DS); Chronos-T5-base [20], a transformer FM that quantizes real-valued series into discrete tokens and forecasts autoregressively on a corpus of synthetic and real-world time series; and Chronos-2 [21], its multivariate extension augmenting the T5 backbone with a group-attention mechanism. Evaluations were taken from [1].

Custom-trained DSR models: an AL-RNN [10] (eq. 1 ), which is used for the experts in DynaMix, a Neural-ODE [35], and a reservoir computer [80]; each is trained from scratch on the same \(T=2000\) context segment provided to the foundation models. Evaluations were taken from [1].

Context parroting: the algorithm of [2], which retrieves the best-matching motif in \(\boldsymbol{C}\) and copies its successor as forecast. As shown in Thm. 2, this baseline corresponds to the \(\alpha=0\) case of DynaBase on a delay-embedded context.

6.2 Performance measures↩︎

6.2.0.1 Long-term metrics

To evaluate the geometric similarity between true and model-generated reconstructions, we employ a measure, \(D_{\text{stsp}}\), based on the Kullback-Leibler (KL) divergence evaluated in state space [1], [11], [23], [25], [29], [75], [76]. This measure quantifies the (mis)match between the ground-truth spatial distribution of trajectory points, \(p_{\text{true}}(\boldsymbol{x})\), and the distribution \(p_{\text{gen}}(\boldsymbol{x}|\boldsymbol{z})\) of points from trajectories freely generated by the model: \[D_{\text{stsp}}\bigl(p_{\text{true}}(\boldsymbol{x}) \,\|\, p_{\text{gen}}(\boldsymbol{x}|\boldsymbol{z})\bigr) = \int p_{\text{true}}(\boldsymbol{x}) \log \frac{p_{\text{true}}(\boldsymbol{x})}{p_{\text{gen}}(\boldsymbol{x}|\boldsymbol{z})} \, d\boldsymbol{x}.\] In practice, we approximate this via a discrete binning of the state space into \(K = m^N\) bins, where \(m\) is the number of bins per dimension and \(N\) is the system dimensionality, estimating occupation probabilities through relative frequency of visits \(\hat{p}_i\): \[D_{\text{stsp}} \approx \sum_{i=1}^{K} \hat{p}_{\text{true};i} \log\frac{\hat{p}_{\text{true};i}}{\hat{p}_{\text{gen};i}}.\] To ensure that the system has reached a steady-state distribution, long trajectories (\(T=10000\)) are sampled from the trained model. For the 3d attractors we set \(m = 30\) bins per dimension, following [48].

To evaluate long-term temporal reconstruction quality, we calculate the Hellinger distance \(D_H\) between the power spectra of the ground-truth and model-generated time series [16], [18], [29], [76]. We first apply dimension-wise Fast Fourier Transforms (FFT) to the time series and then smooth the resulting power spectra with a Gaussian kernel (with \(\sigma=20\)), followed by normalization to enable comparison across dimensions. High-frequency tails dominated by noise are removed. The Hellinger distance between the smoothed true spectrum \(F(\omega)\) and the generated spectrum \(G(\omega)\) is given by \[D_H\bigl(F(\omega), G(\omega)\bigr) = \sqrt{1 - \int_{-\infty}^{\infty} \sqrt{F(\omega)\,G(\omega)}\, d\omega} \;\in [0, 1],\] where values close to \(0\) indicate high spectral agreement and values close to \(1\) indicate complete disagreement. \(D_H\) is reported as the average across all dimensions. Note that both \(D_{\text{stsp}}\) and \(D_H\) are designed to capture long-term reconstruction quality in the limit \(T \to \infty\), after transients have settled and the attractor geometry has been traced out reasonably well (which was the case here for \(T=10000\) steps).

6.2.0.2 Short-term forecast quality

For evaluating short-term predictive performance, we use the Mean Absolute Scaled Error (MASE) [1], [85], which is a normalized \(n\)-step-ahead prediction error aggregated across all \(N\) output dimensions: \[\text{MASE} = \frac{1}{N} \sum_{i=1}^{N} \frac{\dfrac{1}{n} \displaystyle\sum_{t=1}^{n} \bigl|x_{i,t} - \hat{x}_{i,t}\bigr|}{\dfrac{1}{T} \displaystyle\sum_{t=1}^{T} \bigl|x_{i,t} - x_{i,t-1}\bigr|},\] where \(T\) denotes the total length of the ground truth time series, \(n\) is the forecast horizon, \(x_{i,t}\) the true value, and \(\hat{x}_{i,t}\) the model prediction for dimension \(i\) at time \(t\). We evaluate at \(n = 10\) steps.

6.2.0.3 Lyapunov exponent

To numerically estimate the maximum Lyapunov exponent \(\lambda_{\text{max}}\) from a time series \(\boldsymbol{X}\), we use the Rosenstein algorithm [86]. For each state \(\boldsymbol{x}_i\), its nearest neighbor \(\boldsymbol{x}_j\) is identified subject to \(|i - j| > l_t\), which excludes direct temporal neighbors on the same trajectory segment. We set \(l_t = 100\) (based on the mean periodicity [86]). The divergence between neighboring trajectories is then tracked over time steps \(k = 0, \ldots, k_{\max}\), \[d_i(k) = \| \boldsymbol{x}_{i+k} - \boldsymbol{x}_{j(i)+k} \|_2\;.\] Since nearby trajectories diverge as \[d_i(k) \approx d_i(0)\, e^{\lambda_{\text{max}} k \Delta t}\;,\] where \(\Delta t\) is the sampling interval, taking the logarithm and averaging over all pairs gives \[\langle \ln d(k) \rangle \approx \lambda_{\max} k \Delta t + C\;.\] For chaotic systems, this quantity grows approximately linearly with \(k\) for a suitable \(k_{\max}\) (here: \(100\)) [73], with slope \(\lambda_{\max} > 0\), whereas periodic systems yield \(\lambda_{\max} = 0\).

When the equations of motion \(\dot{\mathbf{x}} = f(\mathbf{x})\) are known, the full Lyapunov spectrum \(\{\lambda_i\}\) can be computed via a QR decomposition algorithm [87], which integrates the system along its variational equations \(\dot{Q} = J(\mathbf{x})Q\), where \(J(\boldsymbol{x}) = \partial f / \partial \boldsymbol{x}\) is the Jacobian of \(f\) and \(Q\) is an \(n \times n\) matrix of perturbation vectors initialized to the identity \(I_n\). At regular time intervals, a QR decomposition \(Q = \tilde{Q}R\) is performed and the columns of \(\tilde{Q}\), which form an orthonormal basis, replace \(Q\). The diagonal entries \(R_{ii}\) record the local expansion rates, and the \(k\)-th Lyapunov exponent is obtained as the time-averaged logarithmic growth rate, \[\lambda_k = \lim_{T \to \infty} \frac{1}{T} \sum_{j} \log \left| R_{kk}^{(j)} \right|,\] where the sum runs over all QR factorizations performed up to time \(T\).

6.3 Details on DynaMix reduction↩︎

Here we provide the supporting evidence and derivation details for the successive reduction of DynaMix to the recursive affine NN map eq. 7 described in Sec. 3.2.

6.3.0.1 Linear approximation

As noted in [1], replacing the MLP in eq. 5 with a linear layer does not reduce performance. Here we further show that dropping the softmax nonlinearity \(\sigma\) in eq. 3 and the piecewise-linear activation \(\Phi^*\) in eq. 1 , reducing the mixture of AL-RNN experts to a single affine map \(F_\text{Linear}\) as in eq. 6 , does not cause performance to degrade either. Table 1 confirms this: Linear experts & gating matches, or even improves upon, the original DynaMix checkpoint on all three measures across the DS test set.

6.3.0.2 Latent dimension

The AL-RNN experts in DynaMix operate in an \(M\)-dimensional latent space distinct from the \(N\)-dimensional observation space, requiring the projection \(\boldsymbol{D}\) in eq. 3 . Replacing the expert block with \(F_\text{Linear}\) eliminates this distinction, yielding \(M=N\) and \(\boldsymbol{D}=\boldsymbol{I}\), such that the model operates entirely in observation space. Table 1 (Linear experts & gating, \(M=N\)) confirms this does not reduce performance on the present test set. Working in a single space is also a prerequisite for the \(O(N)\) equivariance exploited in Thm. 1, since rotations and the nearest-neighbor distance must act on the same space in which the state \(\boldsymbol{z}_t\) and the context \(\boldsymbol{C}\) live. Note we can always perform a classical delay-embedding [88], [89] should this dimensionality not be sufficient to properly represent an empirically observed DS.

6.3.0.3 Nearest-neighbor selector

In the publicly available DynaMix checkpoint, the learned attention temperature \(\tau_\text{att}\) converged to \(|\tau_{\text{att},\text{learned}}| \approx 0.06\), indicating that the soft attention eq. 3 is already operating close to its \(\tau_\text{att} \to 0\) limit, i.e. effectively selecting a single nearest context point at each step. Two deliberate departures from DynaMix are made in passing to the hard limit. First, DynaMix injects Gaussian exploration noise \(\varepsilon \sim \mathcal{N}(0, \boldsymbol{\Sigma})\) into the attention eq. 3 ; in favor of a minimal and deterministic model, we set \(\boldsymbol{\Sigma}=0\). Second, DynaMix computes context distances with the \(\ell^1\)-norm, whereas we adopt the Euclidean \(\ell^2\)-norm. The invariance of \(\ell^2\) distances under orthogonal transformations \(R \in O(N)\), is used in Thm. 1. Together these choices yield the deterministic selector \(\tau(\boldsymbol{z}_t) = \mathop{\mathrm{arg\,min}}_s \|\boldsymbol{c}_s - \boldsymbol{z}_t\|_2\).

6.3.0.4 CNN expansion

To make the recursive map fully explicit in terms of individual context points, we expand the CNN operation. Since DynaMix uses a single linear CNN layer with kernel size \(2\), for \(s \in 1\dots T_C - 1\) we have \[\tilde{\boldsymbol{c}}_s = \text{CNN}(\boldsymbol{C})_s = \boldsymbol{K}^{(1)} \boldsymbol{c}_s + \boldsymbol{K}^{(2)} \boldsymbol{c}_{s+1} + \boldsymbol{b},\] where \(\boldsymbol{K}^{(1)}, \boldsymbol{K}^{(2)} \in \mathbb{R}^{N \times N}\) and \(\boldsymbol{b} \in \mathbb{R}^N\) are learned weights. Substituting into eq. 6 with the nearest-neighbor selector and absorbing \(\boldsymbol{B}\boldsymbol{K}^{(1)}\), \(\boldsymbol{B}\boldsymbol{K}^{(2)}\) and the bias into \(\boldsymbol{B}^{(1)}\), \(\boldsymbol{B}^{(2)}\), and \(\tilde{\boldsymbol{h}}\) respectively, yields the recursive affine NN map eq. 7 as stated in the main text.

6.4 Datasets↩︎

Our DS dataset consists of simulated time series of length \(10^5\), produced from \(3d\) DS collected in [4]. For the test set, we used the same \(54\) distinct systems as in [1] to enable a fair comparison among all the baseline models in relation to the originally published results. To evaluate DynaBase in a zero-shot setting, we estimated \(\alpha\) using a training set of \(30\) different chaotic DS, analogous to the setup in [1].

In addition, we conduct experiments on the \(2d\) Selkov system [90], which models the kinetics of an open monosubstrate enzyme reaction and is defined by the equations \[\begin{align} \frac{dx}{dt} &= -x + ay + x^2y, \\ \frac{dy}{dt} &= b -ay - x^2y , \end{align}\] where we chose \(a=0.1\) and \(b=0.5\). The system was solved numerically with integration time step \(\Delta t = 0.3\) using scipy.integrate with the RK45 solver.

For Fig. 12 we used the Lorenz-63 attractor [91], originally formulated by Edward Lorenz in 1963 [91] to model atmospheric convection. It is defined by \[\begin{align} \frac{\text{d}x}{\text{d}t} & = \sigma(y-x)\\ \nonumber \frac{\text{d}y}{\text{d}t} & = x(\rho-z)-y\\ \nonumber \frac{\text{d}z}{\text{d}t} & = xy-\beta z, \end{align}\] where \(\sigma, \rho, \beta\), are control parameters of the system (we fix \(\sigma=10\) and \(\rho=28\), and vary \(\beta\in[0,1]\), producing regimes with point attractors, cyclic, and chaotic dynamics). The system was solved numerically with integration time step \(\Delta t = 0.02\) using scipy.integrate with the RK45 solver.

6.5 Further empirical results↩︎

6.6 Further theoretical results↩︎

6.6.0.1 Details on continuous-time limit.

We begin by writing eq. 10 explicitly as a piecewise-affine discrete-time system \[\boldsymbol{z}_{k+1}=f_\alpha(\boldsymbol{z}_k)=\alpha\boldsymbol{I}\boldsymbol{z}_k+(\boldsymbol{c}_{\tau(\boldsymbol{z}_k)+1}-\alpha\boldsymbol{c}_{\tau(\boldsymbol{z}_k)}) \;.\] For values of \(\alpha\in\mathbb{R}/\{0,1\}\), Thm. 3 in [77] guarantees the existence of an equivalent affine continuous-time system, in the sense that \[\boldsymbol{\zeta}(t_0)=\boldsymbol{z}_{k_0},\;\boldsymbol{\zeta}(t_0+h)=\boldsymbol{z}_{k_0+1} \;,\] where \(h\) denotes the discrete timestep at which observations \(\boldsymbol{z}_k\) are sampled. The equivalent continuous-time system is then given by \[\dv{\boldsymbol{\zeta}}{t} = \frac{\log\alpha}{h}\boldsymbol{\zeta}-\frac{1}{h}\frac{\log\alpha}{1-\alpha}(\boldsymbol{c}_{\tau(\boldsymbol{\zeta})+1}-\alpha\boldsymbol{c}_{\tau(\boldsymbol{\zeta})}) \;.\] For the case \(\alpha=1\), we define \(\frac{\log{\alpha}}{1-\alpha}\rvert_{\alpha=1} :=-1\) by its continuous extension. Such an extension does, however, not exist for \(\alpha=0\). For \(\alpha<0\), the logarithm is complex-valued. Hence, a real-valued continuous-time limit for \(f_\alpha\) exists only for \(\alpha>0\).

6.6.0.2 Computational complexity

The minimal architecture of DynaBase makes it computationally highly efficient for zero-shot inference. By exploiting discrete nearest-neighbor lookup and efficient heuristic search algorithms such as \(k\)-d trees [92], the time complexity of running DynaBase autoregressively over \(T_F\) forecast steps is \[\mathcal{T} = \mathcal{O}(T_F \log T_C),\] with a one-time overhead of \(\mathcal{O}(T_C \log T_C)\) for constructing the \(k\)-d tree. In higher dimensions, query time degrades toward \(\mathcal{O}(T_C)\) in the worst case. Provided the generated trajectory is not retained, the space complexity reduces to \[\mathcal{S} = \mathcal{O}(T_C).\] These bounds allow DynaBase to generate long forecasts over extended contexts at low computational cost — in contrast to standard transformer architectures, which incur quadratic time and space complexity in context length.

7 Proofs of Theorems↩︎

In the following we use the same notation as introduced in Sec. 3.3, with \(\boldsymbol{C}=\{\boldsymbol{c}_1,\dots,\boldsymbol{c}_{T_C}\}\in\mathbb{R}^{N\times T_C}\) a context sequence, \(\tau(\boldsymbol{z})=\mathop{\mathrm{arg\,min}}_{s}\left\lVert\boldsymbol{c}_s-\boldsymbol{z}\right\rVert_2\) the nearest-neighbor lookup, and \[f_{\alpha,\beta}(\boldsymbol{z})=\alpha\boldsymbol{z}+\beta\boldsymbol{c}_{\tau(\boldsymbol{z})}+\gamma\boldsymbol{c}_{\tau(\boldsymbol{z})+1},\qquad \gamma=1-\alpha-\beta,\] the two-parameter DynaBase map; \(f_\alpha\) denotes its self-consistent reduction with \(\beta=-\alpha\), \(\gamma=1\). Ties in \(\tau\) are broken by the smallest index; the tie set has measure zero and does not affect any of the statements below.

7.1 Proof of Theorem 1 (Uniqueness of form)↩︎

7.2 Proof of Theorem 2 (Reduction to context parroting)↩︎

7.3 Proof of Theorem 3 (Long-term dynamics)↩︎

7.4 Proof of Theorem 4 (Boundedness)↩︎

References↩︎