1 Introduction↩︎

Time series analysis is a pivotal field with broad-ranging applications, from weather forecasting [1] and physiological anomaly detection [2], to data imputation [3] and human activity recognition [4]. Its profound practical utility continues to attract significant research and industrial interest.

Inspired by the success of foundation models in language [5], images [6], and videos [7], researchers have been striving to develop general-purpose time series foundation models (TSFMs). Trailblazing efforts like MOMENT [8], Timer [9], and UniTS [10] have shown the potential of unified architectures pretrained on large-scale datasets to address multiple downstream tasks.

Despite these advances, such models have primarily demonstrated zero-shot capability only in forecasting tasks, while adapting to other downstream tasks typically requires additional training [9], [10]. This limitation contrasts with the general expectations of foundation models and has consequently led most recent efforts to focus predominantly on forecasting [11]–[13]. This limitation can be traced to two fundamental dilemmas in existing TSFMs.

Architectural dilemma between granularity and scalability. Most existing TSFMs adopt patch-wise tokenization [14] to improve semantic density and computational efficiency. However, this design inevitably sacrifices point-level temporal details, hindering its effectiveness on reconstruction-based tasks like imputation and anomaly detection. For example, MOMENT, pretrained with patch-level reconstruction, exhibits a significant performance drop when shifting from patch-missing to point-missing imputation (evidenced in Table 4). In contrast, point-wise tokenization [11], [15] preserves fine-grained structure but suffers from low information density and prohibitive computational overhead on long sequences.

Training dilemma for divergent inductive biases. Although recent works attempt a unified modeling paradigm, they often neglect the fundamentally different inductive biases required by each objective: forecasting relies on extrapolation, imputation and anomaly detection necessitate interpolation, and classification requires global abstraction. This heterogeneity indicates that a single, monolithic training objective, such as BERT-style masked reconstruction [8], [16] or GPT-style autoregressive generation [9], is unlikely to simultaneously endow all necessary capabilities.

Collectively, these dilemmas have hindered prior TSFMs from fully generalizing across diverse tasks, often necessitating task-specific fine-tuning to achieve strong performance. To address these challenges, we introduce Zeus, a unified TSFM that bridges point-level fidelity and long-sequence scalability while supporting diverse task-specific inductive biases within a single pretraining framework. Zeusis designed to operate in a fully tuning-free ¹ manner, enabling out-of-box deployment across diverse downstream tasks.

To overcome the tension between granularity and scalability, Zeusadopts a U-shaped multi-scale hierarchy with point-wise tokenization. It follows a fine-to-coarse-to-fine information flow, progressively aggregating fine-grained information into coarse-grained latent representations and then symmetrically refining them. Lightweight Transformer blocks are used at high resolutions to preserve local details, while deeper and wider blocks operate on compressed representations to efficiently capture global dependencies.

Complementing this architecture, we address the challenge of divergent inductive biases with MOTM, a Multi-Objective Temporal Masking strategy that jointly trains Zeusfor extrapolation, interpolation, and global abstraction. By exposing the model to predictive, point-wise, and block-wise corruption patterns during pretraining, MOTM enables Zeusto learn a unified yet versatile representation space that supports out-of-the-box performance across diverse downstream tasks.

Experimentally, Zeusachieves the consistent state-of-the-art performance in a tuning-free manner across widely recognized benchmarks for five downstream tasks (Figure 1), including long time series benchmark [17] for point forecasting and imputation, GIFT-Eval [18] for probabilistic forecasting, UCR anomaly archive [19] for anomaly detection, and UEA archive for classification. These results demonstrate the strong generalizability of Zeusacross diverse tasks. In summary, our contribution is three-fold:

• We present Zeus, a unified multi-scale Transformer that preserves point-level fidelity while efficiently modeling high-level semantics over long sequences.

• We introduce MOTM, a multi-objective temporal masking strategy that aligns diverse task-specific inductive biases within a single pretraining framework.

• To the best of our knowledge, Zeusis the first TSFM to achieve competitive performance across five downstream tasks without task-specific adaptation, as validated on well-established benchmarks.

2 Related Work↩︎

2.1 Task-Specific Time Series Model↩︎

Time series analysis plays a fundamental role in a wide range of applications, including forecasting, imputation, anomaly detection, and classification. Early statistical and traditional machine learning methods typically rely on case-by-case fitting, facing limitations when applied to large-scale data. With the advancement of deep learning, task-specific neural models have become the dominant paradigm [20], [21]. CNN-based models, such as TS2Vec [22], TimesNet [17] and ModernTCN [23], have shown strong potential to support multiple tasks within a unified framework. Transformer variants [14], [24]–[26] further enhance the ability to capture long-range dependencies and complex temporal patterns. Inspired by advances in LLMs, recent work has explored transferring pretrained language models to time series tasks [27]–[29]. Typically, GPT4TS [28] explores the use of pretrained GPT-2 as a frozen backbone for time series analysis, adapting it to downstream tasks through lightweight output projections. However, these methods still rely on task-specific training for each downstream application.

2.2 Time Series Foundation Models↩︎

Building upon task-specific precursors, recent research has increasingly explored pretraining foundation models on large-scale time series corpora to enhance cross-task generalization, with Transformer [30] becoming the predominant architecture. Representative approaches such as MOMENT [8] and TimesBERT [16] adopt BERT-style masked reconstruction objectives to learn contextualized representations by recovering masked patches. UniTS [10] extends this paradigm by incorporating task prompts, facilitating flexible task adaptation. In contrast, Timer [9] adopts a GPT-style autoregressive formulation to cast diverse time series tasks into a unified generative framework. Despite their progress, existing TSFMs remain constrained by architectural and training-design limitations, which hinder fine-grained modeling and necessitate task-specific adaptation in practice.

3 Zeus↩︎

3.1 Overall Framework↩︎

Zeusis a multi-scale encoder-only Transformer architecture for general-purpose time series analysis. As illustrated in Figure 2, it adopts point-wise tokenization to preserve fine-grained temporal resolution, and processes the resulting tokens through a symmetric downsampling–upsampling hierarchy of Transformer encoders. Representations are progressively aggregated from fine to coarse scales and then refined back to fine resolution. This design resolves the dilemma between granularity and scalability: it preserves point-level details required by reconstruction-oriented tasks while promoting semantically compact and computationally efficient representations for large-scale pretraining.

3.2 Tokenization↩︎

Zeusadopts a point-wise tokenization scheme, where each time step is treated as an individual token. We conduct pretraining in a univariate paradigm and employ the channel-independent strategy to handle multivariate time series [14]. Given a time series \(\mathbf{x}=\{x_1, x_2,\cdots, x_T\}\), we first apply instance normalization [31] to remove scale variations. Then each time step is mapped to a hidden representation via a gated embedding layer: \[\mathbf{h}_t^{(0)} = \mathbf{W}_r x_t + \mathbf{W}_d \big( \sigma(\mathbf{W}_g x_t) \odot \mathbf{W}_u x_t \big),\] where \(\sigma(\cdot)\) denotes a nonlinear activation. This design increases the expressive capacity of token embeddings.

We further introduce two learnable tokens: a [MASK] token for masked reconstruction and a [PAD] token for variable-length sequences and multi-scale alignment, enabling unified handling of masking, missing values, and padding. As shown in Figure 2, by replacing temporal tokens with the [MASK] token, diverse downstream tasks can be uniformly formulated under a masked modeling paradigm, allowing Zeusto support multiple tasks within a unified framework. In forecasting tasks, several [MASK] tokens are appended after the historical window; in imputation tasks, missing values are replaced with [MASK]; in anomaly detection tasks, the target segment can be replaced with [MASK], and reconstruction or prediction errors are used as anomaly scores.

3.3 Multi-Scale Architecture↩︎

While point-wise tokenization preserves temporal fidelity, it introduces substantial computational challenges for long sequences. Zeusresolves this tension through a U-shaped hierarchy that progressively adjusts temporal resolution across symmetric scales \(\{s_1,s_2,\cdots,s_K\}\), where \(s_i=s_{K-i+1}\) defines the number of time steps aggregated into a token.

3.3.0.1 Downsampling and Upsampling Stages

This U-shaped architecture consists of a downsampling stage to compress fine-grained information into high-level semantics via pooling operation, and a symmetric upsampling stage, which employs unpooling to progressively restore local details.

In the downsampling stage, given the hidden states \(\mathbf{h}^{(i)}\in\mathbb{R}^{T_i\times d_i}\) at scale \(i< \frac{K-1}{2}\), a pooling layer aggregates \(r=s_{i+1}/s_{i}\) adjacent tokens into a single representation. The pooled representations are then processed by a Transformer encoder to model higher-level temporal dependencies. \[\begin{gather} \mathbf{p}^{(i)}=\mathrm{Reshape}(\mathbf{h}^{(i)},\mathbb{R}^{\frac{L_i}{r}\times (rd_i)})\mathbf{W}_p,\\ \mathbf{h}^{(i+1)}=\mathrm{TrfmEncoder}(\mathbf{p}^{(i)}), \end{gather}\]

where \(\mathbf{W}_p\in \mathbb{R}^{(rd_i)\times d_{i+1}}\) is a learnable linear projection.

Conversely, in the upsampling stages (\(i\geq \frac{K-1}{2}\)), an unpooling layer expands the sequence and integrates high-resolution features from the corresponding scale via residual skip connections:

\[\mathbf{P}^{(i)}=\mathrm{Reshape}(\mathbf{h}^{(i)}\mathbf{W}_u,\mathbb{R}^{(rL_{i})\times d_{i+1}})+\mathbf{h}^{(K-i+1)}.\]

The resulting representations are then fed into a Transformer encoder to refine fine-grained temporal details. In this architecture, lightweight Transformer blocks operate at fine scales to capture local patterns, while deeper and wider blocks process coarser representations to model long-range dependencies. This multiscale design enables Zeusto efficiently balance temporal fidelity and scalability.

3.3.0.2 Transformer Block

Each Transformer block employs multi-head self-attention with rotary positional embeddings [32], together with gated feed-forward networks [33]. To enhance training stability, we adopt RMSNorm [34] and a pre-LN scheme [35]. All attention modules are implemented with FlashAttention v2 [36], enabling memory-efficient and scalable training on long sequences.

3.3.0.3 Quantile Head

Zeusemploys a quantile head that provides \(|Q|\) quantile values for each time step, enabling probabilistic reconstruction instead of point estimation. Formally, the head implements a mapping \(\mathbb{R}^{d_K}\rightarrow \mathbb{R}^{|Q|}\). In our implementation, Zeuspredicts a set of nine quantile levels \(Q= \{0.1,0.2,...,0.9\}\). For classification task, we directly use the globally pooled representations.

3.4 Multi-Objective Temporal Masking↩︎

The key challenge for TSFMs to generalize across tasks lies in the distinct inductive biases required by different objectives. Forecasting requires extrapolation, whereas imputation relies on interpolation. Anomaly detection and classification introduce more complex demands: point anomalies require sensitivity to local variations, while contextual anomalies necessitate modeling global consistency [37]. Moreover, classification calls for both global abstraction and the identification of local shapelets [38]. However, current TSFMs typically employs a simple BERT-style [8], [16] or GPT-style objective [9], which fails to simultaneously capture these heterogeneous inductive biases.

To bridge this gap, we introduce multi-objective temporal masking (MOTM), a unified masking-based training strategy that equips Zeuswith heterogeneous inductive biases. Figure 3 illustrates the pipeline of MOTM. For each instance, the overall corruption ratio is sampled from \(\mathcal{U}(0, 0.5)\) to enhance robustness to varying levels of missing. We then select a temporal scope that jointly cover short-, mid-, and long-term dependencies. Finally, a masking scheme is sampled from a diverse set, including predictive, point, multi-block, and single-block masking, as well as their combinations. Below, we detail each masking strategy and discuss its role in shaping task-relevant inductive biases.

3.4.0.1 Predictive Mask

To bolster extrapolative capacity essential for forecasting, we adopt a predictive masking scheme that masks a suffix of the sequence. Specifically, given a probability \(p\), we mask the last \(L_p=\lfloor Tp\rfloor\) time steps, requiring the model to learn temporal causality and capture long-range dependencies beyond the observed horizon.

3.4.0.2 Point Mask

This widely-used strategy targets point-wise interpolation by randomly masking individual time steps. This fine-grained corruption forces the model to leverage local continuity and short-range correlations, while serving as a regularizer against overfitting to specific patterns.

3.4.0.3 Multi-Block Mask

To simulate the contiguous missingness prevalent in real-world deployments, we introduce a multi-block masking strategy. Inspired by span-based corruption used in language modeling [39], [40], this method shifts the objective from point-wise reconstruction to structured recovery. However, instead of adopting the Poisson distribution commonly used in language modeling, we employ a uniform distribution, which poses a more challenging setting for time series. Specifically, given a masking budget \(L_p=\lfloor Tp \rfloor\), we sample block lengths \(\{\ell_k\}\) from \(\ell_k \sim \mathcal{U}(1,24)\), until \(\sum_k \ell_k \approx L_p\), and randomly distribute the blocks along the sequence. This strategy compels the model to perform interpolation under structured missingness rather than simple local smoothing.

3.4.0.4 Single-Block Mask

Complementary to multi-block masking, we introduce a single-block masking strategy that removes one long contiguous segment from an arbitrary position in the sequence. This strategy encourages the model to maintain global consistency, which is crucial for classification and contextual anomaly detection.

3.4.0.5 Mixed Mask

We further employ a mixed masking scheme that combines multiple masking strategies to increase the difficulty of the pretraining objective. In practice, simpler masks (multi-block and point mask) are paired with harder ones (predictive and single-block mask), ensuring that mixed masking remains challenging yet learnable.

3.4.0.6 Training Objective

Under all masking strategies, Zeusis trained in a masked reconstruction manner with the quantile loss. Given a binary mask \(\mathcal{M}\in\{0, 1\}^{T}\), the loss is computed only on masked positions:

\[{!}{\mathcal{L}=\frac{1}{|Q||\mathcal{M}|}\sum_{t:\mathcal{M}_t=1} \sum_{q\in Q} \begin{cases} q(y_t-\hat{y}_t^q), & \hat{y}_t^q \leq y_t, \\ (1-q)(\hat{y}_t^q-y_t), & \hat{y}_t^q > y_t. \end{cases}}\]

Here, \(\hat{y}_t^q\) denotes the prediction at quantile level \(q\).

3.5 Pretraining Data↩︎

We train Zeuson a large-scale corpus of real and synthetic time series, comprising about 300B observations collected from diverse domains and frequency. The real-world data are mainly sourced from the Chronos datasets [15] and the GiftEvalPretrain datasets [18]. To enhance pattern diversity beyond real-world records, we additionally construct Aegis-Syn, a synthetic dataset that extends KernelSynth [15] with richer temporal structures, particularly non-smooth and discontinuous patterns that are underrepresented by Gaussian process–based generators. To mitigate pattern imbalance in the pretraining corpus, we adopt a balanced sampling strategy [41]. All evaluation datasets are excluded from pretraining to prevent data leakage. Detailed descriptions of the pretraining data and Aegis-Syn are provided in Appendix 10.

4 Experiments↩︎

Based on the above approach, we implemented and trained Zeuswith five scales and 12 Transformer layers, comprising approximately 100M parameters in total. Details are provided in Appendix 6. In this section, we conduct a comprehensive evaluation on well-established benchmarks to assess Zeus’s performance across five downstream tasks: point forecasting (§4.1), probabilistic forecasting (§4.2), imputation (§4.3), anomaly detection (§4.4), and classification (§4.5). See Appendix 9 for detailed experimental settings and baseline descriptions.

5.1pt

4.1 Point Forecasting↩︎

4.1.0.1 Setups

We adopt six widely used long-term forecasting benchmarks [17], [49] to evaluate Zeus’s zero-shot performance in point forecasting task. We consider four different prediction horizons \(\{96,192,336,720\}\), where the best context length is chosen through hyperparameter search. Performance is evaluated using MSE and MAE.

4.1.0.2 Results

As shown in Table 1, Zeusdemonstrates strong overall performance among advanced models. Compared with the previous state-of-the-art model, Zeusachieves an averaged reduction of 9.0% on MSE and 2.3% on MAE. Moreover, Zeusovercomes the common limitation where TSFMs lag behind task-specific models in zero-shot forecasting. When compared to the best-performing TSFM under the zero-shot settings, Timer [9], Zeusachieves substantial improvements, reducing MSE by 40.3% and MAE by 25.3%.

4.2 Probabilistic Forecasting↩︎

4.2.0.1 Setups

Beyond point forecasting, we evaluate our model on the GIFT-Eval benchmark [18] for probabilistic forecasting, which comprises 97 prediction tasks spanning short-, medium-, and long-term forecasting, providing a comprehensive assessment of predictive performance. Following the official protocol, we report MASE and CRPS as evaluation metrics.

4.2.0.2 Results

Table 1 reports the aggregated performance across 97 tasks on GIFT-Eval. Among the advanced pretrained forecasting models, including Chronos-2 [42], TimesFM2.5 [43], and TiRex [13], Zeusranks first on both MASE and CRPS, highlighting its strong zero-shot forecasting capabilities.

4.3 Imputation↩︎

4.3.0.1 Setups

Time series imputation aims to fill in missing values of time series and is critical in real-world applications. Following previous work [9], [16], we evaluate Zeuson the ETT, ECL and Weather datasets by masking \(\{12.5\%,25\%,37.5\%,50\%\}\) time steps in sequences of length 192. Similar to point forecasting tasks, We use MSE and MAE as evaluation metrics.

Existing TSFMs [8], [9], [16] are typically evaluated under patch-wise missing patterns. However, real-world time series exhibit more diverse missing behaviors, including both point-wise missing and contiguous segment missing with variable lengths [50]. To better reflect practical scenarios, we adopt two masking strategies: (1) random masking, which has been widely adopted in prior task-specific models [17], [23], and (2) block masking, where the lengths of contiguous missing segments are sampled from a Geometry distribution with \(p=0.125\).

4.3.0.2 Results

As shown in Table ¿tbl:tab:imputation?, Zeusconsistently achieves superior performance across all datasets and in zero-shot, demonstrating strong robustness to variable-length missing patterns. Patch-based TSFMs (MOMENT, UniTS, and Timer) perform substantially worse than task-specific models, mainly due to the mismatch between their patch-wise pretraining objective and point-level missing patterns at inference, which results in an out-of-distribution scenario. In contrast, Zeuseffectively handles both random and block missing scenarios. Even compared with the strongest task-specific models, Zeusreduces the averaged MSE by 24.4% under random masking and 18.8% under block masking.

4.4 Anomaly Detection↩︎

4.4.0.1 Setups

To provide a comprehensive evaluation, we conduct experiments on a collection of 42 datasets sourced from the UCR anomaly archive [19]. This subset that has been adopted in previous studies [8], covering a sufficiently broad range of domains and data sources. We partition the time series into non-overlapping detection windows [51] and use the reconstruction error as the criterion for anomaly detection [17]. The window size is selected via hyperparameter search and the results of all baselines are reported under the optimal settings. We use adjusted F1-score as the evaluation metric [17].

4.4.0.2 Results

Figure 4 presents the overall performance of anomaly detection tasks. Zeusranks first among all baselines across 42 UCR datasets. Under the zero-shot setting, Zeusoutperforms advanced task-specific models trained in full-shot. Moreover, Zeusachieves a 21.0% improvement in F1-score over UniTS [10], the second-best performing TSFM on this benchmark.

4.5 Classification↩︎

4.5.0.1 Setups

We conduct evaluations on 26 UEA classification datasets [52]. Prior TSFMs relies on fine-tuning [10], [16] or linear probing [8] for classification tasks. In our evaluation, Zeusis primarily assessed in a tuning-free setting using a non-parametric 1-nearest neighbor (1-NN) classifier for evaluation, with optional PCA whitening applied for feature normalization. In addition, we also report linear probing results to assess the linear separability of the learned representations and to facilitate comparison with existing methods. Following prior work [8], we use the accuracy as the evaluation metric.

4.5.0.2 Results

As shown in Figure 5, Zeusachieves superior performance across the 26 UEA classification datasets under different evaluation protocols. In particular, Zeuswith linear probing attains the highest averaged accuracy among all compared methods, outperforming both task-specific models and other pretrained baselines. Notably, Zeuswith 1-NN evaluation also delivers competitive performance, being on par with advanced models such as TimesNet and MOMENT-LP, while outperforming MOMENT under the same 1-NN setting by a clear margin of \(7.0\) percentage points. Overall, these results highlight the effectiveness and generalizability of Zeus’s pretrained representations.

4.6 Ablation Study↩︎

In this section, we investigate the effectiveness of our proposed MOTM by ablating individual masking strategies. When a specific mask is removed, we keep the expected masking ratio fixed by proportionally reallocating it to the other masks. As illustrated in Figure 6 (a), removing the predictive mask leads to a clear performance drop on the GiftEval benchmark, validating its crucial role in enhancing extrapolation capabilities and forecasting precision. Similarly, Figure 6 (b) shows the absence of multi-block mask degrades Zeus’s performance on imputation tasks, indicating its importance for interpolation and local pattern reconstruction. Furthermore, the removal of the single-block mask leads to consistent performance degradation in both anomaly detection and classification, as shown in Figure 6 (c), suggesting that single-block reconstruction guides the model to capture global consistency.

4.7 Model Analysis↩︎

4.7.0.1 Representation Analysis

We conduct a representation analysis to illustrate the roles of different temporal scales in Zeus. Specifically, we select a sample sequence from the ETTm1 dataset and visualize the feature-norm heatmaps of the representations at multiple scales, as shown in Figure 7.

It can be observed that fine-scale representations are highly sensitive to local variations and extreme values. The regions highlighted by red boxes indicate that these representations effectively capture each negative spike in the sequence. In contrast, mid-scale representations are less responsive to local fluctuations; instead, their stripe-like patterns reflect the underlying periodic structures. Coarse-scale representations clearly characterize global pattern changes in the sequence. For instance, the white vertical line marks the transition from small-amplitude fluctuations to periodic negative impulses, which is reflected by a color change in the heatmap from purple to yellow. Moreover, the regions highlighted by yellow boxes demonstrate that large-scale representations are effective in capturing contextual anomalies.

4.7.0.2 Efficiency Analysis

Conventional point-tokenized Transformers suffers from the high computational cost, as processing a sequence of length \(L\) with dimension \(d\) over \(N\) layers incurs \(\mathcal{O}(N L^2 d)\) computations due to full-resolution self-attention. In contrast, Zeusperforms most attention computations at coarser temporal resolutions, resulting in a reduced complexity of \[\mathcal{C}_{\mathrm{\small Zeus}} = \sum_i \mathcal{O}\!\left(N_i \left(\frac{L}{s_i}\right)^2 d_i\right), \label{eq:efficiency}\tag{1}\]

where \(N_i\) and \(d_i\) denote the number of layers and the hidden dimension at scale \(s_i\). Under our model configuration (Appendix 6), Zeusachieves a \(3.8\times\) reduction in self-attention FLOPs compared to a vanilla Transformer with the same depth (Appendix 8.2). Additionally, we conduct an empirical efficiency comparison between Zeusand \(\text{Time-MoE}_\text{base}\) [11], a point-tokenized pretrained model with approximately 113M parameters. Both models are evaluated under the same environment with FlashAttention enabled, ensuring a fair comparison. As shown in Figure 8, Zeusis \(2.1\times\) faster and \(3.1\times\) memory efficient than \(\text{Time-MoE}_\text{base}\), demonstrating its scalability over long context.

5 Conclusion↩︎

In this work, we propose Zeus, a unified multi-scale Transformer with point-wise tokenization and a U-shaped hierarchy that effectively balances fine-grained temporal fidelity with long-sequence efficiency. Complementing the architectural design, we introduce MOTM, a multi-objective temporal masking strategy that jointly supports extrapolation, interpolation, and global abstraction within a single pretraining framework. Moreover, we construct Aegis-Syn, a synthetic dataset that extends KernelSynth with richer temporal patterns. Extensive experiments across five representative downstream tasks demonstrate that Zeusachieves strong and consistent performance in a fully tuning-free setting, highlighting its potential as a truly general-purpose TSFM. We believe this work takes a meaningful step toward versatile and scalable foundation models for time series analysis. Future work will explore multivariate modeling and extend the proposed framework to a broader range of downstream tasks. See Appendix 11 for detailed discussion on limitations and future directions.

Acknowledgments↩︎

This work is supported by the NSFC underGrant Nos.62372430 and 62502505, the Youth Innovation Promotion Association CAS No.2023112, the Postdoctoral Fellowship Program of CPSF under Grant Number GZC20251078, the China Postdoctoral Science Foundation No.2025M77154 and HUA Innovation fundings. We sincerely thank all the anonymous reviewers who gerenously contributed their time and efforts.

Impact Statement↩︎

This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

6 Implementation Details↩︎

6.1 Model Configuration↩︎

Zeushas approximately 100M parameters, and the detailed model configurations are summarized in Table 2. We set the maximum context length to 4096. By using FlashAttention [36] and preventing padded tokens from participating in attention, we ensure that the additional padding does not incur significant computational overhead. The model are trained for 200,000 steps with a global batch size of 512. We employ the AdamW optimizer [53] with a cosine learning rate schedule and a warmup phase of 10,000 steps. The initial learning rate is set to \(1\times10^{-3}\). All experiments are implemented using PyTorch [54] and trained on 4 NVIDIA H100 GPUs.

6.2 Unified Downstream Task Formulation↩︎

This section describes how Zeusis applied to various downstream tasks.

6.2.0.1 Forecasting

Given a historical time series context \(\mathbf{x} \in \mathbb{R}^{T \times C}\), where \(T\) denotes the context length and \(C\) the number of variables, Zeusperforms forecasting by formulating the task as a masked sequence completion problem. Zeusadopt channel independent strategy that treat \(\mathbf{x}\) as \(C\) univariate time series.

To perform forecasting on a horizon \(H\), the context is concatenated with \(H\) [MASK] tokens, i.e., \(\tilde{\mathbf{x}}=[\mathbf{x}, \mathbf{M}]\in \mathbb{R}^{(T+H)\times C}\), and then fed to Zeus. The output of Zeusis a quantile reconstruction tensor \(\mathbf{z}\in\mathbb{R}^{(T+H)\times C \times |Q|}\), where \(|Q|\) denotes the number of quantile levels. We retain the predictions at the final \(H\) steps as the probabilistic forecast \(\mathbf{y}\in\mathbb{R}^{H\times C \times |Q|}\), and the point forecast is obtained by averaging predictions across all quantiles.

6.2.0.2 Imputation

For the imputation task, Zeusfollows the same reconstruction paradigm as used during pretraining. Missing values are replaced with [MASK] tokens and fed into Zeus, which produces quantile predictions for all time steps. The outputs at masked positions are used as imputed values, with point estimates obtained by averaging across quantiles.

6.2.0.3 Anomaly Detection

Zeussupports anomaly detection under two complementary paradigms: prediction-based and reconstruction-based. The prediction-based approach detects anomalies by forecasting future values and measuring prediction errors, which is well suited for streaming scenarios where only past observations are available. In contrast, the reconstruction-based approach masks a target segment and reconstructs it using both past and future context, enabling more accurate detection by exploiting full contextual information. In this work, we provide a formal description of the reconstruction-based formulation, while the prediction-based variant follows an analogous procedure.

Given a historical time series \(\mathbf{X}_{1:L}\in\mathbb{R}^{L\times C}\) and a target window size \(T<L\), we partition the sequence into non-overlapping subsequences using a sliding window with stride \(T\). For each target segment \(\mathbf{X}_{t:t+T}\), we construct the input by masking the target window and concatenating it with its surrounding context, forming \(\tilde{\mathbf{X}} = [\mathbf{X}_{t-W:t},\;\mathbf{M},\;\mathbf{X}_{t+T:t+T+W}]\), where \(W\) denotes the context length and \(\mathbf{M}\) denotes \(T\) consecutive [MASK] tokens. The resulting sequence is then fed into Zeusto reconstruct the masked segment.

The model outputs quantile predictions \(\mathbf{z}\in\mathbb{R}^{T\times C\times |Q|}\) for the target window. An anomaly score is computed based on the reconstruction error, with higher errors indicating a higher likelihood of anomalies. By default, we use MAE as the reconstruction error for anomaly scoring. However, for sequences containing impulsive patterns, we selectively adopt relative MAE as the error metric, which normalizes the absolute error by the signal magnitude. This choice prevents large but expected impulses from being erroneously identified as anomalies.

6.2.0.4 Classification

Zeusperforms classification using a 1-nearest neighbor (1-NN) classifier on the learned representations. Specifically, given the train dataset \(\mathcal{D}_{train}\) and a sample, Zeuspredict label \(\hat{y}\) with its hidden states \(\mathbf{h} \in \mathbb{R}^{T\times C\times d}\) by

\[\begin{gather} \mathbf{z} = \mathrm{Flatten}\left(\mathrm{GlobalPool}(\mathbf{h})\right),\\ i^\ast = \arg\max_{i \in \mathcal{D}_{\text{train}}} \; \text{sim}(\mathbf{z}, \mathbf{z}_i),\\ \hat{y} = y_{i^\ast}, \end{gather}\]

where \(\mathrm{GlobalPool}(\cdot)\) aggregates the temporal dimension, for which we default to max pooling, \(\mathrm{Flatten}(\cdot)\) flattens the channel dimension, and \(\mathrm{sim}(\cdot,\cdot)\) denotes the cosine similarity. \(\mathbf{h}\) can be taken from a specific scale or formed by concatenating representations from multiple scales. In practice, we typically use the representation from the penultimate scale (\(s_4=8\)) or the coarsest scale (\(s_3=32\)).

As an alternative to 1-NN classification, we also consider a linear probe applied on top of \(\mathbf{z}\), where the backbone of Zeusis frozen during training.

6.3 Technical Details of MOTM↩︎

This section provides the technical details of MOTM that is used to pretrain Zeus.

6.3.0.1 Masking Ratio

The overall masking ratio is sampled from a uniform distribution, i.e., \(p \sim \mathcal{U}(0, 0.5)\), with the expected rate being \(0.25\). A low masking ratio is used to simulate short-term forecasting scenarios, where the model need to accurately infer future dynamics from long historical context. In contrast, a high masking ratio poses a more challenging setting and is used to emulate imputation scenarios with severe missingness. Compared to a fixed masking ratio that is typically adopted by prior work [8], [16], a variable masking ratio prevents the model from overfitting to a single missingness level and improves robustness across diverse missing rates.

6.3.0.2 Temporal Range

To increase the diversity of contexts observed by the model, we sample variable sequence lengths and randomly crop the input sequences, padding them to a maximum context length of 4096. Specifically, the sequence length is sampled from a piecewise uniform distribution: with probability 0.2 from \([64, 512]\), 0.2 from \([513, 2048]\), and 0.6 from \([2049, 4096]\). This design ensures that short- and mid-range temporal dependencies are sufficiently observed, while most sequences fall into the long-range regime and require minimal padding, thereby balancing training efficiency.

7 Related Work↩︎

7.1 Systematic Comparison with Closely Related Work↩︎

In this section, we present a systematic comparison with closely related TSFMs. In this work, we use the term time series foundation models to denote pretrained models that are capable of supporting a wide range of downstream time series analysis tasks, in contrast to pretrained forecasting models that are designed exclusively for forecasting. In addition, some LLM-based approaches, such as GPT4TS [28], do not involve pretraining on time series data. Existing studies further indicate that the performance gains do not primarily stem from the LLM backbone [55]. Therefore, we classify these methods as task-specific models. While state-of-the-art pretrained forecasting models are included in the forecasting evaluations, this section focuses on comparisons among TSFMs.

Table 3 summarizes the key differences between Zeusand representative TSFMs, including TimesBERT [16], MOMENT [8], UniTS [10], and Timer [9], in terms of model architecture, pretraining scale, tokenization strategy, and downstream task support. Existing TSFMs predominantly adopt patch-based tokenization, whereas Zeusemploys point-wise tokenization to preserve fine-grained details. In addition, Zeusis pretrained at a larger scale and with a substantially longer context length than prior TSFMs. As TimesBERT is not publicly available, it is not included as a baseline in our experiments.

Beyond these differences, a more salient distinction lies in downstream task coverage. Zeussupports a substantially broader set of time series analysis tasks than existing TSFMs, enabling forecasting, imputation, anomaly detection, and classification in a tuning-free manner. In contrast, existing TSFMs typically support only a subset of downstream tasks or rely on task-specific fine-tuning or adaptations. Overall, this comparison positions Zeusas a more general-purpose foundation model with comprehensive and unified multi-task capabilities.

7.2 Multi-Scale Architectures↩︎

Multi-scale modeling has been widely explored in task-specific models [56]–[60], where hierarchical representations are adopted to capture temporal dependencies at different resolutions. These studies have demonstrated the effectiveness of multi-scale designs for improving model performance. However, despite their success in task-specific settings, most existing TSFMs still rely on single-scale architectures. Only a few exceptions explored multi-scale structures, and Zeusdiffers from these works fundamentally in both information flow and architectural design. For instance, TTM [61] and Xihe [44] follows a fine-to-coarse paradigm, progressively aggregating local patterns, while Kairos [46] adopts a coarse-to-fine mixture-of-size patching strategy. In contrast, Zeusis the first to introduce a fine-to-coarse-to-fine multi-scale architecture, which preserves point-level details while simultaneously enhancing long-sequence scalability.

7.3 Time Series Tokenization↩︎

Time series tokenization has been extensively studied in recent years. Beyond the widely adopted point tokens [62], [63] and patch tokens [14], [64], prior works have also explored alternative tokenization paradigms such as variable-wise [26], [65], frequency-wise [66], phase-wise tokenization [67], and various improvements to patch-wise tokenization [68], [69]. However, in the context of TSFMs, the design space of tokenization is constrained by the requirement to support arbitrary input lengths. As a result, many existing tokenization improvements are difficult to directly apply to TSFMs. In addition, point-wise tokenization is less favored due to its limited scalability under long-context settings. More recently, tokenization strategies specifically designed for TSFMs have been proposed, including domain-specific tokenization [70] and wavelet-based tokenization [71]. Nevertheless, patch-wise tokenization remains the dominant paradigm in current TSFM architectures.

8 Supplementary Analysis↩︎

8.1 Limitations of Patch Tokenization↩︎

In this section, we further analyze the drawbacks of patch tokenization and provide empirical evidence to substantiate our claims.

Most existing pretrained time series models adopt patch tokenization. Although patching effectively increases the semantic density of tokens and reduces the total number of tokens [14], it also introduces several nontrivial limitations.

First, patching inherently entangles fine-grained temporal variations within a patch, which weakens the model’s ability to reason at the point level. This mismatch becomes particularly evident in tasks that require precise temporal localization, such as point-wise imputation and anomaly detection. Moreover, models pretrained with patch-level reconstruction objectives tend to overfit to patch-wise missing patterns, leading to degraded robustness when the corruption pattern shifts to point-level missingness. We provide empirical evidence for these limitations in Table 4. Notably, although point-wise missing constitutes a strictly simpler corruption pattern than patch-wise missing, the performance of MOMENT drops by over 20% when evaluated under point-missing imputation. Last but not least, patch tokenization struggles to capture high-frequency dynamics. Consider a sequence whose period equals the patch length: all patch tokens become identical, causing the Transformer to degenerate into a feed-forward MLP.

8.2 Detailed Efficiency Analysis↩︎

This section provides a detailed comparison of the computational complexity between Zeusand a vanilla Transformer.

8.2.0.1 Baseline: Vanilla Transformer

We consider a vanilla Transformer with point-wise tokenization and the same total number of layers as Zeus. For a sequence of length \(L\) and hidden dimension \(d\), the self-attention module incurs a computational cost of \(\mathcal{O}(BL^2d)\) per layer, which dominates the overall FLOPs. With \(N\) layers, the total attention complexity is \[\mathcal{C}_{\text{vanilla}} = \mathcal{O}(BNL^2d).\]

8.2.0.2 Zeus

Zeusemploys a multi-scale architecture with temporal scales \(\{s_i\}\). The attention complexity at scale \(s_i\) is computed as Eq.(1 ).

8.2.0.3 Quantitative Comparison

Using the configuration in our implementation, most layers in Zeusoperate on temporally downsampled representations (e.g., \(L/8\) and \(L/32\)), while only a small number of layers attend at full resolution. Let \(d=384\) and \(N=12\), then we obtain

\[\begin{gather} \mathcal{C}_{\mathrm{\small Zeus}} = B \cdot L^2 \Big( 3 \cdot 384 + \frac{6}{8^2} \cdot 768 + \frac{3}{32^2} \cdot 768 \Big),\\ \mathcal{C}_{\text{vanilla}} = B \cdot L^2 \cdot 12 \cdot 384,\\ \mathcal{C}_{\mathrm{\small Zeus}} \approx 0.27 \, \mathcal{C}_{\text{vanilla}}, \end{gather}\] which indicates that Zeusachieves an approximately \(3.8\times\) reduction in self-attention FLOPs compared to a vanilla Transformer with the same depth.

9 Experimental Details↩︎

9.1 Point Forecasting↩︎

9.1.0.1 Benchmarks

We conduct experiments on six widely-recognized datasets in long-term forecasting benchmarks [17], [49], including:

• ETT (Electricity Transformer Temperature) [62] contains 7 features of electricity transformer data collected from two separate counties from July 2016 to July 2018. It contains four datasets: ETTh1, ETTh2, ETTm1, ETTm2, where ETTh1 and ETTh2 are recorded every hour, and ETTm1 and ETTm2 are recorded every 15 minutes.

• ECL (Electricity) [24] records the hourly electricity consumption data of 321 clients from 2012 to 2014. Each variable represents a client’s electricity consumption.

• Weather[24] includes 21 meteorological factors collected every 10 minutes from the weather station of the Max Planck Biogeochemistry Institute in 2020.

Following common practice, we consider four prediction horizons \(\{96, 192, 336, 720\}\). For each horizon, the context length is selected via hyperparameter search from the set \(\{512, 720, 1024, 2048, 3072\}\).

9.1.0.2 Baselines

We compare our methods with three categories of baselines: (1) TSFMs: MOMENT [8], Timer [9], UniTS [10]; (2) Pretrained forecasting models: Kairos [46], Toto [12], Sundial [48]; (3) Task-specific models: ModernTCN [23], GPT4TS [28], TimesNet [17], PatchTST [14].

For baselines that include multiple model variants, we report results for the variant with the strongest overall performance (typically the largest ones). Subscripts \(_b\) and \(_l\) denote the base and large variants, respectively. Notably, the prediction head of MOMENT is randomly initialized and therefore requires fine-tuning before being applicable to forecasting, making it unsuitable for zero-shot evaluation. Consequently, we use its reconstruction head instead and formulate the forecasting task as an extrapolative reconstruction task, following the same setup used for Zeus.

9.1.0.3 Metrics

We adopt Mean Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics, where lower values indicate better performance. They are defined as follows.

\[\begin{gather} \text{MSE}(\hat{y}, y)=\frac{1}{N\cdot T}\sum_{i=1}^N\sum_{t=1}^{T} \left(\hat{y}^{i}_{t}-y^{i}_{t}\right)^2, \\ \text{MAE}(\hat{y}, y)=\frac{1}{N\cdot T}\sum_{i=1}^N\sum_{t=1}^{T} |\hat{y}^{i}_{t}-y^{i}_{t}|. \end{gather}\]

9.1.0.4 Full Results

The full results of point forecasting evaluation are provided in Table ¿tbl:tab:full95forecasting?.

9.2 Probabilistic Forecasting↩︎

9.2.0.1 Benchmarks

For probabilistic forecasting, we evaluate Zeus’s performance on the GIFT-Eval benchmark [18]. GIFT-Eval comprises 23 datasets spanning multiple domains, including nature, energy, healthcare, finance, transportation, and cloud operations, covering 144,000 time series with 177 million data points. These datasets define a total of 97 forecasting configurations, including 55 short-term, 21 medium-term, and 21 long-term forecasting tasks, thereby providing a comprehensive assessment of the model’s predictive capabilities. Compared to long-term forecasting benchmarks used in point forecasting evaluation, GIFT-Eval places greater emphasis on short-term forecasting and additionally provides evaluation metrics for probabilistic predictions.

9.2.0.2 Baselines

We include baselines: Chronos-2 [42], TimesFM2.5 [43], TiRex [13], \(\boldsymbol{Xihe}_\textit{ultra}\) [44], FlowState [45], Kairos [46], Moirai2 [47], Toto [12], Sundial [48]. The performance of baselines is taken from the official benchmark reports, using results available up to December 31, 2025.

9.2.0.3 Metrics

Following the official protocol of GIFT-Eval, we report Mean Absolute Scaled Error (MASE) and Continuous Ranked Probability Score (CRPS) as evaluation metrics. Both metrics are lower-is-better.

MASE [72] evaluates point forecasting performance by scaling the mean absolute error (MAE) of a model against that of a seasonal naive benchmark. It is defined as \[\label{eq:mase} \text{MASE} = \frac{\text{MAE}_{\text{model}}}{\text{MAE}_{\text{seasonal naive}}}\tag{2}\]

with the Seasonal Naive MAE being defined as

\[\label{eq:insample95mae} \text{MAE}_{\text{seasonal naive}} = \frac{1}{T-m}\sum_{t=m+1}^{T} |y_t - y_{t-m}|\tag{3}\]

where \(T\) is the length of the training split of the series, \(y_t\) is the value of the series at time \(t\), and \(m\) is the seasonal period.

CRPS [73] is a scoring rule for probabilistic forecasting. Given a predicted distribution with CDF \(F\) and a ground truth value \(y\), the CRPS is defined as:

\[\text{CRPS}(F, y) = \int_{0}^{1} 2 \Lambda_{\alpha}( F^{-1}(\alpha), y ) \, d\alpha,\]

where \(\Lambda_{\alpha}( q, y )\) denotes the quantile loss.

In practice, CRPS is approximated by a discrete sum over a finite set of quantile levels. This approximation, often referred to as the mean weighted quantile loss [74], is given by:

\[\text{CRPS} \approx \frac{1}{K} \sum_{k=1}^{K} \text{wQL}[\alpha_k],\]

where \(K\) is the number of quantile levels, and \(\{ \alpha_1, \alpha_2, \dots, \alpha_K \}\) are the selected quantile levels. The weighted quantile loss \(\text{wQL}[\alpha]\) for each quantile level \(\alpha\) is calculated as:

\[\text{wQL}[\alpha] = 2 \frac{ \sum_{t} \Lambda_{\alpha}( \hat{q}_t(\alpha), y_t ) }{ \sum_{t} | y_t | },\]

where \(\hat{q}_t(\alpha)\) is the predicted \(\alpha\)-quantile at time step \(t\).

As in GIFT-Eval, both MASE and CRPS are further normalized by the performance of the Seasonal Naive forecast on the test split.

9.2.0.4 Full Results

Following prior work [13], [48], we report the aggregated results in the main text, while the full results are provided in the project repository.

9.3 Imputation↩︎

9.3.0.1 Benchmarks

Following prior work [9], [16], we evaluate Zeuson the same six datasets used for point forecasting. We mask \(\{12.5\%, 25\%, 37.5\%, 50\%\}\) time steps in sequences of length 192.

Existing TSFMs are typically evaluated under patch-wise missing patterns. For example, MOMENT uses a patch size of 8 and therefore only considers missing segments of length 8 [8], while Timer uses a patch size of 24 and considers missing segments of length 24 [9]. However, real-world time series exhibit more diverse missing behaviors, including both point-wise missing and contiguous segment missing with variable lengths [50]. To better reflect practical scenarios, we adopt two masking strategies:

1. Random masking simulates point-wise missing observations that occur sporadically in real-world time series. Such missing values often arise from packet loss during data transmission or occasional logging failures. This setting has been widely adopted in prior work [17], [23], [75].

2. Block masking, where the lengths of contiguous missing segments are sampled from a geometric distribution with \(p=0.125\). This strategy reflects structured missing patterns in practice (e.g., sensor outages or system downtimes). The heavy-tailed nature of the geometric distribution aligns well with real-world missing behaviors, where short gaps are common but extended missing segments can still arise occasionally. We deliberately choose a small \(p\) to create more challenging missing patterns and to clearly distinguish this setting from random masking. The probability distribution of missing segment lengths is shown in Figure 9.

9.3.0.2 Baselines

The baselines for the imputation task include TSFMs, namely MOMENT [8], Timer [9] and UniTS [10], as well as advanced task-specific models, including GPT4TS [28], ModernTCN [23], TimesNet [17], PatchTST [14], and DLinear [76].

9.3.0.3 Metrics

Consistent with the point forecasting setting, we adopt Mean Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics.

9.3.0.4 Full Results

5.1pt

The full results of imputation evaluation are provided in Table ¿tbl:tab:full95imputation?.

9.4 Anomaly Detection↩︎

9.4.0.1 Benchmarks

We evaluate the anomaly detection task on the UCR Anomaly Archive [19], which consists of 250 tasks spanning diverse domains such as medicine, sports, entomology, and space science. The dataset contains both realistic and synthetic anomalies, addressing several limitations of earlier anomaly detection benchmarks. We conduct experiments on a subset of 42 tasks selected by MOMENT², which covers a wide range of domains and exhibits sufficient data diversity [8].

Previous work typically evaluates models using a fixed window size. However, datasets in the UCR archive exhibit substantial variation in periodicity, making a fixed window potentially detrimental to model performance. Therefore, we treat the window size as a hyperparameter and select the optimal value via search from \(\{64, 256,512,1024,2048\}\).

9.4.0.2 Baselines

The baselines for anomaly detection include TSFMs—MOMENT [8], Timer [9], and UniTS [10]—as well as task-specific models, including ModernTCN [23], GPT4TS [28], TimesNet [17], PatchTST [14], and Anomaly Transformer [51].

9.4.0.3 Metrics

Following prior work [8], [17], we use the adjusted F1 score as the evaluation metric. Specifically, point adjustment is applied such that if an anomaly is detected at any time point within a ground-truth anomalous segment, the entire segment is considered correctly detected. Precision and recall are then computed based on these adjusted predictions, and the adjusted F1 score is calculated accordingly. Compared to the standard F1 score, the adjusted F1 score better reflects detection quality at the event level and reduces over-penalization caused by slight temporal misalignment.

9.4.0.4 Full Results

Table 5 presents the full results of anomaly detection.

9.5 Classification↩︎

9.5.0.1 Benchmarks

We conduct evaluations on the UEA classification archive [52], which comprises 30 multivariate time series classification datasets. Since four datasets (namely CharacterTrajectories, InsectWingBeat, JapaneseVowels, and SpokenArabicDigits) contain sequences with unequal lengths, some baselines are incompatible with them and are marked as NA in the corresponding works. To enable a fair comparison using dataset-averaged metrics, we conduct evaluations only on the 26 datasets with equal-length sequences.

While prior work on classification tasks relies on fine-tuning [10], [16] or linear probing [8], Zeusis evaluated in a tuning-free setting. Accordingly, we use a non-parametric 1-nearest neighbor (1-NN) classifier for evaluation, with optional PCA whitening applied as a feature normalization step.

9.5.0.2 Baselines

The baselines include two TSFMs, MOMENT [8] and UniTS [10], a pretrained classification model VQ-shape [77], and task-specific models including ModernTCN [23], SVP-T [78], GPT4TS [28], TimesNet [17], PatchTST [14]; and classical methods including Rocket [79], Shapelet Transformation with Random Forest (STRF) [80], and DTW [81].

All pretrained baselines are fine-tuned in the evaluation of their original works and do not support a tuning-free setting. Specifically, MOMENT performs linear probing using an SVM classifier on pooled representations, VQ-Shape trains a linear head for downstream classification, and UniTS fine-tunes prompt tokens for task adaptation. To compare the tuning-free performance with other pretrained baselines, we replace the SVM classifier in MOMENT with an 1-NN classifier and re-evaluate the results, denoted as \(\text{MOMENT}^\dagger\).

9.5.0.3 Metrics

Following prior work [8], we use the accuracy as the evaluation metric.

9.5.0.4 Full Results

The full results for classification tasks are provided in Table 6.

3pt

10 Pretraining Dataset↩︎

We trained Zeuson a diverse corpus of real and synthetic time series, comprising approximately 300B observations. We developed the Aegis-Syn synthetic dataset, enhancing KernelSynth dataset [15] with additional diverse patterns. In training, synthetic data comprises roughly 10% of the sampled sequences. To mitigate the imbalance in pattern distributions across the training data, we adopt the balanced sampling strategy [41] rather than assigning fixed probabilities per dataset. This promotes faster convergence and enhances model performance. To prevent data leakage, we exclude all evaluation datasets from pretraining data.

10.1 Real-World Datasets↩︎

The real-world data used in our study primarily come from two major sources. The detailed dataset composition is provided in Table 8 and 9.

1. Chronos Datasets [15] contain 94B data points. We do not employ the TSMixup data augmentation technique proposed in Chronos.

2. GiftEvalPretrain Datasets [18] is the pretraining dataset collection used in the GIFT-Eval benchmark. It comprises 71 univariate and 17 multivariate datasets spanning seven domains and 13 temporal frequencies, totaling 4.5 million time series and 230B data points. This dataset does not overlap with the test sets of the GIFT-Eval benchmark.

10.2 Aegis-Syn Dataset↩︎

We developed Aegis-Syn, a synthetic dataset designed to encompass diverse common patterns in time series data. Influenced by the pioneering KernelSynth [15], the current mainstream approach for synthetic time series generation in model pretraining is based on Gaussian processes (GPs) [13], [41], [82]. While effective for modeling smooth and stationary dynamics, GPs are inherently ill-suited for representing non-smooth and discontinuous structures that frequently arise in practice, such as abrupt traffic surges or sudden drops in energy consumption.

To overcome this limitation, we construct a component bank that explicitly models diverse temporal structures commonly observed in real-world time series. The bank includes long-term trend components (e.g., linear drift and logistic growth), periodic patterns with both smooth and non-smooth waveforms (e.g., sinusoidal and square-like signals), stochastic noise components capturing random fluctuations, and anomaly components modeling irregular behaviors such as point anomalies and event-level deviations. The complete set of components and their corresponding sampling probabilities are summarized in Table 7. Based on this component bank, each synthetic time series is generated by sampling and composing components according to predefined probability distributions, following the generative procedure described in Algorithm 10.

11 Limitations and Future Work↩︎

11.0.0.1 Multivariate Correlations

Like most pre-trained models and TSFMs, Zeusfocuses on univariate time series and handle multivariate time series with channel independent strategy, which has proven effective in practice, as evidenced by the experimental results and prior work such as PatchTST [14]. Nevertheless, there exist scenarios where incorporating multivariate correlations can lead to significant improvements for downstream tasks. In that cases, adaptation techniques (e.g., CoRA [83]) can be adopted. Future work will explore explicitly incorporating multivariate correlations into this framework.

11.0.0.2 Generalization to Broader Tasks

At present, Zeusis primarily evaluated on five categories of downstream tasks that have been extensively explored in previous studies. On the one hand, some important downstream tasks, such as time series segmentation and causal discovery, are not yet well supported by the current framework. On the other hand, several compatible and promising tasks, such as irregular time series forecasting (forecasting tasks with structural missingness), remain unexplored in our evaluation. We leave these directions for future work.

11.0.0.3 Challenges in Classification

Time series classification in a fully non-parametric setting remains challenging. While Zeuswith a 1-NN classifier shows promising performance, it is not yet able to outperform state-of-the-art baselines with full-shot training. This can be mainly attributed to two reasons. First, some UEA datasets contain extremely short (\(<30\)) and high-dimensional (\(>100\)) sequences, whereas Zeusis better suited for long, univariate time series. Second, the label semantics in classification datasets are highly heterogeneous and strongly task-dependent. To address these issues, we explored contrastive learning [84] and SwAV-style prototype contrastive learning [85], [86], using joint training or post-training strategies to encourage more clustered representations in Zeus. However, the results were not satisfactory. We plan to continue this line of investigation in future work.

12 Case Study↩︎

12.0.0.1 Forecasting Showcases

Figure 11 presents forecast examples from Zeus. The first eight examples illustrate representative patterns from the Aegis-Syn dataset, while the remaining examples demonstrate the zero-shot forecasting capability of Zeus. In all cases, Zeusis able to produce accurate forecasts that closely follow the underlying temporal dynamics.

12.0.0.2 Reconstruction Showcases

Figure 12 presents zero-shot reconstruction examples from Zeus. For clarity, we primarily present the most challenging cases with a single large contiguous missing block. The first four examples are sampled from imputation benchmarks, demonstrating that the model can accurately reconstruct missing segments and preserve the underlying temporal patterns. The remaining four examples are from the UCR Anomaly Archive, showing that the model is able to precisely localize anomalous regions.

12.0.0.3 Clustering Analysis for Classification

Figure 13 visualizes the representations of Zeuson the UEA classification datasets after dimensionality reduction using t-SNE [87] and PCA. As can be observed, the t-SNE projections exhibit clear clustering patterns, while the PCA projections reveal a coherent low-dimensional manifold structure, indicating that Zeuslearns discriminative and structurally meaningful representations.

References↩︎

---

1. In this paper, the term tuning-free refers to performing inference without tuning or retraining any model parameters.↩︎

2. Although MOMENT originally states that 45 tasks are selected, only 42 tasks are reported in its published results. We therefore follow the reported tables and conduct our evaluation on these 42 tasks.↩︎