1 Introduction↩︎

Financial time series (FinTS) forecasting stands as a pivotal pillar within the realm of quantitative finance, offering profound insights that underpin the formulation of lucrative investment strategies [1]–[6]. Unlike general time series prediction challenges [7]–[11], stock prices are not merely statistical series but the manifestation of complex, often chaotic human behavior shaped by many cognitive, emotional, and sociopolitical factors. Accurately forecasting future returns amidst this maelstrom of data can unlock extraordinary financial gains, positioning it as an invaluable tool for strategic decision-making. As a result, FinTSF has emerged as a cutting-edge domain of scholarly exploration, drawing intense interest from the global research community.

Financial time series refers to a sequence of data points ordered chronologically, typically representing asset price factors or market indicators, that reflect the dynamic behavior of financial markets. As shown in 1, existing FinTSF methods can be broadly categorized into six types based on their underlying backbone approach. Early methods were primarily rooted in classic strategies derived from practitioner experience, classic quantitative trading strategies such as momentum [12] and mean reversion [13] strategies. machine learning-based methods [14]–[16] include a wide range of algorithms, including autoregressive models like ARIMA [17] and tree-based models like XGBoost [18], LightGBM [19], and Random Forests [20], each of which offers distinct advantages in capturing non-linear relationships in financial data. In recent years, deep learning-based methods [2], [6], [21]–[25] have demonstrated state-of-the-art (SOTA) performance by leveraging various neural network architectures (e.g. RNN, CNN, Transformer, Mamba, GNN) to model both stock features and inter-stock correlations, becoming a dominant paradigm in the FinTSF field. Moreover, reinforcement learning-based methods [26]–[30] have also emerged as another promising direction to better optimize sequential decision making processes and end-to-end optimization of some key non-differentiable metrics (such as the sharpe ratio, maximum drawdown). Additionally, to better capture the inherent noise and uncertainty within financial markets, generative model-based methods [31]–[35], such as Variational Autoencoders and Diffusion Models, have been adapted to reflect the heightened levels of uncertainty characteristic of the market, accounting for the low signal-to-noise FinTS. Recently, Large Language Models (LLMs) have attracted considerable attention due to their ability to process vast amounts of unstructured data and perform sophisticated reasoning. LLM-based methods have found diverse applications in FinTSF, including: as an enhancer [36], [37], where they utilize news sentiment and other textual information to augment decision making; as a predictor [38], [39], where they leverage extensive time series training to generalize effectively across different domains; and in LLM-based multi-agent systems (MAS) [40]–[42], where autonomous agents are employed to replicate decision-making processes, communication, and interactions.

As the number of proposed methods expands across various settings, the demand for comprehensive and practical empirical evaluations has correspondingly increased. However, as depicted in 2, existing evaluation frameworks often face challenges such as Diversity Gap, Standardization Deficit, and Real-World Mismatch, which hinder their ability to fully assess the performance of these methods in real-world contexts. To address these limitations, we introduce FinTSB, a novel evaluation framework designed to enhance the robustness and applicability of empirical assessments, thereby improving the evaluation capabilities in FinTSF.

Issue : Diversity Gap. Given the inherent complexity of financial markets, which often include different phases of stock movement such as uptrends, downtrends, periods of volatility, and extreme events (black swan events), current evaluation datasets face several limitations. On the one hand, some datasets contain only three to five years of historical data, which fails to comprehensively represent all possible movement patterns, thus hindering the generalization capacity of models and preventing them from effectively handling unseen patterns in the training set. On the other hand, datasets covering over a decade of data suffer from severe distribution shifts, resulting in reduced model accuracy when confronted with current market conditions. These shortcomings prevent existing evaluations from providing a complete picture of stock movement behavior. In addition, FinTS exhibits distinct characteristics in different markets. For example, the Chinese stock market is often characterized by high retail participation and higher volatility, while the U.S. market tends to have a more balanced mix of institutional and retail investors with a generally higher degree of efficiency. Some existing works evaluate models in only one market, which does not provide a holistic view of their performance.

Our approach emphasizes the diversity of FinTS, focusing on both the comprehensiveness of movement patterns and the broad scope of financial markets. This allows for a more thorough assessment of model performance. Specifically, in terms of movement patterns, we advocate a fine-grained analysis of how different methods perform over different periods of market volatility, with the aim of more accurately reflecting real-world scenarios.

Issue . Standardization Deficit: In the existing literature, discrepancies in evaluation criteria lead to inconsistencies in performance comparisons. To address this issue, we classify the current evaluation metrics into three main categories: ranking metrics, which assess the distribution between predicted and actual daily returns; portfolio-based metrics, which evaluate the profitability and risk of investment strategies derived from predictions; and error metrics, which quantify the degree of approximation between predicted and true values. Nevertheless, it is important to note that forecasting errors show little correlation with overall investment returns.

On top of that, current FinTSF methods lack a standardized pipeline for evaluation. Evaluating SOTA models often requires re-running them on proprietary datasets and settings, which is both time-consuming and labor-intensive. In addition, factors such as differences in data pre-processing, feature engineering, and model tuning further complicate fair comparisons. While the Qlib [43] framework offers a wealth of practical tools, its steep learning curve remains a significant barrier. There is an urgent need for a unified, user-friendly, and lightweight evaluation framework in FinTSF, similar to those found in general time series forecasting fields, such as TSLib [44] and TFB [45].

Issue : Real-World Mismatch. The ultimate goal of FinTSF methods is to be deployed in real-world trading scenarios and to generate actual investment returns in the highly competitive, human-brain-armed environment of financial markets. As a result, there are stringent requirements for simulating realistic trading conditions. Existing research often overlooks these constraints, resulting in overly optimistic portfolio metrics. For example, some models still assume short selling in the Chinese A-share market, which is impractical due to restrictions in certain sectors. Moreover, many studies do not take transaction fees into account, which is particularly critical when constructing portfolios based on the prediction of stocks with top-\(k\) returns. High turnover rates without considering transaction costs can result in significant overlooked expenses that negatively impact the profitability of the strategy. Furthermore, many works neglect trading restrictions such as limit-ups or trading halts, which are common in various markets and can have a profound effect on trading strategies. Thus, we emphasize the necessity of incorporating these real-world constraints into evaluations to ensure more objective and accurate assessments of model performance.

The importance of benchmarking in any field cannot be overstated, as it provides a common basis for comparing the performance of different approaches. Similar to computer vision, where benchmarks such as ImageNet [46] or COCO [47] provide standardized datasets against which models can be evaluated, the absence of a consistent benchmark in FinTSF leads to fragmented and incomparable results. Without a standardized evaluation, it becomes difficult to gauge the true effectiveness of models in real-world applications. By addressing the three key issues of establishing a consistent dataset for fair comparisons, ensuring comprehensive evaluation metrics, and considering real-world trading constraints, the FinTSF community will be able to establish a solid foundation for model assessment, ultimately advancing the field and improving its applicability in real-world investment scenarios.

Building on the above motivations, we propose FinTSB, a comprehensive and practical financial time series benchmark. This dataset first tokenizes sensitive information and then pre-processes real historical stock data from multiple financial markets to mitigate distribution shifts. It categorizes the data into four distinct movement patterns based on daily change rates and assesses the data quality through sequence-based metrics. Additionally, we conduct fair and robust evaluations of a diverse set of FinTSF methods, including all six different backbone models, and derive insightful and critical research conclusions. In a nutshell, our contributions are as follows:

• Diversity Inclusion. We collect and pre-process tokenization historical financial time series data to provide FinTSB, which captures all types of movement patterns across various markets.

• Standardization Consistency. We advocate for a comprehensive evaluation of the capabilities of various methods from three perspectives: ranking, portfolio, and error, which ensures a holistic assessment.

• Real-World Alignment. We meticulously design investment strategies that align with real-world market conditions, facilitating practical implementation in actual trading environments.

• In-depth Evaluation. We evaluate a wide range of FinTSF methods and extract key insights that advance the understanding of model performance in the context of financial time series forecasting.

The paper is organized in the following manner: 2 provides a review of related work, while 3 introduces the definitions of FinTSF tasks and relevant concepts. In 4, we present the design of FinTSB, and 5 evaluates existing FinTSF methods using our benchmark. Finally, 6 concludes the paper.

2 Related Work↩︎

To date, there remains a significant gap in the availability of a comprehensive dataset of financial time series. Existing studies typically focus on specific slices of historical stock data for their experiments. For example, ALSP-TF [48], ADB-TRM [49], and CI-STHPAN [2] utilize New York Stock Exchange (NYSE) and NASDAQ stocks for the period from 2013 to 2017. Meanwhile, MASTER [21], FactorVAE [31], LARA [50], and RSAP-DFM [51] select data from the Chinese A-share market, but with distinct temporal ranges: MASTER [21] covers 2008 to 2022, FactorVAE [31] covers 2010 to 2020, and both LARA [50] and RSAP-DFM [51] cover 2008 to 2020. Similarly, LSR-iGRU [6], FinMamba [52], and MCI-GRU [5], as well as THGNN [24], focus on stocks from both the Chinese and U.S. markets, with time slices ranging from 2018 to 2023 and 2016 to 2021, respectively. Qlib [43] provides a wealth of raw data and factor data, from which users can extract the segments they need. The varying time horizons of these data slices pose a challenge to consistent and fair evaluations. Furthermore, after applying Principal Component Analysis (PCA) [53] for dimensionality reduction, we visualized FinTSB and other time-sliced stocks as a hexbin plot in 4. The results indicate that FinTSB covers the most cells, suggesting that reliance on sliced historical data lacks diversity. In contrast, FinTSB offers more comprehensive coverage, thus effectively capturing the complex distribution of financial time series.

3 Preliminaries↩︎

In this section, we will introduce some related concepts and formally define the problem of financial time series forecasting.

3.1 Problem Definition↩︎

3.1.0.1 Definition 1. Stock Context.

Let the set of all stocks be denoted as \(\mathcal{S}=\{s_1, s_2, ..., s_N\}\in\mathbb{R}^{N\times L\times F}\), where \(s_i\) is a specific stock, \(N\) is the total number of stocks, \(L\) is the length of the lookback window and \(F\) is the number of features. For each stock \(s_i\), its data on trading day \(t\) is represented by \(s_i^t\in\mathbb{R}^F\), with the closing price \(p_i^t\) as one of the features. The one-day return ratio is given by \(r_i^t = \frac{p_i^t-p_i^{t-1}}{p_i^{t-1}}\). On any trading day \(t\), stocks are ranked according to their underlying scores \(Y^t=\{y_1^t\geq y_2^t\geq...\geq y_N^t\}\). If \(r_i^t\geq r_j^t\), then \(y_i^t\geq y_j^t\), establishes an overall ranking based on return ratios. This ranking reflects the expected investment returns for each stock on day \(t\), where stocks that achieve higher ranking scores \(Y\) are expected to achieve a higher investment revenue (profit) on day \(t\).

3.1.0.2 Problem 1. Financial time series forecasting.

Formally, given the stock-specific time series information of \(\mathcal{S}\), the goal is to develop a ranking function that predicts the scores \(Y^{L+1}\) for the next day, ordering the stocks \(s_i\) by their expected profitability.

3.2 Sequence Characteristics↩︎

To explore the intrinsic properties of financial time series data, we introduce the following sequence characteristics. This allows for a more thorough evaluation of the sophisticated dynamics within stock market behavior, providing deeper insights into the underlying patterns and trends that drive financial assets.

3.2.0.1 Characteristic 1. Movement Patterns.

Based on the daily return ratio \(r\), derived from the closing price changes, FinTS can be categorized into different periods, each characterized by a dominant movement pattern, including uptrends, downtrends, periods of volatility, and extreme events. Specifically, uptrends are characterized by a higher frequency of trading days with positive \(r\), while downtrends are marked by a higher frequency of trading days with negative \(r\). Extreme events are defined by significant fluctuations in \(r\), representing periods of sharp price movements. Conversely, periods of volatility are identified by a roughly equal number of positive and negative \(r\), indicating more frequent market fluctuations without a clear directional trend.

3.2.0.2 Characteristic 2. Non-Stationarity.

It is well known that stock data, due to their inherent volatility and external influences, typically exhibit non-stationarity, reflecting complex, time-varying patterns that defy simple statistical modeling. Such time series are considered to be integrated of order \(k\), denoted as \(I(k)\), if it becomes stationary after applying \(k\) times differences. For example, a stock series \(s_i^t\) is \(I(1)\) if its first difference \(\Delta s_i^t = s_i^t - s_i^{t-1}\) is stationary. To test for non-stationarity, the Augmented Dickey-Fuller (ADF) test [54] is commonly employed. It tests the null hypothesis of the presence of a unit root, indicating non-stationarity:

\[\Delta s_i^t = \alpha + \beta t + \gamma s_i^{t-1} + \sum_{j=1}^{p} \delta_j \Delta s_i^{t-j} + \epsilon_t\]

Where \(\Delta X_t\) is the differenced series, \(\gamma\) represents the coefficient of the lagged series, and \(\epsilon_t\) is the error term. A rejection of the null hypothesis (\(\gamma = 0\)) indicates stationarity, while a failure to reject it indicates non-stationarity. A smaller ADF test result indicates more stationary time series data.

3.2.0.3 Characteristic 3. Autocorrelation.

Autocorrelation [55] measures the correlation between a time series and its \(k\) lagged values. Strong autocorrelation \(\tau(\cdot)\) suggests that past values of the series have a significant influence on future values, which can be valuable for predictive modeling and identifying underlying patterns. Mathematically, this can be expressed as: \[\tau(s_i)=\frac{\sum_{t=1}^{L-k}(s_i^t-\bar{s_i})(s_i^{t+k}-\bar{s_i})}{\sum_{t=1}^L(s_i^t-\bar{s_i})^2}\]

3.2.0.4 Characteristic 4. Forecastability.

Forecasting is inextricably linked to the time domain. Following ForeCA [56], we can leverage frequency domain properties to assess the forecastability \(\phi(\cdot)\) of a time series. A higher value \(\phi(x)\) indicates that series \(x\) exhibits a lower forecast uncertainty as measured by the entropy. \[\phi(s_i)=1-\frac{H(s_i)}{\text{log}(2\pi)}\] where \(H(\cdot)\) denotes the entropy derived from the Fourier decomposition of the time series.

4pt

4 FinTSB↩︎

4.1 Dataset Details↩︎

4.1.1 Dataset Construction↩︎

We construct the FinTSB dataset by tokenizing and pre-processing a large amount of raw stock data. Specifically, we first divide 15 years of historical stock data into non-overlapping segments to prevent data leakage. Then, we calculate the daily return (change rate) \(r\) for each stock and, based on this metric, categorize the stocks in each fixed 250-day segment into one of four distinct movement patterns. If there are extreme outliers in the change rate, the stocks are classified as black swan events. Next, we rank the remaining stocks based on a positive change rate, selecting the top 300 as uptrends, the bottom 300 as downtrends, and the remaining 300 as periods of volatility. Afterward, we compute sequence characteristics for each pattern and select the five most appropriate ones. It is important to note that in the pre-processing stage, we need to avoid future information leakage, which necessitates normalization at the stock dimension for each trading day, rather than across the time dimension. Through the above steps, we generate five smaller datasets for each of the four movement patterns, resulting in a total of 20 datasets in the FinTSB. This ensures that FinTSB is comprehensive and diverse, accurately reflecting the dynamics of the financial market.

4.1.2 Dataset Overview↩︎

The benchmark, FinTSB, contains a total of 20 datasets, representing four different movement patterns, with each dataset containing 300 stocks over 250 consecutive trading days. We have carefully designed the dataset selection strategy to ensure that there is no overlap between any two datasets. During training, each dataset is split into training, validation, and test sets in a 7:1:2 ratio. The statistics for each movement pattern are summarized in 1. All patterns exhibit strong non-stationarity, with extreme event patterns being particularly prominent, highlighting the low signal-to-noise ratio typical of financial data. Autocorrelation measures the degree to which a stock’s past price movements influence its future behavior. It can be observed that stocks in the uptrend and downtrend patterns tend to exhibit higher autocorrelation, reflecting the persistence of their directional movements, while stocks in volatile periods show weaker autocorrelation due to frequent reversals. The predictability of different movement patterns varies significantly, with uptrends and downtrends generally being more predictable, while periods of volatility and extreme events pose greater forecasting challenges due to their inherent unpredictability and the occurrence of abrupt, large price movements. Overall, FinTSB encompasses a wide variety of sequence indicators, ensuring that it captures the multifaceted nature of FinTS. By integrating these key characteristics, FinTSB enables the exploration of diverse forecasting challenges.

4.2 Comparison Baselines↩︎

To investigate the advantages and limitations of different methods, as well as their adaptability across different patterns, our evaluation covers the six categories of methods mentioned previously. For classic strategies, we choose CSM [12] and BLSW [13]. In terms of ML-based methods, we include XGBoost [18], LightGBM [19], DoubleEnsemble [57] and ARIMA [17]. Among DL-based methods, we choose Linear, LSTM [58], ALSTM [59], GRU [60], GCN [61], GAT [62], TCN [63], Transformer [64], Mamba [65], PatchTST [66], Crossformer [67], iTransformer [68], AMD [7], PDF [9], Localformer [69]. For RL-based methods, we include PPO [26], DDPG [70], SAC [71] and DQN [72]. For Generative-based methods, we choose DDPM [73], DDIM [74] and FactorVAE [31]. For LLM-based methods, we include Timer [75], Time-MoE [39] and Chronos [76].

4.3 Evaluation Metrics↩︎

Existing work on FinTSF evaluation varies. To achieve a consistent and comprehensive assessment of forecast performance, we employ eleven metrics across three dimensions, including ranking, portfolio, and error. The detailed explanation is provided below.

4.3.1 Ranking Metrics↩︎

Ranking metrics systematically assess the performance of predicted ranking scores (returns) \(Y\) in quantitative research, measuring both cross-sectional and predictive power.

Information Coefficient (IC) quantifies the directional alignment between \(Y\) and subsequent true returns \(r\), calculated as the Spearman correlation coefficient. It evaluates the raw predictive power of scores \(Y\), with statistically significant positive IC values indicating economically meaningful forecasting power. \[\text{IC}=\frac{1}{N}\sum_{i=1}^{N}\frac{\sum_{k=1}^{t}(r_i^k-\bar{r_i})(Y_i^k-\bar{Y_i})}{\sqrt{\sum_{k=1}^{t}(r_i^k-\bar{r_i})^2}\cdot\sqrt{\sum_{k=1}^{t}(Y_i^k-\bar{Y_i})^2}}\]

Information Coefficient Information Ratio (ICIR) measures the stability of the performance of \(Y\) by comparing the annualized mean IC with its temporal volatility. (\(\text{ICIR}=\frac{\text{mean}(\text{IC})}{\text{std}(\text{IC})}\))

Rank Information Coefficient (RankIC) is the Spearman correlation metric that employs dual-ranking normalization: both \(Y\) and \(r\) are converted to uniform percentile ranks before calculating the correlation. This process eliminates scaling artifacts and reduces sensitivity to outlier bias, which is particularly beneficial when analyzing non-linear factor-response relationships. \[\text{RankIC}=1-\frac{1}{N}\sum_{i=1}^{N}\frac{6\sum_{k=1}^{t}(R(r_i^k)-R(Y_j^k))^2}{t(t^2-1)}\] where \(R(\cdot)\) is the rank function.

Rank Information Coefficient Information Ratio (RankICIR) evaluates the reliability of rank-based relationships between \(Y\) and \(r\). (\(\text{RankICIR}=\frac{\text{mean}(\text{RankIC})}{\text{std}(\text{RankIC})}\))

4.3.2 Portfolio-Based Metrics↩︎

Portfolio-based metrics evaluate the performance and risk characteristics of investment strategies through simulated portfolio implementation. These metrics provide a perspective for assessing both absolute and risk-adjusted returns, drawdown behavior, and strategy efficiency of FinTSF methods.

Annualized Return Ratio (ARR) measures the geometric mean return of a strategy annualized over the evaluation period and serves as the primary indicator of strategy profitability. (\(\text{ARR}=(1+\text{Total Return})^{\frac{252}{n}}-1\))

Annualized Volatility (AVol) quantifies the dispersion of strategy returns and captures the consistency of performance delivery, with lower values indicating more stable return streams. \(R_p\) denotes the daily return of the portfolio. (\(\text{AVol}=\sqrt{252\cdot\text{Var}(R_p)}\))

Maximum Draw Down (MDD) represents the largest peak-to-trough decline (\(p_{peak}-p_{trough}\)) in portfolio value over the evaluation period and is critical in assessing the strategy’s risk tolerance and ability to preserve capital. (\(\text{MDD}=-\text{max}\bigg(\frac{p_{peak}-p_{trough}}{p_{peak}}\bigg)\))

Annualized Sharpe Ratio (ASR) measures the excess return per unit of total risk, assessing risk-adjusted performance. (\(\text{ASR}=\frac{\text{ARR}}{\text{AVol}}\))

Information Ratio (IR) assesses the ability to generate excess returns relative to a benchmark and the effectiveness of active management. \(\text{IR} = \frac{\text{mean}(R_p-R_b)}{\text{std}(R_p-R_b)}\), where \(R_b\) is the daily return of the market index.

4.3.3 Error Metrics↩︎

Some reviewers emphasize the use of mean square error (MSE) and mean absolute error (MAE) to quantify the discrepancy between predicted \(Y\) and actual returns \(r\). However, it is important to note that a lower MSE or MAE does not guarantee profitability; market impact, position sizing rules and transaction costs ultimately determine the success of the strategy. The formulations are: \(\text{MSE}=\frac{1}{L}\sum_{t=0}^{L}(Y_{i}^t-r_i^t)^2\), \(\text{MAE}=\frac{1}{L}\sum_{t=0}^{L}|Y_{i}^t-r_i^t|\).

4.4 Unified Pipeline↩︎

As discussed in Issue , the use of divergent evaluation criteria has led to differences in model performance. To ensure a fair, comprehensive, and practical evaluation, we introduce a unified pipeline that is structurally divided into the data layer, the training layer, the backtesting layer, and the feedback layer. A detailed description of each module is provided below.

Data Layer stores comprehensive market information in FinTSB, encompassing four different movement patterns. For data pre processing, we implement tokenization (anonymization) and normalization operations. The dataloader dynamically constructs global training/validation/test sets based on the selected movement modes. Researchers can evaluate model performance under identical market conditions by maintaining consistent modes for training and testing phases. Cross-pattern evaluation through transfer learning (training on selected patterns and testing on others) enables granular analysis of strategy adaptability across market regimes, which is particularly valuable for assessing model generalization capabilities. Additionally, historical stock market data verification validates model effectiveness in real-world financial scenarios.

4.5pt

Training Layer integrates dozens of canonical models built on six heterogeneous backbone architectures. We extend Qlib’s foundation by designing an easy-to-use and unified training pipeline that ensures evaluation consistency. The framework maintains model-agnostic compatibility - researchers employing FinTSF paradigm can seamlessly integrate their new models while maintaining evaluation impartiality. In particular, we implement parameter encapsulation through dedicated configuration modules rather than monolithic config files/main functions, significantly enhancing code readability and customization flexibility.

Backtesting Layer currently incorporates two classic strategies: Top\(K\) and Top\(K\)-Drop, with transaction cost simulations reflecting real market conditions. Through multi-dimensional evaluation spanning prediction errors (2 metrics), ranking accuracy (4 metrics), and portfolio performance (5 metrics), we comprehensively quantify model capabilities across 11 rigorously wide-used indicators.

Feedback Layer systematically archives training logs, preserves prediction results, and provides interactive visualization tools. This design facilitates continuous model optimization by tracking performance across training iterations, enables comparative analysis through standardized results formats, and supports decision-making via intuitive graphical representations of strategy behavior.

Users only need to deploy their method at the training layer and configure the configuration file, then FinTSB can automatically run the pipeline in 5, enabling to better understand, compare, and select FinTSF methods for specific financial scenarios.

5 Expeiments↩︎

5.1 Experiment Setup↩︎

Our experiment is trained on the NVIDIA A100 GPU for LLM-based methods and NVIDIA V100 GPU for others, and all models are built using PyTorch [79]. The training, validation, and test sets (from FinTSB) are kept consistent for all models. The lookback horizon \(L\) is set to \(20\) and we will predict the returns \(Y\) on the next trading day. For each method, we adhere to performing hyperparameter searches across multiple sets for optimal results. The training process employs a dual-objective optimization framework, simultaneously minimizing a composite loss function that integrates both point-wise regression loss and pair-wise ranking loss. These complementary objectives are balanced through an adaptive weighting coefficient \(\eta\), which is set to \(5\). \[\mathcal{L}=\frac{1}{L}\sum_{t=1}^{L}(\sum_{i=1}^{N}||Y_i^t-r_i^t||^2+\eta\sum_{i=1}^{N}\sum_{j=1}^{N}max(0,-(Y_i^t-Y_j^t)(r_i^t-r_j^t)))\]

5.2 Trading Protocols↩︎

We adopt the Top\(K\)-Drop strategy [43] to maintain a portfolio on each trading day. The Top\(K\)-Drop strategy improves upon Top\(K\) strategy by dynamically optimizing portfolio turnover. Rather than fully rebalancing holdings daily, it retains stocks that persistently rank in the top-\(K\) cohort and only replaces underperformers. This reduces the frequency of transactions, thereby lowering commission costs in proportion to the actual turnover rate, and maintains exposure to stocks with sustained high scores, avoiding unnecessary exits. Formally, on trading day \(t\), Top\(K\)-Drop constructs an equal-weighted portfolio of \(m\) stocks \(\mathcal{P}^t=\{s_{i_1}^t,s_{i_2}^t,...,s_{i_m}^t\}\), which are selected according to the rank of predicted returns \(Y\). Given the turnover constraint, the number of intersection stocks \(n\) is required to fulfill the condition \(|\mathcal{P}^t\cap\mathcal{P}^{t-1}|\geq m-n\). In our experiment, we set \(m\) at one tenth of the total number of stocks, i.e., \(m=30\), and \(n\) is set to \(5\). Furthermore, we charge a transaction fee at a rate of \(0.1\%\), in line with standard market practice.

1.5pt

5.3 Experiment Results↩︎

The performance results of various FinTSF methods are reported in [tab:mainexp]. We observe that no single method achieves the best performance across all three dimensional metrics. There is significant performance heterogeneity among different methods, even within the same backbone category, which we attribute to the critical factor of how effectively models capture temporal dependencies and cross-sectional relationships to predict future stock movements. Notably, LLM-based approaches exhibit a paradoxical pattern in which performance initially deteriorates with model scaling but subsequently shows marked improvement at larger scales, suggesting the emergence of financial time series specific capabilities that manifest only beyond critical thresholds of model capacity, possibly due to the need for sufficient parameters to disentangle complex market noise and latent factor interactions. Contrary to expectations, modern deep learning techniques do not universally outperform traditional quantitative strategies or tree-based models, underscoring the necessity for rigorous benchmarking across diverse architectures and temporal regimes. This empirical evidence argues for a more nuanced evaluation framework that takes into account both model scalability and FinTS characteristics.

5.4 Transfer Learning Results↩︎

To validate the cross-data generalization capability of FinTSB, we conduct transfer learning experiments by applying models pre-trained on FinTSB to backtest the entire 2024 CSI 300 stock market. The empirical results reveal two key insights. First, as shown by the metrics in 2 and the cumulative return trajectories in 6, the model demonstrates remarkable performance consistency across different market regimes. Second, the superior risk-adjusted returns achieved through this zero-shot transfer learning paradigm highlight FinTSB’s unique advantages in both pattern diversity coverage and temporal robustness, establishing it as a comprehensive benchmark for heterogeneous market behaviors spanning bull, bear, and transitional market phases.

5.5 Inference Efficiency↩︎

It is well known that quantitative trading in real-world markets places high demands on system latency sensitivity. Given this critical operational requirement, we conducted a comparative visualization analysis of representative algorithmic models in 7, focusing on three key performance metrics, ASR performance, computational inference efficiency, and memory footprint allocation.

6 Conclusion↩︎

In this paper, we propose FinTSB, a comprehensive benchmark for FinTSF that addresses three key challenges. By categorizing stock movement patterns into four different types, we ensure a more diverse and representative evaluation, filling the Diversity Gap overlooked in previous studies. Furthermore, we introduce a unified evaluation framework that standardizes performance metrics across multiple dimensions, mitigating the Standardization Deficit and enabling more reliable cross-study comparisons. To bridge the gap between theoretical models and real-world applications, we incorporate critical market structure factors to overcome the Real-World Mismatch that often distorts performance metrics. In general, FinTSB offers a robust platform for advancing the evaluation and development of FinTSF methods, which we believe may pave the way for further research into the practical application of FinTSF.

References↩︎