1 Introduction↩︎

Trajectory forecasting is a core component of autonomous driving systems, as it enables vehicles to anticipate the behavior of other traffic participants. Because human driving is inherently non-deterministic and multimodal, it is not sufficient to predict only a single future trajectory per agent. Instead, trajectory forecasting models must capture multiple plausible futures and quantify uncertainty such that downstream applications can account for possible behaviors.

In practice, the trajectory forecasting task is defined as follows: given a road graph describing the static scene context (, lanes, crosswalks) and the recent state histories of relevant agents (, cars, pedestrians), the model must predict a set of \(K\) future trajectories over a fixed temporal horizon for each agent. Typically, \(K=6\). In addition, each trajectory gets assigned a confidence score. Together, these predictions should represent a multimodal distribution over plausible futures corresponding to the true uncertainty of the world.

In recent years, a diverse set of architectures and training objectives has been proposed for this task [1]–[6]. Interestingly, we observe that these models are strongly shaped by the particular metrics that are used in the most popular benchmarks. Models evaluating on Argoverse 2 [7], whose main metric Brier-minFDE depends on endpoint distance, often employ training objectives that closely mirror these distance-based metrics [8], [9]. Conversely, methods evaluated on the Waymo benchmark [10], which uses soft mAP, are optimized for broader coverage of the space of possible future endpoints, , by using non-maximum suppression to avoid clusters of endpoints [11]–[13]. Some recent papers even train multiple models to optimize different metrics [4], [11], [14].

While it is not inherently problematic to optimize for a benchmark, tightly coupling trajectory forecasting models to evaluation metrics can lead to several issues. Different benchmark metrics encode different, sometimes conflicting, goals: distance-based metrics reward precise final positions, whereas soft mAP expects broad coverage of the future space. As a result, models tuned to one metric tend to perform worse on others, and it becomes hard to assess whether improvements stem from genuinely better forecasting or from better metric-specific tuning. Moreover, benchmark metrics do not directly evaluate the model’s predictive distribution and thus only provide a partial view of its quality.

In this work, we argue for a paradigm change in trajectory forecasting: the primary training objective should be to learn a good predictive distribution, independent of the metrics used to evaluate the forecast. Such a distribution should provide a faithful representation of uncertainty. If it is well-calibrated, it can serve as a general, metric-agnostic interface that can then be exploited by various downstream tasks, including optimizing for evaluation metrics.

We suggest that optimizing for different metrics should become part of the evaluation protocol as a post-processing step applied to the generated predictive distribution. To this end, we build upon and generalize the per-metric optimization proposed by HOME [15] by defining metric-specific policies that map these distributions to the finite set of \(K\) trajectories and confidences required by a particular metric. Our Trajectory-Distribution Evaluation policies (TraDiE policies) are derived from Monte Carlo samples of the predictive distribution for common trajectory prediction metrics. Crucially, these policies are applied to the outputs of a single underlying model, without retraining. This evaluation protocol rewards the model for a good predictive distribution: if it is accurate, policies optimizing for the metric under the distribution will yield good performance.

We argue that adopting this decoupled view will allow the trajectory forecasting community to again focus on the core of the problem, namely how to represent the predictive distribution and how to improve its accuracy. This turns the choice of predictive distribution into a general design decision. It allows systematic evaluation of how future agent trajectories and their uncertainty should be represented and how trajectory forecasting models should be trained, independent of specific benchmark metrics.

As a concrete instantiation, we apply this idea to DONUT [1], a recent trajectory forecasting model that achieves state-of-the-art results on Argoverse 2. Naively applied to Waymo with its original, minFDE-related loss, DONUT’s soft mAP performance is extremely poor, making it an ideal testbed to study the impact of metric-agnostic training. We replace the original loss with negative log-likelihood objectives, which directly optimize the predictive distribution, and find that these models improve both minFDE and (soft) mAP significantly when combined with our policies. This shows that optimizing the distribution directly is indeed better than training for surrogate submetrics. We further evaluate different distribution families for representing agents’ positions and show that mixtures of generalized Gaussian distributions are better suited than mixtures of Laplacian distributions or Gaussian scale mixtures. Beyond DONUT, we apply our metric-specific policies to two strong open-source baselines (MTR [11] and QCNet [2]) and show that they (i) consistently improve MTR across metrics and (ii) expose a distribution mismatch for QCNet under distance-based evaluation.

In summary, our contributions are:

1. We advocate for a paradigm shift in which trajectory forecasting models are trained with metric-agnostic probabilistic objectives that learn well-calibrated predictive distributions, and in which metric optimization is treated as a downstream step via metric-specific policies.

2. We propose sampling-based metric-specific policies for mapping predictive distributions to \(K\) trajectories and confidences, with concrete implementations for minFDE, (soft) mAP and miss rate, enabling evaluation of multiple metrics from a single predictive model without metric-specific retraining.

3. We introduce DONUT-NLL, a negative log-likelihood training scheme for DONUT, which achieves state-of-the-art results on the Waymo benchmark, demonstrating the effectiveness of optimizing the distribution directly.

4. We conduct a systematic comparison of position distributions and empirically find that generalized Gaussian distributions outperform alternatives.

5. We validate that the proposed policies are model-agnostic by applying them to MTR [11] and QCNet [2], highlighting both gains and failure modes when predictive distributions are misspecified.

2 Related Work↩︎

Early work on trajectory forecasting encodes the environment in bird’s-eye-view rasterizations and applies convolutional neural networks to capture local context [15]–[21]. More modern approaches use sparse input representations, where agents’ histories and map elements are modeled as polylines [22], [23]. On top of such structures inputs, graph neural networks have been used to capture relations between agents and the road layout [24], [25]. Current approaches commonly model agents as queries and use attention mechanisms for interaction between agents and map elements [1]–[3], [6], [11], [12], [14], [26]–[30].

To model uncertainty over future trajectories, earlier works [16], [21] have employed VAEs [31], which allow sampling of an arbitrary number of trajectories. Some approaches [15], [32], [33] represent the uncertainty as dense, rasterized heatmaps. Most current models predict a fixed number of possible futures, often represented as the modes of Gaussian mixtures [3], [4], [11]–[14], [27], [34], [35] or Laplacian mixtures [1], [2], [6], [8], [30]. For training, they commonly employ a winner-takes-all loss, in which the trajectory closest to the ground truth gets optimized with negative log-likelihood. Other approaches [9], [36], [37] directly regress the best trajectory with an \(L_1\) or smooth \(L_1\) loss without defining a continuous mixture. A few works [5], [38] employ GPT-like models, which discretize sub-trajectories into a fixed number of classes and learn a categorical distribution per timestep. Such models allow sampling arbitrarily many trajectories at inference time.

As we discuss in more detail in 3.3, many of these models optimize for the main metrics of either Argoverse 2 or Waymo. As a result, models perform well on specific aspects of the trajectory prediction task, rather than improving the predictive distribution itself. Moreover, the objectives induced by these metrics are partly conflicting, so models optimized for one benchmark metric typically perform poorly on the other. In contrast, we propose to train models with metric-agnostic probabilistic objectives that focus on learning good predictive distributions, and to handle different benchmark metrics via metric-specific policies applied at evaluation time, allowing a single calibrated model to be reused across multiple metrics and downstream tasks.

The idea to decouple predictive distribution estimation from reporting has also been explored by HOME [15]. HOME outputs a dense heatmap for the endpoints and introduces a mechanism to post-process this output separately for individual metrics. However, the post-processing is tightly coupled to a specific discrete endpoint representation and cannot be used as a general, model-agnostic solution. In contrast, we derive sampling-based policies that can be applied to any probabilistic forecaster and directly follow the metric definitions without model-specific tuning. We provide a detailed comparison to HOME in App. 10.

3 Evaluation Metrics↩︎

In this section, we will present common evaluation metrics for trajectory forecasting and analyze how previous methods have implicitly or explicitly adapted their optimization pipeline for either distance-based or window-based metrics. For each agent \(n\), these metrics evaluate a set of \(K\) predicted trajectories \(\{\hat{\boldsymbol{X}}_{nk}^\text{pos}\}_{k=1}^K\) where each trajectory \(\hat{\boldsymbol{X}}_{nk}^\text{pos}\) is a sequence \((\hat{\boldsymbol{x}}_{nkt}^\text{pos})_{t=1}^T\) of 2D positions over \(T\) future timesteps, given a ground-truth trajectory \(\boldsymbol{Y}_{n}^\text{pos} = (\boldsymbol{y}_{nt}^\text{pos})_{t=1}^T\). Some metrics additionally consider predicted confidences \(\{\hat{\pi}_{nk}\}_{k=1}^K\) assigned to the trajectories.

3.1 Distance-Based Metrics↩︎

Distance-based metrics evaluate how close a predicted trajectory comes to the ground-truth trajectory. A common example is the minimum final displacement error (minFDE), visualized in 1. For an agent \(n\), it is defined as the Euclidean distance between the ground-truth endpoint \(\boldsymbol{y}_{nT}^\text{pos}\) and the closest endpoint of \(K\) predictions \(\{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K\): \[\text{minFDE}(\boldsymbol{y}_{nT}^\text{pos}, \{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K) = \min_k \|\boldsymbol{y}_{nT}^\text{pos} - \hat{\boldsymbol{x}}_{nkT}^\text{pos}\|_2. \label{eq:minfde}\tag{1}\] The minFDE for a dataset is obtained by averaging over all target agents.

3.2 Window-Based Metrics↩︎

Other metrics rely on windows with a specific size and orientation for evaluation. A simple example is the miss rate (MR), which counts the proportion of agents that are misses, , where none of the predicted endpoints are within a window \(W\) around the ground truth \(\boldsymbol{y}_{nT}\). For a single agent \(n\) a miss is defined as \[\text{Miss}(\boldsymbol{y}_{nT}, \{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K) = \mathbb{I}(\nexists k:\hat{\boldsymbol{x}}_{nkT}^\text{pos} \in W(\boldsymbol{y}_{nT})),\] where \(\mathbb{I}\) is the indicator function. Argoverse 2 defines the window to be a circle with a radius of 2 m. Waymo uses a rectangle aligned to the ground-truth heading \(\boldsymbol{y}_{nT}^\text{hd}\) whose size depends on the agent’s velocity at the last historical timestep.

Average precision (AP) is a window-based metric that also considers confidences ([fig:map]) which are not required to sum to \(1\). To compute AP, predictions for all agents across the dataset are sorted based on their confidence. For each confidence level, precision and recall are computed. A prediction counts as a true positive, if its endpoint is the highest-confidence endpoint within the window \(W\) around the ground-truth endpoint. Predictions with lower confidences within the window either count as false positive (AP) or are ignored (soft AP). Predictions outside the window are considered false positives and windows not hit by any predictions become false negatives. From these, a precision-recall curve is computed ([fig:smooth]), where standard object detection interpolation is applied [39].

To compute mean average precision (mAP), agents are categorized into buckets based on their ground-truth trajectories (, straight, left, stationary). AP is computed independently for each bucket and then averaged. Soft mAP is computed analogously and serves as the main metric for Waymo, using the same window definition as the miss rate.

Distance-based metrics and window-based metrics reward different behaviors from a forecasting model. Distance-based metrics improve if at least one predicted trajectory is very close to the ground truth. A model is rewarded if it puts many trajectories close together into the main mode, even if they are redundant. By contrast, window-based metrics care about coverage: models score well if they have one prediction in the ground-truth window, additional close-by trajectories either bring no benefit or are actively penalized. This incentivizes a spread between predictions. These goals are inherently conflicting: concentrating predictions to minimize distance helps minFDE but hurts coverage, whereas spreading predictions to cover many windows improves mAP or miss rate but worsens distance-based metrics.

3.3 Metric-Driven Modeling↩︎

Many current state-of-the-art models implicitly or explicitly tailor their training objectives and post-processing schemes to either distance-based or window-based metrics. Because these metrics reward conflicting behaviors, such metric-dependent training encourages models to specialize for one metric at the expense of others, rather than to learn a generally useful predictive distribution that can support diverse downstream tasks.

Models optimized to train for distance-based metrics often predict exactly \(K=6\) trajectories and directly train these outputs by minimizing a distance error between the best trajectory and the ground truth. Concretely, they apply the winner-takes-all (WTA) paradigm which selects the best mode based on Euclidean distance and regress the corresponding trajectory with an \(L_1\) loss [9], smooth \(L_1\) loss [36], [37], a Laplacian negative log-likelihood (, an \(L_1\) loss with a learned scale) [1], [2], [8], [30], or a Gaussian negative log-likelihood (, a squared \(L_2\) loss with a learned scale) [4], [11], [14]. This procedure mirrors the definition of minFDE, where only the closest prediction contributes to the metric. This improves distance-based metrics but leads to tightly clustered predictions and poor performance on window-based metrics [4], [11], [14].

Generative, GPT-like trajectory forecasting approaches, which can sample an arbitrary number of trajectories, use \(k\)-means clustering as a post-processing step to select \(K\) trajectories for reporting to the benchmarks [5], [38]. Similarly, several works apply \(k\)-means clustering on ensembles of trajectories to further reduce distance-based errors [2], [30], [36], [37]. In both cases, clustering minimizes the average squared Euclidean distance between sampled trajectories and the chosen cluster centers, which is a direct surrogate for squared minFDE under the model’s predictive distribution. Again, this prioritizes minimizing distance to the ground truth over modeling the full future distribution.

Methods targeting window-based metrics adapt training and post-processing to encourage coverage of many distinct evaluation windows. A common pattern on Waymo is to output \(64\) candidate trajectories and then apply non-maximum suppression (NMS) as post-processing [3], [11], [14], [27], [34]. By suppressing predictions that fall close to each other, NMS decreases the number of trajectory endpoints within a window. This is rewarded by (soft) mAP, where only the highest-confidence hit within each window can contribute to the score. Some methods even make the NMS radius depend on the predicted trajectory length [4], [12], [13], [35], which approximates the velocity-dependent window size used by Waymo and thereby aligns the post-processing even more closely with the specific definition of soft mAP. While these strategies improve mAP and miss rate, they typically worsen minFDE [4], [11], [14].

ModeSeq [6] goes one step further by aligning its training criterion precisely with the window-based metrics. In its early-match-takes-all scheme, a match is defined exactly as a true positive in the benchmark: using a circular window of radius 2 m for Argoverse 2 and a heading-aligned rectangle whose size depends on the last observed velocity for Waymo. This directly trains the model to avoid predicting multiple trajectories within the same window and tailors the optimization to the precise hit definitions of the metrics. While this improves the targeted window-based scores, it further biases the model toward the benchmark objective and away from modeling the full predictive distribution.

As many current methods are tightly coupled to particular evaluation protocols, instead of aiming for general representations, models cannot be easily applied in novel contexts. In the following, we instead advocate training with metrics-agnostic probabilistic objectives that prioritize the quality and calibration of the predictive distribution. We treat benchmark metrics as downstream problems that are addressed at evaluation time via metric-specific policies.

4 Metric-Specific Policies↩︎

Instead of adapting the training procedure to improve a certain metric, we propose to decouple the training objective from the evaluation metrics. Concretely, models should be trained to output a calibrated predictive distribution over the agents’ future trajectories, which is the most general representation of uncertainty. For evaluation, we propose to continue using existing benchmarks, which require reporting \(K\) trajectories with associated confidences \(\{(\hat{\boldsymbol{X}}_{nk}, \hat{\pi}_{nk})\}_{k=1}^K\). To obtain these, we introduce TraDiE policies for Trajectory Distribution Evaluation, which output trajectories and confidences that are optimal for a specific benchmark metric given the predictive distribution. This way, we can assess the quality of the predictive distribution through standard benchmark metrics.

Given a model providing a predictive distribution \(p_n(\boldsymbol{X}_n)\) over future trajectories \(\boldsymbol{X}_n\), we define a distribution-aware policy as a mapping \[\mathcal{P}: p_n(\boldsymbol{X}_n) \mapsto \{(\hat{\boldsymbol{X}}_{nk}, \hat{\pi}_{nk})\}_{k=1}^K.\] Ideally, such a policy produces trajectories and confidences that are optimal for a given evaluation metric under the assumption that the ground truth is drawn from the model’s predictive distribution.

In the following, we design policies for the most commonly used trajectory forecasting metrics, which are evaluated at the endpoint of the forecasting horizons. For these, we only require the predictive endpoint distribution \(p_{nT}(\boldsymbol{x}_{nT})\), , the marginal of \(p_{n}(\boldsymbol{X}_n)\) over the final timestep \(T\). The policies then operate on endpoints instead of full trajectories. Depending on the metric, \(\boldsymbol{x}_{nT}\) is required to contain either only the positions \(\boldsymbol{x}_{nT}^\text{pos}\), or both positions and headings \(\boldsymbol{x}_{nT}^\text{hd}\). Implementation details are provided in App. 12.

4.1 Policies for Distance-Based Metrics↩︎

To obtain the optimal position predictions \(\{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K\) for the minFDE metric given the model’s predicted distribution \(p_{nT}(\boldsymbol{x}_{nT})\), we define the optimization objective for the minFDE policy as \[\mathbb{E}_{\boldsymbol{x}_{nT} \sim p_{nT}} \left[\min_k \|\hat{\boldsymbol{x}}_{nkT}^\text{pos} - \boldsymbol{x}_{nT}^\text{pos}\|_2\right]. \label{eq:riskfde}\tag{2}\] This is the expected minFDE for the predictions \(\{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K\) under the predictive distribution \(p_{nT}(\boldsymbol{x}_{nT})\). We can optimize this by using Monte Carlo sampling to obtain a set of positions, \(S\), from the endpoint distribution following \[S = \{\boldsymbol{x}_{nT}^{(1)}, \dots, \boldsymbol{x}_{nT}^{(M)}\}, \quad \boldsymbol{x}_{nT}^{(m)}\sim p_{nT}(\boldsymbol{x}_{nT}), \label{eq:MCs}\tag{3}\] and minimizing the empirical approximation \[\frac{1}{|S|} \sum_{\boldsymbol{x}_{nT} \in S} \min_k \|\hat{\boldsymbol{x}}_{nkT}^\text{pos} - \boldsymbol{x}_{nT}^\text{pos}\|_2 \label{eq:optfde}\tag{4}\] by treating the endpoints \(\hat{\boldsymbol{x}}_{nkT}^\text{pos}\) as free variables and optimizing them using gradient descent (2, top).

4.2 Policies for Window-Based Metrics↩︎

Computing an optimal policy for average precision is challenging, because it depends on the global ranking of all predictions across all agents and scenes. To obtain a tractable approximation, we design a per-agent heuristic. We first draw a set of candidate endpoints from the predictive distribution and associate each candidate with an evaluation window. We then greedily select endpoints that are covered by the largest number of windows, while ignoring windows that have already been hit in previous steps.

More formally, let \(W_n({\boldsymbol{y}}_{nT})\) denote the window defining a hit centered at the ground-truth endpoint \({\boldsymbol{y}}_{nT}^\text{pos}\) with orientation \({\boldsymbol{y}}_{nT}^\text{hd}\). A specific predicted endpoint \(\hat{\boldsymbol{x}}_{nkT}^\text{pos}\) from the set of predictions \(\{(\hat{\boldsymbol{x}}_{nkT}^\text{pos}, \hat{\pi}_{nk})\}_{k=1}^K\) with confidences \(\hat{\pi}_{n1} \geq \hat{\pi}_{n2} \geq \dots \geq \hat{\pi}_{nK}\) is counted as a true positive (\(\text{TP}\)) if it lies within the window \(W_n({\boldsymbol{y}}_{nT})\) and no prediction with higher confidence lies in the same window. Formally, we define \[\text{TP}_{nk}(\hat{\boldsymbol{x}}_{nkT}^\text{pos}, \boldsymbol{y}_{nT}) = \mathbb{I}\left(\hat{\boldsymbol{x}}_{nkT}^\text{pos} \in W_n(\boldsymbol{y}_{nT}) \land \forall i<k: \hat{\boldsymbol{x}}_{niT}^\text{pos} \notin W_n(\boldsymbol{y}_{nT})\right). \label{eq:tp}\tag{5}\] As the ground-truth endpoint \(\boldsymbol{y}_{nT}\) is not available at inference time, we use samples \(\tilde{\boldsymbol{x}}_{nT}\) from the predictive distribution \(p_{nT}\) instead. The probability that \(\hat{\boldsymbol{x}}_{nkT}\) is the true positive under the predictive distribution is then given by \[\overline{\text{TP}}_{nk}(\hat{\boldsymbol{x}}_{nkT}^\text{pos}) = \mathbb{E}_{\tilde{\boldsymbol{x}}_{nT} \sim p_{nT}} \left[ \text{TP}_{nk}(\hat{\boldsymbol{x}}_{nkT}^\text{pos}, \tilde{\boldsymbol{x}}_{nT})\right].\]

In practice, we approximate this expectation by averaging over samples \(\tilde{\boldsymbol{x}}_{nT}\) of a Monte Carlo set \(S\) (2, bottom). To find predictions \(\{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K\), a greedy heuristic iteratively picks \(\hat{\boldsymbol{x}}_{nkT}^\text{pos}\) for \(k\in\{1,\dots,K\}\) from the same set \(S\) with \[\hat{\boldsymbol{x}}_{nkT}^\text{pos} = \mathop{\mathrm{arg\,max}}_{\boldsymbol{x}_{nT} \in S} \overline{\text{TP}}_{nk}(\boldsymbol{x}_{nT}^\text{pos}),\] , we choose the endpoint to which we assign the highest probability of becoming a true positive. We define the corresponding confidences as \[\hat{\pi}_{nk} = \overline{\text{TP}}_{nk}(\hat{\boldsymbol{x}}_{nkT}^\text{pos}).\]

This approach has several properties that naturally align with improving average precision. By repeatedly choosing the endpoint that is most likely to be the highest-confidence true positive, we increase the chance that true positives appear earlier than false positives in the global ranking, which improves precision across confidence thresholds. The greedy selection is also a standard heuristic [40] for the maximum coverage problem: prioritizing endpoints that cover many yet-uncovered windows reduces the number of false negatives and thus improves recall and AP. Finally, because windows that have already been hit are ignored in subsequent steps, the procedure avoids placing multiple predictions in the same window, which would either be ignored (soft mAP) or penalized (mAP).

A forecast is considered a miss if none of the \(K\) predictions are within a window around the target. The miss rate is the proportion of misses across the dataset. Argoverse 2 defines this region as a circle with a radius of 2 m; Waymo uses the same definition as for the (soft) mAP window. Confidences do not matter for this metric. For an agent \(n\), the probability of a hit (at least one prediction in the window) under the predictive distribution is \[\sum_{k=1}^K\overline{\text{TP}}_{nk}(\hat{\boldsymbol{x}}_{nkT}^\text{pos}). \label{eq:optmr}\tag{6}\] Thus, this metric is optimized when we select \(\{\hat{\boldsymbol{x}}_{nkT}^\text{pos}\}_{k=1}^K\) to maximize 6 , , we aim to maximize the total number of covered windows under the predictive distribution. As our greedy heuristic for (soft) mAP aims to optimize this objective, the same heuristic policy is applicable to soft mAP, mAP, and miss rate.

5 DONUT-NLL↩︎

5.1 Baseline↩︎

To study metric-agnostic training and metric-specific evaluation policies in a controlled setting, we instantiate our approach on top of DONUT [1], a decoder-only transformer for trajectory forecasting that obtains state-of-the-art results on Argoverse 2 [7]. For each agent \(n \in \{1, \dots, N\}\) and each predicted trajectory \(k \in \{1, \dots, K\}\), DONUT predicts a \(T\)-step future trajectory. It parameterizes positions with 1D Laplace distributions over the global \(x\)- and \(y\)-axes by predicting \(\boldsymbol{\mu}_{n}^x, \boldsymbol{\mu}_{n}^y, \boldsymbol{b}_{n}^x, \boldsymbol{b}_{n}^y \in \mathbb{R}^{T \times K}\), and it models heading with von Mises distributions with parameters \(\boldsymbol{\mu}_{n}^\text{hd}, \boldsymbol{\kappa}_{n}^\text{hd} \in \mathbb{R}^{T \times K}\). In addition, mixture weights \(\pi_n \in \mathbb{R}^K\) are obtained from the decoder via a small prediction head, representing non-negative mode probabilities over the \(K\) trajectories.

To better align the positional uncertainty with the kinematics of the agents, we make the following adaptation to DONUT: we use the predicted mean heading \(\boldsymbol{\mu}_{nkt}^\text{hd}\) to align the 1D distributions with the agent’s local longitudinal (lg) and lateral (lt) directions, instead of aligning with global \(x\)- and \(y\)-axes. Denoting the ground-truth observation at timestep \(t\) for agent \(n\) by \(\boldsymbol{y}_{nt} = (\boldsymbol{y}_{nt}^\text{lg}, \boldsymbol{y}_{nt}^\text{lt}, \boldsymbol{y}_{nt}^\text{hd})\), the likelihood at each timestep for a given agent and mode can be computed as \[ p_{nkt}(\boldsymbol{y}_{nt}) = \text{L}(\boldsymbol{y}_{nt}^\text{lg} \mid \boldsymbol{\mu}_{nkt}^\text{lg}, \boldsymbol{b}_{nkt}^\text{lg}) \cdot \text{L}(\boldsymbol{y}_{nt}^\text{lt} \mid \boldsymbol{\mu}_{nkt}^\text{lt}, \boldsymbol{b}_{nkt}^\text{lt}) \cdot \text{vM}(\boldsymbol{y}_{nt}^\text{hd} \mid \boldsymbol{\mu}_{nkt}^\text{hd}, \boldsymbol{\kappa}_{nkt}^\text{hd}) \label{eq:pertimestep}\tag{7}\] with \(\text{L}(\cdot)\) and \(\text{vM}(\cdot)\) denoting 1D Laplace and von Mises densities, respectively.

To optimize the mixture components for the distance-based metrics of the Argoverse 2 benchmark, DONUT uses the winner-takes-all paradigm and selects the best mode \(k_n^*\) using the average Euclidean distance over all timesteps. This component’s log-likelihood is optimized via \[\mathcal{L}_\text{reg} = -\sum_n \sum_{t=1}^T \log p_{nk_n^*t}(\boldsymbol{y}_{nt}).\] A classification loss optimizes the log-likelihood of the mixture components’ weights at the final timestep \(T\), with \(\bar p\) indicating a no-gradient version of the density: \[\mathcal{L}_\text{cls} = -\sum_n \log \sum_{k=1}^K \pi_{nk} \bar p_{nkT}(\boldsymbol{y}_{nT}).\]

5.2 Probabilistic Training Objectives↩︎

To match our paradigm of training a model to directly optimize a predictive distribution rather than specific metrics, we adjust DONUT and propose two new variants: Traj-NLL and Step-NLL. Instead of using the winner-takes-all objective, Traj-NLL directly optimizes the predictive distribution with the per-timestep likelihood in 7 and minimizes the negative log-likelihood (NLL): \[-\log p(\boldsymbol{Y}) = -\sum_n \log \sum_{k=1}^K \pi_{nk} \prod_{t=1}^T p_{nkt}(\boldsymbol{y}_{nt}).\] In this formulation, the mixture weights \(\pi_{nk}\) are shared across all timesteps \(t\), so each component \(k\) corresponds to a complete trajectory hypothesis.

To increase temporal flexibility, we further propose Step-NLL that defines a mixture for each timestep independently, yielding the negative log-likelihood \[-\log p(\boldsymbol{Y}) = -\sum_n \sum_{t=1}^T \log \sum_{k=1}^K \pi_{nkt} \, p_{nkt}(\boldsymbol{y}_{nt}).\] In this second variant, we introduce time-dependent mixture weights \(\pi_{nkt}\), so that the importance of each component is allowed to vary across timesteps. This increases the expressiveness of the model, as different modes can become more or less relevant over time, but it also weakens the implicit coupling between timesteps that is present when a single \(\pi_{nk}\) is shared across the entire horizon.

Using our policies, we systematically investigate different position distribution families as drop-in replacements for DONUT’s original Laplace distribution. A common choice [4], [14], [34] is the Gaussian distribution. Compared to Laplace, it has lighter tails and a less sharp peak. In preliminary experiments, Gaussians performed poorly, which is why we experiment with two generalizations of the Gaussian instead.

The generalized Gaussian distribution includes an additional parameter that controls the density’s shape. Laplace and Gaussian distributions are special cases of the generalized Gaussian. We further consider discrete scale mixtures of Gaussians with a fixed number of \(J=5\) components that share a single mean but differ in scale. We provide more details in App. 13.

6 Experiments↩︎

We evaluate on the large-scale Waymo motion prediction benchmark [10], which reports a number of window-based and distance-based metrics. It consists of approximately 487k training scenes, with 44k scenes for evaluation and 45k scenes for testing. Models must predict \(K=6\) future trajectories over 8 s with assigned confidences, using 1.1 s of historical input, sampled at 10 Hz.

To assess transferability beyond DONUT-NLL, we apply our metric-specific policies to the open-source trajectory forecasting models MTR [11] and QCNet [2]. MTR was originally evaluated on Waymo and is therefore optimized for window-based metrics. It predicts a \(64\)-component Gaussian mixture and uses non-maximum suppression to select \(6\) trajectories. We report its original NMS outputs as the naive baseline and additionally evaluate our policies by sampling from the full \(64\)-component mixture. By contrast, QCNet targets Argoverse 2 and directly outputs \(6\) trajectory modes parameterized as Laplacians. For both models, we vary the training objective and compare the original WTA losses to our probabilistic objectives (Traj-NLL and Step-NLL).

Our experiments are designed to answer six main questions: (i) are our TraDiE policies effective at turning a single distribution into strong performance across heterogeneous metrics, (ii) do our policies detect poorly calibrated predictive distributions, (iii) how does the training objective (WTA, Traj-NLL, Step-NLL) affect the quality of the learned predictive distribution, (iv) does optimizing for distributions instead of metrics favor different design choices, (v) which position distributions are best suited for our paradigm, and (vi) is the analysis of predictive distributions via policies model-agnostic?

In 1, we report results for the WTA, Traj-NLL, and Step-NLL training objectives, both before and after our metric-specific policies. For both NLL variants, which optimize the predictive distribution directly, we find that our policies greatly improve the corresponding metrics. The distance policy consistently reduces minFDE, and the window policy leads to substantial improvements in soft mAP, mAP, and miss rate. This confirms the first key aspect of our paradigm: given a single predictive distribution, the proposed policies can effectively optimize different metrics without retraining the underlying model.

Applying the distance policy to WTA degrades minFDE compared with naive evaluation, since WTA directly optimizes minFDE rather than a calibrated distribution. The result is therefore expected: if the predictive distribution poorly aligns with observed uncertainty, optimizing the metric under that distribution may not yield good results. This suggests that our policies expose miscalibration rather than merely improving benchmark scores through post-processing.

Comparing the different training objectives, we observe that Step-NLL consistently outperforms both WTA and Traj-NLL once TraDiE policies are applied. This directly supports our central hypothesis: optimizing the predictive distribution directly, via NLL, is more beneficial than optimizing a surrogate metric-specific loss once metric-specific policies are used at evaluation time.

We can further observe that design decisions differ between models trained to optimize metrics and models trained for distributions: without policies, the best objective for minFDE is WTA, which directly optimizes the distance-based loss. By contrast, Step-NLL outputs the best predictive distribution, as it consistently outperforms the other objectives if we apply our policies.

In 2, we investigate how the choice of position distribution influences performance under our paradigm. We compare Laplace, generalized Gaussian, and Gaussian scale mixture distributions, each combined with Step-NLL and evaluate them naively and with our metric-specific policies. Once our policies are applied, the generalized Gaussian distribution consistently outperforms the alternatives across all metrics. This suggests that its additional shape flexibility over the Laplacian is beneficial for learning a well-calibrated predictive distribution that can be effectively exploited by the downstream policy.

As before, the naive evaluation again favors different design choices. For minFDE, naive Laplace performs best, likely because its sharper shape encourages the model to place modes close to the ground-truth endpoint. In contrast, naive Gaussian scale mixtures perform best for soft mAP and mAP, presumably because their smoother and broader predictive distributions improve coverage. The consistent advantage of the generalized Gaussian after policy optimization indicates that it learns a better-calibrated predictive distribution. We provide full ablations over all combinations of losses and distribution families in App. 14.

To evaluate whether our findings extend to other models, we apply our policies to MTR [11] and QCNet [2] and report the results in 3. MTR improves consistently, and even outperforms the original post-processing, indicating that the distribution-aware policies transfer to a different architecture. Changing the training loss only has a minor impact in this setup.

For QCNet, minFDE degrades significantly under the distance policy. This suggests that QCNet has not learned a well-calibrated predictive distribution. Our policies directly optimize the expectation of the metric under the model’s output distribution, so if there is any discrepancy with the real-world uncertainty, the policy selects trajectories that are suboptimal in reality. We attribute this mismatch to QCNet’s position distribution parameterization: its Laplacians are oriented according to the last historical heading, and therefore become misaligned with the future motion direction on curved trajectories. To compensate, QCNet inflates the uncertainty into a more circular shape, which makes the minFDE-optimal endpoints spread out. Window-based metrics are less affected because the distribution’s main mass stays concentrated around each mode.

This is exactly what our policies are meant to expose: if the predictive distribution is miscalibrated, policy optimization cannot find good trajectories for real data. In this sense, policy-optimized evaluation enables a direct diagnostic of distribution quality that naive metric computation can obscure.

In 4, we apply both of our metric-specific policies to DONUT-NLL with the Step-NLL loss and the generalized Gaussian position distribution, and compare with state-of-the-art methods. We find that our policies allow a single model to obtain state-of-the-art performance across both distance-based and window-based metrics with the same model weights, whereas existing methods typically perform well for only one of these when naively evaluated. DONUT-NLL outperforms IMPACT (e2e) [4] on minFDE, while using roughly \(5\times\) fewer parameters. For soft mAP, mAP, and miss rate, it even outperforms ensemble methods while using only a single model. Together, these results validate our proposed paradigm: training a metric-agnostic predictive distribution with probabilistic objectives is an effective way to model future trajectories, and the quality of the predictive distribution can be evaluated on benchmarks using our metric-specific policies.

7 Limitations and Discussion↩︎

Our TraDiE policies provide a mechanism for adapting general-purpose trajectory prediction models that are trained to output calibrated predictive distributions to common trajectory forecasting metrics such as minFDE and soft mAP. This allows a single predictive model to be flexibly adapted to different downstream objectives without retraining. The main drawback is the additional computational cost incurred at inference time. Because our policies are intended for evaluation and not for direct deployment on autonomous vehicles, we did not focus on optimizing efficiency. In practical settings, however, an explicit trade-off between output quality and computational efficiency would be required.

Another limitation is that our results are restricted to Waymo. Preliminary experiments on Argoverse 2 suggest that Step-NLL is prone to overfitting, possibly because the step-wise mixture weights introduce additional modeling flexibility. Nevertheless, on Waymo, DONUT-NLL combined with our TraDiE policies is the only method to date that reaches state-of-the-art performance on both window-based and distance-based metrics without requiring retraining.

8 Conclusion↩︎

In this paper, we propose to shift the focus of trajectory forecasting from metric-specific training to learning well-calibrated predictive distributions and treating benchmark metrics as downstream tasks via metric-specific policies. Concretely, we design sampling-based policies for minFDE, miss rate, and (soft) mAP. We instantiate this framework with DONUT-NLL, a negative log-likelihood variant of DONUT. Our experiments on the Waymo motion prediction benchmark confirm that metric-agnostic probabilistic training, combined with our policies, yields a single model that can be adapted post hoc to diverse metrics while surpassing state-of-the-art results. Importantly, this distribution-centric view changes the preferred design choices: DONUT-NLL outperforms WTA, i.e., metric-specific training. This indicates that centering trajectory forecasting on predictive distributions fundamentally changes how forecasting models should be designed. Extending our metric-specific policies to multi-agent forecasting and implementing continuity-aware objectives are interesting directions for future work.

This work was partially funded by the BMBF project 6GEM (16KISK036K). The authors gratefully acknowledge the computing time provided to them at the NHR Center NHR4CES at RWTH Aachen University (project number p0024673). This is funded by the Federal Ministry of Research, Technology and Space, and the state governments participating on the basis of the resolutions of the GWK for national high performance computing at universities (www.nhr-verein.de/unsere-partner).

9 Additional Proofs↩︎

10 Comparison to HOME [15]↩︎

In the following, we discuss the similarities and differences between our approach and HOME [15] in more detail. Conceptually, both approaches share the idea of first learning a distributional representation over endpoints and then applying metric-specific optimization to extract a finite set of trajectories for evaluation. However, HOME relies on heuristics tailored to improve the empirical performance of its specific model. By contrast, our goal is to define model-agnostic policies that can be applied to any probabilistic forecaster and that correspond as closely as possible to optimizing the exact benchmark metrics under the predictive distribution.

HOME subdivides trajectory forecasting into three steps: (1) learn an endpoint heatmap from history and scene information, (2) optimize for different metrics within the heatmap, and (3) apply a second network to connect the endpoint and the last historical point to retrieve a full trajectory.

The heatmap is rasterized using square cells with a side length of 0.5 m. The target heatmap \(\boldsymbol{M}\) defines a circular Gaussian around the observed endpoint with a standard deviation of 2 m. To train the predicted heatmap \(\hat{\boldsymbol{M}}\), they weight a variant of the focal loss with a squared error for each cell \(i\): \[\mathcal{L}_\text{HOME}(\boldsymbol{M}_i, \hat{\boldsymbol{M}}_i) = -(\boldsymbol{M}_i - \hat{\boldsymbol{M}}_i)^2 \, f(\boldsymbol{M}_i, \hat{\boldsymbol{M}}_i),\] with the focal loss variant \[f(\boldsymbol{M}_i, \hat{\boldsymbol{M}}_i) = \begin{cases} \log(\hat{\boldsymbol{M}}_i) & \text{if } \boldsymbol{M}_i = 1,\\ (1-\boldsymbol{M}_i)^4 \log(1-\hat{\boldsymbol{M}}_i) & \text{otherwise}. \end{cases}\] This objective does not produce a normalized, calibrated probability distribution. In contrast, we train models directly with a negative log-likelihood, yielding a normalized predictive distribution that is more suitable for applying policies that optimize evaluation metrics under this distribution.

As defined in Eq. (2), a miss for an agent \(n\) occurs when none of the \(K\) predicted endpoints lies inside the evaluation window \(W(\boldsymbol{y}_{nT})\) around the ground-truth endpoint \(\boldsymbol{y}_{nT}\). Given a predictive endpoint distribution \(p_{nT}\), the optimal policy for miss rate is therefore to choose \(K\) endpoints that maximize the probability of at least one hit, which is exactly the objective formalized in Eq. (11).

HOME instead applies a greedy algorithm that selects endpoints such that the window around each selected endpoint covers as much probability mass as possible. Note that this is a different objective: instead of finding endpoints covering as many windows as possible, they search for windows covering as many endpoints as possible. For Argoverse 2 [7], where all windows are circular with fixed radius, both formulations coincide. However, their method does not match the metric definition when windows depend on heading or velocity, as is the case for the rectangular, heading-aligned windows used in Waymo [10]. In contrast, our policies are derived directly from the miss rate objective and therefore apply to arbitrary window geometries.

Moreover, HOME does not use the exact benchmark window parameters from Argoverse 2 [7]: instead of performing the optimization with circular windows of radius 2 m, they use 1.8 m as this empirically lowers their miss rate. While this is a reasonable engineering choice for maximizing benchmark performance of a particular model, the algorithm does not optimize the miss-rate metric under the learned distribution as defined by the benchmark. In our framework, the policy is part of the evaluation protocol: we commit to using the exact metric definition without additional tuning, so that differences in benchmark scores can be attributed to the quality of the predictive distribution rather than to metric-specific hyperparameters embedded in the policy.

HOME proposes a custom iterative algorithm derived from \(k\)-means to optimize minFDE. Standard \(k\)-means minimizes the sum of squared Euclidean distances between points and their assigned cluster centers. In contrast, optimizing minFDE requires minimizing the non-squared Euclidean distance, so \(k\)-means does not directly optimize minFDE under the predictive distribution. HOME further modifies the centroid update as \[c_k = \frac{1}{N_k} \sum_{i}\mathbb{I}(d_i^k\leq 3) \frac{\hat{\boldsymbol{M}}_i}{d_i^k}\frac{m_i}{d_i^k}x_i,\] where \(x_i\) are the cell positions within the heatmap \(\hat{\boldsymbol{M}}\), \(d_i^k\) is the distance between cell \(x_i\) and centroid \(c_k\), \(m_i = \min_{k} d_i^{k}\) is the distance from \(x_i\) to its closest centroid, and \(N_k\) is a normalization factor.

Unlike classical \(k\)-means, which updates each centroid using only the points currently assigned to it (regardless of their distance), HOME includes cells from all clusters within a radius of 3 m around the centroid to allow for flexible decision boundaries. Additionally, they apply a custom weighting factor per cell based on the ratio between its distance to the closest centroid and its distance to the centroid being updated. These heuristics are reasonable when designing an end-to-end system tuned for Argoverse, but the resulting endpoints cannot be interpreted as solutions to the minFDE optimization problem under an arbitrary predictive distribution. Instead, these design choices are tightly coupled to their specific model and therefore cannot serve as a general, model-agnostic policy for other forecasting approaches. In contrast, our policy directly optimizes minFDE under the predictive distribution \(p_{nT}\) by choosing endpoints that minimize the objective in Eq. (4). It is therefore directly tied to the metric definition and only requires the ability to sample from the predictive distribution.

11 Policy for minADE↩︎

The minimum average displacement error (minADE) is defined analogously to minFDE, but averages the error over all timesteps \(t \in \{1, \dots, T\}\): \[\text{minADE}(\boldsymbol{Y}_{n}^\text{pos}, \{\hat{\boldsymbol{X}}_{nk}^\text{pos}\}_{k=1}^K) = \min_k \frac{1}{T}\sum_{t=1}^T\|\boldsymbol{y}_{nt}^\text{pos} - \hat{\boldsymbol{x}}_{nkt}^\text{pos}\|_2. \label{eq:minade}\tag{8}\]

In principle, we can apply the same approach as for minFDE, except that we need to optimize full trajectories \(\{\hat{\boldsymbol{X}}_{nk}^\text{pos}\}_{k=1}^K\), where each trajectory is a sequence \((\hat{\boldsymbol{x}}_{nkt}^\text{pos})_{t=1}^T\). This requires models to define a predictive distribution over complete trajectories from which we can draw an arbitrary number of samples. Many current architectures do not support this directly: they either parametrize independent per-timestep distributions [1], [3], [4], [12]–[14], [34], or they directly regress a fixed set of \(K\) trajectories [9], [36], [37]. For this reason, we focus on endpoint-based metrics, which only require the endpoint marginal and are therefore more generally applicable across existing forecasting models.

12 Implementation Details↩︎

The Waymo benchmark evaluates all metrics at 3 s, 5 s, and 8 s and averages the results. Accordingly, we apply the corresponding policies to the predictive endpoint distribution at these three horizons separately, and then combine the selected endpoints into a single trajectory for each mode. Intermediate timesteps do not affect these metrics and are therefore ignored.

A practical difficulty arises because confidences for (soft) mAP must be defined per trajectory for submission to the Waymo benchmark. However, our policies operate per horizon and produce separate confidences for each endpoint. To obtain per-trajectory confidences, we first sort the \(K\) candidate endpoints by their policy-derived confidence in descending order for each horizon. We then construct \(K\) trajectories by pairing endpoints with the same rank across horizons: trajectory \(k\) consists of the \(k\)-th most confident endpoint at 3 s, 5 s, and 8 s. This rank-wise pairing ensures that the endpoints belonging to the same trajectory correspond to similar confidence levels across horizons, so that the single confidence we later assign to the trajectory remains similar to the individual confidences produced by the per-horizon policies.

We then assign a single confidence to each trajectory. Rather than averaging the three per-horizon confidences, we choose the confidence of the 8 s endpoint as the trajectory confidence. The longest horizon is both the most challenging and the most informative part of the forecasting task, and it typically dominates the practical usefulness of the predictions. Using the 8 s confidence therefore makes the trajectory-level ranking used by (soft) mAP as faithful as possible to the policy we would ideally apply at the most important horizon, while keeping the evaluation protocol simple and deterministic.

Note that this issue does not arise for minFDE, as confidences do not affect the metric. Any assignment of endpoints to trajectories yields the same minFDE.

Since intermediate timesteps are not evaluated when submitting to the benchmarks, we keep them undefined. As many current models can only produce distributions or samples for individual timesteps, full trajectories can only be produced by post-processing (, interpolation).

We use \(|S| = 3000\) samples for all Monte Carlo sets. To optimize minFDE, we initialize the \(K=6\) endpoints by sampling from \(S\) and apply the Adam optimizer [41] for 300 steps with a learning rate of 0.2. We repeat this procedure 10 times with different random initializations and select the solution with the lowest empirical objective value.

For DONUT-NLL, we mostly use the same architecture and training parameters as DONUT [1], but add the new loss variants. We preprocessed Waymo such that the resulting structure is similar to DONUT’s preprocessing for Argoverse, but added stop signs and traffic lights (with their last observed state) as additional polygons. To decrease the scene size, we ignore stationary agents that are farther than 3 m from the road, we remove all driveways and road polygons far away from the target agent, and we only keep every fifth point of each map polyline. As DONUT requires full observed history, we apply PCHIP interpolation to obtain full agent histories.

13 Position Distributions↩︎

The Laplace distribution is parameterized by a location \(\mu \in \mathbb{R}\) and a scale \(b > 0\) as \[\text{Laplace}(x \mid \mu, b) = \frac{1}{2b}\exp\left[-\frac{|x-\mu|}{b}\right].\]

With mean \(\mu\) and standard deviation \(\sigma > 0\), the Gaussian distribution is defined as \[\mathcal{N}(x \mid \mu, \sigma^2) = \frac{1}{\sqrt{2\pi}\sigma} \exp\left[-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2\right].\]

The generalized Gaussian uses a location \(\mu \in \mathbb{R}\), a scale \(\alpha > 0\), and a shape parameter \(\beta > 0\), and is given by \[\text{GG}(x \mid \mu, \alpha, \beta) = \frac{\beta}{2\alpha\Gamma(1/\beta)}\exp\left[-\left|\frac{x-\mu}{\alpha}\right|^\beta\right],\] where \(\Gamma(\cdot)\) denotes the Gamma function. Laplace and Gaussian distributions are special cases of the generalized Gaussian family with \(\beta=1\) and \(\beta=2\), respectively.

We also use discrete scale mixtures of Gaussians with a fixed number of \(J=5\) components that share a single mean but differ in their scale. We define \[\text{SMG}(x \mid \mu, \{\omega_j, \sigma_j\}_{j=1}^J) = \sum_{j=1}^J \omega_j \, \mathcal{N}(x \mid \mu, \sigma_j^2),\] where \(\mu \in \mathbb{R}\) is the shared location parameter, \(\sigma_j > 0\) are component standard deviations, and \(\omega_j \geq 0\) are mixture weights satisfying \(\sum_{j=1}^J \omega_j = 1\).

We use each of these distribution families as a direct substitute for the 1D longitudinal and lateral Laplace distributions in DONUT. While the number of parameters needed to specify the distribution differs between families, this only changes the dimensionality of the model’s output head and can be implemented by adjusting the size of the final linear layer at each decoding step.

14 Full Ablations↩︎

In 5, we show the full ablations with all losses and distribution families. Overall, applying our TraDiE policies improves performance compared to naive evaluation, except for WTA, where minFDE degrades. This is consistent with the fact that WTA explicitly optimizes for the endpoint error rather than a properly calibrated predictive distribution.

We further observe that optimizing for the quality of the predictive distribution leads to different preferred design choices than optimizing directly for benchmark metrics. With our policies applied, Step-NLL combined with the generalized Gaussian consistently outperforms all other approaches, whereas naive evaluation would favor different losses and position distributions.

15 Limitations↩︎

15.0.1 Continuity of trajectories.↩︎

A clear limitation of our minFDE and mAP policies is that they generally do not produce temporally continuous trajectories. Downstream applications that require \(K=6\) continuous trajectories that are themselves optimal for minFDE or mAP cannot use these policies directly. However, applicability to downstream control or planning is not the purpose of these particular policies. Our goal is to evaluate the quality of the predictive distribution under the metrics used by the benchmarks. Given a calibrated predictive distribution, any downstream task can define its own policy tailored to that specific use case.

15.0.2 Runtime.↩︎

With default parameters, optimizing (soft) mAP with our policy takes approximately 0.03 s per scene, while minFDE is considerably slower at around 0.26 s per scene. This additional cost is incurred only at evaluation time.

Note that we did not aim for efficiency in this work. Our policies are designed for evaluation, not as AV planning-stack components, and they serve as a proof of concept for tailoring policies to a specific downstream task.

16 Visualizations↩︎

We visualize endpoint distributions together with naive endpoints, endpoints optimized for minFDE, and endpoints optimized for soft mAP in [fig:scene_0] [fig:scene_1] [fig:scene_2] [fig:scene_3] [fig:scene_4] [fig:scene_5] [fig:scene_6] [fig:scene_7] [fig:scene_8] [fig:scene_9] for DONUT and DONUT-NLL. In general, endpoints optimized for soft mAP show greater coverage than endpoints obtained under the minFDE policy. In some cases, they even lie on individual outliers when the windows in the main mode of the distribution are already covered by higher-confidence endpoints ([fig:scene_0]). Such outlier endpoints receive low confidences from our policy and therefore have little effect on the (soft) mAP score.

In [fig:scene_9], we can observe a limitation of winner-takes-all training. Because the straight and right-turn futures have much higher likelihood than the left-turn future, the model places all mixture components along these futures to optimize the WTA objective. At the same time, it is aware of the possibility of turning left and therefore inflates the uncertainty of the modes to account for this alternative. This results in very wide modes that do not reflect the true multimodal structure of the future and thus deviate from a well-calibrated predictive distribution.

in 0,...,9

References↩︎