1 Introduction↩︎

Trajectory prediction plays a pivotal role in autonomous driving and robotic navigation, acting as the bridge between perception and planning [2]–[7]. Accurate trajectory prediction allows an autonomous agent to anticipate how surrounding agents will move in dynamic scenes and to plan safe, socially compliant actions. However, real-world environments are highly interactive: pedestrians, vehicles, and robots influence one another, and forecasting future motions therefore requires understanding the intent and interactions of multiple agents rather than purely extrapolating past positions [8]–[13].

With this motivation, recent studies have begun to leverage the contextual reasoning abilities of large language models (LLMs) for trajectory prediction. Language models offer strong capabilities in contextual reasoning and multi-agent understanding [14]–[18]. Vision–language frameworks and language-driven predictors have explored encoding trajectories, intents, and scene descriptions in a language-like format to improve interpretability and scene-level consistency [19]–[22]. However, the large size and heavy computational requirements of LLMs make them unsuitable for deployment on edge devices, where real-time inference and energy efficiency are essential. This limitation prevents their direct use in autonomous vehicles and mobile robots operating under strict resource constraints. To mitigate these constraints, many works explore parameter-efficient tuning and quantization for LLMs [23]–[28].

In trajectory prediction, sequence-to-sequence (Seq2Seq) encoder-decoder architectures are widely recognized to outperform decoder-only models, as demonstrated by recent state-of-the-art approaches [1]. However, the majority of existing quantization research has concentrated on decoder-only models, particularly within the LLaMA family. Specifically, these works demonstrate that weight-only quantization effectively preserves the performance of LLaMA-like models, whereas quantizing activations often leads to significant degradation [25], [27]. It remains unverified whether this finding holds true for the encoder-decoder structure of Seq2Seq models. Our work addresses this gap by investigating efficient quantization strategies specifically for Seq2Seq architectures. We select BitNet [26] due to its ability to perform extreme model compression down to 1.58-bits. We hypothesize that BitNet will be particularly impactful for Seq2Seq models, as its aggressive compression can be applied to both the encoder and decoder, offering substantial size reductions ideal for edge device deployment. Furthermore, we aim to empirically validate whether a selective quantization strategy—separating weights and activations—is also the optimal approach for Seq2Seq models, drawing parallels from the observations in LLaMA-based research.

Building upon this hypothesis, we propose BitTP (Fig. 1), a quantized reasoning framework for efficient trajectory prediction based on Seq2Seq LLMs. Instead of relying on massive LLMs, we transform a standard language‑based prediction model into a lightweight bitlinear architecture that performs low‑bit operations while preserving reasoning capability. Our approach applies quantization selectively: only the model weights are quantized and activations remain in full precision. This selective design draws inspiration from extreme network quantization and sparsity studies [29], [30]. Through systematic experiments, we find that selective quantization achieves the best trade-off between accuracy and efficiency, while full quantization of both weights and activations severely degrades motion reasoning quality and leads to unstable training behaviors. The proposed design, which builds on established multi-agent reasoning and dynamic interaction models [31]–[34], enables the model to process complete past trajectories and generate coherent multi‑step predictions, maintaining the relational reasoning and interpretability of language‑based approaches while achieving substantial improvements. Our model is evaluated on real‑world trajectory prediction datasets, showing that it maintains prediction performance while being suitable for deployment on resource‑limited robots [35]–[40].

Our main contributions are summarized as follows:

• We propose BitTP, a quantized reasoning framework for edge devices, and conduct a systematic analysis of three selective quantization strategies (weight-only, activation-only, and combined) within a sequence-to-sequence architecture.

• We find that the BitTP-Weight strategy, which quantizes only model weights to 1.58-bit while maintaining full-precision activations, achieves the optimal trade-off. It outperforms the full-precision baseline, reducing ADE by 14.29% and FDE by 20.97%, while significantly reducing memory and latency relative to previous approaches.

• We confirm that activation precision is critical for spatio-temporal reasoning in encoder-decoder models, as strategies involving activation quantization resulted in severe instability and performance degradation. Our work validates the practical feasibility of deploying lightweight, accurate language-based models for real-time trajectory prediction.

2 Related Works↩︎

2.1 Trajectory Prediction in Dynamic Environments↩︎

Trajectory prediction has long been a core problem in autonomous driving and robotics. Classical approaches focus on modeling the geometric evolution of agent trajectories from past motion histories. These methods learn spatial and temporal correlations through recurrent or transformer‑based encoders and often rely on pairwise distance or attention mechanisms to capture agent interactions. Examples include pooling‑based recurrent models, generative adversarial frameworks, graph‑based approaches and transformer architectures [2]–[13], [41]–[43]. While effective for short‑term motion forecasting, such geometry‑centric designs struggle to capture higher‑level semantics such as human intent, social norms and multi‑agent coordination[7]. As urban scenes become increasingly complex, accurate forecasting requires reasoning not only about motion geometry but also about latent intentions and inter‑agent dependencies across the scene.

2.2 Reasoning-based and Language-driven Prediction↩︎

Recent research has explored reasoning‑oriented frameworks to enhance the semantic understanding of dynamic scenes. Graph‑based relational reasoning models treat agents and their interactions as nodes and edges in evolving graphs, enabling context‑aware predictions [32]–[34], [44]–[46]. Transformers have been adapted to jointly encode social interactions and temporal dynamics for multi‑agent prediction [4], [5], [31]. Inspired by the expressive reasoning capability of language models, several works represent trajectories, intents and scene descriptions in a language‑like format to improve interpretability and scene‑level consistency [14]–[22]. However, the computational footprint of these models remains prohibitively large. Even compact LLMs require multi‑gigabyte memory and tera‑scale FLOPs for autoregressive inference, leading to high latency and power consumption. This limitation makes direct deployment of language‑driven predictors impractical for real‑time autonomous systems, motivating research toward more efficient reasoning architectures.

2.3 Efficient and Quantized Reasoning for Edge Deployment↩︎

To enable real‑time prediction under limited compute budgets, recent works have investigated model compression, pruning and quantization techniques. Early studies on binary neural networks and distillation paved the way for aggressive quantization and sparsity without catastrophic performance drops [29], [30], [47]. Lightweight language models such as DistilBERT and ALBERT demonstrate that substantial parameter reductions are possible through knowledge distillation and factorized embeddings [17], [18]. Low‑rank adaptation and parameter‑efficient fine‑tuning methods like LoRA further reduce training and inference costs [23]. Recent advances in quantization, including QLoRA, GPTQ, BitNet, AWQ and 8‑bit matrix multiplication, have shown that large language models can be quantized to low precision while retaining accuracy [24]–[28]. Nevertheless, most of these approaches target general NLP tasks or vision transformers and do not directly address the relational reasoning needed for multi‑agent trajectory prediction. In contrast, efficient trajectory reasoning demands architectures that not only compress parameters but also retain the ability to infer interactions and intent over time. This gap motivates our work: we propose a quantized reasoning framework for trajectory prediction that achieves strong reasoning capability with lightweight computation, enabling deployment on edge robots and autonomous vehicles.

3 Method↩︎

This section elaborates on BitTP, our proposed lightweight quantization framework designed for efficient trajectory prediction on edge devices. Sec. 3.1 introduces the backbone architecture and detailing the quantization mechanism (Alg. 3) used to replace standard nn.Linear layers with our custom BitLinear modules. Subsequently, Sec. 3.2 presents our core technical contribution: three selective quantization strategies, namely BitLinear-Both, BitLinear-Activ, and BitLinear-Weight. We describe the technical formulations and mathematical definitions (Eq. 3 5 ) for each strategy.

3.1 Model Quantization↩︎

Our proposed quantized reasoning framework, BitTP, is a general method designed to quantize standard nn.Linear layers, making it broadly applicable to Transformer-based models. While language-based architectures show strong potential for trajectory prediction, recent work suggests that sequence-to-sequence (encoder–decoder) models are better suited for this task than causal language models (CLMs) [1]. We therefore empirically validate our approach using the T5-small model [48] as a representative encoder–decoder Transformer backbone. BitTP adapts this backbone by replacing all nn.Linear layers within the T5 architecture with our custom BitLinear modules, as described below and in Sec. 3.2.

Preliminaries. The T5-small model is composed of 6 encoder blocks and 6 decoder blocks, as described in Fig. 2 and Tab. 1. Within each block, the standard Transformer components are our quantization targets: each Feed-Forward Network (FFN) module contains 2 nn.Linear layers for the gated activation, and each attention module (self-attention or cross-attention) contains 4 nn.Linear layers for the Q, K, V, and O projections. As summarized in Tab. 1, this architecture logically comprises 97 nn.Linear layers in total: 36 in the encoder, 60 in the decoder, and 1 in the final output projection layer (LM Head), which is tied with the input token embeddings (shared embedding projection).

Quantization Mechanism. To perform adaptation, we utilize a replacement helper function, replace_linear_with_quantization. As shown in Algorithm 3, this main entry function selects the appropriate quantization measures based on the target string, an argument that specifies whether to quantize weights, activations, or both. It then invokes the recursive helper function, _replace_linear_base (Algorithm 3), to traverse the T5 model’s module hierarchy and replace every nn.Linear instance with a BitLinear module. While the logical count of nn.Linear layers is 97 (Tab. 1), we note that due to parameter sharing within the Hugging Face transformers library implementation (e.g., combining query, key, and value projections into a single nn.Linear module), the actual number of unique nn.Linear objects replaced by our function is 73. This dynamic replacement allows us to use the same training and evaluation pipeline (Sec. 4) to systematically compare the performance of each quantization strategy against the full-precision baseline.

\(\dagger\): Results reported in the original papers (may use different batch sizes), \(\dagger\dagger\): Results from our backbone implementation.

3.2 Three Selective BitLinear Strategies↩︎

Our primary objective is to adapt the T5-small backbone for efficient deployment on resource-constrained edge devices. To this end, we replace the standard full-precision nn.Linear layers, which perform the operation \(y = x W^T + b\), with custom BitLinear layers. This adaptation inherits the core quantization principles from BitNet b1.58 [26], which quantizes both weights and activations. However, our approach departs from this; we systematically investigate three selective quantization strategies, visualized in the right side of Fig. 2, to identify the optimal balance between prediction accuracy and computational efficiency. These strategies are: (a) BitLinear-Both, quantizing both weights and activations; (b) BitLinear-Activ, applying activation-only quantization; and (c) BitLinear-Weight, applying weight-only quantization.

Scaling Functions and Quantization. The BitLinear operation is defined by its normalization and scaling procedures. First, all input activations \(x\) are normalized via LayerNorm (\(\text{LN}\)) to produce \(x_{\text{norm}} = \text{LN}(x)\). Second, two scaling factors, \(\gamma\) for activations and \(\beta\) for weights, are computed. As visualized in Fig. 2-(a,b), for 8-bit activation quantization (\(Q_{\text{activ}} = [-128, 127]\)), we employ AbsMax scaling: \[\label{eq:absmax} \begin{align}\gamma = \frac{\max(|Q_{\text{activ}}|)}{\max(|x_{\text{norm}}|) + \epsilon} = \frac{127}{\max(|x_{\text{norm}}|) + \epsilon}.\end{align}\tag{1}\] For 1.58-bit ternary weight quantization (\(Q_{\text{weight}} = \{-1, 0, 1\}\)), visualized in Fig. 2 (c), we utilize AbsMean scaling: \[\label{eq:absmean} \begin{align}\beta = \frac{\max(|Q_{\text{weight}}|)}{\text{mean}(|W|) + \epsilon} = \frac{1}{\text{mean}(|W|) + \epsilon}.\end{align}\tag{2}\] The core quantization function, denoted as \(\text{Quant}_Q(z)\), then transforms the scaled values \(z\) into the target integer range \(Q\) using \(\text{Round Clamp}\) operations. A Straight-Through Estimator (STE) is employed during the backward pass to maintain gradient flow.

Selective Quantization Strategies. Based on these functions, we define three BitLinear variants. In each case, the final output \(y\) is computed by performing the linear operation on the quantized values, followed by a de-quantization step (division by the relevant scaling factors) to restore the original scale.
Bitlinear-Both. This variant mirrors the original BitNet [26] by quantizing both weights and activations, targeting maximum theoretical efficiency: \[\label{eq:both} \begin{align}y = \frac{\text{Quant}_{8}(\gamma \cdot \text{LN}(x)) \cdot \text{Quant}_{1.58}(\beta \cdot W)^T + b}{\beta \cdot \gamma}.\end{align}\tag{3}\] Bitlinear-Activ. This strategy maintains full-precision weights but quantizes the 8-bit activations after LayerNorm, aiming for computational savings during the matrix multiplication: \[\label{eq:activ} \begin{align}y = \frac{\text{Quant}_{8}(\gamma \cdot \text{LN}(x)) \cdot W^T + b}{\gamma}.\end{align}\tag{4}\] Bitlinear-Weight. This strategy keeps activations in full precision while quantizing only the weights to 1.58-bit. This strategy is designed to significantly reduce the model’s memory footprint: \[\label{eq:weight} \begin{align}y = \frac{x \cdot \text{Quant}_{1.58}(\beta \cdot W)^T + b}{\beta}.\end{align}\tag{5}\]

In the following sections, we empirically compare these three strategies and show that BitLinear-Weight achieves the best trade-off between accuracy and efficiency for LLM-based trajectory prediction.

4 Experiments↩︎

In this section, we present a comprehensive evaluation of our proposed BitTP framework. We begin in Sec. 4.1 by detailing the experimental setup, including the widely-used ETH/UCY benchmark, our evaluation metrics, and key implementation details. Next, in Sec. 4.2, we conduct a quantitative and qualitative comparison of our three BitTP strategies against state-of-the-art baselines, analyzing their predictive accuracy. Following that, Sec. 4.3 provides a detailed analysis of the inference cost and memory usage for each configuration. Finally, in Sec. 4.4, we investigate training dynamics by analyzing the model’s sensitivity to different learning rates, confirming the stability of our approach. Furthermore, to demonstrate its suitability for edge devices and on-board computers, we validate the feasibility of deploying our model even in environments without GPU acceleration, which is detailed in Sec. 10 of the Supplementary Material.

4.1 Experimental Setup↩︎

Datasets. We adopt the ETH/UCY trajectory datasets, which contain five scenes: ETH, Hotel, Univ, Zara1, and Zara2. Following the standard protocol, each sequence consists of 8 observed steps (3.2s) and 12 prediction steps (4.8s). We employ a leave-one-out evaluation scheme, training on four scenes and testing on the remaining one.

Metric. We report Average Displacement Error (ADE) and Final Displacement Error (FDE), where ADE is the mean positional error over the prediction horizon and FDE is the error at the final timestep.

Model and Implementation Details. All experiments are conducted using the T5-small backbone (Fig. 2) implemented in PyTorch. BitTP models are trained on a single NVIDIA RTX 3090 GPU with AdamW optimizer and an initial learning rate of \(1\times10^{-4}\) using a linear scheduler without warmup steps. The training epoch is set to 8 with a batch size of 128 or 256. Gradient clipping is applied with \(\|g\|_{max}=1.0\). We utilize a trajectory subword tokenizer trained via BPE [1] on pixel-level trajectories to handle continuous input. Stochastic decoding modes are evaluated with temperature 0.7.

Quantization Settings. BitTP introduces lightweight BitLinear modules for weight-only, activation-only, and combined quantization, as described in Fig. 2-(a,b,c). We evaluated multiple precision levels (1.58-bit, 4-bit, 8-bit) and compared them with a BF16 baseline [1]. During inference, stochastic decoding (temperature=0.7) is reported, with top-k sampling disabled.

4.2 Performance Comparison Across BitTP Strategies↩︎

Tab. 2 compares BitTP models with existing state-of-the-art stochastic trajectory prediction methods [1], [41], [49], [50] on the ETH/UCY benchmark. Methods such as SocialVAE and LMTraj-SUP achieve the strongest full-precision performance, with average ADE/FDE around 0.21/0.33 and 0.22/0.32, respectively. These models represent the performance ceiling for encoder–decoder architectures on this benchmark and provide a solid reference point for evaluating quantized variants.

Before evaluating BitTP, we reproduced LMTraj-SUP [1] under BF16, INT8, and INT4 settings. The reproduced BF16 model (LMTraj-SUP††) shows slightly degraded performance compared to the original numbers (AVG 0.35/0.62 vs. 0.22/0.32). Importantly, both INT8 (AVG 0.34/0.59, \(\Delta\) -0.01/-0.03) and INT4 (AVG 0.34/0.60, \(\Delta\) -0.01/+0.02) configurations maintain nearly identical accuracy, indicating that transformer-based trajectory models remain robust under moderate quantization [24], [28], [51], [52].

The three BitTP strategies display distinctly different behaviors. BitTP-Weight (Fig. 2-(c)) achieves the strongest performance, with an average of 0.30 ADE and 0.49 FDE—improving upon the reproduced BF16 baseline (\(\Delta\) -0.05/-0.13) and outperforming all quantized LMTraj-SUP settings. This suggests that weight-only BitLinear layers effectively preserve spatial–temporal structure while benefiting from reduced parameter redundancy. BitTP-Both (Fig. 2-(a)), however, shows degraded accuracy with an average of 1.27/1.13 (\(\Delta\) +0.92/+0.51), as joint encoder–decoder quantization accumulates noise and disrupts attention-layer activations, consistent with the instability observed during training. Lastly, BitTP-Activation (Fig. 2-(b)) performs the worst, achieving an average of 1.84/3.12 (\(\Delta\) +1.49/+2.50), which reflects substantial instability when only activations are quantized; early activation distortion significantly increases gradient variance and disrupts feature-flow consistency. BitTP-Weight (Fig. 2-(c)) achieves the strongest performance, with an average of 0.30 ADE and 0.49 FDE. This corresponds to a relative error reduction of 14.29% in ADE and 20.97% in FDE compared to the reproduced BF16 baseline (\(\Delta\) -0.05/-0.13), outperforming all quantized LMTraj-SUP settings.

To complement our quantitative analysis, Fig. 4 presents a qualitative visualization of stochastic trajectory predictions from the ETH/UCY benchmark. As shown in the scenes from ETH, HOTEL, UNIV, and ZARA, the trajectories predicted by our BitTP-Weight consistently generate plausible paths that closely align with the ground truth. When compared to the full-precision LMTraj-SUP†† baseline, our lightweight model demonstrates a comparable or superior ability to capture complex multi-agent interactions and anticipate future movements. This visual evidence supports our main finding that selective weight-only quantization effectively preserves the model’s critical spatio-temporal reasoning capabilities.

Overall, BitTP-Weight provides the best accuracy and efficiency, demonstrating that low-bit trajectory transformers are feasible when quantization is applied selectively. The weaker performance of BitTP-Both and BitTP-Activation highlights that stability in encoder–decoder architectures is highly sensitive to the location of quantization, emphasizing the importance of preferring weight quantization over activation quantization in practice.

4.3 Inference Cost Comparison Across BitTP Strategies↩︎

To understand the efficiency of different quantization designs, we compare the inference cost of each BitTP strategy using BitTP-Both as the reference baseline (100% memory and 100% inference cost) in Tab. 3. BitTP-Both jointly quantizes all nn.Linear layers in both the encoder and decoder, and therefore represents the full quantization path without any structural reduction. The relative cost of the other variants can thus be interpreted as the proportion of computation and memory they eliminate with respect to this baseline.

BitTP-Activation applies 8-bit quantization only to activations while keeping all weights in high precision. Since most computation in transformer architectures arises from weight matrices, activation-only quantization reduces the overall load only marginally, resulting in 91.09% memory and 74.68% inference cost relative to BitTP-Both. Importantly, this limited efficiency gain comes at the expense of a severe accuracy degradation (1.84 ADE / 3.12 FDE), as quantizing activations directly distorts intermediate feature distributions and destabilizes attention propagation.

BitTP-Weight, which quantizes all weight matrices to 1.58-bit while preserving full-precision activations, yields substantial efficiency improvements. Weight-only quantization reduces parameter storage and matrix multiplication cost, achieving 53.58% memory and 63.12% inference cost. Despite this reduction, BitTP-Weight maintains strong predictive performance (0.30 ADE / 0.49 FDE), surpassing all other quantized variants and closely matching the reproduced BF16 results.

Overall, these results demonstrate that BitTP-Weight is the most practical and efficient design, offering significant computational savings without compromising accuracy. In contrast, BitTP-Activation provides minimal benefit while severely damaging model stability, underscoring the importance of carefully selecting quantization locations in encoder–decoder architectures.

4.4 Learning-Rate Effects in Encoder-Decoder↩︎

Recent findings on BitNet and other decoder-only low-bit models show that unusually high learning rates can improve optimization stability under aggressive quantization, smoothing the loss landscape and mitigating quantization-induced noise [26]. This behavior contrasts with conventional quantization pipelines—where higher learning rates typically destabilize training—and suggests that weight-only quantization may follow a distinct optimization regime. However, prior analyses have focused almost exclusively on decoder-only LLMs. Whether this phenomenon extends to encoder–decoder architectures, where both sides jointly influence generation, has remained unexamined.

Tab. 4 addresses this gap. Across all learning rates (1e-4, 2e-4, 4e-4), BitTP-Weight remains remarkably stable and consistently outperforms the BF16 baseline, achieving its best accuracy at 1e-4 (0.42 ADE / 0.57 FDE). These results indicate that the high–learning-rate robustness reported in BitNet generalizes naturally to seq2seq transformers: weight-only quantization preserves a smooth and optimization-friendly loss landscape even when both encoder and decoder participate in prediction.

Fig. 5-(a) provides deeper insight into these trends under a fixed learning rate (1e-4). BitTP-Weight converges smoothly and monotonically, nearly mirroring the BF16 (LMTraj-SUP) baseline. This confirms that reducing weight precision does not distort early optimization dynamics or impede gradient flow. In contrast, BitTP-Activation exhibits a step-like training pattern: the loss remains flat for an extended period before abruptly dropping once certain optimization thresholds are crossed. This suggests that activation-only quantization introduces early optimization barriers and may require a warm-up phase before meaningful learning can occur. This behavior aligns with the poor accuracy of BitTP-Activation in Tab. 4. BitTP-Both lies between these two extremes. Its convergence is slower and noisier than BitTP-Weight, yet far more stable than BitTP-Activation. Occasional sharp loss spikes indicate that jointly quantizing both encoder and decoder introduces intermittent instability, likely due to accumulated quantization noise propagating across layers.

Fig. 5-(b) examines how each strategy responds to different learning rates (1e-4, 2e-4, 4e-4). BitTP-Weight remains stable across all settings, with nearly identical convergence trajectories. BitTP-Both also maintains reasonable stability, showing only minor fluctuations even as the learning rate increases.

These findings suggest that bit-linear quantization does not fundamentally impair optimization in encoder–decoder transformers; rather, it can even benefit from moderately increased step sizes, echoing observations from decoder-only BitNet models. Activation-only quantization again stands as the outlier. For BitTP-Activation, higher learning rates magnify instability, producing sharp spikes and abrupt loss drops—showing that perturbations in activation precision are far more disruptive than low-bit weights.

In summary, Weight-only quantization learns reliably in encoder–decoder architectures, mirroring the favorable optimization behavior previously reported in decoder-only models. Joint weight–activation quantization (BitTP-Both) remains usable but introduces intermittent noise, while activation-only quantization (BitTP-Activation) suffers from inherent optimization instability that limits its practical utility.

5 Conclusion↩︎

In this paper, we addressed the challenge of deploying large-scale reasoning models for trajectory prediction on resource-constrained edge devices. We proposed BitTP, a quantization framework that transforms a sequence-to-sequence Transformer into a lightweight bitlinear model, and systematically analyzed three selective quantization strategies: BitLinear-Both, BitLinear-Activation, and BitLinear-Weight. Notably, BitTP-Weight resolves the severe training instability and degraded spatio-temporal reasoning seen in activation-quantized variants (BitTP-Both and BitTP-Activation), ultimately surpassing the full-precision baseline. We hypothesize that weight-only quantization acts as an implicit regularizer, alleviating overfitting in the full-precision model—an effect that may be particularly pronounced in datasets with low trajectory diversity and dominant paths such as ETH. In contrast, activation-involving strategies (BitTP-Both and BitTP-Activ) suffered from severe training instability and performance degradation, suggesting that preserving activation precision is critical for stable spatio-temporal reasoning in encoder-decoder frameworks. In conclusion, our work validates that the BitTP-Weight approach makes it practically feasible to deploy sophisticated, LLM-based reasoning for real-time trajectory prediction on real-world edge devices, such as autonomous vehicles and mobile robots.

Acknowledgements↩︎

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2025-22802992), the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. RS-2025-25420118), the Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea government (MSIT) (No. RS-2025-25442149, LG AI STAR Talent Development Program for Leading Large-Scale Generative AI Models in the Physical AI Domain / No. RS-2025-02219277, AI Star Fellowship Support (DGIST)), and the InnoCORE program of the Ministry of Science and ICT (No. N10260003 / No. 26-InnoCORE-01).

6 Implementation Details and Evaluation Metrics↩︎

This section provides a comprehensive breakdown of the model architecture integration, the specific metrics used for performance assessment, and the hardware protocols employed for efficiency benchmarking.

6.1 Backbone and BitLinear Integration↩︎

We expand on the implementation details of the BitTP architecture described in the main paper. Our backbone utilizes the T5-small encoder-decoder structure, consisting of 6 encoder blocks and 6 decoder blocks (\(d_{model}=512, d_{ff}=2048, \text{heads}=8\)).

Quantization Injection Points. We apply our recursive replacement algorithm to inject BitLinear modules into specific sub-layers of the Transformer blocks. The target modules include:

• Attention Projections: The Query (\(\mathbf{W}_Q\)), Key (\(\mathbf{W}_K\)), Value (\(\mathbf{W}_V\)), and Output (\(\mathbf{W}_O\)) matrices in both Self-Attention and Cross-Attention mechanisms.

• Feed-Forward Networks (FFN): The expansion (\(\mathbf{W}_{i}\)) and projection (\(\mathbf{W}_{o}\)) layers within the position-wise FFNs.

6.2 Latency and Memory Measurement Protocol↩︎

To ensure fair and reproducible comparisons across different quantization strategies, we conducted all efficiency measurements under a standardized environment that reflects our experimental configuration.

• Computing Device: All benchmarks were performed on a single NVIDIA RTX 3090 GPU with \(24\text{GB}\) of VRAM.

• Batch Size: We set the inference batch size to \(B=1\)

• Decoding Settings: Unless stated otherwise, measurements include the autoregressive generation time for the full prediction horizon \(T_{\text{pred}}=12\). Crucially, we employed stochastic sampling with a temperature parameter \(\tau=0.7\) (rather than greedy decoding) to account for the computational overhead of the random sampling process required for generating diverse trajectories.

• Precision & Cache: Computations were performed in mixed precision (BF16/FP32) with the Key-Value (KV) cache enabled to optimize sequential decoding throughput.

6.3 Evaluation Metrics↩︎

We evaluate the predictive performance using standard trajectory prediction metrics. Since our model generates multimodal predictions via stochastic sampling, we report the Best-of-\(K\) metrics, where the sample closest to the ground truth is selected for evaluation. In all experiments, we use \(K = 20\) stochastic samples per prediction. 1000 trajectory samples are generated for each input in the main experiment, while only 1 sample per input is used in the CPU-based experiments in Sec. 10.

Let \(\mathbf{Y}_i \in \mathbb{R}^{T \times 2}\) be the ground truth trajectory for pedestrian \(i\), and \(\hat{\mathbf{Y}}_{i}^{(k)}\) be the \(k\)-th predicted sample among \(K\) generated trajectories. If the dataset provides homography matrices \(H\), all coordinates are projected from pixel space to world coordinates (meters) via a mapping function \(\mathcal{M}(\cdot, H)\) before error calculation.

• Average Displacement Error (ADE): The mean Euclidean distance between the predicted path and the ground truth over all time steps. We report the minimum ADE (minADE) across \(K\) samples, averaged over all \(N\) pedestrians in the dataset: \[\text{ADE} = \frac{1}{N} \sum_{i=1}^{N} \min_{k=1}^{K} \left( \frac{1}{T} \sum_{t=1}^{T} \lVert \hat{\mathbf{y}}_{t, i}^{(k)} - \mathbf{y}_{t, i} \rVert_2 \right)\]

• Final Displacement Error (FDE): The Euclidean distance between the predicted position and the ground truth at the final time step \(T\). Similarly, we report the minimum FDE (minFDE): \[\text{FDE} = \frac{1}{N} \sum_{i=1}^{N} \min_{k=1}^{K} \left( \lVert \hat{\mathbf{y}}_{T, i}^{(k)} - \mathbf{y}_{T, i} \rVert_2 \right)\]

• Peak Memory: The peak VRAM usage required to load the model parameters, measured in Megabytes (MB). This metric directly reflects the compression ratio achieved by quantization.

• Inference Latency: The average wall-clock time (milliseconds) required to generate a full trajectory (\(T=12\)) for a single agent (\(B=1\)).

6.4 Latency and Memory Measurement Protocol↩︎

To ensure fair and reproducible comparisons across different quantization strategies, we conducted all efficiency measurements under a standardized environment that reflects our experimental configuration.

• Computing Device: All benchmarks were performed on a single NVIDIA RTX 3090 (24GB) GPU.

• Batch Size: We utilized a batch size of \(1\), online trajectory prediction scenario

• Decoding Settings: Unless stated otherwise, measurements include the autoregressive generation time for the full prediction horizon (\(T_{pred}=12\)). Crucially, we employed stochastic sampling with a temperature of \(\tau=0.7\) (rather than greedy decoding) to account for the computational overhead of the random sampling process required for generating diverse trajectories.

• Precision & Cache: Computations were performed in mixed precision (BF16/FP32) with the Key-Value (KV) cache enabled to optimize sequential decoding throughput.

7 Quantization Methods: Existing Techniques↩︎

This section elaborates on the theoretical underpinnings and implementation details of the quantization strategies compared in this study. For the post-training quantization (PTQ) baselines, we leverage the optimized kernels provided by the bitsandbytes library [24], [28], which implement vector-wise quantization and normal-float data types. We contrast these with our proposed BitTP, which fundamentally alters the linear projection mechanics.

7.1 INT8 Baseline: Vector-wise Quantization with Decomposition↩︎

For the 8-bit baseline, we employ the LLM.int8() methodology. Unlike standard abs-max quantization which can be degraded by outlier features, this method utilizes a mixed-precision decomposition strategy.

Vector-wise Quantization. The matrix multiplication \(\mathbf{Y} = \mathbf{X}\mathbf{W}\) is approximated by scaling rows of \(\mathbf{X}\) and columns of \(\mathbf{W}\). Let \(C_x\) and \(C_w\) be the scaling constants for the input and weight tensors, respectively. The quantized computation is defined as: \[\mathbf{Y} \approx \text{dequant}\left( \text{quant}(\mathbf{X}) \cdot \text{quant}(\mathbf{W}) \right) = S_x \cdot \mathbf{X}_{\text{int8}} \cdot \mathbf{W}_{\text{int8}} \cdot S_w^T\] where \(S_x\) and \(S_w\) preserve the dynamic range of the activations and weights.

Outlier Handling (Decomposition). To handle emergence of extreme outliers in Transformer activations, the matrix multiplication is decomposed into two parts based on an outlier threshold \(\alpha\): \[\mathbf{Y} = \underbrace{(\mathbf{X}_{out} \cdot \mathbf{W}_{out})}_{\text{BF16}} + \underbrace{(\mathbf{X}_{int} \cdot \mathbf{W}_{int})}_{\text{INT8}}\] Dimensions where \(|x_i| > \alpha\) are extracted and computed in high precision (BF16), while the bulk of dimensions are computed in INT8. This ensures numerical stability but introduces the inference latency overhead observed in our experiments.

7.2 4-bit Baseline: NormalFloat (NF4) Quantization↩︎

For the 4-bit baseline, we adopt the NormalFloat4 (NF4) data type. This approach assumes that the weights of a converged neural network follow a zero-mean normal distribution \(\mathcal{N}(0, 1)\) and employs Quantile Quantization.

Information-Theoretic Optimality. Standard integer quantization relies on linear spacing, which is suboptimal for bell-curved distributions. NF4 constructs quantization levels \(q_i\) such that each bin has equal probability mass. The levels are determined by the quantiles of the normal distribution: \[q_i = Q^{-1}\left(\frac{i}{2^k + 1}\right)\] where \(Q^{-1}\) is the inverse cumulative distribution function (CDF) of \(\mathcal{N}(0, 1)\) and \(k=4\) bits. During the forward pass, these 4-bit stored weights are dequantized on-the-fly to BF16 for computation, prioritizing memory efficiency over arithmetic speedup.

8 Training-Time Behavior↩︎

In this section, we analyze the optimization dynamics of training BitTP variants from scratch (or fine-tuning) compared to the BF16 baseline. We focus on training stability and computational overhead. While the primary goal of BitTP is to reduce inference memory and latency, analyzing training efficiency is crucial for understanding the cost of using BitLinear.

Measurement Setup. We measured the wall-clock training time on a single NVIDIA RTX 3090 (24GB) GPU. The training configuration was standardized with a batch size of 128 and a learning rate of \(1\times 10^{-4}\) over 8 epochs.

Training Duration and Overhead. Tab. 5 presents the total training duration required across five datasets (ETH, Hotel, Univ, Zara1, Zara2). We observe distinct overhead patterns depending on the quantization target:

• Weight-Only Efficiency: The BitTP-Weight variant incurs a relatively minor overhead (avg. \(\sim 20\%\)) compared to the baseline. This is because weight quantization (ternarization) involves element-wise operations that are computationally inexpensive relative to the full matrix multiplication cost.

• Activation Quantization Cost: The BitTP-Both and BitTP-Activ variants show a significant increase in training time (\(\sim 1.8\text{--}1.9\times\)). This substantial overhead is attributed to the dynamic activation quantization process. Unlike weights, activations change every forward pass, requiring on-the-fly calculation of scaling factors (e.g., AbsMean) and rounding operations for every token. In our Python-based implementation, this creates a computational bottleneck that limits training throughput.

Consistency Across Datasets. This trend was consistent across all datasets. For instance, in the Univ dataset, training BitTP-Weight required 12 hours, whereas BitTP-Both required 18 hours (\(1.5\times\)), reinforcing the observation that the computational bottleneck lies primarily in the activation quantization path.

9 Qualitative Results of Forecasting the Trajectory↩︎

This section provides qualitative analyses of language-conditioned trajectory forecasting under different quantization strategies. Fig. 6-8 visualize stochastic predictions on the ETH/UCY benchmark and compare the full-precision LMTraj-SUP\(\dagger\dagger\) model with BitTP-Weight, BitTP-Both, and BitTP-Activ. Together, these figures illustrate how weight-only quantization preserves scene-consistent motion, whereas activation quantization leads to severe degradation in both trajectory generation and language outputs, complementing the quantitative results in the main paper.

Multi-scene comparison. Fig. 6 summarizes qualitative trajectory forecasts across five scenes (ETH, HOTEL, UNIV, ZARA1&2) under different quantization strategies. In all scenes, LMTraj-SUP\(\dagger\dagger\) and BitTP-Weight produce future trajectories that closely align with the ground-truth (GT). In contrast, BitTP-Both and BitTP-Activ fail to match the GT in a much more drastic manner. BitTP-Both still generates trajectories; however, the predicted paths are often clearly unrealistic and significantly displaced from the GT routes, sometimes bending into unrealistic directions or overshooting the intended goal. BitTP-Activ exhibits the most severe failure modes: for many examples, it does not produce a usable trajectory at all, either terminating without valid output or entering an effectively unbounded generation regime. The ZARA example is particularly illustrative—although points are plotted, they stem from such runaway decoding that no coherent trajectory can be drawn.

Input and output in the forecasting task. Fig. 7 and 8 decompose the qualitative results from Fig. 6 by explicitly showing the language-based input and output for representative examples on the ETH/HOTEL and UNIV/ZARA scenes. For each case, the observation is formatted as a question–context pair: the context encodes the past trajectory as a token sequence, and the question requests the future motion of pedestrian 0 over the forecasting horizon. The figures then display both the generated text answer and the corresponding trajectory tokens rendered in the scene. LMTraj-SUP\(\dagger\dagger\) and BitTP-Weight produce well-structured answers that correctly refer to the requested forecast (e.g., horizon length, direction of movement) and generate coordinate tokens that form a coherent, scene-consistent trajectory when visualized. In contrast, BitTP-Both and BitTP-Activ reveal how activation-involving quantization corrupts the decoding process itself: the outputs contain incomplete or repetitive phrases, or excessively long token streams that never settle into a valid trajectory sequence. Fig. 7 and 8, therefore, complement Fig. 6 by making explicit how the same failure patterns seen at the trajectory level already manifest at the language-interface level, while BitTP-Weight maintains reliable question–answer behavior and accurate future-path generation.

10 CPU-based Inference and Cost Evaluation↩︎

Fig. 9 presents the execution log of BitTP in a CPU-only environment, exploring the feasibility of deploying an LLM-based predictor under strict hardware constraints. Despite the inherently massive computational requirements of LLMs, by leveraging Quantization-Aware Training (QAT) and a specialized TQ1 kernel [53] within the llama.cpp [54] framework, BitTP-Weight successfully achieves a throughput of 2.18 sequences/s on the ETH subset. This result confirms that running sophisticated trajectory prediction models entirely on resource-constrained on-board computers without GPU acceleration is practically possible. To measure realistic edge-device latency, our CPU evaluation uses single-sample inference, departing from the standard 1,000-sample GPU protocol used for reproducible benchmarking. While this naturally introduces slight stochastic variance, it effectively demonstrates the model’s practical execution speed under strict resource constraints.

Furthermore, the execution log directly validates our memory efficiency analysis: the peak CPU memory consumption (PSS) was measured at only 698.93 MB (including child processes), confirming the exceptionally lightweight memory footprint of our ternary-quantized model. Collectively, these results demonstrate that BitTP-Weight, when aggressively quantized and paired with a highly-optimized inference engine, significantly lowers the hardware barrier, opening up new possibilities for deploying LLMs on CPU-bound edge platforms.

References↩︎