1 Introduction↩︎

Based on Vision-Language Models (VLMs), Vision-Language-Action (VLA) models have demonstrated remarkable performance across a wide range of embodied tasks [1], [2]. By leveraging the rich knowledge embedded in pre-trained VLMs and fine-tuning on high-quality robot demonstration datasets, VLA models can directly predict actions chunks from raw observations and natural language instructions [3], [4]. However, the high parameter count and computational complexity of VLA models pose significant challenges for their deployment on real robotic systems, where low-latency inference is critical yet onboard compute resources are often limited [5]–[7].

To achieve low inference latency and high control frequency, existing works often deploy VLA models on high-end GPUs such as NVIDIA RTX 4090 [1], [3], [8]–[11], which provide substantial compute and bandwidth resources. However, these devices introduce severe power consumption issues [12], [13]. As shown in Figure 1 (a), using an RTX 4090 can reduce battery life by 6.0\(\times\) compared to onboard devices like Jetson Orin, significantly limiting robot working time. This limits existing VLA works to laboratory demos and makes it difficult to expand its application scenarios [14]–[16]. Even when running such computing devices as online servers, it will also bring extra network latency, and the robot’s activity area will be limited [16], [17]. Therefore, deploying VLA models on low-power onboard devices is essential for embodied applications [16], [18]–[21].

However, due to the limited resources of onboard devices, VLA models suffer from substantial inference latency [16], [22], [23]. As shown in Figure 1 (b), state-of-the-art VLA models such as \(\pi_{0.5}\) [3] achieve an inference latency of approximately 1.4 seconds on a Jetson Orin, resulting in a control frequency around 0.7 Hz. This low frequency leads to slow reaction to environmental changes and long pauses between action chunks [24]–[26]. Even if asynchronous inference [27]–[29] can partially hide inference latency by predicting the next chunk while executing the current chunk, we find that it introduces two critical issues. First, when the VLA finishes predicting an action chunk, the external environment has already changed relative to the image input, leading to a prediction-execution misalignment problem, which becomes even more severe with a long inference time. Second, the robot’s reaction speed remains bounded by the VLA inference time, making it difficult to respond in time for complex embodied tasks.

In this paper, we present Jetson-PI, a method for deploying VLA models on low-power onboard devices via foresight-aligned asynchronous inference. To address prediction-execution misalignment under long inference latency, we train a correction module that predicts future VLM representations conditioned on the actions committed for execution. These predicted representations are then provided as future information to the action expert, enabling it to directly predict actions starting from the future time step. This allows the predicted trajectory to better align with the actual environment at execution time. For the slow reaction time issue, we approach the problem from both scheduling and system perspectives. On the scheduling side, as the future correction module can dynamically adjust the foresight horizon, we can invoke the VLM once and subsequently call the action expert multiple times, thus improving control frequency and reducing reaction time. On the systems side, we design an inference framework for VLA on edge platforms based on llama.cpp, and accelerate VLA inference through computation graph reuse and unrolling to cope with the limited bandwidth. We conduct experiments both in simulation and on real robots across multiple edge devices. On Jetson Orin, Jetson-PI achieves 8.66\(\times\) and 5.41\(\times\) improvements in control frequency compared with naive PyTorch and vla.cpp [30]. And it outperforms VLASH [9] by 14.8% in average success rate on the LIBERO benchmark [31].

2 Background↩︎

Vision-language-action models. VLA models have demonstrated strong generalization across a wide range of embodied tasks. State-of-the-art VLA models, such as \(\pi_{0}\) and \(\pi_{0.5}\), typically adopt a “VLM + action expert” architecture [1], [3], [32]–[37]. During each inference, the current image and language instruction are fed into VLM, which computes and passes the resulting KV cache to action expert. The action expert takes this context along with randomly initialized action noise and uses flow matching to predict an action chunk through multiple denoising steps. The detailed architecture of \(\pi_{0}\) and \(\pi_{0.5}\) is shown in Appendix 8. These VLA models are typically deployed under synchronous inference, where the robot must wait for the current action chunk to finish execution before the next inference [25]. Consequently, as shown in Figure 2, the long VLA latency of onboard devices results in long robot pauses between action chunks.

Asynchronous VLA Inference. As shown in Figure 2, asynchronous inference eliminates pauses between chunks by parallelizing execution of the current chunk with inference of the next chunk [28], [38], [39]. Among them, SmolVLA implements naive asynchronous inference and directly switches to new action chunks after prediction [27], and RTC mitigates discontinuity between chunks by inpainting the new chunk to produce smoother trajectories [25].

However, they still can not solve the prediction-execution misalignment that the environment has already changed relative to the input image of the chunk prediction, which leads to prediction accuracy drop. VLASH predicts the robot’s state at the end of VLA inference (\(q_{t+\Delta}=q_t+a_t+...+a_{t+\Delta-1}\)), where \(\Delta\) is VLA latency in terms of action steps. And uses \(q_{t+\Delta}\) to guide the next chunk prediction [9]. But future robot state alone does not reflect environmental changes, and its effectiveness degrades as inference latency increases. In contrast, we train a future correction module directly predicting future environmental representation. This allows the next chunk prediction to account for how the environment will be affected by the committed actions.

World Action Models. Another line of work related to ours is World Action Models (WAMs), which leverage world modeling, i.e., predicting future environment, to guide action prediction [39]–[43]. [44], [45] explicitly predict future images, and [40], [46]–[48] predict latent representations of future states. However, WAMs typically predict environments at the granularity of entire action chunks or sub-tasks, which is usually used as the desired final state and assists in predicting the current action [8], [39]. Our future correction module can be viewed as a lightweight “world model”. The distinction is that our method can dynamically adjust the foresight horizon at action step granularity, which can predict the environment representation after executing the committed actions and assist in predicting the following actions. This enables addressing the perception-execution misalignment across different computing devices with varying VLA inference latencies.

3 When Asynchronous Inference Meets Onboard Computation↩︎

We conduct a detailed analysis of asynchronous VLA inference and identify two critical challenges that limit onboard deployment: perception-execution misalignment and long reaction time.

Perception-execution Misalignment. Perception-execution misalignment arises because the environment continues to evolve while the VLA model performs inference for the next action chunk. As shown in Figure 3 (a)(b), for naive asynchronous and RTC, with VLA inference latency increases, perception-execution misalignment becomes more severe, leading to task success rate drop. While VLASH maintains high accuracy with low \(\Delta\), for longer latency, predicting future robot state alone is insufficient to capture environmental changes. Therefore, a method that can provide future environmental information for asynchronous inference on onboard devices is needed.

Long Reaction Time. Reaction time refers to the interval between an environmental change and the robot’s corresponding response. For asynchronous inference, as shown in Figure 4, the reaction time in terms of action steps ranges from \(\Delta\) to \(\Delta+L\) [49], [50], where \(L\) is the number of actions executed in each chunk (\(L=3\) in Figure 2 (b)). As shown in Figure 3 (c)(d), increasing \(L\) under varying \(\Delta\) leads to a drop in task success rate. Because a larger \(L\) results in a longer average reaction time. In addition, later actions in a predicted chunk tend to be lower quality, as the environment becomes increasingly uncertain when these actions are executed [49]. To reduce \(L\) without introducing pauses between chunks, a straightforward approach is to perform continuous VLA inference (setting \(L=\Delta\)). However, the reaction time is still bounded by \(\Delta\). Consequently, the objective of reducing reaction time is transformed into lowering inference latency, especially for onboard deployment.

Overview. Figure 4 presents an overview of Jetson-PI. To address prediction-execution misalignment, we propose Foresight-Aligned Asynchronous Correction, which adaptively predicts future environment representation conditioned on the committed actions. This allows the action expert to directly predict actions based on the future environment rather than the outdated observation. For the long reaction time problem, on scheduling side, we propose Confidence-based Scheduling Optimization, which dynamically adjusts the invocation frequency of the VLM and the action expert based on the confidence of the future environment prediction. On systems side, we conduct computation graph reuse, intermediate buffering, and unrolling to accelerate VLA inference.

4 Method↩︎

4.1 Foresight-Aligned Asynchronous Correction↩︎

Requirements for the future correction module. To address the prediction-execution misalignment problem, we train a future correction module that predicts the environment representation at the time when VLA inference completes, guiding next chunk prediction. However, the future correction module must satisfy the following requirements: (1) Action-conditioned prediction: The module must predict future environment states conditioned on the actions committed to be executed, since these actions directly determine how the environment evolves. (2) Lightweight design: The module must introduce minimal additional latency, as extra delay would exacerbate the misalignment problem. (3) Adaptability to varying \(\Delta\): as shown in Table [tab:latency95breakdown95vla], VLA inference latency varies across different hardware platforms and even under different power modes on the same device. The module must adapt to a wide range of \(\Delta\) values to remain effective in diverse deployment scenarios.

The design of Foresight-Aligned Asynchronous Correction is shown in Figure 5. To avoid introducing excessive latency, we do not directly predict future images or correct the KV cache at each layer. Inspired by [47], we take the compressed final-layer output of the VLM at \(t\) and the committed actions, then pass them through future correction module that predicts the compressed VLM final-layer output at \(t+\Delta\). Then pass this lightweight correction item to action expert, enabling action expert to directly predict actions starting from \(t+\Delta\). The parameter size of future correction module is just 40M, which is only 1% of the whole VLA and introduces negligible cost. Architecture details of final-layer state compressor and future correction module are shown in Appendix 9.

To ensure that the future correction module effectively benefits action prediction, the training process must address two objectives: (1) enabling action expert to predict actions starting at \(t+\Delta\) with correction item, and (2) enabling future correction module to accurately predict the future environment representation. To this end, we design a two-stage Correction-aware Training pipeline. In the first stage, we use the ground-truth VLM final-layer state at time \(t+\Delta\) and train the compressor and the action expert using the action loss, allowing them to learn how to extract useful information from the future hidden state. In the second stage, we feed the compressed VLM final-layer output at time \(t\) and the action sequence from \(t\) to \(t+\Delta\) into the future correction module. The module is trained to predict the compressed VLM final-layer output at time \(t+\Delta\). Additionally, the module is trained to output a confidence estimate for its future environment prediction, which will later facilitate scheduling optimization. The loss function of the second stage is \[\label{equ:295stage95loss} L_{predict} = \| \hat{h}_{t+\Delta} - h_{t+\Delta} \|_2,~~~~~ L_{total} = L_{predict} + \lambda \| \hat{c} + L_{predict} \|_2,\tag{1}\] where \(\hat{h}_{t+\Delta}\) and \(h_{t+\Delta}\) is predicted and ground-truth compressed VLM final-layer output at \(t+\Delta\) respectively, \(\hat{c}\) is the predicted confidence. During training, we randomly sample \(\Delta\) to enable the future correction module to adapt to varying inference latencies.

4.2 Confidence-based Scheduling Optimization↩︎

Rethinking the role of VLM and action expert. With future correction module, to reduce reaction time, a naive method is to use only the initial image observation, relying solely on the future correction module to predict environmental changes and use the action expert to output all subsequent action chunks without further VLM invocation. However, the prediction of future correction module introduces error. These errors accumulate over time and lead to a drop in the success rate [35]. Here we reconsider the roles of the VLM and the action expert after introducing future correction module. The VLM serves to observe current environment and thereby improve the accuracy of the future correction module’s prediction. The action expert is invoked to predict actions in real time. Balancing their invocation frequencies is key to achieving low reaction time while maintaining accuracy.

Based on this, we design a Confidence-based Scheduling Optimization approach, where the future correction module serves as the scheduler. Its predicted confidence determines when to invoke the VLM and the action expert. Specifically, as shown in Figure [fig:method2], when the confidence is above a threshold \(\theta\), we consider the predicted future environment information to be sufficiently accurate. So the VLM will be skipped, and the action expert directly predicts actions, improving real-time responsiveness. When the confidence falls below \(\theta\), we invoke the VLM to observe the current environment and update both the KV buffer used by the action expert and the hidden state buffer used by the future correction module for subsequent predictions. In this way, our method adaptively adjusts the invocation frequency of the VLM and the action expert, reducing reaction time without dropping accuracy. We provide an example of confidence evolving over time in experiments and a pseudocode of our algorithm in Appendix 10.

Why don’t we parallelize VLM and action expert? [51] deploys \(\pi_0\) on NVIDIA RTX 4090 and parallelizes VLM and action expert inference to reduce reaction time. However, we find this strategy is ineffective for onboard devices. We analyze VLA computation on different devices using the roofline model [40], [52]. The dominant computation in VLA inference is matrix multiplication. For a matrix multiplication operator with input sizes \(M\times K\) (activation) and \(K\times N\) (weight), the theoretical execution time in FP16 precision is \(max(\frac{M*K*N}{compute~throughput},\frac{2*(M*K+K*N)}{bandwidth})\), where \(M\) represents token length and \(N\) represents model embedding dimension. Specifically, the inference latency is compute-bound when the first term dominates and bandwidth-bound when the second term dominates. The value of \(M\) when the two terms are equal corresponds to the balanced arithmetic intensity. As shown in Table 1, on a high-end GPU such as the RTX 4090, the VLM is compute-bound while the action expert is bandwidth-bound. Therefore, parallelizing the two fully utilizes both compute and bandwidth resources. In contrast, onboard devices typically have much tighter bandwidth. Even VLM computation can not reach the balanced arithmetic intensity, and both the VLM and the action expert become bandwidth-bound. Running them in parallel leads to severe contention for the shared bandwidth, ultimately causing both to slow down significantly.

4.3 System Design↩︎

Unique characteristics of VLA inference. Beyond scheduling optimization, we also accelerate VLA inference through system-level design to further reduce reaction time. To the best of our knowledge, no existing deployment framework is specifically designed for edge-side VLA inference. We build our system upon llama.cpp, a popular inference framework for LLMs edge deployment, and adapt it to the unique characteristics of VLA models, which we identify as follows: (1) Deterministic token length. Unlike LLM decoding, where token length grows progressively, given camera configurations and image resolutions, the token length in VLA inference remains largely constant. The only potential variation is the language instruction, which is typically short and exhibits minimal length fluctuation. (2) Cross-component communication overhead. A VLA model consists of three components—ViT, LLM, and action expert—that communicate with each other. For example, ViT encodes visual observations and then passes them to LLM, and LLM generates KV caches and passes them to action expert. Traditional inference frameworks write such intermediate results back to CPU memory after GPU computation, introducing unnecessary communication latency. (3) Repeated action expert invocations. The action expert adopts a flow matching formulation, which requires multiple denoising steps to produce an action chunk. This repeated invocation of the identical computation graph introduces additional latency.

Based on these opportunities, we conduct the following optimizations. (1) Computation graph reuse. As VLA inference has deterministic token length, we directly reuse the CUDA graph from the first inference to save graph construction time. To ensure that the token length remains constant, we pad the language instruction to a fixed length for every inference. (2) GPU-resident Intermediate Buffer. Because the sizes of the intermediate results between different components are deterministic, we can directly reserve fixed-size buffers in GPU memory, avoiding writing them back to CPU memory. (3) Flow matching unroll. We unroll the flow matching computation graph, fusing multiple denoising iterations into a unified computation graph. This reduces the number of graph invocation calls. Although unrolling increases the graph construction time, this overhead is incurred only once during the first inference step, and the graph is reused in subsequent inferences. Our method is orthogonal to quantization [53]–[56], pruning [57]–[59], and other acceleration techniques. Here, we focus on system-level optimizations that do not alter the model computation.

5 Experiments↩︎

We conduct comprehensive experiments in both sim and real to validate the following questions: (1) How does the performance of Jetson-PI compare to other asynchronous inference methods (RTC and VLASH)? (2) How much improvement in reaction time and control frequency does Jetson-PI achieve on onboard devices compared to standard VLA inference? (3) Does Jetson-PI generalize across different inference latencies (varying \(\Delta\)) and different onboard devices (Orin and Thor)?

5.1 Simulation Experiments↩︎

Experiment Setup. We evaluate Jetson-PI on LIBERO [31] benchmark based on \(\pi_{0.5}\) model. We compare Jetson-PI with three baselines: (1) Sync. serves as an optimal baseline and the inferred delay is set to 0 [3]. (2) RTC freezes the actions committed to execute and inpaints the rest for smoother trajectories [25]. (3) VLASH estimates future robot state to guide action prediction [9].

Main Results. The simulation results are shown in Table 2. On average over four sub-datasets and different \(\Delta\), our method outperforms VLASH and RTC by 14.8% and 3.9%. Notably, as \(\Delta\) increases and the perception-execution misalignment problem becomes more severe, both VLASH and RTC exhibit significant performance degradation. In contrast, Jetson-PI maintains consistently high accuracy, as it adaptively provides future environment information for action prediction across varying \(\Delta\). For \(\Delta=9\), Jetson-PI shows 45.6% and 7.0% on success rate over VLASH and RTC, on average across four datasets, showing the effectiveness of our method. Compared to synchronous inference, our method eliminates pauses between action chunks with negligible accuracy loss.

5.2 Latency Evaluation↩︎

Table 3 presents our latency evaluation. Compared to \(\pi_{0.5}\) inference on naive PyTorch, our scheduling optimization achieves 2.11\(\times\) and 1.87\(\times\) reductions in reaction time on Orin and Thor, because action prediction no longer requires invoking the VLM every time. Our graph reuse method achieves a 2.96\(\times\) reduction in reaction time on Orin by eliminating per-inference graph construction overhead. Our GPU-resident intermediate buffering and flow matching unrolling achieve 1.50\(\times\) and 1.59\(\times\) acceleration for the action expert on the two devices, by avoiding KV communication and repeated graph invocation across multiple denoising steps. Overall, we achieve 8.66\(\times\) and 3.48\(\times\) improvements in control frequency on two devices, enhancing the robot’s reaction speed and task performance. Compared with vla.cpp [30], Jetson-PI also achieves 5.41\(\times\) improvements in control frequency by scheduling and system optimizations.

5.3 Real-world Experiments↩︎

Real-world experiments use PrimeBot Research Institute’s X2-W robot, equipped with three cameras (one head camera and two wrist cameras, all at 224×224 resolution). We deploy the XR-1 model on a Jetson Orin and have the robot execute a complex task: folding clothes. This task is divided into 3 subtasks: “put the cloth in the folding area”, “unfold the cloth and fold it neatly”, and “place the folded cloth in storage area”. The robot executes actions at 15 Hz. As shown in Figure 6, under constrained compute resources, Jetson-PI incurs almost no accuracy loss compared to RTX 4090 deployment, and achieves a clear improvement over naive asynchronous. This shows the capability of Jetson-PI to complete complex tasks on edge devices.

5.4 Analysis of Confidence-Based Scheduling↩︎

An example of confidence evolution. Figure 7 presents an example of how confidence evolves during a single task execution. For most of the task duration, confidence remains high, and the future correction module alone is sufficient to predict environmental changes. At critical steps where environment change is hard to predict, such as during grasping and placing, confidence drops and triggers VLM invocation to ensure accurate environment information. This example demonstrates that our method adaptively balances reaction time and environmental accuracy within a single task.

Ablation on confidence threshold. Figure [fig:exp:ablation] illustrates the impact of the confidence threshold \(\theta\) on robot performance. When \(\theta\) is large, the VLM is invoked more frequently, leading to a slight decrease in control frequency. When \(\theta\) is small, the scheduler invokes the VLM less often, which reduces reaction time but allows slightly larger environmental prediction errors, resulting in a modest drop in average confidence. Nevertheless, across different threshold values, our method achieves improvements in both success rate and control frequency compared to the configuration without scheduling optimization, demonstrating the robustness of our approach.

6 Conclusion↩︎

We presented Jetson-PI, a method for deploying VLA models on low-power onboard devices. We identified two challenges in asynchronous VLA inference: perception-execution misalignment and long reaction time. To address these, we use Foresight-Aligned Asynchronous Correction that predicts future VLM latents to guide action prediction. We use confidence-based scheduling optimization and system accelerations to reduce reaction time. Experiments on Jetson Orin show Jetson-PI achieves 8.66\(\times\) and 5.41\(\times\) improvements in control frequency compared with naive PyTorch and vla.cpp with negligible success rate loss.

7 Limitations↩︎

While Jetson-PI successfully enables VLA deployment on low-power onboard devices, significantly improving battery life and activity area compared to high-end GPU setups, the onboard platform remains fundamentally constrained in compute and bandwidth relative to GPU clusters. As model parameter counts and batch sizes continue to scale, the performance gains of Jetson-PI may not fully close the gap with high-end GPU deployment. Nevertheless, we believe that for mobile robotics applications where power and portability are critical, Jetson-PI offers a practical and efficient solution.

8 Model Architecture of \(\pi\) series Models↩︎

Table 4 presents the architectural parameters of the \(\pi\) series models (\(\pi_0\) and \(\pi_{0.5}\)). The inference pipeline of these models proceeds as follows: Given raw observation (2 or 3 images) and a natural language instruction, the ViT encoder first extracts visual features, which are then projected into the token space of the LLM. The LLM processes the concatenated visual and language tokens, providing the KV cache of each layer that serves as conditioning information for the action expert. The action expert is a Diffusion Transformer (DiT) whose input is randomly initialized action noise. At each DiT layer, the AE performs cross‑attention where the queries are derived from the action noise tokens, and the keys and values are formed by concatenating the KV caches from the corresponding LLM layer with the action noise tokens. Through an iterative denoising process based on flow matching, the action expert progressively refines the noise over a fixed number of steps (e.g., 10 steps during inference) to generate a smooth action chunk. This chunk contains a sequence of future actions that can be executed on the robot. Our Jetson-PI further introduces a lightweight future correction module that predicts the future VLM hidden states conditioned on the actions committed for execution, enabling the action expert to directly generate actions aligned with the future environment under asynchronous inference.

9 Architecture of Future Correction Module↩︎

As illustrated in Figure 8, the future correction module takes two inputs:

• The sequence of committed actions \(a_t, a_{t+1}, \dots, a_{t+\Delta-1}\) that are guaranteed to be executed during the inference window.

• The output hidden state of the VLM final-layer at timestep \(t\).

The module processes these inputs through a series of components. First, the action sequence is passed through an MLP and then into a Transformer block. We take the last token of its output state. Second, the VLM final-layer output at time \(t\) is compressed by a Q-Former block to produce a compact representation. These two branches are then concatenated, projected, and fed into two consecutive Transformer blocks.

From this joint representation, the module splits into two heads:

• Correction head: predicts the compressed VLM final-layer hidden state at time \(t+\Delta t\), which is then provided to the action expert as the future environment information. In the training period, the predicted correction item is aligned with the ground-truth compressed VLM final-layer hidden state at time \(t+\Delta t\) as Equation 1 .

• Confidence head: outputs a scalar confidence value \(\hat{c}\) indicating the reliability of the future prediction.

The entire module is lightweight, adding minimal computational overhead, and is trained via the two-stage correction-aware training described in Section 4.1.

10 Algorithm Details of Confidence-based Scheduling Optimization↩︎

Algorithm 9 shows the details of confidence-based scheduling optimization.

11 Training Details↩︎

Table 5 presents our training hyperparameters, including batch size, learning rate, optimizer, number of training steps, and so on.

References↩︎