1 Introduction↩︎

Rapid progress in vision-language models (VLMs) continually expands the capabilities of robot perception and reasoning, yet transferring these capabilities to robot control remains disproportionately difficult. Vision-language-action models (VLAs), which sit at the intersection of vision-language modeling and robot learning, pursue this transfer by fine-tuning pretrained VLMs on robot demonstrations [1]–[3]. Recent systems have improved action prediction through discretized action tokens, learned action experts, or diffusion action heads [1], [4], but each addition moves the VLA further from its underlying VLM in both architecture and training distribution. This architectural and data complexity makes it difficult to isolate what the pretrained VLM actually contributes to control, limiting both systematic capability analysis and the transfer of advances from the broader VLM community. What would it take to make VLA research as accessible as VLM research?

A central source of complexity in VLM-to-VLA adaptation is an output-distribution mismatch: standard VLA fine-tuning trains a VLM pretrained to generate semantically structured language to instead emit bare action-token sequences such as 4 12 98 3 0 0. These tokens carry little of the linguistic or spatial structure seen during pretraining, so learning to act can degrade the representations that support semantic generalization. This unresolved mismatch is especially important for compact VLAs, which use lightweight VLM backbones for efficient deployment but may be more sensitive to distribution shift during fine-tuning [5], [6]. Recent architecture-free VLM-to-VLA work provides cleaner settings for studying this issue by avoiding separate action experts, but existing approaches either predict bare action tokens [7] or replace numeric actions with language-only descriptions [8]. These choices leave a trade-off between direct executability and alignment with the VLM’s pretrained language distribution.

To address this mismatch, we propose CLAP (Causal Language-Action Prediction), a lightweight VLM-to-VLA fine-tuning recipe that reformulates action prediction as language-conditioned action generation. Instead of predicting bare action tokens directly or replacing actions with language-only descriptions, CLAP generates both in a single autoregressive sequence: before predicting discrete numeric action tokens, the model first generates a natural-language action description, such as “move forward, tilt right, and close the gripper”. Because the full sequence is generated autoregressively, causal attention conditions the numeric action tokens on the preceding language description, making it a conditioning intermediate rather than an auxiliary prediction. CLAP requires no action expert, vocabulary modification, architectural change, or manual annotation, while retaining executable numeric action tokens for precise control.

We evaluate CLAP across manipulation, robustness, and capability-transfer analysis using compact Qwen3.5 backbones at 0.8B, 2B, and 4B parameters [9]. On LIBERO, 2B CLAP achieves 90.8% success with a single fine-tuning epoch, outperforming VLA-0 by +14.9 percentage points under identical conditions and approaching fully-trained VLA-0’s performance, showing that the language-action prefix substantially improves learning efficiency. On LIBERO-PRO, CLAP improves robustness under instruction, object, and spatial perturbations, suggesting that the prefix helps beyond in-distribution imitation. On VLABench, the 2B backbone performs strongest across most capability dimensions and also yields the strongest CLAP policy, indicating that parameter count alone does not determine transfer quality in the compact regime. Together, these evaluations suggest that CLAP provides not only an efficient compact VLA recipe, but also a controlled setting for studying how VLM capabilities transfer to robot control.

In summary, our main contribution is CLAP: a minimal VLM-to-VLA recipe that changes only the output representation, jointly generating language-action descriptions and action tokens without an action expert, vocabulary modification, or architectural change. Consistent with the lightweight design, CLAP reaches \(\approx\) 90% success rate on LIBERO within 6.5 hours on 8 GPU, across 0.8B, 2B, and 4B model sizes. We will release weights and code upon publication.

2 Related Work↩︎

2.0.0.1 Vision-language-action models and action representations.

Vision-language-action (VLA) models adapt pretrained vision-language models (VLMs) [10]–[12] to robot control by combining internet-scale vision-language pretraining with behavioral cloning on robot demonstrations [13]–[15]. A central design choice is how to represent low-level actions within or alongside the VLM. One strategy discretizes continuous actions into tokens, casting robot control as autoregressive next-token prediction [4], [7], [16]. Another adds dedicated diffusion or flow-matching action heads to generate continuous actions directly [1], [2], [17]. CLAP builds on the autoregressive direction, but changes the output representation itself: instead of predicting bare discrete action tokens, it inserts a language-action prefix before the action tokens without adding an action expert.

2.0.0.2 Language grounding and intermediate reasoning.

Prior work has used language, affordances, and intermediate reasoning traces to make robot policies more semantically structured. One line of work conditions manipulation policies or high-level plans on natural language, often grounding instructions through learned affordances or embodied multimodal representations [18]–[21]. More recent VLA methods move language or trace-like intermediates closer to action prediction. VLM2VLA [8] represents actions directly as natural-language text, improving alignment with the VLM pretraining distribution but limiting low-level control precision. Embodied reasoning methods such as ECoT [22], ECoT-Lite [23], and TraceVLA [24] rely on auxiliary intermediate representations, including reasoning tokens and visual traces, before action generation, but often incur additional supervision, prompting, or inference-time overhead. LAP [3] uses language-action descriptions as a pretraining signal, but still relies on an action expert for final action generation. By contrast, CLAP places the language-action description directly in the final autoregressive policy output, where it conditions subsequent discrete action tokens while preserving executable numeric control.

2.0.0.3 Capability preservation.

Fine-tuning VLMs on robot action data can disrupt pretrained representations, degrading capabilities such as multimodal understanding, spatial reasoning, and generalization [25], [26]. One line of work addresses this by insulating or preserving the VLM during policy learning. Knowledge Insulation [25] blocks gradients from the action expert, FLOWER [27] decouples action generation from the VLM via an intermediate-fusion Flow Transformer, and GenVLA [26] uses a dual-encoder design with vision-language co-training. These methods improve preservation, but introduce additional modules, training objectives, gradient-blocking schemes, or co-training data. CLAP instead asks whether capability-preserving VLM-to-VLA adaptation can be improved by changing only the output representation while keeping the VLM architecture and training objective unchanged.

2.0.0.4 Compact VLAs.

Capability preservation is especially important for compact VLAs, where efficient deployment must be balanced against limited model capacity. SmolVLA [5], TinyVLA [6], and VLA-Adapter [28] improve efficiency through lightweight backbones, adapter-based tuning, or specialized policy modules. However, they either focus on a specific model scale or rely on additional policy/action components. In contrast, CLAP applies the same architecture-preserving recipe across 0.8B, 2B, and 4B Qwen3.5 backbones, enabling controlled analysis of VLM-to-VLA transfer across compact scales.

3 Preliminaries↩︎

Given an image observation \(o\) and a natural-language instruction \(l\), an autoregressive VLA models a robot policy \(\pi_\theta\) by predicting an action sequence. Each \(h\)-step action chunk \(\boldsymbol{a} = (a_1, \dots, a_h)\) consists of \(h\) sequential \(7\)-DoF commands, where each \(a_t \in \mathbb{R}^7\) encodes end-effector translation, rotation, and gripper state. Following prior work [7], each action dimension is discretized into \(K=1000\) integer bins and represented as text tokens using the VLM’s existing vocabulary. The policy predicts the flattened action-token sequence autoregressively: \[\pi_\theta(\boldsymbol{a} \mid o, l) = \prod_{i=1}^{7h} \pi_\theta(a^{(i)} \mid a^{(<i)}, o, l), \label{eq:vla0-policy}\tag{1}\] where \(a^{(i)}\) denotes the \(i\)-th token in the flattened action sequence and \(a^{(<i)}\) denotes all preceding action tokens. The VLM backbone is thus fine-tuned with the same next-token prediction objective as language pretraining, while its outputs can be directly decoded into robot commands. However, the fine-tuning targets are bare integer sequences that lack the semantic structure of natural language, diverging from the distribution the backbone was pretrained to generate. We refer to this shift as output-distribution mismatch [3], [8]. CLAP mitigates this mismatch by incorporating a language-grounded action description into the autoregressive output.

4 CLAP: Causal Language-Action Prediction↩︎

We use the output-distribution mismatch introduced in Section 3 to design a simple training recipe for compact VLA policies. Our goal is to retain the semantic generalization of the pretrained VLM while producing directly executable low-level robot actions, without requiring an action expert, vocabulary extension, or large-scale robot pretraining. 2 provides an overview.

4.0.0.1 Language-action representation.

The central design question in converting a pretrained VLM into a VLA is what action representation to use. Representing actions as bare numeric tokens yields compact, directly executable outputs, but moves generation away from the VLM’s pretrained language distribution. A promising alternative, introduced by LAP [3], is to express actions as natural-language descriptions, preserving closer alignment with the VLM’s pretrained generation behavior. However, language-only action descriptions are too verbose to specify precise low-level commands efficiently, and still require an external action generator to produce executable robot actions [3], [25]. CLAP addresses this trade-off by prepending a short language-action description to the numeric action tokens within a single autoregressive sequence, extending Eq. 1 as \[\pi_\theta(d, \boldsymbol{a} \mid o, l) = \underbrace{ \prod_{j=1}^{|d|} \pi_\theta\bigl(d^{(j)} \mid d^{(<j)}, o, l\bigr) }_{\text{language-action prefix}} \cdot \underbrace{ \prod_{i=1}^{7h} \pi_\theta\bigl(a^{(i)} \mid a^{(<i)}, d, o, l\bigr) }_{\text{numeric action tokens}}, \label{eq:clap-policy}\tag{2}\] where \(d\) is a natural-language description of the action chunk \(\boldsymbol{a}\). During training, the description is generated deterministically from the ground-truth action chunk \(\tilde{\boldsymbol{a}}\) by a fixed template as \(\tilde{d} = \texttt {T}(\boldsymbol{\tilde{a}} )\). At inference time, the model generates both \(d\) and \(\boldsymbol{a}\) autoregressively. This representation satisfies three desiderata: (i) semantic alignment with the VLM’s pretrained distribution, (ii) direct executability without an external action generator, and (iii) token efficiency for real-time control.

4.0.0.2 Template-based language-action prefix.

We instantiate the language component of this representation with a deterministic template. For each \(h\)-step action chunk, CLAP prepends a compact natural-language summary of the intended motion, such as "move forward 3 cm, tilt right 1 degree, close gripper". This prefix summarizes the chunk-level translation, rotation, and gripper command, while the following numeric tokens encode precise low-level action values. Because the prefix is generated from the ground-truth action chunk using a fixed template, CLAP requires no manual annotation beyond the original robot demonstrations. 1 shows an example output sequence generated from an input image observation and language instruction.

4.0.0.3 Joint generation and training.

Given the template-derived prefix, CLAP trains the model to predict a single concatenated sequence: language-action description tokens followed by numeric action tokens. We optimize the standard autoregressive cross-entropy objective \[\mathcal{L}_{\text{CLAP}} = -\mathbb{E}_{(o,l,a)\sim\mathcal{D}} \left[ \log \pi_\theta(\tilde{d} \mid o,l) + \log \pi_\theta(\tilde{a} \mid o,l,\tilde{d}) \right],\] where \(\mathcal{D}\) is the robot demonstration dataset and \(T\) is the fixed template. Because the sequence is generated autoregressively, each action token causally attends to the preceding language-action prefix, making the prefix a conditioning intermediate rather than an independent auxiliary output. CLAP uses the same VLM backbone, tokenizer, output head, and cross-entropy loss for both language and action tokens, introducing no action expert, auxiliary head, or vocabulary extension.

4.0.0.4 Action masking augmentation.

We consider an optional augmentation that changes the teacher-forced input context without changing the CLAP target sequence. Specifically, we randomly replace a subset of input action tokens with a “?" placeholder while keeping the original tokens as prediction targets. The model must therefore infer the masked values from the image observation, instruction, language-action prefix, and remaining action context. For each sample, the masking fraction is drawn from \(\mathrm{Uniform}(0,p_{\max})\), with a 40% probability of applying no masking. Full details are provided in Appendix. We study the effect of masking in Section 5.2; unless otherwise stated, CLAP uses the unmasked configuration.

5 Experiments↩︎

5.1 Experimental Setup↩︎

5.1.0.1 Benchmarks.

We evaluate CLAP on three benchmarks targeting complementary aspects of VLA performance: (1) in-distribution manipulation, (2) out-of-distribution robustness, and (3) VLM capability analysis. LIBERO [29] evaluates in-distribution manipulation under the standard simulated benchmark protocol; we report results on the four LIBERO suites, covering spatial reasoning, object manipulation, goal-conditioned behavior, and long-horizon tasks. LIBERO-PRO [30] evaluates out-of-distribution robustness by perturbing the original LIBERO tasks, including novel visual object instances, object relocation, instruction rephrasing, and task-configuration changes. VLABench [31] evaluates language-conditioned robot reasoning capabilities; we use it to measure the base Qwen3.5 backbone’s planning and spatial reasoning capabilities before robot fine-tuning, and to analyze how these capabilities relate to downstream VLA performance. Full benchmark details are provided in Appendix.

5.1.0.2 Comparisons.

Our primary baseline is VLA-0, reproduced using the same Qwen3.5 backbones, LIBERO training data, and number of gradient steps. This matched comparison isolates the effect of the language-action prefix. We also report results from large robot-pretrained VLAs, including OpenVLA [4], \(\pi_0\) [2], and SmolVLA [5], as contextual reference points. The gray rows in Table 2 indicate methods evaluated under different backbones, training budgets, or robot-pretraining data, and therefore should not be interpreted as matched comparisons.

5.1.0.3 Implementation.

We instantiate CLAP with Qwen3.5 backbones at 0.8B, 2B, and 4B parameters. All models are trained for one epoch on the LIBERO demonstration data across all four suites with full-parameter fine-tuning, using the same action discretization and fine-tuning protocol as VLA-0. Unless otherwise stated, CLAP uses the language-action prefix without additional augmentation. We study random action masking as an optional augmentation in Section 5.2; the unmasked variant is used as the default configuration. Full hyperparameters are provided in Appendix.

5.2 Learning Efficiency Results↩︎

\(\dagger\) SmolVLA uses community-scale robot pretraining; all sizes in Appendix. \(\ddagger\) Original VLA-0 uses Qwen2.5-VL-3B and 8 epochs [7]

.

5.2.0.1 Does the language-action prefix improve learning efficiency?

Table 2 evaluates performance after one training epoch, isolating the effect of the output representation under matched backbones, data, and training budget. CLAP consistently improves over VLA-0 across all model sizes, with average gains of +13.5, +14.9, and +20.7 points at 0.8B, 2B, and 4B, respectively. At 2B parameters, CLAP achieves the best overall result, reaching 90.8% average success and outperforming VLA-0 by +14.9 points. The gains are broad across task suites: 2B CLAP improves Spatial from 77.8% to 93.0%, Object from 86.6% to 97.4%, Goal from 77.0% to 90.8%, and Long from 62.4% to 82.0%. For context, 2B CLAP also exceeds the reported averages of SmolVLA and OpenVLA and approaches stronger robot-pretrained systems such as \(\pi_{0.5}\), although these comparisons are not controlled for backbone, data, training budget, or pretraining. These gains come from changing the output representation rather than adding data, model capacity, or architectural components. VLA-0 learns a direct mapping from image-language inputs to bare integer action tokens, which provide precise supervision but little semantic structure. CLAP instead inserts a language-action description before the numeric tokens, giving the model an intermediate target closer to its pretrained generation behavior.

5.2.0.2 Does action masking provide additional gains?

Table 3 ablates whether action masking improves over the language-action prefix alone. Masking is not uniformly beneficial: it reduces average success by 3.9 points at 0.8B and 1.7 points at 2B, but improves the 4B model by 3.2 points, from 84.9% to 88.1%. The drop at smaller scales is mainly due to lower Long-horizon performance, suggesting that 0.8B and 2B models benefit from direct access to the full action-token context. In contrast, the 4B model appears better able to exploit the additional reconstruction constraint imposed by masking. These results suggest that action masking should be treated as a validation-dependent augmentation rather than a core component of CLAP.

5.3 Generalization and Capability Transfer Results↩︎

3pt

5.3.0.1 Do CLAP’s learning gains transfer out of distribution?

Table 4 assesses CLAP’s robustness to out-of-distribution perturbations on LIBERO-Pro. CLAP consistently improves over VLA-0 across model scales, with average OOD gains of +5.5, +11.1, and +10.9 points for unmasked CLAP at 0.8B, 2B, and 4B, respectively. The most striking improvement appears on the Spatial suite with novel visual instances. For 4B models, unmasked CLAP improves over VLA-0 by +42.6 pt, while the masked variant improves by +54.4 pt. These results suggest that CLAP improves OOD robustness beyond what is captured by standard in-distribution evaluation on LIBERO. Notably, action masking substantially enhances OOD robustness at 4B scale, despite providing only mixed benefits on LIBERO. This suggests that masking can reduce reliance on action-token shortcuts that become brittle under distribution shift.

5.3.0.2 Does bigger VLM capability mean better transfer?

Figure 3 presents VLABench performance of Qwen3.5 backbones prior to robot fine-tuning, evaluated under 1-shot prompting with and without Chain-of-Thought (CoT) [32]. Across most categories, performance generally improves with model size. However, on Complex and Physics Law tasks, the 0.8B and 2B models match or exceed the 4B model under both prompting conditions. This non-monotonic trend persists downstream: CLAP 2B also outperforms CLAP 4B on LIBERO (Table 2). Base VLM capability thus influences VLA performance, yet parameter count alone does not determine transfer quality in the compact regime.

5.3.0.3 Does reasoning help compact VLAs?

Reading the same evaluation along the CoT axis, we find that removing CoT improves performance in most settings; on Physics Law tasks, the 0.8B model even exceeds the larger 2B and 4B models. Output analysis suggests that longer reasoning traces can introduce format violations or loops that diverge from actionable predictions, consistent with recent reports that extended reasoning may weaken visual grounding [33]. CLAP avoids relying on free-form reasoning at inference time: its structured language-action template provides a compact intermediate representation before numeric action prediction, keeping smaller variants competitive and the approach attractive for compact, deployment-friendly settings.

6 Conclusion↩︎

We presented CLAP, a minimal VLM-to-VLA fine-tuning recipe that addresses output-distribution mismatch by prepending a natural-language action description to each numeric action sequence. With single-epoch training on a single 8-GPU node (under 6.5 hours for all model sizes), 2B CLAP achieves 90.8% on LIBERO, outperforming size-matched VLA-0 by +14.9 points, while also improving OOD robustness on LIBERO-PRO. Multi-scale evaluation across 0.8B, 2B, and 4B reveals that parameter count alone does not determine VLM-to-VLA transfer quality, with the 2B backbone consistently outperforming the 4B variant. We believe making VLA fine-tuning as lightweight and transparent as standard VLM fine-tuning is a necessary step toward bringing the VLM community’s rapid progress to robot learning.

7 Limitations↩︎

We limit our study to a single VLM family and limited benchmark suites. Generalization to other backbones, embodiments, and real-world settings remains to be validated. On the inference side, autoregressive token generation is inherently slower than parallel action decoding (e.g. diffusion or flow-matching heads), but deployment techniques in the LLM community, such as speculative decoding and quantization, are directly applicable and remain unexplored in this work. As CLAP preserves the VLM backbone unchanged, VLM-side advances such as distillation, quantization, and new backbone architectures can be incorporated without redesigning the action pipeline.

Appendix↩︎

8 Multimedia Material↩︎

Please see the video with audio alongside with this PDF. We enclose multimedia material consisting of a graphical abstract summarizing the CLAP framework and real-robot (UR5e) rollouts.

9 Real-Robot Deployment↩︎

The main experiments evaluate CLAP on simulated benchmarks (LIBERO [29], LIBERO-PRO [30]). Here we provide a preliminary assessment of whether our direct VLM-to-VLA recipe transfers to a physical robot. We deploy CLAP on a UR5e manipulator for cluttered pick-and-place, fine-tuning on a limited real-robot demonstration set to examine how model scale affects task performance in real-world scenarios.

9.1 Real-Robot Setup↩︎

4 shows a real-robot setup to validate our model. We use a UR5e collaborative robot arm equipped with a Robotiq 2F-85 parallel gripper. A wrist-mounted RealSense D435 provides an eye-in-hand view, and a second RealSense D435 provides a fixed third-person view. CLAP runs on a workstation with an NVIDIA GeForce RTX 4090 GPU, served via FastAPI, and queried by a ROS client at inference time. The ROS client runs a synchronous observe-query-act loop, at each query, the current side- and wrist-camera images and the language instruction are sent to the inference server via an HTTP POST request, which returns an eight-step action chunk. Each action encodes a Cartesian end-effector delta (position, axis-angle orientation, and gripper state). The client executes each eight-step action chunk in open loop before re-querying the VLA. At the controller’s 15 Hz rate, this yields an effective query rate of \(\sim\) 1.9 Hz.

9.2 Data Collection.↩︎

During data collection, we collect demonstrations on the UR5e via VR controller-based teleoperation. The operator controls the robot using an HTC Vive controller, whose motion is mapped to Cartesian end-effector delta commands at 50 Hz. Data is recorded from two RGB camera streams, consisting of a fixed external camera and a wrist-mounted camera, together with the end-effector pose, gripper state, and a natural-language instruction at each timestep. Each episode lasts up to 1500 steps. For each episode, the scene layout is generated from a seeded episode plan. All three objects (a carrot, a tomato, and a cube) are present simultaneously and are placed at three of five discrete table positions. We use 50 distinct object-to-position assignments from this layout space. Before each demonstration, the operator arranges the scene according to the pre-generated layout. We collect 120 demonstrations in total, with 40 episodes for each target object. The instruction template is randomized per episode from four templates: “place {target} on the plate,” “put {target} onto the plate,” “pick up {target} and place it on the plate,” and “grasp {target} and drop it onto the plate.”

9.3 Training Protocol↩︎

We fine-tune Qwen3.5-0.8B and 2B directly from their pretrained VLM checkpoints using the CLAP framework on the 120 real-robot demonstrations. Notably, we do not initialize from the LIBERO-finetuned models used in the main experiments: because the base VLMs are pretrained on real images, we focus on assessing how much performance can be extracted from VLM-to-VLA adaptation using only real-world data, without an intermediate simulation fine-tuning stage.

9.4 Task Design and Evaluation↩︎

We evaluate pick-and-place tasks with three training objects (carrot, cube, tomato) under multiple instruction templates. 5 summarizes all objects used in the evaluation. Each model is tested under two conditions:

• In-domain (ID): Object sets and layouts seen during training.

• Out-of-domain (OOD): Unseen distractor objects (a silver watering can, an orange toy car) are introduced into the scene.

Table 6 reports success rates over 20 trials per condition. The 2B model achieves the highest success rate (60%) followed by 0.8B (35%), consistent with simulation results where the 2B backbone also outperforms the 0.8B model. The overall success rates are substantially lower than in simulation, reflecting the additional complexity of real-world perception and control with only 120 training demonstrations. Under OOD conditions with unseen distractor objects, the 2B model maintains its in-domain success rate (60%), while the 0.8B model drops substantially to 10%, suggesting that the larger backbone is more robust to visual distractors. This capacity gap is substantially wider than observed on LIBERO (0.8B: 89.6%, 2B: 90.8%), suggesting that simulation benchmarks may underestimate the benefit of model capacity, and that real-world task complexity may correlate more strongly with backbone size.

The results suggest two complementary directions for future work. The first is to further explore the minimal-recipe regime: improving real-world performance with limited demonstrations through better data augmentation, input robustness, or more effective use of the pretrained VLM’s capabilities, without introducing additional architectural components. The second is to scale up robot data while retaining the language-action representation [33], and investigating whether CLAP’s zero-modification architecture benefits similarly from larger data remains an open question.

10 Inference Latency↩︎

Table 7 reports end-to-end inference latency per policy query, measured as the mean over ten consecutive queries after one warm-up query on an RTX 4090. The 0.8B and 2B models exhibit near-identical latency, indicating that inference time is dominated by token generation overhead rather than model size at these scales. The 4B model is moderately slower (\(\sim\)​6.0 s). With \(h{=}8\) (open-loop execution of the full action chunk), effective control rates range from 1.3 to 1.9 Hz.

Comparing CLAP 0.8B against VLA-0 0.8B on the same backbone isolates the overhead of the language-action prefix: CLAP adds approximately 1.0 s per query (+32%), corresponding to the additional \(\sim\)​149 tokens generated for the prefix. GPU memory consumption is identical, confirming that the prefix introduces no architectural overhead.As CLAP preserves the VLM backbone unmodified, standard LLM acceleration techniques such as speculative decoding and quantization are directly applicable and could substantially reduce this overhead.

11 Training Details↩︎

11.1 Training Hyperparameters↩︎

In Section 5 of the main manuscript, we fine-tune Qwen3.5-0.8,2,4B VLM with full-parameter updates on 8 \(\times\) NVIDIA H200 GPUs. Each GPU processes a micro-batch of 16, yielding a global effective batch size of 128 per optimizer step. We define one epoch as a single pass over the LIBERO-spatial, object, goal, long training set under our augmentation pipeline, which in our configuration corresponds to 17,000 gradient steps (about 2.18M training samples).

11.2 Loss Convergence vs. Task Saturation↩︎

Section 5.2 of the main text reports LIBERO performance after a single training epoch (approximately four hours). In extended runs up to six epochs, 6 demonstrates that the closed-loop success saturates at roughly 90% after the first epoch; subsequent epochs continue to reduce training loss but do not improve task success, suggesting overfitting to the demonstration distribution. We therefore adopt the single-epoch checkpoint throughout.

12 Evaluation Protocol↩︎

Each observation encodes two camera views (agentview and wrist) tiled into a single \(224\!\times\!224\) image. The model produces the full <think>…</think> action_tokens sequence in one autoregressive pass using Qwen3.5’s built-in extended-thinking mode (enable_thinking=True); only the 56 integer tokens after the closing </think> tag are forwarded to the robot controller. All eight predicted action steps execute open-loop before the model is re-queried (action horizon \(h{=}8\)).

13 CLAP Language Target Construction↩︎

During training, the language-action prefix is generated deterministically from each ground-truth action chunk using a fixed template-based method [3] that summarizes steps of numerical actions into a short natural-language summary.

13.0.0.1 Inputs.

Let \(\boldsymbol{a} \in \mathbb{R}^{7 \times h}\) denote an action chunk of horizon \(h\), where each \(a_t = (\delta x_t, \delta y_t, \delta z_t, \delta \phi_t, \delta \theta_t, \delta \psi_t, g_t)\) contains end-effector translational deltas in meters, rotational deltas in radians (roll, pitch, yaw), and a gripper state \(g_t \in \{0, 1\}\).

13.0.0.2 Aggregation.

We sum the per-step deltas across the chunk and convert to human-readable units: \[\begin{align} \Delta x &= \mathrm{round}\!\left(100 \sum_{t=1}^{h} \delta x_t\right) \,\,\text{[cm]}, \qquad &\text{and analogously for \Delta y, \Delta z,} \\ \Delta \phi &= \mathrm{round}_{10}\!\left(\frac{180}{\pi} \sum_{t=1}^{h} \delta \phi_t\right) \,\,\text{[deg]}, \qquad &\text{and analogously for \Delta \theta, \Delta \psi,} \end{align}\] where \(\mathrm{round}_{10}\) rounds to the nearest multiple of \(10^{\circ}\). The gripper bit is taken from the last step, \(g_H \geq 0.5 \Rightarrow \texttt{open}\), otherwise close.

13.0.0.3 Phrasing.

Each non-zero aggregated component is mapped to a phrase whose direction word is selected by the sign of the underlying delta:

The selected phrases, followed by the gripper phrase, are joined with ", " into a single sentence \(d\).

13.0.0.4 Final target.

The CLAP target is the language summary wrapped in <think>...</think> tags and followed by the same discretised numeric action tokens used by VLA-0: \[\texttt{<think>}\,d\,\texttt{</think>}\,\, n_1\,n_2\,\ldots\,n_{7\cdot h}\] where \(n_i\) are the per-dimension action bin indices. Concretely, for \(h=8\) with a small forward and downward motion and the gripper closing, the target looks like

At inference, the prefix up to and including the last </think> is stripped and the remaining numeric tokens are decoded back to a continuous action chunk.

14 Action Masking Details↩︎

Following prior work [7], each action chunk is discretized into \(1000\) bins per dimension and serialized as space-separated integers. Cross-entropy loss is computed only over the assistant turn; all prompt-side tokens (system message, images, instruction, and padding) are masked out. As an optional augmentation, a random fraction of input action tokens are replaced with a ? placeholder while keeping the original values as prediction targets, forcing the model to reconstruct masked actions from visual and linguistic context. For each sample, the masking fraction is drawn from \(\mathrm{Uniform}(0,\, p_{\max})\) with \(p_{\max}{=}0.4\). For CLAP, this augmentation is restricted to the numeric tail following to avoid misalignment within the language-action prefix. At inference, a logits processor constrains generation to digits, spaces, and eos; in CLAP mode this constraint is suspended during the span and re-enabled for the numeric tail.

15 Benchmark Details↩︎

15.0.0.1 LIBERO.

LIBERO [29] is a simulated manipulation benchmark comprising four task suites: Spatial, Object, Goal, and Long. Each suite contains 10 tasks with 50 demonstrations per task. We follow the standard evaluation protocol and report success rates averaged over 20 rollouts per task.

15.0.0.2 LIBERO-PRO.

LIBERO-PRO [30] extends LIBERO with four perturbation types designed to stress-test out-of-distribution robustness. Obj replaces objects with novel visual instances of the same category. Pos relocates objects to positions unseen during training. Sem rephrases task instructions while preserving semantics. Task modifies the task configuration itself (e.g., goal location or object arrangement), representing the hardest perturbation type. All four perturbation types are applied to each of the four LIBERO suites.

15.0.0.3 VLABench.

VLABench [31] evaluates task planning across 45 tasks covering six capability dimensions: Common Sense & World Knowledge (background knowledge about object properties and everyday scenarios), Spatial Understanding (precise spatial reasoning about object positions and orientations), Mesh & Texture Understanding (distinguishing objects by shape or surface appearance), Semantic Understanding (language-grounded category-level reasoning), Physics Law (tasks governed by physics constraints such as gravity, support, and rigidity), and Complex (multi-step inference combining multiple dimensions). Each dimension covers 100 episodes; results are averaged over all episodes within each dimension.

The main paper reports 1-shot evaluation of the Qwen3.5 backbones before robot fine-tuning. We repeat the evaluation under 0-shot prompting (Figure 7) and find that the overall capability ranking is consistent, although Physical Law accuracy drops to near zero without the in-context example.

16 SmolVLA Full Size Comparison↩︎

Table 9 reports all three SmolVLA sizes alongside CLAP for reference. SmolVLA uses community-scale robot pretraining, whereas CLAP uses no robot pretraining beyond the LIBERO demonstrations. Despite this difference, CLAP 0.8B and 2B match or exceed all three SmolVLA scales on average success rate with single-epoch training, while CLAP 4B falls below SmolVLA 2.25B, consistent with the non-monotonic scaling observed in the main paper.

\(\dagger\)

SmolVLA uses community-scale robot pretraining.

References↩︎

---

1. Equal contribution.↩︎

2. Work done as an intern at OMRON SINIC X Corporation.↩︎