1 Introduction↩︎

Vision-Language-Action (VLA) models have emerged as a scalable paradigm for robotic control, typically acquiring manipulation skills through imitation learning (IL) over large-scale demonstrations [1]–[5]. However, the rapid growth of robot demonstration datasets does not automatically translate into better policies. In practice, such datasets often contain substantial trajectory redundancy, noisy human demonstrations, suboptimal behaviors, and uneven task coverage [6]–[9]. Training on unfiltered data can repeatedly expose the model to near-duplicate behaviors while also propagating inconsistent or low-quality supervision. These issues make data selection an increasingly important problem for VLA imitation learning: given a large demonstration pool, we aim to retain a compact subset that is more beneficial for policy learning.

Existing data selection methods for imitation learning typically curate demonstrations by estimating sample utility at different granularities. One line relies on trajectory-level signals, such as trajectory-representation similarity for redundancy removal, demonstration reliability, or downstream task feedback [10]–[13]. While these signals provide a global view of demonstration utility, they may collapse a long-horizon trajectory into a single score and obscure which internal stages or behavior compositions are useful; moreover, feedback-based methods often require costly additional model training. Another line estimates utility from state-action-level signals, such as state-action mutual information, task progress, or joint state-action similarity [12], [14]. These fine-grained signals can capture local predictability or redundancy, but they are often either too myopic to characterize coherent long-horizon task semantics or primarily designed for local pruning. This creates a granularity mismatch: effective IL data selection requires evidence that is coarser than individual state-action pairs, yet more structured than holistic trajectory scores.

Inspired by the Minimum Description Length (MDL) principle [15]–[17], we view useful demonstrations as those that expose reusable behavioral regularities. In its two-part form, MDL seeks a description that minimizes the cost of encoding both the model and the data: \[L(x)=\min_{H\in\mathcal{H}}\left[L(H)-\log P(x\mid H)\right],\] where \(L(H)\) is the number of bits required to encode the model \(H\), and \(-\log P(x\mid H)\) is the number of bits required to encode the data \(x\). This formulation operationalizes Occam’s razor: a model is penalized unless it yields a shorter description of the data. When repeating patterns exist, they can be stored in the model rather than redundantly encoded in each data instance. This view implies that a learner tends to compress data by absorbing repeating patterns into its parameters, so as to encode more data under limited parameters and computation budgets. Useful data, therefore, is not merely abundant or locally predictable, but rich in extractable structure (i.e., non-random regularities that a bounded learner can internalize and reuse as shared behavioral subprograms) [18]. For robotic imitation learning, such structure is naturally expressed as primitive composition: a trajectory consists of atomic behavior primitives and transition interfaces. Thus, we estimate trajectory utility by the primitives it exposes, how they are composed, and which transitions support long-horizon execution, yielding a mid-level scoring granularity that captures reusable behavior patterns shared across demonstrations.

In addition, since IL is typically optimized by behavior cloning (BC), selected demonstrations should also provide stable and predictable action supervision. Atypical or noisy realizations of the same behavior pattern may introduce inconsistent actions under similar observations, making the conditional action prediction target harder to fit [7], [19], [20]. Therefore, effective data selection should not only decide which behavioral structures to preserve, but also which concrete demonstrations to imitate. This motivates an IL-friendly selection principle: after identifying useful behavior patterns, one should prefer stable central trajectory realizations that provide more consistent supervision for BC.

Based on these insights, we propose SIEVE, a structure-aware data selection framework for VLA imitation learning. SIEVE first discovers reusable atomic behavior primitives by segmenting trajectories at physically grounded interaction boundaries and clustering segment representations. Each trajectory is then represented as a primitive sequence, which defines a composition pattern and its adjacent transitions. SIEVE allocates the selection budget over composition-pattern buckets to maximize reusable primitive and transition exposure under diminishing returns, and then selects medoid trajectories within each bucket to obtain central, stable, and imitation-friendly realizations. This yields a compact subset that exposes reusable behavioral structure while providing predictable supervision for BC. Our contributions are as follows:

• We propose a primitive-compositional view of trajectory utility, realized by Primitive Discovery and Structural Exposure Allocation, which allocate selection budgets according to reuse-aware primitive and transition exposure under diminishing returns.

• We introduce Learning-Friendly Trajectory Selection, which selects medoid trajectories within each composition-pattern bucket to favor central, stable, and predictable realizations for behavior cloning.

• We present SIEVE, a structure-aware data selection method for VLA imitation learning, and demonstrate its effectiveness across datasets, benchmarks, and models. SIEVE can outperform full-data training using only 50% of demonstrations and 50% of training steps, while consistently improving over competitive baselines under multiple experimental settings.

2 Related Work↩︎

2.1 Vision-Language-Action Models.↩︎

Recent advances in vision-language models (VLMs) [21]–[23] have accelerated the development of Vision-Language-Action (VLA) models, establishing a scalable paradigm for language-conditioned robotic control [1]–[3]. Modern VLAs map visual observations and language instructions to robot actions through generative policy architectures, such as autoregressive action tokenization, diffusion, or flow-matching models [1], [4], [5], [24]–[28]. These models are typically trained on large-scale demonstrations via imitation learning (IL), most commonly formulated as behavior cloning [20]. However, robotic datasets often contain noisy, redundant, and unevenly distributed demonstrations, so scaling training data indiscriminately can waste computation and yield diminishing returns [2], [6]–[9]. This motivates selecting compact yet informative subsets for efficient VLA imitation learning.

2.2 Data Selection for Imitation Learning.↩︎

Some related studies mainly focus on macroscopic dataset-level evaluation, mixing optimization across data sources and VLM data selection [7], [29]–[32]. Beyond dataset-level curation, existing IL data selection methods typically estimate sample utility either at the trajectory or state-action level. Trajectory-level methods use global signals such as representation similarity, reliability, downstream task feedback, or validation-loss-based influence estimation [10]–[13], but they often collapse long-horizon demonstrations into a single score and, for feedback-based criteria, require costly additional model training. State-action-level methods instead rely on local signals such as mutual information, task progress, or joint state-action similarity [12], [14], [33]. However, these criteria are often either too myopic to capture coherent task semantics or primarily designed for local pruning.

3 SIEVE↩︎

We propose SIEVE, a structure-aware data selection method for imitation learning with VLA models. The key idea is to exploit the structural exposure of demonstrations for data selection. To this end, SIEVE first discovers visuo-motor primitives from trajectory segments and represents each trajectory as a composition of these primitives. It then allocates the selection budget based on structural exposure and selects representative, learning-friendly trajectories within each composition pattern. The resulting subset retains informative behavioral structures while improving the efficiency of imitation learning.

3.1 Primitive Discovery↩︎

A demonstration trajectory is typically a composition of several reusable visuo-motor segments. This underlying structural prior motivates us to uncover a vocabulary of primitives from trajectories. We define a primitive as a reusable visuo-motor behavior unit discovered from trajectories, which serves as a proxy for a reusable behavioral subprogram.

3.1.0.1 Trajectory segmentation.

Let \(\mathcal{D}= \{\mathcal{T}_1,\dots,\mathcal{T}_N\}\) denote the original dataset, where each sample \(\mathcal{T}_i\) is a demonstration trajectory. For each trajectory \(\mathcal{T}_i \in \mathcal{D}\), we segment it using end effector (gripper/dexterous-hand) state flips (i.e., grasp/release flips) as physically grounded interaction boundaries to obtain \(L_i\) segments \(\mathcal{T}_i=\{S_i^1,\dots,S_i^{L_i}\}\). To suppress spurious boundaries caused by transient actuation jitter, a state transition is accepted only if it persists for five consecutive frames.

3.1.0.2 Segment representation.

We then extract the representation for each segment using a pretrained video encoder (\(\operatorname{VE}\)). Specifically, for each segment \(S_i^j\) (\(j\in \{1,\dots,L_i\}\)), we uniformly sample 8 frames and encode them with V-JEPA2 [34]. As the start, middle, and end frames provide a compact summary of the segment’s state evolution, we concatenate their representations to form a single feature that captures both semantic content and coarse temporal structure. To suppress noise, improve clustering stability, and reduce computational cost, we further reduce the representation to 256 dimensions with PCA: \[\begin{align} z_i^j \leftarrow &\operatorname{PCA}(\operatorname{Concat}(\operatorname{VE}(S_i^j)_\texttt{start};\\&\operatorname{VE}(S_i^j)_\texttt{mid};\operatorname{VE}(S_i^j)_\texttt{end})). \end{align}\]

3.1.0.3 Primitive discovery via clustering.

We discover primitives by clustering the segment representations using MiniBatch K-Means [35]. Rather than manually specifying the number of clusters, we select \(K\) automatically on a randomly sampled subset of trajectories. We seek a primitive vocabulary that is both reusable across trajectories and discriminative at the trajectory level: useful primitives should recur across demonstrations, while not collapsing structurally different trajectories into nearly identical primitive composition patterns.

For each candidate \(K\), we first cluster the segment representations from the randomly sampled subset and summarize each trajectory \(\mathcal{T}_i\) by the set of clusters it covers: \[\mathcal{C}_i=\{c_i^1,\dots,c_i^{|\mathcal{C}_i|}\} \subseteq \{1,\dots,K\},\] where each \(c_i^u \in \{1,\dots,K\}\) is a cluster index covered by at least one segment in \(\mathcal{T}_i\). Based on these trajectory-level cluster sets, we evaluate each candidate vocabulary using a reuse-aware criterion comprising two components. The first, trajectory-level discriminability \(\mathcal{J}\), measures how distinct different trajectories remain after being represented by cluster coverage. We compute \(\mathcal{J}\) as the median of the average pairwise Jaccard similarities over the cluster sets \(\mathcal{C}_i\) across all sampled trajectories: \[\mathcal{J} = \operatorname{median}_{i=1}^{n}\, \frac{1}{n-1}\sum_{j \ne i}^n \operatorname{Jaccard}(\mathcal{C}_i,\mathcal{C}_j).\] A lower \(\mathcal{J}\) indicates better preservation of structural distinctions between trajectories.

The second component, cross-trajectory reuse \(\mathcal{R}\), quantifies how broadly each discovered cluster recurs across trajectories. \(\mathcal{R}\) is defined as the median occurrence count of each cluster across all trajectories: \[\mathcal{R} = \operatorname{median}_{k=1}^{K}\, \sum_{i=1}^{n} \mathbf{1}[\,k \in \mathcal{C}_i\,].\] A higher \(\mathcal{R}\) implies that the discovered primitives capture widely applicable behavior units.

We choose the number of clusters by maximizing \[K^* = \arg\max_{K} \left[ \bigl(1 - \mathcal{J}\bigr) \log \mathcal{R} \right].\] This criterion favors primitive vocabularies that are broadly reused across trajectories while still preserving trajectory-level discriminability. After selecting \(K^*\), we rerun MiniBatch K-Means on all segment representations, and each resulting cluster is treated as a discovered primitive.

3.2 Structural Exposure Allocation↩︎

After primitive discovery, each trajectory \(\mathcal{T}_i\) is represented by an ordered primitive sequence, referred to as a composition pattern: \[P_i = [c_i^1, \dots, c_i^{|P_i|}],\] where each \(c_i^m \in \mathcal{C}_i\) is a discovered primitive.

We further define a transition as the local compositional interface between two adjacent primitives in a composition pattern. For trajectories with \(|P_i|\ge2\), the transitions are \[e_i^j=(c_i^j\rightarrow c_i^{j+1}), \qquad j=1,\dots,|P_i|-1.\] For trajectories consisting of a single primitive, we introduce a terminal null state \(\varnothing\) and define the terminal transition: \[e_i=(c_i^1\rightarrow\varnothing),\] so that every trajectory contains at least one transition.

Let \(\mathcal{P}=\{P^{(1)},\dots,P^{(|\mathcal{P}|)}\}\) denote the set of unique composition patterns in the dataset, where \(P^{(\ell)}\) denotes the \(\ell\)-th unique composition pattern, corresponding to a pattern bucket containing trajectories with the same composition pattern. This composition-pattern space provides the structural reference for measuring how broadly reusable structures are shared across demonstrations. In SIEVE, we capture such reusable structures through primitive composition, where trajectories expose both atomic behavior primitives and the transition interfaces connecting them. To effectively learn reusable structures, the model should be repeatedly exposed not only to primitives themselves but also to how they are composed through transitions. In our setting, these transition interfaces often coincide with critical state changes (e.g., gripper grasp/release) in the task, and therefore provide informative cues for local behavior progression. Moreover, primitives and transitions that are reused across more composition patterns support a broader range of executable behaviors and should be preferentially preserved.

To retain diverse and important structural information, we allocate the selection budget over the composition-pattern space. Let \[\mathcal{B}=[b_1,\dots,b_{|\mathcal{P}|}]\] denote the allocation vector, where \(b_\ell\) is the number of trajectories retained for composition pattern \(P^{(\ell)}\). We optimize the budget allocation \(\mathcal{B}\) by maximizing the following structural exposure objective: \[F(\mathcal{B}) = \sum_{c \in \mathcal{C}} w_c \log\bigl(1+n_c(\mathcal{B})\bigr) + \sum_{e \in \mathcal{E}} w_e \log\bigl(1+n_e(\mathcal{B})\bigr),\] where \(\mathcal{C}\) and \(\mathcal{E}\) denote the sets of all discovered primitives and transitions, respectively. Here, \(n_c(\mathcal{B})\) and \(n_e(\mathcal{B})\) denote the numbers of occurrences of primitive \(c\) and transition \(e\) among the selected trajectories under allocation \(\mathcal{B}\). The primitive and transition weights are defined by their reuse frequency over the composition-pattern space: \[\begin{align} w_c &= \frac{q_c}{|\mathcal{P}|}, \qquad q_c = \left| \{\, P \in \mathcal{P} : c \in P \,\} \right|, \\ w_e &= \frac{q_e}{|\mathcal{P}|}, \qquad q_e = \left| \{\, P \in \mathcal{P} : e \in P \,\} \right|. \end{align}\] Here, \(q_c\) and \(q_e\) denote the numbers of composition patterns containing primitive \(c\) and transition \(e\), respectively. Consequently, primitives and transitions that participate in more composition patterns receive larger weights. The logarithmic utility introduces diminishing returns, encouraging the budget to expose the model to a broader set of reusable structures rather than repeatedly reinforcing the same ones.

We optimize \(F(\mathcal{B})\) greedily. Starting from \(\mathcal{B}^{(0)}=\mathbf{0}\), each iteration allocates one additional sample to the composition pattern with the largest marginal gain: \[\Delta(P^{(\ell)}\mid\mathcal{B}) = F(\mathcal{B}+\mathbf{b}_\ell)-F(\mathcal{B}),\] where \(\mathbf{b}_\ell\) is the one-hot allocation vector that increases the budget of pattern \(P^{(\ell)}\) by one. We select \[\ell^\star = \mathop{\mathrm{arg\,max}}_{\ell\in\{1,\dots,|\mathcal{P}|\}} \Delta(P^{(\ell)}\mid\mathcal{B}),\] and update \[\mathcal{B} \leftarrow \mathcal{B} + \mathbf{b}_{\ell^\star}.\] After the budget is exhausted, \(\mathcal{B}\) specifies how many trajectories should be retained from each pattern bucket. The actual trajectories are then selected within each pattern bucket in the next stage.

3.3 Learning-Friendly Trajectory Selection↩︎

Given the pattern-level budget allocation, this stage selects representative and learning-friendly trajectories within each pattern bucket. Behavior cloning trains the policy by minimizing \[\mathcal{L}_{\mathrm{BC}}(\theta) = \mathbb{E}_{(s,a)\sim\mathcal{D}} [-\log \pi_\theta(a|s)],\] where \((s,a)\) is a state-action pair sampled from the demonstration dataset \(\mathcal{D}\), and \(\pi_\theta(a|s)\) is the probability assigned by the policy \(\pi_\theta\) to action \(a\) under state \(s\). Demonstrations with more consistent state-action mappings provide clearer supervision and are easier for imitation learning to optimize. Since directly estimating conditional action entropy is impractical, we use representation-space centrality as a practical proxy: trajectories near the center of a composition pattern are less likely to be outliers or ambiguous demonstrations, and thus tend to provide more stable supervision.

For each trajectory \(\mathcal{T}_i\) with composition pattern \(P^{(\ell)}\), we construct a trajectory representation \(x_i\) by concatenating the segment representations along its primitive sequence: \[x_i = \operatorname{Concat}(z_i^1; \dots; z_i^{|P_i|}).\] Within each composition pattern, we compare trajectories using cosine similarity and identify the medoid trajectory \(\mathcal{T}_{\mathrm{med}}\), which has the largest aggregate similarity \(S_i=\sum_{j\neq i}\cos(x_i,x_j)\) to other trajectories in the same pattern. We then rank trajectories by their distance to the medoid, defined as \(d_i=1-\cos(x_i,x_{\mathrm{med}})\), and retain the \(\mathcal{B}[\ell]\) trajectories with the smallest distances, where \(\mathcal{B}[\ell]\) is the budget assigned to composition pattern \(P^{(\ell)}\) by Structural Exposure Allocation. This selects trajectories closest to the pattern bucket center, yielding representative and learning-friendly demonstrations.

4 Experiments↩︎

4.1 Experiment Setup↩︎

Datasets and Evaluation. Here we evaluate SIEVE on three representative robot imitation learning datasets to assess both its data selection effectiveness and its applicability across different embodiments and environments. Unless otherwise specified, Bridge-V2 with SimplerEnv-WidowX serves as the default training and evaluation setting.

• Bridge-V2: We train models on the Bridge-V2 dataset [36], a real-world subset of Open X-Embodiment, containing approximately 53K demonstration trajectories collected with a WidowX robot equipped with a parallel gripper. Policies are evaluated in SimplerEnv-WidowX [37], which includes four manipulation tasks: Stack Cube, Put Carrot, Put Spoon, and Put Eggplant. Evaluation is conducted under multiple unseen kitchen backgrounds and randomized object configurations, providing a challenging out-of-distribution (OOD) benchmark.

• Fractal: We further evaluate SIEVE on the Fractal subset of Open X-Embodiment [2], which contains approximately 87K real-world manipulation trajectories collected with a Google Robot manipulator. Policies are evaluated in SimplerEnv-GoogleRobot [37] on three manipulation tasks: Grasp Coke Can, Move Near, and Close/Open Drawer.

• GR00T-X-Sim: We additionally evaluate SIEVE on the downsampled Humanoid Robot Tabletop Manipulation subset of GR00T-X-Embodiment-Sim [25], which contains 24K simulated demonstration trajectories collected with a humanoid robot equipped with dexterous hands. Policies are evaluated in RoboCasa-GR1 [38], a benchmark comprising 24 tabletop manipulation tasks across diverse scene layouts and object configurations.

Models. We evaluate SIEVE on two representative VLA models, Qwen3-VL-4B-GR00T and Qwen3-VL-4B-OFT. Unless otherwise specified, all main experiments and ablation studies are conducted using Qwen3-VL-4B-GR00T.

• Qwen3-VL-4B-GR00T [25]: This model combines the Qwen3-VL-4B vision-language backbone [22] with a GR00T-style flow-matching policy head that predicts continuous robot actions through conditional flow matching.

• Qwen3-VL-4B-OFT [26]: This model uses the Qwen3-VL-4B backbone and adopts the OpenVLA-OFT action decoding recipe, which performs parallel continuous action prediction with an MLP-based prediction head trained using an L1 regression objective.

Baselines. We compare SIEVE with the following baselines. All experiments use the same training hyperparameters for a fair comparison (see Appendix for details):

• Full-Training: trains on the complete dataset without data selection and serves as the full-data reference.

• Random: uniformly samples demonstrations from the original training set.

• DemInf [14]: selects demonstrations according to state-action mutual information estimates.

• SCIZOR [12]: filters low-quality data by identifying redundant trajectories and suboptimal state-action pairs.

4.2 Main Results↩︎

Table 1 summarizes the main results on Bridge-V2 using Qwen3-VL-4B-GR00T. We evaluate two selection budgets, 50% and 70%, corresponding to 26.5K and 37.1K demonstrations, respectively. For each budget, we consider two training schedules. The first scales the number of training steps proportionally to the selected data size (25K and 35K steps for the 50% and 70% budgets, respectively), reducing both training data and computation. The second trains each selected subset for the same 50K training steps as Full-Training. Comparing these two settings allows us to disentangle improvements brought by data selection from those potentially arising from increased optimization per sample under a fixed training budget.

As shown in Table 1, SIEVE consistently achieves the highest average success rate across all selection budgets and training schedules. Notably, using only 50% of the demonstrations and 25K training steps, SIEVE attains an average success rate of 56.3%, outperforming Full-Training (51.8%), which uses the complete dataset and twice the training steps. This result indicates that SIEVE is able to identify a compact subset that is more effective for imitation learning while substantially reducing both training data and computation.

Compared with existing data selection baselines, SIEVE consistently delivers the best performance. Under the 50% selection budget, SIEVE achieves average success rates of 56.3% and 59.4% under the proportional and 50K training schedules, outperforming SCIZOR (52.2% and 55.5%, respectively). Under the 70% budget, SIEVE further achieves average success rates of 62.3% and 62.5% with 35K and 50K training steps, respectively, again outperforming all competing methods. SIEVE also exhibits consistently stronger per-task performance. It outperforms Random on every task under all evaluation settings, suggesting that the performance gains arise from selecting structurally informative demonstrations rather than simply reducing the training set size. Compared with SCIZOR, SIEVE produces more balanced performance across tasks. While SCIZOR performs particularly well on Put Eggplant In Basket, its performance drops noticeably on Stack Green Cube On Yellow Cube and Put Carrot On Plate. In contrast, SIEVE maintains competitive performance across all four tasks, resulting in the best overall average success rate.

Since SimplerEnv evaluates policies under substantial visual and environmental variations relative to Bridge-V2, these results further demonstrate that the subsets selected by SIEVE generalize well under distribution shift.

4.3 Extended Results↩︎

Performance Across Different Datasets. We further evaluate SIEVE across different training datasets and evaluation benchmarks. For Bridge-V2, Full-Training is trained for 50K steps, whereas for Fractal and GR00T-X-Sim, Full-Training is trained for 100K steps. For all data selection methods, we retain 50% of the training data and train for 50% of the corresponding Full-Training steps (25K for Bridge-V2 and 50K for Fractal and GR00T-X-Sim). As shown in Table 2 (see the Appendix for detailed results), SIEVE achieves the highest average success rate across all three settings. On Bridge-V2 and Fractal, SIEVE achieves average success rates of 56.3% and 76.4%, outperforming Full-Training (51.8% and 75.0%, respectively), while using only half of the training data and half of the training steps. On GR00T-X-Sim, Random selection already slightly outperforms Full-Training. One possible explanation is that both GR00T-X-Sim and RoboCasa-GR1 are simulated tabletop manipulation environments, making RoboCasa-GR1 relatively in-domain for GR00T-X-Sim. In this setting, training on a smaller subset may provide more effective optimization per demonstration. Nevertheless, SIEVE still achieves the highest average success rate among all methods. These results demonstrate that SIEVE generalizes effectively across different datasets, robot embodiments, and evaluation benchmarks.

Performance Across Different Models. We further evaluate whether SIEVE-selected data remains effective across different VLA models. Since this experiment aims to evaluate model generalization rather than compare all data selection methods, we compare SIEVE with Full-Training and Random under the same 50% selection budget. As shown in Table 3, SIEVE consistently achieves the highest average success rate on both Qwen3-VL-4B-GR00T and Qwen3-VL-4B-OFT. On Qwen3-VL-4B-GR00T, SIEVE achieves an average success rate of 56.3%, outperforming both Random (39.6%) and Full-Training (51.8%). On Qwen3-VL-4B-OFT, SIEVE further achieves an average success rate of 56.5%, outperforming Random (26.3%) and Full-Training (38.8%). The gains are particularly evident on tasks such as Put Carrot On Plate and Put Eggplant In Basket, where SIEVE-selected data consistently achieves substantially higher success rates than both reference methods. These results indicate that the effectiveness of SIEVE is not tied to a particular VLA model.

4.4 Ablation Study and Further Analysis↩︎

Ablation of Structural Exposure Allocation. We ablate the two structural exposure terms in SIEVE. Removing the transition or primitive exposure term decreases the average success rate from 56.3% to 50.8% and 51.6%, respectively, indicating that both primitives and transitions contribute to effective data selection. The larger performance drop without transition exposure further suggests that transition interfaces provide important supervision for modeling how reusable behavior units are connected during task execution.

Ablation of Learning-Friendly Trajectory Selection. We ablate the within-bucket sample selection strategy. Replacing the proposed selection with Most-Dissim, which selects trajectories with the lowest aggregate cosine similarity to other samples to maximize trajectory diversity, reduces the average success rate from 56.3% to 40.1%, indicating that atypical trajectories within the same composition pattern are less suitable for imitation learning. Random within-bucket selection performs better than most-dissimilar selection but still trails SIEVE, improving the average success rate only from 53.7% to 56.3%. These results validate our design choice of selecting representative and stable demonstrations within each composition-pattern bucket.

Composition Pattern Redistribution. Figure 3 illustrates how SIEVE reshapes the composition-pattern distribution on Bridge-V2. The original dataset is dominated by a small number of high-frequency patterns, many of which consist of a single primitive, indicating limited compositional diversity. After selection, the distribution becomes substantially more balanced, and the most frequent selected patterns are predominantly multi-primitive sequences, covering a wider range of primitives and transitions. This redistribution suggests that SIEVE shifts the selected subset away from frequency-dominated patterns toward structurally richer composition patterns, thereby exposing more reusable primitives and transitions for imitation learning.

5 Conclusion↩︎

We introduced SIEVE, a structure-aware data selection method for VLA imitation learning. SIEVE selects demonstrations by exposing reusable primitive compositions and transition interfaces, while favoring central and stable realizations within each composition pattern for behavior cloning. Across multiple datasets, benchmarks, and VLA models, SIEVE consistently improves over competitive baselines and can outperform full-data training using fewer demonstrations and training steps. These results highlight the importance of selecting data according to reusable structure, offering a practical route toward more efficient VLA imitation learning.

6 Appendix↩︎

6.1 Training Hyperparameters↩︎

All experiments are conducted using the same training hyperparameters summarized in Table 5, ensuring a fair comparison across different methods. All models are trained on 8 NVIDIA H100 (80GB) GPUs.

6.2 Implementation Details of Primitive Discovery via Clustering.↩︎

To determine the number of primitive clusters, we adopt a practical search strategy. Specifically, for each dataset, we randomly sample 10% of the demonstrations and uniformly evaluate 20 candidate values of \(K\) within a reasonable search range. For each candidate, we perform primitive clustering and compute the clustering score \((1-\mathcal{J})\log\mathcal{R}\) defined in the Primitive discovery via clustering section. The value of \(K\) that maximizes this score is selected for all subsequent experiments. Figure 4 shows the search results on Bridge-V2, Fractal, and GR00T-X-Sim. The clustering score exhibits a clear peak on all three datasets, suggesting that the proposed criterion provides a stable and practical heuristic for determining the number of primitive clusters.

6.3 Details on Performance Across Different Datasets↩︎

Tables 6 and 7 present the detailed results corresponding to Table 2 in the main paper. Consistent with the main results, SIEVE achieves the highest average success rate on both Fractal and GR00T-X-Sim while maintaining competitive performance across individual evaluation categories.

References↩︎

---

1. These authors contributed equally↩︎

2. Corresponding author↩︎