1 Introduction↩︎

Robotic manipulation enables robots to understand scenes and interact with objects to perform precise physical tasks in the real-world environments. Some existing methods [1]–[7] rely solely on 2D visual inputs to predict end-effector actions in 3D space, however, they often struggle in scenarios requiring spatial reasoning, occlusion handling, geometric shape comprehension, or fine-grained object interactions due to their limited understanding of spatial geometry. Given that real-world tasks inherently occur in 3D space, incorporating 3D spatial information is crucial for learning robust and generalizable robotic manipulation policies.

Recent efforts in robotic manipulation have explored various approaches to exploit spatial information. In Fig. 1 (a), point cloud-based approaches [8]–[11] represent 3D geometry explicitly, yet suffer from semantic loss due to sparse sampling and are limited by the high cost of 3D annotations, which constrains pretraining scalability. In contrast, Fig. 1 (b) illustrates image-based methods [12]–[15] that utilize multi-view RGB-D to jointly model semantics and geometry in a shared feature space. These methods exploit structured 2D inputs to obtain dense semantics and benefit from strong 2D pretrained priors, enabling competitive performance. However, the entanglement of semantics and geometry makes these methods sensitive to inherent depth noise in the real-world, which degrades semantic and geometric understanding. As shown in Fig. 1 (d), even minor noise can lead to a significant performance drop of 8.9% in RVT2 [13]. In reality, depth is often compromised by sensor noise, lighting variations, and surface reflections, which severely limit the practical application of such methods in the real-world. Furthermore, the joint modeling primarily retains high-level geometry while neglecting low-level spatial cues that are critical for precise interaction by providing fine-grained 2D-3D correspondences.

The limitations above call for three critical capabilities in robotic manipulation: 1) fine-grained spatial understanding to enable accurate control; 2) robustness to sensor noise to ensure real-world reliability; and 3) low-level spatial cues to support consistent spatial tokens interaction. This raises a fundamental question: How can we construct a robust spatial representation that fulfills these requirements?

To address this, we propose SpatialActor, a novel framework for robust spatial representation in robotic manipulation. Instead of a shared feature space, we decouple semantics and geometry to mitigate cross-modal interference. Furthermore, we decompose geometric information into high-level geometric representations and low-level spatial cues. To construct a robust high-level geometric representation, we propose a Semantic-guided Geometric Module (SGM). Within the SGM, high signal-to-noise semantics from RGB are processed by a large-scale pretrained depth estimation expert [16], [17] to produce a robust but coarse geometric prior. Meanwhile, raw depth inputs retain fine-grained geometric details but are inherently noisy. By adaptively integrating these complementary geometric representations through a gating mechanism, the SGM enhances both robustness and spatial precision, effectively addressing the limitations of individual modalities. For low-level positional cues, we introduce a Spatial Transformer (SPT) that integrates spatial modeling into the transformer layers. By employing spatial position encoding, distinct spatial tokens are endowed with unique spatial indices, facilitating spatial interactions. The model performs view-level interaction to refine token relationships within each view, followed by scene-level interaction that unifies cross-modal cues across the scene, yielding features for the action head.

To comprehensively evaluate our method, SpatialActor, we conduct experiments on 50+ robotic manipulation tasks in both simulation and real-world. In RLBench (18 tasks with 249 variations), SpatialActor achieves an 87.4% average success rate, surpassing state-of-the-art methods by approximately 6.0%, with a notable 53.3% improvement in high-precision spatial tasks like Insert Peg. Our method also shows strong robustness, maintaining higher success rates under noise conditions with improvements of 13.9%, 16.9%, and 19.4% at light, medium, and heavy noise levels, respectively. On ColosseumBench, which evaluates 20 tasks under spatial perturbations, SpatialActor consistently outperforms baselines, showcasing superior spatial generalization. Additionally, in a few-shot setting, adapting a multi-task pre-trained model to 19 novel tasks with only 10 demonstrations per task, SpatialActor achieves 79.2% success compared to 46.9% for RVT-2. Real-world experiments further validate these results, as SpatialActor outperforms RVT-2 across 8 tasks and 15 variations, demonstrating its strong robustness and generalization across diverse scenarios.

2 Related Works↩︎

2.1 Representation Learning for Manipulation↩︎

Early methods relied on proprioceptive sensing [18], [19], which limited their generalization. With the rise of large-scale visual pretraining, many 2D-based approaches [2], [5], [20]–[24] leverage strong visual priors to extract semantics. However, they often lack 3D spatial understanding, limiting their effectiveness in precise manipulation. Point cloud-based methods [8]–[10], [25]–[27] capture explicit 3D structures, offering geometry but are hampered by sparsity. Voxel-based representations [11], [28] reduce sparsity by discretizing space for structured reasoning, yet they incur high computational costs. Multi-view RGB-D approaches [12]–[15], [29], [30] integrate dense 2D semantics with geometry via early fusion or auxiliary supervision, yet such shared feature spaces remain vulnerable to sensor noise and often lack precise spatial corresponding for fine-grained interaction. To address these limitations, we decouple semantics and geometry, and construct geometric representations by fusing complementary high-level expert priors and raw depth together with low-level spatial cues for precise manipulation.

2.2 Vision Foundation Models for Robotics↩︎

Vision foundation models have significantly enhanced robotic perception by incorporating semantic and geometric priors. Visual and multimodal models [31]–[36] leverage diverse datasets to learn strong semantic priors that improve visual understanding, which benefits downstream robotic tasks. However, they focus on the 2D domain and lack spatial understanding capabilities. 3D vision models [37]–[42] integrate semantic information with explicit spatial structures to facilitate effective geometric perception. However, the acquisition and annotation of 3D data are inherently expensive and labor-intensive, which restricts scalability and limits their application in real-world scenarios. Depth estimation experts [16], [17], [43], [44] leverage large-scale pretraining on diverse datasets to translate semantics in images into corresponding geometric structures, robustly inferring geometric information even under challenging conditions such as sensor noise and occlusions. In this paper, we leverage the strong semantic alignment of vision models together with robust geometric priors from depth estimation experts.

3 Method↩︎

3.1 Overall Framework↩︎

Fig. 2 illustrates the overall framework of our approach. The inputs to the robot’s control system are given by \[X = \{ I^v, D^v \}_{v=1}^{V},\, P,\, L,\] where \(I^v \in \mathbb{R}^{H \times W \times 3}\) and \(D^v \in \mathbb{R}^{H \times W}\) denote the RGB image and depth map for view \(v\) (with \(V\) views in total), \(P \in \mathbb{R}^{d_p}\) represents the robot’s proprioceptive state (\(d_p\) indicating its dimension), and \(L\) denotes the language prompt.

For each view \(v\), the RGB images and noisy depth maps are processed separately. The images \(I^v\) and the language instruction \(L\) are fed into a vision-language model (e.g., CLIP [31]) to extract semantic features \({F}_{\text{sem}}^{v}\) and text features \(F_{\text{text}}\). Meanwhile, raw depth maps \(D^v\) are processed by a depth encoder to yield fine-grained but noisy geometric features \(F_{\text{geo}}^{v}\). Subsequently, \(F_{\text{geo}}^{v}\) is enhanced via a Semantic-guided Geometric Module (SGM). In SGM, large-scale pre-trained depth estimation expert is employed to obtain robust yet coarse geometric priors \(\hat{F}_{\text{geo}}^{v}\). A multi-scale gated fusion module then adaptively fuses \(F_{\text{geo}}^{v}\) with \(\hat{F}_{\text{geo}}^{v}\) to produce refined geometric features \(F_{\text{fuse-geo}}^{v}\), preserving details while reducing noise, which are concatenated with \(F_{\text{sem}}^{v}\) to form the final spatial representation \(H^v\).

We further introduce a Spatial Transformer (SPT). Within the SPT, intrinsic and extrinsic parameters, along with depth values, are used to construct a spatial encoding that captures the low-level spatial cues between spatial tokens. The SPT first applies view-level interaction to consolidate intra-view context, followed by scene-level cross-modal interaction to aggregate cross-modal cues into a unified scene representation. Finally, an action head predicts the robot’s 3D end-effector pose and gripper state.

3.2 Semantic-guided Geometric Module↩︎

Real-world depth measurements are often noisy due to sensor limitations and environmental interference, whereas RGB images provide high signal-to-noise semantic cues. Large-scale pretrained depth estimation models (e.g., Depth Anything [16], [17]) learn a smooth semantic-to-geometric mapping, offering robust and generalizable geometric priors. In contrast, raw depth features retain fine-grained, pixel-level details but are highly sensitive to noise.

To leverage these complementary strengths, we extract robust yet coarse-grained geometric priors from RGB inputs via a frozen large-scale pre-trained depth estimation expert: \[\hat{F}_{\text{geo}}^{v} = \mathcal{E}_{\text{expert}}(I^v) \in \mathbb{R}^{H \times W \times C},\] and extract fine-grained but noisy geometry from raw depth using a depth encoder (e.g. ResNet-50 [45]): \[F_{\text{geo}}^{v} = \mathcal{E}_{\text{raw}}(D^v) \in \mathbb{R}^{H \times W \times C}.\]

As shown in Fig. 3 (a), a multi-scale gating mechanism then adaptively fuses these features to yield an optimized geometric representation that preserves fine details while reducing noise and aligning with the semantic cues. \[G^{v} = \sigma\bigl(\mathrm{MLP}\bigl(\mathrm{Concat}(\hat{F}_{\text{geo}}^{v},\, F_{\text{geo}}^{v})\bigr)\bigr),\] \[F_{\text{fuse-geo}}^{v} = G^{v} \odot F_{\text{geo}}^{v} + \bigl(1 - G^{v}\bigr) \odot \hat{F}_{\text{geo}}^{v},\] where \(\sigma\) denotes sigmoid activation and \(\odot\) element-wise multiplication. The gate \(G^{v}\) learns to retain reliable depth details while suppressing noise.

3.3 Spatial Transformer↩︎

For each view \(v\), we denote the spatial features as \(H^v \in \mathbb{R}^{N_v \times D}\). The proprioceptive input \(P\) is projected via an MLP and fused with \(H^v\) by element-wise addition: \[\widetilde{H}^v = H^v + \text{MLP}(P).\]

Given a pixel \((x', y')\) with depth \(d = D^v(x', y')\), its 3D coordinate \([x, y, z]^\top\) in the robot-centric coordinate system is computed via perspective projection: \[[x, y, z, 1]^\top = E^v \left( d \cdot (K^v)^{-1}[x', y', 1]^\top \,\Vert\, 1 \right),\] where \(K^v \in \mathbb{R}^{3 \times 3}\) and \(E^v \in \mathbb{R}^{4 \times 4}\) denote the intrinsic and extrinsic matrices, and \(\Vert\) denotes vector concatenation.

To encode spatial cues, we apply rotary positional encoding to \(\widetilde{H}^v\), where each axis is assigned \(D/3\) dimensions. We define a set of frequencies: \[\omega_k = \lambda^{-2k/d},\quad k = 0, 1, \dots, \frac{d}{2} - 1,\quad d = D/3,\] with \(\lambda=10000\) to control the frequency bandwidth. In the spirit of Fourier feature mappings, we compute axis-wise sinusoidal embeddings as: \[\cos_{\text{pos}} = \left[\cos(\omega_k u)\right]_{u \in \{x, y, z\},\;k = 0, \dots, d/2 - 1},\] \[\sin_{\text{pos}} = \left[\sin(\omega_k u)\right]_{u \in \{x, y, z\},\;k = 0, \dots, d/2 - 1}.\] The final position-encoded features are given by: \[T^v = \widetilde{H}^v \odot \cos_{\text{pos}} + \mathrm{rot}(\widetilde{H}^v) \odot \sin_{\text{pos}},\] where \(\odot\) denotes element-wise multiplication, and \(\mathrm{rot}(\cdot)\) rotates each \((f_{2i}, f_{2i+1})\) feature pair as \((-f_{2i+1}, f_{2i})\).

At the view level, self-attention followed by a feed-forward network (FFN) refines each view’s token representation. At the scene level, tokens from all views are concatenated with language features \(F_{\text{text}}\). Another round of self-attention and an FFN then fuses cross-view and language context, producing the final refined tokens. The tokens are fed into a lightweight decoder (ConvexUp) to generate a per-view 2D heatmap. The target 2D position is obtained via \(\text{argmax}\) and lifted into 3D using the camera model. The action head then uses an MLP on local features around this position to regress the rotation \(\theta = (\theta_x,\theta_y,\theta_z)\) and gripper state \(g\). Together with the 3D translation \((x,y,z)\), these form the final action \(A = (x, y, z, \theta_x, \theta_y, \theta_z, g)\).

The action supervision includes three parts: a cross-entropy loss on per-view 2D heatmaps for translation, cross-entropy losses on discretized Euler angles for rotation, and a binary classification loss for the gripper state.

4 Experiments↩︎

To comprehensively evaluate the effectiveness of SpatialActor, we conduct experiments in both simulation and real-world settings. Specifically, we aim to answer the following key questions: (1) How does SpatialActor compare to state-of-the-art robotic manipulation policies? (2) How robust is SpatialActor under noisy conditions? (3) How well does SpatialActor generalize to few-shot settings? (4) How does SpatialActor perform under spatial perturbations? (5) What is the impact of different components of SpatialActor? (6) How does SpatialActor perform in real-robot setups?

4.1 Comparison with State-of-the-Art Policies↩︎

4.1.0.1 Simulation Environment and Datasets.

We evaluate SpatialActor on RLBench [49], a mainstream multi-task 3D manipulation benchmark built on CoppeliaSim [50]. The simulation environment features a Franka robotic arm with a parallel gripper operating in a tabletop scenario. Observations come from four fixed RGB-D cameras (front, left/right shoulder, wrist) at \(128 \times 128\) resolution. The action space consists of 3D translation, rotation of the end-effector, and binary gripper control. An OMPL-based motion planner [51] is utilized to compute feasible trajectories. Following PerAct [28], we use 18 tasks with 249 variations covering diverse manipulation skills, each with 100 expert demonstrations for training and 25 unseen episodes for evaluation.

4.1.0.2 Implementation Details.

SpatialActor is trained for approximately 40k iterations using a cosine learning rate schedule with an initial 2k-iteration warm-up. Training is performed using 8 GPUs with a total batch size of 192 (24 per GPU) and an initial learning rate of \(2.4 \times 10^{-3}\). Data augmentation includes random spatial translations of up to 12.5 cm along the \(x\), \(y\), and \(z\) axes, as well as rotations of up to \(45^\circ\) around the \(z\) axis. We follow RVT [12], [13], incorporating its virtual view design and two-stage process. Furthermore, we employ CLIP [31] as our vision-language encoder to provide aligned cross-modal representations, and Depth Anything v2 [17] as our geometry expert.

4.1.0.3 Performance on RLBench 18 Tasks.

Tab. ¿tbl:tab:rlbench95results? summarizes the performance of various methods on 18 RLBench tasks with 249 variations. SpatialActor achieves an average success rate of 87.4%, surpassing the previous state-of-the-art by 6.0%. Notably, SpatialActor shows substantial improvements on tasks requiring high spatial precision, such as Insert Peg and Sort Shape. It achieves success rates of 93.3% and 73.3% on these tasks, outperforming RVT-2 by 53.3% and 38.3%, respectively. These results highlight SpatialActor’s superior spatial handling capability.

4.2 Robustness under Noisy Conditions↩︎

4.2.0.1 Experimental Setup.

Depth measurements are inherently affected by sensor noise, lighting variations, and surface reflections. To simulate these challenges, we inject controlled Gaussian noise into reconstructed point clouds. Specifically, we design three noise levels: Light corrupts 20% of the points with a Gaussian standard deviation of 0.05, Middle corrupts 50% of the points with a standard deviation of 0.1, and Heavy corrupts 80% of the points with a standard deviation of 0.1. This setup allows us to evaluate the robustness of our approach under progressively severe noisy conditions.

4.2.0.2 Performance Evaluation.

Tab. ¿tbl:tab:noise95results? shows that under Light, Middle, and Heavy noise, SpatialActor improves average success rates over RVT-2 by 13.9%, 16.9%, and 19.4%, respectively. Notably, in tasks requiring high spatial precision, these gains are even more pronounced. For instance, on Insert Peg task, SpatialActor outperforms RVT-2 by 88.0%, 78.6%, and 61.3% under the respective noise levels.

4.3 Few-Shot Generalization↩︎

We evaluate the few-shot generalization ability of SpatialActor by adapting the multi-task pre-trained model to 19 novel tasks using only 10 demonstrations per task, just one-tenth of the data used during multi-task training. In this few-shot adaptation scenario, the model is initialized with its pre-trained weights and then fine-tuned on the limited data. As shown in Tab. ¿tbl:tab:few95shot95results?, our experiments demonstrate that SpatialActor effectively transfers previously learned skills to new tasks with minimal adaptation data. Overall, SpatialActor achieves an average success rate of 79.2%, compared to 46.9% for RVT-2, yielding an improvement of approximately 32.3%. This significant boost underscores the superior few-shot generalization capability of our approach.

4.4 Spatial Perturbations on ColosseumBench↩︎

4.4.0.1 Setup.

We evaluate the robustness on the Colosseum benchmark [52], which assesses manipulation policies under environmental variations. We report results on 20 tasks under the baseline setting (no perturbation) and three spatial perturbations: manipulation object size (MO-Size), which scales the object being manipulated; receiver object size (RO-Size), which scales an indirectly used object such as a container; and camera pose perturbation (Camera-Perturb), which randomly shifts the camera’s position and orientation to vary the observation viewpoint.

4.4.0.2 Performance Evaluation.

The results in Tab. 1 indicate that under the no-perturbation condition, our method achieves a task-average success rate of 57.4%. When spatial perturbations are introduced, SpatialActor attains 59.2% under MO-Size variations, 62.0% under RO-Size changes, and 54.2% with camera pose perturbations. These results consistently outperform competing methods, demonstrating strong robustness and generalization under spatial variations.

4.5 Ablation Study↩︎

Our ablation study on 18 tasks (Tab. 2) shows that decoupling semantics and geometry improves performance in both noise-free and heavy-noise settings, increasing success rates from 81.4% to 85.1% and from 57.0% to 68.7%, respectively. Introducing the Semantic-guided Geometry Module (SGM) further boosts performance, especially under heavy noise, where performance rises to 73.9%. Finally, the Spatial Transformer (SPT), which provides precise low-level spatial cues, brings the success rates to 87.4% and 76.4% in noise-free and noisy conditions, respectively.

4.6 Real-World Evaluation↩︎

4.6.0.1 Setup.

In real-world experiments, we use a WidowX single-arm robot equipped with an Intel RealSense D435i RGB-D camera. The camera is statically mounted to capture a front view of the workspace. We perform both intrinsic and extrinsic calibration between the camera and the robot to accurately transform the observed point clouds into the robot’s base coordinate system. The system is integrated using a ROS package. Images are originally captured at a resolution of \(1280 \times 720\) and are downsampled to \(128 \times 128\).

4.6.0.2 Dataset Collection.

We conduct experiments on a series of real-world tasks (Fig. 4), including (1) Pick Glue to Box, (2) Stack Cup, (3) Push Button, (4) Slide Block, (5) Place Carrot to Box, (6) Stack Block, (7) Insert Ring onto Cone, and (8) Wipe Table. For each task, we collect 25 demonstrations that capture diverse spatial configurations and object variations. Some tasks are instantiated with multiple variations, for example, the Slide Block task includes yellow, green, and red variants, resulting in a total of 15 variations across the 8 tasks. The trajectories are recorded at 30 fps, and key-frames are extracted to construct the training set.

4.6.0.3 Evaluation.

We evaluate SpatialActor against RVT-2 on various real-world tasks. Single-variant tasks are tested 20 times, and multi-variant tasks 10 times per variant. As shown in Tab. 3, SpatialActorconsistently outperforms RVT-2, with an average improvement of around 20% across tasks, demonstrating effectiveness in real-world scenarios.

To evaluate robustness to distribution shifts, we test SpatialActor under variations in manipulated object, receiver object, lighting, and background (Fig. 5). SpatialActor maintains consistently high performance across these diverse and challenging conditions, clearly demonstrating strong robustness and generalization in complex real-world scenarios.

5 Conclusion↩︎

In this work, we present SpatialActor, a framework for robust spatial representation in robotic manipulation. SpatialActor disentangles semantic and geometric information, with the geometric branch comprising a high-level module (SGM), which fuses semantic-guided geometric priors with depth features, and a low-level module (SPT), which captures fine-grained spatial cues through position-aware interactions. Experiments across 50+ simulated and real-world tasks show that SpatialActor achieves higher success rates and strong robustness, underscoring the value of disentangled spatial representations for reliable manipulation.

Acknowledgments↩︎

This work was supported by the National Science and Technology Major Project of China under Grant No. 2023ZD0121300, the Scientific Research Innovation Capability Support Project for Young Faculty under Grant No. ZYGXQNJSKYCXNLZCXM-I20, and the National Natural Science Foundation of China under Grant No. U24B20173.

6 Hyperparameters↩︎

As detailed in Table 4, we train our model with a per-GPU batch size of 24 on eight GPUs for a total batch size of 192, using the LAMB optimizer at an initial learning rate of \(4\times10^{-3}\) and a cosine decay schedule. A linear warmup runs for the first 2000 steps, followed by around 40 000 training steps (50 epochs).

7 Data Scaling Analysis↩︎

We examine how the size of the training dataset affects our model’s performance on eight challenging RLBench manipulation tasks. As shown in Table 5, the average success rate grows monotonically with the number of training samples, rising from 69.3 % at 25 samples to 73.0 % at 50 samples, 75.4 % at 100 samples, and 80.0 % at 200 samples, before reaching 81.3 % at 500 samples. This monotonic improvement illustrates a clear data-scaling effect, where enlarging the dataset consistently enhances task performance and generalization. For fair comparison with prior work, however, we report all results in the main paper using the 100 sample.

8 Ablation on View Setup↩︎

Table ¿tbl:tab:view? presents success rates for the front-only (single-view) and multi-view configurations on 18 RLBench manipulation tasks. The multi-view setup, which integrates front, left shoulder, and right shoulder cameras, increases the average success rate from 80.0 % to 87.4 %, demonstrating that additional viewpoints enrich spatial context and substantially improve manipulation performance.

9 Ablation on Visual Backbone↩︎

Table ¿tbl:tab:backbone? compares DINO and CLIP as visual backbones on 18 RLBench manipulation tasks. Using CLIP’s language-aligned visual features increases the average success rate from 86.5 % to 87.4 %, indicating that semantic alignment in the encoder contributes to improved task performance.

10 Qualitative Results↩︎

Figures 6 and 7 offer comparisons between RVT-2 and our method on two real-world manipulation tasks. In the PlaceGlueToBox task, RVT-2’s grasp attempts frequently slip or miss the glue stick as a result of noisy depth perception, whereas our method produces stable, secure grasps that reliably lift the stick for subsequent placement. In the InsertRingOntoCone task, RVT-2’s unstable pickups often let the ring slip or drift off-axis, whereas our method reliably acquires the ring and centers it above the cone for smooth insertion.

11 Failure Cases↩︎

As shown in Figure 8 (a), simulation failures include (1) instruction understanding errors, where the Open Drawer task opens the wrong drawer; (2) long-horizon breakdowns, where Place Cups stalls after only a few placements; and (3) semantic grounding confusion, where Stack Cups picks the incorrect cup among similar ones. In real-world trials (Figure 8 (b)), failures arise from (1) pose precision limits, where slight end-effector drift causes Stack Cups to miss its target; (2) instruction mis-understanding, where Stack Blocks grasps the wrong block; and (3) distractor susceptibility, where background clutter diverts the policy during Pick Glue To Plate. These findings point toward promising enhancements, such as incorporating large language models (LLMs) for more accurate instruction parsing, adding episodic memory or belief tracking to support reliable long-horizon planning, and integrating uncertainty-aware pose refinement and attention-based filtering to improve resilience against calibration errors and visual clutter.

12 Robot Setup↩︎

As shown in Figure 9, a WidowX-250 single-arm robot sits adjacent to an Intel RealSense D435i RGB-D camera fixed on a tripod 0.8 m from the workspace. The camera captures synchronized 1280×720 color and depth frames at 30 Hz, which are downsampled to 128×128 for input into SpatialActor. We perform intrinsic and extrinsic calibration to align depth measurements with the robot’s base frame, enabling accurate end-effector control in real-world.

13 Simulation Tasks↩︎

We follow the evaluation protocol of PerAct and benchmark SpatialActor on the same 18 RLBench tasks shown in Figure 10. In total, these tasks cover 249 randomized scene configurations varying in object color, count, placement, shape, size, or category. Table 6 provides each task’s language instruction template, the number of variations, and the type of variation. Furthermore, Figure 11 illustrates the simulation tasks in a few-shot setting.

14 Real-world Tasks↩︎

We evaluate SpatialActor on 8 real-world manipulation tasks illustrated in Figure 12. In total, these tasks comprise 15 randomized variants. Table 7 lists each task’s language instruction template and the corresponding number of variants.

References↩︎