Vision-Based Dribbling for Humanoid Soccer via Privileged Representation Learning
July 14, 2026
Abstract
Recent advances in humanoid robotics have highlighted the importance of deployable loco-manipulation skills. Dribbling a soccer ball while evading active opponents requires simultaneous balance, precise ball control, and awareness of a dynamic adversary under onboard sensing and real-time constraints. Existing approaches typically separate perception and motion, which can be effective in controlled settings but may fail under occlusions, fast ball movements, and complex opponent interactions, since perception is not directly optimized for control.
We propose an integrated approach in which a temporal depth encoder is embedded into a reinforcement learning policy through a task-specific projection layer. We apply this framework to a simulated Booster T1 humanoid robot and show that it is possible to learn vision-based, opponent-aware dribbling directly from depth observations, without explicit state estimation or privileged scene information. The learned policy achieves \(100\%\) success in nominal target-driven dribbling and \(96\%\) success with a single static obstacle, while reaching \(46\%\) success against an actively moving ball-attacker opponent. These results demonstrate that the proposed framework supports robust vision-based dribbling in nominal and moderately dynamic settings, and provides a strong foundation for handling more challenging moving-adversary scenarios.
1 Introduction↩︎
Reinforcement learning has enabled legged robots to master complex locomotion behaviors across challenging terrains [1], [2], and more recently to manipulate objects in the environment while walking - a setting that demands simultaneous control of balance, contact forces, and object dynamics [3]. However, despite the progress that DRL has allowed, loco-manipulation task still remains an open challenge for both quadruped and bipedal robots. If the problem results complex for the first, and a wide range of solutions have been explored [4], [5], for the second the challenge is still harder, due to the mixture of coordination, balance and stability that are needed to execute a task [6].
Soccer dribbling is a particularly demanding instance of this problem: the robot must maintain continuous ball control while navigating toward a goal, adapting its gait to ball velocity, and reacting to perturbations. Previous works have demonstrated dribbling on quadrupeds [7], [8] on diverse terrains, but these systems either rely on human-provided velocity commands or assume that the robot acts in isolation, without an adversary. Compared with quadrupeds, humanoids must also preserve dynamic balance under the perturbations introduced by ball contact.
The soccer setting, especially considering the existence of rapid ball movements and active opponents, fundamentally changes the perception requirements. The robot must now track not only the ball but also the opponent’s position relative to itself, and it must maintain this estimate even when the ball is momentarily occluded - a frequent occurrence on a humanoid, where the ball regularly disappears during close-range dribbling. This challenge has grown qualitatively harder as humanoid hardware has improved. Modern RoboCup humanoid platforms, such as the Booster T1, operate at higher speeds than earlier platforms, increasing the perception and control demands.
The dominant paradigm for addressing the ball control in humanoid soccer has been to decouple perception from control: a detection module (e.g. YOLO-based) localizes the ball in each frame, and a Kalman filter propagates the state estimate across frames and through missed detections [9]. While effective in controlled settings, this modular design has a fundamental limitation for learning-based control: the perception module is optimized independently of the policy, with no guarantee that its output representation is sufficient for the downstream task.
In this work, we address these challenges by learning perception and control jointly through a temporal encoder whose representation is shaped by what the policy actually needs to act well. We formulate the task as an obstacle-aware dribbling problem and address it with a perception-integrated RL system inspired by Rapid Motor Adaptation (RMA) [1]: a privileged encoder provides rich state information during training, which a deployable temporal encoder learns to approximate at inference time. Integrated with this representation, we train a policy under a curriculum that progressively increases opponent difficulty and with a reward shaping system that encourages ball possession and opponent avoidance. All experiments are conducted in simulation on the Booster T1 humanoid platform using mjlab [10], and we outline a principled sim-to-real roadmap as future work. The contributions of this paper are threefold:
We propose a temporal encoder trained through privileged distillation to infer task-relevant ball and opponent states from onboard depth observations, enabling deployable perception without explicit state estimation.
We introduce an obstacle-aware training pipeline that progressively exposes the robot to increasingly challenging dribbling scenarios, from nominal ball control to static blockers and dynamic opponent-like obstacles.
We present and release an end-to-end reinforcement learning system for autonomous humanoid soccer dribbling, integrating perception, balance control, ball handling, and opponent avoidance within a unified policy framework. Code and extra-material are released at https://lab-rococo-sapienza.github.io/learning-to-dribble/.
The remainder of the paper is organized as follows. Section 2 reviews related work, Section 3 presents the proposed methodology, Section 4 reports the experimental results, and Section 5 concludes the paper.
2 Related Work↩︎
Recent advances in deep reinforcement learning (RL) have enabled legged robots to acquire robust and adaptive locomotion skills across a wide range of terrains. [1] introduced an adaptation module that infers latent environment parameters online, allowing for walking with terrain variation and payload changes, and [11] extended the approach to bipedal robots. These works highlight the importance of latent adaptation for handling partial observability and dynamics uncertainty in locomotion. [12] pushes this further with an end-to-end controller characterized by an adaptive curriculum on velocity commands and online system identification, while [13] combines experience replay with an automatic curriculum strategy to achieve agile and adaptive behaviors including fall recovery and high-speed turning. Along a complementary direction, [14] proposes a concurrent teacher-student scheme that jointly trains privileged and deployable policies, improving sample efficiency and sim-to-real robustness in challenging environments. On the bipedal side, [15] introduces a unified RL framework with a dual-history architecture that enables the biped to walk, run, and jump, identifying task randomization as a key source of robustness. Several works have targeted agile locomotion over challenging terrains such as [16] [17]. [18] integrates visual information into a teacher-student approach for robust bipedal locomotion over uneven terrains, using a mixture-of-experts policy with an alignment loss between teacher and student networks.
Soccer and dribbling tasks represent a challenging class of loco-manipulation problems, requiring simultaneous control of locomotion and object dynamics. Early systems relied on modular, model-based pipelines with limited adaptability [19]. On quadrupeds, DribbleBot [7] learns dynamic ball manipulation directly from sensory observations, while DexDribbler [8] achieves dexterous ball control through RL with dynamic supervision. On the other hand, hierarchical frameworks decouple kicking from target-aware planning for precise shooting, leveraging deep reinforcement learning to train both robust motion control policies and planning policies that enable accurate real-world ball targeting [20]. Humanoid soccer introduces additional challenges due to dynamic balance, higher degrees of freedom, and increased sensitivity to perturbations. [21] trains a humanoid to play soccer end-to-end, with agile kicking, fall recovery, and tactical behaviors emerging from self-play, while [22] removes reliance on privileged state by training soccer policies directly from RGB vision. More recently, [23] presents a unified RL controller for humanoid soccer that tightly integrates visual perception with motion control. Complementary work such as [24] addresses robust fall recovery through multi-stage curriculum learning.
In contrast, our work investigates humanoid dribbling in a dynamic environment, where obstacles mimicking opponent behaviors force the policy to plan around interferences, rather than operate in a free field. We build on the RMA paradigm with a temporal-depth encoder that jointly distills perception and control, producing a deployable visual adaptation module trained end-to-end that addresses both the humanoid loco-manipulation gap and the lack of adversaries in prior benchmarks.
3 Methodology↩︎
Problem Formulation. We consider a dribbling task where a humanoid robot must maintain ball control while avoiding opponents. The problem is formulated as a Partially Observable Markov Decision Process (POMDP). \[\mathcal{M}=(\mathcal{S},\mathcal{O},\mathcal{A},p,r,\gamma)\]
The state space \(\mathcal{S}\) includes the full world-frame state of all entities, namely the robot, ball, and opponent obstacles, as well as internal command variables such as the target position. The observation space \(\mathcal{O}\) comprises both proprioceptive signals (e.g., IMU readings and joint positions) and exteroceptive information (e.g., ball position, ball velocity, and obstacle positions). In particular, the exteroceptive information is encoded into a latent vector \(z_t \in \mathbb{R}^{d_z}\) that summarizes historical observations [1]. The latent vector \(z_t\) is produced according to the two-phase training scheme detailed in Sec. [sec:VEL].
The action space \(\mathcal{A}\) contains a joint position vector \(a_t \in \mathbb{R}^{21}\). The vector represents only the body joints, we assume the head is controlled independently. The transition function \(p(s_{t+1} \mid s_t, a_t)\) is induced by the physics simulator 1 and task randomization, and is assumed to be unknown to the policy. As for the rewards, the task-related terms include (i) ball-velocity tracking, (ii) robot-ball proximity, (iii) robot-ball directional alignment, (iv) target progress, and (v) target completion, while additional penalties discourage unsafe obstacle interactions, while regularization terms promote feasible locomotion, smooth actions, stable contacts, and consistent gait patterns.
The policy receives only a partial observation \(o_t \in \mathcal{O}\). Following the RMA formulation, the actor observation is composed of proprioceptive terms together with a latent vector representing exteroceptive information. Formally, the actor observation is: \[o_{t}^{\text{actor}} = \left[ o_t^{\mathrm{prop}}, z_t \right]\] where the proprioceptive component \(o_t^{\mathrm{prop}}\) is composed by:
base angular velocity from the IMU, \(\boldsymbol{\omega}_t^{base}\);
projected gravity in the robot body frame, \(\mathbf{g}_t^{proj}\);
relative joint positions, \(\mathbf{q}_t^{rel}\);
relative joint velocities, \(\dot{\mathbf{q}}_t^{rel}\);
previous action, \(\mathbf{a}_{t-1}\);
gait-phase command, \(\boldsymbol{\phi}_t\);
commanded ball velocity expressed in the robot body frame, \(\mathbf{v}_t^{ball}\);
The proprioceptive component includes IMU measurements, projected gravity, relative joint positions and velocities, the previous action, gait phase, and the commanded ball velocity expressed in the robot frame.
Instead, the latent variable \(z_t\in \mathbb{R}^{d_z}\) provides additional exteroceptive information derived from high-dimensional sensory inputs (e.g., raw obstacle or ball detections), ensuring that the actor never directly observes privileged simulator states.
Curriculum Learning. Training progresses along two axes: (i) phase defining how the latent exteroceptive representation \(z_t\) is obtained, (ii) stage, defining obstacle configuration difficulty.
The task is specified by a persistent target in the world frame \(\mathbf{g}^w \in \mathbb{R}^2\) sampled around the ball position \(\mathbf{b}^w_t\), avoiding velocity resampling at every step.
3.0.0.1 Phase 1 (policy learning).
A four-stage curriculum increases dribbling difficulty as depicted in Figure 1. Stage 0: no obstacles, ball far from the robot. The agent learns approach and dribbling. Stage 1: no obstacles, ball near the robot. The agent refines target-directed dribbling. Stage 2: one static blocker on the ball-target corridor. The agent acquires obstacle avoidance and trajectory recovery. Stage 3: one dynamic opponent on the ball-target corridor interacting with the ball. The agent learns online reactive control.
Different optimization strategies are used for different stages. In particular, Stage 0 uses plain PPO, while stages 1-3 are initialized from the respective previous stage checkpoint and use a DAgger-regularized PPO objective [25] with the stage-0 policy \(\pi_0\) as a teacher:
\[\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{PPO}} + \lambda_{\mathrm{imit}} \mathbb{E}_{s_t}\left[ \left|\left| \pi_\theta(a_t \mid s_t) - \pi_0(a_t \mid s_t) \right|\right|_2^2 \right].\]
The imitation component mitigates catastrophic forgetting, preserving the nominal locomotion and ball-handling strategy learned in Stage 0.
3.0.0.2 Phase 2 (visual adaptation)
Initialized from the Phase 1 Stage 3 checkpoint, the policy is frozen, and a visual encoder is trained to reconstruct the latent \(z_t\) from depth observations.
Visual Encoder Learning. To address partial observability, Phase 2 replaces the privileged latent generator with a visual adaptation module that predicts the same latent from depth observations. The actor input remains unchanged: proprioception concatenated with a latent \(z_t \in \mathbb{R}^{d_z}\) encoding task-relevant exteroceptive information. The two phases differ only in how \(z_t\) is produced (Fig. 2).
3.0.0.3 Phase 1: Privileged encoder.
In the first phase, the latent is generated by a privileged encoder with access to ground-truth task variables, including the state of the ball and the state of the nearest dynamic obstacle. These quantities are represented in the robot coordinate frame and mapped by a small MLP to a latent vector \(z_t\), which is concatenated with the proprioceptive observation and provided to the policy.
3.0.0.4 Phase 2: Visual adaptation encoder.
In the second phase, the privileged encoder is replaced by a visual adaptation module that aims to predict the same latent vector from onboard depth sensing only. Specifically, we use a convolutional neural network (CNN) encoder followed by a gated recurrent unit (GRU). The convolutional layers extract spatial features from each depth frame, while the GRU aggregates them over time to capture temporal continuity and short-horizon motion cues. The recurrent output is then projected into the same latent space used in Phase 1.
Formally, given the current depth frame \(\mathbf{D}_t\) resized to \(108 \times 192\) preserving the original aspect ratio, the visual adaptation module computes \[\mathbf{e}_t = f_{\mathrm{cnn}}(\mathbf{D}_t) \in \mathbb{R}^{64}, \qquad \mathbf{h}_t = f_{\mathrm{gru}}(\mathbf{e}_t,\mathbf{h}_{t-1}) \in \mathbb{R}^{256}, \qquad z_t = f_{\mathrm{proj}}(\mathbf{h}_t) \in \mathbb{R}^{64},\] where \(f_{\mathrm{cnn}}\) is the convolutional encoder, \(f_{\mathrm{gru}}\) is the recurrent temporal aggregator, and \(f_{\mathrm{proj}}\) maps the recurrent state to the policy latent space.
The visual encoder is trained to match the privileged latent through the loss \[\mathcal{L}_{\mathrm{latent}} = \|z_t^{\mathrm{adapt}} - z_t^{\mathrm{priv}}\|_2^2.\] To improve identifiability and stabilize learning, auxiliary prediction heads also regress the ball state and nearest-obstacle state from the latent: \[\mathcal{L}_{\mathrm{adapt}} = \mathcal{L}_{\mathrm{latent}} + \lambda_{\mathrm{ball}}^{\mathrm{pos}}\mathcal{L}_{\mathrm{ball}}^{\mathrm{pos}} + \lambda_{\mathrm{ball}}^{\mathrm{vel}}\mathcal{L}_{\mathrm{ball}}^{\mathrm{vel}} + \lambda_{\mathrm{obs}}^{\mathrm{pos}}\mathcal{L}_{\mathrm{obs}}^{\mathrm{pos}} + \lambda_{\mathrm{obs}}^{\mathrm{vel}}\mathcal{L}_{\mathrm{obs}}^{\mathrm{vel}}.\]
This design preserves a stable actor interface across both phases: only the mechanism used to produce the latent changes, while the policy architecture itself remains unchanged.
4 Experimental Results↩︎
4.1 Simulation Setup and Evaluation Protocol↩︎
All experiments are conducted in simulation using the mjlab framework, on the Booster T1 humanoid dribbling a free soccer ball on a flat terrain.
The policy receives proprioceptive measurements concatenated with a latent vector \(z_t\) produced by the visual adaptation module. At deployment, \(z_t\) is inferred from a head-mounted depth camera.
We evaluate the dribbling policy under three conditions:
No obstacles: target-reaching dribbling without adversaries.
Static obstacle: a single stationary obstacle placed near the ball-to-target corridor.
Ball attacker: a single moving obstacle spawned near the ball-to-target corridor and scripted to chase the ball at a fixed speed.
At each trial, a target is generated in the horizontal plane conditioned on the initial robot-ball configuration. Target placement, obstacle placement, and (for the dynamic case) obstacle speed are sampled independently within predefined ranges; the resulting scene is fixed for the duration of each trial. Obstacle speed is drawn from \([0.1, 0.4]\,\mathrm{m/s}\).
A trial terminates only upon successful target reach (ball-target distance \(\le 0.75\,\mathrm{m}\)), timeout (\(30\,\mathrm{s}\)), robot fall, or ball lost (robot-ball distance above \(2.0\,\mathrm{m}\)). Obstacle collisions do not terminate the trial and are instead recorded as safety metrics. The weights associated with each component of the reward function are outlined in Table 1
| Reward Term | Weight | Reward Term | Weight |
|---|---|---|---|
| Ball velocity tracking | 3.0 | Ball-target reached | 50.0 |
| Ball speed tracking | 2.0 | Robot-obstacle collision | \(-10.0\) |
| Ball heading tracking | 2.0 | Ball-obstacle collision | \(-10.0\) |
| Robot-ball distance | 0.05 | Foot slip | \(-0.1\) |
| Robot-ball yaw | 2.0 | Upright | 1.0 |
| Ball-target progress | 2.0 | Body angular velocity | \(-0.05\) |
| Swing phase | 3.0 | Angular momentum | \(-0.5\) |
| Stance phase | 2.0 | DoF position limits | \(-1.0\) |
| Pose arms | 1.0 | Action rate (L2) | \(-0.1\) |
| Pose legs | 1.0 | Foot swing height | \(-0.25\) |
| Feet distance | \(-6.0\) | Soft landing | \(-1 \times 10^{-5}\) |
| Foot-foot contact | \(-5.0\) | Self-collisions | \(-1.0\) |
| Nonfoot-ball contact | \(-2.0\) |
We report the success rate (SR), time-to-target over successful trials in seconds (T2T), timeout-censored time-to-target in seconds (T2T-C), fall rate (FR), ball-lost rate (LR), robot-obstacle collision rate (RCR), robot-obstacle contacts per trial (RC/t), ball-obstacle collision rate (BCR), ball-obstacle contacts per trial (BC/t), and the minimum ball-obstacle clearance in meters (MBC). The censored time-to-target assigns failed trials the full timeout horizon, preventing low-success policies from appearing artificially fast.
We also report a ball-velocity diagnostic on the same episodes. Let \(v_{\mathrm{ball}}\) denote the realized ball velocity and \(v_{\mathrm{cmd}}\) the commanded ball velocity. We define the vector tracking error as \(e_{\mathrm{vec}} = \lVert v_{\mathrm{ball}} - v_{\mathrm{cmd}} \rVert\), the speed error as \(e_{\mathrm{spd}} = \left| \lVert v_{\mathrm{ball}} \rVert - \lVert v_{\mathrm{cmd}} \rVert \right|\), and the angular error as \(e_{\mathrm{ang}} = \angle\!\left(v_{\mathrm{ball}},\, v_{\mathrm{cmd}}\right)\). In obstacle settings, a good policy may intentionally deviate from the commanded ball velocity to avoid the obstacle while still completing the task successfully.
To distinguish nominal tracking from deliberate avoidance, timesteps in the obstacle conditions are split into blocked and unblocked subsets. A timestep is marked as blocked if at least one active obstacle lies ahead of the ball along the ball-to-target direction, remains before the target, is within \(2.0\,\mathrm{m}\) in forward distance, and lies within \(0.75\,\mathrm{m}\) of the ball-to-target line. Otherwise, it is marked as unblocked.
Finally, we report perception metrics on the same episodes to assess the visual adaptation module during task execution: ball position error \(e_{\mathrm{ball,pos}}\), ball velocity error \(e_{\mathrm{ball,vel}}\), obstacle position error \(e_{\mathrm{obs,pos}}\), obstacle velocity error \(e_{\mathrm{obs,vel}}\), and field-of-view coverage \(c_{\mathrm{fov}}\).
Evaluation is run over 5 random seeds. For each seed, we collect 10 trials per condition and aggregate results across seeds, for a total of 50 trials per condition.
4.2 Dribbling Policy Evaluation↩︎
We first evaluate the final deployable policy, i.e., the policy trained with the full pipeline and executed with the depth-based adaptation encoder. Table 2 summarizes the main task metrics across the three evaluation conditions.
Table 3 reports the velocity diagnostic. In the obstacle-free condition, all timesteps are effectively unblocked. In the two obstacle conditions, we additionally separate the metrics into unblocked and blocked subsets in order to distinguish nominal tracking from obstacle-avoidance behavior.
Table ¿tbl:tab:perception95eval? reports the perception metrics collected during evaluation. These metrics are computed only on timesteps in which the ball lies within the camera field of view. When the ball is outside the field of view, the visual adaptation module is not used to estimate the ball state, and ground-truth ball position is provided instead.
| Condition | SR | T2T | T2T-C | FR | LR | RCR | RC/t | BCR | BC/t | MBC |
| [%] | [s] | [s] | [%] | [%] | [%] | [#/trial] | [%] | [#/trial] | [m] | |
| No obstacles | 100.00 | 11.45 | 11.45 | 0.00 | 0.00 | – | – | – | – | – |
| Static obstacle | 96.00 | 13.29 | 13.95 | 4.00 | 0.00 | 8.00 | 0.08 | 4.00 | 0.10 | 1.73 |
| Ball attacker | 46.00 | 11.82 | 21.64 | 52.00 | 2.00 | 68.00 | 0.72 | 40.00 | 0.70 | 1.36 |
| Condition | Segment | \(e_{\mathrm{vec}}\) [m/s] | \(e_{\mathrm{spd}}\) [m/s] | \(e_{\mathrm{ang}}\) [deg] |
|---|---|---|---|---|
| No obstacles | All timesteps | 0.71 | 0.62 | 28.39 |
| Static obstacle | Unblocked | 0.74 | 0.62 | 35.62 |
| Static obstacle | Blocked | 0.74 | 0.60 | 33.96 |
| Ball attacker | Unblocked | 0.74 | 0.61 | 34.27 |
| Ball attacker | Blocked | 0.79 | 0.60 | 43.00 |
The final policy achieves perfect performance in the no-obstacle condition, with \(100\%\) success and no failures, showing that nominal target-driven dribbling is reliably preserved. Performance remains strong in the static obstacle setting, where success stays high at \(96\%\) with only a modest increase in time-to-target and low collision rates, indicating effective handling of a single stationary obstacle. In contrast, the ball-attacker condition remains substantially more challenging: success drops to \(46\%\), while both fall and collision rates increase markedly, showing that robust dribbling against an actively moving adversary is still an open problem.
The velocity diagnostic is consistent with this trend. Tracking errors remain similar between the no-obstacle and static-obstacle conditions, suggesting that the policy can avoid a stationary obstacle without strongly disrupting the commanded ball motion. In the ball-attacker setting, however, the blocked timesteps exhibit the largest angular and vector errors, indicating stronger deviations from the nominal command during active avoidance. Perception metrics remain relatively stable across conditions, with only a small degradation in ball-state estimation and comparable obstacle-state errors in the two obstacle settings. This suggests that the main limitation in the hardest condition is not a severe perception failure, but the difficulty of closed-loop control against a moving opponent.
4.3 Ablation: Curriculum Stages↩︎
To study the effect of curriculum training, we evaluate the checkpoint produced after each curriculum stage on all evaluation conditions. As described in Section 3, the full curriculum consists of four stages. However, in this ablation, we exclude Stage 0, since it only covers a basic prerequisite skill that is shared by all subsequent stages. For each remaining checkpoint, we report the success rate on the three evaluation conditions defined by the current protocol: no obstacles, static obstacle, and ball attacker. Table [tbl:tab:curriculum95success], where each stage has been evaluated over 5 seeds, each with 10 trials, shows the effectiveness of the curriculum strategy, since stage 1 performs poorly when obstacles are present in the scene.
5 Conclusion↩︎
This paper presented a visual temporal policy for autonomous humanoid soccer dribbling. We formulated the task as an opponent-aware loco-manipulation problem in which the robot must jointly maintain balance, control the ball, and avoid obstacles.
The experimental results show that the proposed policy reliably solves nominal target-driven dribbling, achieving \(100\%\) success in the no-obstacle condition, and also performs well in the single static-obstacle setting, where success remains high at \(96\%\). Performance degrades substantially in the ball-attacker setting, where success drops to \(46\%\) and both fall and collision rates increase markedly. This indicates that the main remaining challenge is robust closed-loop control against an actively moving adversary, rather than nominal dribbling itself. The velocity and perception diagnostics support this interpretation: deviations from the commanded ball motion increase during blocked attacker interactions, while perception errors remain relatively stable across conditions.
Overall, these results suggest that combining temporal visual adaptation with reinforcement learning is a promising approach for humanoid loco-manipulation tasks. However, the current evidence is limited by the small number of independently trained policies and by the fact that all experiments are conducted in simulation.
Beyond humanoid soccer, the same framework could support other embodied robotic tasks requiring perception-driven control under partial observability. Future work will focus on harder and more diverse opponent behaviors, stronger robustness against moving adversaries, and sim-to-real transfer on the Booster T1 humanoid platform.