1 Introduction↩︎

Dynamic catching of fast-moving objects [1]–[3] is important for warehouse automation, aerial delivery, service robotics, and robot sports. Unlike static grasping [4], [5], it requires short-horizon reaction, coordinated arm-hand motion, and robustness to impact under varying incoming conditions.

Existing methods [6] are typically based on online optimization [7], [8] or end-to-end policy learning [9], [10]. Optimization can enforce constraints but is often slow in contact-rich interception. End-to-end reinforcement learning [11], [12] can discover successful behaviors, but direct policy execution [13], [14] offers limited control over whole-trajectory feasibility [15]. A practical solution should combine offline exploration with structured online generation.

Inspired by compliant human interception in tennis (see Fig. 1), we propose an offline-to-online framework for impact-aware catching. RL in simulation generates successful arm-hand trajectories over diverse object initial states. A low-dimensional kinodynamic trajectory manifold is then learned and queried online from the estimated object state [16], arvanitidis2018metric?. The contributions are: (1) an RL pipeline for collecting feasible and impact-aware catching trajectories; (2) a conditioned trajectory manifold for efficient online synthesis; and (3) a trajectory-centric framework combining interception with compliant impact absorption.

2 Problem Formulation↩︎

We consider the problem of catching a fast-moving object in free flight using a 7-DoF manipulator equipped with a five-finger compliant gripper. Let the object state at time \(t\) be \[x_o(t) = \begin{bmatrix} p_o(t)^\top & v_o(t)^\top \end{bmatrix}^\top \in \mathbb{R}^6,\] where \(p_o(t) \in \mathbb{R}^3\) and \(v_o(t) \in \mathbb{R}^3\) denote object position and velocity. The robot state is \[x_r(t) = \begin{bmatrix} q(t)^\top & \dot{q}(t)^\top \end{bmatrix}^\top, \quad u(t) = \tau(t),\] where \(q(t) \in \mathbb{R}^7\) and \(\tau(t) \in \mathbb{R}^7\) are joint coordinates and torques.

Given an estimated object trajectory \(\hat{x}_o(t)\) over \(t \in [0,T]\), the goal is to generate a dynamically feasible catching motion reaching a valid interception state at some catching time \(t_c \in [0,T]\) while ensuring stable post-impact capture. Let \(y_e(q)\) and \(\dot{y}_e(q,\dot{q})\) denote the end-effector pose and spatial velocity. The interception condition is \[y_e(q(t_c)) \approx y_o(t_c), \quad \dot{y}_e(q(t_c),\dot{q}(t_c)) \approx \dot{y}_o(t_c),\] where \(y_o(t_c)\) and \(\dot{y}_o(t_c)\) are the object pose and velocity in the grasp frame.

To improve robustness under high-speed impact, we optimize a trajectory parameterization \(\theta\) on a kinodynamic manifold by minimizing \[\min_{\theta,\, t_c} J = w_1 J_{\mathrm{track}} + w_2 J_{\mathrm{impact}} + w_3 J_{\mathrm{grasp}} + w_4 J_{\mathrm{effort}},\] subject to \[\dot{x}_r = f(x_r,u), \quad q \in \mathcal{Q}, \quad \dot{q} \in \dot{\mathcal{Q}}, \quad \tau \in \mathcal{U},\] and geometric feasibility constraints, \[\mathrm{dist}\big(\mathcal{R}(q(t)),\mathcal{O}(t)\big) \geq 0, \quad t \in [0,t_c),\] where \(J_{\mathrm{track}}\), \(J_{\mathrm{impact}}\), \(J_{\mathrm{grasp}}\), and \(J_{\mathrm{effort}}\) penalize interception error, impact severity, poor grasp geometry, and control effort, respectively.

The objective is to learn a low-dimensional trajectory manifold that maps predicted object motion to feasible arm-hand catching motions for real-time impact-aware generation.

3 Methodology↩︎

The method follows an offline-to-online pipeline. Offline, RL in simulation discovers feasible and impact-aware catching motions over diverse object conditions [17], [18]. Online, the learned manifold generates a feasible reference trajectory from the incoming object state for real-time interception and compliant capture.

3.1 RL-Based Feasible Trajectory Generation↩︎

We first formulate data generation as a finite-horizon Markov decision process. At time step \(k\), the policy observes the object state, robot state, and relative interception geometry, \[s_k = \begin{bmatrix} x_o(k)^\top & x_r(k)^\top & \phi_k^\top \end{bmatrix}^\top,\] where \(\phi_k\) includes relative position, relative velocity, and hand-object alignment. The policy outputs an arm-hand control command \[a_k = \pi_\omega(s_k),\] which may be implemented as target joint increments, joint torques, or another low-level reference. The reward decomposes into three stages matching the catching phases, plus a penalty: \[r_k = R_{\mathrm{s1},k} + R_{\mathrm{s2},k} + R_{\mathrm{s3},k} - R_{\mathrm{pen},k}.\]

Stage 1 (Pre-Catch Tracking) promotes approach and interception alignment before cushioning: \[R_{\mathrm{s1},k} = \lambda_1 r_{\mathrm{dist}} + \lambda_2 r_{\mathrm{align}} + \lambda_3 r_{\mathrm{progress}},\] where \(r_{\mathrm{dist}}\), \(r_{\mathrm{align}}\), and \(r_{\mathrm{progress}}\) encourage proximity, palm–velocity alignment, and progress toward the interception point.

Stage 2 (Soft Contact & Cushioning) handles force unloading once the object enters the soft zone (\(d\!\le\!0.3\) m) and continues through the post-contact drag buffer: \[R_{\mathrm{s2},k} = \lambda_4 r_{\mathrm{vel}} + \lambda_5 r_{\mathrm{drag}} + \lambda_6 r_{\mathrm{impact}},\] where \(r_{\mathrm{vel}}\), \(r_{\mathrm{drag}}\), and \(r_{\mathrm{impact}}\) encourage velocity matching and compliant following while penalizing peak contact force.

Stage 3 (Post-Catch Stabilization) promotes secure retention after buffering: \[R_{\mathrm{s3},k} = \lambda_7 r_{\mathrm{grasp}} + \lambda_8 r_{\mathrm{stable}},\] where \(r_{\mathrm{grasp}}\) and \(r_{\mathrm{stable}}\) favor fingertip enclosure and low residual object–hand motion. The penalty term enforces safety and smooth impact response: \[R_{\mathrm{pen},k} =\lambda_{9} p_{\mathrm{impulse}} + \lambda_{10} p_{\mathrm{col}} + \lambda_{11} p_{\mathrm{limit}} + \lambda_{12} p_{\mathrm{drop}},\] where \(p_{\mathrm{impulse}}\) penalizes abrupt force variation, and the remaining terms suppress collisions, joint-limit violations, and object drops.

After convergence, we retain successful episodes only and build a dataset \[\mathcal{D} = \left\{\left(x_{o,0}^{(i)}, \tau^{(i)}\right)\right\}_{i=1}^{N},\] where \(x_{o,0}^{(i)}\) is the initial object state and \(\tau^{(i)}\) is the corresponding successful catching trajectory. RL therefore serves as an offline feasibility explorer rather than the final online controller.

3.2 Trajectory Representation and Manifold Learning↩︎

RL trajectories generally have different durations and catching times. We therefore align them by the interception instant and resample them to a fixed horizon of \(H\) steps. The \(i\)-th trajectory is represented as \[\xi^{(i)} = \left[q_1^\top, \dots, q_H^\top, \dot{q}_1^\top, \dots, \dot{q}_H^\top\right]^\top.\] Let \(\mathcal{X} \subset \mathbb{R}^{D}\) denote the ambient trajectory space with \(D = 14H\). Only a subset satisfies kinodynamic feasibility: \[\mathcal{F} = \left\{\xi \in \mathcal{X} \,\middle|\, \dot{x}_r = f(x_r,u),\;q \in \mathcal{Q},\;\dot{q} \in \dot{\mathcal{Q}},\;\tau \in \mathcal{U}\right\}.\] Successful catching motions are modeled as a conditional Riemannian manifold \(\mathcal{M}_{x_{o,0}} \subset \mathcal{F}\) indexed by object initial state \(x_{o,0}\). The manifold is parameterized by a latent variable \(z \in \mathbb{R}^{d}\) with \(d \ll D\) through \[\mathcal{M}_{x_{o,0}} = \left\{\xi = G_\varphi(z, x_{o,0}) \mid z \in \mathbb{R}^{d}\right\}.\] This yields a low-dimensional embedded submanifold of the kinodynamic trajectory space. Given a trajectory \(\xi\) and object initial state \(x_{o,0}\), the encoder maps the motion into a latent code, \[z = E_\psi(\xi, x_{o,0}), \quad d \ll \mathrm{dim}(\xi),\] and the decoder reconstructs the trajectory conditioned on the same object state, \[\hat{\xi} = G_\varphi(z, x_{o,0}).\]

The local geometry is characterized by the decoder Jacobian \[J_G(z, x_{o,0}) = \frac{\partial G_\varphi(z, x_{o,0})}{\partial z} \in \mathbb{R}^{D \times d},\] whose column space defines the tangent space \(T_{\xi}\mathcal{M}_{x_{o,0}}\) at \(\xi = G_\varphi(z, x_{o,0})\). A pullback Riemannian metric on the latent space is then given by \[\mathbf{G}(z; x_{o,0}) = J_G(z, x_{o,0})^\top W J_G(z, x_{o,0}),\] where \(W \succeq 0\) weights trajectory coordinates. This metric measures latent perturbations by their induced kinodynamic deformation in trajectory space.

Under this metric, the geodesic connecting two latent states \(z_a\) and \(z_b\) is the minimum-energy curve \[\gamma^{\ast} = \arg\min_{\gamma(0)=z_a,\,\gamma(1)=z_b} \int_0^1 \dot{\gamma}(s)^\top \mathbf{G}(\gamma(s); x_{o,0}) \, \dot{\gamma}(s) \, ds,\] and the corresponding manifold distance is \[d_{\mathcal{M}}(z_a,z_b; x_{o,0}) = \left( \int_0^1 \dot{\gamma}^{\ast}(s)^\top \mathbf{G}(\gamma^{\ast}(s); x_{o,0}) \, \dot{\gamma}^{\ast}(s) \, ds \right)^{\frac{1}{2}}.\] This allows interpolation to follow the intrinsic geometry of the feasible set. The model is trained by minimizing \[\mathcal{L}_{\mathrm{manifold}} = \mathcal{L}_{\mathrm{rec}} + \lambda_1 \mathcal{L}_{\mathrm{smooth}} + \lambda_2 \mathcal{L}_{\mathrm{dyn}} + \lambda_3 \mathcal{L}_{\mathrm{geo}} + \lambda_4 \mathcal{L}_{\mathrm{tan}},\] where \(\mathcal{L}_{\mathrm{rec}} = \|\xi - \hat{\xi}\|_2^2\) penalizes reconstruction error, \(\mathcal{L}_{\mathrm{smooth}}\) encourages temporal smoothness, and \(\mathcal{L}_{\mathrm{dyn}}\) penalizes dynamic infeasibility. To preserve intrinsic geometry, we introduce a geodesic regularization term \[\mathcal{L}_{\mathrm{geo}} = \sum_{(i,j) \in \mathcal{N}} \left( d_{\mathcal{M}}\left(z_i,z_j; x_{o,0}^{(i)}\right) - \|\xi^{(i)} - \xi^{(j)}\|_{W} \right)^2,\] which keeps manifold distances consistent with trajectory dissimilarity in the ambient kinodynamic space. We further impose a tangent-space regularization term \[\mathcal{L}_{\mathrm{tan}} = \sum_{(i,j) \in \mathcal{N}} \left\| \left(\xi^{(j)} - \xi^{(i)}\right) - J_G\left(z_i, x_{o,0}^{(i)}\right)\left(z_j-z_i\right) \right\|_{W}^2,\] where \(\mathcal{N}\) denotes local neighbor pairs. Together, \(\mathcal{L}_{\mathrm{geo}}\) and \(\mathcal{L}_{\mathrm{tan}}\) improve interpolation and online generation. To enable direct generation from the object initial state, we further learn a condition-to-latent mapping \[z = M_\eta(x_{o,0}),\] so that the manifold can be queried without running RL online.

3.3 Online Trajectory Synthesis and Execution↩︎

At test time, perception estimates the incoming object initial state \(x_{o,0}\). The latent predictor \(M_\eta\) produces a low-dimensional code, and the decoder generates the reference trajectory, \[\hat{\xi} = G_\varphi\big(M_\eta(x_{o,0}), x_{o,0}\big).\] Equivalently, online generation can be viewed as solving a low-dimensional search problem on the manifold, \[z^{\ast} = \arg\min_{z \in \mathbb{R}^{d}} \; \ell\big(G_\varphi(z, x_{o,0}), x_{o,0}\big) + \beta C_{\mathrm{dyn}}\big(G_\varphi(z, x_{o,0})\big),\] where \(\ell(\cdot)\) measures interception and impact quality, and \(C_{\mathrm{dyn}}(\cdot)\) penalizes residual feasibility violations. Because the search is performed in latent space rather than full trajectory space, the dimension is greatly reduced and the solution stays close to the learned feasible manifold. The latent update can also be written using the Riemannian gradient induced by \(\mathbf{G}(z; x_{o,0})\), \[z_{k+1} = \mathrm{Exp}_{z_k}\left(-\rho \, \mathbf{G}(z_k; x_{o,0})^{-1} \nabla_z \mathcal{J}(z_k)\right),\] where \(\mathcal{J}(z) = \ell\big(G_\varphi(z, x_{o,0}), x_{o,0}\big) + \beta C_{\mathrm{dyn}}\big(G_\varphi(z, x_{o,0})\big)\) and \(\mathrm{Exp}_{z_k}(\cdot)\) denotes the exponential map on the manifold. This update keeps optimization aligned with the intrinsic geometry of the conditional manifold.

Compared with direct RL execution, this trajectory-centric formulation provides a complete interception motion before contact. Compared with online nonlinear optimization, it shifts heavy computation offline and enables fast trajectory synthesis at run time.

The generated reference is tracked by a low-level compliant controller near contact, improving capture robustness.

2pt

4 Experiments↩︎

4.1 Experimental Setup↩︎

We evaluate in MuJoCo with a 7-DoF xArm and a five-finger dexterous gripper. As shown in Table 2 and Fig. 3, five geometric primitives at two size scales yield ten object variants (mass \(\in[45,55]\) g). Three domain-randomization pipelines (Table 3)—observation, dynamics, and control—are applied to bridge the sim-to-real gap.

3pt

3pt

4.2 Results and Analysis↩︎

Fig. 2 (left) shows representative catching sequences and the right panel compares peak-aligned impact metrics and object vertical velocity, showing smoother compliant capture. Table 1 confirms the advantage: our method attains 85.1% success with peak contact force 32.1 N versus the baseline’s 54.7% and 120.7 N.

4.2.1 Sensitivity to State Estimation Error↩︎

We inject bounded uniform perturbations into the estimated initial object state. As shown in Fig. 4, success remains above 70% under position perturbations up to 0.05 m and velocity perturbations up to 0.3 m/s, owing to the observation-pipeline randomization (Table 3).

5 Conclusion↩︎

This paper presents an offline-to-online framework for impact-aware compliant catching of fast-moving objects. By combining RL-based trajectory exploration with a conditioned kinodynamic manifold, the method enables real-time synthesis of feasible catching motions from the estimated object state without online nonlinear optimization. Experimental results in simulation demonstrate superior catching performance compared to direct policy execution, with higher success rates and significantly reduced impact forces across ten object variants spanning five geometric primitives at two size scales (Fig. 3). Sensitivity analysis further confirms that the framework degrades gracefully under realistic levels of state estimation error.

All experiments in this work are conducted in the MuJoCo simulation environment, and real-robot validation is currently lacking. Real-world factors such as actuator dynamics, sensor noise, and imperfect state estimation may introduce challenges not fully captured in simulation, despite the extensive domain randomization employed during training. Future work will focus on real-robot experiments to evaluate the sim-to-real transferability of the learned manifold and the compliant control strategy.

References↩︎