RigPI: Dynamic Parameter Identification of Rigid Body via VLM-Seeded Differentiable Simulation
June 23, 2026
Abstract
Accurate physical parameter identification of manipulated objects is fundamental to advanced robotic manipulation and the construction of faithful digital twins. However, acquiring physically consistent inertial and frictional properties from real-world interactions remains challenging due to sensing noise, modeling errors, and limited prior knowledge. This paper presents RigPI, a systematic framework for identifying dynamic parameters of both unconstrained rigid bodies and multi-link rigid bodies during robot–object interaction. RigPI integrates vision-based semantic priors, force–torque measurements, and motion observations within a differentiable simulation pipeline. A vision-language model (VLM) provides informed initialization and a constrained search space, while gradient information from a differentiable physics simulator enables efficient and stable parameter refinement. The proposed two-stage optimization strategy alleviates sensitivity to noise and avoids physically implausible solutions. Extensive real-world experiments on objects with revolute and prismatic joints demonstrate that RigPI achieves accurate and stable parameter estimates, and successfully reproduces manipulation trajectories on a real robot with parameter-aware predictive validity. These results highlight the effectiveness and robustness of RigPI for real-world robotic system identification tasks. For code, model, and more supporting materials, please visit our website at https://sites.google.com/view/rigpi.
1 INTRODUCTION↩︎
Accurate dynamic parameter identification of external objects, including both inertial and frictional properties, is essential for advanced robotic manipulation, such as predictive control, and creating digital twins that mirror real-world physics. However, accurate and automatic acquisition of such parameters during manipulation remains a significant challenge. It requires leveraging diverse sources of information from observed interaction data, such as trajectories, force/torque signals, and prior knowledge, while optimizing the parameters in a carefully constrained search space to avoid physically implausible solutions. To this end, we introduce RigPI, which integrates high-level priors with rich mechanical feedback and orchestrates them within a gradient-based optimization pipeline.
Traditional approaches to dynamic parameter identification are typically based on inverse-dynamics formulations, where inertial parameters are estimated as linear functions of joint torques using least-squares or robust variants [1]–[3]. These methods assume precise knowledge of system dynamics and are highly sensitive to measurement noise, which directly corrupts both the regressor and the observation vector in the linear system. They also require carefully designed excitation trajectories to render the regression problem well-posed. As a result, small modeling inaccuracies or sensing disturbances may lead to physically inconsistent or unstable parameter estimates. To mitigate these challenges, recent developments have shifted attention toward differentiable simulation-based identification schemes that evaluate candidate parameters through forward simulation of the system’s time evolution. Recent differentiable physics simulators enable gradient-based system identification in complex, contact-rich environments [4]–[6]. They provide powerful solvers for parameter optimization and offer rich support for geometry, collision, and other environmental features [7], [8].
In this work, we propose RigPI, a real-to-sim-to-real framework that tackles rigid object identification through a systematic process: First, it takes as input RGB-D images of the object and robot–object interaction data (applied forces, torques, and resulting poses), and then aligns the object state from the real world to a differentiable simulator. Second, it adopts a two-stage optimization mechanism to estimate the object’s dynamic parameter vector \(\boldsymbol{\theta}\) through the differentiable simulator. A vision-language model (VLM) is adopted to produce an informed initialization and a feasible search space from images, and then differentiable simulation with NVIDIA Newton [4] provides gradients that drive gradient-based optimization to refine the parameters efficiently.
To evaluate the proposed pipeline, we conduct a comprehensive test on a real-world robotic platform using a diverse set of external objects, including both unconstrained rigid bodies and articulated bodies with revolute and prismatic joints. We assess the accuracy and stability of the identified parameters and further validate their correctness by reproducing manipulation trajectories on real hardware using the parameter-aware predictive validity. Our results demonstrate that RigPI can achieve high-fidelity parameter identification while ensuring stability in real-world robotic applications. In addition, we perform ablation studies to verify the accuracy and necessity of the visual priors in guiding the identification process.
We summarize our contributions as follows:
We present RigPI, a pipeline to identify the dynamic parameters of both unconstrained and multi-link rigid bodies, including inertial and frictional properties, via robotic manipulation.
We conduct real-world robot tasks and evaluations to demonstrate its practical applicability and superior performance on both unconstrained and multi-link rigid objects.
2 RELATED WORKS↩︎
2.1 Differentiable Simulation for Parameter Identification↩︎
Traditional parameter identification is largely based on inverse-dynamics formulations, where joint torques are modeled as linear functions of inertial parameters. Classical work by Atkeson et al. [1] established a least-squares framework for identifying mass, center of mass, and inertia. Subsequent studies improved robustness and adaptability through recursive total least-squares schemes [2] and noise-aware weighted least-squares variants [3]. Despite their effectiveness in structured settings, these methods require carefully designed excitation trajectories, assume simplified contact and friction models, and are highly sensitive to measurement noise and modeling errors. Moreover, their linear assumptions often fail in contact-rich or visually inferred manipulation scenarios, resulting in ill-posed estimates or physically inconsistent parameters.
With the rise of differentiable simulators, modern methods now leverage analytic gradients of physics engines to perform end-to-end optimization, offering a unified framework that integrates sensor feedback and gradient-based optimization for system identification. Early explorations of differentiable dynamics [9] have evolved into general-purpose systems such as DiffTaichi [5], as well as physically accurate and highly optimized engines like Dojo [6] and Newton [4]. These GPU-accelerated simulators provide efficient forward and backward passes for gradient-based optimization. However, they are primarily computation backbones rather than complete identification solutions: performance remains sensitive to initialization and parameter bounds, and they lack mechanisms for constructing well-posed problems from unstructured real-world data.
Recent studies have advanced differentiable physics simulation by enabling optimization over high-dimensional or neural-augmented objects [7], [10], [11], integrating deep learning for robotics applications, and providing systematic frameworks for parameter optimization and model fitting. In robotic system identification, these methods have predominantly focused on inertial parameters of single, monolithic rigid bodies [12], or employed data-driven models such as Graph Networks [13] and physics-informed Deep Lagrangian Networks [14]. Other works have extended differentiable parameter identification to fluids and deformable objects, e.g., DiffStir [15] and DiffCP [16]. While these approaches offer complementary strategies, they do not fully address the systematic identification of unconstrained and multi-link bodies with physics-grounded, interpretable dynamic parameters.
Our framework, RigPI, bridges this gap by formulating the identification problem as a structured two-stage process. In contrast to purely learning-based methods that operate as black boxes, RigPI preserves the physical interpretability of the estimated parameters (\(\boldsymbol{\theta}\)), ensuring both accuracy and transparency.
2.2 Object-Centric Real-World System Identification↩︎
Recent progress in foundation models has revived the idea of inferring physical properties from visual data. Early CNN-based approaches regressed quantities such as mass from simulated interactions [17], [18], but suffered from limited generalization. The emergence of Vision-Language Models (VLMs) enables zero-shot, common-sense physical reasoning, with benchmarks such as PhysBench [19] quantifying these capabilities.
However, current VLM-based systems are not designed for precise real-world system identification. They typically operate in open-loop settings or serve as components within Vision-Language-Action (VLA) pipelines [20], which emphasize task planning rather than quantitative parameter estimation. Nevertheless, recent studies demonstrate the potential of physically grounded reasoning, including VLM-based liquid reasoning [21], robotic reachability analysis [22], and navigation with physical constraints [23]. These efforts suggest that visual priors can be effectively coupled with physical inference beyond semantic reasoning alone.
RigPI addresses this gap by treating the VLM’s output as an informed prior rather than a final answer, formulating an optimization problem that tightly couples object-centric priors with high-fidelity gradient information from a differentiable simulator. This synergy establishes a robust identification framework capable of overcoming the limitations of approaches that rely solely on either semantic priors or differentiable simulation.
3 PROBLEM STATEMENT↩︎
The central problem addressed in this research is dynamic parameter identification for external objects, which aims to estimate mass, inertial, and frictional properties by observing their response to a series of controlled manipulations.
3.1 Dynamic Parameters↩︎
For a rigid body, the concerned parameters, which cover major parameters affecting the Newtonian motion are given as follows:
Mass (\(m\)): A scalar quantity representing the body’s translational inertia.
Center of Mass (\(\boldsymbol{com}\)): A vector in \(\mathbb{R}^3\) specifying the mean position of the mass distribution.
Inertia Tensor (\(\boldsymbol{I}\)): A \(3 \times 3\) symmetric positive-definite matrix characterizing the body’s rotational inertia relative to its center of mass.
Coefficient of Surface Friction (\(\mu\)): A scalar modeling frictional forces arising from contact with external surfaces.
To note, such a definition can be naturally extended to multi-link bodies which have only a single rigid part moved under manipulation along a 1-DoF joint, such as a cabinet or a drawer. For the friction property in unconstrained and multi-link rigid bodies, our model adopts Coulomb friction.
To put the parameters together and make them suitable for optimization, we vectorize all identifiable parameters into \(\boldsymbol{\theta}\): \[\boldsymbol{\theta} = \left( m, \boldsymbol{com}, \boldsymbol{I}, \mu \right).\]
For an unconstrained rigid body, this denotes the vectorized unique elements of mass, center of mass, inertia tensor, and friction coefficient. For a multi-link rigid body, this vector is simply extended to encompass the full set of parameters \(\boldsymbol{\theta}_{i,j}=(m_i, \boldsymbol{com}_i, \boldsymbol{I}_i, \mu_{\text{joint},j})\) from all its constituent links and joints, with \(i\) and \(j\) respectively indexing the links and joints in the articulated structure.
3.2 Parameter Identification↩︎
Ultimately, our goal is to find an optimal parameter vector \(\boldsymbol{\theta}^*\) that best explains the observed physical behavior. However, searching the entire parameter space is inefficient and prone to physically implausible solutions. Therefore, we formulate this as a constrained optimization problem. This requires both an initial parameter guess, \(\boldsymbol{\theta}^{(0)}\), and a corresponding constraint for optimization, \(\mathcal{C}\), to define the feasible search space. The initial guess is a vector of estimated parameters: \[\boldsymbol{\theta}^{(0)} = \left(\hat{m}_0, \widehat{\boldsymbol{com}}_0, \widehat{\boldsymbol{I}}_0, \hat{\mu}_0 \right).\] The optimization is then confined within the range: \[\mathcal{C} = [\boldsymbol{\theta}_{\min}, \boldsymbol{\theta}_{\max}].\] Both the initial guess and the plausible range are derived from high-level base properties (e.g., density, dimensions) inferred from the object’s visual appearance, as summarized in Table 1.
| Parameter | Derived Constraint |
|---|---|
| Mass (\(m\)) | Plausible range \([m_{\min}, m_{\max}]\) |
| Center of Mass (\(\bm{com}\)) | Plausible range \([\bm{com}_{\min}, \bm{com}_{\max}]\) |
| Inertia Tensor (\(\bm{I}\)) | Element-wise bounds \([\bm{I}_{\min}, \bm{I}_{\max}]\) |
| Friction Coeff. (\(\mu\)) | Plausible range \([\mu_{\min}, \mu_{\max}]\) |
The final optimization objective is thus to find the parameter vector \(\boldsymbol{\theta}^*\) that minimizes a loss function \(\mathcal{L}(\boldsymbol{\theta})\) while remaining within this physically-grounded search space, starting from the initial guess \(\boldsymbol{\theta}^{(0)}\). The specific methodology for deriving the initial values and bounds, and for formulating the loss function, will be detailed in Section 4.
4 RigPI METHOD↩︎
Our proposed RigPI addresses the challenge of identifying the physical parameters of rigid parts through a real-to-sim-to-real, differentiable optimization framework. The approach first acquires data from the real world in Sec. 4.1, and then performs parameter optimization in Sec. 4.2 with vision-guided parameter initialization and gradient-based optimization. The overall pipeline of RigPI is depicted in Fig. 2.
4.1 Object State Observation↩︎
In the initial stage, we acquire experimental data by having a robotic manipulator interact with the object of interest, which can be either an unconstrained rigid body or a rigid part of a multi-link body. The robot records two primary types of information for each interaction:
Wrench (\(\boldsymbol{f}, \boldsymbol{\tau}\)): This represents the forces and torques applied to the object under test during manipulation. The interaction is captured by a force vector \(\boldsymbol{f} \in \mathbb{R}^3\) and a torque vector \(\boldsymbol{\tau} \in \mathbb{R}^3\), which is applied to the center of mass of the object being identified.
Object Pose (\(\boldsymbol{x}, \boldsymbol{q}\)): This describes the position and orientation of the object being tracked. The position of its center of mass is given by the vector \(\boldsymbol{x} \in \mathbb{R}^3\). Its orientation is represented by a unit quaternion \(\boldsymbol{q} \in SO(3)\). For a multi-link rigid body, although the whole state includes all joint angles, we focus only on the interacting rigid parts here.
The collected dataset, denoted as \(\mathcal{D} = \{(\boldsymbol{f}_k, \boldsymbol{\tau}_k, \boldsymbol{x}_k, \boldsymbol{q}_k)\}_{k=1}^N\), where \(k\) means the \(k\)-th iteration step and \(N\) means the number of iteration steps, providing a comprehensive record of the object’s dynamic behavior under controlled robotic manipulation.
4.2 Parameter Initialization and Optimization↩︎
After collecting interaction data, our next step is to identify the object’s dynamic parameters by minimizing a loss function that measures the discrepancy between simulated and observed behavior. Since this optimization is highly sensitive to the initial guess, obtaining a good initialization is crucial. To this end, we leverage Vision-Language Models (VLMs), such as Intern-S1 [24] and Gemini-3 Pro [25], to extract an informed initial parameter vector \(\boldsymbol{\theta}^{(0)}\) and define a plausible search space \(\mathcal{C}\). Specifically, we input clear and unoccluded RGB-D images of the target object into the VLM, which can infer its geometric properties (pose, scale, volume, internal structure) and material characteristics, providing initial estimates and bounds for density (\(\rho\)) and friction coefficient (\(\mu\)), as defined in Section 3. Combined with the assumption of uniform mass distribution, this yields a full set of initial parameters and their corresponding bounds.
With the initialized parameter, the identification process can be done via differentiable simulation using the NVIDIA Newton physics engine [4]. The collected wrench data \((\boldsymbol{f}_k, \boldsymbol{\tau}_k)\) drive the simulation, which propagates the object’s state over time given the current parameter guess \(\boldsymbol{\theta}\). For unconstrained rigid bodies, a semi-implicit integrator is used, whereas multi-link rigid bodies are simulated using a Featherstone integrator.
Figure 3: Examples of physical parameter identification using our RigPI system. Each subfigure shows the unconstrained or multi-link rigid body being identified (left) and the predicted values. Units are: Mass [kg], \(\boldsymbol{C}\) [m], \(\boldsymbol{I}\) [kg\(\cdot\)m\(^2\)], Friction (coefficient of friction, unitless). \(\boldsymbol{C}\) denotes center of mass, and \(\boldsymbol{I}\) denotes rotational inertia.. a — Cube, b — Sphere, c — Apple, d — Drawer, e — Door
To guide the optimization, we define an augmented loss function that combines a pose discrepancy term with a penalty for violating physical constraints. The primary loss term, \(\mathcal{L}_{\text{pose}}(\boldsymbol{\theta})\), quantifies the difference between the simulated and empirically observed poses: \[\label{equa:loss32function} \mathcal{L}_{\text{pose}}(\boldsymbol{\theta}) = \frac{1}{N} \sum_{k=1}^N \left( \|\boldsymbol{x}_{k}(\boldsymbol{\theta}) - \boldsymbol{x}_k\|^2 + \|\boldsymbol{q}_{k}(\boldsymbol{\theta}) - \boldsymbol{q}_k\|^2 \right).\tag{1}\] To this, we add a penalty term, \(P(\boldsymbol{\theta}, \mathcal{C})\), which enforces the parameter bounds \(\mathcal{C} = [\boldsymbol{\theta}_{\min}, \boldsymbol{\theta}_{\max}]\). Specifically, \(P(\boldsymbol{\theta}, \mathcal{C})\) is defined as \[\begin{align} P(\boldsymbol{\theta}, \mathcal{C}) = \sum_{i=1}^{d} \Big( &\max\{0, \theta_i - \theta_{\max,i}\}^2 \\ + &\max\{0, \theta_{\min,i} - \theta_i\}^2 \Big), \end{align}\] where \(d\) is the dimension of \(\boldsymbol{\theta}\). This formulation ensures that no penalty is applied if \(\boldsymbol{\theta}\) lies within the valid region \(\mathcal{C}\), while quadratic penalties are imposed for violations of the lower or upper bounds. The final objective is to minimize the total loss, \(\mathcal{L}_{\text{total}}(\boldsymbol{\theta})\): \[\mathcal{L}_{\text{total}}(\boldsymbol{\theta}) = \mathcal{L}_{\text{pose}}(\boldsymbol{\theta}) + \lambda P(\boldsymbol{\theta}, \mathcal{C}),\] where \(\lambda\) is a weighting hyperparameter that controls the strength of the penalty. The differentiable nature of the NVIDIA Newton engine allows for the efficient computation of the gradient \(\nabla_{\boldsymbol{\theta}} \mathcal{L}_{\text{total}}(\boldsymbol{\theta})\).
The final stage employs an iterative gradient descent algorithm to find the optimal parameters \(\boldsymbol{\theta}^*\). The optimization objective is defined as: \[\boldsymbol{\theta}^* = \arg\min_{\boldsymbol{\theta}} \left( \mathcal{L}_{\text{pose}}(\boldsymbol{\theta}) + \lambda P(\boldsymbol{\theta}, \mathcal{C}) \right),\] where \(\lambda\) is a weighting hyperparameter controlling the penalty strength. In all our experiments, we empirically set \(\lambda\) to \(0.1\). Starting from the initial guess \(\boldsymbol{\theta}^{(0)}\) and using an adaptive learning rate, the iterative update rule is given by: \[\boldsymbol{\theta}^{(t+1)} = \boldsymbol{\theta}^{(t)} - \alpha^{(t)} \nabla_{\boldsymbol{\theta}} \mathcal{L}_{\text{total}}(\boldsymbol{\theta}^{(t)}).\] where \(\alpha^{(t)}\) denotes the learning rate (step size) at iteration \(t\). The penalty term creates steep gradients that steer the search away from invalid parameter regions defined by the visual priors. This formulation enforces the constraints implicitly, eliminating the need for specialized constrained-optimization solvers.
We scale \(\alpha\) according to the magnitude of the initial data. Specifically, at each iteration, the step size is set such that the update remains roughly two orders of magnitude smaller than the original parameter scale, which is defined by the initial parameter and the range given by VLM. This ensures that each optimization step is neither too aggressive to cause instability nor too small to slow convergence, effectively adapting the learning rate to the scale of the parameters being optimized. Further discussion about the comparison between dynamic and fixed is shown in Sec 6.3.
5 EXPERIMENTAL SETUP↩︎
5.1 Simulation Setup↩︎
Our differentiable physics simulations were performed using the NVIDIA Newton framework [4]. All computations were accelerated on an NVIDIA GeForce RTX 4060 GPU, enabling efficient gradient calculations that are crucial to our optimization pipeline. The runtime per optimization iteration is \(0.1\) seconds.
5.2 Real-World Setup↩︎
All experiments were conducted on a sturdy optical table, providing a stable foundation for all components.
5.2.1 Robotic Manipulator↩︎
We used a uFactory xArm6 robotic arm, securely fixed to the table and equipped with a custom gripper. Gripper position and orientation (\(\boldsymbol{x}, \boldsymbol{q}\)) were obtained from the xArm6’s internal readings.
5.2.2 Force/Torque Sensor↩︎
Force and torque were measured using an ATI Axia80-M20 sensor at the gripper base, with a resolution of 1/80 N (force) and 1/2000 N·m (torque), enabling precise contact characterization.
5.2.3 Object Setup and Data Transformation↩︎
We used 3D-printed objects for parameter identification: unconstrained rigid bodies (cube, sphere, apple) and two-joint rigid bodies (cabinet, drawer, oven). Objects were first fixed to the table for stability, and all collected data—force, torque, position, orientation—were transformed to the world frame.
5.2.4 Vision Sensor↩︎
We use an Azure Kinect camera to capture RGB-D images of the objects, serving as input for VLM estimation and point cloud reconstruction.
| Object | Mass (kg) | \(I_{xx}\) (kg\(\cdot\)m\(^2\)) | \(I_{yy}\) (kg\(\cdot\)m\(^2\)) | \(I_{zz}\) (kg\(\cdot\)m\(^2\)) | ||
|---|---|---|---|---|---|---|
| Cube | Ground Truth | 2.00e-2 | 5.33e-6 | 5.33e-6 | 5.33e-6 | |
| Init(Intern-S1) | 8.00e-2 / 300.00% | 2.13e-5 / 299.81% | 2.13e-5 / 299.81% | 2.13e-5 / 299.81% | ||
| Init(Gemini-3 Pro) | 5.40e-2 / 170.00% | 1.10e-5 / 106.36% | 1.10e-5 / 106.36% | 1.10e-5 / 106.36% | ||
| Least-Square Baseline | 1.72e-2 / 14.00% | 4.05e-6 / 24.02% | 4.31e-6 / 19.14% | 4.09e-6 / 23.26% | ||
| Gradsim Baseline | 1.82e-2 / 9.00% | 4.29e-6 / 19.51% | 4.67e-6 / 12.38% | 4.05e-6 / 24.02% | ||
| RigPI(Ours) | 2.03e-2 / 1.50% | 5.79e-6 / 8.63% | 5.67e-6 / 6.38% | 5.71e-6 / 7.13% | ||
| Sphere | Ground Truth | 2.00e-2 | 5.00e-6 | 5.00e-6 | 5.00e-6 | |
| Init(Intern-S1) | 7.85e-2 / 292.50% | 1.96e-5 / 292.00% | 1.96e-5 / 292.00% | 1.96e-5 / 292.00% | ||
| Init(Gemini-3 Pro) | 6.87e-2 / 243.50% | 1.73e-5 / 246.00% | 1.73e-5 / 246.00% | 1.73e-5 / 246.00% | ||
| Least-Square Baseline | 1.78e-2 / 11.00% | 3.69e-6 / 26.20% | 4.21e-6 / 15.80% | 3.72e-6 / 17.80% | ||
| Gradsim Baseline | 19e-2 / 9.50% | 5.88e-6 / 17.60% | 4.31e-6 / 13.80% | 5.89e-6 / 17.80% | ||
| RigPI(Ours) | 2.07e-2 / 3.50% | 5.37e-6 / 7.40% | 5.21e-6 / 4.20% | 5.41e-6 / 8.20% | ||
| Apple | Ground Truth | 4.16e-2 | 1.22e-5 | 1.22e-5 | 1.12e-5 | |
| Init(Intern-S1) | 1.30e-1 / 212.50% | 6.50e-5 / 432.79% | 6.50e-5 / 432.79% | 6.50e-5 / 480.36% | ||
| Init(Gemini-3 Pro) | 7.00e-2 / 68.27% | 4.33e-5 / 254.92% | 4.33e-5 / 254.92% | 3.66e-5 / 226.79% | ||
| Least-Square Baseline | 5.19e-2 / 24.76% | 1.61e-2 / 31.97% | 1.75e-2 / 43.44% | 1.56e-2 / 39.29% | ||
| Gradsim Baseline | 4.39e-2 / 5.53% | 1.59e-6 / 30.33% | 1.38e-5 / 13.11% | 1.35e-5 / 20.54% | ||
| RigPI(Ours) | 4.23e-2 / 1.68% | 1.39e-5 / 13.93% | 1.30e-5 / 6.58% | 1.27e-5 / 12.29% | ||
| Drawer | Ground Truth | 1.11 | 1.25e-3 | 1.14e-2 | 1.02e-2 | |
| Init(Intern-S1) | 2.95 / 165.77% | 3.56e-2 / 2748.00% | 3.56e-2 / 2121.05% | 6.02e-2 / 490.20% | ||
| Init(Gemini-3 Pro) | 1.60 / 44.14% | 1.20e-2 / 860.00% | 1.80e-2 / 57.89% | 2.00e-2 / 96.08% | ||
| Least-Square Baseline | 3.29 / 196.40% | 1.07e-2 / 756.00% | 2.94e-2 / 157.89% | 2.88e-2 / 182.35% | ||
| Gradsim Baseline | 1.70 / 53.15% | 1.77e-2 / 41.60% | 1.82e-2 / 59.64% | 1.61e-2 / 57.84% | ||
| RigPI(Ours) | 1.19 / 7.21% | 1.39e-3 / 11.20% | 1.45e-2 / 27.19% | 1.22e-2 / 19.61% | ||
| Cabinet Door | Ground Truth | 8.72e-1 | 8.00e-3 | 8.00e-3 | 1.60e-2 | |
| Init(Intern-S1) | 9.80e-1 / 12.39% | 1.00e-2 / 25.00% | 1.31e-2 / 63.75% | 2.30e-2 / 43.75% | ||
| Init(Gemini-3 Pro) | 7.70e-1 / 11.74% | 7.00e-3 / 12.50% | 7.00e-3 / 12.50% | 1.40e-2 / 12.50% | ||
| Least-Square Baseline | 1.49 / 70.87% | 2.61e-2 / 226.25% | 2.51e-2 / 213.75% | 4.57e-2 / 185.63% | ||
| Gradsim Baseline | 1.13 / 29.59% | 1.21e-2 / 51.25% | 1.30e-2 / 62.50% | 2.52e-2 / 57.50% | ||
| RigPI(Ours) | 8.51e-1 / 2.41% | 7.67e-3 / 4.13% | 8.41e-3 / 5.13% | 1.86e-2 / 16.25% | ||
| Oven Door | Ground Truth | 6.42e-2 | 1.12e-4 | 3.62e-4 | 4.74e-4 | |
| Init(Intern-S1) | 1.25e-1 / 94.70% | 3.50e-4 / 212.50% | 4.50e-4 / 24.31% | 1.03e-3 / 117.30% | ||
| Init(Gemini-3 Pro) | 8.00e-2 / 24.61% | 1.40e-4 / 25.00% | 5.00e-4 / 38.12% | 5.20e-4 / 9.70% | ||
| Least-Square Baseline | 8.22e-2 / 28.01% | 2.64e-4 / 135.71% | 3.15e-4 / 12.98% | 5.63e-4 /18.78% | ||
| Gradsim Baseline | 7.21e-1 / 12.31% | 2.44e-4 / 117.86% | 3.08e-4 / 14.92% | 5.14e-4 / 8.44% | ||
| RigPI(Ours) | 6.67e-1 / 3.89% | 1.26e-4 / 12.50% | 3.28e-4 / 9.39% | 4.43e-4 / 6.54% |
5.3 Evaluation Setup↩︎
5.3.1 Accuracy and Stability of the Identification Process↩︎
Accuracy is measured by comparing estimated parameters with ground-truth (GT) values, while stability is quantified via the Coefficient of Variation (CoV) over five independent runs. High accuracy indicates close alignment with GT, and low CoV reflects robust and repeatable estimation.
5.3.2 Predictive Validity with Identified Parameters↩︎
To evaluate the practical effectiveness of the identified parameters \(\boldsymbol{\theta}^*\), we use them in simulation to predict object motion under novel, previously unseen applied wrenches. The predicted trajectories are then compared against real-world measurements. The trajectory reproduction error can assess how well the parameters generalize beyond the identification phase.
| Object | VLM Initializer | Mass CoV (%) | \(\bm{com}\) CoV (%) | Inertia CoV (%) | Friction CoV (%) |
|---|---|---|---|---|---|
| Cube | Intern-S1 | 10.2 | 25.7 | 18.7 | 6.8 |
| Gemini-3 Pro | 7.4 | 17.2 | 19.0 | 6.1 | |
| Sphere | Intern-S1 | 8.4 | 16.8 | 17.5 | 12.0 |
| Gemini-3 Pro | 12.5 | 8.3 | 14.3 | 10.7 | |
| Apple | Intern-S1 | 9.1 | 15.3 | 14.2 | 15.7 |
| Gemini-3 Pro | 7.3 | 11.0 | 11.4 | 7.3 | |
| Drawer | Intern-S1 | 9.4 | 24.9 | 12.0 | 14.2 |
| Gemini-3 Pro | 7.1 | 17.0 | 15.2 | 21.8 | |
| Cabinet Door | Intern-S1 | 6.0 | 16.7 | 22.4 | 20.7 |
| Gemini-3 Pro | 5.6 | 19.9 | 19.2 | 25.4 | |
| Oven Door | Intern-S1 | 8.1 | 13.9 | 17.9 | 23.7 |
| Gemini-3 Pro | 6.3 | 11.6 | 19.2 | 15.7 |
6 RESULTS↩︎
6.1 Accuracy and Stability of the Identification Process↩︎
We evaluate RigPI’s performance by analyzing the accuracy and stability of parameter identification.Table 2 summarizes mass, center of mass (\(\boldsymbol{com}\)), principal moments of inertia (\(I_{xx}, I_{yy}, I_{zz}\)), and selected joint parameters for both unconstrained and multi-link rigid bodies. We compare GT values, initial guesses from two VLMs (Intern-S1 and Gemini-3 Pro), results from the traditional Least-Square baseline [1] and the GradSim baseline [8], and final parameters from RigPI. Our framework consistently outperforms the baseline, showing significant improvements in parameter accuracy. Figure 3 visualizes RigPI’s output, displaying each object alongside predicted values. Some moments of inertia for multi-link rigid bodies are less accurate due to the limited range of joint motion; physically constrained movements make it difficult to excite all directions, challenging precise estimation.
For parameters that are difficult to measure precisely, such as friction coefficients and \(\boldsymbol{com}\) locations, stability is further assessed via CoV across all objects (Table 3). Low CoV values indicate that RigPI achieves robust and reliable convergence.
6.2 Predictive Validity with Identified Parameters↩︎
To demonstrate the impact of identified parameters \(\boldsymbol{\theta}^*\), we simulate each object’s response to novel robot-applied forces and torques. Predicted trajectories are compared with real robot measurements to evaluate trajectory reproduction.
We quantitatively assess the accuracy of the identified parameters by computing the loss on new trajectories using equation 1 . As Table 4 shows, low loss values across all objects indicate accurate prediction of responses under novel dynamics. Values are averaged over five trajectories. Representative predicted trajectories in Fig. 4 illustrate how the identified parameters capture each object’s dynamics.
| Object | Reproduction Loss (\(\mathcal{L}_{\text{pose}}\)) | Endpt Dist.(m) | |
|---|---|---|---|
| Cube | 9.98e-2 | 8.25e-2 | |
| Sphere | 2.93e-1 | 4.53e-2 | |
| Apple | 1.30e-1 | 5.46e-2 | |
| Drawer | 4.41e-1 | 3.79e-2 | |
| Cabinet Door | 3.94e-1 | 5.04e-2 | |
| Oven Door | 4.38e-1 | 4.21e-2 |
6.3 Ablation Study↩︎
6.3.1 Effectiveness of VLM Priors↩︎
RigPI relies on VLMs to provide reasonable initial guesses and a valid search space. We evaluate zero-shot VLM estimation on diverse objects using a single RGB-D image per object, where the VLM predicts geometric and material properties, which are used to derive mass and inertia, and compared with ground-truth.
We compare it with other initialization methods: randomized initialization (“Rand.”), geometry-aware initialization (“Point Cloud”), and the ground truth.
For randomized initialization, we assign mass, center of mass, and inertia values by sampling uniformly within a given range. For instance, mass is sampled between 0 and the maximum payload, the center of mass is sampled within the workspace bounds (e.g., the tabletop), and the inertia tensor is computed based on these sampled values. For geometry-aware initialization, we reconstruct a point cloud from the RGB image and approximate it by a convex hull. Assuming uniform density, mass and inertia are computed analytically. While this geometry-based method can be effective for near-ideal shapes, its accuracy degrades for irregular or concave objects due to reconstruction errors.
As shown in Table ¿tbl:tab:vlm95results95grid?, all the results are initialized by different methods and optimized later. The VLM produces estimates much closer to ground truth, yielding more reliable initial guesses \(\boldsymbol{\theta}^{(0)}\). As for the randomized initialization, “Rand. + Opt.” shows competitive results with “VLM”. However, randomized initialization can easily lead to rapidly increasing and unstable loss, whereas VLM guidance maintains low and consistent loss. The seemingly good results of “Rand. + Opt.” are primarily due to convergence to physically invalid local minima, where numerical values can become extremely large and optimization unreliable. Unlike other initialization methods, which consistently produce reasonable outcomes, random initialization exhibits high variability across trials: some runs converge well, while others diverge dramatically to extreme and physically incorrect values. Such cases are excluded from the results reported in the table.
6.3.2 Effect of Dynamic Learning Rate \(\alpha\)↩︎
Different schemes for setting \(\alpha\) could also influence the optimization. We compare three different \(\alpha\) settings: 0.01, 0.1, and a dynamic setting. A properly chosen \(\alpha\) prevents updates from being too small, causing slow convergence, or too large, risking physically invalid parameters. Figure 5 illustrates loss curves under different \(\alpha\) settings, confirming that dynamic \(\alpha\) primarily regulates step size for safe and efficient optimization rather than tuning final accuracy.
7 CONCLUSION AND FUTURE WORKS↩︎
In this work, we present RigPI, a robust solution to the ill-posed problem of dynamic parameter identification. Our approach anchors differentiable physics simulation with vision-guided semantic priors, transforming the task into a tractable optimization problem. Experiments show that RigPI yields accurate and stable estimates of inertial and frictional properties for diverse unconstrained and multi-link rigid objects. Integrating prior knowledge with physical reasoning offers a promising path toward physically-aware robotic systems capable of complex manipulation. Future work will focus on autonomous exploration for parameter discovery and integrating these models into planning and control frameworks to further enhance robotic capabilities.