1 Introduction↩︎

Surgical robots have been widely adopted in various medical applications over the past decades due to their high precision and intelligence [1]. In robot-assisted laparoscopic surgery, robots can manipulate the laparoscope to reduce surgeon workload and improve procedural stability [2]. Recent advances in visual servoing [3], [4] and learning-based perception [5], [6] further enable automatic field-of-view (FoV) adjustment during surgery. A fundamental requirement in these systems is the enforcement of the remote center of motion (RCM) constraint at the trocar, which restricts the laparoscope to pivot about a fixed incision point, thereby preventing tissue damage [7].

The RCM methodologies can be broadly categorized into design-based and control-based strategies [7]. Design-based approaches rely on specialized mechanical structures to physically enforce the pivot constraint, as exemplified by commercial systems such as the da Vinci platform. Although mechanically precise, these solutions are often complex and costly [8]. In contrast, control-based RCM strategies enforce the constraint at the kinematic level and can be implemented on general-purpose robotic manipulators without dedicated mechanisms. This flexibility has led to a variety of RCM modeling and control formulations in the literature [9]–[11].

Existing control-based RCM formulations can be divided into two main categories. Calculation-based approaches determine the RCM point through geometric relationships, such as tangent-plane constraints [9] or projection onto the laparoscope axis [10]. These models have been combined with control architectures like quadratic programming (QP) schemes [12], [13]. Variable-based approaches, on the other hand, introduce a virtual joint variable to parameterize the RCM position along the shaft [11], which has been adopted in both rigid and flexible laparoscopic systems [14], [15]. Table ¿tbl:tab:literature? summarizes representative FoV-adjustment works using control-based RCM methods.

While prior studies have analyzed the properties of different RCM generation algorithms [17], [18], existing comparisons neglect several issues in practical applications:

1. How different RCM modeling choices behave under identical image-based visual servoing (IBVS) tasks?

2. How controller architecture interacts with RCM constraint formulation?

3. What practical sensitivities arise from seemingly minor modeling decisions?

Moreover, publicly available implementations that allow reproducible and controlled comparison of these methods are still lacking. To address this gap, this paper develops a unified and open-source simulation infrastructure that integrates three representative RCM modeling methods and six IBVS-compatible control frameworks within a consistent velocity-level formulation. Rather than proposing a new RCM method, the objective of this paper is to provide a structured benchmark that enables systematic evaluation of modeling and controller interactions under identical conditions. Through a series of carefully designed simulation cases, the framework exposes practical sensitivities related to tangent-plane definition, constraint dimensionality, open- versus closed-loop enforcement, and robustness near kinematic singularities. The contributions of this paper are summarized as follows:

1. A unified velocity-level formulation that expresses three representative RCM models within a common constraint structure.

2. A modular benchmark integrating three RCM models and six control frameworks for controlled and reproducible comparison.

3. A systematic study exposing structural sensitivities between modeling choices and controller architectures, including constraint dimensionality and singularity robustness.

4. An open-source playground providing a reproducible baseline for RCM-constrained visual servoing research.

The remainder of this paper is organized as follows: Section II introduces the modeling of the RCM constraint. Section III presents the visual servoing task. Section IV describes the different control frameworks. Section V shows the simulation results, and Section VI makes further discussions. Finally, Section VII concludes the paper.

2 Modeling of the RCM constraint↩︎

This section introduces three representative RCM modeling methods. Fig. 1 illustrates the coordinate frames involved. Frame \(\{s\}\) denotes the world frame in simulation or the robot base frame in practice. The camera frame \(\{c\}\) is located at the laparoscope tip, with its \(z\)-axis aligned with the optical axis and its \(x\)-axis pointing to the right of the image. Frames \(\{e\}\), \(\{b\}\), and \(\{rcm\}\) are defined at the distal end of the laparoscope shaft, the connector, and the RCM point, respectively. These frames share the same initial orientation as \(\{c\}\) and remain aligned during motion. For rigid laparoscopes, \(\{c\}\) and \(\{e\}\) coincide, whereas for flexible systems a bending section exists between them. Figs. 1 (b)-(d) depict the three RCM modeling approaches. In all cases, the RCM position along the shaft is parameterized by a scalar \(\lambda \in (0,1)\), defined as the ratio \(L_{out}/L_{rod}\), where \(L_{out}\) is the distance from \(\{b\}\) to \(\{rcm\}\) and \(L_{rod}\) is the shaft length from \(\{b\}\) to \(\{e\}\). The scalar is denoted as \(\lambda_i\) for the \(i\)-th method.

2.0.1 RCM 1 [9]↩︎

As shown in Fig. 1 (b), this method utilizes a tangent plane at the trocar point to calculate the scalar \(\lambda_{1}\): \[\lambda_{1} = \frac{\hat{\boldsymbol{n}}_{z}^{T}({}^{s}\boldsymbol{p}_{b}-{}^{s}\boldsymbol{p}_{tro})}{\hat{\boldsymbol{n}}_{z}^{T}({}^{s}\boldsymbol{p}_{b}-{}^{s}\boldsymbol{p}_{e})}\] where \({}^{s}\boldsymbol{p}_{b} \in \mathbb{R}^{3}\) and \({}^{s}\boldsymbol{p}_{e}\in \mathbb{R}^{3}\) are the positions of frame \(\{b\}\) and frame \(\{e\}\) in frame \(\{s\}\), respectively, \({}^{s}\boldsymbol{p}_{tro}\in \mathbb{R}^{3}\) is the initial position of the trocar point in frame \(\{s\}\), and \(\hat{\boldsymbol{n}}_{z}\in \mathbb{R}^{3}\) is the unit vector normal to the tangent plane of the trocar point: \[\hat{\boldsymbol{n}}_{z}^{T}\boldsymbol{p}+d=0 \label{eq:trocar95plane}\tag{1}\] where \(\boldsymbol{p}\in\mathbb{R}^{3}\) is the position of an arbitrary point on the tangent plane, and \(d\) is a constant, which is calculated by substituting \(\boldsymbol{p}={}^{s}\boldsymbol{p}_{tro}\) into Eq. (1 ). Furthermore, the position \({}^{s}\boldsymbol{p}_{rcm}\in \mathbb{R}^{3}\) and linear velocity \({}^{s}\boldsymbol{v}_{rcm}\in \mathbb{R}^{3}\) of the RCM point in frame \(\{s\}\) is: \[\begin{align} {}^{s}\boldsymbol{p}_{rcm}&=\lambda_{1}{}^{s}\boldsymbol{p}_{e}+(1-\lambda_{1}){}^{s}\boldsymbol{p}_{b} \\ {}^{s}\boldsymbol{v}_{rcm}&=\lambda_{1}{}^{s}\boldsymbol{v}_{e}+(1-\lambda_{1}){}^{s}\boldsymbol{v}_{b} \label{eq:rcm195velocity} \end{align}\tag{2}\] where \({}^{s}\boldsymbol{v}_{e}\in \mathbb{R}^{3}\) and \({}^{s}\boldsymbol{v}_{b}\in \mathbb{R}^{3}\) are the linear velocities of the distal end of the laparoscope shaft and the connector in frame \(\{s\}\), respectively. To ensure the RCM point does not have any lateral deflection in the tangent plane, the following constraint should be satisfied: \[\begin{bmatrix} \hat{\boldsymbol{n}}_{x}^{T}; \hat{\boldsymbol{n}}_{y}^{T} \end{bmatrix} {}^{s}\boldsymbol{v}_{rcm} = \boldsymbol{0}_{2\times1} \label{eq:rcm195constraint}\tag{3}\] where \(\hat{\boldsymbol{n}}_{x}\in \mathbb{R}^{3}\) and \(\hat{\boldsymbol{n}}_{y}\in \mathbb{R}^{3}\) are unit vectors spanning the tangent plane defined in Eq. (1 ). Substitute Eq. (2 ) to Eq. (3 ): \[\boldsymbol{J}_{rcm,1}\dot{\boldsymbol{\Theta}} = \boldsymbol{0}_{2\times1}\] \[\boldsymbol{J}_{rcm,1} =\begin{bmatrix} \hat{\boldsymbol{n}}_{x}^{T}; \hat{\boldsymbol{n}}_{y}^{T} \end{bmatrix} (\lambda_{1}{}^{s}\boldsymbol{J}_{e}+(1-\lambda_{1}){}^{s}\boldsymbol{J}_{b})\] where \(\dot{\boldsymbol{\Theta}}\in \mathbb{R}^{N_{\Theta}}\) is the joint velocity and \(N_{\Theta}\) is the number of joints of the laparoscopic robot, and \({}^{s}\boldsymbol{J}_{e} \in \mathbb{R}^{3 \times N_{\Theta}}\) and \({}^{s}\boldsymbol{J}_{b} \in \mathbb{R}^{3 \times N_{\Theta}}\) are the Jacobian matrices mapping \(\dot{\boldsymbol{\Theta}}\) to \({}^{s}\boldsymbol{v}_{e}\) and \({}^{s}\boldsymbol{v}_{b}\), respectively.

2.0.2 RCM 2 [10]↩︎

As shown in Fig. 1 (c), this method projects the trocar point onto the axis of the laparoscope shaft to obtain \(\lambda_{2}\): \[\lambda_{2} = {}^{s}\hat{\boldsymbol{z}}_{b}^{T}({}^{s}\boldsymbol{p}_{tro}-{}^{s}\boldsymbol{p}_{b})/L_{rod}\] where \({}^{s}\hat{\boldsymbol{z}}_{b}={}^{s}\boldsymbol{R}_{b}[0;0;1]\) is the unit vector along the \(z\)-axis of frame \(\{b\}\) in frame \(\{s\}\), which coincides with the axis of the laparoscope shaft, and \({}^{s}\boldsymbol{R}_{b} \in SO(3)\) is the rotation matrix of frame \(\{b\}\) in frame \(\{s\}\). Based on this, the position and linear velocity of the RCM point are: \[\begin{align} {}^{s}\boldsymbol{p}_{rcm} &={}^{s}\boldsymbol{p}_{b}+\lambda_{2}L_{rod}{}^{s}\hat{\boldsymbol{z}}_{b} \tag{4} \\ {}^{s}\boldsymbol{v}_{rcm} &={}^{s}\boldsymbol{v}_{b}+{}^{s}\boldsymbol{\omega}_{b} \times (\lambda_{2}L_{rod}{}^{s}\hat{\boldsymbol{z}}_{b}) \tag{5} \end{align}\] where \({}^{s}\boldsymbol{\omega}_{b}\in\mathbb{R}^{3}\) is the angular velocity of the connector in frame \(\{s\}\). Additionally, the position of the RCM point can also be extracted via \({}^{s}\boldsymbol{p}_{rcm} = {}^{s}\boldsymbol{T}_{rcm}(1:3,4)\), where \({}^{s}\boldsymbol{T}_{rcm} \in SE(3)\) is the homogeneous transformation matrix of frame \(\{rcm\}\) in frame \(\{s\}\), which is calculated by: \[{}^{s}\boldsymbol{T}_{rcm} = {}^{s}\boldsymbol{T}_{b} \cdot \text{transl\_z}(\lambda_{2}L_{rod})\] where \({}^{s}\boldsymbol{T}_{b} \in SE(3)\) is the homogeneous transformation matrix of frame \(\{b\}\) in frame \(\{s\}\), and \(\text{transl\_z}(\cdot)\) is a function that generates a homogeneous transformation matrix representing a translation along the \(z\)-axis. Furthermore, the velocity twist \({}^{rcm}\boldsymbol{\mathcal{V}}_{rcm}=[{}^{rcm}\boldsymbol{\omega}_{rcm};{}^{rcm}\boldsymbol{v}_{rcm}]\) of the RCM point can be obtained from that of the connector \({}^{b}\boldsymbol{\mathcal{V}}_{b}=[{}^{b}\boldsymbol{\omega}_{b};{}^{b}\boldsymbol{v}_{b}]\) through the adjoint representation: \[\begin{bmatrix} {}^{rcm}\boldsymbol{\omega}_{rcm} \\ {}^{rcm}\boldsymbol{v}_{rcm} \end{bmatrix} = \begin{bmatrix} \boldsymbol{I}_{3\times3} & \boldsymbol{0}_{3\times3} \\ [{}^{rcm}\boldsymbol{p}_{b}]_{\times} & \boldsymbol{I}_{3\times3} \end{bmatrix} \begin{bmatrix} {}^{b}\boldsymbol{\omega}_{b} \\ {}^{b}\boldsymbol{v}_{b} \end{bmatrix} \label{eq:rcm295adj}\tag{6}\] where \({}^{rcm}\boldsymbol{\omega}_{rcm}\in\mathbb{R}^{3}\) and \({}^{rcm}\boldsymbol{v}_{rcm}\in\mathbb{R}^{3}\) (\({}^{b}\boldsymbol{\omega}_{b}\in\mathbb{R}^{3}\) and \({}^{b}\boldsymbol{v}_{b}\in\mathbb{R}^{3}\)) are the angular and linear velocities of the RCM point (the connector) in frame \(\{rcm\}\) (frame \(\{b\}\)), respectively, \(\boldsymbol{I}_{3\times3}\) and \(\boldsymbol{0}_{3\times3}\) denote the \(3 \times 3\) identity and zero matrices, respectively, \({}^{rcm}\boldsymbol{p}_{b} = [0;0;-L_{out}]\) is the position of frame \(\{b\}\) in frame \(\{rcm\}\), and \([\cdot]_{\times}\) is the skew-symmetric matrix operator.

To satisfy the RCM constraint, the linear velocities of the RCM point at \(x\) and \(y\) directions in frame \(\{rcm\}\) should be controlled: \[\boldsymbol{J}_{rcm,2}(1:2,:)\dot{\boldsymbol{\Theta}} = {}^{rcm}\boldsymbol{v}_{rcm,des}\] \[\boldsymbol{J}_{rcm,2} = \begin{bmatrix} [{}^{rcm}\boldsymbol{p}_{b}]_{\times} & \boldsymbol{I}_{3\times3} \end{bmatrix} {}^{b}\boldsymbol{J}_{b}\] where \({}^{b}\boldsymbol{J}_{b} \in \mathbb{R}^{6 \times N_{\Theta}}\) is the Jacobian matrix mapping \(\boldsymbol{\dot{\Theta}}\) to \({}^{b}\boldsymbol{\mathcal{V}}_{b}\), and \({}^{rcm}\boldsymbol{v}_{rcm,des}\in \mathbb{R}^{2}\) contains the desired linear velocities of the RCM point at \(x\) and \(y\) directions in frame \(\{rcm\}\), which is a zero vector for open-loop control of the position of the RCM point, or a vector calculated based on the RCM error, defined as \({}^{s}\boldsymbol{p}_{tro} - {}^{s}\boldsymbol{p}_{rcm}\), for closed-loop control.

2.0.3 RCM 3 [11]↩︎

As shown in Fig. 1 (d), this method introduces \(\lambda_{3}\) as a virtual joint variable to derive the position and the linear velocity of the RCM point in frame \(\{s\}\): \[{}^{s}\boldsymbol{p}_{rcm} = {}^{s}\boldsymbol{p}_{b}+\lambda_{3}({}^{s}\boldsymbol{p}_{e}-{}^{s}\boldsymbol{p}_{b})\] \[{}^{s}\boldsymbol{v}_{rcm}={}^{s}\boldsymbol{v}_{b} +\dot{\lambda}_{3}({}^{s}\boldsymbol{p}_{e}-{}^{s}\boldsymbol{p}_{b}) +\lambda_{3}({}^{s}\boldsymbol{v}_{e}-{}^{s}\boldsymbol{v}_{b})\] where \(\dot{\lambda}_{3}\) is the velocity of the virtual joint variable \(\lambda_{3}\). It is worth noting that there is another equivalent way [19] to express the linear velocity of the RCM point: \[{}^{s}\boldsymbol{v}_{rcm} = {}^{s}\boldsymbol{v}_{b} + {}^{s}\boldsymbol{v}_{rcm}^{b} - [{}^{s}\boldsymbol{p}_{rcm}^{b}]_{\times} {}^{s}\boldsymbol{\omega}_{b}\] where \({}^{s}\boldsymbol{v}_{rcm}^{b}\) is the linear velocity of the RCM point related to frame \(\{b\}\) in frame \(\{s\}\), and \({}^{s}\boldsymbol{p}_{rcm}^{b} = {}^{s}\boldsymbol{p}_{rcm} - {}^{s}\boldsymbol{p}_{b}\). In this method, the RCM constraint equation is expressed as: \[\boldsymbol{J}_{rcm,3} \begin{bmatrix} \dot{\boldsymbol{\Theta}}; \dot{\lambda}_{3} \end{bmatrix} = {}^{s}\boldsymbol{v}_{rcm,des} \label{eq:rcm395constraint}\tag{7}\] \[\boldsymbol{J}_{rcm,3} = \begin{bmatrix} {}^{s}\boldsymbol{J}_{b}+\lambda_{3}({}^{s}\boldsymbol{J}_{e}-{}^{s}\boldsymbol{J}_{b}) & {}^{s}\boldsymbol{p}_{e}-{}^{s}\boldsymbol{p}_{b} \end{bmatrix}\] where \({}^{s}\boldsymbol{v}_{rcm,des}\in \mathbb{R}^{3}\) is the desired linear velocity of the RCM point in frame \(\{s\}\), which holds the same assignment logic as \({}^{rcm}\boldsymbol{v}_{rcm,des}\) in RCM 2. Eq. (7 ) can be changed to express under frame \(\{rcm\}\) by left multiplying the rotation matrix \({}^{rcm}\boldsymbol{R}_{s}\in SO(3)\), which describes the rotation of frame \(\{s\}\) in frame \(\{rcm\}\), on both sides.

2.0.4 Unified Expression↩︎

In summary, the RCM constraint can be uniformly expressed as: \[\boldsymbol{J}_{rcm,i}\boldsymbol{\dot{u}} = \boldsymbol{v}_{rcm,des} \label{eq:rcm95constraint}\tag{8}\] where \(\boldsymbol{u} = \boldsymbol{\Theta}\) and \(\boldsymbol{\dot{u}} = \dot{\boldsymbol{\Theta}}\) for RCM 1 and RCM 2, whereas for RCM 3, \(\boldsymbol{u} = [\boldsymbol{\Theta}; \lambda_{3}]\) and \(\boldsymbol{\dot{u}} = [\dot{\boldsymbol{\Theta}}; \dot{\lambda}_{3}]\), \(\boldsymbol{v}_{rcm,des} \in \mathbb{R}^{N_{rcm}}\) is the desired velocity of the RCM point, and \(N_{rcm}\) is the number of RCM constraint equations.

3 Visual servoing task↩︎

This section formulates the visual servoing task for automatic FoV adjustment of laparoscopic robots. Generally, the relationship between visual feature velocity and robot joint velocity is given by [20]: \[\dot{\boldsymbol{s}} = \boldsymbol{J}_{t}\dot{\boldsymbol{\Theta}}\] where \(\boldsymbol{s} \in \mathbb{R}^{N_{f}}\) is the visual feature, \(\dot{\boldsymbol{s}}\) is the velocity of the visual feature, \(N_{f}\) is the number of visual features, and \(\boldsymbol{J}_{t}\in \mathbb{R}^{N_{f} \times N_{\Theta}}\) is the Jacobian matrix of the whole laparoscopic robot system.

The objective of visual servoing is to drive \(\dot{\boldsymbol{s}}\) to a desired velocity \(\dot{\boldsymbol{s}}_{des}\), computed from the feature tracking error. Accordingly, the task is written as: \[\boldsymbol{J}_{t,i}\boldsymbol{\dot{u}} = \dot{\boldsymbol{s}}_{des} \label{eq:visual95servoing}\tag{9}\] where \(\boldsymbol{J}_{t,i} = \boldsymbol{J}_{t}\) for \(i=1,2\) and \(\boldsymbol{J}_{t,3} = [\boldsymbol{J}_{t}, \boldsymbol{0}_{N_{f}\times1}]\).

4 Control frameworks↩︎

This section introduces six different control frameworks for achieving the visual servoing task under the RCM constraint. Specifically, two pseudoinverse (PI)-based control frameworks, three QP-based methods, and one inverse-kinematics (IK)-based approach are introduced.

4.1 PI-based Method↩︎

PI-based methods calculate the joint velocity by directly solving Eq. (8 ) and Eq. (9 ), which represent the RCM constraint and visual servoing task, respectively.

4.1.1 PI 1↩︎

This PI-based method treats both RCM constraint and visual servoing task equally. Combine Eq. (8 ) and Eq. (9 ) to obtain the control value [11]: \[\boldsymbol{\dot{u}} = \begin{bmatrix} \boldsymbol{J}_{t,i}; \boldsymbol{J}_{rcm,i} \end{bmatrix}^{\dagger} \begin{bmatrix} \dot{\boldsymbol{s}}_{des}; \boldsymbol{v}_{rcm,des} \end{bmatrix}\] where [\(\cdot\)]\(^{\dagger}\) is the pseudoinverse operator.

4.1.2 PI 2↩︎

In this method, the null-space projection is introduced to prioritize the RCM constraint over the visual servoing task, which is similar to [9]: \[\boldsymbol{\dot{u}} = \boldsymbol{\dot{u}}_{rcm} + \boldsymbol{N}_{rcm,i}\boldsymbol{\dot{u}}_{t}\] \[\boldsymbol{\dot{u}}_{rcm} = \boldsymbol{J}_{rcm,i}^{\dagger}\boldsymbol{v}_{rcm,des}\] \[\boldsymbol{\dot{u}}_{t} = (\boldsymbol{J}_{t,i}\boldsymbol{N}_{rcm,i})^{\dagger}(\dot{\boldsymbol{s}}_{des}-\boldsymbol{J}_{t,i}\boldsymbol{\dot{u}}_{rcm})\] where \(\boldsymbol{N}_{rcm,i} = \boldsymbol{I}-\boldsymbol{J}_{rcm,i}^{\dagger}\boldsymbol{J}_{rcm,i}\) is the null space projector of \(\boldsymbol{J}_{rcm,i}\), and \(\boldsymbol{I}\) is the identity matrix of appropriate size.

4.2 QP-based Method↩︎

In practical applications, laparoscopic robots are subject to physical constraints such as joint angle and velocity limits, which are difficult to incorporate directly into PI-based methods. QP-based frameworks address this limitation by enforcing the RCM constraint and visual servoing task while explicitly incorporating these physical bounds as inequality constraints:

\[[\boldsymbol{u}_{min}; \boldsymbol{\dot{u}}_{min}] \leq [\boldsymbol{u}; \boldsymbol{\dot{u}}] \leq [\boldsymbol{u}_{max}; \boldsymbol{\dot{u}}_{max}]\] where \(\boldsymbol{u}_{min}\) and \(\boldsymbol{u}_{max}\) (\(\boldsymbol{\dot{u}}_{min}\) and \(\boldsymbol{\dot{u}}_{max}\)) are the lower and upper limits of \(\boldsymbol{u}\) (\(\boldsymbol{\dot{u}}\)), respectively.

4.2.1 QP 1↩︎

Both RCM constraint and visual servoing task are modeled as equality constraints in this QP-based method [14]: \[\begin{align} \text{min.} \quad & \boldsymbol{\dot{u}}^{T}\boldsymbol{W}_{u}\boldsymbol{\dot{u}} \\ \text{s.t.} \quad & \boldsymbol{J}_{rcm,i}\boldsymbol{\dot{u}}= \boldsymbol{v}_{rcm,des} \\ & \boldsymbol{J}_{t,i}\boldsymbol{\dot{u}} = \dot{\boldsymbol{s}}_{des} \\ & [\boldsymbol{u}_{min}; \boldsymbol{\dot{u}}_{min}] \leq [\boldsymbol{u}; \boldsymbol{\dot{u}}] \leq [\boldsymbol{u}_{max}; \boldsymbol{\dot{u}}_{max}] \end{align}\] where \(\boldsymbol{W}_{u} \in \mathbb{R}^{N_{u} \times N_{u}}\) is a positive definite weight matrix, and \(N_{u}\) is the dimension of \(\boldsymbol{\dot{u}}\).

4.2.2 QP 2↩︎

In this method, the RCM constraint is modeled as an equality constraint, while visual servoing task is considered in the cost function [12]: \[\begin{align} \text{min.} \quad & \boldsymbol{c}_{s}^{T}\boldsymbol{W}_{s}\boldsymbol{c}_{s} + \boldsymbol{\dot{u}}^{T}\boldsymbol{W}_{u}\boldsymbol{\dot{u}} \\ \text{s.t.} \quad & \boldsymbol{J}_{rcm,i}\boldsymbol{\dot{u}}= \boldsymbol{v}_{rcm,des} \\ & [\boldsymbol{u}_{min}; \boldsymbol{\dot{u}}_{min}] \leq [\boldsymbol{u}; \boldsymbol{\dot{u}}] \leq [\boldsymbol{u}_{max}; \boldsymbol{\dot{u}}_{max}] \end{align}\] where \(\boldsymbol{c}_{s} = \boldsymbol{J}_{t,i}\boldsymbol{\dot{u}} - \dot{\boldsymbol{s}}_{des}\) and \(\boldsymbol{W}_{s} \in \mathbb{R}^{N_{f} \times N_{f}}\) is a positive definite weight matrix.

4.2.3 QP 3↩︎

Hierarchical QP with RCM constraint in the first priority and visual servoing task in the second priority are constructed in this QP-based method [13]. The first QP problem is designed to satisfies the RCM constraint: \[\begin{align} \text{min.} \quad & \boldsymbol{c}_{rcm}^{T}\boldsymbol{W}_{rcm}\boldsymbol{c}_{rcm} + \boldsymbol{\dot{u}}^{T}\boldsymbol{W}_{u}\boldsymbol{\dot{u}} \\ \text{s.t.} \quad & [\boldsymbol{u}_{min}; \boldsymbol{\dot{u}}_{min}] \leq [\boldsymbol{u}; \boldsymbol{\dot{u}}] \leq [\boldsymbol{u}_{max}; \boldsymbol{\dot{u}}_{max}] \end{align}\] where \(\boldsymbol{c}_{rcm} = \boldsymbol{J}_{rcm,i}\boldsymbol{\dot{u}} - \boldsymbol{v}_{rcm,des}\) and \(\boldsymbol{W}_{rcm} \in \mathbb{R}^{N_{rcm} \times N_{rcm}}\) is a positive definite weight matrix. By solving the first QP problem, the control value \(\boldsymbol{\dot{u}}_{rcm}\) is obtained. Furthermore, the second QP problem is designed to achieve the visual servoing task in the null space of the RCM constraint: \[\begin{align} \text{min.} \quad & \boldsymbol{c}_{s}^{T}\boldsymbol{W}_{s}\boldsymbol{c}_{s} + \boldsymbol{\dot{u}}^{T}\boldsymbol{W}_{u}\boldsymbol{\dot{u}} \\ \text{s.t.} \quad & \boldsymbol{\dot{u}} = \boldsymbol{\dot{u}}_{rcm} + \boldsymbol{N}_{rcm,i}\boldsymbol{\dot{u}}_{t} \\ & [\boldsymbol{u}_{min}; \boldsymbol{\dot{u}}_{min}] \leq [\boldsymbol{u}; \boldsymbol{\dot{u}}] \leq [\boldsymbol{u}_{max}; \boldsymbol{\dot{u}}_{max}] \end{align}\]

4.3 IK-based Method↩︎

The IK-based method applies to rigid laparoscopic robots under the assumption that the RCM constraint is perfectly satisfied, allowing a direct relationship between the velocity twists of the connector and the end camera [4]. It first computes the desired end-camera twist from the visual servoing task, then determines the corresponding connector twist under the RCM constraint, and finally derives the control input via inverse kinematics.

For a rigid laparoscopic robot, the relationship between the connector and end-camera velocity twists is given by: \[\begin{bmatrix} {}^{b}\boldsymbol{\omega}_{b} \\ {}^{b}\boldsymbol{v}_{b} \end{bmatrix} = \begin{bmatrix} \boldsymbol{I}_{3\times3} & \boldsymbol{0}_{3\times3} \\ [{}^{b}\boldsymbol{p}_{c}]_{\times} & \boldsymbol{I}_{3\times3} \end{bmatrix} \begin{bmatrix} {}^{c}\boldsymbol{\omega}_{c} \\ {}^{c}\boldsymbol{v}_{c} \end{bmatrix} \label{eq:vc2vb}\tag{10}\] where \({}^{b}\boldsymbol{p}_{c}=[0;0;L_{rod}]\) is the position of frame \(\{c\}\) with respect to frame \(\{b\}\), and \({}^{c}\boldsymbol{\omega}_{c}\in\mathbb{R}^{3}\) and \({}^{c}\boldsymbol{v}_{c}\in\mathbb{R}^{3}\) are the angular and linear velocities of the laparoscope’s end camera in frame \(\{c\}\), respectively. Assuming the RCM constraint is perfectly satisfied, combining Eq. (6 ) and Eq. (10 ) yields: \[\begin{align} {}^{rcm}v_{rcm,x} & = (L_{out}-L_{rod}){}^{c}\omega_{c,y} + {}^{c}v_{c,x} = 0 \\ {}^{rcm}v_{rcm,y} & = (L_{rod}-L_{out}){}^{c}\omega_{c,x} + {}^{c}v_{c,y} = 0 \end{align}\] which further lead to: \[\begin{align} {}^{c}\omega_{c,x} & = -{}^{c}v_{c,y}/(L_{rod}-L_{out}) \\ {}^{c}\omega_{c,y} & = -{}^{c}v_{c,x}/(L_{out}-L_{rod}) \end{align}\] where \(\cdot{}_{,x}\) and \(\cdot{}_{,y}\) denote the \(x\) and \(y\) components of the corresponding vector. Substituting these expressions into Eq. (10 ) gives the form commonly used in [4]: \[\begin{bmatrix} {}^{b}\boldsymbol{\omega}_{b} \\ {}^{b}\boldsymbol{v}_{b} \end{bmatrix} = \begin{bmatrix} 0 & 0 & 0 & 0 & -\frac{1}{L_{in}} & 0 \\ 0 & 0 & 0 & \frac{1}{L_{in}} & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & -\frac{L_{out}}{L_{in}} & 0 & 0 \\ 0 & 0 & 0 & 0 & -\frac{L_{out}}{L_{in}} & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} {}^{c}\boldsymbol{\omega}_{c} \\ {}^{c}\boldsymbol{v}_{c} \end{bmatrix}\] where \(L_{in} = L_{rod} - L_{out}\) denotes the shaft length inside the body. The control input is then obtained via the robot arm inverse kinematics.

5 Simulation Studies↩︎

This section presents simulation studies to evaluate the proposed benchmark across different RCM modeling methods and control frameworks, and to analyze the structural insights revealed under controlled conditions. The primary results are obtained using a rigid laparoscopic robot, while additional experiments on both rigid and flexible systems are provided in the supplementary video.

5.1 Simulation Setup↩︎

A simulation platform is developed in CoppeliaSim to evaluate FoV adjustment, as shown in Fig. 2. The system consists of a 6-DoF UR5 manipulator holding a rigid laparoscope. A red sphere and a green rectangle represent the trocar and the visual tracking target (e.g., a surgical tool), respectively. A dummy attached to the shaft visualizes the RCM point. The coordinate frames follow Section II, with frame \(\{s\}\) coincident with the CoppeliaSim world frame. The image frame \(\{i\}\) is defined at the top-left corner of the camera view, with the \(u\)-axis pointing right and the \(v\)-axis downward. The image resolution is \(256 \times 256\) pixels. Controllers are implemented in MATLAB and communicate with CoppeliaSim via the remote API. Variables are discretized with time step \(\Delta t = 0.1\), and joint velocity is computed as \(\dot{\boldsymbol{\Theta}}^{k} = (\boldsymbol{\Theta}^{k}-\boldsymbol{\Theta}^{k-1})/\Delta t\), where \(k\) denotes the time index.

5.2 4-DoF Visual Servoing↩︎

Following Huang et al. [20], the visual feature vector is defined by the position, scale, and orientation of the target rectangle in frame \(\{i\}\): \[\boldsymbol{s} =[s_1; s_2; s_3; s_4]=[u_c; v_c; Z^{*} \sqrt{a^{*}/a}; \alpha]\] where \([u_c; v_c]\) is the mass center of the target rectangle in the image, \(a\) and \(a^{*}\) are the current and desired area of the target rectangle in the image, and \(Z^{*}\) is the desired depth in frame \(\{c\}\). When the area approaches the expected value, \(s_3\) approximates \(Z^{*}\). \(\alpha = atan2(2\mu_{11}/(\mu_{20}-\mu_{02}))\cdot 90/\pi\) indicates the rotation angle between the long axis of the rectangle and the \(u\)-axis of frame \(\{i\}\), where \(\mu_{11}\), \(\mu_{20}\), and \(\mu_{02}\) are second-order centered moments computed by \(\mu_{ij} = \sum_{n=1}^{4} (u_n - u_c)^i (v_n - v_c)^j\), with \([u_n; v_n]\) denoting the \(n\)-th vertex in frame \(\{i\}\). In the following simulation cases, the desired visual feature is \([128; 128; 100; 70]\).

5.2.1 Case 1: Tangent Plane Selection in RCM 1↩︎

Since the original work [9] does not specify how the tangent plane should be defined, this case evaluates how different tangent-plane selections affect the performance of RCM 1. Two representative choices are compared: planes perpendicular to the \(z\)-axis of frame \(\{s\}\) and frame \(\{rcm\}\), respectively. The former constrains the RCM point within the \(x\)–\(y\) plane of \(\{s\}\), corresponding to the implicit setting in [9], while the latter aligns with the physical interpretation of the trocar constraint and becomes equivalent to RCM 2 with \(\boldsymbol{v}_{rcm,des}=\boldsymbol{0}\). QP 1 is used as the control framework.

In this case, the target moves along a straight line: \[{}^{s}\boldsymbol{p}_{tar}^{k} = {}^{s}\boldsymbol{p}_{tar}^{1} + ({}^{s}\boldsymbol{p}_{tar}^{K}-{}^{s}\boldsymbol{p}_{tar}^{1}) \times t/T \label{eq:line}\tag{11}\] where \({}^{s}\boldsymbol{p}_{tar}^{k}\in \mathbb{R}^3\) is the position of the target at time step \(k\) in frame \(\{s\}\), \({}^{s}\boldsymbol{p}_{tar}^{1} = [-50;525;5]\) mm and \({}^{s}\boldsymbol{p}_{tar}^{K} = [50;525;5]\) mm are the initial position and final position of the target in frame \(\{s\}\), respectively, \(K=T/\Delta t+1\) is the total time step number, \(T=10\) s is the total simulation time, and \(t=(k-1)\Delta t\) is the current time. Fig. 3 (a) shows the initial snapshot of this case, where the initial visual feature is \([178.09;177.69;139.25;64.98]\).

The results are shown in Fig. 4. In both cases, the visual servoing task converges successfully (Fig. 4 (b)). However, when the tangent plane is perpendicular to the \(z\)-axis of \(\{s\}\) (red solid), significant RCM errors accumulate in the \(x\) and \(y\) directions (Fig. 4 (a)), indicating poor constraint enforcement. The deviation is visible in Fig. 3 (b). This occurs because scale adjustment requires insertion of the laparoscope, which introduces lateral motion of the RCM point in frame \(\{s\}\) under this plane definition. In contrast, defining the tangent plane perpendicular to the \(z\)-axis of \(\{rcm\}\) (blue dash) maintains small RCM errors throughout the motion, with maximum errors of \(0.63\) mm and \(-0.43\) mm in the \(x\) and \(y\) directions, respectively.

5.2.2 Case 2: Effects of \(N_{rcm}\) for RCM 3↩︎

This case examines the effect of constraint dimensionality in RCM 3 by comparing \(N_{rcm}=2\) (no constraint along the \(z\) direction) and \(N_{rcm}=3\). The same straight-line trajectory in Eq. (11 ) is used, with QP 1 as the control framework.

As shown in Fig. 5 (b), the visual servoing task converges in both cases. However, when \(N_{rcm}=2\) (red solid), a large RCM error appears in the \(z\) direction, while the \(x\) and \(y\) errors remain small (Fig. 5 (a)). In contrast, enforcing \(N_{rcm}=3\) (blue dash) maintains small errors in all directions. The deviation is evident in Fig. 3 (c), where the RCM point shifts along the shaft when \(N_{rcm}=2\), producing a \(z\)-direction error exceeding \(20\) mm. This behavior arises because the virtual variable \(\lambda_{3}\) is unconstrained along the axial direction, leading to drift of the computed RCM location. These results demonstrate that enforcing all three constraint components (\(N_{rcm}=3\)) is necessary to ensure correct RCM localization in RCM 3.

5.2.3 Case 3: Comparison of Different RCM Methods with Open-Loop and Closed-Loop Formulations↩︎

This case compares open-loop and closed-loop RCM enforcement strategies. Specifically, RCM 1, RCM 2 with \(\boldsymbol{v}_{rcm,des}=\boldsymbol{0}\), and RCM 3 with \(\boldsymbol{v}_{rcm,des}=\boldsymbol{0}\) (denoted as RCM 2 (0) and RCM 3 (0)) operate in open loop. In contrast, RCM 2 and RCM 3 with \(\boldsymbol{v}_{rcm,des}\) computed from the RCM position error operate in closed loop. All methods are evaluated under the straight-line trajectory in Eq. (11 ), with QP 1 as the control framework.

The results are shown in Fig. 6. The open-loop methods (RCM 1 (red solid), RCM 2 (0) (blue dash), and RCM 3 (0) (purple solid)) exhibit non-zero steady-state RCM errors after the visual servoing task converges. As seen in the \(e_x\) and \(e_y\) plots, residual lateral errors persist, while the \(e_z\) plot reveals axial drift, particularly in RCM 3 (0). This indicates that purely geometric enforcement without error feedback cannot eliminate accumulated deviation. In contrast, the closed-loop methods (RCM 2 (green solid) and RCM 3 (yellow dash)) drive the RCM errors toward zero in all directions, maintaining sub-millimeter accuracy throughout the motion. These results highlight the necessity of explicit RCM error feedback to prevent steady-state and axial deviations.

5.2.4 Case 4: Comparison of Different Control Frameworks↩︎

This case compares the performance of different control frameworks including IK, PI 1, PI 2, QP 1, QP 2, and QP 3. IK operates in open loop by enforcing \({}^{rcm}v_{rcm,x}=0\) and \({}^{rcm}v_{rcm,y}=0\), while the remaining five methods use closed-loop RCM 2. Two scenarios are considered.

In the first scenario, the target follows the straight-line trajectory in Eq. (11 ). As shown in Fig. 7, all methods maintain small RCM errors. However, IK (purple solid) cannot eliminate steady-state error due to its open-loop formulation. The rest closed-loop methods limit the peak RCM error to approximately \(0.5\) mm and converge rapidly to near zero.

The second scenario evaluates robustness near singular configurations. The target follows a helical trajectory (Fig. 8 (a)): \[{}^{s}\boldsymbol{p}_{tar}^{k} = \begin{bmatrix} -150+100\cos(2\pi t/T) \\ 525+100\sin(2\pi t/T) \\ 5+250t/T \end{bmatrix}\] The values of the above equation are all in mm, and the total simulation time \(T\) is set to 60s. PI 1, QP 1, and IK are used as the control frameworks in this scenario.

As shown in Fig. 9, PI 1 (blue dash) fails shortly after passing through the singular configuration (around \(50\) s), resulting in large RCM deviation and loss of visual tracking (Fig. 8 (c)). A snapshot at \(t=48\) s in Fig. 8 (c) shows a pronounced deviation of the RCM point from the trocar, accompanied by a loss of tracking in the visual features. IK (green dash) exhibits similar failure. In contrast, QP 1 (red solid) maintains task convergence and RCM enforcement, although transient error fluctuations occur near the singularity (Fig. 9 (a)). These results indicate that QP-based formulations provide improved robustness near singular configurations by explicitly handling task coupling and constraints, whereas PI- and IK-based methods are more sensitive to singularity-induced ill-conditioning.

5.3 Case 5: 2-DoF Visual Servoing↩︎

Some studies consider 2-DoF visual servoing, where only the image position \([u_c; v_c]\) is regulated [15]. Therefore Case 5 compares the performance of different RCM modeling methods and evaluates RCM behavior under this reduced task. RCM 2 and RCM 3 are tested using QP 1 as the controller. For RCM 2, both \(N_{rcm}=2\) and \(N_{rcm}=3\) are considered, denoted as RCM 2 (\(N_{rcm}=2\)) and RCM 2 (\(N_{rcm}=3\)), respectively. For RCM 3, \(\lambda\) is either free or fixed at its initial value (denoted as RCM 3 (Const. \(\lambda\))) to enforce a constant RCM location. The target is static at \([-150;625;5]\) mm in frame \(\{s\}\), with initial image position \([95.83;95.35]\) pixel. The simulation duration is \(T=5\) s.

Snapshots for RCM 2 (\(N_{rcm}=2\)) and RCM 2 (\(N_{rcm}=3\)) are shown in Fig. 10 (a)-(c). Fig. 10 (a) presents the initial configuration at \(t=0\) s, while Figs. 10 (b) and (c) show the final states at \(t=5\) s for RCM 2 (\(N_{rcm}=2\)) and RCM 2 (\(N_{rcm}=3\)), respectively. The corresponding joint-angle variations are provided in Fig. 11, and the evolution of \(\lambda\) is shown in Fig. 10 (d). When RCM 2 (\(N_{rcm}=3\)) is used (blue solid), \(\lambda\) remains constant at \(0.66\) throughout the FoV adjustment, as expected for a fixed RCM point. In contrast, with RCM 2 (\(N_{rcm}=2\)) (orange solid), \(\lambda\) increases from \(0.66\) to \(0.78\), indicating axial retraction of the laparoscope during alignment, as observed in Fig. 10 (b). The configuration cost \(\Delta \boldsymbol{\Theta}^{T}\Delta \boldsymbol{\Theta}\) is \(0.04\) for RCM 2 (\(N_{rcm}=2\)), compared to \(0.07\) for RCM 2 (\(N_{rcm}=3\)), indicating reduced joint motion when the axial constraint is relaxed. A similar increase in \(\lambda\) is observed for RCM 3 (green solid). If a fixed trocar pivot is required, additional constraints on \(\lambda\) are necessary to prevent axial drift.

6 Discussions↩︎

The proposed playground enables systematic analysis of how RCM modeling choices interact with controller architectures in automatic FoV adjustment. The results reveal several structural insights:

1. Impact of RCM modeling choices: RCM formulations differ not only geometrically but also numerically. Tangent-plane-based models (RCM 1) are sensitive to plane definition: defining the plane in frame \(\{s\}\) introduces lateral drift during insertion, whereas defining it in the local RCM frame better preserves the physical trocar constraint. In variable-based models (RCM 3), relaxing axial constraint (\(N_{rcm}=2\)) allows drift of the virtual variable, leading to incorrect RCM localization. These effects stem from how each formulation structures the RCM Jacobian.

2. Open-loop versus closed-loop enforcement: Open-loop formulations enforce geometric consistency but lack error feedback, resulting in steady-state residual errors. Closed-loop formulations incorporate RCM position feedback, enabling convergence of constraint violations to zero and preventing accumulated drift.

3. Controller robustness and singularity sensitivity: Under nominal motion, PI-, IK-, and QP-based methods perform similarly. However, near kinematic singularities, PI- and IK-based methods become sensitive to Jacobian ill-conditioning. Since pseudoinverse-based updates amplify small singular values, constraint coupling can cause rapid error growth and loss of tracking. In contrast, QP-based controllers solve a constrained optimization problem with explicit regularization and inequality handling, improving numerical robustness and stability near singular configurations.

4. Modeling-controller interaction: The overall robustness depends on the interaction between modeling formulation and controller structure. Modeling choices determine the conditioning and dimensionality of the constraint Jacobian, which in turn affects controller sensitivity. Thus, modeling and control cannot be selected independently and their combined structure governs accuracy, stability, and efficiency.

5. Design tradeoffs: Relaxing axial constraints reduces joint motion but introduces RCM drift, revealing a tradeoff between strict constraint enforcement and motion efficiency. Controller selection further influences robustness under challenging trajectories.

Overall, the results indicate that robustness in RCM-constrained visual servoing is governed by the structural coupling between modeling formulation and controller architecture.

7 Conclusions↩︎

This paper presented a unified benchmark framework integrating three representative RCM modeling approaches and six control architectures for automatic FoV adjustment in laparoscopic robots. The comparative study revealed key structural sensitivities in RCM-constrained visual servoing, including the impact of tangent-plane definition, constraint dimensionality, open- versus closed-loop enforcement, and controller robustness near kinematic singularities. The results demonstrate that modeling formulation and controller design are tightly coupled, and their interaction fundamentally determines accuracy, stability, and robustness. Future work will incorporate dynamic effects and evaluate performance under more realistic physical conditions.

Acknowledgments↩︎

The open-source code of the proposed benchmark is available at: https://github.com/zhangj726/RCM-Constrained-IBVS. The authors gratefully acknowledge the assistance of GPT in performing grammar correction and language refinement.

References↩︎

---

1. All authors are with School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China. Corresponding author: Mengtang Li (e-mail: limt29@mail.sysu.edu.cn).↩︎