UTF8gbsn

1 Introduction↩︎

Autonomous strawberry harvesting remains challenging because of the highly unstructured nature of agricultural environments. Unlike structured industrial settings, strawberry plants exhibit clustered fruit distributions, intertwined ripe and unripe berries, and dense, deformable foliage. Severe occlusion and complex plant geometries further increase perception and manipulation difficulty, often reducing harvesting success and increasing fruit damage in real-world applications. Most existing harvesting systems adopt a modular perception–planning–control pipeline, in which ripe fruits are first detected using vision algorithms, followed by predefined motion planning and rule-based grasping [1], [2]. Although these approaches can be effective in relatively structured scenarios, their performance often deteriorates under severe occlusion and environmental disturbances. When target fruits are obstructed by leaves or unripe berries, passive collision avoidance or forced grasping motions are typically used, while the surrounding environment is rarely manipulated to improve harvesting feasibility [3], [4]. Consequently, existing systems remain limited in their ability to actively clear obstacles before fruit detachment.

Reinforcement learning (RL) provides a promising alternative by formulating harvesting as a sequential decision-making problem. Through closed-loop state–action–reward interactions, long-horizon behaviors can be learned without requiring explicit models of complex plant interactions. However, the application of RL to agricultural harvesting remains constrained by deformable plant contact, severe occlusion, the limited feasibility of real-world training, and sim-to-real discrepancies [5].

To address these challenges, a policy-driven reinforcement learning framework with heuristic phase coordination was developed for occlusion-aware strawberry harvesting, as illustrated in Fig. 1. The obstacle-separation, fruit-detachment, and placement workflow was formulated as a sequential decision-making problem. A shared interaction-aware policy generated Cartesian motions across all task phases, while lightweight heuristic logic coordinated phase transitions and gripper events.

The main contributions are summarized as follows:

• Policy-driven interaction-aware harvesting: A shared interaction-aware policy was developed to generate obstacle-separation, fruit-detachment, and placement motions, while heuristic phase coordination organized task progression without replacing policy-based motion control.

• Feasibility-oriented sim-to-real alignment: A shared structured observation space was used across task phases and aligned between simulation and real-world deployment to support zero-shot transfer.

• Real-world validation: The proposed framework achieved an 82.0% success rate in an elevated strawberry cultivation environment without real-world fine-tuning.

2 Related Work↩︎

2.1 Reinforcement Learning in Agricultural Manipulation↩︎

Reinforcement learning (RL) has recently been applied to agricultural manipulation. In strawberry harvesting, policies trained in FruitGym, a MuJoCo-based environment incorporating large-scale domain randomization and Dormant Ratio Minimization, were deployed on a Franka Panda using Cartesian impedance control [6]. Although effective sim-to-real transfer was demonstrated, the task primarily focused on target reaching and grasping rather than active manipulation of occluding plant elements [7]. Beyond strawberry harvesting, a 7-DoF manipulator was trained for grape pruning using proximal policy optimization (PPO) by Yandún et al., with perception outputs incorporated into the state representation [8]. Similarly, detected banana targets were included in the RL state by Lin et al., while the policy primarily learned inverse kinematics solutions [9]. These studies demonstrated the feasibility of using RL as a motion generator conditioned on structured perception inputs. However, active obstacle separation and shared policy control across obstacle separation, fruit detachment, and placement under severe occlusion remain underexplored [10].

2.2 Reinforcement Learning for Manipulation in Unstructured Environments↩︎

Beyond agriculture, deep reinforcement learning has demonstrated strong potential for manipulation in cluttered or deformable environments. Deng et al. applied RL to pushing and picking in cluttered tabletop scenes, where target objects were separated through physical interaction without explicit motion planning [11]. Sim-to-real RL for deformable-object manipulation was investigated by Matas et al., and simulation-trained policies were transferred to real-world cloth manipulation tasks [12]. The SoftGym benchmark was introduced by Lin et al. to study long-horizon control of deformable objects [13]. However, these studies primarily addressed generic manipulation in controlled environments or tasks with single-stage objectives. Agricultural harvesting additionally involves severe occlusion, interleaved fruit distributions, and deformable plant structures, requiring coordinated interaction across multiple task phases. Shared policy control of sequential behaviors, including active obstacle separation, fruit detachment, and placement, remains insufficiently investigated [14].

2.3 Position of This Work↩︎

In contrast to prior agricultural RL methods that primarily focused on target reaching, grasping, or motion generation from structured perception inputs, this work formulated strawberry harvesting as an occlusion-aware sequential decision-making problem involving active environmental interaction. Occluding fruits and plant structures were manipulated to create accessible harvesting space before fruit detachment. An interaction-aware policy generated Cartesian motions across obstacle separation, fruit detachment, and placement, while heuristic phase coordination organized task progression and gripper events. Therefore, the contribution of this work lies in extending policy-based agricultural manipulation from isolated target-directed operations to active obstacle separation within a complete harvesting workflow, rather than in the standalone application of RL or Cartesian impedance control.

3 Method↩︎

The proposed framework integrated visual perception, policy-based Cartesian motion generation, heuristic phase coordination, and compliant robot execution for occlusion-aware strawberry harvesting. The perception and control architecture, simulation environment, observation and action spaces, and policy-learning method are described in the following subsections.

3.1 Perception-Driven Policy Architecture↩︎

A hierarchical control architecture was adopted to decouple policy-level Cartesian decision-making from robot-specific execution. Visual perception served as a supporting component that provided fruit-level semantic and geometric information for policy inference. Specifically, a YOLOv11-based SRR-Net estimated strawberry category, instance mask, spatial position, and maturity through parallel detection, segmentation, and ripeness-regression heads. RetinexNet-based illumination normalization was further applied to reduce lighting-induced color distortion under greenhouse conditions [15].The detected fruit states were fused with robot proprioceptive information to form the structured observation vector described in the following subsection. Based on this input, the interaction-aware policy generated Cartesian motion increments, while heuristic logic coordinated phase transitions and gripper events. The resulting Cartesian commands were converted into joint commands through inverse kinematics and executed using Cartesian impedance control to support compliant interaction with deformable plant structures [16]. This decoupling reduced the dependence of policy inference on platform-specific kinematics and supported transfer across structurally different robotic platforms.

3.2 Simulation Environment and Domain Randomization↩︎

A PyBullet-based simulation environment was constructed to model robot motion and contact interactions during strawberry harvesting. The simulated system consisted of a mobile base, a 7-DoF Franka Panda manipulator, a V-shaped elevated cultivation rack, and plant models containing ripe fruits, unripe fruits, and stems. Fruit bodies and stems were represented as separate links, while ripe and unripe fruits were assigned dark red and green colors, respectively, to approximate the primary visual cue used for maturity recognition. To approximate plant compliance, each stem was connected to the cultivation rack using a point-to-point spring–damper constraint, allowing oscillation and rebound during contact. Domain randomization was applied to plant geometry, occlusion complexity, and physical parameters [17].

(1) Structural Randomization of Plant Geometry: Stem joint angles were independently sampled from a uniform distribution within \(\pm 0.15\) rad (approximately \(\pm 8.6^\circ\)), corresponding to approximately 30% of the original URDF joint limit of \(\pm 0.523\) rad. This range provided plant-pose diversity while avoiding unrealistic deformation. The lateral displacement of the stem tip is approximated as

\[\Delta x \approx L \sin(0.15),\] where \(L\) denotes the stem length. For \(L=0.25~\mathrm{m}\), the resulting displacement was approximately \(3.7~\mathrm{cm}\), which was sufficient to alter occlusion relationships and interaction paths.

(2) Occlusion Complexity Randomization: During each episode, 1–5 unripe strawberries were randomly generated around the target fruit to produce different occlusion levels. This setting exposed the policy to varying interaction complexity and encouraged active obstacle-separation behavior during training.

(3) Physical Parameter Perturbation: Spring–damper parameters were randomized at each episode reset to approximate variations in plant stiffness and contact dynamics. The maximum constraint force was sampled within 48–72 N around a nominal value of 60 N, while the error reduction parameter (ERP) was sampled within 0.18–0.22 around a nominal value of 0.2. Additional parameters, including mass, damping coefficients, contact friction, and initial fruit poses, were also perturbed within bounded ranges. These perturbations increased physical diversity while maintaining numerical stability.

3.3 Observation Space↩︎

A compact 32-dimensional shared structured observation space was designed to support obstacle-aware decision-making across obstacle separation, fruit detachment, and placement. Let the tool center point (TCP) and target-fruit positions be denoted by \(\mathbf{p}_{\mathrm{tcp}}\) and \(\mathbf{p}_{\mathrm{tgt}}\), respectively. Their Euclidean distance is defined as

\[d_t = \left\lVert \mathbf{p}_{\mathrm{tcp}} - \mathbf{p}_{\mathrm{tgt}} \right\rVert_2, \label{eq:distance}\tag{1}\]

The observation vector consisted of four components:

• End-effector state (8D): The TCP position \([x,y,z]\), orientation representation \([q_x,q_y,q_z,q_{\mathrm{pitch}}]\), and target-relative distance \(d_{\mathrm{target}}\) in the world frame were included.

• Target state (5D): The target-relative position \(\Delta\mathbf{p}_{\mathrm{target}}\), Euclidean distance \(d_t\), and distance change \(\Delta d_t\) were included.

• Obstacle representation (15D): The \(K=3\) nearest obstacles were encoded, with each obstacle represented by a five-dimensional feature vector containing its relative position, binary ripeness encoding indicating ripe or unripe fruit, and distance to the TCP.

• Task context (4D): The relative drop-box position and a binary target-fruit grasp-status flag were included.

3.4 Action Space↩︎

Cartesian incremental control was adopted instead of joint-space control to reduce dependence on platform-specific kinematics and improve policy-learning stability. The four-dimensional normalized action is defined as

\[\mathbf{a}_t = \left[ a_{x,t}, a_{y,t}, a_{z,t}, a_{\mathrm{pitch},t} \right] \in [-1,1]^4, \label{eq:action95space}\tag{2}\] where the first three components represent translational actions and the fourth component represents pitch rotation. The normalized action was mapped to TCP motion increments as

\[\begin{align} \Delta \mathbf{p}_{\mathrm{tcp},t} &= \alpha_{\mathrm{pos}} \left[ a_{x,t}, a_{y,t}, a_{z,t} \right]^{\mathrm{T}}, \\ \Delta \theta_{\mathrm{pitch},t} &= \alpha_{\mathrm{rot}}a_{\mathrm{pitch},t}, \end{align} \label{eq:action95mapping}\tag{3}\] where \(\alpha_{\mathrm{pos}}\) and \(\alpha_{\mathrm{rot}}\) denote the translational and rotational scaling factors, respectively. This mapping bounded the motion increments and remained compatible with Cartesian impedance execution. Gripper closure and release were coordinated by the heuristic phase logic and were not included in the continuous action space.

4 Policy Learning↩︎

This section describes the task organization, curriculum strategy, and composite reward used to train the interaction-aware policy. Proximal policy optimization (PPO) [18] was adopted for continuous Cartesian control. Unlike conventional modular pipelines [19], the same policy generated Cartesian motions throughout the harvesting workflow, while heuristic logic coordinated task transitions and gripper events.

4.1 Policy Learning Framework↩︎

Each harvesting episode consisted of two task modes: harvesting and placement. The harvesting mode included active obstacle separation and fruit detachment. When surrounding fruits obstructed the approach to the target, the policy generated interaction motions to progressively create accessible harvesting space before the final approach. The number, direction, and order of pushing motions emerged through policy optimization rather than being specified by predefined motion sequences. During execution, the spatial relationships among the end-effector, target fruit, and surrounding obstacles were continuously updated through the structured observation vector. The policy adapted its Cartesian actions to changes caused by plant contact and obstacle displacement. Heuristic phase coordination evaluated whether the target neighborhood had been sufficiently cleared and coordinated the grasp event, while the approach and interaction motions remained controlled by the learned policy.

After successful fruit detachment was confirmed, the task context was switched to the placement mode, in which the drop-box position served as the target reference. The policy then generated Cartesian motions based on the relative position between the end-effector and the drop box, while avoiding unnecessary contact with surrounding plant structures. The release event was triggered after the placement condition was satisfied. This organization preserved policy-based motion control while coordinating task transitions and gripper events.

4.2 Curriculum Learning↩︎

To accelerate convergence and stabilize policy training, a two-part curriculum strategy was applied.

• Distance curriculum: During early training, the TCP was initialized 15 cm from the target fruit. The initialization distance was then gradually increased to 1.0 m as training progressed.

• Mid-state initialization: With a predefined probability, episodes were initialized from a post-grasp state in which the target fruit had already been grasped, thereby providing sufficient training samples for placement.

This strategy reduced the imbalance between harvesting and placement experiences and facilitated learning of the complete long-horizon workflow.

4.3 Composite Reward Design for the Harvesting Process↩︎

To facilitate learning of interaction-aware behaviors, a composite reward function was designed to guide target approach, active obstacle separation, fruit detachment, and placement. The reward design followed three principles: (i) providing dense guidance for efficient convergence during target approach, (ii) encouraging physically meaningful interactions that cleared occluding fruits before the final approach, and (iii) penalizing unsafe contacts, premature approach, and excessive interaction forces. Through this reward structure, the shared interaction-aware policy was encouraged to create accessible harvesting space before fruit detachment and subsequently complete placement.

The total reward is defined as

\[R_t = r_{\text{dist}} + r_{\text{prog}} + r_{\text{orient}} + r_{\text{push}} + r_{\text{seq}} + r_{\text{success}} + r_{\text{safe}} \label{eq:reward95total}\tag{4}\]

Notation↩︎

Let \(\mathbf{p}_{\text{tcp}}, \mathbf{p}_{\text{tgt}} \in \mathbb{R}^3\) denote the TCP and target positions. The Euclidean distance is defined as

\[d_t = \left\lVert \mathbf{p}_{\text{tcp}} - \mathbf{p}_{\text{tgt}} \right\rVert_2 \label{eq:distance95def}\tag{5}\]

The distance variation is defined as

\[\Delta d_t = d_{t-1} - d_t \label{eq:delta95distance}\tag{6}\]

The obstacle set is denoted as \(\mathcal{O} = \{o_1,\dots,o_K\}\).

4.3.0.1 Distance Reward

\[r_{\text{dist}} = w_{d1} \left(1 - \tanh(\lambda d_t)\right) \label{eq:r95dist}\tag{7}\]

Through this term, dense global attraction toward the target is provided.

4.3.0.2 Progress Reward

\[r_{\text{prog}} = \begin{cases} \lambda_{\text{app}} \Delta d_t, & \Delta d_t > 0 \\ \lambda_{\text{ret}} \Delta d_t, & \Delta d_t \le 0 \end{cases} \label{eq:r95prog}\tag{8}\]

Oscillatory retreat behavior was discouraged by the asymmetric coefficients (\(\lambda_{\text{ret}} > \lambda_{\text{app}}\)).

4.3.0.3 Orientation Constraint

Pitch deviation is defined as

\[\theta_{\text{dist}} = \theta_{\text{pitch}} - \theta_{\text{opt}} \label{eq:theta95dist}\tag{9}\]

The orientation reward is defined as

\[r_{\text{orient}} = \begin{cases} 1 - \dfrac{|\theta_{\text{dist}}|}{\delta_{\text{tol}}}, & |\theta_{\text{dist}}| \le \delta_{\text{tol}} \\ -\beta |\theta_{\text{dist}}|, & \text{otherwise} \end{cases} \label{eq:r95orient}\tag{10}\]

Through this term, the policy was biased toward kinematically feasible grasp configurations.

4.3.0.4 Obstacle Interaction

For each obstacle \(o_i\), its distance to the target fruit was defined as

\[d_{\mathrm{obs\text{-}tgt},t}^{(i)} = \left\lVert \mathbf{p}_{\mathrm{obs},t}^{(i)} - \mathbf{p}_{\mathrm{tgt},t} \right\rVert_2, \label{eq:d95obs}\tag{11}\] where \(\mathbf{p}_{\mathrm{obs},t}^{(i)}\) and \(\mathbf{p}_{\mathrm{tgt},t}\) denote the obstacle and target positions, respectively. The obstacle-separation reward was defined as

\[r_{\mathrm{push},t} = C_{\mathrm{push}} \sum_{o_i \in \mathcal{O}_{\mathrm{obs}}} \mathbb{I} \left( d_{\mathrm{obs\text{-}tgt},t}^{(i)} > d_{\mathrm{safe}} \right) \left( 1-b_{i,t-1} \right), \label{eq:r95push}\tag{12}\] where \(C_{\mathrm{push}}>0\) denotes the fixed separation reward, and \(b_{i,t-1}\in\{0,1\}\) indicates whether obstacle \(o_i\) had previously satisfied the separation condition. Once the reward was triggered, \(b_{i,t}\) was set to 1. Thus, each obstacle was rewarded only once, and additional displacement beyond \(d_{\mathrm{safe}}\) yielded no further reward.

4.3.0.5 Sequential Constraint

The target-neighborhood clearance condition was defined as

\[S_{\mathrm{clear},t} \iff \forall o_i \in \mathcal{O}_{\mathrm{obs}}, \quad d_{\mathrm{obs\text{-}tgt},t}^{(i)} > d_{\mathrm{safe}}. \label{eq:s95clear}\tag{13}\]

The sequential penalty was defined as

\[r_{\mathrm{seq},t} = - C_{\mathrm{seq}} \mathbb{I} \left( \neg S_{\mathrm{clear},t} \land d_t < d_{\mathrm{near}} \right), \label{eq:r95seq}\tag{14}\] where \(C_{\mathrm{seq}}>0\) denotes the penalty coefficient, \(d_t\) denotes the end-effector-to-target distance, and \(d_{\mathrm{near}}\) denotes the near-target threshold. This term penalized premature approach and encouraged obstacle separation before fruit detachment.

4.3.0.6 Success Reward

\[r_{\text{success}} = \mathbb{I}_{\text{picked}} \left( R_{\text{base}} + \max(0, T_{\text{max}} - \eta t) \right) \label{eq:r95success}\tag{15}\]

4.3.0.7 Safety Penalty

\[r_{\text{safe}} = -w_f \max(0, F_{\text{contact}} - F_{\text{limit}}) - w_c \mathbb{I}_{\text{collision}} \label{eq:r95safe}\tag{16}\]

Through this penalty, excessive contact force and unintended collisions were discouraged.

5 Policy Training↩︎

This section describes the simulation-based training setup and implementation details.

5.1 Simulation Infrastructure↩︎

Training was conducted using 16 parallel instances of the PyBullet environment described above on an NVIDIA RTX 6000 GPU. Parallel rollout collection increased training throughput and provided diverse interaction trajectories for policy optimization.

5.2 Algorithm and Network Architecture↩︎

The interaction-aware policy was trained using PPO. PPO was selected for its empirical stability in continuous control tasks and compatibility with parallel simulation. The implementation was based on Stable-Baselines3. An actor–critic architecture was adopted, in which a multilayer perceptron (MLP) with hidden-layer sizes of 512–512–256–512 was used as the feature extractor. The policy and value heads each consisted of two hidden layers with 256 units per layer.

3pt

The learning rate was linearly decreased from \(1\times10^{-4}\) to \(5\times10^{-5}\), while the entropy coefficient was decreased from 0.05 to 0.04 to balance exploration and policy convergence. The PPO clipping range was linearly decreased from 0.2 to 0.1. The discount factor and generalized advantage estimation (GAE) parameter were set to \(\gamma=0.99\) and \(\lambda=0.95\), respectively. The main training configuration is summarized in Table 1.

Training converged after approximately 6.03 million timesteps. As shown in Fig. 2, the average episode reward stabilized around 2357 during the late training stage, while the simulated harvesting success rate reached 89.7%. The value loss decreased from 0.139 to 0.027, corresponding to an approximately 81% reduction, while the policy loss and entropy loss remained near \(-0.006\) and \(-4.60\), respectively. These training diagnostics indicated stable policy optimization. The trained policy parameters were subsequently fixed for real-world deployment.

5.3 Ablation Study↩︎

Two ablation experiments were conducted to evaluate the effects of the obstacle-separation mechanism and curriculum learning. For a fair comparison, the same PPO network architecture, four-dimensional action space, placement procedure, evaluation conditions, and maximum episode length of 900 steps were used across the corresponding experimental groups. The specific components removed in each ablation are described below.

5.3.1 Effect of Obstacle Separation↩︎

The first ablation evaluated the combined contribution of heuristic obstacle-separation coordination and its associated reward guidance. In the baseline, separation activation, separation confirmation, the post-separation fruit-detachment gate, and the obstacle-separation-related reward terms were disabled. The remaining reward components, policy-based motion control, and placement procedure were kept unchanged. Thus, the shared interaction-aware policy remained responsible for Cartesian motion generation in both groups.

Five training repetitions were conducted using seeds 0–4, with each seed trained for 100,000 steps. The harvesting success rates with and without the obstacle-separation mechanism were 88%/60%, 85%/59%, 88%/52%, 93%/58%, and 86%/56%, respectively. The mean success rate increased from 57.0% to 88.0%, corresponding to an improvement of 31.0 percentage points. These results demonstrated that the obstacle-separation mechanism substantially improved harvesting reliability when obstacles were located near the target fruit.

5.3.2 Effect of Curriculum Learning↩︎

The second ablation evaluated early-stage training efficiency. In the uniform-training baseline, the distance curriculum and mid-state initialization were removed, while the complete obstacle-separation mechanism, reward design, placement control, and evaluation conditions remained unchanged. Five paired seeds were trained for 300,000 steps from random initialization, with checkpoints saved every 50,000 steps.

The convergence criterion was defined as the first training step at which the full-task success rate remained at or above 50% for two consecutive checkpoints. The curriculum group satisfied this criterion for all five seeds, whereas the uniform-training group satisfied it for only two seeds. The normalized area under the success-rate curve (AUC) over 0–300,000 steps increased from 43.77% to 64.33%. These results indicated that curriculum learning improved early convergence and learning efficiency.

Overall, the obstacle-separation mechanism primarily improved final harvesting reliability, whereas curriculum learning mainly accelerated early-stage policy training.

6 Sim-to-Real Transfer↩︎

Zero-shot transfer was supported by preserving consistency in observation representation, action semantics, coordinate definitions, and workspace feasibility between simulation and the real system. The transfer strategy focused on task-specific consistency adaptations rather than introducing new normalization or low-level control techniques.

6pt

6.1 Observation Alignment and Sim-to-Real Consistency↩︎

A key challenge in sim-to-real deployment arose from inconsistencies between simulated and real observation representations. To address this issue, a Feasibility-First principle was adopted. The shared structured observation space defined above was restricted to states physically measurable by the real system, while the simulation environment was adapted to reproduce the same observation semantics. Formally, let \(\mathcal{S}^{\mathrm{real}}\) denote the set of physically measurable states. The aligned observation space \(\mathcal{O}\) is defined as

\[\mathcal{O} = \arg\max_{O \subseteq \mathcal{S}^{\mathrm{real}}} \mathcal{I}(O;\pi^*), \label{eq:observation95alignment}\tag{17}\] where \(\mathcal{I}(O;\pi^*)\) denotes the mutual information between a candidate observation subset \(O\) and the optimal policy \(\pi^*\). Under real-world sensing constraints, this formulation retained decision-relevant information while reducing dependence on variables unavailable during deployment.

Three alignment mechanisms were implemented to maintain consistency between simulation and real-world deployment.

6.1.1 Geometric and Coordinate Consistency↩︎

The policy was conditioned on the TCP pose expressed relative to the robot base. Differences in coordinate semantics between simulation and hardware could introduce out-of-distribution observations and produce invalid control commands. Therefore, the base reference frames, coordinate origins, and TCP definitions were aligned between the simulated and real systems. Kinematic consistency was verified using multiple canonical poses, for which the positional deviation between simulation and hardware remained within \(5~\mathrm{mm}\).

6.1.2 Action Semantics Consistency↩︎

The normalized actions generated by the policy were transformed into executable TCP increments through a multi-stage processing pipeline comprising base scaling, distance-adaptive scaling, action smoothing, and directional correction. The same processing sequence and parameter definitions were reproduced during real-world deployment to preserve the action semantics learned in simulation. When a processed command exceeded the predefined safety limit, proportional safety scaling was applied to reduce its magnitude while preserving its direction.

6.1.3 Observation Distribution Alignment↩︎

During training, observations were normalized using the VecNormalize module in Stable-Baselines3, which maintained running estimates of the per-dimension means \(\mu_i\) and variances \(\sigma_i^2\). The stored training-time normalization statistics were reused during deployment to preserve consistency in observation preprocessing. All observation components were also kept consistent in physical meaning, dimensionality, computation method, and coordinate reference, thereby reducing semantic and statistical drift between simulation and real-world deployment.

6.2 Workspace Constraints↩︎

The executable workspace was defined as the intersection of the simulated Franka Panda workspace and the workspace of the LingXtend manipulator used in the real-world experiments. Cartesian commands outside this region were clipped to the corresponding workspace bounds, ensuring that all executed motions remained feasible for both robotic platforms and within predefined mechanical safety limits.

6.3 Real-World Experimental Setup and Results↩︎

Real-world experiments were conducted using the LingXtend hybrid serial–parallel manipulator [20]. A Cartesian impedance controller was implemented in ROS to support compliant interaction with deformable plant structures. Visual observations were acquired using an Intel RealSense D435 camera mounted near the robot base. The experiments were performed in an elevated strawberry cultivation system at a commercial greenhouse operated by Cuihu Agriculture Technology Co., Ltd., Beijing, China, where ripe and unripe fruits were distributed in an interleaved manner.

Five occlusion levels with 1–5 surrounding obstacles were evaluated. Twenty harvesting trials were conducted at each level, resulting in a total of 100 real-world trials. A trial was regarded as successful only when the complete harvesting sequence was accomplished: the occluding fruits were cleared from the approach path, the target strawberry was detached, and the fruit was transported to and released at a position \(5~\mathrm{cm}\) above the drop box. Failure at any stage, including obstacle separation, fruit detachment, target tracking, or placement, resulted in an unsuccessful trial. The proposed framework completed the full harvesting sequence in 82 trials, yielding an overall success rate of 82.0%. As summarized in Table 2, the average total execution time increased from 12.99 s to 21.73 s as the occlusion level increased from 1 to 5.

Among the 18 failed trials, target strawberries positioned close to the cultivation rack restricted feasible approach angles and reduced the available manipulation space. Insufficient stem bending also prevented fruit detachment despite appropriate end-effector positioning, as illustrated in Fig. 3. In addition, displaced unripe strawberries occasionally rebounded because of plant elasticity, reintroducing occlusion and causing target loss or suboptimal approach motions. Harvesting trajectories under different occlusion levels are shown in Fig. 4, while the simulation and real-world success rates are compared in Fig. 5. Despite these failures, the results demonstrated the feasibility of zero-shot sim-to-real transfer without additional real-world fine-tuning.

7 Discussion↩︎

A direct quantitative comparison with existing strawberry-harvesting methods was not conducted because most previous studies evaluated isolated operations, such as target approach, grasping, or fruit detachment, whereas this work evaluated the complete workflow of obstacle separation, fruit detachment, and placement. Moreover, the spring–damper model did not fully reproduce the nonlinear deformation and contact dynamics of real strawberry plants, and experiments were conducted using only one robotic platform in a single elevated cultivation environment. Variations across cultivars, seasons, and cultivation systems, together with fruit and plant damage, were not systematically evaluated. Future work will include standardized full-process comparisons, broader field validation, more realistic plant models, and force- and damage-aware evaluation.

8 Conclusion↩︎

A policy-driven reinforcement learning framework with heuristic phase coordination was developed for occlusion-aware strawberry harvesting. The interaction-aware policy generated Cartesian motions throughout the harvesting workflow, while heuristic logic coordinated task transitions and gripper events. A hierarchical control architecture and consistency-oriented sim-to-real adaptations supported compliant execution and zero-shot deployment on a structurally different robotic platform. The policy achieved success rates of 89.7% in simulation and 82.0% in real-world experiments. As the occlusion level increased from 1 to 5, the average execution time increased from 12.99 s to 21.73 s, primarily because of longer obstacle-separation and target-approach stages, while perception and policy-inference times remained relatively stable. These results demonstrated the feasibility of transferring interaction-aware harvesting behaviors learned in simulation to real-world strawberry harvesting without additional policy fine-tuning.

Acknowledgment↩︎

Figure 1 was created using a generative AI tool (Gemini) based on a text prompt. A demonstration video is available at: https://youtu.be/3X6CT0IuVKc.