1 Introduction↩︎

Unmanned aerial vehicles and unmanned surface vehicles (USVs) are increasingly used together in marine operations. In these heterogeneous robotic systems, UAVs provide fast aerial sensing, wide-area surveillance, and target localization, while USVs offer longer endurance, greater payload capacity, and direct interaction with the water surface. This cooperation is especially valuable in tasks such as search and rescue at sea, offshore inspection, environmental monitoring, and pollution response, where aerial observation and surface-level action must work together with limited human intervention [1]–[3]. In such settings, the ability of a UAV to reliably land on a ship or USV is crucial, since it determines whether the vehicle can be safely recovered, recharged, redeployed, and sustained in long-duration missions.

Despite its practical importance, autonomous UAV landing on wave-disturbed marine platforms remains difficult. Unlike fixed infrastructure [4], ship and USV decks are affected by wave-induced platform motion, producing time-varying heave, roll, and pitch motions. These disturbances create major uncertainty in the relative pose and velocity between the UAV and the landing platform, particularly near touchdown, where small errors may cause large impact forces or instability. Wind, sensor noise, and imperfect state estimation further increase the challenge. Therefore, UAV landing in ocean environments is not simply a trajectory tracking problem, but a coupled decision-and-control task that demands precise timing, robustness, and adaptability.

Recent studies have investigated model-based and optimization-driven solutions for UAV landing on marine platforms. For example, Gupta et al. [5] proposed an MPC-based framework that predicts vessel motion online and performs landing during favorable near-zero-tilt windows. Such methods demonstrate the importance of exploiting the temporal structure of wave-induced platform motion. However, they typically require online motion prediction, carefully designed objective functions, and repeated optimization during flight. Their performance may also depend on the accuracy of the predicted platform motion, which can be degraded by stochastic disturbances, sensor noise, and short observation windows.

Reinforcement learning offers an alternative paradigm by directly learning a feedback policy through interaction, avoiding explicit prediction of future platform motion and repeated online optimization. This is particularly appealing for wave-disturbed landing, where the core challenge lies in making timing-sensitive decisions under uncertain platform motion. However, existing RL-based approaches [6] typically rely on high-dimensional visual inputs and discrete action policies, leading to high training complexity, coarse control, and limited integration with reliable low-level flight controllers, which restricts their applicability in safety-critical dynamic scenarios.

To address these limitations, we propose WaveLander, a hierarchical control framework via RL that decouples vertical landing decision-making from low-level flight stabilization. A conventional low-level flight controller ensures robust tracking, while a learned policy outputs a continuous vertical velocity reference relative to the moving platform. This compact formulation reduces learning complexity, improves interpretability, and enables smooth landing behavior without heuristic switching. The main contributions of this paper are summarized as follows:

• A hierarchical control framework via RL, termed WaveLander, is proposed for UAV landing on wave-disturbed platforms by decoupling high-level vertical landing decisions from low-level flight stabilization.

• A compact platform-relative policy interface maps relative height and platform attitude to a continuous vertical velocity reference, reducing learning complexity and simplifying deployment.

• An attitude-aware reward-shaping design is introduced to encourage descent, hold, and retreat behaviors, enabling the learned policy to produce continuous vertical commands without threshold-based switching during deployment.

• The framework is evaluated through randomized simulation, SITL transfer, and a representative deployment-oriented real-world test under dynamic platform motion.

2 Related Work↩︎

Autonomous UAV landing on moving platforms, especially shipborne and USV decks, has been widely studied using control-theoretic, perception-driven, and learning-based approaches. Existing methods can be broadly grouped into two categories: model-based control methods and learning-based methods.

2.1 Model-based Control Methods↩︎

Traditional approaches typically formulate the landing problem as trajectory tracking, state estimation, or optimal control, relying on explicit models of UAV dynamics and platform motion, often with prior prediction of platform movement. For example, Gupta et al. [5] proposed an MPC-based framework that predicts vessel motion online and schedules landing during favorable time windows. To improve robustness under wave disturbances, Li et al. [7] introduced a vision-guided synchronized motion strategy combining Bi-LSTM-based prediction with PID control, while Li et al. [8] further developed an NMPC-based cooperative tracking and landing framework to exploit favorable heave phases for precise touchdown. In addition, Xu et al. [9] proposed a manipulator-assisted system with a robust NMPC controller to actively capture UAVs under disturbances. Despite their interpretability and safety guarantees, these model-based methods rely heavily on accurate modeling and prediction, which become unreliable in realistic maritime environments with stochastic wave-induced motion, leading to degraded performance under highly dynamic conditions.

2.2 Learning-based Methods↩︎

To improve adaptability under uncertain disturbances, reinforcement learning has been increasingly explored for autonomous landing. Early work by Polvara et al. [6] demonstrated end-to-end UAV landing using a hierarchical Deep Q-Network framework directly from low-resolution visual inputs, highlighting the potential of learning-based policies to generalize across diverse conditions. Building on this direction, Rodriguez-Ramos et al. [10] showed that deep reinforcement learning can learn continuous landing control policies for moving platforms and achieve improved robustness compared with conventional low-level flight controllers. More recent studies have further extended learning-based approaches to dynamic landing scenarios with improved adaptability and success rates [11]. In maritime environments, such methods have also been applied to vision-based ship landing and adaptive landing on highly dynamic platforms [12], [13], demonstrating the ability to capture timing-aware and disturbance-tolerant behaviors directly through interaction, without relying on manually designed landing logic. However, many existing approaches attempt to learn full end-to-end control policies, which increases training complexity and limits interpretability. In contrast, this work adopts a hierarchical formulation in which the RL policy focuses on high-level vertical landing decisions, while a conventional low-level flight controller ensures stabilization and tracking.

3 Methodology↩︎

This section presents WaveLander, a hierarchical control framework via RL for UAV landing on wave-disturbed platforms. The learned component outputs a scalar vertical velocity reference from a compact platform-relative observation, while a conventional low-level flight controller handles stabilization, velocity tracking, and lateral alignment. Timing-related landing behavior is encouraged through reward shaping rather than explicit platform-motion prediction or threshold-based switching.

3.1 System Overview↩︎

As shown in Fig. 1, WaveLander adopts a hierarchical control structure. The high-level policy generates a scalar vertical velocity reference \(v^{\mathrm{ref}}_{z,t}\), while the low-level flight controller tracks the commanded motion and maintains attitude and lateral stability. This separation allows learning to focus on high-level vertical landing decisions while preserving compatibility with standard UAV flight-control stacks.

3.2 Problem Formulation↩︎

The landing task is formulated as a high-level decision-making problem rather than a direct motor-control problem and is modeled as an episodic discounted Markov decision process (MDP) [14], \[\mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{P},\mathcal{R},\gamma),\] where \(\mathcal{S}\) denotes the underlying landing-related state space, \(\mathcal{A}\) is the high-level action space, \(\mathcal{P}\) represents the closed-loop transition dynamics induced by the UAV, the low-level controller, and the wave-disturbed platform, \(\mathcal{R}\) is the reward function, and \(\gamma\in[0,1)\) is the discount factor.

In practice, the actor does not receive the full state \(s_t\in\mathcal{S}\). Instead, it is conditioned on a compact platform-relative observation \(o_t\) for deployment simplicity. At each time step, the policy outputs a scalar high-level action, \[a_t=\pi_\psi(o_t),\] which is mapped to a vertical velocity reference and tracked by the low-level flight controller. The objective is to learn policy parameters \(\psi\) that maximize the expected discounted return: \[\psi^* = \operatorname*{arg\,max}_{\psi} \mathbb{E}_{\pi_\psi} \left[ \sum_{t=0}^{\infty} \gamma^t r_t \right]\]

The optimal high-level policy is then given by \(\pi_{\psi^*}\).

3.3 Wave-Disturbed Platform Model↩︎

To train and evaluate the policy under marine-like motion, the landing platform is modeled as a kinematically driven rigid body with heave, roll, and pitch disturbances. Since the focus is the landing policy rather than high-fidelity hydrodynamics, a lightweight randomized wave generator is used. The disturbance along each motion axis is generated by a superposition of sinusoidal components: \[d_{\mathrm{axis}}(t) = \sum_{i=1}^{K} A_i^{(\mathrm{axis})} \sin \left( \omega_i t+\varphi_i^{(\mathrm{axis})} \right), \label{eq:wave95multisine}\tag{1}\] where \(K\) is the number of components, and \(A_i^{(\mathrm{axis})}\), \(\omega_i\), and \(\varphi_i^{(\mathrm{axis})}\) are the amplitude, angular frequency, and phase of the \(i\)-th component. The frequencies are shared across heave, roll, and pitch, while amplitudes and phases are sampled independently to generate varying platform tilt directions.

To improve generalization, the wave parameters are resampled at the beginning of each episode, with larger amplitudes assigned to lower-frequency components. Temporally correlated stochastic variation is added using an Ornstein–Uhlenbeck process [15]: \[n_{t+1} = \rho n_t+\sigma A_{\max}\epsilon_t, \qquad \epsilon_t\sim\mathcal{N}(0,1), \label{eq:ou95noise}\tag{2}\] where \(\rho\) controls temporal correlation, \(\sigma\) controls the noise strength, and \(A_{\max}\) is the maximum disturbance amplitude. To avoid abrupt platform motion, the resulting signal is smoothed before being applied: \[s_{t+1} = (1-\alpha)s_t + \alpha \left( d_{\mathrm{axis}}(t)+n_t \right), \label{eq:wave95smoothing}\tag{3}\] where \(\alpha\) is the smoothing coefficient. The smoothed signals are applied to heave, roll, and pitch, while yaw is fixed.

3.4 Platform-Relative Perception and Control↩︎

For deployment, the platform-relative state is obtained from an AprilTag mounted on the landing platform. The policy does not use raw images; instead, the detector provides a structured relative pose measurement. Let \(P^{\mathrm{cam}}_{\mathrm{tag}}\) denote the detected tag position in the camera frame. With calibrated camera-to-body extrinsics, the tag position is first expressed in the UAV body frame: \[P^{\mathrm{body}}_{\mathrm{body}\rightarrow\mathrm{tag}} = R^{\mathrm{body}}_{\mathrm{cam}}P^{\mathrm{cam}}_{\mathrm{tag}} + P^{\mathrm{body}}_{\mathrm{body}\rightarrow\mathrm{cam}} . \label{eq:tag95body95transform}\tag{4}\] Let \(p^{\mathrm{body}}_{\mathrm{body}\rightarrow\mathrm{gear}}\) be the fixed vector from the UAV body-frame origin to the lowest landing-gear point. The body-frame vector from the landing gear to the pad reference point is \[P^{\mathrm{body}}_{\mathrm{gear}\rightarrow\mathrm{pad}} = P^{\mathrm{body}}_{\mathrm{body}\rightarrow\mathrm{tag}} - p^{\mathrm{body}}_{\mathrm{body}\rightarrow\mathrm{gear}} . \label{eq:gear95to95pad95body}\tag{5}\] This vector is then expressed in the gravity-aligned world frame: \[P^{\mathrm{world}}_{\mathrm{gear}\rightarrow\mathrm{pad}} = R^{\mathrm{world}}_{\mathrm{body}} P^{\mathrm{body}}_{\mathrm{gear}\rightarrow\mathrm{pad}} . \label{eq:gear95to95pad95world}\tag{6}\] The vertical height used by the policy is \[h = -e_z^\top P^{\mathrm{world}}_{\mathrm{gear}\rightarrow\mathrm{pad}}, \label{eq:relative95height}\tag{7}\] where \(e_z=[0,0,1]^\top\). Thus, \(h_t=0\) corresponds to nominal contact between the landing gear and the pad surface.

The platform position error is defined as \[\mathbf{e}_{xy} = \mathbf{p}^{\mathrm{tag}}_{xy} - \mathbf{p}_{xy}.\] Lateral motion is handled by an SE(2) PID tracking controller: \[\mathbf{v}^{\mathrm{ref}}_{xy} = K_p^{xy} \mathbf{e}_{xy} + K_i^{xy} \int \mathbf{e}_{xy} \, dt + K_d^{xy} \dot{\mathbf{e}}_{xy}.\] For vertical motion, the desired vertical velocity \(v^{\mathrm{ref}}_z\) is directly provided by the RL policy as the action \(a_t = \pi_\psi(o_t)\). The overall desired velocity vector is \(\mathbf{v}^{\mathrm{ref}} = [\mathbf{v}^{\mathrm{ref}}_{xy},\; v^{\mathrm{ref}}_z]^\top\).

Define the velocity error vector as \[\mathbf{e}_v = \mathbf{v}^{\mathrm{ref}} - \mathbf{v}.\] A low-level PID controller with gravity compensation then computes the desired acceleration: \[\mathbf{a}^{\mathrm{des}} = K_p^{v} \mathbf{e}_v + K_i^{v} \int \mathbf{e}_v \, dt + K_d^{v} \dot{\mathbf{e}}_v + [0,\;0,\;g_0]^\top.\] The desired thrust direction is obtained by normalizing \(\mathbf{a}^{\mathrm{des}}\): \[\mathbf{z}_{\mathrm{des}} = \frac{\mathbf{a}^{\mathrm{des}}}{\|\mathbf{a}^{\mathrm{des}}\|}.\] The desired thrust direction \(\mathbf{z}_{\mathrm{des}}\) is then sent to a low-level SE(3) attitude controller, which computes the required body torques and collective thrust.

3.5 Observation and Action Design↩︎

The policy uses a compact platform-relative observation to capture the vertical landing state and the attitude trend of the moving platform. Let \(h_t\) and \(\dot{h}_t\) denote the relative height and vertical velocity.

The platform attitude is represented by the quaternion \(q=(q_{w},q_{x},q_{y},q_{z})\). The roll and pitch angles are computed as \[\phi = \operatorname{atan2} \left( 2(q_{w}q_{x}+q_{y}q_{z}), 1-2(q_{x}^2+q_{y}^2) \right), \label{eq:roll95angle}\tag{8}\] \[\theta = \arcsin \left( \operatorname{clip} \left( 2(q_{w}q_{y}-q_{z}q_{x}),-1,1 \right) \right). \label{eq:pitch95angle}\tag{9}\]

The platform tilt magnitude and its time derivative are defined as \[\theta_{tilt} = \sqrt{\phi^2 + \theta^2}, \qquad \dot{\theta}_{tilt} = \frac{d\theta_{\text{tilt}}}{dt}. \label{eq:tilt}\tag{10}\]

The policy observation is \[o=[h,\dot{h},\theta_{tilt},\dot{\theta}_{tilt}]^\top \in \mathbb{R}^{4}. \label{eq}\tag{11}\]

The deployed vertical command is obtained by clipping the raw policy output: \[v^{\mathrm{ref}}_{z} = \operatorname{clip}\!\left( a, -0.45, 0.45 \right)\;\mathrm{m/s}. \label{eq:action95clip}\tag{12}\]

3.6 Reward Design↩︎

The reward is designed to shape the high-level vertical decision under wave-induced platform motion. The objective is to encourage descent when the platform condition is favorable while discouraging further descent and promoting upward retreat when the UAV is close to an excessively tilted platform. This behavior is induced without introducing a threshold-based switching policy; instead, the landing condition is represented through continuous height- and attitude-dependent gates.

At each control step, the total reward is defined as \[R_t = R_t^{\mathrm{prog}} + R_t^{\mathrm{td}} + R_t^{\mathrm{rate}} + R_t^{\mathrm{time}}, \label{eq:total95reward}\tag{13}\] where the four terms correspond to vertical progress modulation, touchdown quality, command smoothness, and delay suppression, respectively.

3.6.1 Proximity and Attitude Assessment↩︎

The landing condition is evaluated using two continuous gates. The height gate measures the proximity to the platform, while the attitude gate measures the suitability of the platform orientation: \[\begin{align} g_h &= \exp\!\left(-\frac{h_t}{\sigma_h}\right), \\ g_{\theta} &= \exp\!\left( -\frac{\max(0,\,\theta_{\mathrm{tilt},t}-\theta_{\mathrm{safe}})}{\sigma_\theta} \right). \end{align} \label{eq:gates}\tag{14}\] Here, \(h_t\) is the relative height, \(\theta_{\mathrm{tilt},t}\) is the platform tilt magnitude, and \(\theta_{\mathrm{safe}}\) is a soft attitude level used only for reward shaping. It is not the hard success threshold used in evaluation. The parameters \(\sigma_h\) and \(\sigma_\theta\) determine the sensitivity to height and tilt, respectively. Therefore, \(g_h\) increases as the UAV approaches the platform, whereas \(g_\theta\) decreases smoothly when the platform tilt exceeds the desired landing range.

3.6.2 Descent–Retreat Modulation↩︎

The two gates are combined into an adaptive coefficient that determines whether vertical motion is rewarded as descent or retreat: \[\alpha_t = 1 - 2g_h(1-g_\theta). \label{eq:alpha}\tag{15}\] The progress term is defined as \[R_t^{\mathrm{prog}} = -w_{\mathrm{prog}}\,\dot{h}_t\,\alpha_t , \label{eq:progress95reward}\tag{16}\] where \(\dot{h}_t<0\) denotes descent and \(\dot{h}_t>0\) denotes upward retreat. When the UAV is far from the platform, \(g_h\approx 0\), so \(\alpha_t\approx 1\) and descent is encouraged. When the UAV is close to a level platform, \(g_h\approx 1\) and \(g_\theta\approx 1\), resulting again in \(\alpha_t\approx 1\), which encourages continued descent. In contrast, when the UAV is close to a highly tilted platform, \(g_h\approx 1\) and \(g_\theta\approx 0\), giving \(\alpha_t\approx -1\). In this case, further descent is penalized and upward retreat is rewarded. Intermediate cases are handled continuously through the same coefficient.

3.6.3 Touchdown Quality Evaluation↩︎

Touchdown is rewarded according to both platform attitude and vertical impact speed. Let \(C_t\in\{0,1\}\) denote the contact indicator. The touchdown quality term is given by \[R_t^{\mathrm{td}} = C_t w_{\mathrm{td}} \exp\!\left( -\frac{\theta_{\mathrm{tilt},t}^2}{2\sigma_{td,\theta}^2} \right) \exp\!\left( -\frac{\dot{h}_t^2}{2\sigma_{td,\dot{h}}^2} \right), \label{eq:terminal95reward}\tag{17}\] where \(\sigma_{td,\theta}\) and \(\sigma_{td,\dot{h}}\) specify the shaping scales for touchdown tilt and vertical speed, respectively. This term assigns the highest reward to near-level and low-speed touchdowns, while providing a graded quality measure for imperfect contacts. The final success rate reported in evaluation is determined separately by the touchdown criteria, rather than by the soft shaping level \(\theta_{\mathrm{safe}}\).

3.6.4 Command Smoothness and Delay Suppression↩︎

To avoid oscillatory or aggressive command changes, an action-rate penalty is included: \[R_t^{\mathrm{rate}} = -w_a(a_t-a_{t-1})^2 . \label{eq:rate}\tag{18}\] In addition, a small time penalty is applied before touchdown: \[R_t^{\mathrm{time}} = -w_{\mathrm{time}}(1-C_t). \label{eq:time}\tag{19}\] This term discourages excessive waiting while keeping the dominant objective on attitude-aware landing quality.

3.6pt

3.7 Training Setup↩︎

The policy is trained in NVIDIA Isaac Lab [16] using Proximal Policy Optimization (PPO) [17] with the RSL-RL implementation [18]. Running mean–standard deviation normalization is applied to the policy inputs, and the actor normalizer is exported together with the deployed JIT policy. The actor and critic are implemented as GRU-based recurrent networks with ELU activations. The actor uses hidden layers of \([128,64]\), while the critic uses \([256,128]\); both use a GRU hidden dimension of \(64\). The initial action noise standard deviation is set to \(1.5\).

The UAV is modeled as a single prismatic joint constrained to vertical translation. A low-level velocity controller tracks the commanded vertical velocity, with randomized damping gains to capture variability in the closed-loop vertical response. Additional domain randomization is applied during training, including a random action delay of \(0\)–\(3\) control steps, a \(\pm30\%\) mass perturbation, and resampled wave parameters in each episode, covering the number of sinusoidal components, frequency range, and Ornstein–Uhlenbeck noise intensity.

A lightweight curriculum on the platform tilt magnitude is used to improve training stability. The policy is first trained under smaller pad tilt angles and is then progressively exposed to larger tilt disturbances. The final checkpoint is selected according to validation performance under randomized tilt conditions.

To improve robustness against state-estimation errors, Gaussian observation noise is added to the actor inputs but not to the critic. The relative position and vertical velocity observations receive zero-mean noise with standard deviation \(0.02\), while the pad attitude features receive zero-mean noise with standard deviation \(0.01\).

For PPO, each iteration collects \(64\) steps from \(30000\) parallel environments, and training is conducted for \(1000\) iterations. The learning rate is initialized at \(1\times10^{-3}\) and adjusted using an adaptive KL schedule with a desired KL divergence of \(0.01\). Each update uses \(5\) epochs and \(16\) mini-batches per epoch. The discount factor and GAE parameter are set to \(\gamma=0.995\) and \(\lambda=0.97\), respectively. The PPO clip range is \(0.2\), the entropy coefficient is \(0.008\), the value-loss coefficient is \(1.0\), clipped value loss is enabled, and the maximum gradient norm is limited to \(1.0\). Training is conducted on a workstation with an AMD Threadripper PRO 5965WX CPU and an NVIDIA RTX A5000 GPU.

4 Experiments and Analysis↩︎

This section evaluates WaveLander through policy behavior analysis, quantitative baseline comparison, SITL transfer, and a representative real-world test.

4.1 Policy Behavior Analysis↩︎

We analyze the learned high-level policy under selected wave-disturbed platform motions. For visualization, safe touchdown intervals are defined by \(\theta_{\mathrm{tilt},t}\le\theta_{\mathrm{ok}}\), where \(\theta_{\mathrm{ok}}=0.10~\mathrm{rad}\) is the same touchdown attitude threshold used in the quantitative evaluation. Fig. 2 shows three representative cases. In Case 1, the platform remains within the safe tilt bound for most of the rollout, and the policy commands a smooth descent to touchdown. In Case 2, the UAV starts close to the platform under an unfavorable attitude; the policy therefore holds or retreats before selecting a suitable safe window to land. In Case 3, stronger platform motion produces short safe windows. The policy mainly continues descent and commands retreat-like motion only when the UAV is close to the platform under unfavorable tilt, then lands when a safe window appears.

These results indicate that the learned policy is not a fixed descent law. It adapts the vertical command according to relative height, vertical velocity, platform attitude, and tilt rate. The actor uses a GRU-based policy, which retains a short-term memory of past observations. Nevertheless, the observed timing behavior is learned from the reward design rather than from an explicit predictive model of future platform motion.

4.2 Experimental Setup↩︎

The complete framework is evaluated in three settings:

• MuJoCo simulation: quantitative baseline comparison under randomized wave-disturbed platform motions.

• Isaac Sim SITL: transfer to a high-fidelity simulation stack using Pegasus [19] and an ArduPilot-based flight controller.

• Real-world deployment: a representative deployment-oriented test on a physical UAV system.

ROS 2 [20] is used for SITL and deployment integration. All methods share the same hierarchical structure: a low-level flight controller handles attitude stabilization and lateral tracking, while the evaluated strategy provides the vertical velocity command.

4.3 MuJoCo Evaluation and Baseline Comparison↩︎

The quantitative evaluation is conducted in the wave-disturbed MuJoCo environments [21]. A video of the MuJoCo experiments is available online.⁷ The platform uses the same randomized motion model described in Section 3.3, where heave, roll, and pitch are generated by a spectral sum of sinusoidal components with Ornstein–Uhlenbeck noise and exponential smoothing. In this evaluation, six sinusoidal components are used, with frequencies uniformly sampled from \(0.05\) to \(1.0~\mathrm{Hz}\).

WaveLander is compared with a constant-descent baseline that commands a fixed vertical velocity of \(-0.3~\mathrm{m/s}\) after lateral alignment. Both methods use the same lateral controller, so the comparison isolates the vertical landing strategy. Let \(\theta_{\rm td}\) denote the touchdown attitude mismatch. A touchdown is counted as successful when contact occurs with \(\theta_{\rm td}\leq0.1~\mathrm{rad}\). Three platform-motion cases are evaluated, with maximum roll/pitch motion ranges of \(30^\circ\), \(40^\circ\), and \(60^\circ\). WaveLander is evaluated on two UAV configurations, denoted CF2 and X2. These configurations use the same high-level landing policy interface but differ in vehicle dynamics and low-level flight-control characteristics. The platform-motion labels 30\(^\circ\), 40\(^\circ\), and 60\(^\circ\) denote the maximum roll and pitch range applied independently to the moving platform. Therefore, these values should be interpreted as per-axis attitude bounds rather than the maximum resultant tilt angle. When roll and pitch vary simultaneously, the resultant platform tilt can exceed the nominal per-axis bound. For each case, 100 paired random seeds are used for both methods.

6pt

A touchdown with \(\theta_{\mathrm{td}}\le 0.10~\mathrm{rad}\)

indicates a high-quality landing with minimal platform tilt at contact.

As shown in Table 2, WaveLander consistently improves the strict touchdown success rate under the criterion \(\theta_{\rm td}\leq 0.10~\mathrm{rad}\). For CF2, the success rate increases from 33%, 14%, and 8% to 66%, 60%, and 39% across the \(30^\circ\), \(40^\circ\), and \(60^\circ\) platform-motion cases, respectively. For X2, the corresponding improvement is from 27%, 21%, and 5% to 76%, 54%, and 30%. These results indicate that the learned vertical policy improves touchdown timing relative to the constant-descent baseline, especially under the \(30^\circ\) and \(40^\circ\) motion cases.

The CDFs in Fig. 4 further support this interpretation. In the 30\(^\circ\) and 40\(^\circ\) cases, the WaveLander curves are shifted toward smaller touchdown mismatch values compared with the constant-descent baseline, indicating that more trials terminate with favorable platform attitude at contact. In the 60\(^\circ\) stress-test case, the gap narrows because the platform motion is more aggressive and favorable touchdown windows become shorter and less frequent. Even in this case, however, WaveLander retains a higher success rate, suggesting that the learned timing strategy provides improved robustness under wave-induced attitude motion.

4.4 Transfer to Isaac Sim with SITL↩︎

To evaluate deployment compatibility, the trained policy is transferred to NVIDIA Isaac Sim using Pegasus [19] and integrated with an ArduPilot-based flight controller in a software-in-the-loop configuration. ROS 2 connects the simulator, policy inference module, and flight-control interface [20].

The policy is deployed without additional training or parameter tuning. The same four-dimensional observation \(o_t=[h,\dot{h},\theta_{tilt},\dot{\theta}_{tilt}]^\top\) and scalar vertical velocity command are used. This experiment verifies that the learned high-level command can be executed in an autopilot-integrated simulation stack without changing the policy interface.

4.5 Real-World Deployment↩︎

WaveLander is further prepared for deployment on a physical UAV system using the same hierarchical structure as in simulation. The flight controller handles stabilization and lateral tracking, while the learned policy provides the vertical velocity reference.

A representative deployment-oriented test was conducted under easterly wind, with an average wind speed of approximately \(10~\mathrm{kt}\) and gusts up to \(18~\mathrm{kt}\). The observed wave height was approximately \(0.8\)–\(0.9~\mathrm{m}\), with a period of about \(5~\mathrm{s}\).

Fig. 5 shows the deployment sequence, including lateral alignment, vertical descent, and touchdown under the tested condition. This experiment demonstrates the feasibility of implementing the compact platform-relative observation and vertical velocity command interface on a physical UAV system. Large-scale real-world validation under broader sea states is left for future work.

4.6 Discussion↩︎

The experiments provide complementary evidence for the effectiveness and deployability of WaveLander. The policy-behavior analysis shows that the learned actor is not a fixed descent law: it descends smoothly under mild platform motion, but slows, holds, or retreats when the platform attitude becomes unfavorable. The MuJoCo comparison is consistent with this interpretation: WaveLander produces a higher fraction of low-attitude-mismatch touchdowns than the constant-descent baseline, indicating that the main improvement comes from more selective touchdown timing rather than from executing a fixed descent command.

These results indicate that WaveLander improves landing performance mainly by learning touchdown timing. Since the lateral controller is shared across methods, the MuJoCo improvement can be attributed primarily to the learned vertical decision policy rather than to lateral tracking differences. Using only relative height, vertical velocity, platform attitude, and short-term tilt-rate information, the policy learns to exploit favorable touchdown phases without an explicit switching rule or future platform-motion predictor.

The hierarchical design is also important for deployment. By restricting the learned component to a scalar vertical velocity reference, WaveLander remains compatible with conventional low-level flight controllers and reduces the sim-to-real burden. The same compact interface can therefore be inserted as a high-level module above existing stabilization and lateral tracking loops.

Several limitations define the current scope of the study. The wave-platform model is intentionally lightweight and is mainly used to generate repeatable heave–tilt disturbances for policy evaluation, rather than to reproduce full marine hydrodynamics. Similarly, near-surface aerodynamic effects, sensing noise, and latency are not modeled in detail, but the hierarchical design and low-speed vertical command help reduce the sensitivity to these effects. Although the actor is recurrent and observes short-term tilt variation, it does not include an explicit future platform-motion prediction module; instead, the touchdown timing behavior is learned implicitly from the observation history and reward structure. Finally, the real-world experiment is intended as a deployment-oriented validation of the interface and control stack, while a larger-scale field evaluation is left for future work. Future extensions will consider perception uncertainty, richer temporal modeling, and higher-fidelity platform and aerodynamic effects.

5 Conclusion↩︎

This paper presented WaveLander, a hierarchical control framework via RL for UAV landing on wave-disturbed platforms. By restricting the learned component to a scalar vertical velocity reference and relying on a conventional low-level flight controller for stabilization and lateral tracking, the proposed framework reduces dynamic platform landing to a compact observation-based vertical decision task. The policy is trained under randomized wave-induced platform motions and operates without explicit threshold-based switching during deployment. Simulation results show that the proposed approach adjusts vertical landing behavior according to relative height and platform attitude, improving robustness over the constant-descent baseline. Future work will focus on perception uncertainty, higher-fidelity aerodynamic effects, temporal observation augmentation, and large-scale real-world validation.

References↩︎

---

1. *The first two authors contributed equally.↩︎

2. \(\dagger\)Corresponding authors: Ling Shi and Pengyu Wang.↩︎

3. \(^{1}\)Chun-Kit Li is with the Department of Mathematics, The Hong Kong University of Science and Technology, Hong Kong SAR. ckliaq@connect.ust.hk↩︎

4. \(^{2}\)Iok Long Sit is with the Division of Integrative Systems and Design, The Hong Kong University of Science and Technology, Hong Kong SAR. ilsit@connect.ust.hk↩︎

5. \(^{3}\)Ming Fung Siu, Ka Yu Kui, Hin Wang Lin, Pengyu Wang and Ling Shi are with the Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR. {mfsiu, kykui, hwlinaa, pwangat}@connect.ust.hk, eesling@ust.hk↩︎

6. \(^{4}\)Chun-Kit Li, Ka Yu Kui, Hin Wang Lin, Pengyu Wang and Ling Shi are also with the Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR. In addition, Ling Shi is with the Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR.↩︎

7. Supplementary video: https://youtu.be/XaS_Jkk0rGU↩︎