1 Introduction↩︎

In recent years, quadrotor Unmanned Aerial Vehicles (UAVs) have been widely adopted in logistics, search and rescue, and exploration missions, owing to their exceptional maneuverability and agility [1], [2]. However, navigation in these unstructured and dynamic environments requires the system to perceive sudden obstacles and execute evasive maneuvers within short time. This creates a fundamental conflict between the demand for high-speed flight and the computational latency required to guarantee rigorous collision avoidance.

Traditional navigation pipelines decompose autonomy into sequential perception, planning, and control modules, achieving reliable performance through explicit algorithmic structures and well-established safety margins [3]–[5]. However, this modular design inevitably incurs cumulative latency, as perception updates, global replanning, and trajectory optimization are repeatedly executed in a closed loop, leading to delayed reactions in rapidly changing environments [6]. Moreover, planning in densely cluttered spaces often requires high-dimensional search or optimization, causing computational costs to scale poorly with environmental complexity and limiting real-time performance at high flight speeds.

In contrast, learning-based navigation has emerged as an attractive alternative by directly mapping sensory observations to control commands [7]–[11]. Such end-to-end policies significantly reduce processing latency and have demonstrated superior agility in high-speed flight scenarios compared to modular pipelines [12]. Despite these advantages, purely learning-based approaches typically lack theoretical safety guarantees and are vulnerable to out-of-distribution generalization failures, which fundamentally constrain their reliability in safety-critical applications.

To address the lack of formal safety guarantees in purely end-to-end navigation policies while preserving their agility in complex environments, we propose a hybrid navigation framework that tightly couples reinforcement learning with model-based safety mechanisms. The key idea is to exploit the low-latency and expressive power of end-to-end policies for high-speed maneuvering, while enforcing safety through explicit physical constraints. During training, we incorporate model-based priors into the learning process by shaping the reward with global path guidance and safety-related constraints, encouraging the policy to acquire navigation behaviors that are both efficient and risk-aware. During deployment, we further introduce a real-time safety correction module based on high-order control barrier functions, which projects policy outputs onto a provably safe set to guarantee collision avoidance under unforeseen disturbances and distribution shifts. In addition, we employ robust sim-to-real transfer strategies to ensure consistent performance on real quadrotor platforms.

Our main contributions are summarized as follows:

• A training-time safety-aware navigation framework that integrates global path structure and local collision avoidance into end-to-end reinforcement learning. The proposed approach alleviates local minima in Euclidean distance objectives and enables more foresighted navigation behaviors in cluttered environments.

• A safety enforcement mechanism based on control barrier functions, which guarantees collision avoidance by correcting policy outputs in real time. This mechanism complements learning-based agility with formal safety constraints, enabling high-speed flight without sacrificing reliability under distribution shifts.

• A comprehensive experimental validation across simulation and real-world platforms. Extensive benchmarks demonstrate superior performance over both traditional planners and learning-based baselines, while real-world experiments validate robust sim-to-real transfer and agile navigation in densely cluttered environments.

2 Related Work↩︎

2.1 Modular Navigation for UAV↩︎

Traditional optimization-based navigation methods typically rely on explicit 3D environmental representations constructed from depth measurements and manually derived safety constraints [3], [13], [14]. Hard-constrained approaches, for example, decouple planning into extracting a Safe Flight Corridor(SFC)-represented by cubes, spheres, or polyhedrons and subsequently generating trajectories via convex optimization liu2017sfc?. While ensuring global optimality within the corridor, these methods often depend on heuristic time allocation strategies, which can degrade trajectory quality and lead to conservative dynamic constraints that limit agility.

Conversely, soft-constrained methods using Euclidean Signed Distance Fields (ESDF) jointly optimize planning and control, leveraging gradient information to converge upon feasible trajectories within continuous space. However, maintaining real-time ESDF imposes substantial computational overhead on onboard processors due to the expensive calculation of line integral calculations, necessitating a difficult trade-off between mapping precision and update frequency. To mitigate this, Ego-Planner [14] circumvents explicit ESDF construction by iteratively generating safe guidance paths to provide obstacle avoidance gradients. Nevertheless, such methods lack theoretical convergence guarantees and remain susceptible to entrapment in unsafe local minima within complex, non-convex environments. Furthermore, the cascading architecture of these multi-module systems introduces significant parameter complexity and cumulative latency, leading to state estimation instability during high-speed flight and limiting robustness in dynamic scenarios.

2.2 Learning-Based Methods↩︎

Recent studies underscore the immense potential of learning-based methodologies, which obviate the need for dense mapping or explicit environmental modeling. By directly mapping sensory observations to control commands, these approaches circumvent the cumulative latency and computational burden associated with modular trajectory optimization, demonstrating impressive maneuverability [15]–[17]. NavRL [18] introduces a PPO-based framework that enables safe and reactive navigation in dynamic environments with moving obstacles, achieving low collision rates through careful reward design and curriculum training. Similarly, approaches such as  [10] and  [19] proposes an RL pipeline that dynamically adapts flight speed based on environmental clutter, striking an effective balance between success rate and agility in unknown cluttered spaces. Furthermore, emerging research has leveraged differentiable simulators and optimization-embedded networks to refine controller performance. hybrid approaches [20] embed traditional trajectory optimization within neural networks to generate dynamically feasible trajectories directly from depth inputs, bridging perception and planning while ensuring kinematic constraints without explicit mapping.

While these learning-based approaches have propelled UAV agility to new heights, their inherent lack of theoretical completeness often compromises reliability in safety-critical scenarios. Hybrid architectures integrating Control Barrier Functions (CBFs) have emerged as a prominent approach to address this limitation. For example, the RL-CBF framework proposed by Cheng et al. couples model-free RL algorithms with model-based CBF controllers to ensure safety throughout the training process [21]. To overcome the challenges of integrating hard constraints into gradient-based learning, Emam et al. introduced a differentiable Robust Control Barrier Function safety layer embedded within a Soft Actor-Critic framework [22]. Concurrently, recent approaches have demonstrated the effectiveness of incorporating global path structures as privileged information during training to prevent policies from becoming trapped in non-convex obstacles. For instance, Lee et al. utilize Time-of-Arrival (ToA) maps to define explicit velocity setpoint gradients, effectively guiding the policy through highly cluttered spaces and dead ends [23]. While these prior works elegantly embed analytical models into neural networks, we propose a distinct, position-based reward within a PPO framework [24]. By avoiding strict velocity tracking, this hybrid strategy bridges agility and safety, endowing the policy with enhanced robustness and global reachability without overly constraining maneuverability.

3 Methodology↩︎

3.1 Problem Formulation↩︎

We formulate the real-time interaction between the simulated UAV and its complex, dynamic surroundings as a Markov Decision Process (MDP). Formally, the MDP is defined as the tuple \((\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R}, \gamma)\), where \(\mathcal{S}\) denotes the state space, \(\mathcal{A}\) the action space, \(\mathcal{P}: \mathcal{S} \times \mathcal{A} \times \mathcal{S} \rightarrow [0, 1]\) the state-transition probability, \(\mathcal{R}: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}\) the reward function, and \(\gamma \in [0, 1)\) the discount factor. The actor-critic network maps directly from \(\mathcal{S}\) to \(\mathcal{A}\) rather than a multi-stage or explicit curriculum learning approach. The simulation step size is set to \(0.01\) s, so both the policy output and the control execution operate at \(100\) Hz. Detailed descriptions of the observation space, action space, and reward design are provided in the subsequent subsections 3.1.1,  3.1.2 and  3.1.3.

3.1.1 Observation Space↩︎

The training loop initiates by acquiring the current observation of the UAV state. The observation input to our policy network is structured as a multi-modal composite vector \(\mathbf{o}_t\), integrating both exteroceptive environmental perception and the proprioceptive state of the UAV. Specifically, the observation is defined as: \[\mathbf{o}_t = \left[ \mathbf{D}_t, \mathbf{v}_t, \mathbf{R}_t, \mathbf{a}_{t-1}^{\pi}, \mathbf{d}^{xy}_t, z^{\text{self}}_{t}, z^{\text{goal}}_{t} \right],\] where the components are detailed as follows:

• \(\mathbf{D}_t \in \mathbb{R}^{100 \times 60}\): the depth image captured by the onboard camera.

• \(\mathbf{v}_t \in \mathbb{R}^3\): the linear velocity vector expressed in the body coordinate frame.

• \(\mathbf{R}_t \in SO(3)\): the rotation matrix representing the current attitude of the UAV.

• \(\mathbf{a}_{t-1}^{\pi}\): the action executed at the previous time step, comprising roll, pitch, yaw, and total thrust. This historical action serves as a reference to ensure control smoothness.

• \(\mathbf{d}^{xy}_{t} = [\Delta x, \Delta y]^\top\): the normalized horizontal position difference relative to the target.

• \(z^{\text{self}}_{t}\) and \(z^{\text{goal}}_{t}\): the current altitude of the UAV and the target altitude, respectively.

3.1.2 Action Space↩︎

The policy network processes the aforementioned observations and outputs a four-dimensional action vector \(\mathbf{a}_t^{\pi} \in \mathbb{R}^4\), defined as: \[\mathbf{a}_t^{\pi} = [f_n, \phi_{ref}, \theta_{ref}, \psi_{ref}]^\top,\] this vector consists of the mass-normalized collective thrust \(f_n\) and the desired attitude angles (roll \(\phi_{ref}\), pitch \(\theta_{ref}\), and yaw \(\psi_{ref}\)). The control commands are updated at a fixed frequency of 100 Hz to guarantee real-time responsiveness. Subsequently, the low-level onboard flight controller solves for the required rotational speeds of individual motors to achieve precise tracking of these high-level commands.

To ensure operational safety, real-world UAVs typically enforce bounded attitude commands to prevent aggressive maneuvers that may lead to loss of control. Accordingly, instead of issuing low-level control inputs such as direct motor speeds, we adopt attitude-level control commands as the policy output. This higher-level control abstraction significantly reduces the sim-to-real gap by aligning the action space in simulation with practical flight controllers used on physical platforms, thereby improving the robustness of sim-to-real transfer [25].

3.1.3 Reward Function Design↩︎

In this work, the overall reward \(r_t\) is defined as a weighted sum of distinct components: \[\begin{align} r_t =\;& \alpha_1 r_{\mathrm{navigation}} + \alpha_2 r_{\mathrm{safety}} + \alpha_3 r_{\mathrm{smooth}} \\[4pt] &+ \alpha_4 r_{\mathrm{collision}} + \alpha_5 r_{\mathrm{success}} \\[4pt] &+ \alpha_6 r_{\mathrm{speed}} + \alpha_7 r_{\mathrm{height}} . \end{align}\] Specifically, the total reward is composed of navigation guidance, safety regulation, control regularization, and terminal signals. The model-based components \(r_{\mathrm{navigation}}\) and \(r_{\mathrm{safety}}\), detailed in Section 3.3, provide global path guidance and safety-aware shaping during training. To ensure executable and stable control behavior, the smoothness term \(r_{\mathrm{smooth}}\) penalizes both large control magnitudes and abrupt changes between consecutive actions, encouraging dynamically feasible and smooth trajectories. Terminal rewards \(r_{\mathrm{collision}}\) and \(r_{\mathrm{success}}\) are applied upon collision events and successful task completion, respectively, providing clear episodic feedback. In addition, auxiliary terms. \(r_{\mathrm{speed}}\) and \(r_{\mathrm{height}}\) softly constrain the UAV’s velocity and altitude within predefined safety limits, preventing unrealistic or unsafe behaviors during exploration. The overall reward is formulated as a weighted sum, where \(\alpha_i\) denotes the coefficient of the \(i\)-th term. These weights are empirically tuned to balance safety, agility, and task completion during training.

3.2 Network Architecture↩︎

We employ the Proximal Policy Optimization (PPO) algorithm within an asymmetric actor-critic framework. This architecture leverages a privileged learning paradigm: while the critic network is trained on noise-free ground truth states to facilitate stable value estimation, the actor network operates solely on noisy observations to ensure robust deployment under real-world uncertainties.

The complete network structure is illustrated in Fig. 1.To effectively process multimodal inputs, the actor utilizes a heterogeneous neural architecture. The visual stream processes a \(100 \times 60\) depth image via a Convolutional Neural Network (CNN) composed of sequential convolutional, activation (ReLU), and pooling layers, extracting a compact visual feature vector \(f_{\text{CNN}}\) that encodes spatial geometry. We directly concatenate this visual embedding with the raw proprioceptive observations, which consist of the quadrotor’s linear velocity, rotation matrix, previous action, and goal-relative displacement. This unified vector is subsequently fed into a Gated Recurrent Unit (GRU). The GRU maintains a temporal hidden state \(h_t\) to aggregate historical context, enabling the policy to mitigate partial observability and smooth sensor noise over time. Finally, the GRU output is passed to a Multi-Layer Perceptron (MLP) head, which predicts the Gaussian distribution parameters for the continuous action command \(\mathbf{a}_t\).

3.3 Model-based Reward Shaping↩︎

We begin by formulating the navigation reward \(r_{\mathrm{navigation}}\). Conventional distance-based rewards typically rely on Euclidean distance to shape the policy. Although this offers dense feedback, it often leads to convergence failures in non-convex environments. The agent, driven by the greedy minimization of Euclidean distance, is prone to the local minima trap, failing to navigate around complex obstacles. To overcome this, we propose a geodesic-guided reward mechanism. Instead of Euclidean distance, we utilize a precomputed cost map derived from Dijkstra algorithm, which represents the true shortest-path distance considering obstacles. This map serves as a dense potential field during training. By applying trilinear interpolation, we extract continuous cost values \(\mathbf{V}_t\) at the real-time coordinates of the UAV. The reward is then formulated based on the progress \(\boldsymbol{\delta}\) along this potential field. This approach provides the agent with global geometric awareness, effectively decomposing the long-horizon task into accessible local subgoals and guiding the UAV out of local minima.

Let \(\Phi_g\) denote the precomputed geodesic distance field (via Dijkstra) for a given scene \(g\). The navigation reward \(r_{\mathrm{navigation}, t}\) at time step \(t\) is defined as the clipped progress made along this potential field: \[{!}{ r_{\mathrm{navigation}, t} = \lambda \cdot \text{clamp}( \underbrace{\text{Interp}(\Phi_g, \mathbf{p}_{t-1})}_{\mathbf{V}_{t-1}} - \underbrace{\text{Interp}(\Phi_g, \mathbf{p}_t)}_{\mathbf{V}_t}, -C, C ), }\] where \(\text{Interp}(\cdot)\) represents the trilinear interpolation function that maps the continuous UAV position \(\mathbf{p}\) to the discrete Dijkstra cost grid, \(\lambda\) is a scaling factor, and the \(\text{clamp}\) function ensures numerical stability by bounding the reward within \([-C, C]\).

To enhance flight safety, we incorporate a shaping reward based on Control Barrier Function [21], [26], [27]. We first construct a ESDF from the obstacle map, where \(d(\mathbf{x})\) denotes the minimum distance from position \(\mathbf{x}\) to the nearest obstacle. The barrier function is defined as \(b(\mathbf{x}) = d(\mathbf{x}) - d_{\text{safe}}\), where \(d_{\text{safe}}\) is a prescribed safety margin. According to CBF theory, safety is guaranteed if the forward-invariance condition \(\dot{b}(\mathbf{x}) + \gamma b(\mathbf{x}) \ge 0\) is satisfied. Instead of enforcing this constraint explicitly, we encode the violation of this condition into the reward function. Specifically, at each timestep \(t\), we compute the time derivative of the barrier function as \(\dot{b}(\mathbf{x}_t) = \nabla d(\mathbf{x}_t) \cdot \mathbf{v}_t\), utilizing the gradient of the ESDF and the current velocity vector of the UAV. The safety reward \(r_{\text{safety}}\) is then formulated as: \[r_{\text{safety}} = \text{clip} \left( \dot{b}(\mathbf{x}_t) + \gamma b(\mathbf{x}_t) , \delta_{\text{min}} , 0 \right ),\] where \(\gamma\) is the CBF coefficient. We impose a clipping threshold \(\delta_{\text{min}} = -2.0\) on the safety reward. This lower bound prevents unbounded penalties from destabilizing the value function approximation and causing gradient explosion, thereby ensuring numerical stability during training. This mechanism encourages the UAV to proactively align its velocity with the ESDF gradient to satisfy the barrier condition, effectively anticipating and avoiding collision risks.

Although we utilize privileged map information to construct \(r_{navigation}\) and \(r_{safety}\) to enhance the policy’s navigation capability and safety, both of these rewards are strictly used only during the training phase and not during deployment.

3.4 HOCBF-based Correction Loop↩︎

Since reward shaping only acts as a soft incentive rather than a hard constraint, the agent can still suffer severe failures. To provide theoretical safety guarantees during real-world deployment, we integrate a safety filter based on High-Order Control Barrier Functions (HOCBF) [28]. This optimization layer minimally deviates from the raw reference acceleration \(\mathbf{a}_{\text{raw}}\)which can be derived from the policy output \(\mathbf{a}_t^\pi\) to strictly satisfy the safety constraints. We formulate this as a Quadratic Program (QP): \[\mathbf{a}^* = \underset{\mathbf{a}} {\arg\min}\;\frac{1}{2} | \mathbf{a} - \mathbf{a}_\text{raw} |^2 \quad \text{s.t.} \quad \mathcal{C}(\mathbf{r}_t, \mathbf{a}) \geq 0 .\] For an obstacle \(i\), we define the candidate barrier function \(B_i(\mathbf{r}_t) = \| \mathbf{r}_t \|^2 - r_{safe}^2\), where \(\mathbf{r}_t\) denotes the relative position vector extending from the UAV to the nearest obstacle point identified within the local point cloud reconstructed from real-time depth observations and \(r_{safe}\) is the safety radius. Considering the second-order nature of quadrotor dynamics, we employ the HOCBF formulation to ensure the forward invariance of the safe set. The associated safety constraint is defined as: \[\ddot{B}_i(\mathbf{r}_t) + \alpha_1\dot{B}_i(\mathbf{r}_t) + \alpha_0 B_i(\mathbf{r}_t) \geq0,\] where \(\alpha_0, \alpha_1 > 0\) are adjustable coefficients. In practice, these coefficients are empirically tuned to balance the trade-off between strict safety enforcement and trajectory reachability. Larger coefficients introduce excessive conservatism, while smaller coefficients lead to insufficient optimization effects. By substituting the time derivatives \(\dot{B}_i(\boldsymbol{r}_t) = 2 \mathbf{r}_t \cdot \mathbf{v}_t\) and \(\ddot{B}_i(\boldsymbol{r}_t) = 2 \| \mathbf{v}_t \|^2 + 2 \mathbf{r}_t \cdot \mathbf{a}_t\), we derive a linear inequality constraint on the control action \(\mathbf{a}\):

\[\underbrace{2 \mathbf{r}_t^{\top}}_{\mathbf{A}_{\mathrm{cbf}}} \mathbf{a}_t \ge \underbrace{-2\|\mathbf{v}_t\|^2 - \alpha_1 \dot{B}_i(\mathbf{r}_t) - \alpha_0 B_i(\mathbf{r}_t)}_{b_{\mathrm{cbf}}}.\] Here, \(\mathbf{A}_{\text{cbf}}\) encapsulates the gradient direction for collision avoidance, while \(b_{\text{cbf}}\) quantifies the minimum required control effort to counteract the inertia of the system and ensure boundary invariance. This linear constraint allows the QP solver to efficiently compute safe control commands \(\mathbf{a}^*\) in real-time, effectively correcting unsafe maneuvers while preserving the intended flight trajectory. We evaluate the computational overhead of the HOCBF filter, and the result shows that the average optimization time for a single HOCBF execution is merely \(90.61\,\mu\mathrm{s}\), with the maximum optimization time strictly bounded below \(384.5\,\mu\mathrm{s}\). This ensures that the filter does not introduce any noticeable control latency during actual deployment.

4 Experiments↩︎

To validate the efficacy of the proposed framework, we first evaluate training performance across various reward configurations. Subsequently, ablation studies are conducted by deploying the policy in diverse, unseen environments to assess its impact on success rates. Furthermore, we benchmark the proposed method against state-of-the-art approaches, including the planning-based Ego-Planner [14] and the learning-based NavRL [18], YOPO [29], and DiffPhys [11], demonstrating that our method consistently maintains the target velocity setpoint while ensuring rigorous safety. Real-world feasibility is verified through indoor experiments with varying obstacle configurations, compared against Ego-PlannerV2 [5]. Finally, we demonstrate the high-speed navigation capability of the framework in a cluttered outdoor forest, achieving agile flight at speeds up to 7.5 m/s.

4.1 Simulations↩︎

4.1.1 Training Results↩︎

Our framework is trained in Isaac Lab using large-scale parallel simulation, deploying 1,000 quadrotors simultaneously across 16 procedurally generated scenes of increasing difficulty. Each scene contains randomly placed trees and cylinders, with obstacle density gradually rising alongside the difficulty level. At the beginning of each episode, the initial position of the drone is uniformly randomized within the free space located before the obstacle field. An episode is considered successful if the drone reaches the goal region, defined as being within a 5-meter radius of the target position.

To investigate the impact of reward shaping on learning efficiency and navigation performance, we compare three reward formulations: (i) Distance, which utilizes the Euclidean distance to the target as the navigation reward; (ii) Dijkstra, which replaces the Euclidean metric with a Dijkstra-based potential value for global guidance; and (iii) Dijkstra + CBF, which augments the Dijkstra reward with an additional CBF-based safety term to penalize proximity to obstacles. The learning curves in Fig. [2][a](#fig:training) demonstrate that, under identical training conditions, the Dijkstra-based navigation reward substantially accelerates policy learning and achieves a consistently higher training success rate than the Distance baseline. Moreover, incorporating the CBF-based safety reward encourages safer exploration and yields further improvements in final performance, achieving the highest success rate among all compared variants.

4.1.2 Ablation Studies↩︎

We assess the individual contributions of the Dijkstra-based navigation reward, the CBF-based safety reward, and the HOCBF safety filter through ablation experiments across eight unseen cluttered geometric scenes with dimensions of \(64\times32\times3\) m. Unlike the training scenes, these scenes are composed of randomly placed inclined or vertical cylinders, frame structures, and thin poles that induce highly non-convex traversability (Fig. [3][b](#fig:benchmark_traj)), while the deployed policy relies solely on its standard onboard observations without access to the privileged map information used during training for reward construction. We execute 50 independent trials per scene, where the start and goal positions are randomly sampled within rectangular regions of size \(1\,\mathrm{m}\times32\,\mathrm{m}\times2\,\mathrm{m}\) at the starting line and the end of the environment, respectively, with the results summarized in Tab. 1. Overall, the results strongly validate our hierarchical design: global guidance ensures reachability, while the hybrid safety mechanism (soft shaping + hard filter) guarantees robustness across the entire speed envelope.

Global guidance for reachability. The Euclidean-distance baseline suffers rapid degradation at higher velocities due to its myopic nature, frequently leading to failure in complex clutter. In contrast, integrating the Dijkstra-based potential field significantly enhances reachability by encoding global topological information, which provides consistent gradients to steer the agent away from local minima (e.g., U-shaped traps). This strategy effectively distills long-horizon planning capabilities into the reactive policy without incurring online search latency. Notably, the slightly better performance at \(5\,\mathrm{m/s}\) than at \(3\,\mathrm{m/s}\) can be interpreted as a constraint-induced reachability phenomenon. With a stricter speed cap, the vehicle makes less forward progress per decision step and remains longer in the vicinity of non-convex obstacle structures, increasing its exposure to local trapping configurations. Moreover, because the nominal action is smaller in magnitude in the low-speed regime, the CBF-based correction constitutes a larger relative modification and can suppress the forward-progress component of the command. Consequently, failures at \(3\,\mathrm{m/s}\) are more likely to arise from insufficient task progress and local stagnation than from loss of safety.

Necessity of the hybrid safety architecture. Augmenting the policy with a CBF-based safety reward enhances performance in low-to-medium speed regimes. To isolate this contribution, we evaluated our formulation against a Geometric Inflation baseline across 160 trials in 16 unseen scenes at \(5\,\mathrm{m/s}\), which enforced a doubled static safety margin of \(0.4\,\mathrm{m}\) via terminal collision penalties. The inflation approach underperformed our CBF-guided policy, yielding both a lower success rate (\(87.08\%\) vs. \(92.50\%\)) and a reduced average trajectory clearance (\(0.89\,\mathrm{m}\) vs. \(0.92\,\mathrm{m}\) ESDF). Simply enlarging static margins artificially obstructs narrow corridors, whereas our continuous CBF formulation provides velocity-aware, physics-grounded gradients that dynamically enhance safety without sacrificing reachability. However, under high-dynamic conditions (\(7\,\mathrm{m/s}\) and \(9\,\mathrm{m/s}\)), soft reward shaping exhibits a slight decline at high velocities, significant system inertia renders safety margins extremely sensitive to control inputs, making it challenging for the RL agent to approximate stiff constraints solely via reward feedback. This limitation underscores the critical necessity of our HOCBF safety filter. Acting as a deterministic safeguard during deployment, the filter solves a real-time QP to project raw commands onto a provably safe set. In complex simulation tests, the HOCBF filter was triggered in \(13.06\%\) of the total control steps, successfully compensating for the neural policy’s approximation errors in critical boundary cases. Consequently, the full framework effectively bridges the gap between learning-based agility and model-based safety.

0pt

0pt

4.1.3 Benchmarks↩︎

We benchmark our proposed framework against four representative baselines: the planning-based Ego-Planner [14] and the learning-based NavRL [18], YOPO [29], and DiffPhys [11]. Each method was evaluated with its native observation design under the same target-speed budget. Experiments were conducted on 16 unseen benchmark scenes, all with the same dimensions as those used in Sec. 4.1.2 and excluded from training (Fig. [3][b](#fig:benchmark_traj)), categorized into two types: (1) 12 cluttered geometric scenes composed of randomly distributed primitives that induce highly non-convex traversability, and (2) 4 procedurally generated obstacle fields based on Perlin noise. Across these two scene types, scene difficulty increased with obstacle density, as reflected by the median free-space ESDF, which decreased from \(1.24\,\mathrm{m}\) to \(0.49\,\mathrm{m}\). For each method and target speed, we conducted 160 independent trials in total, and a run was counted as successful only if the UAV completed the scene without collision and finished within \(5\,\mathrm{m}\) of the goal. Representative success-rate results are reported in Tab. 2.

As detailed in Tab. 2, Ego-Planner [14] remains viable only at the lowest target speed and fails almost completely beyond \(3\,\mathrm{m/s}\), reflecting the latency bottleneck of iterative mapping and replanning in dense clutter. Among the learning-based baselines, NavRL [18] degrades most sharply with speed, dropping from \(43.12\%\) at \(3\,\mathrm{m/s}\) to nearly zero thereafter. This behavior is likely attributable to the mismatch between its relatively regularized training scenes and the highly cluttered benchmark environments, together with its static-obstacle input design, which limits robustness as the reaction time shrinks at higher speed. YOPO [29] is more stable than NavRL and can track the target speed reasonably well, but its depth-conditioned selection over a finite set of short-horizon motion primitives remains vulnerable in highly non-convex clutter, where a locally favorable primitive can still steer the vehicle into dead ends or collisions. DiffPhys [11] achieves higher success rates than NavRL and YOPO, but its realized speed saturates near \(3\,\mathrm{m/s}\) (Fig. [3][a](#fig:benchmark_traj)). This limited high-speed scalability likely stems from tying the camera heading to commanded motion. During aggressive maneuvers, large pitch excursions shift the camera view towards the ground or sky, severely reducing forward obstacle visibility. Consequently, the policy adopts conservative behavior, trading speed for perceptual stability and failing at higher target speeds.

Conversely, our method achieves the optimal safety-agility trade-off, maintaining the highest success rates and realized velocities across all speeds and configurations (Fig. [3][a](#fig:benchmark_traj)). Specifically, the success rate remains at \(96.88\%\), \(96.25\%\), \(73.75\%\), and \(56.88\%\) from \(3\) to \(9\,\mathrm{m/s}\), with the average successful speed increasing from \(3.00\) to \(7.58\,\mathrm{m/s}\). These results validate the advantage of combining low-latency learning with model-based safety: the learned policy preserves aggressive speed tracking, while the safety layer enforces reliable near-obstacle behavior and improves robustness as the environment and target speed become increasingly challenging.

4.2 Real-World Experiments↩︎

To further validate the proposed framework, we deployed the trained policy on a physical quadrotor and conducted extensive experiments in both cluttered indoor environments and unstructured outdoor forests.

We construct a cluttered indoor environment, as shown in Fig. 4. The policy is evaluated using networks trained with target speed constraints of \(3 \text{ m/s}\), \(5 \text{ m/s}\), and \(7 \text{ m/s}\). To assess generalization capabilities, the quadrotor is tasked with navigating back and forth through a \(15 \text{ m}\) long obstacle course, with layouts randomly reconfigured before each flight.

Our policy achieves exceptionally high success rates in obstacle avoidance and navigation across diverse speeds and configurations. Compared to the traditional Ego-PlannerV2 [5], it demonstrates superior flight stability and maximum achievable velocity. As shown in Fig. 4 (where red circles denote collisions), Ego-PlannerV2 [5] failures increase significantly at higher target speeds. In contrast, our method consistently achieves successful navigation while maintaining an actual velocity that closely tracks the target setpoint for the majority of the trajectory.

To provide a more principled evaluation of real-world scene complexity, we further quantify the difficulty of the indoor test environments using the traversability metric following [30]. Four indoor environments with increasing clutter levels are constructed, and the policy is evaluated under three target-speed settings, namely 3 m/s, 5 m/s, and 7 m/s. For each combination of environment complexity and target speed, 10 independent real-world trials are conducted.

As shown in Fig. 5, the proposed method achieves a 100% success rate for all target speeds in the two lower-difficulty environments. In more cluttered scenes, the policy remains robust: the 5 m/s setting fails only once in the most difficult environment, while the 7 m/s setting achieves success rates of 90% and 80% in the two most challenging environments, respectively. The corresponding velocity statistics further show that the realized speed remains close to the target speed and does not exhibit a significant decrease as the environment complexity increases. These results indicate that the proposed policy maintains both safety and agility under increasing clutter, rather than relying on conservative speed reduction.

In outdoor forest scenarios (Fig. 6), we employ a policy trained with a setup modified only by reducing obstacle density and increasing the target speed limit compared to the original training configuration. The quadrotor successfully achieves stable, high-speed, long-distance obstacle avoidance, maintaining an average velocity of \(7 \text{ m/s}\) over a distance exceeding \(35 \text{ m}\) to reach the target.

Furthermore, we evaluated our policy under dynamic conditions in simulation and the real world. Despite being trained solely in static environments, the UAV successfully generalized to unknown dynamic scenarios, achieving a \(64\%\) success rate across 100 simulated trials with dense moving obstacles, and completing all 10 real-world flight tests without collisions against an obstacle moving at \(0.5 \text{ m/s}\). The failures primarily occur when moving obstacles approach from the sensor’s blind spots, rendering the local perception horizon insufficient for successful reactive maneuvers.

5 Conclusion↩︎

In this paper, we present a hybrid reinforcement learning framework integrating model-free policy optimization with model-based physical priors for high-speed quadrotor obstacle avoidance. The priors accelerate training convergence via geometric guidance and ensure feasible control deployment through real-time filtering. Currently, hardware limitations restrict real-world flight speeds from reaching the policy’s maximum simulated capabilities. Future work will focus on enhancing quadrotor hardware performance and extending this method to specific, task-oriented scenarios.

References↩︎