1 Introduction↩︎

Space-Air-Ground Integrated Networks (SAGINs) are increasingly recognized as a promising architecture for latency-sensitive IoT services, by integrating terrestrial devices, UAV edge nodes, and high-altitude platform stations (HAPS) into a unified computing–communication system [1]–[3]. With this multi-tier structure, tasks can be flexibly offloaded across heterogeneous nodes, improving service coverage and resilience. However, achieving efficient and delay-aware scheduling in SAGIN remains a challenging problem due to the tight coupling between task association, wireless transmission, and distributed computation under dynamic queue backlogs, and heterogeneous resource constraints.

Recent studies have built strong foundations for SAGIN-enabled communication and computing. Existing works cover global transmission optimization and wide-area data collection [4], hierarchical HAPS–UAV aerial computing [3], multi-agent offloading in aerial-edge systems [1], and multi-HAP-assisted offloading [2]. Other efforts investigate joint offloading/association/resource allocation [5], perception-aware offloading [6], and task-prediction-driven edge offloading [7]. At the communication and control layers, prior work also addresses UAV-assisted robustness [8], hybrid FSO/RF coverage and deployment [9], channel modeling and estimation [10], RIS-enabled multifunctional architectures shenMultifunctionalRIsEnabledSAGIN2026?, outage/fairness analysis [11], blockchain-supported trusted offloading [12], and game-theoretic resource distribution [13]. Despite these advances, three limitations remain from a delay-aware scheduling perspective. First, many methods rely on macro-scale delay abstractions (e.g., average-rate or static-capacity assumptions), which miss fine-grained service dynamics. Second, queue evolution is often modeled coarsely, weakening cross-interval carry-over under bursty traffic arrivals. Third, while joint optimization formulations can be theoretically appealing, they are often difficult to deploy practically due to mixed discrete-continuous coupling and high computational complexity.

To address these challenges, we formulate the delay-aware task offloading and resource scheduling problem in a three-tier SAGIN as a non-convex MINLP. We then propose HALO, a hierarchical auction-assisted learning framework that combines auction-based task association with HPPO-based resource allocation, supported by a macro–micro slot design for fine-grained system dynamics. The main contributions of this work are summarized as follows:

• We propose a macro–micro slot scheduling framework for three-tier SAGIN that models transmission and computation dynamics, enabling accurate latency characterization and consistent queue evolution.

• We design a UAV-centric auction for task association, combining latency-aware utility and resource-aware bidding to enable efficient and adaptive task–UAV matching under dynamic network conditions.

• We propose HALO, a two-stage hierarchical learning framework that combines auction-based task association with HPPO-based resource allocation, enabling effective coordination between discrete association decisions and continuous control across multiple timescales.

We evaluate HALO against single-layer DRL baselines, including PPO, DDPG, and SAC. Simulation results, averaged over multiple traffic loads, demonstrate consistent performance gains in task success rate, with an average improvement of 8.70 percentage points (+11.4% relative) over PPO.

The remainder of this paper is organized as follows. Section II presents the system model and problem formulation. Section III introduces the proposed HALO framework. Section IV provides simulation results under varying traffic conditions. Section V concludes the paper.

2 System Model and Problem Formulation↩︎

As illustrated in Fig. 1, we consider a three-tier SAGIN with ground IoT devices \(\mathcal{N}=\{1,\dots,N\}\), UAVs \(\mathcal{M}=\{1,\dots,M\}\) as aerial-edge nodes, and one HAPS as the high-altitude cloud tier. Each IoT device generates delay-sensitive tasks that may be processed locally, offloaded to an associated UAV, or further relayed to the HAPS. UAVs provide aerial edge computing and relay services, while the HAPS offers wider-area cloud-level computation. In the following, we first describe the task arrival and dynamic queueing model under a mixed macro–micro timescale framework. We then present the channel, communication, and computation models, define the resulting end-to-end task delay, and finally formulate the joint task offloading and resource scheduling problem.

2.1 Task Arrival and Dynamic Queueing Model↩︎

We adopt a mixed-timescale framework: macro-slots capture Poisson task arrivals with rate \(\lambda\), while micro-slots enable fine-grained resource scheduling. At each macro-slot, a task generated by IoT device \(i\) is defined as: \[\phi_i \triangleq (D_i, C_i, T_i^{\max}, t_i^{arr}),\] where \(D_i\) denotes the data size, \(C_i\) is the computation intensity, \(T_i^{\max}\) is latency constraint, and \(t_i^{arr}\) is the arrival time. Each task is partitioned into local, UAV, and HAPS components with ratios \(\alpha_i, \beta_i, \gamma_i\), respectively, satisfying \(\alpha_i + \beta_i + \gamma_i = 1\). The task partitioning variables are determined at macro slots by the Manager agent (detailed in Section 3). Whereas resource allocation is performed dynamically at each micro-slot rather than reserved over the entire task lifetime. Urgency-aware scheduling (e.g., based on remaining slack time) is applied to prioritize tasks. Completed transmission and computation stages are immediately removed, freeing resources for reuse and efficient operation under stochastic traffic while maintaining consistent task progress across micro-scheduling intervals.

2.2 Channel Model↩︎

2.2.1 IoT-to-UAV Link ( Air-to-Ground (A2G) Channel)↩︎

For IoT-UAV access links, blockage is modeled via probabilistic LoS/NLoS A2G propagation. The LoS probability between IoT \(i\) and UAV \(m\) is: \[P_{LoS}(\theta_{i,m}) = {1}/{[1 + a \exp \left( -b (\theta_{i,m} - a) \right)]},\] where \(\theta_{i,m} = \frac{180}{\pi}\arcsin\!\left(\frac{H_m}{d_{i,m}}\right)\), \(H_m\) is UAV altitude, \(d_{i,m}\) is 3D distance between Iot device \(i\) and UAV \(m\), and \(a,b\) are environment parameters. Then \(P_{NLoS}=1-P_{LoS}(\theta_{i,m})\). Average path loss (dB) is: \[\bar{L}_{i,m} = P_{LoS}(\theta_{i,m})\!\left(L_{FS}\right) +P_{NLoS}\!\left(L_{FS}+\eta_{NLoS}\right),\] with \(L_{FS}=20\log_{10}\!\left({(4\pi f_c d_{i,m})}/{c}\right)\), where \(\eta_{NLoS}\), \(f_c\), and \(c\) denote excess loss in NLoS link, carrier frequency and \(C\) speed of light, respectively. The corresponding channel gain is \[h_{i,m}=10^{-\bar{L}_{i,m}/10}.\]

2.2.2 UAV-to-HAPS Link (Air-to-Air(A2A) Channel)↩︎

For UAV-HAPS backhaul, high-altitude propagation is LoS-dominant. The corresponding channel gain is: \[h_{m,H}=\rho_0 d_{m,H}^{-\alpha_{AA}},\] where \(\rho_0=(c/4\pi f_c)^2\) is the 1 m reference gain, \(d_{m,H}\) is UAV-HAPS distance, and \(\alpha_{AA}\approx2\) is path loss exponent.

2.3 Communication Model↩︎

Let \(T\) be macro-slot duration (index \(t\in\{1,2,\dots\}\)), and let each macro-slot contain \(K\) micro-slots of duration \(\Delta t\) (index \(k\in\{1,2,\dots,K\}\)) such that \(T = K\Delta t\).

2.3.1 IoT-to-UAV Uplink (Access Link)↩︎

If task \(i\) uses bandwidth fraction \(w_i^{U}[t,k]\) and power \(p_i^{IoT}[t,k]\) at micro-slot \(k\), its uplink rate to UAV \(m\) is: \[\label{eq:rate95uav} \begin{align} R_{i,m}^{U}[t,k] &= w_i^{U}[t,k]B^{U}\\ &\quad\times \log_2\!\left(1+\frac{p_i^{IoT}[t,k]h_{i,m}}{N_o w_i^{U}[t,k]B^{U}+I_{i,m}[t,k]}\right), \end{align}\tag{1}\] where \(B^{U}\) is the total bandwidth available at each UAV, \(N_o\) is the AWGN power spectral density (PSD) and \(I_{i,m}[t,k]\) denotes inter-cell interference (ICI) on task \(i\).

Intra-cell FDMA eliminates intra-cell interference; only ICI is considered. For victim task \(i\) (UAV \(m\)) and interferer \(j\) (UAV \(v\neq m\)), define \(B_i[t,k]=w_i^{U}[t,k]B^{U}\), and \(B_j[t,k]=w_j^{U}[t,k]B^{U}\). The interference PSD from \(j\) at UAV \(m\) is \[\label{eq:PSD} \rho_{j,m}[t,k]=\frac{P_{j,m}^{rx}[t,k]}{B_j[t,k]}=\frac{p_j^{IoT}[t,k]h_{j,m}}{w_j^{U}[t,k]B^{U}},\tag{2}\] where \(h_{j,m}=\rho_o d_{j,m}^{-2}\). With independent FDMA across cells, expected overlap bandwidth is \[\label{eq:Overlap} \mathbb{E}\!\left[B_{i,j}^{overlap}[t,k]\right]=\big(w_i^{U}[t,k]w_j^{U}[t,k]\big)B^{U}.\tag{3}\] Using Eq. 2 and Eq. 3 , the ICI from task \(j\) to task \(i\) is: \[\begin{align} I_{j\to i}[t,k] &=\rho_{j,m}[t,k]\cdot \mathbb{E}\!\left[B_{i,j}^{overlap}[t,k]\right]\\ &=w_i^{U}[t,k] p_j^{IoT}[t,k]h_{j,m}. \end{align}\] Thus, the total interference on task \(i\) is \[\label{eq:interference} I_{i,m}[t,k]=w_i^{U}[t,k]\sum_{v\in\mathcal{M}\setminus\{m\}}\sum_{j\in\mathcal{N}_v[t,k]} p_j^{IoT}[t,k]h_{j,m},\tag{4}\] where \(\mathcal{N}_v[t,k]\) is the active tasks set associated with UAV \(v\), determined by the auction mechanism described in Section 3. For offloaded data \((\beta_i+\gamma_i)D_i\), let the generation time coordinate be \((t_i^{arr},0)\) and the uplink completion time coordinates be \((t_i^{fin,uav,tx},k_i^{fin,uav,tx})\). The transmission duration is: \[\small T_{i,tx}^{UAV} = (t_i^{fin,uav,tx}-t_i^{arr}) T + k_i^{fin,uav,tx}\Delta t.\] By aggregating the uplink rate over all active micro-slots within this interval, the completion condition is succinctly given by: \[\small \sum_{t=t_i^{arr}}^{t_i^{fin,uav,tx}} \sum_{k} R_{i,m}^{U}[t,k]\Delta t \ge (\beta_i+\gamma_i)D_i,\] where the inner sum runs up to \(K\) for intermediate macro-slots, and up to \(k_i^{fin,uav,tx}\) in the final macro-slot.

2.3.2 UAV-to-HAPS Uplink (Backhaul Link)↩︎

At micro-slot \(k\), UAV \(m\) allocates \(w_i^{H}[t,k]\) and \(p_{i,m}^{UAV}[t,k]\) for relayed task \(i\). Neglecting inter-UAV backhaul interference, the rate is: \[\small R_{i,m}^{H}[t,k] = w_i^{H}[t,k]B^{H} \times \log_2\left(1+\frac{p_{i,m}^{UAV}[t,k]h_{m,H}}{N_0 w_i^{H}[t,k]B^{H}}\right),\] where \(B^H\) denotes the total bandwidth available at HAPS. For the relayed data \(\gamma_iD_i\), backhaul starts and ends at \((t_i^{fin,uav,tx},k_i^{fin,uav,tx})\) and \((t_i^{fin,haps,tx},k_i^{fin,haps,tx})\), respectively. The total backhaul duration \(T_{i,tx}^{HAPS}\) is calculated as the exact time elapsed between these two boundary coordinates. The capacity constraint over this period is: \[\label{eq:bh95constraint}\small \sum_{t=t_i^{fin,uav,tx}}^{t_i^{fin,haps,tx}} \sum_{k} R_{i,m}^{H}[t,k]\Delta t \ge \gamma_iD_i.\tag{5}\]

2.4 Computation Model↩︎

CPU frequencies are scheduled at the micro-slot level. The total computation workload of task \(i\) is \(D_iC_i\), which is split into local, UAV, and HAPS parts according to \(\alpha_i, \beta_i\), and \(\gamma_i\). Similar to the transmission model, the exact computation duration for each tier is calculated as the time elapsed between its respective start and end coordinates.

2.4.1 Local IoT Computing↩︎

For the local portion, processing starts at \((t_i^{arr},0)\) and ends at \((t_i^{fin,loc,comp},k_i^{fin,loc,comp})\), yielding the local computation duration \(T_{i,comp}^{loc}\). By aggregating the local CPU frequency \(f_i^{loc}[t,k]\) over this period, the processed workload constraint is: \[\small \sum_{t=t_i^{arr}}^{t_i^{fin,loc,comp}} \sum_{k} f_i^{loc}[t,k]\Delta t \ge \alpha_iD_iC_i.\]

2.4.2 UAV Edge Computing↩︎

Edge processing can only begin after the A2G transmission is completed. It starts at \((t_i^{fin,uav,tx},k_i^{fin,uav,tx})\) and ends at \((t_i^{fin,uav,comp},k_i^{fin,uav,comp})\), taking a duration of \(T_{i,comp}^{UAV}\). The allocated UAV CPU frequency \(f_i^{U}[t,k]\) must satisfy: \[\small \sum_{t=t_i^{fin,uav,tx}}^{t_i^{fin,uav,comp}} \sum_{k} f_i^{U}[t,k]\Delta t \ge \beta_iD_iC_i.\]

2.4.3 HAPS Cloud Computing↩︎

Similarly, HAPS processing starts after the A2A backhaul is completed and ends at \((t_i^{fin,haps,tx},k_i^{fin,haps,tx})\) and \((t_i^{fin,haps,comp},k_i^{fin,haps,comp})\), respectively, with a duration of \(T_{i,comp}^{HAPS}\). The required cloud workload constraint is: \[\small \sum_{t=t_i^{fin,haps,tx}}^{t_i^{fin,haps,comp}} \sum_{k} f_i^{H}[t,k]\Delta t \ge \gamma_iD_iC_i.\]

2.5 Total Delay Definition↩︎

Under the partial offloading scheme, the local, edge, and cloud computation branches are executed in parallel. Queueing delays are implicitly captured by the boundary coordinates of the scheduled micro-slots. The edge and cloud branches share the initial A2G transmission phase \(T_{i,tx}^{UAV}\) and then diverge. The overall task makespan is defined as the maximum latency among the three branches: \[T_i^{total}=\max\big\{T_i^{Local},\;T_i^{Edge},\;T_i^{Cloud}\big\},\] where the latency of each individual branch is given by: \(T_i^{Local}= T_{i,comp}^{loc}\), \(T_i^{Edge} = T_{i,tx}^{UAV}+T_{i,comp}^{UAV}\), and \(T_i^{Cloud} = T_{i,tx}^{UAV}+T_{i,tx}^{HAPS}+T_{i,comp}^{HAPS}\).

2.6 Problem Formulation↩︎

Define \(\mathbf{A}=\{\alpha_i,\beta_i,\gamma_i\}\), \(\mathbf{F}=\{f_i^{loc}[t,k],f_i^{U}[t,k],f_i^{H}[t,k]\}\), \(\mathbf{W}=\{w_i^U[t,k],w_i^H[t,k]\}\), \(\mathbf{P}=\{p_i^{IOT}[t,k],p_{i,m}^{UAV}[t,k]\}\). The optimization problem is: \[\tag{6} \begin{align} &\max_{\mathbf{A},\mathbf{W},\mathbf{P},\mathbf{F}}\;\frac{1}{|\mathcal{N}_{all}|}\sum_{i\in\mathcal{N}_{all}}\mathbb{I}\!\left(T_i^{total}\le T_i^{\max}\right)\tag{7}\\ &\text{s.t.}\quad\alpha_i+\beta_i+\gamma_i=1,\;\alpha_i,\beta_i,\gamma_i\in[0,1],\;\forall i,\tag{8}\\ &\quad\sum_{i\in\mathcal{N}_m^{tx}[t,k]}w_i^U[t,k]\le1,\;\forall m,t,k,\tag{9}\\ &\quad\sum_{i\in\mathcal{M}_H^{tx}[t,k]}w_i^H[t,k]\le1,\;\forall t,k,\tag{10}\\ &\quad p_i^{IoT}[t,k]\le P_{\max}^{IoT},\;\forall i,t,k,\tag{11}\\ &\quad\sum_{i}p_{i,m}^{UAV}[t,k]\le P_{\max}^{UAV},\;\forall m,t,k,\tag{12}\\ &\quad f_i^{loc}[t,k]\le F_{\max}^{IoT},\;\forall i,t,k,\tag{13}\\ &\quad\sum_{i}f_i^{U}[t,k]\le F_{\max}^{UAV},\;\forall m,t,k,\tag{14}\\ &\quad\sum_{i}f_i^{H}[t,k]\le F_{\max}^{HAPS},\;\forall t,k.\tag{15} \end{align}\]

Problem 6 is a non-convex MINLP. The task success rate indicator objective and coupling among \(\mathbf{A},\mathbf{W},\mathbf{P},\mathbf{F}\) induce bilinear and quadratic-fractional SINR terms, making the problem NP-hard. In addition, active sets (e.g., \(\mathcal{N}_m[t,k]\)) vary stochastically with arrivals/departures, so static convex methods are not directly applicable.

3 Proposed Solution Framework↩︎

To address the formulated problem, we propose HALO presented in Fig. 2, a hierarchical auction-assisted learning framework for joint task association and resource allocation.

3.1 UAV-Centric Online Auction Association↩︎

From [14] stochastic local search-based auction we proposed a auction algorithm for UAV association. For each arriving task \(i\) defined by the tuple \(\phi_i = (D_i, C_i, T_i^{\max}, t_i^{arr})\), only UAVs within communication coverage are eligible bidders. The bidder set is strictly: \[\mathcal{M}_i=\{m\in\mathcal{M}\mid d_{i,m}\le R_m\},\] where \(R_m\) is the coverage radius of UAV \(m\). UAVs outside coverage are never considered for bidding or assignment.

The task value reflects urgency and is defined as \(v_i=D_i/T_i^{\max}\). The association cost accounts for task size and propagation distance: \(c_{i,m}=D_i\left(1+{d_{i,m}}/{d_0}\right),\quad m\in\mathcal{M}_i\). where \(d_0\) is a reference distance used to normalize the distance effect. Thus, larger tasks and farther UAVs incur higher association cost. We apply batch-wise min–max normalization:

\(\tilde{v}_i=\mathrm{Norm}(v_i)\), \(\tilde{c}_{i,m}=\mathrm{Norm}(c_{i,m})\), and define pre-price utility \(\pi_{i,m}=\tilde{v}_i-\tilde{c}_{i,m}\). Let \(p_i^{(0)}=0\) be the task price. In auction round \(r\) (distinct from the micro-slot index \(k\)), \[U_{i,m}^{(r)}=\pi_{i,m}-p_i^{(r)},\qquad m_i^\star=\arg\max_{m\in\mathcal{B}_i}U_{i,m}^{(r)},\] where \(\mathcal{B}_i\subseteq\mathcal{M}_i\) is the capacity-aware bidder set. Specifically, \(\mathcal{B}_i\) only includes UAVs possessing sufficient unallocated bandwidth and computation resources to satisfy the physical limits \(P_{\max}^{UAV}\) and \(F_{\max}^{UAV}\) established in 12 and 14 . With \(U_i^{(1)}\) and \(U_i^{(2)}\) the best and second-best net utilities, the bid increment and price update are: \[\Delta b_i^{(r)}=(U_i^{(1)}-U_i^{(2)})+\varepsilon,\qquad p_i^{(r+1)}=p_i^{(r)}+\Delta b_i^{(r)}.\] Task \(i\) is assigned to \(m_i^\star\). The set of tasks associated with UAV \(m\) at time \((t,k)\) is denoted by \(\mathcal{N}_m[t,k]\), which forms the active association set used in the communication model. If \(\mathcal{M}_i=\varnothing\), the task remains unassigned.

3.2 Two-Timescale Hierarchical PPO Architecture↩︎

To prevent action-space explosion and non-stationarity caused by differing timescales, we adopt a hierarchical Manager-Worker DRL architecture.

3.2.1 Stable Policy Optimization via PPO↩︎

Both Actor-Critic agents use PPO to ensure stable updates and avoid congestion. Let \(r(\theta)=\frac{\pi_\theta(a|s)}{\pi_{\theta_{\mathrm{old}}}(a|s)}\) be policy ratio. The clipped objective is: \[\small L^{\mathrm{CLIP}}(\theta)=\hat{\mathbb{E}} \left[ \min\!\left( r(\theta)\hat{A},\; \mathrm{clip}(r(\theta),1-\epsilon,1+\epsilon)\hat{A} \right) \right],\] where \(\epsilon\) is the clipping coefficient and \(\hat{A}\) is the GAE-estimated advantage. PPO ensures conservative policy evolution via clipping and cross-timescale variance reduction via GAE.

3.2.2 Macro-Level Manager Agent (Task Splitting)↩︎

At macro-slot \(t\) (e.g., 1 s), the Manager assigns task splitting for arrivals.

• State Space (\(\mathbf{s}^M_t\)): Processing up to \(N_{\max}\) tasks (with zero-padding or truncation), each task \(i\) is encoded as \[\big[\tilde{d}_i,\;\tilde{\ell}_i,\;\tilde{q}^{\mathrm{uav}}_i,\;\tilde{q}^{\mathrm{haps}},\;\tilde{u}_i\big],\] capturing normalized size, distance, UAV/HAPS backlogs, and urgency. Concatenation yields \(\mathbf{s}^M_t \in \mathbb{R}^{5N_{\max}}\).

• Action Space (\(\mathbf{a}^M_t\)): The Manager outputs \(\mathbf{z}^M_t \in \mathbb{R}^{3N_{\max}}\). For each task \(i\), a 3-D slice is Softmax-normalized for local/edge/cloud splitting: \[\mathbf{a}_{i,t}^{M}= \begin{bmatrix} \alpha_i\; \beta_i \;\gamma_i \end{bmatrix}^{\top} =\mathrm{Softmax}\!\left(\mathbf{z}_{i,t}^{M}\right).\]

• Reward Function (\(r_t^M\)): Evaluated after all micro-slots based on on-time (\(N_{\mathrm{succ}}\)) and failed (\(N_{\mathrm{fail}}\)) completions: \[r_t^M=\rho_{\mathrm{succ}}N_{\mathrm{succ}}-\rho_{\mathrm{fail}}N_{\mathrm{fail}}.\]

3.2.3 Micro-Level Worker Agent (Resource Scheduling)↩︎

At micro-slot \(k\) (e.g., 50 ms), the Worker executes fine-grained resource allocation.

• State Space (\(\mathbf{s}^W_{k,j}\)): Each resource queue \(j\) is encoded as a Top-\(K\) vector (sorted by least-slack-time): \(\mathbf{s}^W_{k,j}\in\mathbb{R}^{2K}\), comprising \(K\) pairs of normalized residual workload and urgency \([\tilde{L}_{m},\tilde{U}_{m}]\).

• Action Space (\(\mathbf{a}^W_{k,j}\)): An MLP outputs \(\mathbf{a}^W_{k,j}\in\mathbb{R}^{2K}\). For tx, the segments represent bandwidth and power logits: \[\mathbf{a}_{k,j}^{\mathrm{tx}}= \begin{bmatrix} \mathrm{Softmax}(\mathbf{z}^{B}_{k,j})\; \sigma(\mathbf{z}^{P}_{k,j}) \end{bmatrix}^{\top}.\] To satisfy hardware limits, requested powers sharing a transmitter are rescaled: \[\tilde{p}_{e}[k]= \frac{p^{\mathrm{req}}_{e}[k]}{\max\!\left(1,\sum_{j\in\mathcal{E}^{\mathrm{tx}}_{n}[k]}p^{\mathrm{req}}_{j}[k]\right)}.\] Computation queues apply Softmax directly for CPU-share weights.

• Reward Function (\(r_{t,k}^W\)): The worker adopts a dense micro-step progress reward: \[r_{t,k}^W = \sum_{j\in\mathcal{A}_{\mathrm{tx}}(t,k)} \Delta b_j(t,k) + \sum_{j\in\mathcal{A}_{\mathrm{comp}}(t,k)} \Delta c_j(t,k).\] \[\Delta b_j(t,k)=\min\!\left\{R_j(t,k)\Delta k,\; b_j^{\mathrm{rem}}(t,k)\right\},\] \[\Delta c_j(t,k)=\min\!\left\{f_j(t,k)\Delta k,\; c_j^{\mathrm{rem}}(t,k)\right\}.\]

The training process of the hierarchical PPO framework is summarized in Algorithm 3.

4 Simulation Results↩︎

4.1 Simulation Setup↩︎

We generate the random IOT devices uniformly distributed in a 600m width square, use k-means clustering for the UAVs deployment and set the HAPS position at the center of the square. We evaluate the proposed HPPO framework against three single-layer DRL baselines: PPO, DDPG, and SAC. All methods are tested in same SAGIN environment, with identical physical parameters, task generation process, and action constraints. For fair comparison, all algorithms are trained with same interaction budget and evaluated with multi-seed testing.

4.2 Convergence and Performance Comparison↩︎

All algorithms are trained for 1000 macro-step interactions. Fig. 4 (a) shows the reward evolution at \(\lambda=300\) tasks/min. HPPO exhibits the fastest convergence: its smoothed reward increases sharply in the early stage (first \(\sim\)​200 episodes), while PPO, DDPG, and SAC improve more slowly and with larger oscillations. By the end of training, HPPO stabilizes around a higher reward level (approximately 10.6), compared with PPO (\(\sim\)​9.9), DDPG (\(\sim\)​9.6), and SAC (\(\sim\)​8.6), indicating better sample efficiency and a more stable policy update trajectory. This behavior is consistent with hierarchical macro–micro decomposition, which improves long-horizon credit assignment.

We then evaluate trained policies at \(\lambda\in\{300,350,400\}\) tasks/min over 10 independent 10-minute episodes. As shown in Fig. 4 (b), HPPO consistently achieves the highest success rate, with values of 95%, 85%, and 65% at 300/350/400 tasks/min, respectively. Using relative improvement \(\Delta_{\text{rel}}=(S_{\text{HPPO}}-S_{\text{baseline}})/S_{\text{baseline}}\): HPPO outperforms PPO by about 8.0%, 14.9%, and 13.8% at 300/350/400; it outperforms DDPG by about 18.8%, 41.7%, and 44.4%; and outperforms SAC by about 86.3%, 102.4%, and 80.6%. Averaged over the three loads, HPPO provides roughly +11.4% improvement over PPO, +32.4% over DDPG, and +89.9% over SAC. Moreover, although all methods degrade as traffic increases, HPPO maintains the largest absolute success-rate margin under high load, highlighting better scalability under congestion.

Fig. 4 (c) reports average completion latency under increasing traffic. HPPO remains the lowest-latency method across all loads (approximately 6 s, 9 s, and 19 s), while PPO is around 9 s, 14 s, and 25 s; DDPG around 12 s, 23 s, and 42 s; and SAC around 32 s, 48 s, and 60 s. In terms of latency reduction ratio \(R_{\text{lat}}=(L_{\text{baseline}}-L_{\text{HPPO}})/L_{\text{baseline}}\), HPPO reduces latency by approximately 33.3%/35.7%/24.0% versus PPO, 50.0%/60.9%/54.8% versus DDPG, and 81.3%/81.3%/68.3% versus SAC at 300/350/400 tasks/min. These results jointly show that HPPO not only improves task success under heavy traffic, but also controls queueing delay more effectively, yielding a superior reliability–timeliness trade-off.

5 Conclusion↩︎

In this paper, we proposed HALO, a hierarchical auction-assisted learning framework for delay-aware task offloading and resource scheduling in three-tier SAGIN. By combining a UAV-centric auction mechanism for task association with a two-timescale HPPO-based resource allocation strategy, HALO effectively coordinates discrete and continuous decisions under dynamic network conditions. The proposed macro-micro slot model enables fine-grained tracking of transmission and computation progress, improving latency awareness and system responsiveness. Simulation results demonstrate that HALO consistently outperforms conventional DRL baselines in both task success rate and latency, while maintaining robust performance under increasing traffic loads. These results highlight the effectiveness of hierarchical learning in managing complex SAGIN environments. Future work will explore graph-structured state modeling for larger-scale SAGIN scenarios.

Acknowledgment↩︎

This work is supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC) under the CREATE TRAVERSAL and NSERC DISCOVERY programs, and in part by the Ontario Research fund-Research Excellence (ORF-RE) program under project number RE12-024.

References↩︎