1 Introduction↩︎

The evolution towards next-generation (xG) wireless systems envisions artificial intelligence (AI)-native architectures wherein intelligent, resilient communication protocols autonomously emerge to manage unprecedented network dynamics [1]. Central to this vision is the medium access control (MAC) protocol, which orchestrates channel access among numerous nodes. As network topologies become increasingly varying and heterogeneous, the prevailing paradigm of designing static, human-engineered MAC protocols is rendered obsolete, necessitating protocol emergence solutions that can learn and adapt in real-time [2]. This need is particularly acute in challenging scenarios such as uplink data transmission scheduling (UDTS) within cellular networks [3], [4].

Deep reinforcement learning (DRL) and its multi-agent counterpart (MARL) have become a promising approach in this domain, modeling network nodes – such as base stations (BSs) and user equipments (UEs) – as autonomous agents that learn coordinated control messages and policies through trial-and-error interaction [5]–[8]. Despite substantial potential for enhancing network key performance indicators (KPIs), up-to-date formulations typically resort to fully centralized or fully decentralized control paradigms but neglect the intrinsic game-theoretic relationships among network entities [9], [10]. Meanwhile, the UDTS in cellular networks is inherently characterized by resource competition under centralized coordination. Therefore, compared to alternative cooperative games [11], the Stackelberg game (SG), a hierarchical leader-follower game, can grant the flexibility of UEs under the guidance of the BS. Moreover, it can elegantly capture the underlying dynamics [12] and guide policy convergence in MARL frameworks [13], rendering it a more natural and faithful solution. However, generalization in dynamic network environments remains a critical bottleneck [14], as there commonly emerges a fluctuating number of agents (i.e., attached UEs).

Recently, characterized by impressive comprehension and generative reasoning capabilities, large language models (LLMs) have emerged as a compelling solution to intractable protocol design challenges [15]. Their inherent ability for processing variable-length sequences suggests a natural aptitude for operating in dynamic game environments, overcoming the architectural rigidity of non-LLM methods [10]. Nevertheless, instantiating SG agents with LLMs introduces unique challenges. A fundamental discrepancy arises between the pre-training objective of an LLM – generating plausible text – and the strategic reasoning needed for a leader or follower in an SG, which is to maximize network utility. Addressing this significant gap requires functional alignment of model behavior with strategic goals. Existing LLM-based systems, however, typically depend on meticulously curated, domain-specific datasets or expert prompt engineering for domain adaptation [16]. This dependency constrains their capacity for exploratory learning, the very mechanism required for the emergence of novel, high-performance protocols. Furthermore, bridging the semantic gap between the LLMs’ open-domain vocabulary and the game action space of MAC protocols is essential yet nontrivial. Reconciling generalization, exploration, and structured action generation is therefore critical [17] for achieving stable and efficient policy convergence in such dynamic game settings.

To overcome the limitations above, we propose an LLM-empowered MARL framework that conceptualizes the MAC protocol emergence for UDTS as a dynamic multi-follower Stackelberg game (MFSG). This game is instantiated through LLM-empowered agents, where the BS serves as the leader agent and broadcasts a macroscopic scheduling intent as downlink control messages (DCMs). The UEs, acting as followers, interpret this intent in the context of their local observations and articulate their best responses as uplink control messages (UCMs). Crucially, we reformulate the MARL paradigm by endowing both the BS and UEs with language-oriented policy models that naturally handle variable-length input and output sequences while retaining the exploratory capabilities of DRL. We employ proximal policy optimization (PPO) to facilitate a continuous, feedback-driven alignment of the agents’ policies with evolving network dynamics. This synergistic design overcomes the architectural rigidity of conventional MARL and mitigates the data dependency of LLMs. We also incorporate a protocol action grammar (PAG) to guarantee reliability and efficiency by enforcing domain-specific constraints on agent decisions. To this end, we theoretically analyze the convergence of an equilibrium within this learning framework. We summarize the key differences between our algorithm and highly related literature in Table ¿tbl:tab:32summary?.

Our key contributions are threefold, as follows.

• We establish a dynamic MFSG formulation for MAC protocol emergence in the UDTS scenario, explicitly capturing the inherently hierarchical interactions between BS and a dynamically varying number of UEs. As a pioneering effort, this game-theoretic model fundamentally differs from existing paradigms [3], [4] by formulating asymmetric leader-follower relationships.

• We propose a novel LLM-empowered MARL framework that fundamentally reconciles the powerful generalization capabilities of LLMs with the exploratory learning essential for protocol discovery, driven by PPO. The concerted effort makes our design uniquely capable of adapting to a dynamic network without retraining. It also incorporates a PAG mechanism to further enhance efficiency and reliability.

• We provide a theoretical analysis of our framework, establishing not only the guaranteed existence and local convergence of a Stackelberg equilibrium (SE) in the dynamic game, but also the robustness against information disparities. Extensive simulations have validated the superiority and robustness of our proposed algorithm.

This paper is structured as follows. Section II details the related works. Section III presents the system model and formulates the problem. Section IV details the LLM-based decision maker and its PPO training. Section V evaluates performance via simulations, and the main conclusion is drawn in Section VI.

2 Related Works↩︎

2.1 DRL for Communication Protocol↩︎

Leveraging its data-driven exploration and trial-and-error mechanisms, DRL is suited to optimize and even discover protocols for task-specific applications. One research line focuses on optimizing the functional blocks and operational parameters of existing protocols [18]–[22]. For example, Ref. [18], [19] introduces DRL to automatically revise the features of protocol blocks in response to various network conditions, demonstrating superior throughput and latency compared to traditional methods. Ref. [20] proposes specialized wireless networks (SpecNets), wherein agents dynamically adjust settings such as contention windows and frame aggregation to meet diverse performance demands. Another research thrust focuses on augmenting the adaptability of communication protocols. Ref. [21] demonstrates the adaptability of the MARL-based protocol across various dynamic network conditions and coexistence scenarios. Ref. [22] employs multi-task learning and a scalable transformer-based critic to train a single, generalizable model that can adapt to dynamic network sizes and traffic conditions. In contrast, several works have considered protocol emergence through training from scratch. Ref. [3] establishes a DRL framework to learn channel access policies and signaling simultaneously. Subsequent works extend this with more advanced MARL algorithms and optimization tricks [5], [6], [8], [9], achieving notable progress in the convergence, adaptability, and scalability of emergent protocols. Ref. [23] even explores interpretability by encoding symbolic structure via a probabilistic logic programming language (ProbLog). Nevertheless, a persistent challenge for existing DRL-based solutions is their limited generalization [24], rendering them brittle in highly dynamic network environments and often necessitating costly retraining or architectural redesigns.

2.2 LLMs for Communication Protocol↩︎

The advent of LLMs is set to revolutionize wireless protocol design. Benefiting from their remarkable capabilities in understanding, reasoning, and generalization, alongside their extensive general knowledge bases, LLMs have achieved considerable success in various downstream tasks within wireless networking [25]. On the one hand, LLMs can serve as base models that are fine-tuned on communications-specific datasets to support resource management [26], standard-protocol interpretation [27], and protocol design [28], [29]. On the other hand, LLMs also facilitate direct zero-shot or few-shot domain adaptation. Ref. [30] leverages knowledge acquired via in-context learning and uses knowledge distillation to guide and accelerate protocol emergence in multi-agent navigation systems. Recently, Ref. [31] employs prompt engineering to auto-tune LLM system prompts for generating semantic signaling tokens for BSs and UEs, enabling zero-shot adaptation to environmental dynamics. Ref. [24] further showcases the coordination of agentic LLM workflows for autonomous and interpretable protocol design. While effective in achieving high performance, LLMs without post-training remain path-dependent on domain-specific datasets and highly engineered prompts, which constrain their ability to discover fundamentally new protocols through exploration. Our previous work [10] makes an initial attempt to reconcile exploration with generalization by fine-tuning an LLM-based policy using PPO. Yet, it fell short of capturing the intrinsic game-theoretic interactions among agents.

2.3 Stackelberg Game in Wireless Communication↩︎

Distinct from broader computational game-theoretic paradigms, such as evolutionary games that typically characterize symmetric population-level adaptation [32], SG features an intrinsic leader-follower structure and has been widely used to model hierarchical strategic interactions in wireless networks [33]–[35]. In [33], the SG framework effectively models the pricing strategies between service providers (leaders) and end-users (followers) in complex multi-tenant network slicing environments, demonstrating faster convergence and greater economic efficiency. Ref. [34] employs an SG to model energy trading between a power station and a transmitter in an intelligent reflection surface-aided wireless-powered multicast system. The SG framework also delineates the fundamental dynamics governing network interactions and even accelerates convergence [13], guiding agents toward a more efficient equilibrium. However, to our best knowledge, MFSG has not been leveraged to develop MAC protocols.

3 System Model and Problem Formulation↩︎

3.1 System Model↩︎

3.1.1 Basics of UDTS↩︎

As illustrated in Fig. 1, we investigate the uplink data transmission scheduling (UDTS) within a single-cell orthogonal frequency division multiple access (OFDMA) system where a BS serves multiple UEs. The set of \(I\) active UEs is denoted by \(\mathcal{I}_t = \{{u}_{1}, \cdots, {u}_{I}\}\) at time \(t\), where \(I\) may vary due to the arrivals and departures of UEs, reflecting the dynamic nature of the network topology. Consistent with 3GPP TS 38.321 [36], the BS assigns each UE dedicated physical uplink/downlink control channels (PUCCH/PDCCH) for signaling and a shared contention-based channel (PUSCH) for MAC data protocol data unit (dPDU) transmission. The available bandwidth is divided into \(M\) orthogonal resource block groups (RBGs) in the frequency domain, and time is discretized into slots of fixed duration, termed as transmission time intervals (TTIs). An operational cycle, or episode, spans \(T\) TTIs. This scenario necessitates efficient MAC protocols that jointly optimize signaling and access strategies to mitigate RBG collisions while balancing KPIs like throughput, resource efficiency, and fairness. At time slot \(t\), UE \({u}_{i}\) constructs and transmits a UCM \(u_{{i},t}\), encapsulating its local state (e.g., buffer status report and interaction history) and transmission intent. In response, the BS integrates these UCMs with its global observations to issue a DCM \(d_{{i},t}\) to each UE, conveying its macroscopic scheduling intent. These signaling exchanges, typically conveyed over the dedicated PUCCH and PDCCH, are crucial for coordinating access and establishing the operational context for data delivery.

3.1.2 Delivery Model↩︎

We model the arrival of new data at each UE as a Bernoulli process [3], [5], where a dPDU of a fixed size \(\xi^{\rm gen}\) (in bits) is generated with probability \(p_a\) in each TTI. Meanwhile, each UE is equipped with a first-in-first-out (FIFO) buffer of capacity \(B_{\rm cap}\) to temporarily store dPDUs, with the buffer occupancy denoted by \(b_{i,t}\in [0, B_{\rm cap}]\). The UE transmission action, denoted by \(a^{\rm bit}_{{i},t,{ u}}\), is an \(M\)-bit binary bitmap, where each bit indicates whether the \(m\)-th RBG is selected for data transmission. Following 3GPP TS 38.214 [37], the maximum transport block size (TBS) of \({u}_{i}\) in time slot \(t\) is given by \[N_{i,t,{ u}}= {\rm tr}(\boldsymbol{1}^Ta^{\rm bit}_{{i},t,{ u}})\cdot N_{\rm rb}^{\rm rbg}\cdot N_{\rm sc}^{\rm rb} \cdot N_{\rm symbl}^{\rm sh} \cdot \nu_{i,t}\] where \(N_{\rm rb}^{\rm rbg}\) is the number of RBs per RBG, \(N_{\rm sc}^{\rm rb}\) is the number of subcarriers per RB, \(N_{\rm symbl}^{\rm sh}\) is the number of symbols per slot, and \(\nu_{i,t}\) denotes spectral efficiency. Crucially, \(\nu_{i,t}\) is a time-varying quantity that depends on the instantaneous signal-to-noise ratio (SNR) of the link and the corresponding modulation and coding scheme (MCS) selected by the UE. The system employs an automatic repeat request (ARQ) process for buffer management, ensuring UEs remove bits from their buffers after the BS confirms a successful data reception.

3.1.3 Channel Model↩︎

Without loss of generality, we model the PUSCH as a packet erasure channel with a nonzero transport block error rate (TBLER). If an RB is collision-free and the transmission is error-free, the BS successfully receives the transmitted data. Moreover, the channel quality for UE \({u}_{i}\) at time \(t\) is denoted by \(c_{i,t,{ u}}\), which evolves between several discrete states. Its transitions reflect short-term channel fluctuations (e.g., fast fading) and thus affect the instantaneous spectral efficiency \(\nu_{i,t}\). In parallel, the BS acquires corresponding channel state information (CSI), \(\hat{c}_{i,t,{ u}}\), which may differ from the UE’s locally true state. This mismatch between \(c_{i,t,{ u}}\) and \(\hat{c}_{i,t,{ u}}\) abstracts the aggregate leader-side information deficit caused by practical CSI acquisition imperfections, such as reciprocity errors, feedback/processing delays, and channel aging, rather than separately modeling their individual PHY-layer sources.

3.1.4 Decision Model↩︎

A sequential decision-making process governs both UE-BS signaling and the UE’s transmission behavior. For each UE \({u}_{i}\), define its observation at time \(t\) as \(o_{i,t,{ u}}=(c_{i,t,{ u}},b_{i,t},d_{{i},t},a_{i,t-1,u})\) whereas the taken action \(a_{i,t,{ u}}=(a^{\rm bit}_{{i},t,{ u}},u_{{i},t})\). Meanwhile, the BS’s observation is given by \(o_{t,{ b}}=(\{\hat{c}_{i,t,{ u}}\}_{i\in \mathcal{I}_t}),\{u_{i,t-1}\}_{i\in \mathcal{I}_t},\{d_{i,t-1}\}_{i\in \mathcal{I}_t})\) and its action is defined as \(a_{t,{ b}}=(d_{{1},t},\cdots,d_{{I},t})\). Crucially, both BS and UE observations are primarily composed of the preceding interactive UCMs/DCMs, which allows the learned policies to condition subsequent RBG decisions on prior signaling and thereby induce task-grounded operational meanings to optimize system performance. Each agent’s behavior is dictated by a policy that maps its observations to a probability distribution over the action space. Let \(\pi_b\) be the leader’s policy and \(\pi_{{i},u}\) the follower’s policy, such that \(a_{t,{ b}}\sim \pi_b(o_{t,{ b}})\) and \(a_{i,t,{ u}}\sim \pi_{{i},u}(o_{i,t,{ u}})\).

3.2 Stackelberg Game Formulation↩︎

We formally model the sequential and interactive decision-making process as a dynamic MFSG as illustrated in Fig. 2. In this game, the BS (leader) enforces macroscopic decisions to optimize network-wide performance, while the UEs (followers) retain the flexibility to optimize fine-grained transmission parameters based on local information and BS’s guidance. Our objective is to develop a framework that facilitates the emergence of leader–follower policies, driving the system toward a desirable SE.

As illustrated in Fig. 2, the utility of each agent is coupled with the decisions of others². Specifically, the utility for a follower \({u}_{i}\in\mathcal{I}_t\) at time slot \(t\) is defined as a weighted sum of transmission efficiency and its consistency with the leader’s DCM: \[\label{eq:ue32unity} F_{i,t,{ u}}= \varrho_1 \cdot \frac{\xi_{i,t,{ u}}^{\rm rec}}{\xi_{i,t,{ u}}^{\rm tx}} + \varrho_2 \cdot C_{i,t,{ u}},\tag{1}\] where \(\varrho_1,\varrho_2\) are constants, \(\xi_{i,t,{ u}}^{\rm rec}\) denotes the successfully received dPDUs from \({u}_{i}\) at time slot \(t\), and \(\xi_{i,t,{ u}}^{\rm tx}\) represents the total number of transmission attempts made by \({u}_{i}\). \(C_{i,t,{ u}}\) quantifies the normalized Hamming similarity [39] between the UE’s transmission bitmap \(a^{\rm bit}_{i,t,{ u}}\) and the BS’s intended allocation bitmap \(d^{\rm bit}_{i,t}\) (i.e., the bit-level representation of \(d_{i,t}\) and can be directly derived from \(a_{t,{ b}}\)). In other words, \[\label{eq:32consistency32term} C_{i,t,{ u}}=\frac{1}{M} \sum\nolimits_{m=1}^M\left(1-\left(a_{i, t, u, m}^{\rm bit} \oplus d_{i, t, m}^{\rm bit}\right)\right).\tag{2}\] Given the leader’s policy \(\pi_b\), each follower seeks to find its best-response policy \(\pi_{{i},u}^\ast\) that maximizes its expected discounted long-term utility \(J_{i,u}\) over the distribution of UE numbers \(\mathbb{D}(I)\), \[\label{eq:32long-term32ue32unity} J_{i,u}= \mathbb{E}_{I_t\sim\mathbb{D}(I)}\mathbb{E}_{\pi_{{i},u}, \pi_b, \pi_{{-i},u}}\left[\sum\nolimits_{t=0}^{T-1} \gamma^t F_{i,t,{ u}}\right],\tag{3}\] where \(\gamma\in [0,1]\) is the discount factor. Conversely, the leader’s utility \(F_{t,{ b}}\) aligns with the system-level objectives, defined as a weighted sum of the average user throughput and inter-user fairness: \[\label{bs:ue32unity} F_{t,{ b}}\triangleq \frac{1}{|\mathcal{I}_t|\cdot TTI}\sum_{i\in \mathcal{I}_t} \hat{\xi}_{i,t,{ u}}^{\rm rec}+\varepsilon\frac{(\sum_{i\in \mathcal{I}_t} x_{i,t,{ u}})^2}{|\mathcal{I}_t|\cdot\sum_{i\in \mathcal{I}_t}(x_{i,t,{ u}})^2},\tag{4}\] where the \(\varepsilon\) balances the scaled throughput against fairness, which is quantified using Jain’s Fairness Index (JFI) [40], \(x_{i,t,{ u}}\) represents the cumulative number of RBGs utilized by UE \({u}_{i}\) up to time \(t\). At this equilibrium, the leader’s policy is a universal solution to Eq. 5 , anticipating that all followers will adhere to their best-response policies. \[\label{eq:32long-term32bs32unity} \max_{\pi_b} J_b=\max_{\pi_b}\mathbb{E}_{I_t\sim\mathbb{D}(I)}\mathbb{E}_{\pi_b,\boldsymbol{\pi}^\ast_u}\left[\sum\nolimits_{t=0}^{T-1} \gamma^t F_{t,{ b}}\right],\tag{5}\] where \(\boldsymbol{\pi}^\ast_u = (\{\pi^\ast_{i,t,{ u}}\}_{i\in\mathcal{I}_t})\).

Consequently, the problem is to find a set of policies \((\pi^\ast_{t,{ b}},\{\pi^\ast_{i,t,{ u}}\}_{i\in\mathcal{I}_t})\) that constitutes an expected Stackelberg equilibrium (ESE) over \(\mathbb{D}(I)\), where no player has an incentive to unilaterally deviate from their policy. Solving such an equilibrium is nontrivial due to the implicit coupling of utilities and the high-dimensional state–action space, which renders traditional optimization intractable. While MARL is a promising alternative, its fixed architectures fail to generalize to a dynamic environment. To address this, we propose an LLM-driven MARL framework that jointly leverages MARL’s exploratory power and LLMs’ generalization capability to learn robust equilibria, as detailed in the following section.

4 PPO-based LLM Decision Maker for Protocol Emergence↩︎

In this section, we recast the search for the SE as an LLM-empowered multi-agent learning process, where all agents operate in a semantic-generalized UTDS environment. Both the BS and UEs leverage parameterized LLM as their decision-making policies. Within each TTI, based on its observation, the BS initiates its action \(a_{t,{ b}}\), and the UEs react accordingly. Ultimately, both agents receive their respective rewards from the environment. PPO with expert seeding is employed to ground the LLMs’ performance, ensuring the policies are functionally aligned with the dynamic environment.

4.1 Framework of RL-based Protocol Emergence↩︎

We begin by outlining the conventional MARL approach for protocol emergence, establishing a baseline, and highlighting its limitations. Following the work [6], the UDTS is formulated as a decentralized partially observable Markov decision process (Dec-POMDP), where agents are trained concurrently to learn both transmission and signaling actions using PPO. The UCM and DCM are treated as special discrete actions of the UEs and the BS, respectively. While these signaling actions do not directly alter the environment, they influence the decisions of other agents by being part of their observable states. Taking the BS as an example, we illustrate its training process. As generally defined by the PPO framework, the BS owns an actor network to represent its policy \(\pi_b\) and a critic network \(V(o_{t,{ b}})\) for its value function, parameterized by \(\theta_b\) and \(\phi_b\) respectively. The collective goal is to find a policy that maximizes the objective function defined in 5 . During the interaction with the environment, the BS agent collects a series of experience trajectories \(\tau\) composed of transition tuples and \[\label{eq:rollout} \tau_b = \left\{ \left(o_{t,{ b}},a_{t,{ b}},F_{t,{ b}},\pi_b(a_{t,{ b}}\lvert o_{t,{ b}}),V(o_{t,{ b}}),A_{t,{ b}} \right)\right\}_{t=0}^{\lvert T-1\rvert}\tag{6}\] captures current observation, action, reward, policy probability, value estimate, and the advantage estimate. The advantage \(A_t\) is computed via generalized advantage estimation (GAE) \[\label{eq:GAE} A_{t,{ b}}= \sum\nolimits_{l=0}^{T}(\gamma\lambda)^l(F_{t+l,{ b}} + \gamma V(o_{t+l+1,{ b}}) - V(o_{t+l,{ b}})),\tag{7}\] indicating whether an action taken is better or worse than the policy’s average, and \(\lambda\) denotes the GAE factor. Updating the critic network \(\phi_b\) involves minimizing the squared temporal-difference (TD) residuals \[\label{eq:critic95loss} L(\phi_b)=\mathbb{E}_{\tau_b\sim\mathcal{B}_b}\Big[ \sum\nolimits_{t=1}^T \big(V(o_{t,{ b}})-\hat{R}_{t,{ b}}\big)^2\Big],\tag{8}\] where \(\mathcal{B}_b\) denotes experience buffers, \(\hat{R}_t = \sum_{t^\prime = t}^{T} \gamma^{t^{\prime}-t} F_{t^\prime,{ b}}\) is the discounted reward-to-go. The actor network \(\theta_b\)³ is updated via the sampled policy gradient from the clipped surrogate objective function \[\begin{align} &J(\theta_b) =\mathbb{E}_{\tau_b\sim\mathcal{B}_b}\bigg[\sum_{t=1}^T \big[\min \big(\eta_{t,{ b}}A_{t,{ b}}, \operatorname{clip}(\eta_{t,{ b}}, 1\!-\!\epsilon, 1\!+\!\epsilon) A_{t,{ b}}\big) \nonumber\\ & -D_\text{KL}\big(\pi_b^{\rm old}(\cdot|o_{t,{ b}})\lvert\lvert\pi_b(\cdot|o_{t,{ b}})\big) + H\big((\pi_b(\cdot|o_{t,{ b}})\big)\big]\bigg], \label{eq:actor95loss} \end{align}\tag{9}\] where \(\eta_{t,{ b}}= \frac{\pi_b(a_{t,{ b}}\lvert o_{t,{ b}})}{\pi_b^{\rm old}(a_{t,{ b}}\lvert o_{t,{ b}})}\) denotes the important ratio, the operator \(D_\text{KL}(\cdot\|\cdot)\) computes the KL-divergence, the policy entropy \(H\) encourages exploration, meanwhile both the clipping function \(\text{clip}(\cdot)\) and the hyperparameter \(\epsilon\) prevent excessively large policy updates.

4.2 Semantic-Generalized MFSG↩︎

It can be observed that the traditional policy \(\pi_b\)’s input and output dimensions, which process the UCM and DCM vectors, scale with the total number of UEs. This generates an incompatibility with standard fixed-neuron architectures (e.g., multi-layer perceptron (MLP)), as any environment shift may invalidate the learned policy, forcing costly redesign and retraining from scratch. To address this kind of generalization challenge in dynamic environments, we take advantage of LLMs, which inherently process variable-length input and output sequences. Under this paradigm, fluctuations in the number of UEs merely alter the prompt’s length rather than disrupting the fundamental training pipeline. Specifically, we introduce a natural language vocabulary \(\mathcal{W}\) and a mapping function set \(\boldsymbol{f}\) to adapt to the linguistic modality. We assume both the BS and UEs utilize the same family of LLMs, thereby sharing \(\mathcal{W}\). As depicted in Fig. 3, raw numerical observations \(o_{t,{ b}},o_{i,t,{ u}}\) are transformed into a comprehensive LLM-processable prompt through numeric-to-text conversion and meta-prompt assembly: \[\label{eq:32pag32embedding} \begin{align} \tilde{o}_{i,t,{ u}}&\triangleq f_{o,u}(o_{i,t,{ u}}) \oplus \psi_u ,\;\tilde{o}_{i,t,{ u}}\subset \mathcal{W}\\ \tilde{o}_{t,{ b}}&\triangleq f_{o,b}(o_{t,{ b}}) \oplus \psi_b ,\;\tilde{o}_{t,{ b}}\subset \mathcal{W}, \end{align}\tag{10}\] where the tilde notation distinguishes the linguistic space from the primitive numerical space, \(f_{o,u},f_{o,b}\in \boldsymbol{f}\) convert the numerical observation vector into their linguistic representation, and \(\psi_u,\psi_b\) are meta-prompts containing natural language task descriptions and agent identity identifiers. The LLM then autoregressively generates a structured response sequence that encapsulates the agent’s action and signaling. Rather than open-domain text, these outputs function as task-grounded semantic control messages [31], whose operational meanings are strictly induced by leader-follower interactions and spatial RBG allocations⁴. Notably, this textual output must be parsed via an inverse mapping to be executed within the environment.

4.3 Generative Policy Architecture with PAG↩︎

To ensure the reliability of protocol emergence, we employ PAG to constrain the LLM’s generative process within the valid protocol-action subspace. As depicted in Fig. 4, unlike standard stochastic sampling on an open-domain vocabulary \(\mathcal{W}\) that may yield unparseable tokens or hallucinations [41], PAG enforces a strict mapping from the hidden states to the admissible token set \(\mathcal{W}^-_b,\mathcal{W}^-_u\subset \mathcal{W}\). Precisely, the BS’s output is constrained to only the numerical tokens within \(\mathcal{W}^-_b=\{``0", ``1",\cdots, ``I_t "\}\) that indicate the allocation of corresponding RBGs. Here, “0" indicates no allocation and others specify the allocation to the \(i\)-th UE. Each DCM action is then represented as a sequence \(\tilde{a}_{t,{ b}}= \{w_0,\cdots,w_{M}\}\), \(w\in\mathcal{W}^-_b\) of fixed length \(M\), corresponding to the maximum number of allocable RBs. The probability of generating a specific, valid DCM is factorized autoregressively \[\label{eq:llm95generation} \mathbb{P}(\tilde{a}_{t,{ b}}\lvert \tilde{o}_{t,{ b}}) = \prod\nolimits_{j=0}^{\lvert \tilde{a}_{t,{ b}}\rvert} \mathbb{P}(w_j \lvert \tilde{o}_{t,{ b}},w_{<j} ) ,\tag{11}\] where \(\mathbb{P}(w_j \lvert \tilde{o}_{t,{ b}},w_{<j} )\) represents the probability of generating token \(w_j\) given prompt \(\tilde{o}_{t,{ b}}\) and preceding tokens \(w_{<j}\). Crucially, to enforce the PAG constraints in practice, we manipulate the conditional probability \(\mathbb{P}(w_j | \cdot)\) via logit masking. Let \(\mathbf{z}_j \in \mathbb{R}^{|\mathcal{W}|}\) denote the raw logits output by the LLM at step \(j\). We define a constraint mask \(\mathbf{M} \in \mathbb{R}^{|\mathcal{W}|}\) such that \(\mathbf{M}_k = 0\) if token \(k \in \mathcal{W}^-_b\) and \(-\infty\) otherwise. The valid conditional probability is then derived by \[\label{eq:32PAG32logit32masking} \mathbb{P}(w_j | \tilde{o}_{t,b}, w_{<j}) = \frac{\exp(\mathbf{z}_{j, w_j} + \mathbf{M}_{w_j})}{\sum_{k \in \mathcal{W}} \exp(\mathbf{z}_{j, k} + \mathbf{M}_k)}.\tag{12}\] This operation forces the probability of invalid tokens to zero, effectively pruning the search space. To achieve computational efficiency, we obtain the log-probabilities from Eq. 11 and subsequently normalize these scores using a softmax function. This yields a policy distribution over the action space \(\tilde{\mathcal{A}}_b\) as \[\pi_b(\tilde{a}_{t,{ b}}\lvert \tilde{o}_{t,{ b}}) = \frac{\exp\{\sum_{j=0}^{\lvert \tilde{a}_{t,{ b}}\rvert} \log\mathbb{P}(w_j \lvert \tilde{o}_{t,{ b}},w_{<j})\}}{\sum_{\tilde{a}_{t,{ b}}\in \tilde{\mathcal{A}}_b} \exp\{\sum_{j=0}^{\lvert \tilde{a}_{t,{ b}}\rvert} \log\mathbb{P}(w_j \lvert \tilde{o}_{t,{ b}},w_{<j})\} }.\] Similarly, we can specify \(\mathcal{W}^-_u=\{``0",``1"\}\) to cater to the UE’ action \(a^{\rm bit}_{i,t,{ u}}\) and derive the UE policy \(\pi_{{i},u}\). Such policy formulation fully exploits the LLM’s pre-trained generative priors to score the plausibility of actions, thereby obviating the need for auxiliary, randomly initialized output layers. It is thus robust to diverse action spaces without necessitating architectural modifications.

4.4 PPO-driven Functional Alignment and Adaptation↩︎

The training mechanism follows the principles from Section 4.1, but we tailor it substantially to accommodate the unique hierarchical structure of our dynamic MFSG. We instantiate LLM-enpowered actor \((\theta_b, \theta_u)\) for BS and UEs, respectively, where dynamic UEs utilize the same neural network by parameter sharing. Their corresponding critic networks are realized by replacing the final layer of the initial LLM decoder block with a single-value head MLP. The training procedure, delineated in Algorithm 5, is orchestrated around the hierarchical experience coupling. Specifically, after an initial environment reset (line 5), the process unfolds within each TTI to precisely capture the game’s sequential dependencies. The leader BS first executes its action \(a_{t,{ b}}\) (line 7), which consequently alters the observational space for all followers UEs. The followers then react based on this updated context (line 8), and their joint actions collectively determine the subsequent observations and rewards (lines 9-10). Although the trajectory data collected from this process is inherently coupled by the hierarchical interaction (lines 12-13), the leader and followers independently update their respective policy and value networks (lines 16-21). This process enables the policies to progressively align with the environment dynamics by grounding each update in causally correct, interdependent experiences.

4.5 Proof of Convergence↩︎

The co-adaptation of leader and follower policies in our dynamic MFSG creates an inherently non-stationary learning dynamic. This section therefore provides a formal proof that the gradient-based PPO alignment guides the framework to an equilibrium. We begin with fundamental assumptions and definitions.

Consistent with standard practice in learning-in-games [42], [43], we only focus on local properties of the equilibrium and thus DSE is adopted.

To analyze the convergence of our PPO-based alignment, we also define the following policy update dynamics [43], [45].

Furthermore, we make the following assumptions.

To ensure that the protocol emergence process has a well-defined and reachable target, we begin by formally defining and proving the existence of such an equilibrium within our system. This result is formalized as Theorem 1.

As completely proven in Appendix 7.1, we analyze the equilibrium hierarchically. First, we show that the follower subgame is a potential game [11]. Hence, for any \(I_t\), it admits an NE, i.e., the followers’ best-response correspondence is nonempty. Next, in the dynamic-follower setting, we invoke Weierstrass’ Theorem [48] to establish the existence of a solution to the leader’s problem. Together, these results imply the existence of an ESE.

Based on the existence of equilibrium, we present a general convergence theorem, which states that the policy gradient update converges in a fixed-number MFSG.

We leave the proof in Appendix 7.2 while providing a proof sketch here. The logic is to show that the update map, \(\Lambda(\boldsymbol{\varTheta})=\boldsymbol{\varTheta}+\alpha_b \Omega(\boldsymbol{\varTheta})\), is a local contraction around the DSE fixed point \(\boldsymbol{\varTheta}^*\). The analysis hinges on the property, derived in the appendix from Assumptions 1-3, that the symmetric part of the gradient field’s Jacobian, \(M\left(\boldsymbol{\varTheta}^*\right)=\frac{1}{2}\left(\mathbf{J}_{\boldsymbol{\Omega}}^{\top}+\mathbf{J}_{\boldsymbol{\Omega}}\right)\), is negative definite (\(M\left(\boldsymbol{\varTheta}^*\right) \prec 0\)). This allows for the selection of a learning rate \(\alpha_b=\kappa_1 / \kappa_2\), where \(\kappa_1=\sigma_{\min}\left(-M\left(\boldsymbol{\varTheta}^*\right)\right)\), \(\kappa_2=\sigma_{\max }\left(\mathbf{J}_{\boldsymbol{\Omega}}^{\top} \mathbf{J}_{\boldsymbol{\Omega}}\right)\) and \(\sigma\) denotes a singular value, which guarantees contraction by satisfying \[\left\|I+\alpha_b J_{\boldsymbol{\Omega}}\left(\boldsymbol{\varTheta}^*\right)\right\|_2^2 \leq 1-\kappa_1^2 / \kappa_2<1.\] This result confirms that the spectral radius \(\rho\left(\nabla \Lambda\left(\boldsymbol{\varTheta}^*\right)\right)<1\), proving that \(\Lambda\) is a contraction and implying that under local smoothness and boundedness assumptions, though it does not ensure a global optimum for arbitrary LLM policies, it yields local geometric convergence to the DSE at the claimed rate.

In our dynamic MFSG setting, where the number of followers changes stochastically and all followers share a common LLM policy, we derive the following corollary.

We leave the proof in Appendix 7.4, which extends Theorem 2 by analyzing the averaged learning dynamics over the distribution of followers. The simulation results in Fig. 7 further demonstrate that convergence is not only theoretically possible but also achievable in practice.

To quantify the uncertainty induced by network anomalies, we further investigate the robustness of the converged equilibrium under environmental shifts.

Theorem 3 ensures the learned equilibrium maintains stability rather than suffering catastrophic failure under mild environment shifts. Moreover, a larger \(K\), i.e., a longer interaction horizon during practical testing can reduce the empirical bias. We leave the proof in Appendix 7.5.

4.6 Complexity Analysis↩︎

We analyze the framework’s efficiency by examining its computational complexity, space complexity, and the effective exploration complexity. Regarding computational time, the floating point operations (FLOPs) for a single forward pass of an LLM with \(n\) layers and hidden dimension \(\mathcal{d}_{\rm model}\) scales quadratically with sequence length \(\mathcal{l}\) as \(\mathcal{O}(\mathcal{n}\mathcal{l}^2\mathcal{d}_{\rm model})\). This scaling is intrinsically asymmetric. For the leader, the prompt length is a function of \(I_t\), i.e., \[\mathcal{l}_{t,{ b}}\approx \mathcal{l}_{\psi_b}+ \sum_{i=1}^{I_t} \mathcal{l}_{i,u}\triangleq \mathcal{O}(I_t),\] where \(\mathcal{l}_{\psi_b}\) is the constant length of the meta-prompt and \(\mathcal{l}_{i,u}\) is the length of the tokenized UCM from UE followers. And the UE sequence length is constant \(\mathcal{l}_{i,u}= \mathcal{O}(1)\). Consequently, generating an action sequence of \(M\) incurs a cost of \(\mathcal{O}(M \cdot I_t^2 \cdot \mathcal{n}\mathcal{d}_{\rm model})\) for the leader and \(\mathcal{O}(M\cdot I_t\cdot \mathcal{n}\mathcal{d}_{\rm model})\) for all \(I_t\) followers. With \(I_t\) varying over time according to a distribution, the total training complexity over \(e^{\rm max}\) epochs is best characterized by its expected value \[\mathcal{O}\Big(e^{\rm max} T\cdot M\mathcal{n}\mathcal{d}_{\rm model}\big(\mathbb{E}[I_t^2]+\mathbb{E}[I_t]\big)\Big).\] In contrast, the per-decision inference cost is an instantaneous measure dependent on the current \(I_t\), given by the sum of the leader’s and followers’ forward generation costs \(\mathcal{O}\big( M I_t^2\mathcal{n}\mathcal{d}_{\rm model}+I_tM\mathcal{n}\mathcal{d}_{\rm model}\big)\).

The exploration efficiency is fundamentally dictated by the action space complexity. Standard decoding operates on a high-dimensional probability simplex (e.g., \(|\mathcal{W}| \sim 10^5\) for Llama series [49]), suffering from the curse of dimensionality. PAG functions as a dimension reduction operator, projecting this sparse optimization problem onto a compact, task-valid subspace where \(|\mathcal{W}^-| \ll |\mathcal{W}|\). This \(\mathcal{O}(10^4)\)-fold dimensionality reduction regularizes the policy search landscape, transforming intractable open-ended generation into a tractable discrete control task.

The total space complexity during training is the sum of storage for all network parameters and experience replay buffers, totaling \(\mathcal{O}\big(\mathcal{n}\mathcal{d}^2_{\rm model}+|\mathcal{B}_b|+|\mathcal{B}_u|\big)\). During inference, this requirement reduces to storing only the actor network parameters, yielding a space complexity of \(\mathcal{O}\big(\mathcal{n}\mathcal{d}^2_{\rm model}\big)\).

5 Simulation Results↩︎

In this section, we first detail the simulation setup and the baseline methods. We then compare the performance of our proposed framework against the baselines under various scenarios. Additionally, we include ablation studies and hyperparameter sensitivity analysis.

5.1 Simulation Setup↩︎

We consider a UDTS environment where the number of UEs \(I_t\) varies within the set \(\{3, 4, 5\}\). Each episode spans \(T=24\) TTIs, with a TTI duration of \(5\times10^{-3}\) s. UEs commence with an empty and sufficiently large buffer to prevent overflow. By default, three distinct \(p_a\in\{0.1,0.3,0.7\}\) are assigned to UEs cyclically to model heterogeneous traffic loads, with a fixed dPDU size \(\xi^{\rm gen}=256\) bits. Each UE’s channel transitions between “poor",”medium", and “good" states with a changing frequency in \([1,10]\) ms, where each state maps to a spectral efficiency \(\nu_{i,t}\) via a pre-defined MCS lookup table. Moreover, we set \(N^{\rm rb}_{\rm sc}=12\), \(N^{\rm sh}_{\rm symbol}=14\), aligning with the commonly deployed 3GPP new radio (NR) configuration [36], [37].

We employ open-source Llama3-1B [49] as the actor-critic backbone. The critic network is augmented with a three-layer value head (\(1024\) hidden neurons per layer and ReLU activations) to produce scalar state values. The actor’s temperature linearly decays from \(3.0\) to \(0.3\) during training. To facilitate online training, we perform \(5\) epochs of PPO updates after collecting \(10\) full episodes, after which the buffer \(\mathcal{B}\) is cleared for new collections. The PPO updates utilize a mini-batch size of \(128\), an Adam optimizer with a learning rate of \(5\times10^{-4}\), and a cosine learning rate decay schedule. We totally train for \(2,000\) epochs, periodically evaluating the performance in fixed-UE settings. Experiments are conducted on a server running Ubuntu 22.04.5, equipped with an AMD EPYC 7763 64-Core CPU, 1 TiB of RAM, and eight NVIDIA A800 GPUs. The software stack includes Python 3.12, CUDA 12.2, and PyTorch 2.5. Furthermore, we consider the following baseline methods:

• Heuristic: An enhanced S-ALOHA [31] where each UE initially transmits a dPDU with a probability \(0.5\), which is then adaptively decreased upon a collision.

• MAPPO-S: Symbolic MAPPO algorithm for protocol emergence, as presented in [6], which is retrained for each specific scenario to adapt to environmental shifts.

• MAPPO-G: The same MAPPO architecture, but trained in one scenario and directly applied to others without retraining to evaluate its generalization capabilities.

• G-MAC: A state-of-the-art baseline inspired by [50], which achieves structural generalization across varying numbers of devices by utilizing temporal policy serialization.

See Appendix 8 for more detailed simulation setup and the baselines’ implementation details. Finally, we introduce the following network KPIs to evaluate the performance.

• Throughput: We define the average throughput of our system as the number of dPDU bits received by BS per active UE per unit of time, computed over all TTIs by \[\label{eq:TH} {\rm Th} = \frac{1}{T}\sum_{t=1}^{T}\Big(\frac{1}{|\mathcal{I}_t|}\sum_{i\in \mathcal{I}_t}\xi^{\rm rec}_{i,t,{ u}}\Big).\tag{13}\]

• Fairness Index: We utilize the Jain’s Fairness Index [40] as detailed in the second term of Eq. 5 .

• Resource Efficiency: The resource efficiency is defined as the received dPDUs divided by the total RBG utilization \[\label{eq:Eff} {\rm Eff} = \frac{\sum_{t=1}^{T}\sum_{i\in \mathcal{I}_t}\xi^{\rm rec}_{i,t,{ u}}}{\sum_{i\in \mathcal{I}_T}x_{i,T,u}}.\tag{14}\]

• Packets Success Rate: We measure the ratio of successfully delivered dPDUs by \((\sum \xi^{\rm rec}) / (\sum \xi^{\rm tx})\).

• Latency: We consider the average end-to-end delay (in slots) from dPDU generation to successful reception, as well as the 95% tail latency.

• Queue Stability: This is evaluated via the average and maximum buffer occupancy across all UEs, serving as an indicator of congestion control.

5.2 Numerical Results and Analysis↩︎

6pt

5.2.1 Comparative Analysis with Baselines↩︎

Fig. 6 presents a comparative evaluation of the proposed framework against baselines in terms of average throughput and fairness. Both the MAPPO-G and G-MAC are trained exclusively in the 5-UEs scenario. As observed, our method consistently dominates all baselines across varying configurations. In the high-contention regime with \(5\) UEs and \(p_a=0.75\), the proposed framework achieves a throughput of approximately \(1.2\), surpassing the retrained MAPPO-S and Heuristic by \(46.6\%\) and \(81.1\%\), respectively. Notably, while the G-MAC effectively mitigates the catastrophic degradation seen in MAPPO-G by adopting a novel BS policy structure [50], our method still outperforms it by approximately \(38.5\%\), verifying a superior generalization of the LLM-empowered framework. In terms of fairness, as shown in the second row of Fig. 6, our approach maintains a near-optimal fairness index across all scenarios. In stark contrast, both the Heuristic and MAPPO-G exhibit sharp deterioration in the 5 UEs scenario, with fairness falling below \(0.6\). Accordingly, the consistently high performance across different network scales demonstrates our framework’s strong generalization capability, operating on dynamic network topologies without the need for retraining.

Fig. 7 illustrates the convergence of the system-wide utility during the PPO training. The utility steadily improves and converges to approximately \(0.8\) across all evaluated scenarios after approximately \(1,250\) training epochs, empirically validating the convergence guarantee established in Corollary 1.

5.2.2 Contribution of SG Mechanism↩︎

To validate the superiority of our proposed MFSG mechanism, we conduct an ablation study comparing its performance against a centralized “Dictator” scheme. In this baseline, UEs (followers) forfeit their autonomy, rigidly adhering to the BS’s (leader’s) scheduling commands without local optimization. As depicted in Fig. 8, the SG framework yields average gains of approximately \(24.74\%\) in RBG efficiency and \(17.58\%\) in system utility. This performance gap confirms UEs are not merely passive executors. Unlike the rigid Dictator baseline, our framework empowers UEs to leverage their private local information for fine-grained decision-making, thereby handling heterogeneous network complexities more effectively.

5.2.3 Impact of Heterogeneous Traffic↩︎

To validate robustness under realistic conditions, we extend the traffic model beyond the standard Bernoulli process⁷. We introduce periodic traffic to represent URLLC tasks characterized by fixed arrival intervals, and bursty traffic to model eMBB tasks like web browsing or variable bit rate (VBR) video, formulated as an on-off process with high arrival rates during active periods [51]. Table ¿tbl:tab:mixed95traffic95results? details the system performance under these traffic compositions with \(5\) UEs. In the baseline homogeneous scenario (Case 1), the system exhibits a low average latency of \(2.95\) slots and consistent queue lengths. When deterministic periodic traffic is introduced (e.g., Case 3), the average latency and queue occupancy further improve by approximately \(2.7\%\) and \(18.9\%\), respectively. This enhancement suggests that the framework may be implicitly adapting to the regularity of periodic flows, thereby mitigating contention for the remaining users. The inclusion of bursty traffic (e.g., Case 5) imposes significant stress on the network, evidenced by the rise in the 95% tail latency to \(10.03\) slots and increased maximum queue lengths. Even in Case 6, the fully heterogeneous setting, the protocol achieves a balanced performance with a moderate latency increase of \(16.9\%\) and queue length increase of \(13.6\%\) compared to the baseline. These results corroborate that the framework maintains system stability without performance collapse, demonstrating robust adaptability to diverse and complex traffic dynamics.

5.2.4 Impact of Different RBGs↩︎

Fig. 8 evaluates the framework’s scalability by varying the number of RBGs (\(M\)) from 3 to 8. The throughput analysis (left) confirms that our method scales most effectively with increased resources, peaking at approximately 0.90 when \(M=8\). This performance significantly outperforms the strongest baseline, G-MAC (0.83), and surpasses the Heuristic and MAPPO-S methods by wide margins. Notably, MAPPO-G exhibits a generalization failure in high-resource settings, with throughput dropping sharply to 0.43 at \(M=8\). Regarding fairness (right), our approach demonstrates superior robustness, consistently maintaining an index above 0.92 across all scenarios. In contrast, while G-MAC offers competitive throughput, its fairness remains comparatively low (peaking at \(\approx 0.70\)); similarly, MAPPO-S suffers a significant fairness decline as \(M\) increases. Overall, our method achieves the optimal trade-off between aggregate throughput and equitable resource distribution.

Fig. 9 evaluates the framework’s performance and adaptability when varying the number of RBGs while maintaining other configurations unchanged. The left chart indicates that our method scales most effectively with increased resources, yielding a substantial performance gain of \(52.5\%\) as \(M\) expands from \(3\) to \(8\). With \(M=8\) RBGs, our method achieves the highest throughput, outperforming G-MAC and MAPPO-S by around \(7.9\%\) and \(36.4\%\), respectively. Conversely, MAPPO-G exhibits a severe generalization failure with the performance degrading paradoxically as resources increase. The fairness results on the right further highlight the robustness of our approach, which maintains a high fairness index regardless of the number of RBGs, ensuring equitable resource distribution. While G-MAC provides competitive throughput, its fairness fluctuates significantly, lagging behind our method by roughly \(24.7\%\) at \(M=8\).

5.2.5 Impact of Number of UEs↩︎

Fig. 10 evaluates the throughput scalability of the proposed framework against baselines as the network density increases from \(2\) to \(8\) UEs. Our method demonstrates superior robustness in all scenarios. Specifically, at the peak load of \(8\) UEs, the proposed algorithm achieves substantial throughput gains of \(38.31\%\) and \(84.12\%\) over the strongest baselines, G-MAC and MAPPO-S, respectively. In contrast, Heuristic and MAPPO-G suffer severe degradation in dense networks, plummeting to throughput levels that are \(74.76\%\) and \(93.79\%\) lower than the proposed scheme, respectively. These results validate that the LLM-empowered protocol optimizes channel utilization more effectively than standard MARL and heuristic schemes under dynamic, large-scale conditions.

In Table 2, we extend the evaluation to a multi-cell setting by adopting the hyper-cellular network (HCN) architecture [52], which consists of one controlling BS (CBS) providing continuous macro coverage and \(N\) data BSs (DBSs) deployed for localized capacity. Each DBS cell serves four local UEs and one shared edge UE with heterogeneous traffic intensities \(p_{a}\in\{0.1,0.2,0.4,0.6,0.8\}\). This topology enforces active inter-cell coupling, as shared UEs create dynamic load dependencies between adjacent DBSs, necessitating coordinated resource partitioning by the CBS. The total resource pool scales linearly with the network size, maintaining an average of \(M=5\) per cell. At the network level, the CBS partitions the global RBGs among the DBSs using three policies – Equal (static), Proportional (demand-aware), and Random – while each DBS acts as the local leader, performing inference and executing the Stackelberg game following the same procedure as in the single-cell experiments. By embedding our framework into the HCN architecture, the system demonstrates robust scalability with near-linear throughput growth, proving that the locally learned MAC policy transfers effectively to larger, CBS-coordinated multi-cell deployments. More related discussion is provided in Appendix 10.1.

5.2.6 Impact of Policy Backbones↩︎

In Fig. 11, we investigate the impact of the underlying policy architecture on our framework. We introduce a “Vanilla Transformer" baseline, which comprises standard, randomly initialized self-attention layers capable of processing variable-length sequences, and compare it against pre-trained LLMs ranging from varying parameter sizes. Detailed experimental setups are provided in the Appendix 8.6. As shown in Fig. 11 (Left and Middle), the vanilla transformer consistently underperforms compared to other pre-trained counterparts, showing a tendency to overfit and poor generalization. This gap is further elucidated by the training dynamics in Fig. 11 (Right), where the baseline appears to plateau earlier. This implies that pre-trained priors may offer a superior initialization, potentially steering the optimization trajectory away from the premature stagnation observed in the scratch-trained baseline. Moreover, the performance across different LLMs (from GPT-2 Small to Llama 3.2-1B) is remarkably consistent, suggesting that the task does not exhibit a strong scaling law; rather, a certain threshold of model capacity is sufficient.

5.2.7 Impact of TBLER↩︎

Fig. 12 assesses the robustness of the proposed framework under varying TBLER conditions⁸. The results indicate that as the TBLER increases from \(0.0001\) to \(0.5\), the system throughput exhibits a smooth decline from approximately \(0.7\) to \(0.32\), in line with theoretical expectations. Notably, even under severe channel degradation where throughput is nearly halved, the fairness index consistently remains above \(0.92\), thereby substantiating the robustness of the learned MAC protocol.

6 Conclusion and Future Works↩︎

In this paper, we have focused on the critical challenge of autonomously establishing generalizable and adaptive MAC protocols for dynamic wireless networks, where traditional MARL methods require costly retraining and redesign. We have proposed a novel LLM-empowered framework that recasts protocol emergence as a dynamic MFSG, capturing the intrinsic hierarchical interactions among network agents. By employing LLMs as policy networks within a PPO-driven RL loop, the framework natively handled variable-length inputs and outputs while preserving the exploratory learning required for protocol discovery. Theoretical analyses have guaranteed the existence of equilibria the local convergence of the proposed method under specific assumptions. Simulation results have demonstrated that our method can robustly adapt to dynamic network conditions and consistently outperform various heuristic and conventional MAPPO baselines across diverse KPIs and environmental settings. Future work will investigate the longitudinal stability of distilled edge policies under repeated distillation–deployment cycles, with particular attention to teacher-anchored refresh strategies for mitigating potential policy drift.

10.3 Feasibility and Practical Deployment↩︎

The practical implementation of LLM-empowered MAC protocols hinges on addressing computational cost, communication overhead, and inference latency constraints. Regarding computational feasibility, the intensive training process can be offloaded to cloud clusters with abundant power and computing resources (e.g., edge servers with high-end GPUs), while model compression techniques[66] and hardware evolution [67] are rapidly bridging the gap between theoretical frameworks and real-world deployment. For example, recent benchmarks on consumer-grade system-on-chips (SoCs), such as Snapdragon 8 Elite Gen 5 [68] and Apple A19 [69], demonstrate the capability to execute 7–13B parameter models locally. For ultra-dense networks, as discussed in Appendix 10.1, future architectures can leverage hierarchical MAC protocols [70] or mean-field games [62], [63] to further abstract individual interactions, thereby maintaining performance stability without prohibitive computational growth.

As for communication overhead, a distinction must be drawn between internal agentic reasoning and physical transmission. While the input state space utilizes verbose textual descriptions to facilitate semantic generalization, the output signaling (i.e., UCMs and DCMs) is strictly confined by the PAG to a discrete, low-dimensional subspace. Specifically, the BS’s macroscopic scheduling intent is physically transmitted as a sequence of \(M\) token indices, yielding a core payload of \(M \times \lceil \log_2(I_t + 1) \rceil\). In our experimental configuration with \(5\) RBGs and \(5\) UEs, this corresponds to a compact payload of approximately \(15\) bits. Even when scaling to larger scenarios (e.g., \(M=20, I_t=20\), resulting in approximately \(100\) bits), the payload remains manageable. Crucially, the base payload fits comfortably within a single 5G NR control channel element (CCE), which comprises 72 resource elements (REs) and can carry over 100 encoded bits under QPSK modulation [71], [72]. For larger payloads, standard CCE aggregation techniques (e.g., aggregation level 2 or 4) can be employed. This implies that the signaling overhead does not require new physical channels or excessive bandwidth and can be seamlessly embedded within standard 5G NR PDCCH structures with high reliability and negligible transmission latency.

Given the stringent latency requirements inherent to MAC protocols, our proposed framework is primarily envisioned for offline protocol discovery and near-real-time control updates. In this deployment paradigm, the comprehensive LLM operates in the cloud or during an offline phase to discover and adapt optimal protocols [73]. Once a stable policy emerges, its behavior can be distilled into a lightweight, specialized neural network (student) [30], [31] or a symbolic rule set [23] that runs efficiently on specific hardware with microsecond-level latency. We have preliminarily explored this concept in Appendix 9.3, where our lightweight student model can effectively inherit the generalization capability of the teacher. Furthermore, complementary algorithmic optimizations (e.g., model pruning and quantization) and the integration of AI-specialized accelerators (e.g., NPUs and FPGAs [74], [75]) can be leveraged to further reduce inference time and accelerate the feasibility of real-world deployment.

10.4 Discussion on Semantic PAG-Constrained Protocols↩︎

We clarify that a semantic protocol within this framework denotes a task-oriented representation rather than the standalone lexical interpretability of surface tokens. As detailed in Section IV-B, semantic comprehension occurs during the input phase, where raw numerical states are transformed into language-generalized prompts (Eq. 11) for contextual reasoning. Consequently, while our PAG design (Section IV-C) constrains outputs to numerical \(\mathcal{W}_b^-\) and \(\mathcal{W}_u^-\) tokens, these are not mere heuristic values. Rather, they are structured, executable actions distilled from the LLM’s semantic reasoning. This aligns with recent LLM-empowered MAC designs, which parse natural-language contexts into structured, executable action fields [31], and with language-guided emergent communication, where task-grounded messages are quantized symbols rather than unconstrained text [14], [30]. Translating these emergent representations into human-readable formal logic (e.g., via ProbLog extraction [23]) remains a distinct, orthogonal direction for post-hoc interpretability rather than a prerequisite for semantic protocol emergence.

10.5 Physical Justification of the Weak-Coupling Premise↩︎

While Assumption 3 mathematically structures the local geometry of the gradient field, it is fundamentally an emergent property of the proposed Stackelberg game rather than an arbitrary restriction. In conventional non-coordinated MAC protocols (e.g., ALOHA), dense contention inevitably leads to strong inter-UE coupling, rendering cross-Hessians significant. However, in our hierarchical MFSG, the leader (BS) actively orchestrates resource access by broadcasting allocation intents (DCMs). Near the equilibrium \(\boldsymbol{\varTheta}^*\), the BS policy \(\theta_b^*\) tend to output highly orthogonal intents to maximize system utility, and the followers’ best-response policies \(\theta_u^*\) are also largely aligned with these intents to maximize Eq. 2. Consequently, at \(\boldsymbol{\varTheta}^*\), the agents’ action probability are highly concentrated on disjoint sub-spaces of the available RBGs. Because the UEs are effectively segregated into orthogonal resources, a marginal perturbation in UE \(i\)’s policy near \(\boldsymbol{\varTheta}^*\) has a limited impact on the marginal utility of UE \(j\). Thus, the cross-coupling is substantially weakened near the equilibrium, providing an operational justification for Assumption 3.

10.6 Limitation↩︎

One limitation is our reliance on a simplified OFDMA uplink baseline without complex mobility or inter-cell interference. Additionally, CSI imperfection is modeled as an aggregate BS-UE observation mismatch rather than a decomposed full physical layer process. Future work should involve comprehensive system-level simulations and explicitly parameterize estimation error, reciprocity calibration error, and feedback/processing delay to support PHY-MAC co-design analysis. Furthermore, we have not yet conducted hardware-in-the-loop (HIL) validation or full end-to-end timing profiling on commercial NPU/FPGA-based edge hardware. In practical deployment, the full LLM teacher is expected to be replaced by a distilled, quantized, or cached lightweight executor in the PDCCH slot-critical loop.

References↩︎

---

1. R. Tan and R. Li are with the College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China. (email: {ttrx, lirongpeng}zju.edu.cn?).

   Fei Wang, Chenghui Peng, and Shaoyun Wu are with Huawei Technologies Company Ltd., Shanghai 210026, China. (email: {wangfei76, pengchenghui, wushaoyun}huawei.com?).

   Z. Zhao is with Zhejiang Lab as well as Zhejiang University, Hangzhou 310027, China. (email: zhaozf@zhejianglab.org).

   H. Zhang is with Macau University of Science and Technology, Macau, China (email: hgzhang@must.edu.mo).

   Corresponding Author: Rongpeng Li↩︎

2. While we formulate UEs as homogeneous expected-utility maximizers to preserve analytical tractability and strict convergence guarantees, future extensions could integrate prospect-theoretic primitives [38] to accommodate bounded rationality and behavioral heterogeneity in more practical scenarios.↩︎

3. As the policy \(\pi\) is a direct mapping from \(\theta\), updating the actor network’s parameters constitutes an update to the policy itself.↩︎

4. See Appendix 10.4 for further clarification.↩︎

5. This weak-coupling condition is imposed strictly in a local neighborhood of the equilibrium \(\boldsymbol{\varTheta}^\ast\) rather than arbitrary overloaded contention regimes; it characterizes a coordinated operational regime where the BS’s intents and UEs’ best responses have effectively mitigated persistent RBG overlap, thereby structurally decoupling the agents’ utilities.↩︎

6. We here adopt the leader’s utility for robustness analysis as it explicitly encapsulates the system-wide performance. Here, \(J_{\mathcal{D}}(\boldsymbol{\theta}^*) \triangleq \mathbb{E}_{\boldsymbol{s} \sim \mathcal{D}}[F_{t,b}(\boldsymbol{\theta}^*, \boldsymbol{s})]\), where the state \(\boldsymbol{s}\) abstracts a joint state trajectory over an episode. In this context, network anomalies are rigorously modeled as stochastic perturbations to the trajectory distribution \(\mathcal{D}\).↩︎

7. For detailed experimental settings of these traffic models, please refer to Appendix 8.↩︎

8. As standard practice in system-level link abstraction [4], [5], TBLER serves to model residual MAC-layer packet erasures rather than decomposing PHY-layer sources.↩︎

9. We assume all UEs share the common \(N_m\) for simplicity.↩︎

10. See Appendix 10.5 for a detailed physical justification↩︎

11. This configuration is specifically selected to accommodate the transmission of small, fixed-size data payloads \(\xi^{gen}=256\) bits. These values correspond to the low-index MCS entries specified in 3GPP TS 38.214 [37], representing a robust transmission regime typical of cell-edge users or power-limited IoT devices.↩︎

12. To amplify the conflict between throughput and fairness explicitly, we construct a static stress test scenario. We let five UEs be initialized with sufficiently large buffers (\(b_{i,t}=30,000\)) to ensure continuous contention, and are assigned fixed, highly heterogeneous spectral efficiencies of \(\nu_{i,t}\in [0.15,0.45, 0.85, 1.0, 2.0]\) bits/s/Hz. This setup necessitates a harder decision between maximizing system rate and preventing starvation than dynamic random-arrival settings.↩︎