1 Introduction↩︎

As many-core SoCs continue to scale [1], the memory traffic generated by large numbers of processing elements increases sharply, making high-bandwidth and scalable nocs essential for sustaining performance. At the same time, providing industry-standard AXI4 [2] interfaces at the network boundaries remains critical for seamless integration of existing IPs, usually designed with AXI4 initiator and/or target interfaces [3]–[5].

Deadlock freedom remains a central requirement in nocs, both at the routing and protocol levels [6]–[8]. Endpoint behavior in AXI4-based systems creates protocol dependencies that are invisible to the routing algorithm itself. Thus, even when deterministic XY routing is employed, protocol-level deadlocks may still arise if AXI4-capable endpoints internally couple read and write transactions [7]. [fig:deadlock] illustrates a representative case in which a DMA engine (AXI4 initiator) issues a read burst request to a remote memory . As the read data returns, each AXI4 beat is immediately forwarded into a write burst directed to a local L1 spm (AXI4 target) through the AXI4 crossbar . If, before the DMA completes, an external initiator issues a write burst to the same L1 spm , its request may occupy the local noc link . The DMA can then no longer make forward progress on the read-response path , and a protocol-level circular wait arises at the network endpoint, stalling the entire system. More generally, the challenge of protocol- or message-dependent deadlocks caused by the interaction between nocs and endpoint protocols has long been recognized in the literature [7]–[9]. To eliminate protocol-level deadlocks without constraining endpoint behavior, AXI4 read and write channels must also be decoupled so that they cannot block one another on the same noc link.

One solution is to use vcs [10], [11]. As shown in [fig:old-router], vcs provide separate buffers for different data streams while preserving a shared physical link. In our case, AXI4 read and write transactions can be assigned to distinct buffers, so that returning read data and incoming write data no longer compete for the same downstream path in the local interconnect. A flit may occupy the shared link only if the corresponding downstream buffer has free space, ensuring that the link is released in the next cycle. A second solution is the multiplane approach [12], [13], which assigns different traffic classes to separate physical links, thereby avoiding inter-class blocking on shared links.

Prior work has established the conventional view that vcs are more complex than multiplane designs [13]–[15], largely because earlier studies considered classical vc routers targeted at improving link utilization, which incur significant control overhead. Under this assumption, paying for additional link resources has often been seen as preferable to the area and timing cost of full vc support.

However, the design space of modern nocs is shifting, fundamentally changing the trade-off between router complexity and link resources ([fig:challenges]). As accelerator-centric systems continue to scale, integrating hundreds of thousands of AI cores, on-chip networks must sustain aggregate bandwidth demands reaching tens of   [16]–[19]. To sustain this bandwidth, noc link widths have therefore increased, both in proprietary, closed-source solutions [20]–[23] and in open-source designs such as FlooNoC [24]. The latter stands out as the first open-source AXI4-compliant noc, supporting high-throughput transfers with links up to 1024 bits [25]. At such link widths, the routing-resource cost of physical-plane duplication becomes critical. Moreover, as these fabrics scale toward wafer-scale integration, reliability also becomes increasingly important. Achieving reliability often relies on redundancy, with examples of triple-redundant interconnects incurring up to a 3\(\times\) increase in link width, further exacerbating wiring demand and routing pressure [26]–[30].

Under these stringent constraints, vc implementations re-emerge as a promising option to preserve protocol-level deadlock freedom while saving valuable routing resources. To this end, we propose Preemptive vcs, a novel lightweight vc router design for deadlock-free AXI4 traffic separation which preserves the resource parsimony of vcs while approaching the area and timing characteristics of multiplane router designs.

To summarize our contributions, we:

• Extend an open-source AXI4 NoC design (FlooNoC [24]) with four deadlock-free AXI4 traffic-class separation schemes, including a multiplane design and three lightweight vc-based designs. Among these, we propose a novel design, Preemptive vcs, which balances timing, area, routing resources and bandwidth utilization ¹.

• Integrate the proposed designs into a representative mesh-based soc to demonstrate their applicability in a realistic AXI4-capable tiled system ².

• Evaluate timing, area, routing resource usage and bandwidth utilization for all designs in TSMC 7 nm, showing that Preemptive vcs can save 76% of routing resources while maintaining comparable frequency and incurring only 3% area overhead relative to the multiplane solution.

2 Related Work↩︎

Originally introduced to break cyclic channel dependencies and guarantee deadlock-free routing [10], vcs were soon adopted to improve link utilization by allowing multiple logical flows to share the same physical link [11]. This evolution led to the conventional four-stage router microarchitecture ([fig:old-router]), designed to dynamically map incoming packets to the available vcs. In this architecture, routing computation (RC) and switch allocation (SA) must be complemented by substantial vc control and arbitration logic to support dynamic allocation (VA). Consequently, interest in reduced-complexity vc designs has resurfaced to mitigate this overhead. Xu et al. [31] simplify vc and switch arbitration through partially constrained allocation schemes, but still retain a dynamic allocator to adapt vcs to traffic fluctuations. Shim et al. [32] statically bind vcs to source–destination traffic paths encoded in routing tables, targeting routing optimization rather than traffic-class separation. In contrast, our work targets protocol-driven AXI4 traffic-class decoupling: by statically associating each logical channel with one vc, much of the allocation, control, and arbitration complexity present in conventional and prior reduced-complexity vc routers can be removed, enabling lightweight vcs for deadlock-free traffic-class separation..

Multiple physical networks, or multiplanes, have long been used as an alternative to vcs to separate interdependent traffic classes and avoid protocol-level deadlock. Hansson et al. [7] note that many nocs eliminate request–response dependencies by introducing separate physical networks. The trade-off between vcs and multiplanes was previously studied by Yoon et al. [13], who showed that multiplanes can achieve simpler routers and higher frequency than classical vc routers, at the cost of fabric duplication. However, while their study targets conventional dynamically allocated vc routers, our work revisits this trade-off under different design constraints, where FlooNoC’s lightweight single-cycle router ([fig:floonoc]) and the static use of vcs for AXI4 traffic-class separation fundamentally change the cost–performance balance.

Following the multiplane approach, other works have adopted multiple physical networks for traffic-class separation. In the gpgpu domain, designs such as RAPID [33] and DA2 [34] use two separate 128-bit networks to exploit the request–reply traffic asymmetry of gpgpu memory systems. Similarly, tiled manycore systems such as OpenPiton [35] and ESP [36], [37] rely on multiple physical networks, using, respectively, three and six 64-bit physical planes to provide bandwidth and avoid deadlock. While multiplanes have been adopted successfully for message-class separation, they have mostly been demonstrated on relatively narrow links, which are not suited for modern large-scale, data-intensive AI-oriented socs. In such systems, wide links up to 1024 bits are common to sustain high-bandwidth data movement [20]–[23].

FlooNoC [24] stands out as the first open-source AXI4-compliant noc targeting AI accelerators with very wide links. As shown in [fig:floonoc], it implements two logical networks, one wide (up to 1024 bits) and one narrow (typically 64 bits), on top of three shared physical planes: one narrow channel for AXI4 requests (narrow and wide AW and AR, and narrow W), one narrow channel for AXI4 responses (narrow and wide B, and narrow R), and one wide channel for AXI4 data (wide W and R). This organization makes FlooNoC’s original multiplane choice inexpensive: only narrow traffic is physically separated, while the costly wide data plane remains shared between AXI4 read and write data streams. In the deadlock scenario of 1, however, the wide AXI4 read and write data streams must also be separated. A pure multiplane solution would therefore duplicate the wide tile-to-tile link, sharply increasing routing pressure.

3 Architecture↩︎

The deadlock scenario in [fig:deadlock] stems from the interaction between wormhole-routed AXI4 transactions and IP blocks, such as DMAs, that internally couple read and write streams. This coupling can create circular dependencies and lead to protocol-level deadlocks, which can be avoided by breaking at least one of Coffman’s conditions [6]. We study the design space through a physically-aware lens, since the router-complexity vs. link-resource trade-off discussed in 1 is critical to system-level feasibility. We evaluate four architectural solutions (3), each targeting a different deadlock condition. We focus on wide data routers and the respective tile-to-tile links, which represent the main bottleneck for decoupling AXI4 read and write data streams in large accelerator-centric systems.

3.0.1 Multiplane↩︎

The baseline solution extends the multiplane approach adopted in FlooNoC. While FlooNoC already separates narrow AXI4 request and response traffic, it uses a single shared wide plane for both read and write channels. To decouple these traffic classes, we instantiate two independent wide planes, one for read and one for write data, each with its own tile-to-tile physical link and router datapath. As a result, read and write data streams no longer contend for the same wide physical channel, breaking Coffman’s mutual exclusion condition. The main cost is the duplication of the wide tile-to-tile link, which doubles the number of wide pins and routed tracks crossing tile boundaries. Since the wide link accounts for the majority (78%) of the tile-to-tile link resources [24], the multiplane solution substantially increases routing pressure.

3.0.2 Naive↩︎

To reduce routing overhead, an alternative approach is to use vcs. We first evaluate a Naive valid/ready implementation [38], where a single physical data link is shared across vcs , while the valid and ready handshake signals are replicated per vc ³. This allows the sender to select the active stream and the receiver to apply independent backpressure for each traffic class. To break Coffman’s hold and wait condition, data is injected onto the shared physical link only when the corresponding downstream receiver is ready, by masking the upstream valid with the downstream ready. However, this introduces a ready-to-valid combinational dependency, creating a logic path from the receiving router, through the transmitting router, and back to the receiving router . In tile-based architectures, this path spans twice the tile width (d), making physical delay a critical timing contributor and significantly limiting the achievable frequency.

3.0.3 CreditBased↩︎

To remove the ready-to-valid combinational dependency, a CreditBased protocol can be adopted by extending the router interface with a credit signal that indicates flit consumption in the downstream input buffer [39]. Each output vc has a credit counter , initialized to the downstream FIFO depth (4), decremented when a flit is transmitted , and incremented when the downstream router returns a credit after the corresponding FIFO entry is freed . A flit is injected onto the link only if the corresponding counter is non-zero , guaranteeing downstream buffer space and breaking Coffman’s hold and wait condition.

Compared to the Naive solution, this scheme removes the long combinational backpressure path by registering the downstream credit. However, the returned credit is visible upstream only one cycle after the downstream flit consumption , delaying the counter increment. As shown in 4, this credit delay can reduce throughput by up to 33%. To hide it and preserve full bandwidth, the downstream FIFO must absorb one additional flit, implying a minimum depth of three flits .

3.0.4 Preemptive↩︎

To the best of our knowledge, prior vc designs have not targeted Coffman’s no preemption condition. We therefore propose a novel Preemptive scheme: any valid input may acquire the shared physical link, but can be preempted by another valid stream if its downstream receiver is not ready. Link ownership is selected through round-robin arbitration among the valid streams, while the downstream ready signals are used to determine whether the currently selected stream may retain the link in the following cycle.

Compared to the Naive solution, this approach restores the target frequency by removing the long ready-to-valid combinational path. This is achieved by registering the downstream ready , preventing it from driving valid combinationally: ready only affects next-cycle link ownership, allowing a stalled stream (ready == 0) to be preempted in the next cycle. At the same time, unlike the CreditBased design, the Preemptive solution does not need additional input buffering to sustain full bandwidth. The sender does not wait for delayed credit availability before driving the link; instead, it speculatively asserts valid for the selected stream. If that stream cannot make progress (ready == 0), its flit remains buffered at the source side and another valid stream with an available downstream receiver can take over the shared link in the next cycle. Thus, the feedback-latency window that the CreditBased scheme must hide with a deeper downstream FIFO is avoided.

4 Results↩︎

To evaluate the proposed designs in a realistic AXI4-capable tiled system, we integrate them into a \(4 \times 4\) mesh soc, where each tile contains a Snitch cluster [40] connected to a FlooNoC network extended with the deadlock-free wide-router variants.

Performance is evaluated through cycle-accurate simulation in QuestaSim 2023.4, while physical implementation is carried out in TSMC 7 nm technology using Fusion Compiler 2024.09 under worst-case conditions (SS, \(-40^\circ\)C, 0.675 V).

4.1 Performance evaluation↩︎

To assess the impact of the four router designs, we consider a representative use case in which data are broadcast in burst mode to all 16 tiles using a binary-tree algorithm [41]. 5 shows the runtime for different transfer sizes. To highlight the impact of insufficient buffering in the CreditBased design, we also include a configuration with only two input buffers.

All designs except for the two-buffer CreditBased configuration achieve the same performance as the multiplane solution, showing that lightweight vcs do not introduce runtime degradation. As discussed in 3, the two-buffer CreditBased design suffers from a 33% throughput loss, translating into an asymptotic 1.5\(\times\) increase in runtime (3 instead of 2 cycles for every 2 data beats). For small transfers, the reduced link utilization is partially hidden by fixed costs, such as transfer setup, round-trip latency, and tile synchronization, while for larger sizes the runtime overhead reaches the expected 1.5\(\times\).

4.2 Area and timing evaluation↩︎

In large-scale systems, tile-to-tile wire delay has a major impact on the achievable frequency, since noc links typically span the full tile width and may extend even further in low-diameter noc topologies with bypass connections [42]. At the same time, the exact tile dimensions depend on the target system, and the corresponding tile-to-tile link length may vary accordingly. Moreover, performing place-and-route on the complete cluster tile would make the critical path strongly dependent on the internal compute logic of the cluster, such as the FPU, caches, or other local blocks, thereby obscuring the actual timing impact of the proposed router implementations. For this reason, we place and route two FlooNoC routers at a representative tile-to-tile distance of 750 μm [24], capturing inter-router wire delay and excluding unrelated cluster logic.

We start by assessing the Multiplane solution relative to the FlooNoC baseline [24]. Adding a second 512-bit router doubles the area ([fig:area-breakdown]) while preserving the achievable frequency. However, it increases routing-resource usage by 78% ([fig:routing-resources]), significantly increasing routing complexity.

The Naive vc implementation matches the baseline in routing cost, but the ready-to-valid dependency extends the critical path, lowering the achievable frequency to 1.42 GHz, a 15% reduction relative to the Multiplane solution ([fig:at95plot]). This result highlights the strong impact of tile-to-tile interconnect delay in realistic tile-based systems.

The CreditBased design removes this dependency and restores timing to the level of the Multiplane approach. However, the additional input buffer required to sustain full throughput increases area by 28%, highlighting the high cost of the extra buffering demanded by this solution.

Finally, the Preemptive approach keeps the router area close to that of the Multiplane design, with a negligible \(3\%\) overhead coming from the lightweight arbitration and masking logic. Compared to the Naive vc design, it restores the achievable frequency to the level of the Multiplane approach. At the same time, unlike the Multiplane solution, the Preemptive approach increases routing-resource usage by only 1% over the baseline, due solely to the additional valid and ready signals required for the two virtual channels.

5 Conclusion↩︎

In this work, we presented a detailed analysis of architectural solutions to prevent protocol-level deadlocks in nocs with AXI4 interfaces (1). We proposed Preemptive vcs, a novel lightweight vc implementation that incurs no frequency degradation relative to multiplane designs, with a modest router area overhead of just 3%, while saving up to 76% of link routing resources, a precious commodity in modern nocs with wide, high-bandwidth physical links.

References↩︎