As modern compute workloads scale significantly, data movement is a growing bottleneck. This impacts FPGAs as well: data must frequently move between various blocks through programmable routing. A characteristic feature of contemporary FPGA architectures is the inclusion of dedicated direct interconnects between homogeneous blocks within a column, which can be exploited to reduce reliance on the programmable routing network thereby reducing data movement. These include cascading BRAMs into deeper logical memories, chaining DSPs for MAC operations and CLB carry chains for larger adders [1], [2]. However, these hardened paths do not address data movement between heterogeneous blocks, particularly between BRAMs and DSPs, which still traverses the global routing network.

This limitation is acute in Deep Learning (DL) inference, which is dominated by General Matrix–Vector Multiplication (GEMV). In a GEMV operation, a DSP multiplies a BRAM-resident weight against a runtime-streamed activation through its two input ports. Because DL accelerators [3] instantiate thousands of such BRAM-DSP pairs operating in parallel, the resulting high-volume BRAM-to-DSP traffic has no direct path, so it traverses the global routing network, creating routing congestion and lengthening the critical path. Recent in-/near-BRAM compute architectures cut this data movement but require fundamental changes to the BRAM architecture & circuitry as well as the CAD toolchain, raising area, programming, and commercial-viability concerns [4], [5].

To enable efficient BRAM-to-DSP data transfer without disrupting existing FPGA development flows, we propose an enhanced FPGA architecture that has direct paths between BRAMs and DSPs. BRAM output is directly connected to the input of a DSP a few columns away, directly facing this BRAM. Importantly, the original BRAM and DSP routing interfaces are preserved, maintaining full backward compatibility. Since current CAD flows support only homogeneous (same-type) direct interconnects, they cannot correctly leverage the proposed heterogeneous (cross-type) connections, leading to placement and routing failures. We enhance the placement engine with simple updates to fully utilize the proposed cross-type paths resulting in no failures and yielding better Quality-of-Results (QoR).

Our contributions are:

• Architecture. The first hardened, cross-type direct path between BRAM output ports and DSP inputs, with negligible area and timing overhead and full backward compatibility with default BRAM/DSP interfaces.

• CAD. An additive placer extension that recognizes cross-type direct connections as placement macros and updates the placement priority of the remaining macros after every successful placement.

• Quantified benefit. Evaluation shows up to \(+25\%\) Fmax and \(-49\%\) wirelength on DL layer designs, with no degradation on non-DL designs.

Same-type Direct Paths. Modern FPGAs are heterogeneous, tile-based fabrics whose CLB, BRAM, and DSP columns communicate over a flexible but expensive island-style routing network that occupies over half the area and dominates critical-path delay [6]. To sidestep this cost, vendors harden direct interconnects between same-type resources within a column (Fig. 1): BRAMs cascade into deeper logical memories, DSPs chain to forward partial results, and carry propagation when CLBs are configured as adders. Prior works also leverage these chain optimize their placement to hold high frequency under heavy congestion [7], [8]. However, these changes remain on the same logic resources, we instead harden a BRAM–DSP path to improve efficiency of traffic between different logic resources.

In/near BRAM compute. Recent proposals push compute into or near the BRAMs and make invasive changes to the BRAM. Some add bit-serial processing elements into the BRAM tile, while other adds a small SRAM buffer and custom ALUs near the BRAMs [4], [5]. Beyond the additional area and power overhead, such architectures need a new programming model and are incompatible with existing FPGA toolchains. Our work instead proposes minimal architecture change that come with full-compatibility to existing CAD flow.

At the column level, two BRAM variants, BRAM-A and BRAM-B, are interleaved in each BRAM column, alternating vertically (Fig. 2, left). Both variants retain full cascade capability and the standard output that drives the general routing, so existing BRAM-DSP communication patterns continue to work unchanged. BRAM-A additionally exposes a directout port directly connected to the paired DSP, bypassing the general routing. At the port level (Fig. 2, right), a multiplexer (mux) splits BRAM-A’s output between the out path and directout; symmetrically, the DSP’s first input is selected between the dedicated directin and the standard input. Both muxes reside inside the BRAM and DSP tiles themselves, consuming no routing resources, and are configured per design instance, so an application not using the direct path falls back to the standard datapath with no netlist or programming change. The horizontal distance \(d\) between the BRAM and the DSP is an architectural parameter. Fig. 2 illustrates \(d = 2\), but the column interleaving generalizes to any value, which we sweep in our evaluation in Section [sec:sec:results] to characterize how the direct-path benefit scales with the BRAM–DSP distance.

The hardened connection only carries a signal when the paired BRAM and DSP sit at a fixed \((\Delta x, \Delta y)\) offset, where \(\Delta x\) is the BRAM–DSP distance. Baseline VTR places macros individually, with no architecture- or connectivity awareness that the two must be aligned. Because BRAM and DSP tiles also occupy different \(y\)-offsets in the grid, the logical BRAM-DSP pair in the design almost never lands at the required position during placement. The direct path is unroutable on most runs especially for large designs, and the succeeded runs leave the surrounding logic far from BRAM-DSP pair, inflating wirelength and degrading Fmax. We add two extensions, at macro creation and at initial placement (Fig. 3), leaving the annealing engine and router untouched, so gains reflect the architecture, not a retuned optimizer.

Cross-type macros. We introduce cross-type macros that bind a BRAM-DSP pair or a BRAM-BRAM-DSP triple when cascading at the architecturally-mandated \((\Delta x, \Delta y)\) offset during macro creation. The macro moves as a rigid unit through both initial placement and simulated annealing, so the required adjacency is guaranteed for the entire flow.

Connectivity-driven ordering. VTR places macros in priority order, where a macro’s priority rises with \(\rho\), its number of already-placed neighbors; so, heavy-connected macros are placed early and their neighbors are tightly clustered. But VTR refreshes \(\rho\) only after a macro is placed by centroid heuristic, which itself needs an already-placed neighbor; early placements therefore fall back to random or exhaustive search, \(\rho\) stalls, and the initial macros scatter. We refresh \(\rho\) after any successful placement, weighting each increment by \(w(M)\): \[w(M)= \begin{cases} w_{\max}, & \text{if } M \text{ is a cross-type direct macro,}\\[2pt] 1, & \text{if } \beta\log_2(1{+}\phi(M))+\gamma\log_2(1{+}|M|)\ge T,\\[2pt] 0, & \text{otherwise,} \end{cases} \label{eq:c4weight}\tag{1}\] where \(\phi(M)\) is the summed member count of macros \(M\) connects to (higher when one is a cross-type macro). \(|M|\) is \(M\)’s own member count (a cascaded BRAM pair connected to a DSP gives \(|M|=3\) ). \(\beta\) and \(\gamma\) weight the two log terms and \(T\) sets which same-type macros earn the unit increment. A cross-type macro saturates at \(w_{\max}\) and is placed before the surrounding region fills. The tuple \((w_{\max},\beta,\gamma,T)\) is exposed as a set of tunable knobs that trade a tighter clustering for cross-type macros and their connected macros against a looser one.Together, the extensions yield a more structured layout, raise routing success on cascade and direct-connection designs.

Tools, architecture, and metrics. We use VTR to evaluate the architecture and CAD enhancements. All experiments use the Agilex-10-like template [9] with DSPs in int_sop_4 mode and BRAMs in 512\(\times\)​40 mode, and we report two primary metrics: frequency (Fmax) and total routed wirelength (WL). Design Compiler [10] on FreePDK45 [11] confirms the added multiplexers and direct path are negligible in delay and area relative to the blocks. For simplicity, we model the direct path as the same wire delay as regular path with no added area. Precisely modeling the direct path’s delay and area would require circuit-level simulation and is left to future work. The BRAM-DSP distance \(d\) is the swept parameter: the main results sweep it while all other experiments fix \(d=4\), the architecture’s default. All experiments use a fixed grid sized so every benchmark sits at 70-80% utilization, typical of real deployments, away from both extreme routing congestion. Each configuration reports the average frequency and wirelength over 20 random seeds; because the seed strongly perturbs VPR’s initial placement, we assume a \(\pm5\%\) band as seed noise and ignore any \(\Delta F_\mathrm{max}\) within it.

Benchmarks. We evaluate the proposed architecture and CAD flow on three types of common DL layers. Table 1 reports the resource usage of all six configurations. Each DL layer keeps its weights in BRAM and streams activations from the top module ports. FC is a fully-connected layer (GEMV-like, with ReLU activation and LayerNorm); Conv is a single convolution layer with ReLU and BatchNorm; and Attention is from BERT-Tiny with multiple GEMMs and softmax. We choose each layer’s dimensions so that it instantiates two weight-storage configurations: a small variant whose weights fit in a single \(512\)-deep BRAM and a large variant whose weights require cascaded BRAM pairs. Unlike FC and Conv, where enlarging the weight matrix spills weights into a cascaded BRAM at a fixed DSP count, Attention’s projection weights are fixed with model size; its only scalable on-chip memory is the intermediate buffer between GEMM and softmax, which grows with the sequence length \(N\). We therefore double \(N\) to deepen that buffer into a cascaded chain, reusing the same weights and DSP array. Hence Attention’s BRAM grows without simply doubling.

Although the proposed architecture targets DL inference, its minor fabric changes could disturb place-and-route on broader workloads. To rule this out we evaluate five resource-diverse non-DL VTR-suite designs of varied size and LUT/DSP/BRAM mix: or1200, mkSMAdapter4B, bgm, LU8PEEng, and stereovision2.

Fig. 4 compares the proposed system (proposed architecture \(+\) proposed CAD) against the baseline system (baseline architecture \(+\) default VTR). Wirelength is consistently reduced, by up to \(\sim\)​49% as expected, but frequency gains are not uniform: the critical path depends on each design’s RTL, and the BRAM-DSP connection is never itself critical in any of the six benchmarks, the soft logic is. FC and Conv run only lightweight activation and address generation, so their critical path is the placement-sensitive address-generator fanout, which the direct path’s tighter placement shortens up to \(+25\%\). Attention is instead bound by the softmax reduction tree and LayerNorm variance computation, which the direct path cannot touch, so its frequency stays flat. Larger benchmarks are expected to achieve similar wirelength reductions; frequency gains will depend on the critical path. However, the proposed architecture offers a broader benefit by reducing pressure on the global routing fabric, potentially improving overall routability and timing closure across the design. Investigating these effects on larger benchmarks such as Koios [9] and Titan is future work [12].

In addition to DSP chains and BRAM cascades, the proposed direct BRAM–DSP connection enables larger heterogeneous macros, e.g., a U-shaped macro with multiple cascaded BRAMs in one column, a BRAM-DSP direct link, and a DSP chain in another column. Such macros are harder to relocate during simulated annealing, since they induce much larger placement perturbations. Fig. 5 sweeps DSP-chain length and BRAM-cascade depth on a synthetic benchmark exercising the direct connection; the baseline is the same architecture under default VTR. We observe that even as the fused macro grows to the largest cascade and chaining size, it stays placeable across the entire grid and frequency stays similar to the baseline, imposing no placement penalty under high utilization.

To better assess the cost of proposed architecture and CAD changes, we evaluate the frequency and wirelength of proposed architecture in non-DL VTR benchmarks, as well as the runtime and memory footprint of proposed CAD. Fig. 6 shows the proposed architecture leaves both frequency and wirelength essentially unchanged from the baseline, preserving the FPGA’s general-purpose flexibility. For CAD, running the proposed and the baseline CAD on the same idle server across all benchmarks yields nearly identical runtime and memory footprint (plots omitted for space).

We present a lightweight FPGA enhancement: a hardened cross-type BRAM–DSP direct path and an additive placer extension for the resulting macros, fully compatible with existing FPGA CAD flows. On an Agilex-10-like template it delivers up to \(+25\%\) Fmax and \(-49\%\) wirelength on DL layers, while leaving non-DL benchmarks and the baseline CAD flow performance essentially unchanged. Future work will involve evaluating the proposed architecture and CAD changes using larger benchmark suites such as Koios [9] and Titan [12].

References↩︎