1 Introduction↩︎

Cortical dynamics are scale-free. The power spectrum of the electro- and magneto-encephalogram, of the local field potential, and of the electrocorticogram all follow an aperiodic \(1/f^{\beta}\) law over a broad band [1], and this aperiodic exponent \(\beta\) is not a nuisance: it tracks excitation–inhibition balance, arousal, anaesthesia, and, most cleanly, sleep depth [2], [3]. Deeper non-REM sleep produces a steeper spectrum (larger \(\beta\)), so a sleep-stage label is, physically, a coarse label on a spectral scaling exponent. Alongside the \(1/f\) spectrum, cortex exhibits the other hallmarks of operation near a critical point—neuronal avalanches with power-law size and duration distributions [4] and a branching ratio near unity [5]—which is precisely the regime in which scale-invariance is a genuine property of the data rather than an imposed prior.

This makes neural data an unusually principled testbed for a model whose inductive bias is scale invariance. The renormalization group (RG), the framework physics uses to relate descriptions of a system across scales [6], [7], has a growing set of correspondences with deep learning [8]–[11]. The RG-Flow Transformer operationalizes that correspondence: it runs ordinary associative self-attention in parallel with a scale-aware stream governed by a learnable anomalous dimension \(\gamma\), coarse-grains its representation with block-spin pooling, and gates the two streams with an entropy threshold.

There is a second, more pragmatic reason to expect an inductive-bias advantage here: neural data are scarce. A single night, a single session, a single subject yields limited labelled data, and a correct prior earns its keep exactly when data are too few for a flexible model to discover the structure on its own. A companion study on abundant synthetic data found no accuracy advantage for RG-Flow over a parameter-matched vanilla transformer; scarcity is the regime the present study is designed to probe.

We therefore make three contributions on the real PhysioNet Sleep-EDF corpus, under a strict leakage-free by-subject hold-out:

1. a parameter-matched benchmark of RG-Flow against a vanilla transformer and a hierarchy-only ablation on 5-class AASM sleep staging;

2. a per-subject data-budget sweep that looks for the crossover predicted when data are scarce; and

3. a test of whether RG-Flow’s learned \(\gamma\) tracks the measured aperiodic exponent \(\beta\) out-of-sample—an interpretable, physically-grounded quantity the vanilla model has no analogue for.

The analysis protocol is fixed and fully deterministic, so the underpowered study reported here can be replaced by a larger run without any change to the surrounding argument.

2 Theoretical Framework↩︎

2.1 The scale-free brain as a field↩︎

We map a raw EEG epoch onto a microscopic field configuration \(\phi(t)\) defined at the shortest resolvable lattice spacing (the sampling interval). Following the statistical-mechanics formalism, the underlying distribution of such configurations is governed by an effective data-generating distribution analogous to a partition function, \[P(\phi) = \frac{1}{Z}\, e^{-\mathcal{H}_0[\phi]}, \qquad Z = \int \mathcal{D}\phi\, e^{-\mathcal{H}_0[\phi]},\] where \(\mathcal{H}_0[\phi]\) is an effective microscopic Hamiltonian whose coupling constants dictate short-range interactions. High-frequency fluctuations play the role of ultraviolet (UV) modes at large momentum; low-frequency structure plays the role of infrared (IR) stable states. This analogy is unusually well-motivated for cortical data: the EEG power spectrum is scale-free, following an aperiodic \(1/f^\beta\) law [1], and cortex sits near a critical point with power-law neuronal avalanches [4] and a branching ratio near unity [5]. Scale-invariance is therefore a property of the data, not an imposed prior—which is exactly what an RG-motivated model is designed to exploit.

2.2 Latent Kadanoff block-spin transformation↩︎

To move across observation scales, the neural representation must systematically integrate out short-range fluctuations up to a scaling factor \(s\). Let \(\Lambda\) be the UV cutoff corresponding to the highest resolved frequency. A Kadanoff block-spin transformation coarse-grains the field by integrating out modes \(k>\Lambda'\) with \(\Lambda'<\Lambda\): \[P_{\text{macro}}(\Phi) = \int \mathcal{D}\phi\, T[\Phi,\phi]\, P_{\text{micro}}(\phi).\] The discrete latent mapping realized in our architecture is \[\Phi_T = \sigma\!\left( \mathbf{W}_{\text{macro}} \sum_{t\in\text{block}} \big( V_t\cdot \mathbf{W}_{\text{dis}}\,\phi_t \big) \right),\] where \(\mathbf{W}_{\text{dis}}\) is a learnable disentanglement matrix that orthogonalizes UV noise prior to spatial projection and \(V_t\) is the scale-weight of step \(t\). Because deep NREM sleep steepens the \(1/f^\beta\) spectrum, block coarse-graining suppresses exactly the high-frequency modes whose relative weight distinguishes sleep stages—the operation the label depends on.

2.3 The Callan–Symanzik invariance condition↩︎

A core premise of the architecture is that an authentic structural regime should remain invariant under adjustments to the observation-scale parameter \(\mu\). This is expressed by the Callan–Symanzik (CS) equation for an \(n\)-point latent correlation function \(G^{(n)}\): \[\left[ \mu\frac{\partial}{\partial\mu} + \beta(g)\frac{\partial}{\partial g} + n\,\gamma(g) \right] G^{(n)}(p_i;\mu,g) = 0.\] We do not enforce this equation as a hard invariance; instead we use it as motivation for two soft, differentiable mechanisms (the second contributes a Callan–Symanzik drift penalty to the loss rather than a constraint on the forward pass):

1. The beta-function surrogate (\(\beta(g)\)). In a renormalizable field theory the beta function \(\beta(g)\equiv\mu\,\partial g/\partial\mu\) governs the scale-dependent running of the coupling, and its zeros are the RG fixed points [7]. We do not claim to compute this object. We use a bounded logistic surrogate \(\beta_{\mathrm{s}}(g)=g(1-g)\) acting on the sigmoid-normalized attention coupling \(g\in(0,1)\); it shares the two qualitative properties we exploit—fixed points at \(g\!=\!0\) (irrelevant/decoupled) and \(g\!=\!1\) (marginal/preserved), and a single relevance maximum between them—and defines a differentiable relevance filter \(R(g)=1-|\beta_{\mathrm{s}}(g)|\) that damps couplings far from a fixed point. This is an inductive bias inspired by the RG, not a numerical solution of the RG flow.

2. The anomalous-dimension parameter (\(\gamma\)). By analogy with the field-theoretic anomalous dimension, which sets the scaling correction a field acquires under coarse-graining, we introduce a per-head learnable scalar \(\gamma\) that reweights attention scores by \(\mu^{-\gamma}\) across the scale schedule \(\mu_\ell\). It is a trainable scale-sensitivity parameter motivated by—not identified with—\(\gamma(g)\).

We stress the epistemic status of these components. The mapping between coarse-graining in deep networks and the renormalization group has a rigorous basis in specific cases—an exact correspondence between variational RG and restricted Boltzmann machines [8], information-theoretic real-space RG learned by neural networks [10], normalizing-flow RG [9], and the hierarchical structure that makes “cheap” deep learning effective [11]. Our architecture operationalizes the same intuition as a set of soft, differentiable biases (bounded coupling flow, scale-weighted attention, entropy-gated synchronization, and a Callan–Symanzik drift penalty) rather than as an exact RG transformation. Whether those biases help on real neural data is the empirical question of this paper.

2.4 Why sleep EEG matches the bias↩︎

The aperiodic exponent \(\beta\) of the cortical power spectrum is a genuine scaling exponent, and sleep depth shifts it: deep non-REM sleep has the steepest \(1/f^\beta\) spectrum (Section 3). Two consequences follow. First, the 5-class sleep-staging label is partly a coarse read-out of \(\beta\), so a model that represents scale explicitly has something real to represent. Second, \(\beta\) gives an external, physically-meaningful referent for the model’s learned scale-sensitivity parameter \(\gamma\): if the RG bias is doing what it claims, the learned \(\gamma\) should carry information about the measured \(\beta\). The vanilla transformer has no comparable internal quantity, so this interpretability test is unique to RG-Flow—and is reported (Section 5) regardless of whether it improves accuracy.

3 Methods↩︎

3.1 Data: Sleep-EDF↩︎

We use the PhysioNet Sleep-EDF Expanded corpus (Sleep Cassette study) [12], [13], whole-night polysomnography with expert hypnogram annotations. This draft uses \(5\) subjects (night 1). From each recording we take the two EEG derivations (Fpz–Cz and Pz–Oz) at 100 Hz, band-pass filter to 0.3–35 Hz, crop to the annotated sleep period (\(\pm 30\) min of wake) to remove long lights-on padding, and epoch into 30 s windows aligned to the hypnogram. AASM stages are mapped to five classes (W, N1, N2, N3 with S3+S4 merged, REM); we additionally define a coarse two-class high-\(\beta\) (deep NREM, N2/N3) versus low-\(\beta\) split that mirrors the spectral-depth axis. Every epoch carries its subject identifier for leakage-free grouping.

3.2 Spectral exponent \(\beta\)↩︎

For each epoch we estimate the aperiodic exponent \(\beta\) from the Welch power spectral density of the Fpz–Cz channel using the specparam (FOOOF) spectral-parametrization model in fixed-knee-free mode over 1–35 Hz [14]. Fits below \(R^2=0.90\) are excluded from the regression target; \(97\%\) of epochs pass this cut. The resulting \(\beta\) increases monotonically with sleep depth—median \(\beta\) rises from \(1.58\) (N1) and \(1.67\) (wake) through \(1.84\) (REM) to \(2.43\) (N2) and \(3.17\) (N3)—confirming it as a real, label-linked scale exponent (Fig. 2): the deep-NREM spectrum is markedly steeper than the light stages, exactly the spectral-depth axis the coarse two-class split targets. It serves as both a continuous regression target and the referent for the learned \(\gamma\).

3.3 The Bi-Scale Block architecture↩︎

The RG-Flow engine is a stack of Bi-Scale Blocks. The scale metric \(\mu_\ell\) grows exponentially with block depth \(\ell\) (\(\mu_\ell=\mu_0\rho^\ell\)). Within each block the input representation \(\mathbf{X}_{\ell-1}\) is processed simultaneously by two parallel streams (Fig. 1).

System 1 (associative attention) computes standard contextual weights via multi-head attention (MHA). System 2 (scale-aware RG flow) penalizes query–key dot products by the observation scale \(\mu_\ell\) raised to the learnable anomalous dimension \(\gamma\), \[\mathbf{S} = \frac{\mathbf{Q}\mathbf{K}^{\!\top}}{\sqrt{d_k}\cdot \mu_\ell^{\gamma}},\] and a relevance filter \(\mathbf{R}=1-|\beta(\mathbf{g})|\) acts as a soft noise gate. The two streams merge through the Topological Sync Bridge, which computes the Shannon entropy \(H\) of System 1’s attention distribution and, when \(H>\tau\), injects System 2’s stable macro-attractor back into the stream. Training uses a composite objective balancing cross-entropy, the \(\beta\)-regression MSE, and a multi-scale Callan–Symanzik drift penalty \[L_{\text{Total}} = L_{\text{CE}} + w_{\text{reg}} L_{\text{MSE}} + \lambda_{\text{CS}}\left( \frac{1}{N}\sum_{\ell=1}^{N} \mathbb{V}\mathrm{ar}\!\left[\ln(\mathbf{g}_\ell) - 2\gamma_\ell\ln(\mu_\ell)\right]\right).\]

3.4 Models↩︎

All models share a 1D convolutional patch stem (patch length 30 samples) that reduces each 3000-sample epoch to 100 tokens, so attention is tractable and the front-end is identical across models. We compare three:

• RG-Flow — the full architecture of Section 2 (163,850 parameters).

• Vanilla — a positional-encoded transformer encoder over the same tokens, with width tuned to match RG-Flow’s parameter count (162,910 parameters).

• Ablation — RG-Flow with the scale-aware physics switched off (RG physics disabled, no sync bridge), leaving only the coarse-graining hierarchy (163,850 parameters).

Each has a classification head (5-class stage) and a regression head (continuous \(\beta\)). The near-identical parameter counts make “does the RG bias help” separable from “does capacity help”; the ablation further separates the RG physics from the coarse-graining hierarchy alone.

3.5 Evaluation protocol↩︎

We use leave-one-subject-out group k-fold: each fold holds out one whole subject for test and one for validation, training on the rest. No subject ever appears in more than one split, so there is no by-subject leakage. Models are selected on validation loss (early stopping) and evaluated once on the held-out test subject. We report 5-class accuracy, macro-F1 (robust to the strong class imbalance of sleep data), and the out-of-sample \(\beta\)-recovery \(R^2\), each as mean \(\pm\) 95% CI across \(5\) seeds and all folds.

3.6 Scarce-data budget sweep↩︎

To probe the inductive-bias crossover, we cap the number of training epochs per subject at a budget \(\in\{50,100,200,\text{all}\}\) and rerun the full k-fold benchmark at each budget. The hypothesis is that RG-Flow’s scale prior helps most in the few-epochs regime and that the gap closes as data grow; we report seed-to-seed stability as a first-class metric alongside the mean.

3.7 Scope and reproducibility↩︎

The benchmark reported here is complete for the stated cohort: three models \(\times\) \(5\) seeds \(\times\) four data budgets \(\times\) leave-one-subject-out folds over \(5\) subjects, run to early stopping. All results are deterministic under fixed random seeds, so extending the study to more subjects amounts to rerunning the same protocol on the larger cohort. The cohort size (\(5\) subjects) is the binding limitation: it widens the confidence intervals in the scarce-data regime and is the reason no single accuracy contrast at a data-limited budget reaches significance (Section 4).

4 Results↩︎

4.1 Full-budget benchmark↩︎

Table 1 reports held-out 5-class sleep-staging performance at the full per-subject budget, mean \(\pm\) 95% CI over \(5\) seeds and all leave-one-subject-out folds, for the three parameter-matched models (RG-Flow 163,850, Vanilla 162,910, Ablation 163,850 parameters). RG-Flow reaches 77.3% accuracy (macro-F1 66.5%) versus Vanilla’s 77.0% (67.5%) and the ablation’s 76.0% (65.0%). The paired RG-Flow\(-\)Vanilla accuracy difference is +0.4percentage points (Wilcoxon \(p=0.294\), \(d_z={+0.07}\)), and RG-Flow\(-\)Ablation is +1.3points (\(p=0.034\)*).

4.2 Scarce-data budget sweep↩︎

Figure 3 sweeps the per-subject training budget. The scarce-data hypothesis predicts RG-Flow’s scale prior should help most at the smallest budget (50epochs/subject) and the gap should close as data grow. We do not observe this crossover. At the smallest budget the RG-Flow\(-\)Vanilla accuracy difference is -4.4points (\(p=0.072\))—that is, the vanilla transformer is numerically ahead in exactly the scarce regime where the inductive bias was expected to help, though the difference does not reach significance at five subjects. The three models converge to a statistical tie at the full budget. The point estimate therefore runs opposite to the hypothesis at every data-limited budget, and no budget shows a significant RG-Flow advantage. We additionally report seed-to-seed accuracy stability (right panel of Fig. 3); the three models are comparably stable, with the vanilla transformer marginally the most stable at the largest budgets.

4.3 \(\beta\)-recovery↩︎

Beyond staging accuracy, RG-Flow carries a regression head trained to recover the continuous spectral exponent \(\beta\). Out-of-sample it reaches \(R^2=0.416\) versus 0.403for Vanilla (paired \(\Delta={+0.013}\), \(p=0.542\)). This is a physically-meaningful, interpretable quantity; whether the model’s internal scale parameter \(\gamma\) carries the same information is examined next.

5 Interpretability: does \(\gamma\) track \(\beta\)?↩︎

The distinctive claim of the RG-Flow Transformer is not only accuracy but interpretability: its learnable anomalous dimension \(\gamma\) is a scalar, per-head, per-depth quantity that should—if the scale-aware bias is doing what it is designed to—carry information about the physical scaling exponent \(\beta\) of the input. The vanilla transformer has no comparable internal quantity, so this analysis is unique to RG-Flow.

Figure 4 relates the model’s learned \(\gamma\) (averaged over layers) to its out-of-sample \(\beta\)-recovery \(R^2\) across the budget sweep. Read against the trained models (\(100\) RG-Flow runs), three observations follow.

First, \(\gamma\) departs from its zero initialization but stays small in magnitude: across all runs \(|\gamma|\) never exceeds \(0.087\), so the scale-aware stream applies a gentle correction rather than a dominant reweighting of attention. Second, the sign of that correction is budget-dependent: in the scarce regime \(\gamma\) is slightly positive (mean \({+0.011}\) at \(50\) epochs/subject) and it turns negative as data grow (mean \({-0.025}\) at the full budget). Third—and most informative—the learned \(\gamma\) co-varies with the interpretability payoff: more-negative \(\gamma\) accompanies higher out-of-sample \(\beta\)-recovery, a statistically clear association across runs (Pearson \(r={-0.36}\), \(p\)<0.001, \(n=100\)). The internal scale parameter is therefore not inert; it moves with the very external spectral exponent the architecture was designed to track.

We stress that this interpretability result stands independently of the accuracy comparison. Even where RG-Flow and the vanilla transformer are statistically indistinguishable on staging accuracy, only RG-Flow exposes a physically-readable scale parameter, and only RG-Flow can be asked whether that parameter has tracked the data’s spectral exponent.

6 Conclusion↩︎

We built a leakage-free, real-data test of whether an explicit renormalization-group inductive bias helps a transformer on scarce neural signals, using PhysioNet Sleep-EDF sleep staging as the task and the aperiodic spectral exponent \(\beta\) as both a label axis and an interpretability referent. The contribution is a fixed, reproducible protocol: a preprocessing front-end, a per-epoch \(\beta\) estimator, a parameter-matched three-way comparison (RG-Flow, vanilla, hierarchy-only ablation), a data-budget sweep, and a \(\gamma\)-vs-\(\beta\) interpretability probe.

Three findings hold on this cohort (\(5\) subjects, \(5\) seeds, leave-one-subject-out). First, the machinery works end-to-end on real EEG: \(\beta\) is a clean, label-linked scale exponent (deep NREM steepest, \(97\%\) good fits), and all three models stage sleep at accuracies in the high-\(70\%\) range. Second, and against the study’s own hypothesis, the scarce-data crossover does not appear: RG-Flow does not pull ahead when data are few—the vanilla transformer is numerically ahead at every data-limited budget—and the models tie at the full budget (RG-Flow\(-\)Vanilla accuracy \({+0.4}\) pp, \(p=0.294\)). The RG physics is moreover indistinguishable from its hierarchy-only ablation, so what little structure the bias adds is attributable to coarse-graining, not to the scale-aware flow. Third, the interpretability claim survives independently of accuracy: RG-Flow recovers the continuous spectral exponent out-of-sample (\(R^2=0.416\)), its learned anomalous dimension \(\gamma\) moves systematically with data budget, and more-negative \(\gamma\) accompanies higher \(\beta\)-recovery (Pearson \(r={-0.36}\), \(p\)<0.001)—a physically-readable internal quantity the vanilla architecture has no analogue for.

The binding limitation is cohort size. With \(5\) held-out subjects the scarce-regime confidence intervals are wide, and while the point estimates run consistently against the inductive-bias hypothesis, none reaches significance there; a larger cohort is needed to decide whether the negative scarce-data result is real or merely underpowered. Because the analysis is deterministic under fixed random seeds, that larger study is a rerun of the same protocol on a wider cohort, leaving the surrounding argument unchanged. On the present evidence, the honest verdict is that an explicit renormalization-group bias does not improve scarce-data sleep staging, but it does buy a physically-meaningful, interpretable readout at no accuracy cost—which is where we would direct future work on this architecture for neural data.

References↩︎