1 Introduction↩︎

The last 50 years in topology has arguably been the half-century of low-dimensional topology, with many astounding groundbreaking advances in the topology of \(3\)-manifolds [1]–[3] and \(4\)-manifolds [4], [5]. We will show how innovative topological invariants developed to study low-dimensional (i.e., \(3\)- or \(4\)-dimensional) manifolds may be put to good use in the analysis and design of modern AI models.

We begin with a simple example. Consider two interlocking rings in \(\mathbb{R}^3\), the Hopf link. Can a neural network learn to classify points on these rings into their respective connected components? This deceptively simple question reveals deep connections between topology and machine learning. Under the manifold hypothesis [6], real-world data concentrates near low-dimensional manifolds embedded in high-dimensional space. When class manifolds are topologically entangled, like in a Hopf link, classification requires one to geometrically “untangle” them. We will see how such topological obstructions impose fundamental constraints on neural network architectures. Due to space constraints, we limit ourselves to one specific invariant, the linking number, to illustrate our general framework of using low-dimensional topological invariants to study neural network architectures. A companion paper treats more sophisticated invariants in detail [7].

We emphasize that dimension itself is a powerful, if not all-powerful, attribute in machine learning [8], [9] — there is almost nothing that one cannot do through simply increasing dimension, including effecting topological changes; although curse-of-dimensionality often also accompanies a dimension increase. The flip side of this coin is that dimension is a factor that masks the effects of all other aspects of a neural network architecture. Our approach removes this confounding factor by fixing width throughout, first at \(d = 3\), and later increasing it to higher \(d\).

Existing neural network theory largely ignores topological structure other than dimension (a topological invariant). Universal approximation theorems [10], [11] show that sufficiently wide networks approximate any function, but say nothing about which architectures succeed on which data. Depth–width tradeoffs [12], [13] and approximation rates [14], [15] characterize function complexity, not data complexity. Topological data analysis [16] studies intrinsic data shape but not how embeddings in ambient space constrain learning. Meanwhile, practitioners have observed that certain architectural features like skip connections, nonmonotonic activations, attention, etc, consistently outperform alternatives. Engineering reasons have been proffered for some of these: training stability, information flow, incremental layer-wise transformations, etc. We will provide a topological perspective.

When does topological structure create fundamental barriers for neural networks, and which mechanisms overcome them? The Hopf link (Figure 1) provides a canonical example: two interlocking circles with linking number \(\mathop{\mathrm{link}}= 1\), measuring how many times one curve winds through the other. Linear separability requires \(\mathop{\mathrm{link}}= 0\). Indeed, if a hyperplane separates two curves, each curve can be continuously contracted to a point inside its own half-space without ever leaving it, yielding a link homotopy to a disjoint pair of points whose linking number is zero. We prove that width-3 feedforward networks with ReLU activations preserve linking numbers: invertible affine layers preserve \(\mathop{\mathrm{link}}\) as homeomorphisms, and monotonic activations preserve \(\mathop{\mathrm{link}}\) via straight-line homotopies. Since the Hopf link has \(\mathop{\mathrm{link}}= 1 \neq 0\), no such network can achieve linear separability, regardless of depth. Similarly, decision regions cannot be made disjoint and convex, so logit-based classifiers cannot achieve perfect accuracy.

How do modern architectures escape from such topological traps? The key is folding: coordinate-wise nonmonotonic transformations like the absolute value \(|x|\). A ResNet block can synthesize \(|x| = x + 2\cdot\text{ReLU}(-x)\) using only ReLU and skip connections. Nonmonotonic activations (GELU, Swish) fold directly. Attention mechanisms create input-dependent convex combinations that locally approximate folding. These mechanisms break the homotopy argument underlying our impossibility theorems, enabling topological transformations that monotonic networks cannot perform.

Our contributions. We study the expressivity and learnability of neural network with low-dimensional topology:

1. Topology–ML connection. We prove that classification of linked data requires unlinking before a linear readout (Proposition 1).

2. Topological expressivity. We prove that width-\(d\) feedforward networks with continuous coordinate-wise monotonic activations cannot separate linked manifolds in \(\mathbb{R}^d\) (Sections 3–4). As a corollary, universal approximation with such activations requires width at least \(d+1\). We then show that skip connections, attention, nonmonotonic activations, and width expansion all provide ways to fold the data representation and thereby unlink linked manifolds (Section 5).

3. Learnability beyond expressivity. Experiments on Hopf links and higher-dimensional linked spheres show that limited topological expressivity imposes an accuracy ceiling under our training protocol. Further experiments on synthetic data and CIFAR-10 show that architectures with folding mechanisms gain a classification advantage, and this advantage is more pronounced at data points near the links (Section 6).

4. Extrinsic TDA. We present an algorithm for estimating linking between class manifolds from point-cloud samples, using spatial graphs, cycle bases, and Gauss integrals. Applied to CIFAR-10, the algorithm shows that class pairs with stronger linking tend to be harder for topologically less expressive architectures to classify (Section 6.6).

2pt

Table 1 offers a preview of our main expressivity results: which architectures can perform which topological transformations under width constraints. We will develop the theory in Sections [sec:preliminaries]–5, validate predictions experimentally in Section 6, and demonstrate real-world relevance via CIFAR-10 linking detection in Section 6.6.

2 Related Work↩︎

Topological data analysis extracts intrinsic shape features (holes, connected components) via persistent homology [16]–[18]. [19] study how deep networks change Betti numbers of data manifolds; we study how narrow networks preserve linking numbers, an extrinsic invariant depending on how manifolds are embedded in an ambient space, not an intrinsic invariant of a manifold itself like the Betti numbers. This extrinsic perspective connects to early intuitions about neural networks “untangling” data [20]. Recent works have identified other topological limitations: higher-order message-passing cannot compute homology [21], E(3)-invariant GNNs cannot distinguish enantiomers [22], and topological obstructions constrain generative models [23]. These concern what networks can detect; we study what they can transform, a question also relevant to world models that must represent geometric structure [24]. Width bounds for universal approximation [25]–[27] show that width-\(d\) networks cannot approximate all functions on \(\mathbb{R}^d\); we on the other hand show that arbitrarily deep width-\(d\) networks cannot separate linked data in \(\mathbb{R}^d\). Neural networks for molecular dynamics lift \(\mathbb{R}^3\) coordinates to 64- to 384-dimensional features [28]–[30]; our theory offers a topological rationale for this practice.

3 The Hopf Link: An Intuitive Example↩︎

Consider two interlinked circles in 3D space forming the Hopf link (for topological background, see [31], [32]).

Question: Can a width-3 ReLU network classify points on these circles into separate classes?

Here a width-\(d\) feedforward network is \(F = A_L \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }\sigma \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }\cdots \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }\sigma \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }A_1\) with affine \(A_i(x) = W_i x + b_i\), \(W_i \in \mathbb{R}^{d \times d}\), and coordinatewise activation \(\sigma\), e.g., ReLU. Our results apply to any architecture whose intermediate representations have dimensions \(\leq d\) after flattening any spatial axes and whose nonlinearities are coordinatewise. This includes CNNs, since convolutional layers are affine maps and the flattened representation size \(C \cdot H \cdot W\) is the operative width, as well as bottleneck autoencoders with width-\(d\) bottlenecks.

Classifying samples from the two Hopf link circles exposes a topological constraint on narrow neural networks; we develop the argument step by step, leaving technical details to Appendix 10.

3.1 Linking Numbers↩︎

The core of our argument is a topological invariant that quantifies how two closed curves are intertwined in 3D space.

3.2 Network Operations and Linking Number Preservation↩︎

We analyze each layer type in a width-3 ReLU network through the following key lemmas.

3.3 Classification Requires Unlinking↩︎

Linear separability is the goal of representation learning for classification: the final layer (classifier head) applies a linear transformation to features, so perfect classification requires that class representations become linearly separable. The task imposes a strict topological constraint:

This exposes the conflict: to classify the Hopf link perfectly, a network must change the linking number from \(\pm 1\) to \(0\).

3.4 Separation Impossibility for the Hopf Link↩︎

This example demonstrates the core mechanism: linking numbers create barriers that narrow monotonic networks cannot overcome, regardless of depth. The next section generalizes to higher-dimensional linked manifolds and arbitrary coordinate-wise monotonic activations.

4 Higher Dimension, More Activations↩︎

The Hopf link example gives the core mechanism in a visual setting. In this section we switch notation from the curves \(X,Y \subset \mathbb{R}^3\) of Section 3 to manifolds \(M^m,N^n \subset \mathbb{R}^d\), and generalize from ReLU to arbitrary coordinate-wise monotonic activations.

4.1 Higher-Dimensional Linking Numbers↩︎

The concept of linking generalizes from curves to higher-dimensional manifolds. For two closed, oriented, disjoint manifolds \(M^m\) and \(N^n\) in ambient space \(\mathbb{R}^d\) where \(d = m + n + 1\), the linking number is well-defined via degree.

In the layerwise arguments below, \(F\) denotes the composition of the first \(j\) network layers, for some \(j\), applied to both manifolds. The images \(F(M)\) and \(F(N)\) may no longer be manifolds; whenever they remain disjoint, \(\mathop{\mathrm{link}}(F(M), F(N))\) means the linking number of the maps \(F\mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }\iota_M\) and \(F\mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }\iota_N\) parametrizing the images \(F(M)\) and \(F(N)\).

The key lemmas from Section 3 generalize:

4.2 General Monotonic Activations↩︎

The Hopf link analysis used ReLU, but the impossibility extends to any coordinate-wise monotonic activation \(\sigma(x) = (\sigma_1(x_1), \ldots, \sigma_n(x_n))\) where each \(\sigma_i\) is continuous and monotonic, e.g., sigmoid, tanh, Leaky ReLU, ELU. This shows the obstruction is fundamental to monotonicity, not an artifact of ReLU’s piecewise-linear form.

The key to generalizing beyond ReLU is the concept of link homotopy: a simultaneous continuous deformation of multiple components that keeps them disjoint throughout. Unlike single-component homotopy, where one set moves while others stay fixed, link homotopy moves all components together via the same formula \(H_t(x) = (1-t)x + t\sigma(x)\). This simultaneity is essential: if only one component moves, it can collide with stationary components even for nondecreasing activations.

The result applies equally to smooth activations (sigmoid, tanh) and piecewise-linear activations (ReLU): the crucial property is monotonicity, not differentiability. Thus the topological barrier is intrinsic to the coordinate-wise monotonic structure.

5 Breaking Topological Constraints: Architectural Mechanisms↩︎

The impossibility results in Sections 3–4 demonstrate fundamental limitations of narrow networks with coordinate-wise monotonic activations. These constraints can be overcome through architectural approaches while preserving narrow width. We explore four complementary mechanisms, with supporting constructions in Appendices 11 and 13.

5.1 Non-Monotonic Activation Functions↩︎

The first approach breaks the monotonicity constraint underlying our impossibility theorems. While ReLU and other monotonic activations preserve topological rigidity, nonmonotonic functions create geometric flexibility needed to overcome linking obstructions.

Important nonmonotonic activations (Figure 3, bottom) include GELU [33] (\(x \cdot \Phi(x)\), standard in Transformers), Swish/SiLU [34], [35] (\(x \cdot \sigma(x)\)), and Mish [36] (\(x \cdot \tanh(\text{softplus}(x))\)). The impossibility results critically depend on Lemma 1: monotonic activations preserve linking numbers via straight-line homotopies that avoid intersections. This fails for nonmonotonic functions.

5.2 Skip Connections and Residual Networks↩︎

Residual networks [37] maintain monotonic activations but break the purely feedforward constraint through skip connections. The key insight is that ResNet can express nonmonotonic functions:

While ResNets can be viewed as discretizations of Neural ODEs [38], the discrete formulation is strictly more expressive for topological transformations. Neural ODEs generate continuous flows (diffeomorphisms) that must preserve topological invariants. Discrete ResNet blocks can implement folding maps that change these invariants; the \(|x| = x + 2\mathop{\mathrm{ReLU}}(-x)\) identity relies crucially on discrete, non-infinitesimal residuals.

5.3 Attention Mechanisms↩︎

The Transformer architecture [39] introduces self-attention mechanisms enabling global information routing. We study the topological expressivity of attention by considering pure transformers.

5.4 Width Threshold and Design Implications↩︎

Width \(\geq d+1\) eliminates the classification obstruction. Our impossibility results are tight: width-\((d+1)\) ReLU networks achieve universal approximation on compact subsets of \(\mathbb{R}^d\) [25], and subsequent work sharpens minimum-width thresholds for ReLU, leaky-ReLU, and compact-domain settings [40]–[43]. This suffices to map any disjoint configuration of class manifolds into disjoint scalar class labels, i.e., to achieve linear separability; it does not require “unlinking” as an ambient-topology operation, only producing the right value of a continuous label function.

Design guidance. For topologically complex data, width should be read locally as well as globally: a wide model may contain concrete narrow maps, such as the \(d_{\mathrm{model}}\to d_k,d_v\) projections in a single attention head, a latent bottleneck, or the reduced channel inside a bottleneck residual block. In such settings: (1) expand MLP width slightly beyond input dimension before narrowing; (2) prefer nonmonotonic activations (GELU, SiLU, Mish) in bottleneck layers; (3) use skip connections; or (4) leverage attention. These mechanisms change topology directly rather than relying solely on dimension expansion.

6 Experiments↩︎

We validate our theoretical results with experiments on the Hopf link classification task. These experiments verify both the impossibility results for monotonic activations and the escape mechanisms described in the previous section, with experimental and detection details in Appendices 14–16.

6.1 Experimental Setup↩︎

Thickened Hopf link. We construct volumetric data by sampling points from two thickened interlinked tori. Each curve is thickened by adding small perturbations normal to the curve surface. Dataset: 6000 points (3000 per class), split 80/20 train/validation.

Training protocol. Adam/AdamW with learning rate \(10^{-3}\), up to 800 epochs with early stopping (patience 100–200). Cross-entropy loss.

6.2 ReLU vs GELU on Hopf Link↩︎

We compare monotonic (ReLU) versus nonmonotonic (GELU) activations across 12 depths (3–20 layers) with 30 random seeds each (720 total experiments).

2pt

Table 2 confirms the theory: ReLU stays at or below the topological ceiling and degrades with depth as optimization difficulty compounds the expressivity barrier (Theorem 2); GELU consistently achieves \(\sim\)​90% at moderate depths, breaking the barrier. At extreme depths (\(\geq 16\)) both activations suffer optimization instability (large stds). Across 30 seeds, GELU’s best run reaches \(100.0\%\) at depths 5–12 while ReLU never exceeds \(92.8\%\), matching the prediction.

6.3 ResNet vs Plain ReLU↩︎

We compare plain ReLU feedforward networks against ReLU ResNets, both at width 3.

2pt

Across 30 seeds, ResNet reaches a 100% best run at every depth from 3 to 8, and mean accuracy stays at \(96.6\)–\(98.5\%\) across all depths. Plain ReLU never exceeds \(92.3\%\) and has much lower means with high seed variance, confirming both an expressivity barrier and depth-dependent optimization difficulty. Skip connections provide a mechanism to overcome topological barriers even with monotonic activations, as predicted by Theorem 5.

6.4 Mechanistic Interpretability: How ResNet Unlinks a Point/Circle Pair in \(\mathbb{R}^2\)↩︎

The disk-annulus separation task is the ambient-dimension \(d = 2\) instance of our linking framework, given by Theorem 4 with \(m=0\), \(n=1\), and \(d=2\): a point inside the annulus and the annulus’ inner boundary circle form a \(0\)-manifold/\(1\)-manifold link with \(|\mathop{\mathrm{link}}| = 1\), equivalently winding number \(\pm 1\). By our theory, width-\(2\) monotonic feedforward networks cannot linearly separate them, but a width-\(2\) ResNet can.

Figure 6 shows the layer-by-layer transformation. ResNet implements “folding” operations via \(|x| = x + 2\mathop{\mathrm{ReLU}}(-x)\) (Equation 1 ), enabling nonmonotonic transformations that separate the topologically linked components.

6.5 Higher-Dimensional Linking: \(\mathbb{S}^n \sqcup \mathbb{S}^n\) in \(\mathbb{R}^{2n+1}\)↩︎

We extend experiments to higher dimensions using linked hyperspheres: two \(n\)-spheres embedded in \(\mathbb{R}^{2n+1}\) with linking number \(\pm 1\). For \(n=2\), this gives \(\mathbb{S}^2 \sqcup \mathbb{S}^2 \subset \mathbb{R}^5\) classified by width-5 networks. We quantify topological difficulty via linking number: a single link has only one local entanglement region, so networks can achieve high accuracy by being wrong only in that small patch; placing \(k\) disjoint copies forces the network to handle larger entanglement regions (Table 4).

2pt

At small \(k\), monotonic architectures with skip connections reach the expressivity ceiling (ReLU+Skip attains 100% at \(k=1\), 98.7% at \(k=2\)). As \(k\) increases, the best nonmonotonic architectures (GELU/Swish) hold a \(3\)–\(4\)pp accuracy advantage over the best monotonic architecture at \(k\geq 10\), consistent with nonmonotonic activations being able to resolve each local entanglement separately. The effect at \(k=1\) is weaker than for \(n=1\) (Table 2) due to the linking region’s smaller volume fraction in higher dimensions.

6.6 Linking in Real Data: CIFAR-10↩︎

As a suggestive proof-of-concept, we investigate whether topological linking detectable by our algorithm corresponds to classification difficulty in real image data. We emphasize that the evidence below is correlational rather than causal: classification on real images involves many confounding factors beyond ambient topology, and the construction relies on 3D PCA projection rather than the native data manifold. Using the linking detection algorithm, we analyze CIFAR-10 class manifolds projected to 3D via PCA.

Link detection. With 20\(\times\) data augmentation (1.05M samples) and \(k\)-NN graph construction (\(k=15\), mutual edges), we detect linked cycles between bird and deer classes at \(\varepsilon = 0.034\) (0.22% of the 3D bounding box diagonal). The witness cycles have linking number \(\mathop{\mathrm{link}}= -1\) with Gauss integral \(-1.004\) (Figure 7). This binary search only locates the onset scale; the all-pair consistency study uses fixed thresholds, and detected graph-cycle witnesses persist as \(\varepsilon\) increases because the filtered \(k\)-NN edge set only grows. This is a reproducible linking signal in PCA-3D, not a claim about linking of the full \(3072\)D pixel manifold.

10-class linking consistency. Running detection 11 times across all 45 class pairs yields a linking-consistency summary. Of 45 pairs, 27 are strongly linked (\(>\)​70% consistency), 8 are weak or unlinked (\(<\)​30%), and 10 are ambiguous. High examples include deer–dog (91%), bird–cat (91%), cat–dog (82%), and automobile–truck (91%); low controls include frog–ship (18%), airplane–horse (18%), and cat–ship (27%).

Classification experiments. We train width-bounded CNNs (all intermediate layers \(\leq 3072\)D) on both binary and 10-class classification tasks, comparing monotonic (ReLU, ELU, SELU, LeakyReLU) versus nonmonotonic (GELU, Swish, Mish) activations.

At L8 no-skip (Table 5), all nonmonotonic activations outperform all monotonic activations on the linked deer-dog pair (+1.2% gap), while no such pattern exists on the unlinked frog-ship pair (control). With skip connections, the effect disappears: monotonic networks match nonmonotonic performance, as predicted by Theorem 5. As a localization diagnostic, we retrained bird–deer L8 no-skip ReLU/GELU classifiers, detected one PCA-3D link witness, and stratified the unaugmented test set by distance to that witness. The activation gap is largest near the witness and decays with distance (Table 6); this is diagnostic rather than causal evidence, since distance is measured in PCA-3D and can mix topology with local geometry. PCA-3D linking consistency also correlates with within-CIFAR class-pair confusion (\(r \approx 0.48\), \(p < 0.001\)), outperforming pixel-space distance metrics.

Expressivity vs.optimization. Our theoretical results concern expressivity, meaning what functions network architectures can represent, not optimization, meaning what functions gradient descent actually finds. Even when two architectures have identical expressivity ceilings, they may exhibit different training dynamics: convergence speed, sensitivity to learning rate, propensity to find particular local minima, etc. The observed CIFAR-10 gaps thus reflect a combination of (i) the topological expressivity barrier predicted by our theory, and (ii) optimization dynamics that our theory does not address.

7 Conclusion↩︎

This work develops a new lens for neural network expressivity analysis: from approximation rates to geometric transformation requirements. Rather than asking whether a network can approximate a target function, we ask what geometric operations on embedded data it can perform. For the pairwise linking and folding phenomena studied here, this perspective extends to mechanistic understanding: ResNet skip connections, attention mechanisms, and nonmonotonic activations all break the monotonicity required by our impossibility argument via a “folding” construction.

Our framework focuses on ambient-sensitive (extrinsic) topological invariants, fundamentally distinguishing it from traditional topological data analysis which studies intrinsic properties. The linking number of two manifolds depends on how they are embedded in \(\mathbb{R}^n\), not merely on the manifolds themselves. This extrinsic perspective captures the algorithmic reality: neural networks must transform data within the ambient space, making embedding geometry as important as intrinsic data structure.

We expect this work to open broader applications of low-dimensional topology and geometric topology to machine learning. The invariant we study, the pairwise linking number, is an initial example. A companion paper treats richer invariants from link theory and knot theory, such as Milnor’s \(\bar{\mu}\)-invariants and knot types, in detail [7]. Similarly, the task correspondence we establish between unlinking and classification likely represents an initial example of a broader pattern yet to be fully explored. Understanding this interplay between data geometry and algorithmic capability offers new directions for both theoretical foundations and principled architecture design.

Impact Statement↩︎

This paper presents theoretical and empirical work on neural network expressivity. We identify fundamental limitations of certain architectures, which may inform more principled architecture design. We also propose a link detection algorithm that characterizes topological complexity in datasets, which may inform dataset difficulty analysis.

Acknowledgments↩︎

JR and LH are partially supported by a Vannevar Bush Faculty Fellowship ONR N000142312863.

8 Network Architecture↩︎

This appendix collects the network-architecture definitions used in the main text (§8.1) and the classifier-head/linear-separability equivalence (§8.2).

8.1 Network Architectures↩︎

The bottleneck dimension \(d\) alone determines the topological constraint; the input dimension \(n\) is irrelevant. To break the obstruction one must widen the bottleneck or place full-width nonmonotonic/skip-augmented layers before compression.

8.2 Classifier Heads and Linear Separability↩︎

A standard classifier head maps features \(z \in \mathbb{R}^d\) to logits \(\ell = Wz + b \in \mathbb{R}^c\), predicting \(\hat{y} = \arg\max_i \ell_i\). The class-\(i\) decision region \(D_i = \{z \in \mathbb{R}^c : z_i > z_j \text{ for all } j \neq i\}\) is the intersection of \(c-1\) open half-spaces, hence convex, and the \(\{D_i\}\) are pairwise disjoint.

Hence proving that a width-\(d\) network cannot separate linked components is equivalent to proving that no classifier head achieves perfect accuracy on them.

9 Invertible Architectures and Ambient Homeomorphisms↩︎

An ambient homeomorphism of \(\mathbb{R}^d\) is a continuous bijection \(h: \mathbb{R}^d \to \mathbb{R}^d\) with continuous inverse. By the invariance of domain (Brouwer 1912; [44]), any continuous injective map \(f: U \to \mathbb{R}^d\) on an open \(U \subseteq \mathbb{R}^d\) is an open map onto its image, hence a homeomorphism onto \(f(U)\), though not in general an ambient homeomorphism of \(\mathbb{R}^d\). The architectures we analyze in this appendix (flow-based models, Neural ODEs, normalizing flows) are constructed to be ambient homeomorphisms by design: their forward maps are explicit continuous bijections \(\mathbb{R}^d \to \mathbb{R}^d\) with continuous inverses. For such architectures, the lemmas below apply globally; for a continuous injective width-\(d\) feedforward network, they apply on the image of the data manifold under that network, which suffices for the linking-preservation conclusion.

9.1 Invertible Architectures in Machine Learning↩︎

Invertible architectures whose forward map is an ambient homeomorphism therefore cannot change linking number at any depth. This includes reversible residual networks (RevNet [45], i-RevNet [46]) and normalizing flows [47], [48], which are invertible by construction; and ODE-based flows (Neural ODEs [38], FFJORD [49], Flow Matching [50]), whose ODE integration of a Lipschitz vector field produces a diffeomorphism. [51] verify this constraint empirically on concentric-ring classification with Neural ODEs; their augmented variant works precisely by lifting to higher dimension (width expansion). Discrete-time ResNets are not subject to this constraint: their layer-by-layer composition need not be a homeomorphism, and a discrete block can implement \(|x| = x + 2\mathop{\mathrm{ReLU}}(-x)\) (Theorem 5), which is not invertible. One-step direct evaluators, e.g., MeanFlow [52], similarly escape the homeomorphism constraint by avoiding ODE integration entirely.

10 Proofs for Sections 3 and 4: Linking-Number Preservation↩︎

This appendix proves the main linking-preservation and impossibility theorems for both the \(\mathbb{R}^3\)/ReLU setting of Section 3 and the general width-\(d\)/coordinate-wise-monotonic setting of Section 4. The general results subsume the \(\mathbb{R}^3\)/ReLU special case; we present the general proofs and indicate where the elementary \(\mathbb{R}^3\) argument specializes.

10.1 Linking number and link homotopy↩︎

10.2 Preservation under monotonic activations↩︎

For the \(\mathbb{R}^3\)/ReLU special case with curves \(X,Y\), the same conclusion admits an elementary one-coordinate-at-a-time argument: write \(\sigma = G_3 \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }G_2 \mathbin{\mathchoice {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} {\vcenter{\scriptscriptstyle\circ}} }G_1\) where \(G_j\) applies ReLU only to coordinate \(j\), and homotope \(X\) then \(Y\) through each \(G_j\) in turn. A collision \(H^X_t(x) = y\) would force \(G_j(x) = G_j(y)\) at the endpoint, contradicting \(\sigma(X) \cap \sigma(Y) = \varnothing\). We refer to this elementary form below as the sequential ReLU homotopy.

10.3 Rank-deficient transformations force intersection↩︎

In \(\mathbb{R}^3\), every rank-deficient affine map factors through a rank-\(2\) orthogonal projection \(P: \mathbb{R}^3 \to \mathbb{R}^2\); the combinatorial linking formula then equates \(\mathop{\mathrm{link}}(X, Y)\) with a signed sum over \(P(X) \cap P(Y)\), so non-zero linking forces a non-empty projection intersection. This is the special case used to prove Theorem 2.

10.4 Main impossibility theorems↩︎

11 Width Upper Bound for Topological Unlinking↩︎

This section shows that width \(d+1\) is sufficient to eliminate the classification obstruction: a width-\((d+1)\) ReLU network can linearly separate any disjoint compact configuration in \(\mathbb{R}^d\) regardless of linking. The construction does not unlink the classes as ambient subsets of \(\mathbb{R}^d\); it maps them into disjoint scalar intervals, which is the operation classification actually requires.

Width \(d+1\) is therefore tight: width \(d\) is impossible by the lower bound below, while width \(d+1\) suffices.

12 Width Lower Bound for Universal Approximation↩︎

The contribution here is the proof technique, not the bound itself. Prior width-\(d\) insufficiency results constrain either the activation class or its regularity: [25] (ReLU, level-set components), [26] (activations approximable by injections, level-set topology), [40]–[43] (ReLU, leaky-ReLU, and compact-domain minimum-width analyses), and [27] (monotone Lipschitz activations, approximation-theoretic). Our argument requires only continuous coordinate-wise monotonic activations, with no Lipschitz regularity, smoothness, or approximation by injections, and obtains the bound as a direct corollary of ambient topological invariance, complementing the width-\((d+1)\) upper bound of Appendix 11.

13 Detailed Constructions Breaking Topological Constraints↩︎

13.1 Why nonmonotonic activations break homotopy preservation↩︎

The monotonic-activation preservation lemma (Lemma 10) used the straight-line homotopy \(H_t(x) = (1-t)x + t\sigma(x)\), which is a link homotopy precisely because monotonicity rules out coordinate-wise collisions during interpolation. For \(\sigma(x) = |x|\), the same straight-line interpolation crosses the fold line \(x_i = 0\) from negative to positive sides: distinct points with mirror-image coordinates can collide on the fold. Thus \(\mathop{\mathrm{link}}(\sigma(M), \sigma(N))\) need not equal \(\mathop{\mathrm{link}}(M, N)\), and the rigidity that drives the impossibility theorems disappears.

13.2 Hopf link unlinking via absolute value activations↩︎

We give an explicit five-step construction (Figure 9) that takes the Hopf link \(X(t) = (\cos t, \sin t, 0)\), \(Y(s) = (0, 1+\cos s, \sin s)\) (the same Hopf link as in Example 1, with the \(X\)-axis offset of the main text swapped to the \(Y\)-axis by a \(90^\circ\) rotation of coordinates, chosen so the first fold acts on \(y\)) to a linearly separable configuration in \(\mathbb{R}^3\) using only coordinate-wise \(|\cdot|\) and affine maps:

1. Apply \(|\cdot|\) to the \(y\)-coordinate: \(X \mapsto (\cos t, |\sin t|, 0)\).

2. Affine \((x,y,z) \mapsto (x, 1-y, z)\): now \(X = (\cos t, 1 - |\sin t|, 0)\) and \(Y = (0, -\cos s, \sin s)\).

3. Apply \(|\cdot|\) to the \(x\)- and \(y\)-coordinates.

4. Apply \(|\cdot|\) to the \(z\)-coordinate.

After step 4, both curves lie in the positive octant; an explicit hyperplane (shaded in Figure 9) separates them.

Extension to activations with a local extremum. The recipe generalizes to any activation \(\sigma: \mathbb{R}\to \mathbb{R}\) with a strict local extremum on an open interval \(I \subset \mathbb{R}\), e.g., the local minimum of GELU near \(-0.5\), of Swish/SiLU near \(-1.3\), of Mish near a similar point: since the data is compact, an affine pre-shift \(x \mapsto ax + b\) rescales the relevant data coordinate into \(I\), on which \(\sigma|_I\) is nonmonotonic. The straight-line homotopy \(t \mapsto (1-t)x + t\sigma(x)\) on \(I\) admits a fold-line collision in exactly the same way as \(|\cdot|\) does on \(\mathbb{R}\), so the same disjoint-tube argument fails and the same unlinking construction goes through with \(\sigma|_I\) in place of \(|\cdot|\). We do not require \(\sigma|_I\) to equal \(|\cdot|\), only to fold the rescaled data into a region with reduced crossing structure.

13.3 ResNet absolute-value synthesis↩︎

13.4 Transformer absolute-value synthesis↩︎

14 Experimental Details↩︎

14.1 Hopf Link Parametrization and Thickening↩︎

Hopf link parametrization. \(X(t) = (\cos t, \sin t, 0)\) and \(Y(s) = (1 + \cos s,\, 0,\, \sin s)\).

Thickening procedure. Sample points as \(\gamma(t) + \varepsilon \cdot \mathbf{n}(t)\) where \(\mathbf{n}(t)\) is a unit normal to the curve, \(\varepsilon \sim \mathcal{U}(0, r)\) with \(r = 0.15\), and high-frequency oscillations \(0.3\sin(100t)\) are added to preserve topology.

14.2 Network Architectures and Training Protocol↩︎

All architectures use width \(3\). FFNs are fully connected with depths \(3\)–\(20\); ReLU ResNets use residual blocks \(x \mapsto x + g(x)\) with 2-layer width-\(3\) subnetworks \(g\). We train with Adam for FFNs and AdamW for ResNets at learning rate \(10^{-3}\), batch size \(128\), up to \(800\) epochs, early stopping with patience \(100\)–\(200\), and cross-entropy loss on \(6000\) points (\(3000\) per class) split \(80/20\) for train/validation. Code: github.com/7pocheR/low_dimensional_topology.

14.3 ReLU vs GELU: Detailed Observations↩︎

Table 2 reveals three patterns. (i) Across 30 seeds, ReLU’s best run never exceeds the \(\sim\)​90% topological ceiling at any depth (max \(92.8\%\) at depth 3); mean accuracy is bounded above by the ceiling and additionally degrades with depth as optimization difficulty compounds the expressivity barrier, consistent with Theorem 2. (ii) GELU mean accuracy stays at \(89\)–\(91\%\) for depths 3–12 and the best run achieves \(100\%\) at depths 5–12, confirming that nonmonotonic activations escape the constraint (§5.1). (iii) At extreme depths (\(\geq 16\)) both activations suffer optimization instability (large stds); depth alone cannot compensate. Unlike standard universal-approximation results where depth substitutes for width, here depth provides no escape from the topological barrier.

14.4 Higher-Dimensional Linked Spheres Construction↩︎

For the higher-dimensional experiments (Section 6.5), we construct two \(n\)-spheres linked in \(\mathbb{R}^{2n+1}\) with linking number \(\pm 1\). The construction uses explicit parametrizations \(\tilde{A}, \tilde{B}: \mathbb{S}^n \to \mathbb{R}^{2n+1}\) derived from stereographic projection.

Parametrization. For \(u = (u_0, u') \in \mathbb{S}^n \subset \mathbb{R}^{n+1}\) with \(u' = (u_1, \dots, u_n)\), set \(a = 1 - u_0/\sqrt{2}\) and define \(\tilde{A}(u) = (u'/a,\, -u_0/(\sqrt{2}\,a),\, \mathbf{0}_n) \in \mathbb{R}^n \times \mathbb{R}\times \mathbb{R}^n = \mathbb{R}^{2n+1}\), where \(\mathbf{0}_n\) is the zero vector in \(\mathbb{R}^n\). Symmetrically, for \(v = (v_0, v') \in \mathbb{S}^n\) set \(b = 1 - v_0/\sqrt{2}\) and define \(\tilde{B}(v) = (\mathbf{0}_n,\, v_0/(\sqrt{2}\,b),\, v'/b)\). The images \(\tilde{A}(\mathbb{S}^n), \tilde{B}(\mathbb{S}^n)\) are disjoint \(n\)-spheres in \(\mathbb{R}^{2n+1}\) with \(\mathop{\mathrm{link}}(\tilde{A}(\mathbb{S}^n), \tilde{B}(\mathbb{S}^n)) = 1\) (verified numerically via the higher-dimensional Gauss linking integral).

Geometric structure. The embedding places \(\tilde{A}(\mathbb{S}^n)\) in the \(X\)-\(Z\) subspace (first \(n\) coordinates plus middle coordinate, with \(Y=\mathbf{0}\)) and \(\tilde{B}(\mathbb{S}^n)\) in the \(Z\)-\(Y\) subspace (middle coordinate plus last \(n\) coordinates, with \(X=\mathbf{0}\)). These coordinate subspaces intersect only along the shared middle coordinate axis, and the linking arises from the spheres’ interlocking configuration around this axis.

Minimum separation. The minimum distance between points on the two spheres is \(d_{\min} = 2(\sqrt{2} - 1) \approx 0.828\), independent of \(n\). This ensures the spheres remain well-separated.

Targeted thickening. To create training data with non-trivial volume, we use targeted thickening: each sphere is thickened within its complementary (normal) subspace to preserve the linking structure. In the \((X, Z, Y)\) split with \(X, Y \in \mathbb{R}^n\), \(Z \in \mathbb{R}\): \(\tilde{A}\) lies in the \(X\)-\(Z\) subspace (\(Y = 0\)) and is thickened in the \(Y\)-direction as \(\tilde{A}_\rho(u, \eta) = \tilde{A}(u) + (0, 0, \eta)\); \(\tilde{B}\) lies in the \(Z\)-\(Y\) subspace (\(X = 0\)) and is thickened in the \(X\)-direction as \(\tilde{B}_\rho(v, \zeta) = \tilde{B}(v) + (\zeta, 0, 0)\), with \(\eta, \zeta \sim \text{Uniform}(B_n(\rho))\) on the \(n\)-dimensional radius-\(\rho\) ball. Our experiments use \(\rho = 0.5\).

Multi-copy placement. To amplify the topological barrier (Section 6.5), we place \(k\) disjoint copies of the linked pair using \(L^1\)-ordered grid placement: copy centers are integer lattice points in \(\mathbb{Z}^{2n+1}\) enumerated in nondecreasing \(L^1\) norm order, scaled by spacing \(s = 10\). Specifically, we enumerate shells \(\{v \in \mathbb{Z}^{2n+1} : \|v\|_1 = m\}\) for \(m = 0, 1, 2, \ldots\) and take the first \(k\) vectors. This spreads copies isotropically rather than along a single axis, preventing networks from exploiting directional biases. Each copy contributes an independent entanglement region, so networks must overcome \(k\) local obstructions simultaneously.

14.5 Linking Scaling Experiments: Variance and Dimensional Effect↩︎

For the higher-dimensional linking experiments (Table 4, \(\mathbb{S}^2 \sqcup \mathbb{S}^2\) in \(\mathbb{R}^5\) with \(k\) disjoint copies), we tracked accuracy variance across 100 seeds in addition to best-case ceilings. At small \(k\), ReLU+Skip is both the most reliable and reaches the best ceiling: at \(k=1\) it attains \(100\%\) best and a \(97.0\pm 3.5\) mean test accuracy, whereas plain ReLU’s mean is \(89.4\pm 11.4\) with seeds occasionally collapsing far below the topological ceiling. As \(k\) grows, monotonic ceilings (ReLU, ReLU+Skip) decline faster than nonmonotonic ceilings (GELU, Swish): the nonmonotonic-minus-monotonic best-case gap is mildly negative for \(k\leq 5\), becoming \(+3.7, +4.4, +3.9\)pp at \(k=10, 20, 50\). The pattern is consistent with skip connections resolving the single entanglement at \(k=1\), while nonmonotonic activations are required to resolve each of many local entanglements at larger \(k\). Mean accuracies decline for all architectures as \(k\) grows, but nonmonotonic mean accuracies remain \(4\)–\(6\)pp above monotonic at \(k\geq 10\), e.g., GELU mean \(81.0\pm 3.7\) vs.ReLU+Skip mean \(76.4\pm 3.9\) at \(k=10\).

Why higher dimensions need larger \(k\). At \(n=1\) (\(\mathbb{S}^1 \sqcup \mathbb{S}^1\) in \(\mathbb{R}^3\)) the topological advantage of skip/nonmonotonic architectures is visible at \(k=1\) (Table 2); at \(n=2\) (\(\mathbb{S}^2 \sqcup \mathbb{S}^2\) in \(\mathbb{R}^5\)) one needs \(k > 5\) to see it. The reason is geometric: monotonic networks can achieve high accuracy by sacrificing accuracy on a small overlap region of characteristic size \(\varepsilon\) near each linked location, whose volume fraction scales as \(\Theta(\varepsilon^{2n+1})\) in ambient \(\mathbb{R}^{2n+1}\). For \(n=1\) that fraction is \(\sim 10\%\); for \(n=2\) the same \(\varepsilon\) gives \(\sim \varepsilon^2\) smaller (the \(\sim 90\% \to \sim 99\%\) drop at \(k=1\) implies \(\varepsilon \approx 0.3\)). Real high-dimensional datasets are expected to compensate by having more linking opportunities (volume growing as \(\Theta(\mathrm{poly}(n))\)); our multi-copy design synthetically restores the cumulative obstruction so the experiment remains controlled across \(n\).

14.6 ResNet Skip Connection Visualization↩︎

Figure 10 shows the full resolution visualization of the ResNet skip connection mechanism on the disk-annulus (\(\mathbb{S}^0\)-\(\mathbb{S}^1\)) separation task. This experiment demonstrates that a depth-3 width-2 ResNet with ReLU activations learns to implement the folding operation \(|x| = x + 2\mathop{\mathrm{ReLU}}(-x)\) predicted by Theorem 5.

Training details. The ResNet architecture uses 3 residual blocks with width-2 hidden layers. Training uses Adam optimizer with learning rate 0.001 for 5000 epochs on 50,000 points per class. The visualization shows the block 1 transformation (input \(\to\) first residual block output). Multiple random seeds (including seed 103 shown) consistently learn similar folding transformations, demonstrating that the theoretical mechanism is reliably discovered by gradient descent.

14.7 Width Expansion Eliminates the Obstruction↩︎

Theorem 9 predicts that increasing width past the critical \(d+1\) threshold removes the topological barrier. We verify this in \(\mathbb{R}^7\) on \(\mathbb{S}^3 \sqcup \mathbb{S}^3\) with \(k = 10\) copies, depth \(5\), \(15\) seeds per width (Table 7). Critical width \(d = 7\) caps at \(\approx 88\%\) (mean) / \(93\%\) (max), and accuracy improves overall as width is expanded (with small finite-seed nonmonotonicities), saturating near \(99{-}100\%\) around width \(\approx 5d\). The same pattern holds in \(\mathbb{R}^5\) (\(k = 10\), depth \(5\)): plain ReLU at width \(20\) already reaches \(98.5\%\) mean, and adding nonmonotonic activations or skip connections only marginally changes this, confirming that the architectural mechanisms (folding, skip) are most valuable when width is constrained below the critical threshold and become substitutable with width when width is relaxed.

14.8 Layer-by-Layer Linking-Number and Min-Distance Tracking↩︎

To verify mechanistically that the impossibility theorem reflects what the network actually does, we track both the Gauss linking integral \(\mathop{\mathrm{link}}\) and the minimum inter-class distance \(d_{\min}\) layer-by-layer for a width-\(3\), depth-\(5\) network on the Hopf link, comparing ReLU, GELU, and ReLU+skip (Table 8).

Interpretation. ReLU forces the two curves into intersection by \(L_1\) (minimum distance collapses to \(0\)); the apparent fractional \(\mathop{\mathrm{link}}\) values at \(L_0\), \(L_1\) are not meaningful linking numbers but artifacts of the Gauss integral becoming ill-conditioned as \(|x - y| \to 0\) (linking number is only defined for disjoint curves). After \(L_1\), the network has destroyed the geometric structure rather than resolved the topology, consistent with Lemma 2 and the impossibility theorem. GELU and ReLU+skip, by contrast, achieve \(\mathop{\mathrm{link}}= 0\) while keeping the curves disjoint: after the first unlinking layer (\(d_{\min}\) small but positive), subsequent layers increase \(d_{\min}\) steadily, implementing the genuine unlinking that monotonic feedforward networks cannot.

15 Linking Detection Algorithm for Point Cloud Data↩︎

This section gives the algorithm used to detect topological linking between two finite point clouds \(\mathcal{X}, \mathcal{Y} \subset \mathbb{R}^d\). The pipeline is: (i) project to \(\mathbb{R}^3\) via PCA, (ii) build \(k\)-NN spatial graphs per class, (iii) extract a fundamental cycle basis per graph, (iv) compute Gauss linking numbers over pairs of basis cycles.

15.1 Graph construction↩︎

For each class \(\mathcal{X} = \{x_1, \dots, x_n\} \subset \mathbb{R}^3\), the \(\varepsilon\)-filtered \(k\)-NN graph \(G\) has vertex set \(\{1, \dots, n\}\) and edge \((i,j)\) whenever \(j\) is among the \(k\) nearest neighbors of \(x_i\) and \(\|x_i - x_j\| \le \varepsilon\). The threshold \(\varepsilon\) (typically a percentile of nearest-neighbor distances) suppresses spurious long edges. For fixed point cloud and \(k\)-NN relation, the edge set is monotone in \(\varepsilon\): increasing \(\varepsilon\) only adds eligible \(k\)-NN edges, so already detected witness cycles remain graph cycles at larger thresholds. The mutual variant additionally requires \(i \in \mathrm{kNN}(j)\), yielding sparser but more symmetric graphs robust to density variation.

15.2 Fundamental cycle basis via spanning forest↩︎

This reduces detection from a search over infinitely many cycle pairs to a finite computation over \(O(\beta_\mathcal{X} \cdot \beta_\mathcal{Y})\) basis pairs, where \(\beta = m - n + c\) is the first Betti number.

15.3 Gauss linking number↩︎

For disjoint piecewise-linear cycles \(C_1, C_2 \subset \mathbb{R}^3\), we evaluate the Gauss integral \(\mathop{\mathrm{link}}(C_1, C_2) = \frac{1}{4\pi} \oint_{C_1} \oint_{C_2} \frac{(x - y) \cdot (dx \times dy)}{|x - y|^3}\) via midpoint quadrature on subdivided edges. Algorithm 11 packages the full detection pipeline.

15.4 Parameters and complexity↩︎

Typical choices: \(k \in [6, 15]\) (denser \(k\) exposes more cycles at the cost of compute), \(\varepsilon\) at the 70th percentile of nearest-neighbor distances, \(N_{\mathrm{sub}} \in \{4, 8\}\) subdivisions per edge, and minimum cycle length \(\ge 4\) (smaller cycles are construction artifacts). Use mutual \(k\)-NN for heterogeneous density.

Complexity: \(k\)-NN graph in \(O(n \log n)\) via \(k\)-d trees; spanning forest and cycle-basis extraction in \(O(n + m) = O(kn)\); Gauss integral per cycle pair in \(O(L_1 L_2 N_{\mathrm{sub}}^2)\) for cycle lengths \(L_1, L_2\). Total: \(O(n \log n + \beta_\mathcal{X} \beta_\mathcal{Y} L^2 N_{\mathrm{sub}}^2)\). For our CIFAR-10 setting (\(n \sim 10^4\), \(\beta \sim 10^2\), \(L \sim 20\)) the algorithm runs in tens of seconds per class pair.

16 CIFAR-10 Linking Experiments: Details↩︎

This appendix gives the data-preparation, detection-parameter, and architectural details for the suggestive CIFAR-10 linking analysis of Section 6.6. We emphasize throughout that the evidence here is correlational, not causal, and rests on a 3D PCA projection rather than the native data manifold; the synthetic experiments (Appendix 14) are the primary validation of the theory.

16.1 Data preparation and link detection↩︎

Augmentation and projection. We apply \(20\times\) training-set augmentation (random horizontal flip, random crop with 4-pixel padding, color jitter \(\pm 0.2\) for brightness/contrast/saturation) to CIFAR-10 (yielding \(1{,}050{,}000\) samples; \(105{,}000\) per class), flatten + standardize, and project to \(\mathbb{R}^3\) via PCA on the combined augmented dataset.

Detection parameters. We run Algorithm 11 with \(k = 15\) mutual nearest neighbors and minimum cycle length \(30\). For the displayed bird–deer witness, we separately binary-search the threshold \(\varepsilon\) to locate the smallest scale at which any pair of class cycles links. At \(\varepsilon = 0.0338\) (\(0.22\%\) of the 3D bounding-box diagonal \(15.555\)), the bird (58-vertex) and deer (40-vertex) classes form interlocked cycles with \(\mathop{\mathrm{link}}= -1\) (Gauss integral \(-1.0036\)); see Figure 12.

Linking-consistency definition. For a class pair \((X_i, X_j)\) and \(N\) independent runs (each regenerating the augmented dataset with a fresh random seed and recomputing graphs/cycles), the linking consistency is \(\mathrm{consistency}(X_i, X_j) = \tfrac{1}{N} \sum_r \mathbf{1}[|\mathop{\mathrm{link}}_r(X_i, X_j)| \ge 1]\). Variation across runs stems from augmentation randomness, sampling-dependent \(k\)-NN structure, and cycle-vertex placement; consistency measures how robustly a pair’s linking survives sampling variability rather than being a property of the underlying manifold.

The consistency runs do not binary-search \(\varepsilon\) independently for each pair. They use the same pre-arranged threshold sequence for all \(45\) pairs, \((0.11, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.3, 1.0, 30.0)\). Because the \(\varepsilon\)-filtered graph is monotone in \(\varepsilon\), a detected witness remains available at larger thresholds; the sequence therefore measures how small a scale suffices for detection, rather than whether some tuned threshold exists.

16.2 10-class linking consistency↩︎

Eleven runs across all \(45\) CIFAR-10 class pairs give a deliberately coarse robustness measure rather than a definitive topological classification. The distribution is still informative: \(27/45\) pairs are strongly linked (consistency \(\ge 0.7\)), \(8/45\) are weak or unlinked (\(\le 0.3\)), and the remaining \(10/45\) are ambiguous. Representative high-consistency pairs include deer–dog (91%), bird–cat (91%), cat–deer (91%), cat–dog (82%), and automobile–truck (91%). Representative low-consistency pairs include frog–ship (18%), airplane–horse (18%), airplane–automobile (27%), cat–ship (27%), and deer–truck (27%). We use this summary, rather than the full \(45\)-entry matrix, because the point is robustness of the signal across augmentation seeds and not a claim that each CIFAR-10 pair has an intrinsic binary linked/unlinked label.

16.3 CNN architecture↩︎

To probe whether the bounded-width topological obstruction applies to CNNs on CIFAR-10, we use width-bounded CNNs whose every intermediate representation has flattened dimension \(\le 3072\) (matching the input). Each spatial-resolution stage quadruples channels while halving each spatial dimension, preserving the \(C \cdot H \cdot W = 3072\) budget: \(3 \times 32^2 \to 12 \times 16^2 \to 48 \times 8^2 \to 192 \times 4^2\). Depth variants L5/L8/L11 use 1/2/3 ConvBlocks per stage; ConvBlocks are \(3 \times 3\) convs followed by activation (no batch normalization). When enabled, skip connections span each conv block: \(x \mapsto x + \mathcal{G}(x)\). Training: Adam at \(10^{-3}\), batch size \(128\), up to \(100\) epochs with patience-\(15\) early stopping on the standard CIFAR-10 train/test split.

16.4 10-class confusion vs.linking↩︎

For each of \(42\) trained models (7 activations \(\times\) 3 depths \(\times\) 2 skip settings) we compute the Spearman rank correlation between the linking-consistency scores and the symmetric confusion rate \(\mathrm{conf}(i,j) = P(\hat{y}=j \mid y=i) + P(\hat{y}=i \mid y=j)\) across all \(45\) class pairs. The pattern is stable across L5/L8/L11 and across activation choice: mean Spearman \(r \approx 0.47\) with \(p < 0.003\) throughout. Representative L11/no-skip values are ReLU \(r=0.436\) (\(p=0.0028\)), GELU \(r=0.501\) (\(p=0.0005\)), and Mish \(r=0.497\) (\(p=0.0005\)).

The same runs show a large separation in confusion rates. High-link pairs (consistency \(\ge 0.7\)) have about \(3\times\) the confusion of low-link pairs (consistency \(\le 0.3\)): ReLU gives \(6.4\%\) vs.\(2.1\%\), GELU gives \(6.0\%\) vs.\(1.8\%\), and Mish gives \(5.9\%\) vs.\(2.1\%\). We report representative values rather than the full per-activation table because the rows are redundant; the relevant observation is that the correlation persists across architectures and activations.

16.5 Distance-metric and projection controls↩︎

A natural concern is whether linking consistency simply measures inter-class distance. We compared linking consistency against two pixel-space distance metrics (mean Spearman \(|r|\) across all \(42\) models): linking consistency reaches \(\mathbf{0.479}\), average pairwise distance \(0.412\), minimum-distance \(0.356\). Linking is therefore not reducible to a simple distance proxy; it captures additional signal about how class manifolds are intertwined rather than merely how far apart they sit on average.

We also repeated detection under \(20\) random 3D projections. In the same projection-control protocol, PCA detected links in \(21/45\) class pairs, while random projections detected \(31\)–\(43/45\) pairs depending on the projection. Per-pair random-projection frequency ranged from \(11/20\) (frog–ship) to \(20/20\) (deer–dog), preserving the same weak/strong ordering but with less discrimination than PCA consistency (\(55\)–\(100\%\) vs.\(18\)–\(91\%\)). These random-projection frequencies vary projection subspaces, whereas the linking-consistency summary above varies augmentation seeds, so they should be read as complementary controls rather than the same estimator.

Caveats. The evidence above is suggestive, not causal. (i) Linking is detected in 3D PCA space, not the native \(3072\)D pixel space, so some detected linking may be projection artifacts and some genuine high-dimensional linking will be invisible after PCA. (ii) The linked-pair gap on the binary task is modest (\(1.2\%\)) and concentrated on the few hundred points that participate in detected cycles (\(\sim 0.1\%\) of each class). (iii) The 10-class confusion correlation is robust but partially confounded by semantic similarity; the within-dataset comparison controls factors like resolution and collection methodology but cannot disentangle topology from semantic similarity entirely. Multi-projection aggregation and higher-dimensional topological data analysis are natural next steps for stronger conclusions.

References↩︎