Proof. See Appendix 7. ◻

Proof. See Appendix 8. ◻

Proof. The proof directly follows from applying Proposition 8 in Appendix 7 to the separable Hilbert space \(\mathbb{D}^{s,2:d}_T\) (resp. \(\mathcal{H}^2_T(\mathbb{R}^{d})\) when \(s=0\)). ◻

Proof. See Appendix 9. ◻

Proof. See Appendix 9. ◻

Proof. See Appendix 10. ◻

Proof. First, we consider the (adapted) chaoslet orthonormal basis of \(\mathcal{H}^2_T(\mathbb{R}^{d})\), graded by homogeneous Wiener-chaos degree \(\mathfrak{C}=\bigcup_{k=0}^{\infty} \mathfrak{C}_k\). Then, we consider the chaoslet expansion of \(X \in \mathcal{H}^2_T(\mathbb{R}^{d})\): \(X = \sum_{k=0}^{\infty}\sum_{\mathfrak{m} \in \mathfrak{C}_k} \beta_{k,m} \mathfrak{m}\), with \(\beta_{k,m} \in \mathbb{R}^{d}\), and the non-increasing rearrangement, say \((\beta_r^{\star})_{r \geq 1}\), of the scalar family \((|\beta_{k,m}|_{\mathbb{R}^{d}})_{k \geq 0, \mathfrak{m} \in \mathfrak{C}_k}\). Since \(X\) is by assumption \(S\)-compressible, we have \(\beta_r^{\star} \leq C r^{-S}\) for all \(r \in \mathbb{N}_{+}\). Hence, by using that the chaoslets form an orthonormal basis of \(\mathcal{H}^2_T(\mathbb{R}^{d})\), the best \(N\)-term approximation is obtained by retaining the \(N\) largest coefficients in \(\mathbb{R}^{d}\)-norm; we denote this \(N\)-term truncation by \(X_{N}\). Thus, we have \[\|X-X_N\|^2_{\mathcal{H}^2_T(\mathbb{R}^{d})}=\sum_{r>N}|\beta_r^{\star}|^2 \leq C_X^2 \sum_{r>N} r^{-2 S},\] where the last inequality follows from the compressibility bound. Since \(S > \frac{1}{2}\) implying \(2S > 1\), we have \(\sum_{r>N} r^{-2 S} \lesssim_{X} N^{1-2 S}\) and thus \(\|X-X_N\|_{\mathcal{H}^2_T(\mathbb{R}^{d})} \lesssim_{X} N^{-\left(S-\frac{1}{2}\right)}\). This concludes the proof. ◻

Proof of Proposition 8. Set \(a_n\stackrel{\upshape\tiny def.}{=}\vartheta_n \xi_n\) for each \(n\in \mathbb{N}_+\). Since \((\xi_n)_{n=1}^{\infty}\) are \((C,c)\)-sub-Gaussian⁷, they have finite moments of every positive order, and in particular \(\mathbb{E}[|\xi_1|^{p_{\mathrm{cmp}}}]<\infty\). Thus, by the Fubini-Tonelli Theorem we have \[\mathbb{E}\Big[\sum_{n=1}^{\infty}|a_n|^{p_{\mathrm{cmp}}}\Big] = \sum_{n=1}^{\infty}|\vartheta_n|^{p_{\mathrm{cmp}}}\, \mathbb{E}[|\xi_1|^{p_{\mathrm{cmp}}}] = \mathbb{E}[|\xi_1|^{p_{\mathrm{cmp}}}] \|\vartheta\|_{\ell^{p_{\mathrm{cmp}}}}^{p_{\mathrm{cmp}}} < \infty .\] Therefore, \(\sum_{n=1}^{\infty}|a_n|^{p_{\mathrm{cmp}}}<\infty,\) \(\mathbb{P}\)-a.s., and so \(a\stackrel{\upshape\tiny def.}{=}(a_n)_{n=1}^{\infty}\in \ell^{p_{\mathrm{cmp}}},\) \(\mathbb{P}\)-a.s. Since \(0<p_{\mathrm{cmp}}<2\), one has the continuous embedding \(\ell^{p_{\mathrm{cmp}}}\subseteq \ell^2\). Indeed, if \(a=(a_n)_{n=1}^{\infty}\in \ell^{p_{\mathrm{cmp}}}\), then \(\{n\in \mathbb{N}_+:\;|a_n|>1\}\) is finite, and hence \[\sum_{n} |a_n|^2 = \sum_{|a_n|>1} |a_n|^2 + \sum_{|a_n| \leq 1} |a_n|^2 \leq \sum_{|a_n|>1} |a_n|^2 + \sum_{|a_n| \leq 1} |a_n|^{p_{\mathrm{cmp}}} <\infty .\] Then, \(\sum_{n=1}^{\infty}\|a_nUe_n\|_H^2 = \sum_{n=1}^{\infty}|a_n|^2 < \infty\), \(\mathbb{P}\)-a.s., as \(U\) is unitary. Thus, the series \(X = \sum_{n=1}^{\infty} a_n\,Ue_n = \sum_{n=1}^{\infty}\vartheta_n \xi_n\,Ue_n\) converges in \(H\) \(\mathbb{P}\)-a.s. In particular, \(X\) is a well-defined \(H\)-valued random variable, being the almost sure limit of the measurable partial sums \(X_M\stackrel{\upshape\tiny def.}{=}\sum_{n=1}^{M}\vartheta_n \xi_n\,Ue_n\). Let \(b_m\stackrel{\upshape\tiny def.}{=}\langle X,e_m\rangle_H\) for each \(m\in \mathbb{N}_+\), and write \(b\stackrel{\upshape\tiny def.}{=}(b_m)_{m=1}^{\infty}\). Since \((e_m)_{m=1}^{\infty}\) is an orthonormal basis of \(H\), \(b\) is precisely the coefficient sequence of \(X\) in the basis \(e_{\cdot}\). For each \(M\in \mathbb{N}_+\) and \(m\in \mathbb{N}_+\), \(\langle X_M,e_m\rangle_H = \sum_{n=1}^{M}\, a_n\langle Ue_n,e_m\rangle_H\). Since \(X_M\to X\) in \(H\), \(\mathbb{P}\)-a.s., we obtain \(b_m = \lim_{M\to\infty}\langle X_M,e_m\rangle_H = \lim_{M\to\infty}\sum_{n=1}^{M}a_n\langle Ue_n,e_m\rangle_H\), \(\mathbb{P}\)-a.s., for all \(m\in \mathbb{N}_+\). As \(A_U\stackrel{\upshape\tiny def.}{=}\big(\langle Ue_n,e_m\rangle_H\big)_{m,n=1}^{\infty}\) defines a bounded linear operator on \(\ell^{p_{\mathrm{cmp}}}\), and since \(a^{(M)}\stackrel{\upshape\tiny def.}{=}(a_1,\dots,a_M,0,0,\dots)\) converges to \(a\) in \(\ell^{p_{\mathrm{cmp}}}\), \(\mathbb{P}\)-a.s., we have \(A_Ua^{(M)}\to A_Ua\) in \(\ell^{p_{\mathrm{cmp}}}\), \(\mathbb{P}\)-a.s., and therefore coordinate-wise. Thus \(b=A_Ua\), \(\mathbb{P}\)-a.s. Therefore, if we write \(K_{p_{\mathrm{cmp}}}(U)\stackrel{\upshape\tiny def.}{=}\|A_U\|_{\ell^{p_{\mathrm{cmp}}}\to \ell^{p_{\mathrm{cmp}}}},\) then \(\|b\|_{\ell^{p_{\mathrm{cmp}}}} \le K_{p_{\mathrm{cmp}}}(U)\|a\|_{\ell^{p_{\mathrm{cmp}}}}\), \(\mathbb{P}\)-a.s. Thus, \(b\in \ell^{p_{\mathrm{cmp}}}\), \(\mathbb{P}\)-a.s.

Now fix \(\omega\) in the full-probability event on which both \(a(\omega)\in \ell^{p_{\mathrm{cmp}}}\) and \(b(\omega)\in \ell^{p_{\mathrm{cmp}}}\). Let \((b_k^{\star})_{k=1}^{\infty}\) denote the decreasing rearrangement of \((|b_m|)_{m=1}^{\infty}\). Since \(k\,(b_k^{\star})^{p_{\mathrm{cmp}}} \le \sum_{j=1}^{k}(b_j^{\star})^{p_{\mathrm{cmp}}} \le \sum_{j=1}^{\infty}(b_j^{\star})^{p_{\mathrm{cmp}}} = \|b\|_{\ell^{p_{\mathrm{cmp}}}}^{p_{\mathrm{cmp}}}\), it follows that \(b_k^{\star} \le \|b\|_{\ell^{p_{\mathrm{cmp}}}}\,k^{-1/p_{\mathrm{cmp}}}\) for every \(k\in \mathbb{N}_+\). Because \(e_{\cdot}\) is an orthonormal basis, the best \(N\)-term approximation error of \(X\) in the basis \(e_{\cdot}\) is obtained by retaining the \(N\) largest coefficients in modulus; equivalently, \(\|X-\Pi_N(X)\|_H^2 = \sum_{k>N}(b_k^{\star})^2\). Consequently, \(\|X-\Pi_N(X)\|_H^2 \le \|b\|_{\ell^{p_{\mathrm{cmp}}}}^2 \sum_{k>N}k^{-2/p_{\mathrm{cmp}}}\). Since \(0<p_{\mathrm{cmp}}<2\), one has \(2/p_{\mathrm{cmp}}>1\), and therefore there exists a constant \(C_{p_{\mathrm{cmp}}}>0\), depending only on \(p_{\mathrm{cmp}}\), such that \(\sum_{k>N}k^{-2/p_{\mathrm{cmp}}} \le C_{p_{\mathrm{cmp}}}\,N^{1-2/p_{\mathrm{cmp}}},\) for every \(N\in \mathbb{N}_+\). Taking square roots yields \[\|X-\Pi_N(X)\|_H \le C_{p_{\mathrm{cmp}}}^{1/2} \|b\|_{\ell^{p_{\mathrm{cmp}}}}\, N^{\frac{1}{2}-\frac{1}{p_{\mathrm{cmp}}}} = C_{p_{\mathrm{cmp}}}^{1/2} \|b\|_{\ell^{p_{\mathrm{cmp}}}}\, N^{-S_{\mathrm{app}}},\] where \(S_{\mathrm{app}}\stackrel{\upshape\tiny def.}{=}\frac{1}{p_{\mathrm{cmp}}}-\frac{1}{2}>0\). The bound on \(A_U\) gives for every \(N\in \mathbb{N}_+\), \[\|X-\Pi_N(X)\|_H \le C_{p_{\mathrm{cmp}}}^{1/2} K_{p_{\mathrm{cmp}}}(U) \|a\|_{\ell^{p_{\mathrm{cmp}}}}\, N^{-S_{\mathrm{app}}} \qquad \mathbb{P}-a.s.\] We now define \(\mathbf{C} \stackrel{\upshape\tiny def.}{=} C_{p_{\mathrm{cmp}}}^{1/2} K_{p_{\mathrm{cmp}}}(U) \|(a_n)_{n=1}^{\infty}\|_{\ell^{p_{\mathrm{cmp}}}} = C_{p_{\mathrm{cmp}}}^{1/2} K_{p_{\mathrm{cmp}}}(U) \big(\sum_{n=1}^{\infty}|\vartheta_n \xi_n|^{p_{\mathrm{cmp}}}\big)^{1/p_{\mathrm{cmp}}}\). Since each \(\omega\mapsto |\vartheta_n \xi_n(\omega)|^{p_{\mathrm{cmp}}}\) is measurable, the partial sums \((S_M(\omega)\stackrel{\upshape\tiny def.}{=}\sum_{n=1}^{M}|\vartheta_n \xi_n(\omega)|^{p_{\mathrm{cmp}}})_{M=1}^{\infty}\) are measurable, and \(\sum_{n=1}^{\infty}|\vartheta_n \xi_n|^{p_{\mathrm{cmp}}} = \lim_{M\to\infty}S_M\) is measurable as monotone limit of measurable functions. Hence \(\mathbf{C}\) is measurable and is a bona-fide random variable. Moreover, by the first part of the proof, \(\mathbf{C}<\infty\), \(\mathbb{P}\)-a.s. Therefore, \(\|X-\Pi_N(X)\|_H \le \mathbf{C}\,N^{-S_{\mathrm{app}}}\) for every \(N\in \mathbb{N}_+\), \(\mathbb{P}\)-a.s., which is equivalent to \(\mathbb{P}\big( \sup_{N\in \mathbb{N}_+} N^{S_{\mathrm{app}}}\|X-\Pi_N(X)\|_H \le \mathbf{C} \big) = 1.\) This proves the claim. ◻

Proof. First, we count the number of (distinct) dyadic sampling times. For every \(i=i_0,\ldots,i_0+I_{\rm H}\), we set \(S_i^T\stackrel{\upshape\tiny def.}{=}\left\{\frac{j T}{2^{i+1}}\,:\,j=0,\ldots,2 J_{\rm H} + 2\right\}\); whence, the set in the statement can be written as \(\mathcal{S}_{I_{\rm H}, J_{\rm H}, i_0}^T = \bigcup_{i=i_0}^{i_0+I_{\mathrm H}} S_i^T\). Notice that, since \(\frac{2 J_{\rm H}+2}{2^{i+1}} \leq \frac{2 J_{\rm H}+2}{2^{i_0+1}}=\frac{J_{\rm H}+1}{2^{i_0}}\leq 1\), all points of \(\mathcal{S}_{I_{\rm H}, J_{\rm H}, i_0}^T\) belong to \([0,T]\). Since multiplication by \(T>0\) is injective, it is enough to count the number of distinct points in the set \(\mathcal{S}_{I_{\mathrm H},J_{\mathrm H},i_0}^1\stackrel{\upshape\tiny def.}{=}\bigcup_{i=i_0}^{i_0+I_{\mathrm H}} \left\{ \frac{j}{2^{i+1}}: j=0,\ldots,2J_{\mathrm H}+2\right\}\). Now, set \(S_{i_0}^{1}\stackrel{\upshape\tiny def.}{=}\left\{\frac{j}{2^{i_0+1}}\,:\,j=0,\ldots,2J_{\mathrm H}+2\right\}\); it contains exactly \(2 J_{\mathrm H}+3\) points. For every finer level \(i=i_0+1,\ldots,i_0+I_{\mathrm H}\), the points with even numerator, already appeared at the previous level. Therefore, the only new points are those with odd numerator \(\frac{2 a + 1}{2^{i+1}}\), \(a=0,\ldots,J_{\mathrm{H}}\). Equivalently, if we write \(r=i+1\), the new reduced dyadic rationals are \(\frac{q}{2^{r}}\), \(r=i_0+2,\ldots,i_0+I_{\mathrm H}+1\) and \(q\in\{1,3,\ldots,2J_{\mathrm H}+1\}\). Therefore, the following disjoint decomposition holds true: \[\mathcal{S}_{I_{\mathrm H},J_{\mathrm H},i_0}^{1} = S_{i_0}^{1} \sqcup \left\{ \frac{q}{2^r}: r=i_0+2,\ldots,i_0+I_{\mathrm H}+1,\; q\in\{1,3,\ldots,2J_{\mathrm H}+1\} \right\}.\] Therefore, \(\#\mathcal{S}_{I_{\mathrm H},J_{\mathrm H},i_0}^{1}= (2J_{\mathrm H}+3)+ I_{\mathrm H}(J_{\mathrm H}+1)=(J_{\mathrm H}+1)(I_{\rm H}+2)+1\). In particular, if \(0=s_0<s_1<\cdots<s_{n_{\mathrm{grid}}}\le T\) is the increasing enumeration of the distinct elements of \(\mathcal{S}_{I_{\mathrm H},J_{\mathrm H},i_0}^T\), then \(n_{\mathrm{grid}}+1 = (J_{\mathrm H}+1)(I_{\mathrm H}+2)+1\). We need to verify that for every Brownian component \(r \in [D]_{+}\), the normalized Gaussian coordinate in Eq. 6 can be computed from the Brownian samples on this finite grid. In order to do so, it is sufficient to rewrite the sampling times appearing in Eq. 6 with the common denominator \(2^{i+1}\) and notice that, since \(0 \leq k \leq J_{\rm H}\), the resulting numerators all belong to \(\{0,\ldots,2 J_{H}+2\}\). Whence, all three sampling times belong to \(S_i^{T} \subset S_{I_{\mathrm H},J_{\mathrm H},i_0}^{T}\). Finally, each \(W_{s_j}\) is a \(D\)-dimensional Brownian vector. Therefore, the Brownian sample matrix \(\mathbf{W}^{I_{\rm H}, J_{\rm H}, i_0} \stackrel{\upshape\tiny def.}{=}[W_{s_0}^{\top},\ldots,W_{n_{\rm grid}}^{\top}]^{\top} \in \mathbb{R}^{(n_{\rm grid}+1)\times D}\). This concludes the proof. ◻

Proof. Fix any \(j \in [R]_{+}\). Since \(A\) is lower triangular, \(A_{j,i}=0\) for every \(i>j\) and, therefore, \((A \Xi)_{j:}=\sum_{i=1}^{n_{\rm grid}+1} A_{j,i} \xi_{s_{i-1}}=\sum_{i=1}^{j} A_{j,i} \xi_{s_{i-1}}\). Now, for each \(i \leq j\), since \(s_{i-1} \leq s_{j-1}\) and \(\mathcal{F}_{\cdot}=(\mathcal{F}_{t})_{0 \leq t \leq T}\) is a filtration, we have \(\mathcal{F}_{s_{i-1}} \subseteq \mathcal{F}_{s_{j-1}}\). Hence, \(\xi_{s_{i-1}} \in L^2(\mathcal{F}_{s_{j-1}};\mathbb{R}^{q})\), for every \(i \in [j]_{+}\). Since \(A\) is deterministic and the sum is finite, it follows that \((A \Xi)_{j:} \in L^2(\mathcal{F}_{s_{j-1}};\mathbb{R}^{q})\). Since \(j \in [R]_{+}\) is arbitrary, this concludes the proof. ◻

Proof. We write the rows of \(\mathbf{Z}^{I_{\mathrm H},J_{\mathrm H},i_0}\) as \((Z_{i_\ell,k_\ell})_{\ell=1}^{R_{\rm H}}\) ordered so that \(\frac{(k_\ell+1)T}{2^{i_\ell}}\) is non-decreasing in \(\ell\). Fix \(\ell \in [R_{\rm H}]_{+}\). By Lemma 3, \(\frac{k_\ell T}{2^{i_\ell}}, \frac{(2k_\ell+1)T}{2^{i_\ell+1}}, \frac{(k_\ell+1)T}{2^{i_\ell}}\) belong to \(\{s_0,\ldots,s_{n_{\mathrm{grid}}}\}\). Therefore, there exist unique indices \(a_\ell,b_\ell,c_\ell\in\{0,\ldots,n_{\mathrm{grid}}\}\) such that \(s_{a_\ell}=\frac{k_\ell T}{2^{i_\ell}}, s_{b_\ell}= \frac{(2k_\ell+1)T}{2^{i_\ell+1}}, s_{c_\ell}=\frac{(k_\ell+1)T}{2^{i_\ell}}\). Define \(A^{I_{\mathrm H},J_{\mathrm H},i_0} \in \mathbb{R}^{R_{\mathrm H}\times(n_{\mathrm{grid}}+1)}\) row by row as follows \(A^{I_{\mathrm H},J_{\mathrm H},i_0}_{\ell,m}\stackrel{\upshape\tiny def.}{=}\frac{2^{i_\ell/2}}{\sqrt T}\mathbf{1}_{\{m=a_\ell\}}-\frac{2^{i_\ell/2+1}}{\sqrt T}\mathbf{1}_{\{m=b_\ell\}}+ \frac{2^{i_\ell/2}}{\sqrt T}\mathbf{1}_{\{m=c_\ell\}}\) for \(m=0,\ldots,n_{\mathrm{grid}}\). Then, for each \(\ell=1,\ldots,R_{\mathrm H}\), \[\begin{align} \left( A^{I_{\mathrm H},J_{\mathrm H},i_0} \mathbf{W}^{I_{\mathrm H},J_{\mathrm H},i_0} \right)_{\ell:} &= \tfrac{2^{i_\ell/2}}{\sqrt T} W_{s_{a_\ell}} - \tfrac{2^{i_\ell/2+1}}{\sqrt T} W_{s_{b_\ell}} + \tfrac{2^{i_\ell/2}}{\sqrt T} W_{s_{c_\ell}} \\ &= \tfrac{2^{i_\ell/2}}{\sqrt T} W_{\frac{k_\ell T}{2^{i_\ell}}} - \tfrac{2^{i_\ell/2+1}}{\sqrt T} W_{\frac{(2k_\ell+1)T}{2^{i_\ell+1}}} + \tfrac{2^{i_\ell/2}}{\sqrt T} W_{\frac{(k_\ell+1)T}{2^{i_\ell}}} \\ &= Z_{i_\ell,k_\ell}., \end{align}\] and, therefore, \(A^{I_{\mathrm H},J_{\mathrm H},i_0} \mathbf{W}^{I_{\mathrm H},J_{\mathrm H},i_0}=\mathbf{Z}^{I_{\mathrm H},J_{\mathrm H},i_0}\). Moreover, by construction, every non-zero entry in the \(\ell^{th}\) row of \(A^{I_{\mathrm H},J_{\mathrm H},i_0}\) occurs in a column indexed by a time no later than \(\frac{k_{\ell}+1}{2^{i}\ell}\), so \(A^{I_{\mathrm H},J_{\mathrm H},i_0}\) is causal with respect to the chosen ordering. Finally, since \(\frac{k_\ell T}{2^{i_\ell}}<\frac{(2k_\ell+1)T}{2^{i_\ell+1}}<\frac{(k_\ell+1)T}{2^{i_\ell}}\), the indices \(a_\ell, b_\ell, c_\ell\) are pairwise distinct. Hence the \(\ell^{th}\) row of \(A^{I_{\mathrm H},J_{\mathrm H},i_0}\) has exactly three non-zero entries. Since the matrix has \(R_{\mathrm H} =(I_{\mathrm H}+1)(J_{\mathrm H}+1)\) rows, it follows that \(\left\|A^{I_{\mathrm H},J_{\mathrm H},i_0}\right\|_0 =3R_{\mathrm H}=3(I_{\mathrm H}+1)(J_{\mathrm H}+1)\). This concludes the proof. ◻

Proof. If \(m \in \{0,1\}\), then \(h_m(\cdot)\) is affine; whence, it can be represented by a \(\operatorname{ReLU-MLP}\) with a bounded number of parameters and (hidden) layers. In particular, after increasing the universal constant \(C\) if necessary, we obtain the claimed bound in Lemma 6. Therefore, we assume \(m \geq 2\), and define the rescaled polynomial \(q_{m,B}(u)\stackrel{\upshape\tiny def.}{=}h_m(Bu)=\sum_{r=0}^{m}h_{m,r}B^r u^r\), for every \(u \in [-1,1]\). Whence, the coefficient \(\ell^{1}\)-norm of \(q_{m,B}(\cdot)\) is bounded by \(\sum_{r=0}^{m}|h_{m,r}|B^r\leq A_{m,B}\). By [@petersen2024mathematical], there exists a universal constant \(C_0>0\) such that, after setting \(\delta \stackrel{\upshape\tiny def.}{=}\min\Big\{1,\,\frac{\varepsilon}{C_0A_{m,B}}\Big\}\), we deduce that there exists a \(\operatorname{ReLU-MLP}\) \(\Psi_{m,B,\delta}:\mathbb{R}\to\mathbb{R}\) such that \[\sup_{u\in[-1,1]} \left|q_{m,B}(u)-\Psi_{m,B,\delta}(u)\right|\le C_0\delta A_{m,B}\le \varepsilon.\] Moreover⁹, \(\operatorname{depth}(\Psi_{m,B,\delta}) \le C_0\log_2(m/\delta)\), \(\operatorname{width}(\Psi_{m,B,\delta}) \le C_0 m\) and \(\operatorname{size}(\Psi_{m,B,\delta})\le C_0m\log_2(m/\delta)\). We set \(\Phi^{\mathrm{Herm}}_{m,B,\varepsilon}(x)\stackrel{\upshape\tiny def.}{=}\Psi_{m,B,\delta}(x/B)\). Since \(x\in[-B,B]\) implies \(x/B\in[-1,1]\), we obtain \[\sup_{x\in[-B,B]} \left| h_m(x) - \Phi^{\mathrm{Herm}}_{m,B,\varepsilon}(x) \right| \le \varepsilon.\] Importantly, the affine map \(x\mapsto x/B\) can be absorbed into the first layer of the network. Whence, it does not change the depth, width, or number of non-zero parameters, up to the universal constant. Finally, since \(\delta =\min\left\{1,\frac{\varepsilon}{C_0A_{m,B}}\right\}\), and since \(A_{m,B}\ge 1\), \(m\ge2\), and \(0<\varepsilon<1\), we have, after increasing \(C\), \(\log_2(m/\delta)\le C\left(1+\log_2\left(\frac{mA_{m,B}}{\varepsilon}\right) \right)\). This gives the stated depth and size bounds. The width bound is \(C m\). This concludes the proof. ◻

Proof. First, we construct the one-dimensional cutoff map \(\mathcal{L}:\mathbb{R}\to\mathbb{R}\), and then parallelize it coordinate-wise. For later use in the lemma, we will specialize the parameter \(\eta\) in 20 to \(\eta=1\). \[\tag{20} \begin{align} \mathcal{L}(x) &= \begin{cases} 0, & x\le -\eta-\eta/\beta,\\ -\eta-\beta(x+\eta), & -\eta-\eta/\beta \le x\le -\eta,\\ x, & -\eta\le x\le \eta,\\ \eta-\beta(x-\eta), & \eta\le x\le \eta+\eta/\beta,\\ 0, & x\ge \eta+\eta/\beta. \end{cases} \end{align} \qquad \vcenter{ \begin{figure}\includegraphics[width=0.8\textwidth]{_pdflatex/mgcuydbw.png}\tag{21}\end{figure} }\] We compute as a two-layer \(\operatorname{ReLU}\)-MLP as \(\mathcal{L}=\mathcal{L}_2\circ\mathcal{L}_1\), where \[\label{eq:MLP} \begin{align} \mathcal{L}_1(x) & \stackrel{\upshape\tiny def.}{=} \left( \begin{pmatrix} 1 & -1 & 0 & 0\\ 0 & 0 & -\beta & 0 \\ 0 & 0 & 0 & -\beta \end{pmatrix} \operatorname{ReLU} \bullet \left( \begin{pmatrix} x + 1 \\ x - 1 \\ x - 1 \\ -x - 1 \end{pmatrix} \right) \right) + \begin{pmatrix} -1 \\ 0 \\ 0 \end{pmatrix} \\ \mathcal{L}_2(u_1,u_2,u_3) & \stackrel{\upshape\tiny def.}{=} (1,-1)^{\top} \operatorname{ReLU} \bullet \left( \begin{pmatrix} 1 & 1 & 0 \\ -1 & 0 & 1 \end{pmatrix} \begin{pmatrix} u_1\\ u_2 \\ u_3 \end{pmatrix} \right) . \end{align}\tag{22}\] Thus \(\mathcal{L}(x)=x\) on \([-1,1]\), \(\mathcal{L}(x)=0\) on \((-\infty,-1-1/\beta]\cup[1+1/\beta,\infty)\), and \(\mathcal{L}(\mathbb{R})\subseteq[-1,1]\). Observe that \(\mathcal{L}_1\) has width \(4\), depth \(1\), and \(13\) non-zero parameters; while \(\mathcal{L}_2\) has width \(3\), depth \(1\), and \(9\) non-zero parameters. Together, \(\mathcal{L}\) thus has width \(4\), depth \(2\), and can be expressed using \(22\) non-zero parameters

Now, we parallelize this construction over the \(q_{\rm in }\) coordinates; define \(\mathcal{L}^{\uparrow}:\mathbb{R}^{q_{\rm in }}\to\mathbb{R}^{q_{\rm in }}\) by \[\mathcal{L}^{\uparrow}(x) \stackrel{\upshape\tiny def.}{=} \bigl( \mathcal{L}(x_1),\dots,\mathcal{L}(x_{q_{\rm in }}) \bigr).\] Then \(\mathcal{L}^{\uparrow}(x)=x\) for every \(x\in[-1,1]^{q_{\rm in }}\), and \(\mathcal{L}^{\uparrow}(\mathbb{R}^{q_{\rm in }})\subseteq[-1,1]^{q_{\rm in }}\). In particular, \(\mathcal{L}^{\uparrow}(x)=x\) for every \(x\in[0,1]^{q_{\rm in }}\). Observe also that \(\mathcal{L}^{\uparrow}\) has width \(4 q_{\rm in }\), depth \(2\), and can be expressed using at most \(22 q_{\rm in }\) non-zero parameters; it implements \(\mathcal{L}\) in 20 component-wise.

By setting \(g \stackrel{\upshape\tiny def.}{=} f\circ \mathcal{L}^{\uparrow}\) and using that \(\mathcal{L}^{\uparrow}(\mathbb{R}^{q_{\rm in }})\subseteq[-1,1]^{q_{\rm in }}\), and that \(f\) is continuous, the constant \(C_{f} \stackrel{\upshape\tiny def.}{=} \max\big\{ 1, \sup_{z\in[-1,1]^d}\|f(z)\|_{\infty} \big\}\) is finite. The map \(g\) agrees with \(f\) on \([0,1]^{q_{\rm in }}\), but it need not vanish outside the localization region. We therefore add one scalar cutoff gate. Define \(r(x) \stackrel{\upshape\tiny def.}{=} \max_{i\in [q_{\rm in }]_{+}} \,\{ \operatorname{ReLU}(x_i-1),\,\operatorname{ReLU}(-x_i-1) \}\). Then \(r(x)=0\) for every \(x\in[-1,1]^{q_{\rm in }}\), while \(r(x)\ge 1/\beta\) whenever \(\|x\|_{\infty}\ge 1+1/\beta\). Set \(q(x) \stackrel{\upshape\tiny def.}{=} 2 C_{f}\beta\, r(x)\). Finally, we define the components of \(\bar{f}\), for \(j\in[q_{\rm out}]_{+}\), by \[\label{eq:def95bar95f} \bar f_j(x) \stackrel{\upshape\tiny def.}{=} \frac{1}{2}\, \operatorname{ReLU}\bigl(g_j(x)+C_{f}-q(x)\bigr) - \frac{1}{2}\, \operatorname{ReLU}\bigl(-g_j(x)+C_{f}-q(x)\bigr) .\tag{23}\] We now verify the two desired properties. If \(x\in[0,1]^{q_{\rm in}}\), then \(\mathcal{L}^{\uparrow}(x)=x\), and hence \(g(x)=f(x)\). Moreover, \(r(x)=0\), so \(q(x)=0\). Since \(|g_j(x)|=|f_j(x)|\le C_{f}\), we have \(\bar f_j(x) = \frac{1}{2}\bigl(g_j(x)+C_{f}\bigr) - \frac{1}{2}\bigl(-g_j(x)+C_{f}\bigr) = g_j(x) = f_j(x)\). Thus \(\bar f(x)=f(x)\) on \([0,1]^{q_{\rm in}}\). Conversely, suppose that \(\|x\|_{\infty}\ge 1+1/\beta\). Then \(r(x)\ge 1/\beta\), and therefore \(q(x)\ge 2C_{f}\). Since \(\mathcal{L}^{\uparrow}(x)\in[-1,1]^{q_{\rm in}}\), the definition of \(C_{f}\) gives \(|g_j(x)|\le C_{f}\) for every \(j\in[q_{\rm out}]_{+}\). Hence \(g_j(x)+C_{f}-q(x)\le 0\) and \(-g_j(x)+C_{f}-q(x)\le 0\). Both ReLU terms in the definition of \(\bar f_j(x)\) therefore vanish, and so \(\bar f_j(x)=0\) for every \(j\in[q_{\rm out}]_{+}\). Thus \(\bar f(x)=0\) whenever \(\|x\|_{\infty}\ge 1+1/\beta\).

It remains to finish recording the network complexity. Composing the first affine layer of \(\Phi\) with the output of \(\mathcal{L}^{\uparrow}\) costs at most an additional \(S_{\operatorname{in}}(\Phi)\) non-zero parameters, since each non-zero input weight of \(\Phi\) is applied to a coordinate produced by the localization block. Consequently, the branch computing \(g=f\circ\mathcal{L}^{\uparrow}\) satisfies \(\operatorname{depth}(g) \leq L+2\), \(\operatorname{width}(g) \leq W+4q_{\rm in}\), and \(\operatorname{size}(g) \leq S + S_{\operatorname{in}}(\Phi) + C \,q_{\rm in}\). The scalar branch computing \(r\) uses the standard ReLU maximum identity \(\max\{a,b\} = b+\operatorname{ReLU}(a-b)\), iterated as a binary tree over the \(2 q_{\rm in}\) non-negative quantities \(\operatorname{ReLU}(x_i-1)\) and \(\operatorname{ReLU}(-x_i-1)\), \(i\in[q_{\rm in}]_{+}\). This gives depth at most \(\ell_{q_{\rm in}}+2\) and size \(O(q_{\rm in})\), after increasing the universal constant if necessary. The affine rescaling \(r\mapsto q=2C_{f}\beta r\) costs one additional non-zero parameter. Let \(L_g\) and \(L_q\) denote the depths of the branches realizing \(g\) and \(q\), respectively. We choose the above realizations so that \(L_g=L+2\) and \(L_q\le \ell_{q_{\rm in}}+2\). Set \(m\stackrel{\upshape\tiny def.}{=}\max\{L+2,\ell_{q_{\rm in}}+2\}\), \(n_g\stackrel{\upshape\tiny def.}{=}m-L_g\), and \(n_q\stackrel{\upshape\tiny def.}{=}m-L_q\). Then \(n_g,n_q\in\mathbb{N}_{+}\), \(n_g\le \ell_{q_{\rm in}}\), and \(n_q\le L+2\).

We now equalize the depths of the two branches. For the \(g\)-branch, use the exact ReLU identity \[\operatorname{Id}_{\mathbb{R}^q_{\rm in}}(z) = \begin{pmatrix} I_{q_{\rm out}} & -I_{q_{\rm out}} \end{pmatrix} \operatorname{ReLU} \bullet \begin{pmatrix} I_{q_{\rm out}}\\ -I_{q_{\rm out}} \end{pmatrix} z, \qquad z\in\mathbb{R}^{_{q_{\rm out}}}.\] Each such identity block has width \(2 q_{\rm out}\) and exactly \(4 q_{\rm out}\) non-zero weights. Composing the \(g\)-branch with \(n_g\) such blocks preserves the realized map exactly and increases its depth to \(m\). The resulting additional size is at most \(4 q_{\rm out} n_g \le 4 q_{\rm out}\ell_d\).

For the \(q\)-branch, observe that \(q(x)\ge0\) for every \(x\in\mathbb{R}^{q_{\rm in}}\). Hence, on the range of the \(q\)-branch, the scalar identity is realized by \(\operatorname{Id}_{[0,\infty)}(s) = \operatorname{ReLU}(s)\) for \(s\ge0\). Each such padding block has width \(1\) and one non-zero weight. Composing the \(q\)-branch with \(n_q\) such blocks preserves the realized map exactly and increases its depth to \(m\). The additional size is at most \(n_q \le L+2\). Thus, after padding, the two branches realize the same functions \(g\) and \(q\), respectively, and both outputs are available at depth \(m\). The padding increases the width by at most \(2 q_{\rm out} +1\) and the size by at most \(4 q_{\rm out} \ell_{q_{\rm in}} + L + 2 \le C(q_{\rm out} \ell_{q_{\rm in}} + L)\) for a universal strictly positive constant \(C>0\).

It remains to implement the terminal gate. Let \((z,s)\in\mathbb{R}^{q_{\rm out}}\times[0,\infty)\) represent the output of the padded branches, where \(z=g(x)\) and \(s=q(x)\). Define the affine map \(A:\mathbb{R}^{q_{\rm out}+1}\to\mathbb{R}^{2 q_{\rm out}}\) by \((A(z,s))_{2j-1}=z_j+C_{f}-s,\) and \((A(z,s))_{2j}=-z_j+C_{f}-s\) for each \(j\in[q_{\rm out}]_+\). and all \(2 q_{\rm out}\) biases are equal to \(C_{f}\). Therefore \(A\) has \(4 q_{\rm out}\) non-zero weights and \(2 q_{\rm out}\) non-zero biases. Let \(R(z,s)\stackrel{\upshape\tiny def.}{=}\operatorname{ReLU}\bullet A(z,s)\in\mathbb{R}^{2 q_{\rm out}}.\) Finally, define the linear map \(T:\mathbb{R}^{2 q_{\rm out}}\to\mathbb{R}^{q_{\rm out}}\) by \((Ty)_j \stackrel{\upshape\tiny def.}{=} \frac{1}{2}y_{2j-1}-\frac{1}{2}y_{2j}\) for each \(j\in[q_{\rm out}]_+\). The map \(T\) has exactly \(2 q_{\rm out}\) non-zero weights. Hence \(T\circ R\) is realized by one ReLU layer of width \(2 q_{\rm out}\) and has at most \(4 q_{\rm out}+2q_{\rm out}+2q_{\rm out}=8q_{\rm out}\) non-zero parameters. Moreover, for every \(x\in\mathbb{R}^{q_{\rm in}}\) and every \(j\in[q_{\rm out}]_{+}\), \[(T\circ R)_j(g(x),q(x)) = \frac{1}{2} \operatorname{ReLU}\bigl(g_j(x)+C_{f}-q(x)\bigr) - \frac{1}{2} \operatorname{ReLU}\bigl(-g_j(x)+C_{f}-q(x)\bigr) = \bar f_j(x).\] Therefore, the terminal gate realizes \(\bar f\) exactly, increases the depth by one, increases the width by at most \(2 q_{\rm out}\), and increases the size by at most \(8 q_{\rm out}\). Combining the padding and terminal-gate estimates, the additional size beyond the two branches is bounded by \(4 q_{\rm out} \ell_{q_{\rm in}}+L+2+8 q_{\rm out} \le C(q_{\rm out}\ell_{q_{\rm in}}+L+q_{\rm out}),\) after increasing the universal constant \(C>0\) if necessary. Combining the estimates yields \(\operatorname{depth}(\bar{\Phi}_{\beta}) \le \max\{L+2,\ell_{q_{\rm in}}+2\}+1\), \(\operatorname{width}(\bar{\Phi}_{\beta}) \le W+4 q_{\rm in}+2q_{\rm out}+2\), and \(\operatorname{size}(\bar{\Phi}_{\beta}) \le S + S_{\operatorname{in}}(\Phi) + C\bigl(q_{\rm in}+q_{\rm out}\ell_{q_{\rm in}}+L+q_{\rm out}\bigr)\), after increasing \(C>0\) if necessary. This concludes the proof. ◻

Proof. It is sufficient to apply Lemma 6 only in the scalar identity case; i.e., by localizing the identity map on the interval \([-R,R]\) with transition width \(1/\beta\), and then applying the resulting scalar map coordinate-wise. The asserted complexity bounds follow directly from Lemma 7. ◻

Proof. By [@petersen2024mathematical] there exists a \(\operatorname{ReLU-MLP}\) \(\tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}:\mathbb{R}^{q_{\times}}\to\mathbb{R}\) supported in \([-R_{\times}+\frac{1}{\beta_{\times}}, R_{\times}-\frac{1}{\beta_{\times}}]^{q_{\times}}\) and sending \([-R_{\times},R_{\times}]^{q_{\times}}\) to \([-R_{\times}^{q_{\times}},R_{\times}^{q_{\times}}]\) such that \[\sup_{x_j\in[-R_{\times},R_{\times}]} \, \Big| \prod_{j=1}^{q_{\times}} x_j - \tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}(x_1,\dots,x_{q_{\times}}) \Big| \le \varepsilon_{\times}.\] Moreover, we have \(\operatorname{depth}(\tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}) \leq C \log(q_{\times})\left( 1+ \Big|\log_2\left(\frac{\varepsilon_{\times}}{q_{\times} R_{\times}^{q_{\times}}}\right)\Big| \right)\), \(\operatorname{width}(\tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}) \leq C q_{\times}\), and \(\operatorname{size}(\tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}) \leq C q_{\times}\left(1+ \Big|\log_2\left(\frac{\varepsilon_{\times}}{q_{\times} R_{\times}^{q_{\times}}}\right)\Big| \right)\). Now, let \(p_{R_{\times},\beta_{\times}}:\mathbb{R}^{q_{\times}} \to \mathbb{R}^{q_{\times}}\) be the depth-\(1\) \(\operatorname{ReLU-MLP}\) Corollary 1, and define the \(\operatorname{ReLU-MLP}\) in the statement of the lemma as \(\Phi^{\times,R_{\times},\beta_{\times}}_{q_{\times}}\stackrel{\upshape\tiny def.}{=}\tilde{\Phi}^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}} \circ p_{R_{\times},\beta_{\times}}\). Then, for every \(x\in [-R_{\times}-\frac{1}{\beta_{\times}}, R_{\times}+\frac{1}{\beta_{\times}}]^{q_{\times}}\), since \(p_{R_{\times},\beta_{\times}}(x)=x\), we have \[\Big| \prod_{j=1}^{q_{\times}} x_j - \Phi^{\times,-R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}(\mathbf{x}) \Big| = \Big| \prod_{j=1}^{q_{\times}} x_j - \widetilde{\Phi}^{\times,-R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}(\big(p_{R_{\times},\beta_{\times}}(\mathbf{x})\big) \Big| = \Big| \prod_{j=1}^{q_{\times}} x_j - \widetilde{\Phi}^{\times,-R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}(\mathbf{x}\big) \Big|\leq \varepsilon_{\times}.\]

Taking the supremum over \(\mathbf{x}\in [-R_{\times}-\frac{1}{\beta_{\times}}, R_{\times}+\frac{1}{\beta_{\times}}]^{q_{\times}}\) yields the desired estimate. Since \(p_{R_{\times},\beta_{\times}}(\mathbb{R}^{q_{\times}})\subseteq [-R_{\times}-\frac{1}{\beta_{\times}}, R_{\times}+\frac{1}{\beta_{\times}}]^{q_{\times}}\) sends \([-R_{\times}-\frac{1}{\beta_{\times}}, R_{\times}+\frac{1}{\beta_{\times}}]^{q_{\times}}\) to \([-R_{\times}^{q_{\times}}, R_{\times}^{q_{\times}}]\), it follows that \(\Phi^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}} (\mathbb{R}^{q_{\times}}) \subseteq [-R_{\times},R_{\times}]^{q_{\times}}\). Now, let \(\mathbf{x}\in \mathbb{R}^{q_{\times}}\setminus (-R_{\times},R_{\times})^{q_{\times}}\). Then there exists \(i\in [q_{\times}]_{+}\) such that \(x_i\not\in (-R_{\times},R_{\times})\). By Corollary 1, \(p_{R_{\times},\beta_{\times}}(x)_{i}=0\) and, therefore, \(\prod_{j=1}^{q_{\times}} p_{R_{\times},\beta_{\times}}(x)_{j}=0\). Since \(p_{R_{\times},\beta_{\times}}(\mathbf{x})\in [-R_{\times}^{q_{\times}},R_{\times}^{q_{\times}}]^{q_{\times}}\), the approximation property of \(\widetilde{\Phi}^{\times,R_{\times}}_{q_{\times},\varepsilon_{\times}}\) gives \[\begin{align} \big| \Phi^{\times,R_{\times},\beta_{\times}}_{q_{\times},\varepsilon_{\times}}(\mathbf{x}) \big| &= \Big| \widetilde{ \Phi}^{\times,R_{\times}}_{q_{\times},\varepsilon_{\times}}\big(p_{R_{\times},\beta_{\times}}(\mathbf{x})\big) - \prod_{j=1}^{q_{\times}} p_{R_{\times},\beta_{\times}}(x)_{j} \Big| \le \varepsilon. \end{align}\] Finally, composition with the depth \(1\) width \(4 q_{\times}\) network \(p_{R_{\times},\beta_{\times}}\) increases the depth by \(1\), while the width remains of order \(q_{\times}\) and the number of non-zero parameters increases by only \(O(q_{\times})\). After enlarging the absolute constant \(C\) if necessary, the claimed complexity bounds follow. ◻

Proof. If \(n=0\), then \(h_{\boldsymbol{\alpha}}\equiv1\) and the claim is represented exactly by a constant \(\operatorname{ReLU-MLP}\), and the stated bounds hold after increasing the positive constant \(C>0\). We, therefore, assume that \(n \geq 1\). Now, if \(Q_n=1\), then the tensorized polynomial is univariate and the claim follows directly from Lemma 6, again after increasing the universal constant \(C\). Whence, hereafter, we assume \(Q_n \geq 2\); for every pair \((j,r)_{j \in [J_n]_{+}, r \in [D]_{+}}\), we construct a uni-variate \(\operatorname{ReLU-MLP}\) \(\Phi_{j,r}:\mathbb{R}\rightarrow\mathbb{R}\). If \(\alpha_{j,r}=0\), then \(h_{\alpha_{j,r}} \equiv 1\), and we let \(\Phi_{j,r}\equiv 1\). Otherwise, Lemma 6 yields a \(\operatorname{ReLU-MLP}\) \(\Phi_{j,r}:\mathbb{R}\rightarrow\mathbb{R}\) such that \(\sup_{x \in [-B,B]}|h_{\alpha_{j,r}}(x)-\Phi_{j,r}(x)| \leq \delta\), where \(\delta=\frac{\varepsilon}{2 Q_n A_{\boldsymbol{\alpha},B}^{Q_n-1}}\) (\(< 1\)). Moreover, for every \(x \in [-B,B]\), \(|h_{\alpha_{j,r}}(x)| \le A_{\alpha_{j,r},B}-1 \le A_{\boldsymbol{\alpha},B}-1\). Since \(\delta<1\), it follows that \(|\Phi_{j,r}(x)|\le A_{\boldsymbol{\alpha},B}\) for every \(x \in [-B,B]\). In particular, setting, \(\Psi(\boldsymbol{x}) \stackrel{\upshape\tiny def.}{=}\Phi_{j,r}(x_{j,r})_{j \in [J_n]_{+}, r \in [D]_{+}} \in \mathbb{R}^{Q_n}\), we have, as desired, that \(\Psi([-B,B]^{J_n \times D}) \subseteq [-A_{\boldsymbol{\alpha},B},A_{\boldsymbol{\alpha},B}]^{Q_n}\). Now, apply Lemma 8 with (the admissible choice of) \(q_{\times}=Q_n, R_{\times}=2A_{\boldsymbol{\alpha},B}, \beta_{\times}=A_{\boldsymbol{\alpha},B}^{-1}, \varepsilon_{\times}=\frac{\varepsilon}{2}\), yields a \(\operatorname{ReLU-MLP}\) \(\Psi_{Q_n}^{\times}:\mathbb{R}^{Q_n}\to\mathbb{R}\) such that \[\sup_{\boldsymbol{y}\in[-A_{\boldsymbol{\alpha},B},A_{\boldsymbol{\alpha},B}]^{Q_n}} \left| \prod_{i=1}^{Q_n}y_i - \Phi^\times_{Q_n}(\boldsymbol{y}) \right| \le \frac{\varepsilon}{2}.\] We now consider the composition \(\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon} \stackrel{\upshape\tiny def.}{=}\Psi_{Q_n}^{\times} \circ \Psi\), and fix \(\boldsymbol{x} \in [-B,B]^{J_n \times D}\). Moreover, we choose any enumeration \(\{(j_i, r_i)\,:\,i=1,\ldots,Q_n\}\) of \([J_n]_{+} \times [D]_{+}\), and we set \(u_i\stackrel{\upshape\tiny def.}{=}h_{\alpha_{j_i, r_i}}(x_{j_i, r_i})\) and \(v_i \stackrel{\upshape\tiny def.}{=}\Phi_{j_i, r_i}(x_{j_i, r_i})\), \(i \in [Q_n]_{+}\); in particular, \(h_{\boldsymbol{\alpha}}(\boldsymbol{x})=\prod_{i=1}^{Q_n} u_i\) and \(\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon}(\boldsymbol{x})=\Phi^{\times}_{Q_n}(v_1,\ldots,v_{Q_n})\). Whence, \[|h_{\boldsymbol{\alpha}}(\boldsymbol{x})-\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon}(\boldsymbol{x})| \leq \left| \prod_{i=1}^{Q_n}u_i - \prod_{i=1}^{Q_n}v_i \right| + \left| \prod_{i=1}^{Q_n}v_i - \Phi^\times_{Q_n}(v_1,\ldots,v_{Q_n}) \right| \leq \left| \prod_{i=1}^{Q_n}u_i - \prod_{i=1}^{Q_n}v_i \right| + \frac{\varepsilon}{2}.\] In order to estimate the first term, we use the telescoping identity \[\prod_{i=1}^{Q_n}u_i-\prod_{i=1}^{Q_n}v_i = \sum_{m=1}^{Q_n} \left(\prod_{i=1}^{m-1}v_i\right) (u_m-v_m) \left(\prod_{i=m+1}^{Q_n}u_i\right),\,\,\text{and}\,\,\left| \prod_{i=1}^{Q_n}u_i-\prod_{i=1}^{Q_n}v_i \right| \le Q_n\delta A_{\boldsymbol{\alpha},B}^{Q_n-1} = \frac{\varepsilon}{2},\] where we use the fact that \(|u_i|\le A_{\boldsymbol{\alpha},B}\), \(|v_i|\le A_{\boldsymbol{\alpha},B}\), and \(|u_i-v_i| \leq \delta\). Combining the two estimates yields \(\left|h_{\boldsymbol{\alpha}}(z)-\Phi^{\rm Herm}_{\boldsymbol{\alpha},B,\varepsilon}(z) \right|\le \varepsilon\). Taking the supremum over \(\boldsymbol{x} \in[-B,B]^{J_n\times D}\) proves the approximation claim. Finally, we verify the complexity bounds. For every \((j,r)\) with \(\alpha_{j,r}\ge1\), Lemma 6 and the choice of \(\delta\) give a univariate network with \(\operatorname{width}(\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon}) \leq C\alpha_{j,r}\), \(\operatorname{depth}(\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon}) \leq C\left[ 1 \!+\! \log_2\left( 1 \!+\! \frac{ 2Q_n\alpha_{j,r} A_{\alpha_{j,r},B} A_{\boldsymbol{\alpha},B}^{Q_n-1} }{ \varepsilon } \right) \right]\), and \(\operatorname{size}(\Phi^{\rm Herm}_{\boldsymbol{\alpha}, B, \varepsilon}) \leq C\alpha_{j,r} \left[ 1 \!+\! \log_2\left( 1 \!+\! \frac{ 2Q_n\alpha_{j,r} A_{\alpha_{j,r},B} A_{\boldsymbol{\alpha},B}^{Q_n-1} }{ \varepsilon } \right) \right]\). Notice that for entries with \(\alpha_{j,r}=0\), the constant network contributes only bounded complexity. After parallelizing all \(Q_n\) univariate networks and padding shallower networks to a common depth, the univariate stage \(\Psi\) has \(\operatorname{depth}(\Psi) \leq C\left[ 1+ \max_{\substack{j \in [J_n]_{+}\\ r\;\in [D]_{+}}} \log_2\left( 1+ \frac{ 2Q_n\alpha_{j,r} A_{\alpha_{j,r},B} A_{\boldsymbol{\alpha},B}^{Q_n-1} }{ \varepsilon } \right) \right]\), \(\operatorname{width}(\Psi) \leq C(Q_n+n)\), and \(\operatorname{size}(\Psi) \leq C(Q_n+n) \left[ 1+ \max_{\substack{j \in [J_n]_{+}\\ r\;\in [D]_{+}}} \log_2\left( 1+ \frac{ 2Q_n\alpha_{j,r} A_{\alpha_{j,r},B} A_{\boldsymbol{\alpha},B}^{Q_n-1} }{ \varepsilon } \right) \right]\). The product stage is controlled by Lemma 8. For \(q_{\times}=Q_n\), \(R_{\times}=2A_{\boldsymbol{\alpha},B}\), and \(\varepsilon_{\times}=\varepsilon/2\), we have \(\frac{q_{\times}R_{\times}^{q_{\times}}}{\varepsilon_{\times}}= \frac{ 2^{Q_n+1}Q_nA_{\boldsymbol{\alpha},B}^{Q_n} }{ \varepsilon }\). Thus, \(\Phi^\times_{Q_n}\) has \(\operatorname{depth}(\Phi^\times_{Q_n}) \leq C L_{Q_n} \left[ 1+ \log_2\left( \frac{ 2^{Q_n+1}Q_nA_{\boldsymbol{\alpha},B}^{Q_n} }{ \varepsilon } \right) \right]\), \(\operatorname{width}(\Phi^\times_{Q_n}) \leq CQ_n\), and \(\operatorname{size}(\Phi^\times_{Q_n}) \leq CQ_n \left[ 1+ \log_2\left( \frac{ 2^{Q_n+1}Q_nA_{\boldsymbol{\alpha},B}^{Q_n} }{ \varepsilon } \right) \right].\) Since, \(\Phi^{\rm Herm}_{\boldsymbol{\alpha},B,\varepsilon}=\Phi^\times_{Q_n}\circ\Psi\), its depth is bounded by the sum of the depths of the two stages, the width by the maximum of the widths, and the number of non-zero parameters by the sum of the numbers of non-zero parameters. After increasing the universal constant \(C\), this gives exactly the stated bounds. ◻

Proof. Morally, the proof hinges on Lemmas 7 and 9. Precisely, we identify \(\mathbb{R}^{J_n \times D}\) with \(\mathbb{R}^{Q_n}\) and write \(h_{\boldsymbol{\alpha}}(\boldsymbol{z})=\prod_{i=1}^{Q_n}h_{\alpha_i}(z_i)\). In addition, we set \(B_{+} \stackrel{\upshape\tiny def.}{=}B+1\) and \(\mathcal{C}_{\boldsymbol{\alpha}, B} \stackrel{\upshape\tiny def.}{=}1 + \sup_{\boldsymbol{z} \in [-B_{+},B_{+}]^{Q_n}} |h_{\boldsymbol{\alpha}}(\boldsymbol{z})|\). By Lemma 9, for every \(\delta\in(0,1)\), there exists a \(\operatorname{ReLU-MLP}\), say \(\Psi_{\boldsymbol{\alpha}, B_{+}, \delta}\) (with \(\delta\) chosen below) such that \(\sup_{\boldsymbol{z}\in[-B_+,B_+]^{Q_n}} \left| h_{\boldsymbol{\alpha}}(\boldsymbol{z}) - \Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z}) \right| \le \delta\). For every coordinate, we define the \(\operatorname{ReLU-MLP}\) -realizable hat function \[\theta_B(t)\stackrel{\upshape\tiny def.}{=}1-\operatorname{ReLU}(|t|-B) + \operatorname{ReLU}(|t|-B-1), \qquad |t|=\operatorname{ReLU}(t)+\operatorname{ReLU}(-t).\] In particular, we have \(0\le\theta_B(t)\le1\), \(\theta_B(t)=1\) for \(t \in [-B,B]\), and \(\theta_B(t)=0\) for \(t \not\in (-B_{+},B_{+})\). As a consequence, \(\prod_{i=1}^{Q_n}\theta_B(z_i)=1\) on \([-B, B]^{Q_n}\), while this product vanishes outside \((-B_+,B_+)^{J_n\times D}\). At this point, we apply Corollary 1 in dimension one with (localization) radius \(\mathcal{C}_{\boldsymbol{\alpha},B}+2\) and margin \(1\). This gives a \(\operatorname{ReLU-MLP}\) \(P^{(1)}_{B_{+},1}:\mathbb{R}\rightarrow [-(\mathcal{C}_{\boldsymbol{\alpha}, B}+2),(\mathcal{C}_{\boldsymbol{\alpha}, B}+2)]\) which is the identity on \([-(\mathcal{C}_{\boldsymbol{\alpha}, B}+1), (\mathcal{C}_{\boldsymbol{\alpha}, B}+1)]\) and vanishes outside \([-(\mathcal{C}_{\boldsymbol{\alpha}, B}+2),(\mathcal{C}_{\boldsymbol{\alpha}, B}+2)]\). Since, provided that \(\delta \leq 1\), \(|\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})|\leq \mathcal{C}_{\boldsymbol{\alpha}, B}+1\) on \([-(B_{+}+1),(B_{+}+1)]^{Q_n}\), we have that \(P^{(1)}_{B_{+},1}(\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z}))=\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})\) for every \(\boldsymbol{z} \in [-(B_{+}+1),(B_{+}+1)]^{Q_n}\). Now, we use Lemma 8 with \(q_{\times}=Q_n+1\), \(R_{\times}=\mathcal{C}_{\boldsymbol{\alpha}, B}+3\), \(\beta_{\times}=1\), and \(\delta_{\times}\in (0,1)\), and apply the resulting product network to the \(Q_n+1\) scalar inputs \(P^{(1)}_{B_{+},1}(\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})), \theta_B(z_1),\ldots,\theta_B(z_{Q_n})\). By choosing the multiplication accuracy \(\delta_{\times}>0\) sufficiently small, we obtain a \(\operatorname{ReLU-MLP}\) \(\Phi_{\boldsymbol{\alpha}, B, \varepsilon}(z)\stackrel{\upshape\tiny def.}{=}\Phi_{Q_n+1, \delta_{\times}}^{\times, R_{\times},1}(P^{(1)}_{B_{+},1}(\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})), \theta_B(z_1),\ldots,\theta_B(z_{Q_n}))\). If we choose \(\delta,\delta_{\times}>0\) with \(\delta+\delta_{\times}\le\varepsilon\), we can obtain the local-uniform estimate in the following way. Let \(\boldsymbol{z} \in [-B,B]^{Q_n}\), then \(\prod_{i=1}^{Q_n}\theta_B(z_i)=1\) and \(P^{(1)}_{B_{+},1}(\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z}))=\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})\); whence, \[\left| h_{\boldsymbol{\alpha}}(z) - \Phi_{\boldsymbol{\alpha},B,\varepsilon}(\boldsymbol{z}) \right| \le \left| h_{\boldsymbol{\alpha}}(\boldsymbol{z}) - \Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z}) \right| + \delta_{\times} \le \delta+\delta_{\times} \le \varepsilon.\] It remains to verify the global growth bound. If \(\boldsymbol{z}\in[-B_+,B_+]^{J_n\times D}\), then \(P^{(1)}_{B_{+},1} (\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z}))=\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})\) and \(0 \leq \prod_{i=1}^{Q_n} \theta_{B}(x_i) \leq 1\), so \[|\Phi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})| \leq |\Psi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})|+\delta_{\times} \leq |h_{\boldsymbol{\alpha}}(\boldsymbol{z})| + \delta+\delta_{\times} \leq |h_{\boldsymbol{\alpha}}(\boldsymbol{z})| + 1.\] Instead, if \(\boldsymbol{z} \notin [-B, B]^{Q_n}\), then \(\theta_{Q_n}(x_i)=0\) for at least one coordinate \(i\), and hence \[|\Phi_{\boldsymbol{\alpha},B_+,\delta}(\boldsymbol{z})| \leq \delta_{\times} \leq 1 \leq |h_{\boldsymbol{\alpha}}(\boldsymbol{z})| + 1.\] This proves the global growth bound for every \(\boldsymbol{z} \in \mathbb{R}^{J_n \times D}\).
Finally, we need to record the complexity. To this end, we remind and write that \(h_m(z)=\sum_{\ell=0}^{m} c_{m,\ell} x^{\ell}\), \(A_{m,B}=\sum_{\ell=0}^{m}|c_{m,\ell}|B^{\ell}\), and \(A_{\boldsymbol{\alpha},B} = 1 + \max_{j\in [J_n]_{+}, r \in [S]_{+}} A_{\alpha_{j,r},B}\). Moreover, define \(\ell_{Q_n} \stackrel{\upshape\tiny def.}{=}\left\lceil \log_2(Q_n\vee2)\right\rceil\), \[L_{\boldsymbol{\alpha},B,\varepsilon_{\mathrm{loc}}} \stackrel{\upshape\tiny def.}{=} 1+ \max_{\substack{j \in [J_n]_{+}, r \in [D]_{+}}} \log_2\left( 1+ \frac{ 4 Q_n\alpha_{j,r} A_{\alpha_{j,r},B_+} A_{\boldsymbol{\alpha},B_+}^{Q_n-1} }{ \varepsilon } \right) + L_{Q_n} \left( 1+ \log_2\left( \frac{ 2^{Q_n+2}Q_n A_{\boldsymbol{\alpha},B_+}^{Q_n} }{ \varepsilon } \right) \right)\] and \[M_{\boldsymbol{\alpha},B,\varepsilon} \stackrel{\upshape\tiny def.}{=} 1+ \log_2\left( \frac{ 4(Q_n+1) (A_{\boldsymbol{\alpha},B_+}+3)^{Q_n+1} }{ \varepsilon_{\mathrm{loc}} } \right)\] The local Hermite approximant \(\Phi_{\boldsymbol{\alpha},B_+,\delta}(\cdot)\) has the complexity given by Lemma 9 evaluated at radius \(B_+\) and accuracy \(\delta\). The coordinate masks \(\theta_B(z_i)\) add \(O(Q_n)\) neurons and non-zero parameters. The scalar localization map \(P_{B_+,1}^{(1)}\) has \(O(1)\) complexity. Therefore, the final product network has \(Q_n+1\) inputs and contributes depth of order \(L_{Q_n+1}M_{\boldsymbol{\alpha},B,\varepsilon}\), width of order \(Q_n+1\), and size of order \((Q_n+1)M_{\boldsymbol{\alpha},B,\varepsilon}\). Combining these costs and increasing the universal constant \(C\) gives the stated complexity in the statement of the lemma. This completes the proof. ◻

Proof of Proposition 9. First, for \(j \in [J_n]_{+}\) and \(r \in [D]_{+}\), we observe \(Z_{j,r} = Z_{i_j,k_j}^{r}=\int_0^T \psi_{i_j,k_j}(s)\,\mathrm{d}W_s^{r}\). In particular, \[\mathbb{E}[Z_{j,r}Z_{j',r'}]=\mathbf{1}_{\{r=r'\}}\int_0^T \psi_{i_j,k_j}(s)\psi_{i_{j'},k_{j'}}(s)\,\mathrm{d}s = \mathbf{1}_{\{r=r'\}}\mathbf{1}_{\{j=j'\}};\] whence \(\boldsymbol{Z}=(Z_{j,r})_{j \in [J_n]_{+},r \in [D]_{+}}\) is a standard Gaussian vector in \(\mathbb{R}^{J_n \times D}\). In particular, \(h_{\boldsymbol{\alpha}}(\boldsymbol{Z})\in L^2(\Omega)\) because \(h_{\boldsymbol{\alpha}}\) is a polynomial. Now, as in the proof of Lemma 9, we can consider the case \(n \geq 1\). Fix \(\varepsilon>0\). Since \(1+|h_{\boldsymbol{\alpha}}(\boldsymbol{Z})|\in L^2(\Omega)\), there exists \(B_{\boldsymbol{\alpha}, \varepsilon}>0\) such that \[\left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty>B_{\boldsymbol{\alpha},\varepsilon}\}} h_{\boldsymbol{\alpha}}(\boldsymbol{Z})\right\|_{L^2(\Omega)}\le \frac{\varepsilon}{4},\quad\text{and}\quad \left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty>B_{\boldsymbol{\alpha},\varepsilon}\}} \left(1+|h_{\boldsymbol{\alpha}}(\boldsymbol{Z})|\right) \right\|_{L^2(\Omega)} \le \frac{\varepsilon}{4}\] Now, we set \(\varepsilon_{\mathrm{loc}} \stackrel{\upshape\tiny def.}{=}\min\{\frac{1}{2},\frac{\varepsilon}{2}\}\). By Lemma 10 applied with \(B=B_{\boldsymbol{\alpha},\varepsilon}\) there exists a \(\operatorname{ReLU-MLP}\) \(\Phi_{\boldsymbol{\alpha},\varepsilon}:\mathbb{R}^{J_n\times D}\to\mathbb{R}\) such that \[\begin{align} & \sup_{\boldsymbol{z}\in[-B_{\boldsymbol{\alpha},\rho}, B_{\boldsymbol{\alpha},\varepsilon}]^{J_n\times D}} \left| h_{\boldsymbol{\alpha}}(\boldsymbol{z}) - \Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{z}) \right| \le \varepsilon_{\mathrm{loc}}, \quad \text{and} \\ & |\Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{z})| \le 1+|h_{\boldsymbol{\alpha}}(\boldsymbol{z})|, \quad \text{ for all } \boldsymbol{z}\in\mathbb{R}^{J_n\times D} \end{align}\] By setting \(B_\varepsilon \stackrel{\upshape\tiny def.}{=}B_{\boldsymbol{\alpha},\varepsilon}\), we use that \(|\Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{z})| \le 1+|h_{\boldsymbol{\alpha}}(\boldsymbol{Z})|\) to obtain \[\begin{align} \left\| h_{\boldsymbol{\alpha}}(\boldsymbol{Z}) - \Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z}) \right\|_{L^2(\Omega)} &\le \left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty\le B_\varepsilon\}} \left( h_{\boldsymbol{\alpha}}(\boldsymbol{Z}) - \Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z}) \right) \right\|_{L^2(\Omega)} \\ &\quad+ \left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty>B_\varepsilon\}} h_{\boldsymbol{\alpha}}(\boldsymbol{Z}) \right\|_{L^2(\Omega)} \\ &\quad+ \left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty>B_\varepsilon\}} \Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z}) \right\|_{L^2(\Omega)} \\ & \leq \varepsilon_{\mathrm{loc}} + \frac{\varepsilon}{4} + \left\| \mathbf{1}_{\{\|\boldsymbol{Z}\|_\infty>B_\varepsilon\}} (1+|h_{\boldsymbol{\alpha}}(\boldsymbol{Z})|) \right\|_{L^2(\Omega)} \\ & \leq \frac{\varepsilon}{2} + \frac{\varepsilon}{4} + \frac{\varepsilon}{4} = \varepsilon. \end{align}\] Finally, since every coordinate of \(\boldsymbol{Z}\) is \(\mathcal{F}_i\)-measurable and \(\Phi_{\boldsymbol{\alpha},\varepsilon}(\cdot)\) is deterministic and Borel measurable, we have \(\Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z})\in \mathcal{F}_i\). Moreover, \(|\Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z})| \le 1+|h_{\boldsymbol{\alpha}}(\boldsymbol{Z})|\in L^2(\Omega)\) and so \(\Phi_{\boldsymbol{\alpha},\varepsilon}(\boldsymbol{Z})\in L^2(\mathcal{F}_t)\) if \(\boldsymbol{Z}\) is \(\mathcal{F}_t\)-measurable . The complexity is the one supplied by Lemma 10, evaluated at \(B=B_{\boldsymbol{\alpha},\varepsilon}\) and \(\varepsilon_{\mathrm{loc}}= \min\left\{\frac{1}{2},\frac{\varepsilon}{2}\right\}\). This concludes the proof. ◻

Proof. For the sake of notation, we write \(a \stackrel{\upshape\tiny def.}{=}a_{p,q}\), \(c\stackrel{\upshape\tiny def.}{=}c_{p,q}\), \(b \stackrel{\upshape\tiny def.}{=}b_{p,q}\), \(\delta \stackrel{\upshape\tiny def.}{=}\delta_{\mathrm t}\), \(A\stackrel{\upshape\tiny def.}{=}A_p\). In particular, \(c-a\) coincides with \(b-c\) and it is equal to \(\frac{T}{2^{p+1}}\). Since \(\eta_{\mathrm t}<2^{-(p+2)}\), we have that \(2 \delta < c-a=b-c\). Therefore, the four intervals \([a,a+\delta]\), \([c-\delta,c]\), \([c,c+\delta]\), \([b-\delta,b]\) are pairwise disjoint (up to endpoints). By construction \(\psi^{(\eta_{\mathrm t})}_{p,q} = \psi_{p,q}\) on the complement of the union of the previous intervals. Moreover, on each of the four transition intervals, the error is affine, with maximum absolute value \(A\). More precisely, \[|\psi_{p,q}(t)-\psi^{(\eta_{\mathrm t})}_{p,q}(t)|= \begin{cases} A\frac{a+\delta-t}{\delta}, \qquad \,\,\, t\in[a,a+\delta]\\ A\frac{t-(c-\delta)}{\delta}, \qquad t\in[c-\delta,c]\\ A\frac{c+\delta-t}{\delta}, \qquad \,\,\, t\in[c,c+\delta]\\ A\frac{t-(b-\delta)}{\delta}, \qquad t\in[b-\delta,b] \end{cases}\] Thus, each transition interval contributes \(\int_0^\delta A^2\left(\frac{s}{\delta}\right)^2\,\mathrm{d}s = \frac{A^2\delta}{3}\) to the squared \(L^2([0,T])\)-error. Hence \(\left\| \psi_{p,q} - \psi^{(\eta_{\mathrm t})}_{p,q} \right\|_{L^2([0,T])}^2 = 4\frac{A^2\delta}{3}\). Since \(A^2=\frac{2^p}{T}\), \(\delta=T\eta_{\mathrm t}\), we obtain \[\left\| \psi_{p,q} - \psi^{(\eta_{\mathrm t})}_{p,q} \right\|_{L^2([0,T])}^2 = \frac{4}{3}2^p\eta_{\mathrm t} = \frac{2^{p+2}}{3}\eta_{\mathrm t}.\] Taking square roots gives the desired result. ◻

Proof. Define \(\Phi_{p,q;\eta_{\mathrm t}} \stackrel{\upshape\tiny def.}{=}\psi^{(\eta_{\mathrm t})}_{p,q}\). The first claim follows from Lemma 11 and the representation in Eq. 24 ; whence, it remains only to prove the support identity. If \(t<a_{p,q}\), then all arguments of the clipped ramp in the definition of \(\psi^{(\eta_{\mathrm t})}_{p,q}\) are negative; whence \(\Phi_{p,q;\eta_{\mathrm t}}(t)=0\). If \(t\ge b_{p,q}\), then all three ramp terms are equal to \(1\), and therefore \(\Phi_{p,q;\eta_{\mathrm t}}(t) = A_p(-1+2-1) = 0\). Thus, \(\operatorname{supp}(\Phi_{p,q;\eta_{\mathrm t}}) \subseteq [a_{p,q},b_{p,q}]\). On the other hand, \(\Phi_{p,q;\eta_{\mathrm t}}\) is non-zero on subintervals of \((a_{p,q},b_{p,q})\) arbitrarily close to every point of the support. Hence its closed support is exactly \([a_{p,q},b_{p,q}]\). The same is true for the Haar atom \(\psi_{p,q}\). This concludes the proof. ◻

Proof of Proposition 10. By Lemma 12, there is a ReLU MLP \(\Phi_{p,q;\eta_{\mathrm t}}:\mathbb{R}\to\mathbb{R}\) of depth \(1\), width \(6\), and with \(18\) non-zero parameters such that \(\operatorname{supp}(\Phi_{p,q;\eta_{\mathrm t}})=\operatorname{supp}(\psi_{p,q})= [a_{p,q},b_{p,q}]\) and \(\left\|\psi_{p,q} \!-\! \Phi_{p,q;\eta_{\mathrm t}}\right\|_{L^2([0,T])}=\frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}}\). This proves the existence of the deterministic \(\operatorname{ReLU}\) time network. Now, we first prove the predictability and support statement. Since \(t\mapsto \Phi_{p,q;\eta_{\mathrm t}}(t)\) is deterministic and continuous, it is Borel measurable. Therefore, if \(u\in\mathbb{S}_T^2(\mathbb{R}^d)\) is \(\mathcal{F}_{\cdot}\)-predictable, then \(u^{p,q;\eta_{\mathrm t}}_t = u_t\Phi_{p,q;\eta_{\mathrm t}}(t)\) is predictable as the product of a predictable process and a deterministic Borel function. Moreover, because \(\Phi_{p,q;\eta_{\mathrm t}}(t)=0\) for \(t\notin[a_{p,q},b_{p,q}]\), \(u^{p,q;\eta_{\mathrm t}}_t=0\) for every such \(t\). Therefore, \(\operatorname{supp}\left(t\mapsto u^{p,q;\eta_{\mathrm t}}_t\right) \subseteq [a_{p,q},b_{p,q}]\) almost surely. We also have \(u^{p,q;\eta_{\mathrm t}}\in \mathcal{H}_T^2\). Indeed, \(\Phi_{p,q;\eta_{\mathrm t}}\) is bounded and supported in \([a_{p,q},b_{p,q}]\), and therefore \[\mathbb{E}\left[\int_0^T |u_t\Phi_{p,q;\eta_{\mathrm t}}(t)|^2\,\mathrm{d}t\right] \le \|\Phi_{p,q;\eta_{\mathrm t}}\|_{L^\infty([0,T])}^2 T \mathbb{E}\left[ \sup_{0\le s\le T}|u_s|^2 \right] <\infty\] We next prove the approximation estimate. By the definition of the \(\mathcal{H}_T^2\)-norm, \[\left\| u^{p,q;\eta_{\mathrm t}} - u^{p,q} \right\|_{\mathcal{H}_T^2}^2 = \mathbb{E}\left[\int_0^T |u_t|_{\mathbb{R}^d}^2 \left| \Phi_{p,q;\eta_{\mathrm t}}(t)-\psi_{p,q}(t) \right|^2\right]\,\mathrm{d}t .\] Since \(u\in\mathbb{S}_T^2\), we obtain \[\left\| u^{p,q;\eta_{\mathrm t}} - u^{p,q} \right\|_{\mathcal{H}_T^2}^2 \leq \|u\|_{\mathbb{S}_T^2}^2 \left\| \Phi_{p,q;\eta_{\mathrm t}} - \psi_{p,q} \right\|_{L^2([0,T])}^2 .\] By applying Lemma 12 and taking square roots gives that the latter quantity is less than or equal to \(\|u\|_{\mathbb{S}_T^2} \frac{2^{1+p/2}}{\sqrt{3}} \sqrt{\eta_t}\). Now, it remains to prove the final statement for a random coefficient. Let \(U\in L^2(\mathcal{F}_{a_{p,q}})\). The process \(t\mapsto U\Phi_{p,q;\eta_{\mathrm t}}(t)\) is adapted because \(\Phi_{p,q;\eta_{\mathrm t}}\) vanishes before \(a_{p,q}\), while for \(t\ge a_{p,q}\) the random variable \(U\) is \(\mathcal{F}_t\)-measurable. Since \(\Phi_{p,q;\eta_{\mathrm t}}\) is continuous, this process is predictable. Similarly, \(t\mapsto U\psi_{p,q}(t)\) is predictable. Finally, by Fubini’s theorem, \[\begin{align} & \left\| U\Phi_{p,q;\eta_{\mathrm t}} - U\psi_{p,q} \right\|_{\mathcal{H}_T^2}^2= \mathbb{E}\left[\int_0^T |U|^2 \left| \Phi_{p,q;\eta_{\mathrm t}}(t)-\psi_{p,q}(t) \right|^2\,\mathrm{d}t\right]=\mathbb{E}\left[|U|^2\right] \left\| \Phi_{p,q;\eta_{\mathrm t}} - \psi_{p,q} \right\|_{L^2([0,T])}^2 \end{align}\] Taking square roots and applying Lemma 12 completes the proof. ◻

Proof of Theorem 4. We first prove the density statement in the scalar case. The vector-valued case will then follow component-wise by stacking the corresponding scalar \(\operatorname{NeuralChaos}\) blocks. Let \(\mathfrak{C}^{\rm all}\) be the family in Eq. 8 , which is an orthonormal basis of \(\mathcal{H}_T^2\). Whence, every finite linear combinations of chaoslets are dense in \(\mathcal{H}_T^2\). In particular, for a fixed \(\varepsilon>0\), we may choose a finite set \(\overline{\mathcal{F}} \subset \mathfrak{C}^{\rm all}\) and coefficients \((\beta_m)_{m \in \overline{\mathcal{F}}}\) such that the quantity \(\overline{Y}_{\cdot} \stackrel{\upshape\tiny def.}{=}\sum_{m \in \overline{\mathcal{F}}} \beta_m m\) satisfies \(\|X_{\cdot}-\overline{Y}_{\cdot}\|_{\mathcal{H}^2_T} \leq \frac{\varepsilon}{2}\). Whence, we need to prove that we can approximate \(\overline{Y}_{\cdot}\) by a \(\operatorname{NeuralChaos}\) model (cf. Section 3). To this end, we consider a non-constant scalar chaoslet \(m=\mathfrak{C}^{\boldsymbol{\alpha_{(p,q)}}}\). By definition (cf. Eq. 7 ), \(\mathfrak{C}_{(p,q)}^{\boldsymbol{\alpha}} (t,\omega)=\psi_{p,q}(t)u^{(T)}_{\boldsymbol{\alpha};p,q}(\omega)\), where \(u^{(T)}_{\boldsymbol{\alpha};p,q}(\omega)= \prod_{(i,k)\in \mathcal{I}_{p,q}} \, \prod_{r=1}^{D} \, h_{\alpha_r^{i,k}}\big( Z_{i,k}^r(\omega) \big)\). Now, set, again, \(a_{p,q}\stackrel{\upshape\tiny def.}{=}\frac{qT}{2^p}\). By the definition of \(\mathcal{I}_{p,q}\), every Brownian Haar coordinate entering the quantity \(u^{(T)}_{\boldsymbol{\alpha};p,q}(\omega)\) is measurable no later than \(a_{p,q}\); whence \(u^{(T)}_{\boldsymbol{\alpha};p,q}\in L^2(\mathcal{F}_{a_{p,q}})\). Besides, \(\|u^{(T)}_{\boldsymbol{\alpha};p,q}\|_{L^2(\Omega)}=1\). Now, we observe that since \(\boldsymbol{\alpha}\) is finitely supported, only finitely many Brownian Haar coordinates appear in \(u^{(T)}_{\boldsymbol{\alpha};p,q}\); we let \(\Lambda_{\boldsymbol{\alpha};p,q} \stackrel{\upshape\tiny def.}{=} \left\{ (i,k)\in \mathcal{I}_{p,q}: \exists r\in[D]_{+}\;\text{such that}\;\alpha^r_{i,k}>0 \right\}\) and define \(\boldsymbol{Z}_{\boldsymbol{\alpha};p,q} \stackrel{\upshape\tiny def.}{=}\left( Z^r_{i_j,k_j}\right)_{j \in J_{\boldsymbol{\alpha}}, r \in D_{+}}\), \(\widetilde{\alpha}_{j,r} \stackrel{\upshape\tiny def.}{=}\alpha^r_{i_j,k_j}\), so \(u^{(T)}_{\boldsymbol{\alpha};p,q}=h_{\widetilde{\boldsymbol{\alpha}}}(\boldsymbol{Z}_{\boldsymbol{\alpha};p,q})\), where \(h_{\widetilde{\boldsymbol{\alpha}}}(\boldsymbol{z})\stackrel{\upshape\tiny def.}{=}\prod_{r=1}^{D} \prod_{j=1}^{J_{\boldsymbol{\alpha}}} h_{\widetilde{\boldsymbol{\alpha}}}(z_{j,r})\). Applying Proposition 9, for every \(\rho>0\), there exists a \(\operatorname{ReLU-MLP}\) \(\Phi_{\boldsymbol{\alpha},\rho}:\mathbb{R}^{J_{\boldsymbol{\alpha}} \times D} \rightarrow \mathbb{R}\) such that \[\mathbb{E}\left[|u^{(T)}_{\boldsymbol{\alpha};p,q}-\Phi_{\boldsymbol{\alpha},\rho}(\boldsymbol{Z}_{\boldsymbol{\alpha};p,q})|^2\right]^{\frac{1}{2}} \leq \rho.\] Next, by Lemma 12, for every sufficiently small \(\eta_{\mathrm t}>0\), there exists a \(\operatorname{ReLU-MLP}\) \(\Phi_{p,q;\eta_{\mathrm t}}:\mathbb{R} \rightarrow \mathbb{R}\) of depth \(1\), width \(6\), and at most \(18\) non-zero parameters such that \[\|\psi_{p,q}-\Phi_{p,q;\eta_{\mathrm t}}\|_{L^2([0,T])} = \frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}}\] and \(\operatorname{supp}(\Phi_{p,q;\eta_{\mathrm t}}) = \operatorname{supp}(\psi_{p,q}) = [a_{p,q},b_{p,q}]\). We thus define the single-chaoslet neural approximation as \(\widehat{\mathfrak{C}}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)}(t,\omega) \stackrel{\upshape\tiny def.}{=}\Phi_{p,q;\eta_{\mathrm t}}(t)\,\Phi_{\boldsymbol{\alpha},\rho}(\boldsymbol{Z}_{\boldsymbol{\alpha};p,q})\), which is predictable and square-integrable. Moreover, \[\left\| \mathfrak C^{\boldsymbol{\alpha}}_{(p,q)} - \widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)} \right\|_{\mathcal{H}_T^2} \le \left\| (\psi_{p,q}-\Phi_{p,q;\eta_{\mathrm t}}) u^{(T)}_{\boldsymbol{\alpha};p,q} \right\|_{\mathcal{H}_T^2} + \left\| \Phi_{p,q;\eta_{\mathrm t}} \left( u^{(T)}_{\boldsymbol{\alpha};p,q} -\Phi_{\boldsymbol{\alpha},\rho}(\boldsymbol{Z}_{\boldsymbol{\alpha};p,q}) \right)\right\|_{\mathcal{H}_T^2}\] The first term on the right-hand side of the inequality if finite and, since \(\|u^{(T)}_{\boldsymbol{\alpha};p,q}\|_{L^2(\Omega)}=1\), it is equal to \(\frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}}\) (see above). The second term, instead is easily bounded by \(\rho \|\Phi_{p,q;\eta_{\mathrm t}}\|_{L^2([0,T])}\), which can be bounded by \(1+ \|\Phi_{p,q;\eta_{\mathrm t}}-\psi_{p,q}\|_{L^2([0,T])} = 1+ \frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}}\). In particular, \[\left\| \mathfrak C^{\boldsymbol{\alpha}}_{(p,q)} - \widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)} \right\|_{\mathcal{H}_T^2} \le \frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}} + \rho \left( 1+ \frac{2^{1+p/2}}{\sqrt3}\sqrt{\eta_{\mathrm t}} \right)\] As a consequence, given \(\delta>0\), we can opportunely choose \(\eta_{\mathrm t}\) and then \(\rho>0\) such that the previous quantity is bounded by \(\delta\). At this point, we need to prove that \(\widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)}\) is produced by the \(\operatorname{NeuralChaos}\) architecture in Section 3. We therefore choose a deterministic grid \(=(t_i)_{i=1}^{M}\), \(0=t_0<t_1<\cdots<t_M=T\) \(a_{p,q}\) and all Brownian sampling times needed to compute are at-most evaluated only at these grid times only. Step 1 in Subsection 8.1 gives a lower-triangular linear lift that computes these Brownian Haar coordinates from the sampled Brownian values. The row-wise head at information time \(a_{p,q}\) is chosen to be the \(\operatorname{ReLU-MLP}\) \(\Phi_{\boldsymbol{\alpha}; p, q, \rho}\). Instead, the deterministic time network is chosen to be \(\Phi_{p,q;\eta_{\mathrm t}}\). Therefore, one row/head/mask block of \(\operatorname{NeuralChaos}\) realizes \((t,\omega)\to \Phi_{p,q;\eta_{\mathrm t}}\Phi_{\boldsymbol{\alpha}; p, q, \rho}\). Notice that if the literal Section 3 mask convention includes the additional threshold factor \(\sigma\left(\frac{t-a_{p,q}}{\lambda}\right)\), we replace the time factor by \(\Phi_{p,q;\eta_{\mathrm t}}(t) \sigma\left(\frac{t-a_{p,q}}{\lambda}\right)\). Since \(\Phi_{p,q;\eta_{\mathrm t}}\) is bounded and the modification is supported on a deterministic interval of length \(O(\lambda)\), the additional \(\mathcal{H}_T^2\)-error can be made arbitrarily small by taking \(\lambda>0\) sufficiently small; this error is absorbed into the one-chaoslet tolerance. We now approximate the finite scalar chaoslet sum \(\overline{Y}=\sum_{m\in \overline{F}}\beta_m m\). We set \(B_{\overline{F}} \stackrel{\upshape\tiny def.}{=}\sum_{m\in F}|\beta_m\), and for every non-constant \(m\in \overline{F}\), construct a \(\operatorname{NeuralChaos}\) block \(\widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)}\) as above such that¹⁰ \(\left\| \mathfrak C^{\boldsymbol{\alpha}}_{(p,q)} - \widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)} \right\|_{\mathcal{H}_T^2} \leq \frac{\varepsilon}{2 (1+B_{\overline{F}})}\). Now, we define \(\widehat{\overline{Y}} \stackrel{\upshape\tiny def.}{=}\sum_{\overline{\mathcal{F}}} \beta_m \widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)}\). The \(\operatorname{NeuralChaos}\) class is closed under finite parallelization: we use one block for each \(\widehat{\mathfrak C}^{\boldsymbol{\alpha},\rho,\eta_{\mathrm t}}_{(p,q)}\) and sum the resulting scalar outputs in the final affine layer. Whence \(\widehat{\overline{Y}} \in \mathfrak{N}\mathfrak{C}_{T,D,1}\), and together with \(\|X_{\cdot}-Y_{\cdot}\|_{\mathcal{H}^2_T}\le \frac{\varepsilon}{2}\), we obtain the density in the scalar case.

We now prove the quantitative statement. Assume that \(X_{\cdot}\in\mathcal{H}_T^2(\mathbb{R}^d)\) is \(S\)-compressible, with \(S>1/2\), and belongs to \(\mathbb{D}_T^{s,2:d}\), with \(s>0\). By Proposition 1, for every \(N\in\mathbb{N}_{+}\) and every \(P\in\mathbb{N}\), there exist distinct chaoslets \(\mathfrak m^{(1)},\ldots,\mathfrak m^{(N)}\in\cup_{k=0}^{P}\mathfrak C_k\) and coefficients \(a_1,\ldots,a_N\in\mathbb{R}^d\) such that \[\left\| X_{\cdot} - \sum_{\nu=1}^{N}a_\nu \mathfrak m^{(\nu)} \right\|_{\mathcal{H}_T^2(\mathbb{R}^d)} \lesssim_X N^{-\left(S-\frac{1}{2}\right)} + (1+P)^{-s/2}.\] Set \(Y_{N,P} \stackrel{\upshape\tiny def.}{=} \sum_{\nu=1}^{N}a_\nu \mathfrak m^{(\nu)}\). Let \(C_X^{\rm app}>0\) be an \(X\)-dependent constant for the preceding estimate. Since \(N\ge1\) and \((1+P)^{-s/2}\le1\), there exists a constant \(A_X>0\), depending only on \(X\), such that \(\|Y_{N,P}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} \le A_X\) for all \(N\) and \(P\). For example, one may take \(A_X \stackrel{\upshape\tiny def.}{=} 1+\|X_{\cdot}\|_{\mathcal{H}_T^2(\mathbb{R}^d)}+2C_X^{\rm app}\). Since the chaoslets are orthonormal and the retained chaoslets are distinct, \[\sum_{\nu=1}^{N}|a_\nu|_{\mathbb{R}^d} \le \sqrt{N} \left( \sum_{\nu=1}^{N}|a_\nu|_{\mathbb{R}^d}^2 \right)^{1/2} = \sqrt{N}\, \|Y_{N,P}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} \le \sqrt{N}A_X.\] Now, we define, \(\rho_{N,X,\varepsilon}\stackrel{\upshape\tiny def.}{=} \frac{\varepsilon}{8(1+\sqrt{N}A_X)}.\) For each retained chaoslet \(\mathfrak m^{(\nu)}\), construct a scalar \(\operatorname{NeuralChaos}\) approximant \(\widehat{\mathfrak m}^{(\nu)}\) as above with one-chaoslet error at most \(\rho_{N,X,\varepsilon}\). This is achieved by using stochastic-head accuracy \(\rho_{N,X,\varepsilon}\) and by choosing the corresponding time-mask parameter \(\eta_{\mathrm t}^{(\nu)}\) sufficiently small. We define \(\widehat X_{N,P,\varepsilon} \stackrel{\upshape\tiny def.}{=} \sum_{\nu=1}^{N}a_\nu \widehat{\mathfrak m}^{(\nu)}\). By finite parallelization and final affine summation, \(\widehat X_{N,P,\varepsilon}\in\mathfrak N\mathfrak C_{T,D,d}\). Moreover, \[\begin{align} \|Y_{N,P}-\widehat X_{N,P,\varepsilon}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} &\le \sum_{\nu=1}^{N} |a_\nu|_{\mathbb{R}^d} \|\mathfrak m^{(\nu)}-\widehat{\mathfrak m}^{(\nu)}\|_{\mathcal{H}_T^2} \\ &\le \sqrt{N}A_X\,\rho_{N,X,\varepsilon} \le \varepsilon. \end{align}\] Therefore, \[\begin{align} \|X_{\cdot}-\widehat X_{N,P,\varepsilon}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} &\le \|X_{\cdot}-Y_{N,P}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} + \|Y_{N,P}-\widehat X_{N,P,\varepsilon}\|_{\mathcal{H}_T^2(\mathbb{R}^d)} \\ &\lesssim_X N^{-\left(S-\frac{1}{2}\right)} + (1+P)^{-s/2} + \varepsilon. \end{align}\] This proves the quantitative approximation estimate.

It remains to record the complexity. Since each retained chaoslet belongs to \(\cup_{k=0}^{P}\mathfrak C_k\), its Wiener-chaos degree is at most \(P\). Hence its stochastic Hermite-Haar factor contains at most \(P\) active scalar Brownian Haar coordinates. Each such coordinate is a linear combination of three Brownian samples. Thus the causal lifting stage contributes at most \(3P\) non-zero linear coefficients per retained chaoslet, up to a universal constant. The deterministic time mask has depth \(1\), width \(6\), and at most \(18\) non-zero parameters. By the definition of \(D_{\mathrm H}(P,\rho)\), \(W_{\mathrm H}(P,\rho)\), and \(S_{\mathrm H}(P,\rho)\), the stochastic head for each retained chaoslet has depth, width, and number of non-zero parameters bounded by \(D_{\mathrm H}(P,\rho_{N,X,\varepsilon})\), \(W_{\mathrm H}(P,\rho_{N,X,\varepsilon})\), \(S_{\mathrm H}(P,\rho_{N,X,\varepsilon})\), respectively.

After parallelizing the \(N\) retained chaoslet blocks, the depth is the maximum of the block depths, while widths and numbers of non-zero parameters add. The final affine layer implements the vector coefficients \(a_\nu\in\mathbb{R}^d\), \(\nu=1,\ldots,N\), and contributes at most \(Nd\) additional non-zero parameters. Therefore, for a universal constant \(C>0\), \[\operatorname{depth} \left( \widehat X_{N,P,\varepsilon} \right) \le C \left[ 1+ D_{\mathrm H} \left( P,\rho_{N,X,\varepsilon} \right) \right],\] \[\operatorname{width} \left( \widehat X_{N,P,\varepsilon} \right) \le C \left[ d+ N \left( 1+ W_{\mathrm H} \left( P,\rho_{N,X,\varepsilon} \right) \right) \right],\] and \[\operatorname{size} \left( \widehat X_{N,P,\varepsilon} \right) \le C\,N \left[ P+d+1+ S_{\mathrm H} \left( P,\rho_{N,X,\varepsilon} \right) \right].\] Finally, the finite sampling grid is obtained by collecting all Brownian sampling times required by the retained chaoslets, together with the corresponding information times \(a_{p,q}\). Since each retained chaoslet uses at most \(3P\) Brownian sample times and one information time, the grid can be chosen with \(M_{\mathrm{grid}} \le C\,N(P+1)\). This completes the proof of Theorem 4. ◻

Proof of Proposition 6. Fix \(\delta\in (0,T)\), set \(N_\delta\stackrel{\upshape\tiny def.}{=}\big\lceil \tfrac{T}{\delta}\big\rceil\), and set \(t_i\stackrel{\upshape\tiny def.}{=}(i\delta)\wedge T\) for each \(i=0,\dots,N_{\delta}\). Define the “Euler-Maruyama update-rule” as the map \[f_\delta:[0,\infty)\times \mathbb{R}^{d_X+2D}\to \mathbb{R}^{d_X}, \qquad f_\delta(t,x,w,u) \stackrel{\upshape\tiny def.}{=} x+b(t,x)\bigl(((t+\delta)\wedge T)-t\bigr)+\Sigma(t,x)(u-w) .\] Since \(b\) and \(\Sigma\) are Lipschitz, they are Borel-measurable; thus, so is \(f_\delta\). Now define recursively \(X^{(0)}\stackrel{\upshape\tiny def.}{=}\xi\) and for each \(i=0,\dots,N_{\delta}-1\) define \(X^{(i+1)} \stackrel{\upshape\tiny def.}{=} f_\delta(t_i,X^{(i)},W_{t_i},W_{t_{i+1}})\). A simple induction using Grönwall’s inequality shows that, for each \(i=0,\dots,N_{\delta}\), we have \(X^{(i)}\in L^2(\mathcal{F}_{t_i};\mathbb{R}^{d_X})\). Now, define the piecewise-constant process by \[\hat{X}_t^{\delta} \stackrel{\upshape\tiny def.}{=} \sum_{i=0}^{N_\delta-1} \mathbf{1}_{[t_i,t_{i+1})}(t)\, \mathbb{E}\big[X^{(i)}\mid \mathcal{F}_{t_i}\big]\] where \(0\le t\le T\). Since \(X^{(i)}\) is \(\mathcal{F}_{t_i}\)-measurable, one has \(\mathbb{E}[X^{(i)}\mid \mathcal{F}_{t_i}] = X^{(i)}\) \(\mathbb{P}\)-a.s.for each \(i=0,\dots,N_{\delta}-1\); therefore \(\hat{X}_t^{\delta} = \sum_{i=0}^{N_\delta-1} \mathbf{1}_{[t_i,t_{i+1})}(t)\, X^{(i)}\) and so \(\hat{X}^{\delta}\in \mathfrak{B}\).

It remains to show that the family of processes \((\hat{X}^{\delta}_\cdot)_\delta\) converges to \(X_{\cdot}\) in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\). Consider the classical continuous-time Euler-Maruyama approximations \((\bar{X}^{\delta}_{\cdot})_{\delta>0}\) to \(X_{\cdot}\); namely, \[\bar{X}_t^{\delta} \stackrel{\upshape\tiny def.}{=} X^{(i)} + b(t_i,X^{(i)})(t-t_i) + \Sigma(t_i,X^{(i)})(W_t-W_{t_i})\] for \(t\in [t_i,t_{i+1})\) and each \(i=0,\dots,N_{\delta}-1\). The main result of [@maruyama1955continuous] implies \(\mathcal{S}_T^2(\mathbb{R}^{d_X})\)-norm convergence, i.e., \[\lim_{\delta\downarrow 0} \, \mathbb{E}\Big[\sup_{0\le t\le T}\|X_t-\bar{X}_t^{\delta}\|^2\Big]=0.\] Hence, by the usual inequalities between the \(\mathcal{H}_T^2\) and \(\mathcal{S}_T^2\) norms, we have \[\label{eq:convergence95EM95approx} 0 \le \lim_{\delta\downarrow 0} \, \mathbb{E}\biggl[ \int_0^T\, \|X_t-\bar{X}_t^{\delta}\|^2\,dt \biggr] \le \lim_{\delta\downarrow 0} \, T \mathbb{E}\biggl[ \sup_{0\le t\le T}\|X_t-\bar{X}_t^{\delta}\|^2 \biggr] = 0 .\tag{25}\] We only need to show that \(\hat{X}^{\delta}_{\cdot}\) converges to \(\bar{X}_{\cdot}^{\delta}\) to obtain the conclusion. For any \(i=0,\dots,N_{\delta}-1\) and any \(t\in [t_i,t_{i+1})\) we have \[\bar{X}_t^{\delta}-\hat{X}_t^{\delta} = b(t_i,X^{(i)})(t-t_i) + \Sigma(t_i,X^{(i)})(W_t-W_{t_i}).\] Therefore, using the basic inequality \((a+b)^2\le 2a^2+2b^2\), \[\begin{align} \mathbb{E}\int_{t_i}^{t_{i+1}} \|\bar{X}_t^{\delta}-\hat{X}_t^{\delta}\|^2\,dt \le 2\, \mathbb{E}\int_{t_i}^{t_{i+1}} \|b(t_i,X^{(i)})\|^2 (t-t_i)^2\,dt + 2 \mathbb{E}\int_{t_i}^{t_{i+1}} \|\Sigma(t_i,X^{(i)})(W_t-W_{t_i})\|^2\,dt. \end{align}\] Since \(\int_{t_i}^{t_{i+1}} (t-t_i)^2\,\mathrm{d}t\le \frac{\delta^3}{3}\); while \(\mathbb{E}\big[\|\Sigma(t_i,X^{(i)})(W_t-W_{t_i})\|^2\mid \mathcal{F}_{t_i}\big] = (t-t_i)\|\Sigma(t_i,X^{(i)})\|_{HS}^2\); then \[\label{eq:bounding95nicething} \mathbb{E}\int_{t_i}^{t_{i+1}} \|\bar{X}_t^{\delta}-\hat{X}_t^{\delta}\|^2\,dt \le C\delta^2\, \mathbb{E}\big[1+\|X^{(i)}\|^2\big] .\tag{26}\] Using the standard uniform second-moment bound for the Euler scheme, there exists some finite constant \(C_T>0\) such that, after summing over \(i=0,\dots,N_\delta-1\) in 26 , we find that \[\label{eq:bound95me95bby} \|\bar{X}^{\delta}-\hat{X}^{\delta}\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2 = \mathbb{E}\int_0^T \|\bar{X}_t^{\delta}-\hat{X}_t^{\delta}\|^2\,dt \le C_T\,\delta .\tag{27}\] Upon setting \(\delta>0\) small enough, together 25 and 27 yield \[\lim_{\delta\downarrow 0} \, \mathbb{E}\biggl[ \int_0^T\, \|\hat{X}^{\delta}_t-X_t\|^2\,dt \biggr] \le \lim_{\delta\downarrow 0} \, \mathbb{E}\biggl[ \int_0^T\, \|X_t-\bar{X}_t^{\delta}\|^2\,dt \biggr] + \lim_{\delta\downarrow 0} \, \|\bar{X}^{\delta}-\hat{X}^{\delta}\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2 = 0,\] which completes the proof. ◻

Proof of Proposition 7. We first prove the negative topological result, namely (i), before using some of the tools developed therein to conclude the negative measure-theoretic result (ii).
Proof of (ii) - Topological Non-Representativeness: Let us begin by simplifying our problem as follows. For each \(N\in\mathbb{N}_+\), define \(\mathfrak{S}_N\) to denote the set of all processes \(X\in \mathcal{H}_T^2(\mathbb{R}^{d_X})\) for which there exist deterministic times \(0=t_0\leq t_1\leq \cdots \leq t_N=T\) and random variables \(\xi_i\in L^2(\mathcal{F}_{t_i};\mathbb{R}^{d_X})\), \(i=0,\dots,N-1\), such that \[X_t = \sum_{i=0}^{N-1} \mathbf{1}_{[t_i,t_{i+1})}(t)\,\xi_i \qquad \mathrm{d}t\otimes d\mathbb{P}\text{-a.e.}\] By construction, \(\mathfrak{B} \subseteq \bigcup_{N=1}^{\infty}\mathfrak{S}_N\). By 14 , for any \(X_{\cdot}\in\mathfrak{B}\), there exists some “frequency” \(0<\delta <T\) such that \(X\) is constant on the deterministic grid intervals \([t_i,t_{i+1})\cap[0,T],\) so \(X\in \mathfrak{S}_N\) for \(N\stackrel{\upshape\tiny def.}{=}\lceil T/\delta\rceil\).

Since a set is meagre if it is contained in a countable union of nowhere dense sets, it is enough to show that each \(\mathfrak{S}_N\) is nowhere dense in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\). To this end, we show that each \(\mathfrak{S}_N\) is a closed subset of \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\) with empty interior. To see that each \(\mathfrak{S}_N\) is closed, let \((X^n)_{n\geq 1}\subseteq \mathfrak{S}_N\) converge in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\) to some \(X\). For each \(n\), choose a representation \[X^n_t = \sum_{i=0}^{N-1} \mathbf{1}_{[t_i^n,t_{i+1}^n)}(t)\,\xi_i^n, \qquad \xi_i^n\in L^2(\mathcal{F}_{t_i^n};\mathbb{R}^{d_X}),\] with \(0=t_0^n\leq t_1^n\leq \cdots \leq t_N^n=T.\) Passing to a subsequence, we may assume that, for every \(i=0,\dots,N\), the sequence \((t_i^n)_n\) converges to some \(t_i\in[0,T]\), and moreover is monotone in \(n\); consequently \(0=t_0\leq t_1\leq \cdots \leq t_N=T.\) Now, fix \(i\in\{0,\dots,N-1\}\) such that \(t_i<t_{i+1}\). Let \(I\subset (t_i,t_{i+1})\) be a compact sub-interval. Then, for all sufficiently large \(n\), one has \(I\subset [t_i^n,t_{i+1}^n),\) and therefore \(X_t^n=\xi_i^n\) \(\mathrm{d}t\otimes d\mathbb{P}\)-a.e.on \(I\times\Omega\). Hence, for \(n,m\in \mathbb{N}_+\) large enough, \[|I|\, \mathbb{E}\big[\|\xi_i^n-\xi_i^m\|^2\big] = \mathbb{E}\biggl[ \int_I \|X_t^n-X_t^m\|^2\,\mathrm{d}t \biggr] \le \|X^n-X^m\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2\] where \(|I|\) denotes the Lebesgue measure/length of the interval \(I\). Since \((X^n)_n\) is Cauchy in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\), it follows that \((\xi_i^n)_n\) is Cauchy in \(L^2\), hence converges in \(L^2\) to some \(\xi_i\in L^2(\Omega;\mathbb{R}^{d_X})\). The \(\mathbb{P}\)-a.s.continuity of \(W_{\cdot}\) implies that, since each \(\xi_i^n\) is \(\mathcal{F}_{t_i^n}\)-measurable and \((t_i^n)_n\) is monotone with limit \(t_i\), the continuity of the Brownian filtration implies that \(\xi_i\) is \(\mathcal{F}_{t_i}\)-measurable; and thus \(X_t=\xi_i\) \(\mathrm{d}t\otimes d\mathbb{P}\)-a.e.on \(I\times \Omega\). Since \(I\subset (t_i,t_{i+1})\) was arbitrary, we upgrade this deduction to the entire sub-interval \((t_i,t_{i+1})\); i.e. \(X_t=\xi_i\) \(\mathrm{d}t\otimes d\mathbb{P}\)-a.e.on \((t_i,t_{i+1})\times \Omega\). For indices \(i\) with \(t_i=t_{i+1}\), set \(\xi_i\stackrel{\upshape\tiny def.}{=}0\). We thus conclude that \[\label{eq:nice95representation} X_t = \sum_{i=0}^{N-1} \, \mathbf{1}_{[t_i,t_{i+1})}(t)\,\xi_i \qquad \mathrm{d}t\otimes \mathrm{d}\mathbb{P}\text{-a.e.} ,\tag{28}\] with \(\xi_i\in L^2(\mathcal{F}_{t_i};\mathbb{R}^{d_X})\). Hence \(X_{\cdot}\in \mathfrak{S}_N\), and so \(\mathfrak{S}_N\) is closed.

We now only need to show that it has empty interior. Again, fix an arbitrary \(X\in \mathfrak{S}_N\), \(\varepsilon>0\), and choose any representation of \(X_{\cdot}\) as in 28 . Then there must exist some \(j\in \{0,\dots,N-1\}\) such that \(t_j<t_{j+1}\), and hence some \(t_j<c<d<t_{j+1}\). Let \(B_t\stackrel{\upshape\tiny def.}{=}W_t^{(1)}\) be the first coordinate of \(W\), let \(e_1^{d_X}\) be the first canonical vector of \(\mathbb{R}^{d_X}\), fix some \(\eta>0\), and consider the “bridge-type” process \(R_t^{\eta} \stackrel{\upshape\tiny def.}{=} \eta\,\mathbf{1}_{[c,d)}(t)\,(B_t-B_c)e_1^{d_X}\). Then, for each \(\eta>0\), the process \(R^{\eta}_{\cdot}\) belongs to \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\) and its norm can be estimated as \[\label{eq:norme95me95up} \|R_{\cdot}^{\eta}\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2 = \eta^2\int_c^d \, \mathbb{E}[(B_t-B_c)^2]\,\mathrm{d}t = \eta^2\int_c^d (t-c)\,\mathrm{d}t .\tag{29}\] Now, fix \(\varepsilon>0\); and note that by 29 we may choose \(\eta>0\) small enough so that \(\|R_{\cdot}^{\eta}\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}<\varepsilon.\) We claim that \(X_{\cdot}+R_{\cdot}^{\eta} \notin \mathfrak{S}_N\) for any \(\eta>0\). This would conclude our argument, since it would show that for any \(\varepsilon>0\) the \(\varepsilon\)-ball about \(X_{\cdot}\) in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\) must contain a point outside of \(\mathfrak{S}_N\); meaning that \(\mathfrak{S}_N\) has empty interior (since every point in an open subset of a metric space must contain all sufficiently small balls about that point).
Indeed, on the interval \([c,d)\), the process \(X\) is equal to the constant random variable \(\xi_j\), whereas \(t\mapsto R_t^{\eta}(\omega)\) is continuous and non-constant on \([c,d)\) for \(\mathbb{P}\)-a.e.\(\omega\). Therefore, for \(\mathbb{P}\)-a.e.\(\omega\), the path \[t\mapsto (X_t+R_t^{\eta})(\omega)\] is continuous and non-constant on \([c,d)\). On the other hand, any element of \(\mathfrak{S}_N\) has, for \(\mathbb{P}\)-a.e.\(\omega\), a path which is piecewise constant on a deterministic partition with at most \(N\) intervals. Such a path, if continuous on \([c,d)\), must be constant there; which cannot happen for \(X_{\cdot}+R_{\cdot}^{\eta}\). Hence \(X_{\cdot}+R_{\cdot}^{\eta}\) does not belong to \(\mathfrak{S}_N\) for any \(\eta>0\). Thus, \(\mathfrak{B}\) is contained in a countable union of the nowhere dense sets \((\mathfrak{S}_N)_{N=1}^{\infty}\); whence \(\mathfrak{B}\) is meagre in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\).

Proof of (i) - Probabilistic non-representativeness: Our approach is simple, using the inclusion \(\mathfrak{B} \subseteq \bigcup_{N=1}^{\infty}\mathfrak{S}_N\) we have \[\label{eq:measure95me95this} \mu_G(\mathfrak{B}) \le \sum_{n=1}^{\infty}\, \mu_G(\mathfrak{S}_n) .\tag{30}\] Our strategy is then simply to show that each \(\mu_G(\mathfrak{S}_n)\) on the right-hand side of 30 has \(\mu_G\)-measure zero; which we do by showing that each \(\mathfrak{S}_n\) is contained in a countable union of proper closed linear subspaces; whence each of which must have \(\mu_G\)-measure \(0\) since \(Q\) has non-singular covariance operator, and thus so must \(\mathfrak{S}_n\).

More specifically, fix an arbitrary \(N\in \mathbb{N}_+\). For each pair of rational numbers \(a,b\in \mathbb{Q}\cap[0,T]\) with \[\label{eq:big95bad95indices} 0\le a<b\le T \,\, and \,\, \frac{T}{2N}<b-a .\tag{31}\] Define the linear subspace \(M_{a,b} \stackrel{\upshape\tiny def.}{=} \big\{ X\in \mathcal{H}_T^2(\mathbb{R}^{d_X}): \exists\;\xi\in L^2(\mathcal{F}_a;\mathbb{R}^{d_X}) \text{ s.t.} X_t=\xi \quad \mathrm{d}t\otimes \mathrm{d}\mathbb{P}\text{-a.e. on }[a,b)\times\Omega \big\}\). We claim \(M_{a,b}\) is closed in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\); indeed, if \((X_{\cdot}^{(n)})_{n=1}^{\infty}\) is a sequence in \(M_{a,b}\) converging to some \(X_{\cdot}\in \mathcal{H}_T^2(\mathbb{R}^{d_X})\) then, for each \(n\), we may choose \(\xi_n\in L^2(\mathcal{F}_a;\mathbb{R}^{d_X})\) such that \[X_t^n=\xi_n \qquad \mathrm{d}t\otimes \mathrm{d}\mathbb{P}\text{-a.e.\;on }[a,b)\times\Omega .\] Then \((b-a)\,\mathbb{E}[\|\xi_n-\xi_m\|^2] = \mathbb{E}\Big[\int_a^b \|X_t^n-X_t^m\|^2\,\mathrm{d}t\Big] \le \|X^n-X^m\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2\). Thus \((\xi_n)_n\) is Cauchy in \(L^2(\mathcal{F}_a;\mathbb{R}^{d_X})\), hence converges to some \(\xi\in L^2(\mathcal{F}_a;\mathbb{R}^{d_X})\). Passing to the limit gives \(X_t=\xi\) \(\mathrm{d}t\otimes \mathrm{d}\mathbb{P}\text{-a.e. on }[a,b)\times\Omega\); whence the limiting process \(X_{\cdot}\) also belongs to \(M_{a,b}\). Next, we now show that each \(M_{a,b}\) has positive co-dimension in \(\mathcal{H}_T^2(\mathbb{R}^{d_X})\). Let \(B_t\stackrel{\upshape\tiny def.}{=}W_t^{(1)}\) be the first coordinate of the Brownian motion, let \(e_1^{d_X}\) be the first canonical vector of \(\mathbb{R}^{d_X}\), and define \(Y_t\stackrel{\upshape\tiny def.}{=}\mathbf{1}_{[a,b)}(t)\,(B_t-B_a)e_1^{d_X}\). Then \(Y_{\cdot}\in \mathcal{H}_T^2(\mathbb{R}^{d_X})\), but \(Y_{\cdot}\notin M_{a,b}\); since, on \([a,b)\), the process \(t\mapsto (B_t-B_a)e_1^{d_X}\) is not \(\mathrm{d}t\otimes \mathrm{d}\mathbb{P}\)-a.e.equal to any \(\mathcal{F}_a\)-measurable random variable. Hence \(M_{a,b}\subsetneq \mathcal{H}_T^2(\mathbb{R}^{d_X})\). Thus, \(\mu_G(M_{a,b})=0\) for every \((a,b)\) satisfying 31 .

In order to conclude that \(\mathfrak{S}_N\) has \(\mu_G\)-measure zero, we will capture it in the union of countably many of these “bad” \(M_{a,b}\). Indeed, let \(\Gamma_N\) denote the countable set of \((a,b)\in \mathbb{Q}^2\) satisfying 31 . Clearly \(\Gamma_N\) is countable; thus it remains to show that \[\label{eq:capture95me95this} \mathfrak{S}_N \subseteq \bigcup_{(a,b)\in \Gamma_N} M_{a,b} .\tag{32}\] Indeed, let \(X\in \mathfrak{S}_N\). Then, again, by 14 there exists a deterministic partition \(0=t_0\le t_1\le \cdots \le t_N=T\) such that \(X\) is constant on each interval \([t_i,t_{i+1})\). Since \(\sum_{i=0}^{N-1}(t_{i+1}-t_i)=T,\) there must exist some \(j\in [N-1]\) such that \(t_{j+1}-t_j\ge \frac{T}{N}.\) By density of the rationals in the reals, there exists some \(a,b\in \mathbb{Q}\) such that \(t_j<a<b<t_{j+1}\) and \(b-a>\frac{T}{2N}\). Since \(X\) is constant on \([t_j,t_{j+1})\), it is in particular constant on \([a,b)\). Moreover the corresponding constant random variable is \(\mathcal{F}_{t_j}\)-measurable, hence also \(\mathcal{F}_a\)-measurable because \(\mathcal{F}_{t_j}\subseteq \mathcal{F}_a\), thus \(X\in M_{a,b}\), which proves 32 . Hence, \[\mu_G(\mathfrak{S}_N) \le \sum_{(a,b)\in\Gamma_N}\mu_G(M_{a,b}) = 0.\] Since \(N\in\mathbb{N}_+\) was arbitrary, using 30 gives \[\mu_G(\mathfrak{B}) = 0,\] which completes the proof. ◻

Proof of Lemma 1. Let \(L \in [0,+\infty)\) be a common Lipschitz constant for \(b\) and \(\Sigma\) in the variables \((x,a)\), uniformly in \(t\). Set \(M_b\stackrel{\upshape\tiny def.}{=}\sup_{(t,x,a)} |b(t,x,a)|_{\mathbb{R}^{d_X}}\) and \(M_\Sigma\stackrel{\upshape\tiny def.}{=}\sup_{(t,x,a)} \|\Sigma(t,x,a)\|_{\mathrm F}\). For a fixed \(\alpha\in \mathcal{H}_T^2(\mathbb{R}^{d_\alpha})\), the coefficients \((t,\omega,x)\mapsto b(t,x,\alpha_t(\omega))\) and \((t,\omega,x)\mapsto \Sigma(t,x,\alpha_t(\omega))\) are progressively measurable, globally Lipschitz in \(x\), and bounded. Hence the SDE admits a unique strong solution with continuous adapted paths. In particular, \(X^\alpha\) is predictable. We next show that \(X^\alpha\in \mathcal{H}_T^2(\mathbb{R}^{d_X})\). By the elementary inequality \(|u+v+w|^2\le3(|u|^2+|v|^2+|w|^2)\), Cauchy–Schwarz, and Itô isometry, for every \(t\in[0,T]\), \[\begin{align} \mathbb{E}\left[|X_t^\alpha|_{\mathbb{R}^{d_X}}^2\right] &\le 3\mathbb{E}\left[|x_0|_{\mathbb{R}^{d_X}}^2\right] + 3\mathbb{E}\left[ \left| \int_0^t b(s,X_s^\alpha,\alpha_s)\,ds \right|_{\mathbb{R}^{d_X}}^2 \right] + 3\mathbb{E}\left[ \left| \int_0^t \Sigma(s,X_s^\alpha,\alpha_s)\,\mathrm{d}W_s \right|_{\mathbb{R}^{d_X}}^2 \right] \\ &\le 3\mathbb{E}\left[|x_0|_{\mathbb{R}^{d_X}}^2\right] + 3T^2M_b^2 + 3TM_\Sigma^2. \end{align}\] Integrating over \(t\in[0,T]\) gives \(X^\alpha\in \mathcal{H}_T^2(\mathbb{R}^{d_X})\). It remains to prove Lipschitz continuity of \(\Gamma\). In order to do so, we let \(\alpha,\beta\in \mathcal{H}_T^2(\mathbb{R}^{d_\alpha})\), and write \(\Delta X_t\stackrel{\upshape\tiny def.}{=}X_t^\alpha-X_t^\beta\) and \(\Delta\alpha_t\stackrel{\upshape\tiny def.}{=}\alpha_t-\beta_t\). Then \[\Delta X_t = \int_0^t \left[ b(s,X_s^\alpha,\alpha_s) - b(s,X_s^\beta,\beta_s) \right]\mathrm{d}s + \int_0^t \left[ \Sigma(s,X_s^\alpha,\alpha_s) - \Sigma(s,X_s^\beta,\beta_s) \right]\mathrm{d}W_s.\] Using Cauchy–Schwarz, Itô isometry, and the Lipschitz property, we obtain \[\begin{align} \mathbb{E}\left[|\Delta X_t|_{\mathbb{R}^{d_X}}^2\right] &\le 2T \int_0^t \mathbb{E}\left[ \left| b(s,X_s^\alpha,\alpha_s) - b(s,X_s^\beta,\beta_s) \right|_{\mathbb{R}^{d_X}}^2 \right]\mathrm{d}s \\ &\quad+ 2 \int_0^t \mathbb{E}\left[ \left\| \Sigma(s,X_s^\alpha,\alpha_s) - \Sigma(s,X_s^\beta,\beta_s) \right\|_{\mathrm F}^2 \right]\mathrm{d}s \\ &\le 2(T+1)L^2 \int_0^t \mathbb{E}\left[ |\Delta X_s|_{\mathbb{R}^{d_X}}^2 + |\Delta\alpha_s|_{\mathbb{R}^{d_\alpha}}^2 \right]\mathrm{d}s. \end{align}\] Set \(f(t)\stackrel{\upshape\tiny def.}{=}\mathbb{E}\left[|\Delta X_t|_{\mathbb{R}^{d_X}}^2\right]\), \(C_0\stackrel{\upshape\tiny def.}{=}2(T+1)L^2\). Then \(f(t) \le C_0\int_0^t f(s)\,\mathrm{d}s + C_0\int_0^t \mathbb{E}\left[ |\Delta\alpha_s|_{\mathbb{R}^{d_\alpha}}^2 \right]\mathrm{d}s\). By Grönwall’s lemma, \(f(t) \le C_0e^{C_0T} \int_0^T \mathbb{E}\left[ |\Delta\alpha_s|_{\mathbb{R}^{d_\alpha}}^2 \right]\mathrm{d}s\). Integrating again over \(t\in[0,T]\), we obtain \[\|X^\alpha-X^\beta\|_{\mathcal{H}_T^2(\mathbb{R}^{d_X})}^2 \le T C_0 e^{C_0T} \|\alpha-\beta\|_{\mathcal{H}_T^2(\mathbb{R}^{d_\alpha})}^2.\] This concludes the proof. ◻