1 Introduction↩︎

In statistical problems involving a vector \(\boldsymbol{\beta}_0 = (\beta_{01},...,\beta_{0n})\in \mathbb{R}^n\) of unobserved parameters of the same kind, a popular strategy for borrowing strength is to assume that the parameters are i.i.d. (or at least exchangeable) draws from some common distribution, \[\begin{align} \label{eq:prior-iid} \beta_{0j} & \overset{iid}{\sim}G_0, \qquad j=1,...,n, \end{align}\tag{1}\] but regard the fixed distribution \(G_0\) as unknown. This conceptually simple idea allows one to exploit the symmetry among the parameters to incorporate regularization (shrinkage) into the given likelihood function, while permitting the form of regularization to be learned from the data rather than specified in advance. To emphasize, we assume throughout that 1 is part of the data generating mechanism, i.e., that the true model for the data includes some fixed and unknown distribution \(G_0\) from which the true parameter values \(\beta_{0j}\) are drawn independently. Under this general empirical Bayes (EB) setup, and assuming \(n\) is large, the current article is broadly concerned with nonparametric estimation of the mixing distribution \(G_0\), known as a deconvolution problem.

In the statistical literature, deconvolution problems are classically considered within a sequence model, where for each of the unknown parameters \(\beta_{0j}\in \Theta\) we observe an independent noisy measurement from some common and known (marginal) likelihood, \[\begin{align} \label{eq:lik-sequence} Y_j|\beta_{0j} &\overset{ind}{\sim} f(y_j|\beta_{0j}), \qquad j=1,...,n. \end{align}\tag{2}\] Under the ‘separable’ two-level model specified by 1 2 , the pairs \((Y_j, \beta_{0j})\) are i.i.d., in particular the components of \(\boldsymbol{Y}= (Y_1,...,Y_n)\) are i.i.d. copies from \(f_{G_0}(y) := \int f(y\lvert \beta)G_0(d\beta)\), a ‘convolution’ of the known likelihood \(f\) with the unknown \(G_0\). ¹ The most classical deconvolution method in the sequence model is the nonparametric maximum likelihood estimator [4], [5], which seeks a maximizer of the (log-) likelihood \(\ell(G) := \sum_{j=1}^n \log f_{G}(y_j)\), over all possible distributions \(G\). The NPMLE is known to be discrete with at most \(n\) support points [5], [6], a property that can be leveraged to speed up computation [7], but on the other hand could be considered a disadvantage if \(G_0\) is believed to have a density. Various other methods for estimating \(G_0\), and approximations, both discrete and continuous, to the NPMLE under the sequence model have been proposed over the years, offering advantages of different kinds [8]–[10].

An alternative approach to producing a deconvolution estimator of the (fixed) distribution \(G_0\) in 1 is the nonparametric Bayes approach, which posits another hierarchy in the model by placing a ‘prior on priors’, \[\begin{align} \label{eq:prior-hier-Bayes} \beta_j|G & \overset{iid}{\sim}G, \qquad G\sim \Pi, \end{align}\tag{3}\] where we write \(\beta_j\) and \(G\) to distinguish from the true coefficients \(\beta_{0j}\) and distribution \(G_0\), and where 2 should now be understood as the conditional distribution of the data given \(\boldsymbol{\beta}=\boldsymbol{\beta}_0\) (for an arbitrary \(\boldsymbol{\beta}_0\)). Above, \(\Pi\) is a completely specified prior with a rich support in the space of distributions on \(\Theta\), to accommodate the nonparametric nature of the problem. The fully Bayes model specified by 3 and 2 automates information sharing between the samples through the calculation of posteriors (not only of \(G\) but also of the parameters \(\beta_j\), or the predictive posterior of a fresh sample \(Y_{n+1}\)), because under 3 the pairs \((Y_j, \beta_j)\) are not independent, only conditionally independent given \(G\). As such, one can obtain a deconvolution estimate of the original \(G_0\) in 1 by summarizing the posterior \(\Pi(\cdot|\boldsymbol{Y})\) of \(G\). Assuming a sequence model for the data, [11] first proposed to operationalize the nonparametric Bayes framework by taking \(\Pi\) to be a mixture of Dirichlet processes; [12], [13] worked out examples for specific likelihood functions, and [14] treated the general case. Later, [15] extended this construction by showing how posterior distributions can still be calculated analytically when \(\Pi\) is chosen to be a Pólya tree (PT) prior. Pólya trees form a class of prior distributions that generalizes Dirichlet processes while retaining computational tractability. One notable advantage of PT priors over Dirichlet processes (or mixtures thereof) is that they can be chosen so that \(G\) has a density almost surely, which makes PTs more suitable for modeling continuous distributions \(G_0\).

Recently, a number of papers have appeared that move beyond the conventional sequence model 2 , and tackle the nonparametric deconvolution problem in a linear regression model, \[\begin{align} \label{eq:lik-linear} \boldsymbol{Y}\lvert \boldsymbol{\beta}_0 \sim \mathcal{N}(X\boldsymbol{\beta}_0, \sigma^2I_m), \end{align}\tag{4}\] where \(X\in \mathbb{R}^{m\times n}\) is a fixed and known covariate matrix whose columns \(X_j\in \mathbb{R}^m\) are assumed linearly independent, and the noise term \(\sigma^2>0\) can be regarded as known or unknown. Maintaining the assumption 1 that the parameters \(\beta_{0j}\), now representing the regression coefficients, are i.i.d., results as before in a two-level model with an unknown prior; however, even at the methodological (algorithmic) level, the deconvolution problem becomes substantially harder compared to the sequence case, because under 4 the likelihood of each \(Y_i\) depends on the entire vector \(\boldsymbol{\beta}_0\). From the nonparametric empirical Bayes (EB) perspective, which regards \(G_0\) as strictly fixed, the NPMLE is still a perfectly sensible estimator of \(G_0\), but its implementation—to maximize the marginal log-likelihood of \(\boldsymbol{Y}\) over all \(G\)—is considerably more difficult under 4 , for example it is generally not a convex problem anymore. To circumvent the intractability of the NPMLE, [16] proposed to model \(G_0\) as a scale mixture of zero-mean Gaussians (more precisely, they model the rescaled coefficients \(\beta_{0j}/\sigma\) as i.i.d.), and employ variational Bayes techniques that approximate the posterior within the family of product (posterior) distributions. Compared to the variational approach of [17], this simplifies the computation while using a much more flexible family of priors. Essentially, [16] show how in optimizing the variational objective their approach allows to benefit from reductions to the much simpler normal sequence model, the special case obtained in 4 when \(m=n\) and \(x_{ij} = 1(i=j), 1\leq i,j\leq n\). Encouraged by the computational advantages, [18] carried out a theoretical analysis of (a variant of) the variational EB method of Kim et al. in an asymptotic setting where \(m,n\to \infty\), and under some mild assumptions established consistency of the estimator, as well as of the exact NPMLE. Still, because the advocated (variational) method uses the naive mean-field approximation, it can be inaccurate if covariates are strongly correlated. Intended for those situations, [19] assume a bounded \(G_0\) as in Mukherjee et al., but propose a more ambitious method for approximating the NPMLE, which uses the variational representation while avoiding the mean-field approximation of the posterior. Their estimator employs modern techniques and is characterized by a system of gradient flow equations which are solved with an MCMC expectation-maximization approach. The authors prove consistency of the estimator to the NPMLE in a high dimensional setting, under conditions on \(X\) that build on but relax those of Mukherjee et al.

A different, nonparametric Bayes approach to the deconvolution problem in the linear model (and more generally, in generalized linear models) was presented in [1], who proposed to extend the ideas of [15] from the sequence model to regression models. Thus, assuming the data now truly follows 4 and 1 for a fixed and unknown \(G_0\), they replace 1 with the working assumption 3 , where \(\Pi\) is a (truncated) Pólya tree prior on distributions. To handle the more complicated likelihood compared with the sequence model, they use an MCMC approach along with a Gibbs sampler that exploits a known conjugacy property of PTs, specifically, that under 3 the posterior of \(G\) given the unobserved parameters \(\boldsymbol{\beta}\) is again a Pólya tree. Weinstein et al. report encouraging simulation results both in a (mixed-effects) linear regression example and in logistic regression examples, demonstrating that in terms of estimating the CDF of the true \(G_0\), their hierarchical Bayes method improves significantly over different competitors, parametric and non-parametric.

These encouraging empirical results motivate us in the current article to carry out a theoretical analysis of the method, which is not included in the original work of [1]. Specifically, in a suitable high dimensional setting and assuming the pair \((\boldsymbol{Y}, \boldsymbol{\beta}_0)\) follows the frequentist (in terms of \(G_0\)) model given by 4 and 1 , we set out to establish posterior consistency of \(G\) under the postulated PT model, i.e., that the conditional distribution \(\Pi(\cdot\lvert \boldsymbol{Y})\) of \(G\) converges in a certain sense to a point mass at \(G_0\). Our work builds on and generalizes results from [2], where posterior consistency and corresponding rates for PTs are established in the density estimation problem, the setting in which the density of a continuous \(G_0\) in 1 is to be estimated when observing the coefficients \(\boldsymbol{\beta}\) directly without noise. By contrast, we consider an empirical Bayes setting where only the noisy observations \(Y_i\) (and \(X\)) from 4 are available, and \(\boldsymbol{\beta}\) is latent.

While posterior consistency of \(G\) and rates of convergence for Bayesian nonparametric methods in empirical Bayes problems (where \(\boldsymbol{\beta}\) is unobserved) have been studied before [20], previous work is, to the best of our knowledge, limited to the sequence model; as noted before, the regression model is considerably different and more difficult to analyze, because the likelihood of each \(Y_i\) in 4 involves the entire (common) vector \(\boldsymbol{\beta}\), as opposed to the sequence model in which each \(Y_i\) has its own parameter \(\beta_{i}\). Our main theoretical result shows that, assuming \(G_0\) has a density \(g_0\), and in an appropriate asymptotic regime where \(m,n\to\infty\) and the design matrix is sufficiently well-conditioned, the posterior for the density \(g\) of \(G\) under the postulated model concentrates around the true density \(g_0\) in sup-norm. As in [2], the contraction rate reflects a bias–variance tradeoff, which is accounted for by the approximation error from truncating the Pólya tree on the one hand and the stochastic error from estimating bin probabilities from \(n\) samples on the other hand. However, as opposed to the problem analyzed in [2], in our setting the admissible truncation depth, and hence the achievable resolution, is further constrained by the difficulty of recovering \(\boldsymbol{\beta}\) from the noisy observations \(\boldsymbol{Y}\) in 4 . When this additional constraint is inactive, there is no loss relative to the idealized setting in which the coefficients are observed directly, and the rate matches that of [2].

The rest of the paper is organized as follows. In Section 2 we set up the problem formally and state some basic assumptions. Section 3 recalls the Pólya tree-based hierarchical model from [1]. In Section 4 we prove the main result of the paper, Theorem 2, which asserts posterior concentration of the PT under the true (unknown and nonrandom) density of the coefficients. We supplement the main theorem with a corollary showing consistency, in squared \(\ell_2\) loss, of the posterior mean estimator of the coefficient vector \(\boldsymbol{\beta}_0\), as well as a result for random Gaussian designs, in which case we obtain a simple growth condition on \(m,n\) ensuring posterior convergence. Section 5 contains a high-level outline of the proof of the main result.

2 Data-generating model and problem setup↩︎

The vector \(\boldsymbol{Y}= (Y_1,...,Y_m)\) of observations is assumed to be generated from the linear model 4 , where \(m\ge n\), \(X\in\mathbb{R}^{m\times n}\) is a fixed design matrix with full column rank, and \(\sigma^2>0\) is taken to be known. We allow \(\sigma^2\) to depend on \((m,n)\), although this dependence is suppressed in the notation. Additionally, the components of the unobserved coefficient vector \(\boldsymbol{\beta}_0=(\beta_{01},\dots,\beta_{0n})\in\mathbb{R}^n\) are assumed to be i.i.d. draws as specified in 1 , where \(G_0\) is fixed and unknown. Throughout, we assume that \(G_0\) admits a density \(g_0\) supported on a known compact interval \([a,b]\subset\mathbb{R}\). We assume that \(g_0\) is \(\alpha\)-Hölder smooth for some \(\alpha\in(0,1]\), and is bounded above and below away from zero, i.e., that there exist constants \(L_0<\infty\) and \(0<m_0\le M_0<\infty\) such that \[m_0 \le g_0(x)\le M_0, \qquad |g_0(x)-g_0(y)|\le L_0 |x-y|^\alpha, \qquad x,y\in[a,b].\] For every \(g_0\), the model given by 1 and 4 defines a joint distribution on \((\boldsymbol{\beta}_0,\boldsymbol{Y})\), which we denote by \(\mathbb{P}_{g_0}\). This is the true data-generating law. Expectations under \(\mathbb{P}_{g_0}\) are correspondingly denoted by \(\mathbb{E}_{g_0}\).

Following [1], to draw inference on \(g_0\), we postulate the hierarchical Bayes model given by 3 and 4 , specifying a truncated Pólya tree prior of depth \(L\) on the unknown mixing density \(g\). For each fixed \(\boldsymbol{y}\in \mathbb{R}^m\), we will use \(\Pi_L(\cdot\mid \boldsymbol{y})\) to denote the posterior of \(g\) under this postulated model, and \(\Pi_L(\cdot\mid \boldsymbol{Y})\) for the corresponding random object obtained by plugging \(\boldsymbol{y}= \boldsymbol{Y}\). Our main objective in this article is to show that, under suitable conditions on the design matrix \(X\), the resulting posterior concentrates around \(g_0\) in sup-norm as \(m,n\to\infty\). Throughout, posterior contraction is always understood with respect to the true law. For example, a statement of the form \[\label{eq:post-contraction} \mathbb{E}_{g_0}\!\left[\Pi_L\!\left(\|g-g_0\|_\infty>C_0\varepsilon_n \mid \boldsymbol{Y}\right)\right]\to 0\tag{5}\] means that the posterior arising from the postulated model concentrates near the true density \(g_0\), when expectation over \(\boldsymbol{Y}\) is taken under \(\mathbb{P}_{g_0}\).

3 A hierarchical Bayes approach↩︎

We recall the truncated Pólya tree prior from [1] and the resulting hierarchical Bayes model.

3.1 The level-\(L\) truncated Pólya tree prior↩︎

Let \(L = L(m,n) \in \mathbb{N}\) denote the truncation level (or depth) of the tree, to be specified later as a function of \((m,n)\).

Dyadic partition of the support. For each level \(l\ge 0\), partition \([a,b]\) into \(2^l\) dyadic subintervals \[I_k^l := \begin{cases} \left[a+(b-a)\frac{k}{2^l},\, a+(b-a)\frac{k+1}{2^l}\right), & k=0,\dots,2^l-2,\\[0.4em] \left[b-\frac{b-a}{2^l},\, b\right], & k=2^l-1. \end{cases}\] Thus \(\{I_k^l:0\le k<2^l\}\) is a partition of \([a,b]\) for every \(l\).

Construction of the truncated Pólya tree prior. For each node \((l,k)\), with \(l=0,\dots,L-1\) and \(k=0,\dots,2^l-1\), let \[V_{l,k}\overset{ind}{\sim}\mathrm{Beta}(1,1)\] denote the random split proportion assigned to the left child. These variables determine a random probability distribution \(G\) on the level-\(L\) partition recursively as follows. Starting from \(G(I_0^0)=1\), define for each \(l=0,\dots,L-1\) and \(k=0,\dots,2^l-1\), \[G(I_{2k}^{\,l+1}) = V_{l,k}\,G(I_k^l), \qquad G(I_{2k+1}^{\,l+1}) = (1-V_{l,k})\,G(I_k^l).\] In other words, the mass assigned to a terminal cell \(I_k^L\) is the product of the split proportions along the unique path from the root to that cell. The truncated Pólya tree prior then identifies \(G\) with the piecewise-constant density \[g(x)=\sum_{k=0}^{2^L-1}\frac{2^L}{b-a}\,G(I_k^L)\,\mathbb{1}\{x\in I_k^L\}, \qquad x\in[a,b].\] We denote the resulting prior on densities \(g\) by \(\Pi_L\).

3.2 Hierarchical Bayes model↩︎

Incorporating the level-\(L\) truncated PT prior \(\Pi_L\) on \(g\) leads to the following hierarchical Bayes model, to which we refer as the working model: \[\begin{align} &g \sim \Pi_L, \tag{6}\\ &\beta_1,...,\beta_n \mid g \overset{iid}{\sim} g \tag{7}\\ &\boldsymbol{Y}\mid \boldsymbol{\beta}\sim \mathcal{N}(X\boldsymbol{\beta},\sigma^2 I_m), \tag{8} \end{align}\] where we used \(\beta_j\) here instead of \(\beta_{0j}\) to distinguish the coefficients from those of the true model. We rely on the posterior distribution of \(g\) given \(\boldsymbol{Y}\) under the working model to estimate the true density \(g_0\).

A key feature of the Pólya tree construction is that the posterior distribution of \(g\) given \(\boldsymbol{\beta}\) remains a truncated Pólya tree [21], [22]. To state this formally, define the empirical bin counts, \[N_{\boldsymbol{\beta}}(I_k^l):=\sum_{j=1}^n \mathbb{1}\{\beta_j\in I_k^l\}, \qquad l=0,\dots,L,\quad k=0,\dots,2^l-1,\] and the left/right child counts, \[N_{\boldsymbol{\beta}}^{(0)}(I_k^l):=N_{\boldsymbol{\beta}}(I_{2k}^{l+1}), \qquad N_{\boldsymbol{\beta}}^{(1)}(I_k^l):=N_{\boldsymbol{\beta}}(I_{2k+1}^{l+1}), \qquad l=0,\dots,L-1,\quad k=0,\dots,2^l-1.\] Then, under the \(\mathrm{Beta}(1,1)\) prior, we have \[V_{l,k}\mid \boldsymbol{\beta}\sim \mathrm{Beta}\big(1+N_{\boldsymbol{\beta}}^{(0)}(I_k^l),\,1+N_{\boldsymbol{\beta}}^{(1)}(I_k^l)\big),\] independently across nodes \((l,k)\). Equivalently, the conditional distribution \(\Pi_L(\cdot\mid \boldsymbol{\beta})\) is again a truncated Pólya tree, with updated Beta parameters determined by the bin counts of \(\boldsymbol{\beta}\).

3.3 Posterior distribution↩︎

Since under the working model \(g\) and \(\boldsymbol{Y}\) are independent given \(\boldsymbol{\beta}\), the posterior distribution of \(g\) can be written as the mixture \[\Pi_L(A\mid \boldsymbol{y}) = \int \Pi_L(A\mid \boldsymbol{\beta})\,\Pi_{L,\beta}(d\boldsymbol{\beta}\mid \boldsymbol{y}),\] for every measurable set \(A\) of densities supported on \([a,b]\). Here, \(\Pi_{L,\beta}(\cdot \mid \boldsymbol{y})\) denotes the posterior distribution of \(\boldsymbol{\beta}\) under 6 8 (analogously, \(\Pi_{L,\beta}\) denotes its prior distribution).

Although the posteriors \(\Pi_L(\cdot\mid \boldsymbol{y})\) and \(\Pi_{L,\beta}(\cdot\mid \boldsymbol{y})\) of \(g\) and \(\boldsymbol{\beta}\) are not available in closed form, posterior inference can be carried out via a Gibbs sampling algorithm, alternating between updates of \(\boldsymbol{\beta}\) and \(g\). Conveniently, the update of \(g\) is direct, since \(\Pi_L(\cdot |\boldsymbol{\beta})\) is again a truncated Pólya tree with Beta parameters updated by the empirical bin counts.

4 Main results↩︎

We now state our main posterior contraction result for the truncated Pólya tree procedure introduced in the previous section. Our analysis builds on results of [2], where posterior contraction for Pólya tree priors is studied under the density estimation setting, i.e., in the case where one directly observes the i.i.d. samples \(\beta_{0j}\) from \(g_0\) (in that case, of course, the likelihood of \(\boldsymbol{Y}\lvert \boldsymbol{\beta}\) is irrelevant). We therefore begin by recalling the corresponding direct-observation benchmark, adapted to our notation and assumptions. Then, we state our main theorem for the regression model, which extends this benchmark to the harder case where the coefficients are latent and only the noisy linear observation \(\boldsymbol{Y}\sim \mathcal{N}(X\boldsymbol{\beta}_0, \sigma^2I_m)\) is available.

For a truncation level \(L\in\mathbb{N}\), define \[\label{eq:epsilon-def} \varepsilon_n(L) := (b-a)^{\alpha}2^{-\alpha L} + (b-a)^{-1}\sqrt{\frac{L\,2^L}{n}}.\tag{9}\] The first term is the approximation error from representing the density on the level-\(L\) dyadic partition (‘quantization’), and the second is the stochastic error from estimating the corresponding bin probabilities from \(n\) samples. For each fixed truncation level \(L\), the quantity \(\varepsilon_n(L)\) is the level-\(L\) bias–variance scale governing the sup-norm accuracy of a truncated Pólya tree posterior.

The role of the theory below is to determine which truncation levels are admissible under the two models for the data. In the ‘noiseless’ setting of [2] the coefficients are observed directly, and truncation levels up to \(L_{\mathrm{Cast}}\), defined below, are admissible. In our regression setting, by contrast, the empirical bin counts are not observed and must instead be inferred from the noisy linear observation \(\boldsymbol{Y}\). This creates an additional admissibility constraint: the dyadic partition cannot be finer than the resolution at which the latent coefficients, and hence their bin counts, are recoverable from the regression model. The resulting contraction rate is obtained by evaluating \(\varepsilon_n(L)\) at the largest truncation level satisfying both the direct-observation admissibility constraint and this additional regression-induced constraint. When the latter constraint is inactive, the rate reduces to the direct-observation rate of [2].

4.1 Review: posterior contraction from direct observations↩︎

We first record the direct-observation benchmark corresponding to [2]. Since here \(\boldsymbol{\beta}_0=(\beta_{01},\dots,\beta_{0n})\) is observed, the problem reduces to density estimation from i.i.d. samples from \(g_0\). The theorem below adapts Castillo’s posterior contraction result for Pólya trees to our notation and assumptions, including compact support \([a,b]\) and the truncated Pólya tree prior described in Section 3.

Let \[L_{\mathrm{Cast}} := \left\lfloor \log_2\!\left( c_{\mathrm{Cast}}\left(\frac{n}{\log n}\right)^{\frac{1}{2\alpha+1}} \right) \right\rfloor\] for a sufficiently small constant \(c_{\mathrm{Cast}}>0\). This is the largest truncation level allowed by the direct-observation argument. For \(L\le L_{\mathrm{Cast}}\), let \(\Pi_L(\cdot\mid \boldsymbol{\beta}_0)\) denote the posterior distribution on \(g\) under the level-\(L\) truncated Pólya tree prior given the direct observations \(\boldsymbol{\beta}_0\).

Define \[\Lambda_n(l):=\sqrt{(l+L)\,n\,2^{-l}}, \qquad P_0(I_k^l):=\int_{I_k^l} g_0(x)\,dx.\] For \(M>0\), let \(\mathcal{B}=\mathcal{B}(M,L)\) be the set of all \(\boldsymbol{\beta}\in [a,b]^n\) such that \[\big|N_{\boldsymbol{\beta}}(I_k^l)-nP_0(I_k^l)\big| \le M \{\Lambda_n(l)\vee(l+L)\}\] simultaneously for all \(0\le l\le L\) and \(0\le k<2^l\). Thus, \(\mathcal{B}\) is the event that the empirical bin counts are uniformly close to their expectations under \(g_0\). With this notation, the direct-observation benchmark can be stated as follows.

4.2 Posterior contraction from noisy linear observations↩︎

We now turn to the main technical contribution, which can be viewed as an extension of Castillo’s result from the directly-observed coefficients model to the noisy linear model. Relative to the direct-observation benchmark, the new element is an additional restriction on the truncation level, reflecting the accuracy with which the latent coefficients can be recovered from \(\boldsymbol{Y}\). We encode this restriction below through \(L_{\mathrm{noise}}\).

Let \(\lambda_{\min}(X^\top X)\) denote the smallest eigenvalue of \(X^\top X\). For a sufficiently small constant \(c_{\mathrm{noise}}>0\), define \[L_{\mathrm{noise}} := \max\left\{ \ell\in\mathbb{N}: \frac{2^{3\ell/2}}{\ell^{3/2}} \le c_{\mathrm{noise}}\,\frac{\lambda_{\min}(X^\top X)}{\sigma^2 n^{3/2}} \right\}.\] We choose the truncation level as \[\label{eq:Lmn-def} L:=\min\{L_{\mathrm{Cast}},\,L_{\mathrm{noise}}\}.\tag{10}\] Thus \(L_{\mathrm{Cast}}\) is the upper bound from [2], while \(L_{\mathrm{noise}}\) is the additional resolution limit due to observing \(\boldsymbol{Y}\) instead of \(\boldsymbol{\beta}_0\). We can now state the main result of the paper.

Thus the contraction rate has the same functional form as in the direct-observation problem, but evaluated at the smaller admissible depth \(L=\min\{L_{\mathrm{Cast}},\,L_{\mathrm{noise}}\}\). When \(L_{\mathrm{noise}} \ge L_{\mathrm{Cast}}\), the regression problem incurs no additional loss and the Castillo direct-observation rate is recovered.

4.2.0.1 Posterior mean estimation of the coefficients.

The preceding theorem concerns recovery of the mixing density \(g_0\). As part of its proof, we show and use an auxiliary contraction bound for the latent coefficient vector \(\boldsymbol{\beta}\), stated as Proposition 4 in Appendix 8. Thus, \(\boldsymbol{\beta}\) contraction is not derived from the density contraction statement; instead, it relies on the given likelihood function, and is one of the elements used to prove posterior convergence of \(\Pi_L\). In addition to its use in the proof of Theorem 2, Proposition 4 also yields the following consequence for the Bayes estimator, under squared loss, of the coefficient vector \(\boldsymbol{\beta}_0\). Let \[\widehat{\boldsymbol{\beta}}(\boldsymbol{Y}) := \int \boldsymbol{\beta}\,\Pi_{L,\beta}(d\boldsymbol{\beta}\mid \boldsymbol{Y})\] denote the posterior mean of the coefficient vector under the working Pólya tree model.

Corollary 1 concerns a different target from Theorem 2. For coefficient recovery in the regime considered here, the regression likelihood drives the estimation of \(\boldsymbol{\beta}_0\), while the truncation level \(L\) enters through the complexity of the induced marginal prior on \(\boldsymbol{\beta}\). Thus smaller truncation levels make the displayed sufficient condition easier to satisfy, but this should not be interpreted as a recommendation for the density recovery problem. For instance, when \(L=0\), the working prior on \(g\) is the uniform density on \([a,b]\); such a prior may still yield consistent estimation of \(\boldsymbol{\beta}_0\) under a well-conditioned design, but it cannot consistently estimate a non-uniform mixing density \(g_0\).

4.3 Random designs↩︎

We next state a random-design consequence of Theorem 2. In this subsection, probability statements involving \(X\) are with respect to the random-design law, denoted by \(\mathbb{P}_X\). For a random design, write \(L_{\mathrm{noise}}(X)\) for \(L_{\mathrm{noise}}\) evaluated at the realized matrix \(X\), and set \[L(X):=\min\{L_{\mathrm{Cast}},L_{\mathrm{noise}}(X)\}.\]

The relevant quantity is \[\frac{\lambda_{\min}(X^\top X)}{\sigma^2 n^{3/2}},\] which controls \(L_{\mathrm{noise}}(X)\). For Gaussian random designs with i.i.d. rows whose second-moment matrices have eigenvalues uniformly bounded away from zero, Lemma 9 in Appendix 9 shows that, provided the design is sufficiently tall, \[\lambda_{\min}(X^\top X)\gtrsim m\] with probability tending to one. Thus, under the growth condition \[\frac{m}{\sigma^2 n^{3/2}}\to\infty,\] we obtain \(L(X)\to\infty\) in \(\mathbb{P}_X\)-probability, which is the random-design analogue of the fixed-design assumption in Theorem 2. This yields the following random-design consequence.

5 Proof sketch of Theorem 2↩︎

We now explain the main ideas behind the proof of Theorem 2; the full proof appears in Appendix 7. At a high level, the proof transfers the direct-observation Pólya tree benchmark of Theorem 1 to the regression setting. This transfer is not automatic: the benchmark applies once the coefficient vector has the correct empirical bin counts, whereas in the regression model the coefficients are latent. The main additional work, and the main technical challenge, is therefore to show that, under suitable conditioning of the design matrix and for truncation levels \(L\le L_{\mathrm{noise}}\), the posterior for \(\boldsymbol{\beta}\) concentrates tightly enough around the realized vector \(\boldsymbol{\beta}_0\) for those count conditions to remain valid. We break the proof outline into the following steps.

Step 1. Reduction to the direct-observation benchmark. Denote the complement of the target sup-norm ball by \[T_g:=\{g:\|g-g_0\|_\infty > C_0\,\varepsilon_n(L)\}.\] Using the posterior mixture representation from Section 3, \[\Pi_L(T_g\mid \boldsymbol{Y}) = \int \Pi_L(T_g\mid \boldsymbol{\beta})\,\Pi_{L,\beta}(d\boldsymbol{\beta}\mid \boldsymbol{Y}),\] we can separate the contribution from “good” coefficient vectors and “bad” ones. That is, for any set \(\mathcal{B}\subset [a,b]^n\), \[\begin{align} \Pi_L(T_g\mid \boldsymbol{Y}) &= \int_{\mathcal{B}} \Pi_L(T_g\mid \boldsymbol{\beta})\,\Pi_{L,\beta}(d\boldsymbol{\beta}\mid \boldsymbol{Y}) + \int_{\mathcal{B}^c} \Pi_L(T_g\mid \boldsymbol{\beta})\,\Pi_{L,\beta}(d\boldsymbol{\beta}\mid \boldsymbol{Y}) \nonumber\\ &\le \sup_{\boldsymbol{\beta}\in\mathcal{B}} \Pi_L(T_g\mid \boldsymbol{\beta}) + \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}\mid \boldsymbol{Y}). \label{eq:proof-decomp-pointwise} \end{align}\tag{11}\] Taking expectation under \(\mathbb{P}_{g_0}\) yields \[\label{eq:tower} \mathbb{E}_{g_0}\!\left[\Pi_L(T_g\mid \boldsymbol{Y})\right] \le \sup_{\boldsymbol{\beta}\in\mathcal{B}} \Pi_L(T_g\mid \boldsymbol{\beta}) + \mathbb{E}_{g_0}\!\left[\Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}\mid \boldsymbol{Y})\right].\tag{12}\] This is formalized in Appendix 7 as Lemma 6.

We choose \(\mathcal{B}=\mathcal{B}(M,L)\) to be the good-count event from Theorem 1, namely the set of vectors whose empirical bin counts are uniformly close to the corresponding probabilities under \(g_0\). For this choice, the first term in 12 is controlled directly by the direct-observation benchmark Theorem 1: \[\sup_{\boldsymbol{\beta}\in\mathcal{B}(M,L)} \Pi_L(T_g\mid \boldsymbol{\beta})\le C_1e^{-c_0L}.\] Therefore, the proof of Theorem 2 reduces to showing that the posterior for the latent coefficient vector \(\boldsymbol{\beta}\) places asymptotically negligible mass outside \(\mathcal{B}(M,L)\).

Step 2. From bin-count control to \(\ell_2\)-control of \(\boldsymbol{\beta}\). We next explain why \(\boldsymbol{\beta}\) membership in \(\mathcal{B}(M,L)\) can be enforced by showing that \(\boldsymbol{\beta}\) is sufficiently close to the true coefficient vector \(\boldsymbol{\beta}_0\) in \(\ell_2\).

For each cell \(I_k^l\), by the triangle inequality, \[\begin{align} \big|N_{\boldsymbol{\beta}}(I_k^l)-nP_0(I_k^l)\big| &\le \big|N_{\boldsymbol{\beta}}(I_k^l)-N_{\boldsymbol{\beta}_0}(I_k^l)\big| + \big|N_{\boldsymbol{\beta}_0}(I_k^l)-nP_0(I_k^l)\big|. \label{eq:count-triangle} \end{align}\tag{13}\] The second term is uniformly small over all cells whenever \(\boldsymbol{\beta}_0\in\mathcal{B}(M,L)\), by definition of \(\mathcal{B}(M,L)\). This event occurs with high \(\mathbb{P}_{g_0}\)-probability, by Theorem 1.

The new issue in the regression model is the first term in 13 , which measures how much the bin counts change when \(\boldsymbol{\beta}_0\) is replaced by a posterior draw \(\boldsymbol{\beta}\). A count can change only through coordinates \(j\) that cross a cell boundary when moving from \(\beta_{0j}\) to \(\beta_j\). The key observation is that such a crossing can occur only if either \(\beta_{0j}\) and \(\beta_j\) are sufficiently far apart, or if \(\beta_{0j}\) is close to a cell boundary, such that a small separation between \(\beta_{0j}\) and \(\beta_j\) can still cross a cell boundary.

Lemma 7 formalizes this observation by bounding a cell’s count discrepancy in terms of two quantities: the number of true coefficients lying near the relevant cell boundary, and the \(\ell_2\)-distance between \(\boldsymbol{\beta}\) and \(\boldsymbol{\beta}_0\). The first quantity is controlled uniformly over all cells and levels by Lemma 8, with high \(\mathbb{P}_{g_0}\)-probability. Together, these two lemmas show that posterior draws \(\boldsymbol{\beta}\) have the required empirical bin counts whenever they are sufficiently close to \(\boldsymbol{\beta}_0\) in \(\ell_2\). More precisely, the proof reduces the remaining task to showing that the posterior probability of \[\|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2 \gtrsim \frac{L^{3/2}}{n^{1/2}2^{3L/2}}\] vanishes. Thus, after the count-stability argument, it remains to prove posterior \(\ell_2\)-contraction of the latent coefficient vector \(\boldsymbol{\beta}\).

Step 3. Posterior contraction of the latent coefficients. The required \(\ell_2\)-contraction is provided by Proposition 4. It shows that, for suitable constants \(A,c>0\), \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2 > A\sigma \sqrt{ \frac{n+2^L\log(n+1)}{\lambda_{\min}(X^\top X)} } \;\middle|\; \boldsymbol{Y} \right) \right] \le 2e^{-cn}.\] The factor \(\lambda_{\min}(X^\top X)\) quantifies how well the design separates different coefficient vectors: better conditioning of \(X\) yields stronger recovery of \(\boldsymbol{\beta}_0\).

Since \(L\le L_{\mathrm{Cast}}\), the term \(2^L\log(n+1)\) is \(O(n)\), so the posterior \(\ell_2\)-radius is essentially \[\sigma\sqrt{\frac{n}{\lambda_{\min}(X^\top X)}}.\] The definition of \(L_{\mathrm{noise}}\) ensures that this radius is small enough relative to the count-stability threshold from Step 2. Indeed, \(L\le L_{\mathrm{noise}}\) implies \[\frac{\sigma^2 n}{\lambda_{\min}(X^\top X)} \lesssim \frac{L^{3/2}}{n^{1/2}2^{3L/2}}.\] Thus, under the truncation choice in Theorem 2, posterior draws of \(\boldsymbol{\beta}\) are close enough to \(\boldsymbol{\beta}_0\) to preserve the empirical bin count conditions required by the direct-observation benchmark.

Step 4. Conclusion. Combining the count-stability argument from Step 2 with the posterior \(\ell_2\)-contraction from Step 3 gives \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}(M,L)\mid\boldsymbol{Y}) \right]\longrightarrow 0.\] Together with the direct-observation bound from Step 1 and the decomposition 12 , this yields \[\mathbb{E}_{g_0}\!\left[ \Pi_L(T_g\mid\boldsymbol{Y}) \right]\longrightarrow 0.\] Since \(T_g=\{g:\|g-g_0\|_\infty>C_0\varepsilon_n(L)\}\), this is exactly the conclusion of Theorem 2.

Acknowledgments↩︎

A.W. was supported by the Israeli Science Foundation (ISF) under grant no. 2679/24.

6 Proof of Theorem 1↩︎

This appendix proves the direct-observation benchmark, Theorem 1. It follows the argument of [2], adapted to our setup and notation (support \([a,b]\), truncated Pólya tree, \(\mathrm{Beta}(1,1)\) at all levels). Throughout this appendix, set \[B:=b-a,\] and write \(C,c>0\) for finite positive constants whose values may change from line to line. We will also repeatedly use the consequence of the choice \(L\le L_{\mathrm{Cast}}\), that \[L2^L/n=o(1).\]

Because this appendix almost identically follows the structure of [2], we at times omit certain computational details that are unchanged from that work, and refer the reader to [2] for these details.

A.1. Haar basis on \([a,b]\)↩︎

Let \(T:[a,b]\to[0,1]\) be the affine map \(T(x)=(x-a)/B\). Let \[\phi=\mathbb{1}_{[0,1]}, \qquad \psi=-\mathbb{1}_{[0,1/2)}+\mathbb{1}_{[1/2,1]}\] on \([0,1]\). Define the rescaled Haar functions on \([a,b]\) by \[\varphi(x):=B^{-1/2}\mathbb{1}_{[a,b]}(x), \qquad \psi_{lk}(x):=B^{-1/2}2^{l/2}\psi\{2^lT(x)-k\}, \quad l\ge 0,\quad 0\le k<2^l .\] Then \[\{\varphi\}\cup\{\psi_{lk}:l\ge 0,\;0\le k<2^l\}\] is an orthonormal basis of \(L^2([a,b])\). For a square-integrable function \(f\), write \[f_{lk}:=\langle f,\psi_{lk}\rangle_2, \qquad f_\varphi:=\langle f,\varphi\rangle_2 .\] Since \(g_0\) is \(\alpha\)-Hölder on \([a,b]\), with \(\alpha\in(0,1]\), the Haar coefficients satisfy \[\label{eq:app-haar-holder-bound} \sup_{l\ge 0}\sup_{0\le k<2^l} \left(\frac{2^l}{B}\right)^{1/2+\alpha}|g_{0,lk}|<\infty ,\tag{14}\] which is the analogue on \([a,b]\) of the corresponding bound on \([0,1]\) recalled in [2].

For an integer \(L\ge 0\), and a function \(f\) in \(L^2([a,b])\), let \(f^{(L)}\) denote the \(L^2\)-projection of \(f\) onto the linear span of \(\{\varphi\}\cup\{\psi_{lk}:0\le l\le L-1,\;0\le k<2^l\}\). Denote \(f^{(L^c)}:=f-f^{(L)} .\)

This convention matches the depth-\(L\) truncated Pólya tree: a draw \(g\) from the prior is constant on the terminal cells \(I_k^L\) and hence satisfies \(g=g^{(L)}\).

A.2. Good bin-count set↩︎

For \(0\le l\le L\) and \(0\le k<2^l\), recall the notation \[N_{\boldsymbol{\beta}}(I_k^l):=\sum_{j=1}^n\mathbb{1}\{\beta_j\in I_k^l\}, \qquad P_0(I_k^l):=\int_{I_k^l}g_0(x)\,dx,\] \[\Lambda_n(l):=\sqrt{(l+L)n2^{-l}}.\] For \(M>0\), define \[\label{eq:app-B-def} \mathcal{B}(M,L) := \left\{ \boldsymbol{\beta}\in[a,b]^n: \left|N_{\boldsymbol{\beta}}(I_k^l)-nP_0(I_k^l)\right| \le M\{\Lambda_n(l)\vee(l+L)\} \;\text{for all }0\le l\le L,\;0\le k<2^l \right\}.\tag{15}\]

A.3. Auxiliary bounds↩︎

Let \(\Pi_L(\cdot\mid\boldsymbol{\beta})\) denote the Pólya tree posterior under direct observations \(\boldsymbol{\beta}\). Let \[\bar g:=\mathbb{E}_{\Pi_L}[g\mid\boldsymbol{\beta}]\] be the posterior mean density, and define \[\bar P(I_k^l):=\int_{I_k^l}\bar g(x)\,dx, \qquad P_0(I_k^l):=\int_{I_k^l}g_0(x)\,dx .\] For a parent cell \(I_k^l\), write \[\bar Y_{l,k}:= \frac{\bar P(I_{2k}^{l+1})}{\bar P(I_k^l)}, \qquad y_{l,k}:=\frac{P_0(I_{2k}^{l+1})}{P_0(I_k^l)} .\] The complementary right-child split proportions are \(1-\bar Y_{l,k}\) and \(1-y_{l,k}\).

A.4. Proof of Theorem 1↩︎

7 Proof of Theorem 2↩︎

We prove Theorem 2. Throughout this appendix, write \(C,c>0\) for finite positive constants whose values may change from line to line.

Fix \(C_0>0\), chosen large below, and set \[T_g := \{g:\|g-g_0\|_\infty > C_0\,\varepsilon_n(L)\}.\] As in the proof of Theorem 1, because \(L\to\infty\) and \(L\le L_{\mathrm{Cast}}\), \[B^\alpha 2^{-\alpha L}\to0, \qquad B^{-1}\sqrt{\frac{L2^L}{n}}\to0.\] Hence \(\varepsilon_n(L)\to0\).

For \(0\le l\le L\), write \[R_l:=\Lambda_n(l)\vee(l+L).\] For \(M>0\), define \[\mathcal{B}(M,L) := \left\{ \boldsymbol{\beta}\in[a,b]^n: \big|N_{\boldsymbol{\beta}}(I_k^l)-nP_0(I_k^l)\big| \le M R_l, \quad 0\le l\le L,\;0\le k<2^l \right\}.\] Choose \(M>0\) large enough that Theorem 1 applies with both \(M\) and \(M/2\). Define \[\mathcal{B}_0:=\mathcal{B}(M/2,L), \qquad \mathcal{B}_1:=\mathcal{B}(M,L).\]

By Theorem 1, there exist \(C,c>0\) such that \[\label{eq:appB-beta0-in-B0} \mathbb{P}_{g_0}\!\left(\boldsymbol{\beta}_0\in\mathcal{B}_0\right) \ge 1-Ce^{-cL},\tag{22}\] and, after increasing \(C_0\) if necessary, \[\label{eq:appB-direct-observation-on-B1} \sup_{\boldsymbol{\beta}\in\mathcal{B}_1} \Pi_L(T_g\mid \boldsymbol{\beta}) \le Ce^{-cL}.\tag{23}\]

B.1 Tower decomposition↩︎

Combining Lemma 6 with 23 , we obtain \[\mathbb{E}_{g_0}\!\left[\Pi_L(T_g\mid \boldsymbol{Y})\right] \le Ce^{-cL} + \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \right].\] Thus it remains to prove \[\label{eq:appB-goal-beta-B1} \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \right] \to 0.\tag{24}\]

B.2 Reducing failure of \(\mathcal{B}_1\) to bin-count instability↩︎

We work first on the event \(\{\boldsymbol{\beta}_0\in\mathcal{B}_0\}\), whose complement has \(\mathbb{P}_{g_0}\)-probability at most \(Ce^{-cL}\) by 22 . On this event, uniformly over \(0\le l\le L\) and \(0\le k<2^l\), \[\big|N_{\boldsymbol{\beta}_0}(I_k^l)-nP_0(I_k^l)\big| \le \frac{M}{2} R_l.\] If, in addition, \(\boldsymbol{\beta}\notin\mathcal{B}_1\), then for some \(l,k\), \[\big|N_{\boldsymbol{\beta}}(I_k^l)-nP_0(I_k^l)\big|>M R_l.\] Therefore, by the triangle inequality, \[\big|N_{\boldsymbol{\beta}}(I_k^l)-N_{\boldsymbol{\beta}_0}(I_k^l)\big| > \frac{M}{2} R_l.\] Define \[U := \bigcup_{l=0}^L\bigcup_{k<2^l} \left\{ \big|N_{\boldsymbol{\beta}}(I_k^l)-N_{\boldsymbol{\beta}_0}(I_k^l)\big|>\frac{M}{2}R_l \right\}.\] Then \[\Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \le \mathbb{1}_{\{\boldsymbol{\beta}_0\notin\mathcal{B}_0\}} + \Pi_{L,\beta}(U\mid \boldsymbol{Y}).\] Taking \(\mathbb{E}_{g_0}\) and using 22 , we obtain \[\label{eq:appB-reduce-to-U} \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \right] \le Ce^{-cL} + \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(U\mid \boldsymbol{Y}) \right].\tag{25}\]

B.3 A deterministic bin-count stability lemma↩︎

For an interval \(I\subset[a,b]\) and \(\tau>0\), define \[I^{+\tau}:=\{x\in[a,b]:\mathrm{dist}(x,I)\le \tau\}, \qquad I^{-\tau}:=\{x\in I:\mathrm{dist}(x,I^c)\ge \tau\}.\] Thus \(I^{+\tau}\setminus I^{-\tau}\) is the \(\tau\)-neighborhood of the boundary of \(I\), restricted to \([a,b]\).

Applying Lemma 7 with \(u=\boldsymbol{\beta}\), \(v=\boldsymbol{\beta}_0\), and \(I=I_k^l\), we obtain, for any choice of \(\tau_l>0\), \[\label{eq:appB-bin-stab-applied} \big|N_{\boldsymbol{\beta}}(I_k^l)-N_{\boldsymbol{\beta}_0}(I_k^l)\big| \le N_{\boldsymbol{\beta}_0}\!\left((I_k^l)^{+\tau_l}\setminus (I_k^l)^{-\tau_l}\right) + \frac{\|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2}{\tau_l^2}.\tag{26}\]

B.4 Controlling the boundary-strip counts↩︎

We now choose \[\label{eq:appB-tau-choice} \tau_l:=c_\tau\frac{R_l}{n}, \qquad 0\le l\le L,\tag{27}\] where \(c_\tau>0\) is a sufficiently small constant.

Define the good boundary-count event \[E_{\partial} := \left\{ N_{\boldsymbol{\beta}_0}\!\left((I_k^l)^{+\tau_l}\setminus (I_k^l)^{-\tau_l}\right) \le \frac{M}{4} R_l \quad \text{for all }0\le l\le L,\;0\le k<2^l \right\}.\]

B.5 Reduction to an \(\ell_2\)-posterior tail↩︎

We now control the term \(\Pi_{L,\beta}(U\mid \boldsymbol{Y})\) appearing in 25 . On the event \(E_{\partial}\), if \(U\) occurs, then for some \(0\le l\le L\) and \(0\le k<2^l\), \[\big|N_{\boldsymbol{\beta}}(I_k^l)-N_{\boldsymbol{\beta}_0}(I_k^l)\big|>\frac{M}{2}R_l.\] By 26 and the definition of \(E_{\partial}\), \[\frac{M}{2}R_l < \frac{M}{4}R_l + \frac{\|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2}{\tau_l^2}.\] Therefore \[\|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2 > \frac{M}{4}\tau_l^2R_l.\] Define \[\label{eq:appB-aL-def} a_L := \frac{M}{4} \min_{0\le l\le L} \left\{ \tau_l^2R_l \right\}.\tag{28}\] Then \[E_{\partial}\cap U \subseteq \left\{ \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2>a_L \right\}.\] Hence, \[\Pi_{L,\beta}(U\mid \boldsymbol{Y}) \le \mathbb{1}_{E_{\partial}^c} + \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2>a_L \;\middle|\; \boldsymbol{Y} \right).\] Taking \(\mathbb{E}_{g_0}\) and applying Lemma 8, we obtain \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(U\mid \boldsymbol{Y}) \right] \le Ce^{-cL} + \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2>a_L \;\middle|\; \boldsymbol{Y} \right) \right].\] Combining this with 25 , \[\label{eq:appB-beta-not-B1-bound} \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \right] \le Ce^{-cL} + \mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2>a_L \;\middle|\; \boldsymbol{Y} \right) \right].\tag{29}\]

It remains to lower bound \(a_L\). By 27 , \[\tau_l^2R_l = c_\tau^2\frac{R_l^3}{n^2}.\] Thus \[a_L \asymp \min_{0\le l\le L}\frac{R_l^3}{n^2}.\] Since \(R_l\ge \Lambda_n(l)\) and \[\Lambda_n(l) = \sqrt{(l+L)n2^{-l}},\] we have, for all \(0\le l\le L\), \[\Lambda_n(l) \gtrsim \sqrt{Ln2^{-L}}.\] Therefore \[R_l^3 \ge \Lambda_n(l)^3 \gtrsim (Ln2^{-L})^{3/2}.\] It follows that \[\label{eq:appB-aL-lower} a_L \gtrsim \frac{(Ln2^{-L})^{3/2}}{n^2} = \frac{L^{3/2}}{n^{1/2}2^{3L/2}}.\tag{30}\]

B.6 Latent-coefficient contraction and conclusion↩︎

We now invoke the posterior contraction result for the latent coefficient vector. Namely, by Proposition 4, for a sufficiently large constant \(A > 0\), \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2 > A \sigma \sqrt{\frac{n + 2^L\log(n+1)}{\lambda_{\min}(X^\top X)}} \;\middle|\; \boldsymbol{Y} \right) \right] \to 0.\]

Because \(L \leq L_{\mathrm{Cast}}\), \(2^L \log(n+1) = O(n)\). Thus, for \(A\) sufficiently large, \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}\!\left( \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2 > A \sigma \sqrt{\frac{n}{\lambda_{\min}(X^\top X)}} \;\middle|\; \boldsymbol{Y} \right) \right] \to 0.\]

By the definition of \(L_{\mathrm{noise}}\) and the fact that \(L\le L_{\mathrm{noise}}\), \[\frac{2^{3L/2}}{L^{3/2}} \le c_{\mathrm{noise}}\,\frac{\lambda_{\min}(X^\top X)}{\sigma^2 n^{3/2}}.\] Equivalently, \[\frac{\sigma^2 n}{\lambda_{\min}(X^\top X)} \le c_{\mathrm{noise}}\,\frac{L^{3/2}}{n^{1/2}2^{3L/2}}.\] Combining this with 30 , and choosing \(c_{\mathrm{noise}}>0\) sufficiently small relative to the constants in 30 and Proposition 4, we obtain \[A^2\sigma^2\frac{n}{\lambda_{\min}(X^\top X)} \le a_L.\] Therefore, \[\left\{ \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2^2>a_L \right\} \subseteq \left\{ \|\boldsymbol{\beta}-\boldsymbol{\beta}_0\|_2 > A\sigma\sqrt{\frac{n}{\lambda_{\min}(X^\top X)}} \right\}.\] Using Proposition 4 in 29 gives \[\mathbb{E}_{g_0}\!\left[ \Pi_{L,\beta}(\boldsymbol{\beta}\notin\mathcal{B}_1\mid \boldsymbol{Y}) \right] \longrightarrow 0.\] Together with the decomposition from Lemma 6 and the bound \[\sup_{\boldsymbol{\beta}\in\mathcal{B}_1}\Pi_L(T_g\mid \boldsymbol{\beta})\le Ce^{-cL},\] this yields \[\mathbb{E}_{g_0}\!\left[ \Pi_L(T_g\mid \boldsymbol{Y}) \right] \longrightarrow 0,\] which proves Theorem 2. 0◻

8 Posterior contraction of \(\boldsymbol{\beta}\)↩︎

This appendix proves the latent-coefficient contraction bound used in Appendix 7, and derives Corollary 1 on the \(\ell_2\) error of the posterior mean.

Recall that \(\Pi_{L,\beta}\) denotes the marginal prior distribution of \(\boldsymbol{\beta}=(\beta_1,\dots,\beta_n)\) induced by the level-\(L\) truncated Pólya tree prior on \(g\), after integrating out \(g\). Let \(\pi_{L,\beta}\) denote its Lebesgue density on \([a,b]^n\), \[\pi_{L,\beta}(\boldsymbol{\beta}) = \int \prod_{j=1}^n g(\beta_j)\,\Pi_L(dg), \qquad \boldsymbol{\beta}=(\beta_1,\dots,\beta_n)\in [a,b]^n.\]

9 Proof of Corollary 2↩︎

This appendix proves Corollary 2. We first prove a lemma that lower bounds \(\lambda_{\min}(X^\top X)\) with high probability under the random-design assumptions. We then use this lemma to reduce the random-design corollary to the fixed-design result of Theorem 2 on high-probability events under the design law.

References↩︎