Proof. We first prove the result for the case where \(\boldsymbol{\Sigma}\) is full-rank, and then extend it to the general case.

9.1.0.2 Case 1: Full-Rank \(\boldsymbol{\Sigma}\).

Note that \(\frac{1}{n}\tilde{\mathbf{X}}^\top \tilde{\mathbf{X}} = \mathbf{I}_d\). The optimization problem can be rewritten in the transformed space as: \[\begin{align} \min_{\theta} \left( \frac{1}{2n} \|\mathbf{Y}-\mathbf{X}\theta \|_2^2 + \lambda \|\theta -\beta \|_{\boldsymbol{\Sigma}} \right) &= \min_{\bar{\theta}} \left( \frac{1}{2n} \|\mathbf{Y}-\tilde{\mathbf{X}}\bar{\theta} \|_2^2 + \lambda \| \bar{\theta} - \bar{\beta} \|_2 \right). \end{align}\] By the first-order optimality condition for the \(\ell_2\)-regularized problem (or Lemma 13), the minimizer is exactly \(\bar{\beta}\) if \(\lambda > \|\nabla \bar{f}(\bar{\beta}) \|_2\).

We calculate the gradient norm at \(\bar{\beta}\): \[\begin{align} \|\nabla \bar{f}(\bar{\beta}) \|_2 &= \left\| \frac{1}{n}\tilde{\mathbf{X}}^\top (\tilde{\mathbf{X}} \bar{\beta} - \mathbf{Y}) \right\|_2 \\ &= \left\| \frac{1}{n}\tilde{\mathbf{X}}^\top (\tilde{\mathbf{X}} \bar{\beta} - \tilde{\mathbf{X}} \bar{\theta}^\star - \boldsymbol{\varepsilon}) \right\|_2 \\ &\leq \left\| \frac{1}{n}\tilde{\mathbf{X}}^\top \tilde{\mathbf{X}} (\bar{\beta} - \bar{\theta}^\star) \right\|_2 + \left\| \frac{1}{n}\tilde{\mathbf{X}}^\top \boldsymbol{\varepsilon} \right\|_2 \\ &= \|\bar{\beta} - \bar{\theta}^\star\|_2 + \left\| \frac{1}{n}\tilde{\mathbf{X}}^\top \boldsymbol{\varepsilon} \right\|_2. \end{align}\] The first term equals \(\|\beta - \theta^\star\|_{\boldsymbol{\Sigma}}\). For the second term, since the rows of \(\tilde{\mathbf{X}}\) are isotropic, the Hanson-Wright inequality (Lemma 16) implies that with probability at least \(1-\kappa\): \[\left\| \frac{1}{n}\tilde{\mathbf{X}}^\top \boldsymbol{\varepsilon} \right\|_2 \le c_0 \frac{\sqrt{d \log(1/\kappa)}}{\sqrt{n}}.\] Thus, the condition on \(\lambda\) ensures \(\bar{\beta}\) is the minimizer, implying \(\beta\) minimizes the original objective.

9.1.0.3 Case 2: Rank-Deficient \(\boldsymbol{\Sigma}\).

Let \(\mathcal{R}:=\mathrm{Range}(\boldsymbol{\Sigma})=\mathrm{row}(\mathbf{X})\), and let \(\mathbf{P}\) be the orthogonal projection onto \(\mathcal{R}\). Since \(\mathrm{Null}(\boldsymbol{\Sigma})=\mathrm{Null}(\mathbf{X})\), for every \(v\in\mathbb{R}^d\), \[\mathbf{X}v=\mathbf{X}\mathbf{P}v, \qquad \|v\|_{\boldsymbol{\Sigma}}=\|\mathbf{P}v\|_{\boldsymbol{\Sigma}}.\] Hence both the loss and the penalty in the linear objective depend on \(\theta\) only through \(\mathbf{P}\theta\), with \(\beta\) entering only through \(\mathbf{P}\beta\): \[f(\theta)=f(\mathbf{P}\theta), \qquad \|\theta-\beta\|_{\boldsymbol{\Sigma}}=\|\mathbf{P}\theta-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}.\] Because the linear case is unconstrained, the original problem is exactly equivalent to the reduced problem over \(u\in\mathcal{R}\), \[\min_{u\in\mathcal{R}}\left\{f(u)+\lambda\|u-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}\right\}.\] On \(\mathcal{R}\), the restricted operator \(\boldsymbol{\Sigma}|_{\mathcal{R}}\) is positive definite, so Case 1 applies to the reduced \(r=\mathrm{rank}(\boldsymbol{\Sigma})\)-dimensional problem with projected parameters \(\mathbf{P}\theta^\star\) and \(\mathbf{P}\beta\). The stochastic term is of order \(\sqrt{r/n}\), which is bounded by the displayed \(\sqrt{d/n}\) term. Since \(\|\mathbf{P}\beta-\mathbf{P}\theta^\star\|_{\boldsymbol{\Sigma}}=\|\beta-\theta^\star\|_{\boldsymbol{\Sigma}}\), the assumed lower bound on \(\lambda\) implies that \(\mathbf{P}\beta\) is a minimizer of the reduced problem. Therefore \(g(\beta)=f(\beta)\). Moreover, if \(\beta'\) is any minimizer of the original objective, then \(\mathbf{P}\beta'\) is a minimizer of the reduced problem, so \(\|\mathbf{P}\beta'-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}=0\). Equivalently, \(\|\beta'-\beta\|_{\boldsymbol{\Sigma}}=0\). ◻

Proof. If \(\boldsymbol{\Sigma}\) is rank-deficient, let \(\mathcal{R}:=\mathrm{Range}(\boldsymbol{\Sigma})=\mathrm{row}(\mathbf{X})\) and let \(\mathbf{P}\) be the orthogonal projection onto \(\mathcal{R}\). As in the previous proposition, \(f(\theta)=f(\mathbf{P}\theta)\) and \(\|\theta-\beta\|_{\boldsymbol{\Sigma}}=\|\mathbf{P}\theta-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}\). Thus the unconstrained problem reduces exactly to \(\mathcal{R}\), where \(\boldsymbol{\Sigma}|_{\mathcal{R}}\) is positive definite; the stochastic bound on this \(r=\mathrm{rank}(\boldsymbol{\Sigma})\)-dimensional space is no larger than the displayed \(\sqrt d\) bound. It therefore suffices to prove the result on the working subspace, and we write the proof as if \(\boldsymbol{\Sigma}\) were full-rank.

Fix an arbitrary \(\beta\in\mathbb{R}^d\), and let \(\hat{\theta}=\hat{\theta}(\beta)\). Let \(\tilde{\theta}\) be the minimizer of \(f(\theta)\) (OLS). The estimators in the transformed space, \(\bar{\hat{\theta}} := \boldsymbol{\Sigma}^{1/2}\hat{\theta}\) and \(\bar{\tilde{\theta}} := \boldsymbol{\Sigma}^{1/2}\tilde{\theta}\), are the minimizers of \[\begin{align} \min_{\bar{\theta}} \left( \bar{f}(\bar{\theta}) + \lambda \|\bar{\theta} - \bar{\beta}\|_2 \right) \quad \text{and} \quad \min_{\bar{\theta}} \bar{f}(\bar{\theta}), \end{align}\] respectively.

9.1.0.4 Step 1: Perturbation via Regularization.

Since \(\nabla^2 \bar{f}(\bar{\theta}) = \mathbf{I}_d\), the function \(\bar{f}\) is 1-strongly convex. The regularization term \(h(\bar{\theta}) := \lambda \|\bar{\theta} - \bar{\beta}\|_2\) is convex and \(\lambda\)-Lipschitz with respect to \(\|\cdot\|_2\). By Lemma 11, \[\|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2 \leq \lambda,\] which is equivalent to \(\|\hat{\theta} - \tilde{\theta}\|_{\boldsymbol{\Sigma}} \leq \lambda\).

9.1.0.5 Step 2: OLS Estimation Error.

The unregularized estimator \(\tilde{\theta}\) is the OLS solution, and \[\begin{align} \|\bar{\tilde{\theta}} - \bar{\theta}^\star\|_2 &\le \|\nabla \bar{f}(\bar{\theta}^\star)\|_2 = \left\| \frac{1}{n} \sum_{i=1}^n \varepsilon_i \tilde{x}_i \right\|_2, \end{align}\] where \(\tilde{x}_i = \boldsymbol{\Sigma}^{-1/2}x_i\) are isotropic. Applying the Hanson-Wright inequality (Lemma 16), with probability at least \(1-\kappa\): \[\|\bar{\tilde{\theta}} - \bar{\theta}^\star\|_2 \leq c_0 \sqrt{\frac{d}{n}\log\left(\frac{1}{\kappa}\right)}.\] This is equivalent to \(\|\tilde{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}} \le c_0 \sqrt{\frac{d}{n}\log(\frac{1}{\kappa})}\).

9.1.0.6 Conclusion.

Combining the two bounds yields \[\begin{align} \|\hat{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}} &\le \|\hat{\theta} - \tilde{\theta}\|_{\boldsymbol{\Sigma}} + \|\tilde{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}} \\ &\le \lambda + c_0 \sqrt{\frac{d}{n}\log\left(\frac{1}{\kappa}\right)}. \end{align}\] The only random event used above is the OLS noise event in Step 2, which does not depend on \(\beta\). Hence the bound holds simultaneously for every \(\beta\) and every corresponding minimizer \(\hat{\theta}(\beta)\). ◻

Proof. We first consider the case where \(\boldsymbol{\Sigma}\) is full-rank. The original optimization problem is equivalent to the transformed problem: \[\begin{align} \beta \in \mathop{\arg \min}_{\theta \in \mathbf{C}} \left( f(\theta) + \lambda \|\theta -\beta \|_{\boldsymbol{\Sigma}} \right) \iff \bar{\beta} \in \mathop{\arg \min}_{\bar{\theta} \in \bar{\mathbf{C}}} \left( \bar{f}(\bar{\theta}) + \lambda \|\bar{\theta} -\bar{\beta} \|_{2} \right). \end{align}\] Since \(\bar{f}\) is convex on \(\bar{\mathbf{C}}\) and \(\bar\beta\in\bar{\mathbf{C}}\), by Lemma 14, it suffices to show that \(\|\nabla \bar{f}(\bar{\beta})\|_2 < \lambda\).

We compute the gradient \(\nabla \bar{f}(\bar{\beta})\): \[\begin{align} \nabla \bar{f}(\bar{\beta}) &= \frac{1}{n}\sum_{i=1}^n \left( -y_i \tilde{x}_i + \psi'(\tilde{x}_i^\top \bar{\beta})\tilde{x}_i \right). \end{align}\] Substituting the model equation \(y_i = \psi'(\tilde{x}_i^\top \bar{\theta}^\star) + \varepsilon_i\), we decompose the gradient into a noise term and a bias term: \[\begin{align} \nabla \bar{f}(\bar{\beta}) &= -\frac{1}{n}\sum_{i=1}^n \varepsilon_i \tilde{x}_i + \frac{1}{n}\sum_{i=1}^n \left( \psi'(\tilde{x}_i^\top \bar{\beta}) - \psi'(\tilde{x}_i^\top \bar{\theta}^\star) \right) \tilde{x}_i. \end{align}\] By the Mean Value Theorem, there exists \(\xi_i\) between \(\tilde{x}_i^\top \bar{\beta}\) and \(\tilde{x}_i^\top \bar{\theta}^\star\) such that: \[\psi'(\tilde{x}_i^\top \bar{\beta}) - \psi'(\tilde{x}_i^\top \bar{\theta}^\star) = \psi''(\xi_i) \tilde{x}_i^\top (\bar{\beta} - \bar{\theta}^\star).\] Substituting this back, we obtain: \[\begin{align} \nabla \bar{f}(\bar{\beta}) &= -\frac{1}{n}\sum_{i=1}^n \varepsilon_i \tilde{x}_i + \left( \frac{1}{n}\sum_{i=1}^n \psi''(\xi_i) \tilde{x}_i \tilde{x}_i^\top \right) (\bar{\beta} - \bar{\theta}^\star). \end{align}\] Let \(\tilde{\mathbf{Q}} := \frac{1}{n}\sum_{i=1}^n \psi''(\xi_i) \tilde{x}_i \tilde{x}_i^\top\). Under Assumption 2, we have \(\psi''(\cdot) \in [\alpha_\ell, \alpha_u]\). Since \(\frac{1}{n}\sum_{i=1}^n \tilde{x}_i \tilde{x}_i^\top = \mathbf{I}_d\), the matrix \(\tilde{\mathbf{Q}}\) satisfies: \[\alpha_\ell \mathbf{I}_d \preceq \tilde{\mathbf{Q}} \preceq \alpha_u \mathbf{I}_d \implies \|\tilde{\mathbf{Q}}\|_{\operatorname{op}} \le \alpha_u.\]

We now bound the norm of the gradient. Using the triangle inequality: \[\begin{align} \|\nabla \bar{f}(\bar{\beta})\|_2 &\leq \left\| \frac{1}{n}\sum_{i=1}^n \varepsilon_i \tilde{x}_i \right\|_2 + \|\tilde{\mathbf{Q}}\|_{\operatorname{op}} \|\bar{\beta} - \bar{\theta}^\star\|_2 \\ &\leq \left\| \frac{1}{n}\sum_{i=1}^n \varepsilon_i \tilde{x}_i \right\|_2 + \alpha_u \|\beta - \theta^\star\|_{\boldsymbol{\Sigma}}. \end{align}\] Since the whitened covariates \(\tilde{x}_i\) are isotropic, we apply the Hanson-Wright inequality (Lemma 16). With probability at least \(1-\kappa\): \[\left\| \frac{1}{n}\sum_{i=1}^n \varepsilon_i \tilde{x}_i \right\|_2 \leq c_0 \frac{\sqrt{d \log(1/\kappa)}}{\sqrt{n}}.\] Thus, provided that \(\lambda > \alpha_u \|\beta - \theta^\star\|_{\boldsymbol{\Sigma}} + c_0 \frac{\sqrt{d \log(1/\kappa)}}{\sqrt{n}}\), the condition \(\|\nabla \bar{f}(\bar{\beta})\|_2 < \lambda\) holds, implying \(\beta\) is a minimizer over \(\mathbf{C}\).

For the case where \(\boldsymbol{\Sigma}\) is rank-deficient, let \(\mathcal{R}:=\mathrm{Range}(\boldsymbol{\Sigma})=\mathrm{row}(\mathbf{X})\), and let \(\mathbf{P}\) be the orthogonal projection onto \(\mathcal{R}\). As above, \(\mathrm{Null}(\boldsymbol{\Sigma})=\mathrm{Null}(\mathbf{X})\), so the GLM linear predictors and the seminorm are unchanged by projection: \[x_i^\top\theta=x_i^\top\mathbf{P}\theta, \qquad \|\theta-\beta\|_{\boldsymbol{\Sigma}} = \|\mathbf{P}\theta-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}.\] The constrained domain causes no further difficulty. Here \(\mathbf{C}=\mathbf{B}(0,\xi)\), and orthogonal projections are contractions; hence \[\mathbf{P}\mathbf{C}=\mathbf{C}\cap\mathcal{R}\subseteq\mathbf{C}.\] Thus replacing any feasible \(\theta\) by \(\mathbf{P}\theta\) preserves all GLM linear predictors and the \(\boldsymbol{\Sigma}\)-seminorm penalty while keeping the parameter feasible. The original constrained problem is therefore equivalent, in objective value and minimizer projections, to the reduced problem over \(u\in\mathbf{P}\mathbf{C}=\mathbf{C}\cap\mathcal{R}\), with \(\beta\) and \(\theta^\star\) replaced by \(\mathbf{P}\beta\) and \(\mathbf{P}\theta^\star\). Because the reduced feasible set remains inside \(\mathbf{C}\), the curvature condition in Assumption 2 is valid throughout the reduced problem. On \(\mathcal{R}\), the restricted operator \(\boldsymbol{\Sigma}|_{\mathcal{R}}\) is positive definite, so the full-rank argument applies in dimension \(r=\mathrm{rank}(\boldsymbol{\Sigma})\), with the stochastic \(\sqrt r\) term bounded by the displayed \(\sqrt d\) term. Since \(\|\mathbf{P}\beta-\mathbf{P}\theta^\star\|_{\boldsymbol{\Sigma}}=\|\beta-\theta^\star\|_{\boldsymbol{\Sigma}}\), the same lower bound on \(\lambda\) implies that \(\mathbf{P}\beta\) minimizes the reduced problem. Equivalently, \(\beta\) is a minimizer of the original constrained problem, and any minimizer \(\beta'\) satisfies \(\|\mathbf{P}\beta'-\mathbf{P}\beta\|_{\boldsymbol{\Sigma}}=0\), i.e., \(\|\beta'-\beta\|_{\boldsymbol{\Sigma}}=0\). ◻

Proof. First, we establish the uniform lower bound on the Hessian. The Hessian of the transformed loss is given by: \[\nabla^2 \bar{f}(\bar{\theta}) = \frac{1}{n}\sum_{i=1}^n \psi''(\tilde{x}_i^\top \bar{\theta}) \tilde{x}_i \tilde{x}_i^\top.\] Under Assumption 2, the second derivative of the cumulant generating function satisfies \(\psi''(z) \ge \alpha_\ell\) for all \(z\) in the domain. Additionally, by the construction of the whitened covariates \(\tilde{x}_i = \boldsymbol{\Sigma}^{-1/2}x_i\), the empirical covariance is isotropic: \[\frac{1}{n}\sum_{i=1}^n \tilde{x}_i \tilde{x}_i^\top = \boldsymbol{\Sigma}^{-1/2} \left( \frac{1}{n}\sum_{i=1}^n x_i x_i^\top \right) \boldsymbol{\Sigma}^{-1/2} = \boldsymbol{\Sigma}^{-1/2} \boldsymbol{\Sigma}\boldsymbol{\Sigma}^{-1/2} = \mathbf{I}_d.\] This implies \(\nabla^2 \bar{f}(\bar{\theta}) \succeq \alpha_\ell \mathbf{I}_d\) for all \(\bar{\theta} \in \bar{\mathbf{C}}\), establishing \(\alpha_\ell\)-strong convexity.

Now, applying the definition of strong convexity at the minimizer \(\bar{\tilde{\theta}}\): \[\bar{f}(\bar{\theta}) \ge \bar{f}(\bar{\tilde{\theta}}) + \nabla \bar{f}(\bar{\tilde{\theta}})^\top (\bar{\theta} - \bar{\tilde{\theta}}) + \frac{\alpha_\ell}{2}\|\bar{\theta} - \bar{\tilde{\theta}}\|_2^2.\] Since \(\bar{\mathbf{C}}\) is a convex set and \(\bar{\tilde{\theta}}\) is the minimizer over \(\bar{\mathbf{C}}\), the first-order optimality condition states that \(\langle \nabla \bar{f}(\bar{\tilde{\theta}}), \bar{\theta} - \bar{\tilde{\theta}} \rangle \ge 0\) for all \(\bar{\theta} \in \bar{\mathbf{C}}\). Dropping the non-negative linear term yields the desired result: \[\bar{f}(\bar{\theta}) \ge \bar{f}(\bar{\tilde{\theta}}) + \frac{\alpha_\ell}{2}\|\bar{\theta} - \bar{\tilde{\theta}}\|_2^2.\] ◻

Proof. If \(\boldsymbol{\Sigma}\) is rank-deficient, let \(\mathcal{R}:=\mathrm{Range}(\boldsymbol{\Sigma})=\mathrm{row}(\mathbf{X})\), and let \(\mathbf{P}\) be the orthogonal projection onto \(\mathcal{R}\). For any \(\theta\in\mathbf{C}\), the GLM loss and the matrix seminorm depend only on \(\mathbf{P}\theta\), and the projected feasible set is \[\mathbf{P}\mathbf{C}=\mathbf{C}\cap\mathcal{R}\subseteq\mathbf{C},\] because \(\mathbf{C}=\mathbf{B}(0,\xi)\). Therefore the regularized and unregularized constrained problems reduce exactly to \(\mathbf{P}\mathbf{C}\) on \(\mathcal{R}\), with \(\beta\) replaced by \(\mathbf{P}\beta\). The curvature bounds remain valid on this reduced feasible set, and \(\boldsymbol{\Sigma}|_{\mathcal{R}}\) is positive definite. The proof below applies on \(\mathcal{R}\); converting back gives the same \(\boldsymbol{\Sigma}\)-seminorm bounds, and the stochastic term is controlled by \(\sqrt{\mathrm{rank}(\boldsymbol{\Sigma})}\le\sqrt d\). We therefore write the argument in full-rank notation.

We proceed by analyzing the problem in the whitened domain. Let \(\bar{\theta} := \boldsymbol{\Sigma}^{1/2}\theta\) and define the transformed domain \(\bar{\mathbf{C}} := \{ \boldsymbol{\Sigma}^{1/2}\theta \mid \theta \in \mathbf{C}\}\). The transformed loss function is given by \(\bar{f}(\bar{\theta}) := f(\boldsymbol{\Sigma}^{-1/2}\bar{\theta})\).

Let \(\bar{\hat{\theta}} = \boldsymbol{\Sigma}^{1/2}\hat{\theta}\) and \(\bar{\tilde{\theta}} = \boldsymbol{\Sigma}^{1/2}\tilde{\theta}\) be the minimizers in \(\bar{\mathbf{C}}\) for the regularized and unregularized problems, respectively.

9.2.0.2 Step 1: Lower Bound via Strong Convexity.

By Lemma 1, \(\bar{f}\) is \(\alpha_\ell\)-strongly convex on \(\bar{\mathbf{C}}\). Using the property of strongly convex functions and the first-order optimality condition for \(\bar{\tilde{\theta}}\) (i.e., \(\langle \nabla \bar{f}(\bar{\tilde{\theta}}), \bar{\theta} - \bar{\tilde{\theta}} \rangle \ge 0\) for all \(\bar{\theta} \in \bar{\mathbf{C}}\)), we have: \[\begin{align} \bar{f}(\bar{\hat{\theta}}) - \bar{f}(\bar{\tilde{\theta}}) &\ge \langle \nabla \bar{f}(\bar{\tilde{\theta}}), \bar{\hat{\theta}} - \bar{\tilde{\theta}} \rangle + \frac{\alpha_\ell}{2} \|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2^2 \\ &\ge \frac{\alpha_\ell}{2} \|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2^2. \end{align}\]

9.2.0.3 Step 2: Upper Bound via Regularization.

From the optimality of \(\bar{\hat{\theta}}\) for the regularized objective: \[\bar{f}(\bar{\hat{\theta}}) + \lambda \|\bar{\hat{\theta}} - \bar{\beta}\|_2 \le \bar{f}(\bar{\tilde{\theta}}) + \lambda \|\bar{\tilde{\theta}} - \bar{\beta}\|_2.\] Rearranging terms and applying the reverse triangle inequality: \[\begin{align} \bar{f}(\bar{\hat{\theta}}) - \bar{f}(\bar{\tilde{\theta}}) &\le \lambda \left( \|\bar{\tilde{\theta}} - \bar{\beta}\|_2 - \|\bar{\hat{\theta}} - \bar{\beta}\|_2 \right) \\ &\le \lambda \|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2. \end{align}\]

9.2.0.4 Step 3: Combining Bounds.

Combining the inequalities from Steps 1 and 2: \[\frac{\alpha_\ell}{2} \|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2^2 \le \lambda \|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2.\] Assuming \(\|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2 > 0\) (otherwise the bound holds trivially), we divide by the norm to obtain: \[\|\bar{\hat{\theta}} - \bar{\tilde{\theta}}\|_2 \le \frac{2\lambda}{\alpha_\ell}.\] Converting back to the original space using \(\|\bar{v}\|_2 = \|v\|_{\boldsymbol{\Sigma}}\), we get \(\|\hat{\theta} - \tilde{\theta}\|_{\boldsymbol{\Sigma}} \le \frac{2\lambda}{\alpha_\ell}\).

9.2.0.5 Step 4: Final Error Bound.

The unregularized GLM estimator \(\tilde{\theta}\) satisfies the standard concentration bound (derived from the \(\alpha_\ell\)-strong convexity and Hanson-Wright inequality on the gradient): \[\|\tilde{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}} \le \frac{1}{\alpha_\ell} \|\nabla \bar{f}(\bar{\theta}^\star)\|_2 \le \frac{c_0}{\alpha_\ell} \sqrt{\frac{d \log(2/\kappa)}{n}}.\] Applying the triangle inequality \(\|\hat{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}} \le \|\hat{\theta} - \tilde{\theta}\|_{\boldsymbol{\Sigma}} + \|\tilde{\theta} - \theta^\star\|_{\boldsymbol{\Sigma}}\) yields the final result. ◻

Proof. Recall the closed-form solution for the pooling estimator: \[\hat{\beta}_{F_{\mathbf{S}}} = \mathbf{V}_{\mathbf{S}}^{-1} \sum_{k \in \mathbf{S}} \mathbf{X}_k^\top \mathbf{Y}_k = \theta^\star + \mathbf{V}_{\mathbf{S}}^{-1} \sum_{k \in \mathbf{S}} \mathbf{X}_k^\top (\mathbf{X}_k \eta_k^\star + \boldsymbol{\varepsilon}_k).\] We analyze the error in the \(\boldsymbol{\Sigma}_j\)-norm, decomposing it into a bias term and a variance term: \[\begin{align} \|\hat{\beta}_{F_{\mathbf{S}}} - \theta^\star\|_{\boldsymbol{\Sigma}_j} &\le \underbrace{\left\| \mathbf{V}_{\mathbf{S}}^{-1} \sum_{k \in \mathbf{S}} \mathbf{V}_k \eta_k^\star \right\|_{\boldsymbol{\Sigma}_j}}_{\text{Bias}} + \underbrace{\left\| \mathbf{V}_{\mathbf{S}}^{-1} \sum_{k \in \mathbf{S}} \mathbf{X}_{k}^\top \boldsymbol{\varepsilon}_k \right\|_{\boldsymbol{\Sigma}_j}}_{\text{Variance}}. \end{align}\]

10.2.0.1 Bias Term.

Using the balancedness assumption \(\boldsymbol{\Sigma}_j \preceq B \boldsymbol{\Sigma}_{\mathbf{S}} = \frac{B}{N_\mathbf{S}} \mathbf{V}_{\mathbf{S}}\), we have \(\boldsymbol{\Sigma}_j^{1/2} \mathbf{V}_\mathbf{S}^{-1} \boldsymbol{\Sigma}_j^{1/2} \preceq \frac{B}{N_\mathbf{S}} \mathbf{I}\). Let \(\mathbf{e}\) be the stacked vector of \(\{ \mathbf{X}_k \eta_k^\star \}_{k \in \mathbf{S}} \in \mathbb{R}^{N_\mathbf{S}}\). Then \[(\text{Bias})^2 \le \frac{B}{N_\mathbf{S}} \mathbf{e}^\top \mathbf{X}_\mathbf{S}(\mathbf{X}_\mathbf{S}^\top \mathbf{X}_\mathbf{S})^{-1} \mathbf{X}_\mathbf{S}^\top \mathbf{e}\le \frac{B}{N_\mathbf{S}} \|\mathbf{e}\|_2^2.\] Since \(\|\mathbf{e}\|_2^2 \le N_\mathbf{S}\delta^2\), we obtain \(\text{Bias} \le \sqrt{B} \delta\).

10.2.0.2 Variance Term.

Let \(\mathbf{X}_\mathbf{S}\) be the stacked inlier design and let \(\boldsymbol{\varepsilon}_\mathbf{S}\) be the stacked inlier noise vector. The squared variance term is the quadratic form \[\boldsymbol{\varepsilon}_\mathbf{S}^\top \mathbf{X}_\mathbf{S}\mathbf{V}_\mathbf{S}^{-1}\boldsymbol{\Sigma}_j\mathbf{V}_\mathbf{S}^{-1}\mathbf{X}_\mathbf{S}^\top \boldsymbol{\varepsilon}_\mathbf{S}.\] Since \(\mathbf{V}_\mathbf{S}=N_\mathbf{S}\boldsymbol{\Sigma}_\mathbf{S}\), Assumption 1 gives \[\mathbf{V}_\mathbf{S}^{-1/2}\boldsymbol{\Sigma}_j\mathbf{V}_\mathbf{S}^{-1/2}\preceq \frac{B}{N_\mathbf{S}}\mathbf{I}.\] Thus the quadratic-form matrix has trace at most \(Bd/N_\mathbf{S}\) and operator norm at most \(B/N_\mathbf{S}\). Applying the Hanson-Wright inequality (Lemma 16) with failure level \(\kappa/(4m)\) for each task \(j\) gives, on a union-bound event of probability at least \(1-\kappa/4\), \[\text{Variance} \le c_0 \sqrt{\frac{B d\zeta}{N_\mathbf{S}}}.\] Combining both terms yields the result. ◻

Proof. Let \(\mathcal{E}_{\mathrm{pool}}\) be the pooling-estimator event from Lemma 2; it has probability at least \(1-\kappa/4\). Let \(\mathcal{E}_{\mathrm{inv}}\) be the event that the taskwise noise bound used in Proposition 1 holds for every \(j\in\mathbf{S}\), each invoked with failure level \(\kappa/(4m)\). By a union bound, \(\mathbb{P}(\mathcal{E}_{\mathrm{inv}})\ge 1-\kappa/4\), and hence \(\mathbb{P}(\mathcal{E}_{\mathrm{pool}}\cap\mathcal{E}_{\mathrm{inv}})\ge1-\kappa/2\). On this intersection, a sufficient condition for \(\beta \in R_j\) is \(\lambda > \|\theta_j^\star - \beta\|_{\boldsymbol{\Sigma}_j} + c_0 \sqrt{\frac{d \zeta}{n}}\). Substituting \(\beta = \hat{\beta}_{F_{\mathbf{S}}}\): \[\begin{align} \|\theta_j^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} &\leq \|\theta_j^\star - \theta^\star\|_{\boldsymbol{\Sigma}_j} + \|\theta^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} \\ &\leq \delta + \left( \sqrt{B}\delta + c_0 \sqrt{\frac{B d\zeta}{N_\mathbf{S}}} \right), \end{align}\] where we used Lemma 2. The assumed lower bound on \(\lambda\) ensures this condition holds strictly for all \(j \in \mathbf{S}\). Thus, \(\hat{\beta}_{F_{\mathbf{S}}}\) lies in the invariant region of every task \(j \in \mathbf{S}\). For each \(j\in\mathbf{S}\), define the positive slack \[\Delta_j := \lambda-\left(\|\theta_j^\star-\hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n}}\right) >0.\] Since \(\beta\mapsto \|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j}\) is continuous and the stochastic term in Proposition 1 does not depend on \(\beta\), there exists \(r_j>0\) such that \[\|\beta-\hat{\beta}_{F_{\mathbf{S}}}\|_2<r_j \quad\Longrightarrow\quad \|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j}+c_0\sqrt{\frac{d\zeta}{n}}<\lambda.\] Hence Proposition 1 implies \(f_j(\beta)=g_j(\beta)\) throughout the Euclidean ball \(B_{r_j}(\hat{\beta}_{F_{\mathbf{S}}})\). Let \(r:=\min_{j\in\mathbf{S}} r_j>0\). Then \[F_\mathbf{S}(\beta)=G_\mathbf{S}(\beta) \qquad\text{for all }\beta\in B_r(\hat{\beta}_{F_{\mathbf{S}}}).\] On \(\mathrm{Range}(\boldsymbol{\Sigma}_\mathbf{S})\), \(F_\mathbf{S}\) is strictly convex, so \(\hat{\beta}_{F_{\mathbf{S}}}\) is its unique minimizer in the prediction-relevant subspace and therefore a local minimizer of \(G_\mathbf{S}\) as well. Finally, \(G_\mathbf{S}\) is convex as a sum of infimal convolutions of convex functions, so any local minimizer is global. Hence \(\hat{\beta}_{F_{\mathbf{S}}}\) minimizes \(G_\mathbf{S}\). ◻

Proof. Under the stated event, \(\hat{\beta}_{G_\mathbf{S}}=\hat{\beta}_{F_\mathbf{S}}\). Moreover, by Lemma 2 and the assumed lower bound on \(\lambda\), for every \(j\in\mathbf{S}\), \[\|\theta_j^\star-\hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n}} \le (\sqrt{B}+1)\delta + c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}} + c_0\sqrt{\frac{d\zeta}{n}} < \frac{\lambda}{2}.\] If \(\|\bar{\beta}-\bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \le \lambda/(2\sqrt{B})\), then Assumption 1 gives \[\|\beta-\hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} \le \sqrt{B}\|\beta-\hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_\mathbf{S}} = \sqrt{B}\|\bar{\beta}-\bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \le \frac{\lambda}{2}.\] Hence \[\|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n}} < \lambda,\] so Proposition 1 implies \(\beta\in R_j\) for all \(j\in\mathbf{S}\). Since the stochastic term in Proposition 1 is precisely the taskwise noise bound included in the conditioning event and does not depend on \(\beta\), this implication holds simultaneously throughout the displayed neighborhood.

Therefore \(G_\mathbf{S}\) and \(F_\mathbf{S}\) agree throughout this neighborhood. For the linear loss, \[\nabla^2 \bar{f}_j(\bar{\beta}) = \boldsymbol{\Sigma}_\mathbf{S}^{-1/2}\boldsymbol{\Sigma}_j\boldsymbol{\Sigma}_\mathbf{S}^{-1/2},\] which is independent of \(\bar{\beta}\). Summing over \(j\in\mathbf{S}\) and using \(\boldsymbol{\Sigma}_\mathbf{S}=|\mathbf{S}|^{-1}\sum_{j\in\mathbf{S}}\boldsymbol{\Sigma}_j\) yields \[\nabla^2 \bar{G}_\mathbf{S}(\bar{\beta}) = \nabla^2 \bar{F}_\mathbf{S}(\bar{\beta}) = \sum_{j\in\mathbf{S}}\boldsymbol{\Sigma}_\mathbf{S}^{-1/2}\boldsymbol{\Sigma}_j\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} = |\mathbf{S}|\mathbf{I}_d.\] ◻

Proof. Using the pooled whitening notation introduced above, let \(\bar{\hat{\beta}}_{G} := \boldsymbol{\Sigma}_{\mathbf{S}}^{1/2}\hat{\beta}_{G}\) and \(\bar{\hat{\beta}}_{G_\mathbf{S}} := \boldsymbol{\Sigma}_{\mathbf{S}}^{1/2}\hat{\beta}_{G_{\mathbf{S}}}\). Minimizing \(G\) is equivalent to minimizing \(\bar{G} = \bar{G}_\mathbf{S}+ \bar{L}\).

10.3.0.1 Step 1: Pooling Range and Strong Convexity.

Let \(\mathcal{E}_{\text{pool}}\) denote the event where Lemma 2, Proposition 5, and the taskwise invariant-region noise bounds hold. By the failure-budget convention above, the pooling-estimator event is allocated failure \(\kappa/4\) and the taskwise noise events are allocated total failure \(\kappa/4\), so \(\mathbb{P}(\mathcal{E}_{\text{pool}})\ge 1-\kappa/2\ge 1-\kappa\). By Proposition 5, \(\hat{\beta}_{F_\mathbf{S}} = \hat{\beta}_{G_\mathbf{S}}\) on this event. Moreover, Claim 1 shows that for all \(\beta\) in the neighborhood \[\|\bar{\beta} - \bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \leq \frac{\lambda}{2\sqrt{B}},\] the transformed inlier objective has exact quadratic curvature: \[\nabla^2 \bar{G}_\mathbf{S}(\bar{\beta}) = \nabla^2 \bar{F}_\mathbf{S}(\bar{\beta}) = |\mathbf{S}| \mathbf{I}_d.\]

10.3.0.2 Step 2: Lipschitz Continuity of Outliers.

Since \(\varepsilon < \frac{1}{3B} \le \frac{1}{2B+1}\), Lemma 3 ensures that \[\operatorname{Lip}(\bar{L}) \leq \sum_{j \in \mathbf{S}^c} \operatorname{Lip}(\bar{g}_j) \le m\varepsilon \sqrt{B} \lambda.\] We apply Lemma 11 with parameters \(r = \frac{\lambda}{2\sqrt{B}}\), \(\rho = |\mathbf{S}|\), and \(\lambda' = m\varepsilon \sqrt{B}\lambda\). Since \(m\varepsilon \sqrt{B} \lambda < (1-\varepsilon)m \frac{\lambda}{2\sqrt{B}} \le \rho r\), this yields \[\|\bar{\hat{\beta}}_{G} - \bar{\hat{\beta}}_{G_\mathbf{S}} \|_2 \leq \frac{m\varepsilon \sqrt{B}\lambda}{|\mathbf{S}|} \leq \frac{\varepsilon}{1-\varepsilon} \sqrt{B} \lambda.\] Translating back to the original domain, for any \(j \in \mathbf{S}\), \[\|\hat{\beta}_{G} - \hat{\beta}_{G_\mathbf{S}} \|_{\boldsymbol{\Sigma}_j} \leq \frac{\varepsilon}{1-\varepsilon} B \lambda.\]

10.3.0.3 Step 3: Total Error Bound.

Combining this perturbation bound with Lemma 2, on \(\mathcal{E}_{\text{pool}}\) we obtain \[\begin{align} \label{equation5932error32bound32linear} \|\hat{\beta}_{G} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} &\leq \|\hat{\beta}_{G_{\mathbf{S}}} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} + \|\hat{\beta}_{G} - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_j} \nonumber \\ &\leq (\sqrt{B}+1)\delta + c_0 \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{\varepsilon}{1-\varepsilon} B \lambda \nonumber \\ &\leq (\sqrt{B}+1)\delta + c_0 \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{1}{2}\lambda. \end{align}\tag{8}\]

10.3.0.4 Step 4: Invariant Region Check.

To verify that \(\hat{\beta}_G \in R_j\), Proposition 1 requires \[\|\hat{\beta}_{G} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} + c_0 \sqrt{\frac{d \zeta}{n}} < \lambda.\] Substituting 8 , the left-hand side is bounded by \[(\sqrt{B}+1)\delta + c_0 \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{1}{2}\lambda + c_0 \sqrt{\frac{d \zeta}{n}},\] which is strictly smaller than \(\lambda\) by the assumed lower bound on \(\lambda\). Hence \(\hat{\beta}_G \in R_j\) for all \(j \in \mathbf{S}\). By Proposition 1, every corresponding taskwise minimizer satisfies \(\|\hat{\theta}_j-\hat{\beta}_G\|_{\boldsymbol{\Sigma}_j}=0\), and therefore \(\|\hat{\theta}_j-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}=\|\hat{\beta}_G-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}\). ◻

Proof. Let \[M_j:=\boldsymbol{\Sigma}_{\mathbf{S}}^{-1/2}\boldsymbol{\Sigma}_j\boldsymbol{\Sigma}_{\mathbf{S}}^{-1/2}.\] In pooled-whitened coordinates, \[\bar g_j(w) = \inf_z\left\{\bar f_j(z)+\lambda_j\|z-w\|_{M_j}\right\},\] where the infimum is over the pooled-whitened subspace. By Assumption 1, \(M_j\preceq B\mathbf{I}_d\). Fix \(w,w'\) and \(\eta>0\), and choose \(z_\eta\) such that \[\bar g_j(w')+\eta > \bar f_j(z_\eta)+\lambda_j\|z_\eta-w'\|_{M_j}.\] Then \[\begin{align} \bar g_j(w)-\bar g_j(w') &\le \lambda_j\bigl(\|z_\eta-w\|_{M_j}-\|z_\eta-w'\|_{M_j}\bigr)+\eta \\ &\le \lambda_j\|w-w'\|_{M_j}+\eta \le \sqrt{B}\lambda_j\|w-w'\|_2+\eta. \end{align}\] Letting \(\eta\downarrow0\) and exchanging \(w,w'\) proves the claim. ◻

Proof. Let \(\mathcal{E}_{\mathrm{safe}}\) be the event obtained by applying the uniform form of Proposition 2 to each task with failure probability \(\kappa/(2m)\) and taking a union bound. Then \(\mathbb{P}(\mathcal{E}_{\mathrm{safe}})\ge1-\kappa/2\), and on this event, with \(\lambda=q\sqrt{d\zeta/n}\) and \(q\gtrsim1\), for every task \(j\in[m]\), \[\|\hat{\theta}_j-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \frac{d\zeta}{n}+\lambda^2 \lesssim q^2\frac{d\zeta}{n}.\] This proves part (a), including all outlier tasks, without any balancedness or relatedness assumption.

It remains to prove the sharper inlier bound in part (b). We analyze the estimator in two regimes depending on the magnitude of the task heterogeneity \(\delta\) relative to the regularization parameter \(\lambda = q\sqrt{\frac{d \zeta}{n}}\). Throughout this part, \(B\lesssim \min(1/\varepsilon,m)\); choosing the hidden universal constant small enough gives the numerical conditions needed in Proposition 6, and also \(N_\mathbf{S}=|\mathbf{S}|n\asymp mn\).

10.4.0.1 Case 1: Proposition 6 Applies.

Suppose \[\lambda > 2(\sqrt{B}+1)\delta +2c_0\sqrt{\frac{d\zeta}{n}} +2c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Then Proposition 6 applies. Its proof constructs a transfer event \(\mathcal{E}_{\mathrm{transfer}}\) with \(\mathbb{P}(\mathcal{E}_{\mathrm{transfer}})\ge1-\kappa/2\) under the present good-event budget. On \(\mathcal{E}_{\mathrm{safe}}\cap\mathcal{E}_{\mathrm{transfer}}\), which has probability at least \(1-\kappa\), for all inlier tasks \(j \in \mathbf{S}\): \[\begin{align} \|\hat{\theta}_{j} - {\theta}_{j}^\star \|_{\boldsymbol{\Sigma}_j} &= \|\hat{\beta}_{G} - {\theta}_{j}^\star \|_{\boldsymbol{\Sigma}_j} \\ &\leq (\sqrt{B}+1)\delta + c_0 \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{\varepsilon}{1-\varepsilon}B \lambda. \end{align}\] Squaring and using \((a+b+c)^2 \lesssim a^2+b^2+c^2\), we obtain \[\begin{align} \|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} &\lesssim B \delta^2 + \frac{B d \zeta}{mn} + \left(\frac{\varepsilon}{1-\varepsilon}B\right)^2 \lambda^2 \\ &\lesssim \min\left(B \delta^2, \, q^2 \frac{d \zeta}{n}\right) + \frac{B d \zeta}{mn} + q^2B^2\varepsilon^2 \frac{d \zeta}{n}. \end{align}\] The last inequality uses the displayed condition, which implies \(B\delta^2\lesssim \lambda^2=q^2d\zeta/n\).

10.4.0.2 Case 2: Proposition 6 Does Not Apply.

If the displayed condition fails, then \[\lambda \le 2(\sqrt{B}+1)\delta +2c_0\sqrt{\frac{d\zeta}{n}} +2c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Since \(q\gtrsim1\) is chosen large enough that \(q\ge 8c_0\), we can absorb the middle term into the left-hand side and obtain \[\frac{\lambda}{2} \lesssim \sqrt{B}\delta +\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}},\] where we used \(B\ge1\). Hence at least one of the following alternatives must hold: \[\lambda\lesssim \sqrt{B}\delta \qquad\text{or}\qquad \lambda\lesssim \sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Equivalently, \[q^2\frac{d\zeta}{n}\lesssim B\delta^2 \qquad\text{or}\qquad q^2\frac{d\zeta}{n}\lesssim \frac{Bd\zeta}{N_\mathbf{S}}.\] On the already-defined event \(\mathcal{E}_{\mathrm{safe}}\), \[\begin{align} \|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} &\lesssim \frac{d \zeta}{n} + \lambda^2 \lesssim q^2 \frac{d \zeta}{n}. \end{align}\] The preceding alternatives show that this safety bound is controlled by either the heterogeneity term or the pooling term, and therefore by the right-hand side claimed in part (b).

10.4.0.3 Conclusion.

Combining the two cases, we conclude that with probability at least \(1-\kappa\), for all \(j \in \mathbf{S}\), \[\|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} \lesssim \frac{B d \zeta}{mn} + \min\left(B \delta^2, \, q^2 \frac{d \zeta}{n}\right) + q^2B^2 \varepsilon^2 \frac{d \zeta}{n}.\] The outlier tasks are already covered by the universal safety bound proved at the beginning. ◻

Proof. Using the pooled whitening notation introduced above, let \(\bar{\hat{\beta}}_{F_\mathbf{S}} := \boldsymbol{\Sigma}_\mathbf{S}^{1/2} \hat{\beta}_{F_\mathbf{S}}\).

11.2.0.1 Step 1: Gradient Bound via Strong Convexity.

The Hessian of the transformed loss satisfies \(\nabla^2 \bar{F}_\mathbf{S}(\bar{\beta}) \succeq \alpha_\ell |\mathbf{S}| \mathbf{I}_d\), so \(\bar{F}_\mathbf{S}\) is \(\mu\)-strongly convex with \(\mu=\alpha_\ell |\mathbf{S}|\). Let \(\bar{\hat{\beta}}_{F_\mathbf{S}}\) be the minimizer of \(\bar{F}_\mathbf{S}\). Strong convexity implies \[\bigl\langle \nabla \bar{F}_\mathbf{S}(\bar{\theta}^\star) - \nabla \bar{F}_\mathbf{S}(\bar{\hat{\beta}}_{F_\mathbf{S}}),\, \bar{\theta}^\star - \bar{\hat{\beta}}_{F_\mathbf{S}} \bigr\rangle \ge \mu \|\bar{\theta}^\star - \bar{\hat{\beta}}_{F_\mathbf{S}}\|_2^2.\] Since \(\bar{\hat{\beta}}_{F_\mathbf{S}}\) minimizes \(\bar F_\mathbf{S}\) over the convex set \(\bar\mathbf{C}\) and \(\bar\theta^\star\in\bar\mathbf{C}\), the constrained first-order condition gives \[\left\langle \nabla \bar F_\mathbf{S}(\bar{\hat{\beta}}_{F_\mathbf{S}}),\, \bar\theta^\star-\bar{\hat{\beta}}_{F_\mathbf{S}}\right\rangle\ge0.\] Therefore Cauchy–Schwarz yields \[\|\bar{\hat{\beta}}_{F_\mathbf{S}} - \bar{\theta}^\star\|_2 \le \frac{1}{\alpha_\ell |\mathbf{S}|}\|\nabla \bar{F}_\mathbf{S}(\bar{\theta}^\star)\|_2.\]

11.2.0.2 Step 2: Gradient Decomposition.

We calculate the gradient at the centroid \(\bar{\theta}^\star\): \[\begin{align} \nabla \bar{F}_\mathbf{S}(\bar{\theta}^\star) &= \frac{1}{n} \sum_{j \in \mathbf{S}} \sum_{i=1}^n \left( m(\tilde{x}_{ji}^\top \bar{\theta}^\star)-y_{ji} \right) \tilde{x}_{ji} \\ &= -\underbrace{\frac{1}{n} \sum_{j \in \mathbf{S}} \sum_{i=1}^n \varepsilon_{ji} \tilde{x}_{ji}}_{\text{Noise}} + \underbrace{\frac{1}{n} \sum_{j \in \mathbf{S}} \sum_{i=1}^n \left( m(x_{ji}^\top \theta^\star)-m(x_{ji}^\top \theta_j^\star) \right) \tilde{x}_{ji}}_{\text{Bias}}, \end{align}\] where \(\tilde{x}_{ji} = \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} x_{ji}\) and \(m=\psi'\) is the mean function.

11.2.0.3 Step 3: Bounding the Noise Term.

The noise term is a sum of independent sub-Gaussian random vectors. Since \[\sum_{j \in \mathbf{S}} \sum_{i=1}^n \tilde{x}_{ji} \tilde{x}_{ji}^\top = \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \Bigl(\sum_{j \in \mathbf{S}} \mathbf{X}_j^\top \mathbf{X}_j\Bigr) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} = N_\mathbf{S}\mathbf{I}_d,\] the covariance of the sum is isotropic. Applying the Hanson-Wright inequality with failure level \(\kappa/4\): \[\|\text{Noise}\|_2 \le c_0 \frac{\sqrt{N_\mathbf{S}d\zeta}}{n} = c_0 \sqrt{\frac{|\mathbf{S}| d\zeta}{n}}.\]

11.2.0.4 Step 4: Bounding the Bias Term.

By the Mean Value Theorem, \(m(x_{ji}^\top \theta^\star) - m(x_{ji}^\top \theta_j^\star) = u_{ji} x_{ji}^\top (\theta^\star - \theta_j^\star)\), where \(u_{ji} \in [\alpha_\ell, \alpha_u]\). Let \(\mathbf{Q}_j := \frac{1}{n} \sum_{i=1}^n u_{ji} x_{ji} x_{ji}^\top\). Note that \(\alpha_\ell \boldsymbol{\Sigma}_j \preceq \mathbf{Q}_j \preceq \alpha_u \boldsymbol{\Sigma}_j\). The bias term can be written as: \[\text{Bias} = \sum_{j \in \mathbf{S}} \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j (\theta^\star - \theta_j^\star) = \sum_{j \in \mathbf{S}} \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \boldsymbol{\Sigma}_\mathbf{S}^{1/2} (\theta^\star - \theta_j^\star).\] We bound the norm of each term in the sum: \[\begin{align} \|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j (\theta^\star - \theta_j^\star)\|_2 &\le \|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j^{1/2}\|_{\operatorname{op}} \|\mathbf{Q}_j^{1/2} (\theta^\star - \theta_j^\star)\|_2 \\ &\le \|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j^{1/2}\|_{\operatorname{op}} \sqrt{\alpha_u} \|\theta^\star - \theta_j^\star\|_{\boldsymbol{\Sigma}_j}. \end{align}\] Using \(\mathbf{Q}_j \preceq \alpha_u \boldsymbol{\Sigma}_j\) and Assumption 1 (\(\boldsymbol{\Sigma}_j \preceq B \boldsymbol{\Sigma}_\mathbf{S}\)), we have: \[\|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j^{1/2}\|_{\operatorname{op}}^2 = \|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \mathbf{Q}_j \boldsymbol{\Sigma}_\mathbf{S}^{-1/2}\|_{\operatorname{op}} \le \alpha_u \|\boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \boldsymbol{\Sigma}_j \boldsymbol{\Sigma}_\mathbf{S}^{-1/2}\|_{\operatorname{op}} \le \alpha_u B.\] Moreover, by the global covariate normalization \(\|\boldsymbol{\Sigma}_j\|_{\operatorname{op}}\le 1\) from the Regularity Assumptions and the task-relatedness condition \(\|\theta_j^\star-\theta^\star\|_2\le\delta\), we have \[\|\theta^\star-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}\le \|\theta^\star-\theta_j^\star\|_2\le \delta.\] Thus, \(\|\text{Bias term}_j\|_2 \le \sqrt{\alpha_u B} \sqrt{\alpha_u} \delta = \alpha_u \sqrt{B} \delta\). Summing over \(j \in \mathbf{S}\): \[\|\text{Bias}\|_2 \le \sum_{j \in \mathbf{S}} \alpha_u \sqrt{B} \delta = |\mathbf{S}| \alpha_u \sqrt{B} \delta.\]

11.2.0.5 Step 5: Final Bound.

Combining the bounds: \[\begin{align} \|\hat{\beta}_{F_{\mathbf{S}}} - \theta^\star\|_{\boldsymbol{\Sigma}_\mathbf{S}} &\le \frac{1}{\alpha_\ell |\mathbf{S}|} \left( c_0 \sqrt{\frac{|\mathbf{S}| d\zeta}{n}} + |\mathbf{S}| \alpha_u \sqrt{B} \delta \right) \\ &= \frac{c_0}{\alpha_\ell} \sqrt{\frac{d\zeta}{n |\mathbf{S}|}} + \frac{\alpha_u}{\alpha_\ell} \sqrt{B} \delta \\ &= \frac{c_0}{\alpha_\ell} \sqrt{\frac{d\zeta}{N_\mathbf{S}}} + \tau \sqrt{B} \delta. \end{align}\] The task-specific bound follows from \(\|\cdot\|_{\boldsymbol{\Sigma}_j} \le \sqrt{B} \|\cdot\|_{\boldsymbol{\Sigma}_\mathbf{S}}\). ◻

Proof. Let \(\mathcal{E}_{\mathrm{pool}}^{\mathrm{glm}}\) be the event from Lemma 4; it has probability at least \(1-\kappa/4\). Let \(\mathcal{E}_{\mathrm{inv}}^{\mathrm{glm}}\) be the event that the taskwise GLM invariant-region noise bound from Proposition 3 holds for every \(j\in\mathbf{S}\), each invoked with failure level \(\kappa/(4m)\). Then \(\mathbb{P}(\mathcal{E}_{\mathrm{pool}}^{\mathrm{glm}}\cap\mathcal{E}_{\mathrm{inv}}^{\mathrm{glm}})\ge1-\kappa/2\).

The goal is to show that the unregularized pooling estimator \(\hat{\beta}_{F_{\mathbf{S}}}\) minimizes the regularized objective \(G_\mathbf{S}(\beta) = \sum_{j \in \mathbf{S}} g_j(\beta)\). For GLMs, \(\nabla^2 f_j(\beta)=\frac{1}{n}\mathbf{X}_j^\top \mathbf{D}_j(\beta)\mathbf{X}_j \succeq \alpha_\ell \boldsymbol{\Sigma}_j\), where \(\mathbf{D}_j(\beta)\) is the diagonal matrix of \(\psi''\)-values. Hence \[\nabla^2 F_\mathbf{S}(\beta) = \sum_{j\in\mathbf{S}}\nabla^2 f_j(\beta) \succeq \alpha_\ell\sum_{j\in\mathbf{S}}\boldsymbol{\Sigma}_j = \alpha_\ell |\mathbf{S}| \boldsymbol{\Sigma}_\mathbf{S} \succ 0,\] so \(F_\mathbf{S}\) is strictly convex and \(\hat{\beta}_{F_{\mathbf{S}}}\) is its unique minimizer.

By Proposition 3, the condition \(\hat{\beta}_{F_{\mathbf{S}}} \in R_j\) is satisfied if: \[\label{eq:32invariant32condition32check} \lambda > \alpha_u \|\theta_j^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} + c_0 \sqrt{\frac{d \zeta}{n}}.\tag{9}\] We bound the error term \(\|\theta_j^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j}\) using the triangle inequality: \[\begin{align} \|\theta_j^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} &\leq \|\theta_j^\star - \theta^\star\|_{\boldsymbol{\Sigma}_j} + \|\theta^\star - \hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j}. \end{align}\] For the first term, using the bounded covariate assumption implies \(\|\theta_j^\star - \theta^\star\|_{\boldsymbol{\Sigma}_j} \le \delta\). For the second term, we apply Lemma 4: \[\|\hat{\beta}_{F_{\mathbf{S}}} - \theta^\star\|_{\boldsymbol{\Sigma}_j} \leq B \tau \delta + \frac{c_0}{\alpha_\ell}\sqrt{\frac{B d\zeta}{N_\mathbf{S}}}.\] Combining these, the condition 9 becomes: \[\begin{align} \lambda > \alpha_u \left( \delta + B\tau\delta + \frac{c_0}{\alpha_\ell}\sqrt{\frac{B d\zeta}{N_\mathbf{S}}} \right) + c_0 \sqrt{\frac{d \zeta}{n}}. \end{align}\] This is exactly the assumed lower bound on \(\lambda\). Thus, on \(\mathcal{E}_{\mathrm{pool}}^{\mathrm{glm}}\cap\mathcal{E}_{\mathrm{inv}}^{\mathrm{glm}}\), \(\hat{\beta}_{F_{\mathbf{S}}}\) lies in the invariant region \(R_j\) for all \(j \in \mathbf{S}\). For each \(j\in\mathbf{S}\), define the positive slack \[\Delta_j^{\mathrm{glm}} := \lambda-\left(\alpha_u\|\theta_j^\star-\hat{\beta}_{F_{\mathbf{S}}}\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n}}\right) >0.\] Since \(\beta\mapsto \|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j}\) is continuous and the stochastic term in Proposition 3 does not depend on \(\beta\), there exists \(r_j>0\) such that \[\|\beta-\hat{\beta}_{F_{\mathbf{S}}}\|_2<r_j \quad\Longrightarrow\quad \alpha_u\|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j}+c_0\sqrt{\frac{d\zeta}{n}}<\lambda.\] Hence Proposition 3 implies \(f_j(\beta)=g_j(\beta)\) throughout the Euclidean ball \(B_{r_j}(\hat{\beta}_{F_{\mathbf{S}}})\). Let \(r:=\min_{j\in\mathbf{S}}r_j>0\). Then \[F_\mathbf{S}(\beta)=G_\mathbf{S}(\beta) \qquad\text{for all }\beta\in B_r(\hat{\beta}_{F_{\mathbf{S}}}).\] Since \(\hat{\beta}_{F_{\mathbf{S}}}\) is the unique minimizer of \(F_\mathbf{S}\), it is a local minimizer of \(G_\mathbf{S}\) as well. Finally, \(G_\mathbf{S}\) is convex as a sum of infimal convolutions of convex functions, so any local minimizer is global. Hence \(\hat{\beta}_{G_\mathbf{S}}=\hat{\beta}_{F_{\mathbf{S}}}\). ◻

Proof. Using the pooled whitening notation introduced above, let \(\bar{\hat{\beta}}_{G} := \boldsymbol{\Sigma}_{\mathbf{S}}^{1/2}\hat{\beta}_{G}\) and \(\bar{\hat{\beta}}_{G_\mathbf{S}} := \boldsymbol{\Sigma}_{\mathbf{S}}^{1/2}\hat{\beta}_{G_{\mathbf{S}}}\) denote the minimizers of the transformed global objective \(\bar{G} = \bar{G}_\mathbf{S}+ \bar{L}\) and the inlier objective \(\bar{G}_\mathbf{S}\), respectively. Let \(\mathcal{E}_{\mathrm{glm}}\) be the good event constructed in Proposition 7: the GLM pooling error holds and the taskwise GLM invariant-region noise bounds hold for all \(j\in\mathbf{S}\). The pooling part is allocated failure \(\kappa/4\), and the taskwise invariant-region noise bounds are allocated failure \(\kappa/(4m)\) per task, so \[\mathbb{P}(\mathcal{E}_{\mathrm{glm}})\ge 1-\frac{\kappa}{2}\ge 1-\kappa.\]

11.3.0.1 Step 1: Pooling Estimator and Strong Convexity.

By Proposition 7, the pooling estimator coincides with the regularized inlier estimator, i.e., \(\hat{\beta}_{F_\mathbf{S}} = \hat{\beta}_{G_\mathbf{S}}\). Furthermore, invoking Claim 2, for all \(\beta\) in the neighborhood \(\|\bar{\beta} - \bar{\hat{\beta}}_{G_\mathbf{S}} \|_2 \leq \frac{\lambda}{2 \alpha_u\sqrt{ B}}\), the transformed inlier loss is strongly convex: \[\begin{align} \nabla^2 \bar{G}_\mathbf{S}(\bar{\beta}) \succeq \alpha_\ell |\mathbf{S}| \mathbf{I}_d. \end{align}\]

11.3.0.2 Step 2: Perturbation via Outliers.

We bound the Lipschitz constant of the transformed outlier term \(\bar{L}\). Using the assumption \(B < \frac{1}{4\tau \varepsilon}\) and Lemma 5: \[\begin{align} \operatorname{Lip}(\bar{L}) \leq m\varepsilon \sqrt{B} \lambda. \end{align}\] We apply the constrained perturbation lemma (Lemma 12) over the convex set \(\bar\mathbf{C}\), with \(f=\bar G_\mathbf{S}\), \(g=\bar L\), \(x_0=\bar{\hat{\beta}}_{G_\mathbf{S}}\), \(\rho = \alpha_\ell |\mathbf{S}|\), and \(r = \frac{\lambda}{2\alpha_u \sqrt{B}}\). Since \(|\mathbf{S}| \ge (1-\varepsilon)m\), and we may assume \(B \ge 1\) without loss of generality, the condition \(B < \frac{1}{4\tau \varepsilon}\) implies \(\varepsilon \le \frac{1}{4}\) and hence \(\frac{\varepsilon}{1-\varepsilon} \le \frac{1}{3}\). Moreover, \[\frac{\operatorname{Lip}(\bar{L})}{\rho r} = \frac{2\alpha_u \varepsilon B}{\alpha_\ell} \cdot \frac{m}{|\mathbf{S}|} \le \frac{2\tau \varepsilon B}{1-\varepsilon} < 1,\] so the condition \(\operatorname{Lip}(\bar{L}) < \rho r\) holds. Applying the lemma yields: \[\begin{align} \|\bar{\hat{\beta}}_G - \bar{\hat{\beta}}_{G_{\mathbf{S}}} \|_2 &\leq \frac{m\varepsilon \sqrt{B} \lambda}{\alpha_\ell |\mathbf{S}|} \leq \frac{\varepsilon}{1-\varepsilon} \frac{\sqrt{B} \lambda}{\alpha_\ell} \le \frac{2\varepsilon \sqrt{B}\lambda}{\alpha_\ell}, \end{align}\] where we used \(|\mathbf{S}| \ge (1-\varepsilon)m\) and \(\varepsilon \le 1/4\). Converting back to the \(\boldsymbol{\Sigma}_j\)-norm using \(\boldsymbol{\Sigma}_j \preceq B \boldsymbol{\Sigma}_\mathbf{S}\), we obtain: \[\begin{align} \|\hat{\beta}_{G} - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_j} \leq \sqrt{B} \|\bar{\hat{\beta}}_G - \bar{\hat{\beta}}_{G_{\mathbf{S}}} \|_2 \leq \frac{2\varepsilon B \lambda}{\alpha_\ell}. \end{align}\]

11.3.0.3 Step 3: Total Error and Invariant Region.

Combining the perturbation bound with the pooling error (Lemma 4): \[\begin{align} \label{equation5932bound32GLM32detail} \|\hat{\beta}_{G} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} &\leq \|\hat{\beta}_{G_{\mathbf{S}}} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} + \|\hat{\beta}_{G} - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_j} \nonumber \\ &\leq (\tau B +1)\delta + \frac{c_0}{\alpha_\ell} \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{2\varepsilon B \lambda}{\alpha_\ell}. \end{align}\tag{10}\] To prove the zero-seminorm personalization statement, we check the condition for the invariant region (Proposition 3). We need \(\lambda > \alpha_u \|\hat{\beta}_{G} - \theta_{j}^\star \|_{\boldsymbol{\Sigma}_j} + c_0 \sqrt{\frac{d \zeta}{n}}\). Substituting the error bound from 10 into the right-hand side: \[\begin{align} \text{RHS} &\leq \alpha_u \left[ (\tau B +1)\delta + \frac{c_0}{\alpha_\ell} \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{2\varepsilon B \lambda}{\alpha_\ell} \right] + c_0 \sqrt{\frac{d \zeta}{n}} \\ &= \alpha_u (\tau B +1)\delta + \tau c_0 \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + c_0 \sqrt{\frac{d \zeta}{n}} + 2 \tau \varepsilon B \lambda. \end{align}\] By the assumption \(B < \frac{1}{4\tau \varepsilon}\), we have \(2 \tau \varepsilon B \le \frac{1}{2}\). Thus, the \(\lambda\) term on the right-hand side is at most \(\frac{1}{2}\lambda\). The assumed lower bound on \(\lambda\) then ensures that the remaining deterministic and stochastic terms are also bounded by \(\frac{1}{2}\lambda\).

Therefore, on the event \(\mathcal{E}_{\mathrm{glm}}\), the condition holds. Proposition 3 then implies that every task-specific minimizer at center \(\hat{\beta}_G\) satisfies \(\|\hat{\theta}_j-\hat{\beta}_G\|_{\boldsymbol{\Sigma}_j}=0\) for all \(j\in\mathbf{S}\). Hence the corresponding prediction-seminorm errors coincide, and the bound follows directly from 10 . ◻

Proof. Let \[M_j:=\boldsymbol{\Sigma}_{\mathbf{S}}^{-1/2}\boldsymbol{\Sigma}_j\boldsymbol{\Sigma}_{\mathbf{S}}^{-1/2}.\] For \(w\in\bar\mathbf{C}\), the constrained infimal convolution becomes \[\bar g_j(w) = \inf_{z\in\bar\mathbf{C}} \left\{\bar f_j(z)+\lambda_j\|z-w\|_{M_j}\right\}.\] By Assumption 1, \(M_j\preceq B\mathbf{I}_d\). Fix \(w,w'\in\bar\mathbf{C}\) and \(\eta>0\), and choose \(z_\eta\in\bar\mathbf{C}\) such that \[\bar g_j(w')+\eta > \bar f_j(z_\eta)+\lambda_j\|z_\eta-w'\|_{M_j}.\] Then \[\begin{align} \bar g_j(w)-\bar g_j(w') &\le \lambda_j\bigl(\|z_\eta-w\|_{M_j}-\|z_\eta-w'\|_{M_j}\bigr)+\eta \\ &\le \lambda_j\|w-w'\|_{M_j}+\eta \le \sqrt{B}\lambda_j\|w-w'\|_2+\eta. \end{align}\] Letting \(\eta\downarrow0\) and exchanging \(w,w'\) proves the claim on \(\bar\mathbf{C}\). ◻

Proof. Let \(\eta := \frac{\lambda}{2\alpha_u \sqrt{B}}\). Consider any \(\beta\) such that \(\|\bar{\beta} - \bar{\hat{\beta}}_{G_{\mathbf{S}}} \|_2 \leq \eta\). First, we bound the distance in the task-specific norm \(\|\cdot\|_{\boldsymbol{\Sigma}_j}\). Using the assumption \(\boldsymbol{\Sigma}_j \preceq B \boldsymbol{\Sigma}_\mathbf{S}\): \[\begin{align} \|\beta - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_j} &\leq \sqrt{B} \|\beta - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_{\mathbf{S}}} \\ &= \sqrt{B} \|\bar{\beta} - \bar{\hat{\beta}}_{G_{\mathbf{S}}} \|_2 \\ &\leq \sqrt{B} \eta = \frac{\lambda}{2\alpha_u}. \end{align}\]

Next, we apply the triangle inequality and the error bound for the pooling estimator (Lemma 4). Recall that under the favorable event, \(\hat{\beta}_{G_\mathbf{S}} = \hat{\beta}_{F_\mathbf{S}}\). Thus: \[\begin{align} \|\beta - \theta_j^\star \|_{\boldsymbol{\Sigma}_j} &\leq \|\beta - \hat{\beta}_{G_{\mathbf{S}}} \|_{\boldsymbol{\Sigma}_j} + \|\hat{\beta}_{G_{\mathbf{S}}} - \theta_j^\star \|_{\boldsymbol{\Sigma}_j} \\ &\leq \frac{\lambda}{2\alpha_u} + \left( (\tau B + 1)\delta + \frac{c_0}{\alpha_\ell} \sqrt{\frac{B d \zeta}{N_\mathbf{S}}} \right). \end{align}\]

To ensure \(\beta\) lies in the invariant region \(R_j\), Proposition 3 requires \(\lambda > \alpha_u \|\beta - \theta_j^\star \|_{\boldsymbol{\Sigma}_j} + c_0 \sqrt{\frac{d \zeta}{n}}\). Substituting the bound above, it suffices to show: \[\begin{align} \lambda > \alpha_u \left( \frac{\lambda}{2\alpha_u} + (\tau B + 1)\delta + \frac{c_0}{\alpha_\ell} \sqrt{\frac{B d \zeta}{N_\mathbf{S}}} \right) + c_0 \sqrt{\frac{d \zeta}{n}}. \end{align}\] Simplifying the right-hand side: \[\begin{align} \lambda > \frac{\lambda}{2} + \alpha_u (\tau B + 1)\delta + \tau c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}} + c_0 \sqrt{\frac{d \zeta}{n}}. \end{align}\] Rearranging terms, this condition is equivalent to \[\begin{align} \frac{\lambda}{2} > \alpha_u (\tau B + 1)\delta + \tau c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}} + c_0 \sqrt{\frac{d \zeta}{n}}, \end{align}\] which is exactly the assumed lower bound on \(\lambda\).

Consequently, for all \(\beta\) in the neighborhood, \(\beta \in R_j\) holds for all \(j \in \mathbf{S}\). This implies \(G_\mathbf{S}(\beta) = F_\mathbf{S}(\beta)\) locally. Finally, we compute the Hessian lower bound. The Hessian of the transformed inlier loss is: \[\begin{align} \nabla^2 \bar{G}_\mathbf{S}(\bar{\beta}) &= \nabla^2 \bar{F}_\mathbf{S}(\bar{\beta}) \\ &= \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \left( \sum_{j \in \mathbf{S}} \nabla^2 f_j(\beta) \right) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2}. \end{align}\] For GLM, \(\nabla^2 f_j(\beta) = \frac{1}{n} \mathbf{X}_j^\top \mathbf{D}_j \mathbf{X}_j\), where \(\mathbf{D}_j\) is a diagonal matrix with entries \(\psi''(\cdot) \ge \alpha_\ell\). Thus, \(\nabla^2 f_j(\beta) \succeq \alpha_\ell \boldsymbol{\Sigma}_j\). Summing over inliers: \[\begin{align} \nabla^2 \bar{G}_\mathbf{S}(\bar{\beta}) &\succeq \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \left( \sum_{j \in \mathbf{S}} \alpha_\ell \boldsymbol{\Sigma}_j \right) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \\ &= \alpha_\ell \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} (|\mathbf{S}| \boldsymbol{\Sigma}_\mathbf{S}) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \\ &= \alpha_\ell |\mathbf{S}| \mathbf{I}_d. \end{align}\] This completes the proof. ◻

Proof. Let \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{glm}}\) be the event obtained by applying Proposition 4 taskwise with failure probability \(\kappa/(2m)\). By a union bound, \(\mathbb{P}(\mathcal{E}_{\mathrm{safe}}^{\mathrm{glm}})\ge1-\kappa/2\), and on this event, for all \(j\in[m]\), \[\|\hat{\theta}_{j}-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \frac{d\zeta}{n}+\lambda^2 \lesssim q^2\frac{d\zeta}{n}.\] This proves part (a), so the remainder proves the sharper inlier guarantee in part (b). Recall that \(q \gtrsim 1\), so we may choose the implicit constant large enough that \(q \ge 8c_0\). Also, since part (b) assumes \(B \lesssim \min(1/\varepsilon,m)\), the favorable regime of Proposition 8 is available and \(N_\mathbf{S}= |\mathbf{S}|n \asymp mn\).

11.4.0.1 Case 1: Proposition 8 Applies.

Suppose \[\lambda > 2\alpha_u (\tau B +1) \delta + 2 c_0 \sqrt{\frac{d \zeta}{n}} + 2 \tau c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\] Then Proposition 8 applies. Its proof constructs a transfer event \(\mathcal{E}_{\mathrm{transfer}}^{\mathrm{glm}}\) with \(\mathbb{P}(\mathcal{E}_{\mathrm{transfer}}^{\mathrm{glm}})\ge1-\kappa/2\) under the present good-event budget. On \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{glm}}\cap\mathcal{E}_{\mathrm{transfer}}^{\mathrm{glm}}\), which has probability at least \(1-\kappa\), for all inlier tasks \(j \in \mathbf{S}\), \[\|\hat{\theta}_{j} - {\theta}_{j}^\star \|_{\boldsymbol{\Sigma}_j} \leq (\tau B +1)\delta + \frac{c_0}{\alpha_\ell} \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{2\varepsilon B\lambda}{\alpha_\ell}.\] Squaring and using \((a+b+c)^2 \lesssim a^2+b^2+c^2\), \[\|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} \lesssim B^2\delta^2 + \frac{B d \zeta}{N_\mathbf{S}} + q^2B^2 \varepsilon^2 \frac{d \zeta}{n}.\] Moreover, the displayed condition implies \(\lambda \gtrsim B\delta\), hence \[B^2\delta^2 \lesssim q^2\frac{d\zeta}{n}.\] Therefore, \[\|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} \lesssim \frac{B d \zeta}{mn} + \min\left(B^2\delta^2,\, q^2\frac{d\zeta}{n}\right) + q^2B^2 \varepsilon^2 \frac{d \zeta}{n}.\]

11.4.0.2 Case 2: Proposition 8 Does Not Apply.

If the displayed condition fails, then \[\lambda \le 2\alpha_u (\tau B +1) \delta + 2 c_0 \sqrt{\frac{d \zeta}{n}} + 2 \tau c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\] Since \(q \ge 8c_0\), we can absorb the middle term into the left-hand side and obtain \[\frac{\lambda}{2} \le 2\alpha_u (\tau B +1) \delta + 2 \tau c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\] Hence at least one of the following two alternatives must hold: \[\lambda \lesssim B\delta \qquad\text{or}\qquad \lambda \lesssim \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\] Equivalently, \[q^2\frac{d\zeta}{n} \lesssim B^2\delta^2 \qquad\text{or}\qquad q^2\frac{d\zeta}{n} \lesssim \frac{B d\zeta}{N_\mathbf{S}}.\] In either subcase, \[q^2\frac{d\zeta}{n} \lesssim \frac{B d\zeta}{mn} + \min\left(B^2\delta^2,\, q^2\frac{d\zeta}{n}\right).\] On the already-defined event \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{glm}}\), the GLM personalization bound gives \[\|\hat{\theta}_{j} - {\theta}_{j}^\star \|^2_{\boldsymbol{\Sigma}_j} \lesssim \frac{d \zeta}{n} + \lambda^2 \lesssim q^2\frac{d\zeta}{n}.\] Combining the last two displays yields the same theorem-level bound in this regime.

This completes the proof. ◻

Proof. The theorem relates the in-sample MSE to the population MSE.

12.0.0.1 Part 1: Via Comparability.

The first inequality \(\mathcal{E}_j(\hat{\theta}_j) \le \nu_j \mathcal{E}_j^{\operatorname{in}}(\hat{\theta}_j)\) follows directly from the definition of the comparability constant \(\nu_j\) (Definition 7 ) and the definition of the MSEs. Specifically, \(\|\cdot\|_{\bar{\boldsymbol{\Sigma}}_j}^2 \le \nu_j \|\cdot\|_{\boldsymbol{\Sigma}_j}^2\).

12.0.0.2 Part 2: Via Intrinsic Dimension (Safety).

For the second inequality, we consider the case where \(\nu_j\) might be large. Assume that the parameter domain is known to be bounded, \(\mathbf{C}=\mathbf{B}(0,\xi)\), and that \(\theta_j^\star\in\mathbf{C}\). Let \[\hat{\theta}_j^\xi \in \mathop{\mathrm{argmin}}_{\theta\in \mathbf{B}(0,\xi)} \|\theta-\hat{\theta}_j\|_{\boldsymbol{\Sigma}_j}^2\] be the \(\|\cdot\|_{\boldsymbol{\Sigma}_j}\)-norm projection of \(\hat{\theta}_j\) onto \(\mathbf{B}(0,\xi)\).

Since \(\theta_j^\star\in\mathbf{B}(0,\xi)\), the projection property gives \[\|\hat{\theta}_j^\xi-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \le \|\hat{\theta}_j-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 = \mathcal{E}_j^{\operatorname{in}}(\hat{\theta}_j).\] Moreover, both \(\hat{\theta}_j^\xi\) and \(\theta_j^\star\) belong to \(\mathbf{B}(0,\xi)\), so \[\|\hat{\theta}_j^\xi-\theta_j^\star\|_2 \le 2\xi .\] Let \(U_j\) be the high-probability upper bound on \(\|x_{ji}\|_2\). Applying Lemma 8 to the bounded estimator \(\hat{\theta}_j^\xi\), we obtain \[\mathcal{E}_j(\hat{\theta}_j^\xi) \lesssim \|\hat{\theta}_j^\xi-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 + \frac{U_j^2}{n} \|\hat{\theta}_j^\xi-\theta_j^\star\|_2^2 .\] Combining the previous two displays yields \[\mathcal{E}_j(\hat{\theta}_j^\xi) \lesssim \mathcal{E}_j^{\operatorname{in}}(\hat{\theta}_j) + \frac{\xi^2 U_j^2}{n}.\] This confirms that even if the spectral ratio \(\nu_j\) diverges, the projected estimator has population risk controlled by the in-sample risk plus a \(1/n\)-order safety term. ◻

Proof. Recall \(\hat{\beta}_{F_\mathbf{S}} = \mathbf{V}_\mathbf{S}^{-1} \sum_{k \in \mathbf{S}} \mathbf{X}_k^\top \mathbf{Y}_k\) and write \[\|\hat{\beta}_{F_{\mathbf{S}}} - \theta^\star\|_{\boldsymbol{\Sigma}_j} \le \text{Bias}_j + \text{Variance}_j.\]

14.2.0.1 Bias Term.

Let \(\eta_k^\star := \theta_k^\star - \theta^\star\) and \(a := \sum_{k \in \mathbf{S}} \mathbf{V}_k \eta_k^\star.\) Then \(\text{Bias}_j = \| \mathbf{V}_\mathbf{S}^{-1} a \|_{\boldsymbol{\Sigma}_j}\). Using \(\boldsymbol{\Sigma}_j \preceq B \boldsymbol{\Sigma}_\mathbf{S}= \frac{B}{N_\mathbf{S}}\mathbf{V}_\mathbf{S}\), we have \[\text{Bias}_j^2 \le \frac{B}{N_\mathbf{S}} \bigl\| \mathbf{V}_\mathbf{S}^{-1/2} a \bigr\|_2^2.\] Next, by \(\boldsymbol{\Sigma}_k \preceq B \boldsymbol{\Sigma}_\mathbf{S}\) we have \(\mathbf{V}_k \preceq \frac{n_k B}{N_\mathbf{S}} \mathbf{V}_\mathbf{S}\), hence \[\bigl\| \mathbf{V}_\mathbf{S}^{-1/2} \mathbf{V}_k \eta_k^\star \bigr\|_2 \le \sqrt{\frac{n_k B}{N_\mathbf{S}}} \, \|\eta_k^\star\|_{\mathbf{V}_k}.\] Therefore, \[\bigl\| \mathbf{V}_\mathbf{S}^{-1/2} a \bigr\|_2 \le \sum_{k \in \mathbf{S}} \bigl\| \mathbf{V}_\mathbf{S}^{-1/2} \mathbf{V}_k \eta_k^\star \bigr\|_2 \le \sqrt{\frac{B}{N_\mathbf{S}}} \sum_{k \in \mathbf{S}} n_k \|\eta_k^\star\|_{\boldsymbol{\Sigma}_k}.\] Under the bounded covariate assumption, \(\|\eta_k^\star\|_{\boldsymbol{\Sigma}_k} \le \|\eta_k^\star\|_2\), and the task-relatedness condition gives \(\sqrt{n_k}\|\eta_k^\star\|_2 \le \delta_0\). Hence \[\text{Bias}_j \le \frac{B}{N_\mathbf{S}} \sum_{k \in \mathbf{S}} \sqrt{n_k}\,\delta_0 = \alpha \frac{B\delta_0}{\sqrt{n^\star}}.\]

14.2.0.2 Variance Term.

Using Hanson–Wright with failure level \(\kappa/(4m)\) for each \(j\in\mathbf{S}\), a union bound, and \(\boldsymbol{\Sigma}_j \preceq \frac{B}{N_\mathbf{S}} \mathbf{V}_\mathbf{S}\), \[\text{Variance}_j \le c_0 \sqrt{\operatorname{Tr}(\boldsymbol{\Sigma}_j \mathbf{V}_\mathbf{S}^{-1}) \zeta} \le c_0 \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\]

For the task-specific error, apply triangle inequality: \[\|\hat{\beta}_{F_{\mathbf{S}}} - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \le \|\hat{\beta}_{F_{\mathbf{S}}} - \theta^\star\|_{\boldsymbol{\Sigma}_j} + \|\theta^\star-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}.\] The first term is bounded above. For the second, \(\|\theta^\star-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}\le \|\theta^\star-\theta_j^\star\|_2 \le \delta_0/\sqrt{n_j}\) by definition of task relatedness. Since \(\alpha B\ge 1\), this yields \[\|\hat{\beta}_{F_{\mathbf{S}}} - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \lesssim \alpha\frac{B \delta_0 }{\sqrt{n_j}} + \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}.\] ◻

Proof. Let \(\mathcal{E}_{\mathrm{pool}}^{\mathrm{gs}}\) be the event from Lemma 6, which has probability at least \(1-\kappa/4\). Let \(\mathcal{E}_{\mathrm{inv}}^{\mathrm{gs}}\) be the event that the taskwise invariant-region noise bound from Proposition 1 holds for all \(j\in\mathbf{S}\), each invoked with failure level \(\kappa/(4m)\). Then \(\mathbb{P}(\mathcal{E}_{\mathrm{pool}}^{\mathrm{gs}}\cap\mathcal{E}_{\mathrm{inv}}^{\mathrm{gs}})\ge1-\kappa/2\). On this intersection, it suffices to verify for each \(j\in \mathbf{S}\): \[\frac{\lambda}{\sqrt{n_j}} > \|\theta_j^\star-\hat{\beta}_{F_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n_j}}.\] We bound the first term by triangle inequality: \[\|\theta_j^\star-\hat{\beta}_{F_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} \le \|\theta_j^\star-\theta^\star\|_{\boldsymbol{\Sigma}_j} + \|\theta^\star-\hat{\beta}_{F_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j}.\] Since \(\|\boldsymbol{\Sigma}_j\|_{\operatorname{op}}\le 1\) and \(\sqrt{n_j}\|\theta_j^\star-\theta^\star\|_2\le \delta_0\), we have \[\|\theta_j^\star-\theta^\star\|_{\boldsymbol{\Sigma}_j}\le \frac{\delta_0}{\sqrt{n_j}}.\] By Lemma 6, \[\|\theta^\star-\hat{\beta}_{F_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} \le \alpha\frac{B\delta_0}{\sqrt{n^\star}} + c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}} \le \alpha\frac{B\delta_0}{\sqrt{n_j}} + c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Combining the last three displays and using \(\alpha B\ge 1\) (we may assume \(B\ge1\)), the assumed lower bound on \(\lambda/\sqrt{n_j}\) implies \(\hat{\beta}_{F_\mathbf{S}}\in R_j\) for all \(j\in \mathbf{S}\). For each \(j\in\mathbf{S}\), define the positive slack \[\Delta_j^{\mathrm{gs}} := \frac{\lambda}{\sqrt{n_j}} - \left( \|\theta_j^\star-\hat{\beta}_{F_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n_j}} \right) >0.\] Since \(\beta\mapsto \|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j}\) is continuous and the stochastic term in Proposition 1 does not depend on \(\beta\), there exists \(r_j>0\) such that \[\|\beta-\hat{\beta}_{F_\mathbf{S}}\|_2<r_j \quad\Longrightarrow\quad \frac{\lambda}{\sqrt{n_j}} > \|\theta_j^\star-\beta\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n_j}}.\] Hence Proposition 1 implies \(f_j(\beta)=g_j(\beta)\) throughout the Euclidean ball \(B_{r_j}(\hat{\beta}_{F_\mathbf{S}})\). Let \(r:=\min_{j\in\mathbf{S}} r_j>0\). Then \[F_\mathbf{S}(\beta)=G_\mathbf{S}(\beta) \qquad\text{for all }\beta\in B_r(\hat{\beta}_{F_\mathbf{S}}).\] Since \(F_\mathbf{S}\) is strictly convex with Hessian \(N_\mathbf{S}\boldsymbol{\Sigma}_\mathbf{S}\succ 0\), \(\hat{\beta}_{F_\mathbf{S}}\) is its unique minimizer, and therefore a local minimizer of \(G_\mathbf{S}\) as well. By convexity, this local minimizer is global, so \(\hat{\beta}_{F_\mathbf{S}}\) minimizes \(G_\mathbf{S}\). This proves the claimed statement on \(\mathcal{E}_{\mathrm{pool}}^{\mathrm{gs}}\cap\mathcal{E}_{\mathrm{inv}}^{\mathrm{gs}}\), whose probability is at least \(1-\kappa/2\). ◻

Proof. The proof follows the same logic as the equal-sample-size case, but we verify the weighted scaling explicitly. Fix \(\beta\) such that \(\|\bar{\beta} - \bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \le \frac{\lambda}{2\sqrt{B n^\star}}\). For any \(j \in \mathbf{S}\), using \(\boldsymbol{\Sigma}_j \preceq B\boldsymbol{\Sigma}_\mathbf{S}\): \[\begin{align} \|\beta - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} &\le \sqrt{B}\|\beta - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_\mathbf{S}} = \sqrt{B}\|\bar{\beta} - \bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \\ &\le \frac{\lambda}{2\sqrt{n^\star}} \le \frac{\lambda}{2\sqrt{n_j}}. \end{align}\] By Proposition 9, we have \(\hat{\beta}_{G_\mathbf{S}}=\hat{\beta}_{F_\mathbf{S}}\). Hence, by Lemma 6, \[\|\hat{\beta}_{G_\mathbf{S}} - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \lesssim \alpha \frac{B\delta_0}{\sqrt{n_j}} + \sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Hence \[\begin{align} \|\beta - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} &\lesssim \frac{\lambda}{2\sqrt{n_j}} + \alpha \frac{B\delta_0}{\sqrt{n_j}} + \sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}. \end{align}\] Proposition 1 (applied with task-wise \(\lambda_j=\lambda/\sqrt{n_j}\)) requires \[\frac{\lambda}{\sqrt{n_j}} > \|\beta-\theta_j^\star\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n_j}}.\] The displayed assumption, with \(C_{\mathrm{gs}}\) chosen sufficiently large, leaves enough slack after the \(\lambda/(2\sqrt{n_j})\) term to verify this condition. Therefore \(\beta \in R_j\) for all \(j \in \mathbf{S}\), so \(G_\mathbf{S}(\beta)=F_\mathbf{S}(\beta)\) locally. Consequently, \(\nabla^2 \bar{G}_\mathbf{S}(\bar{\beta})=\nabla^2 \bar{F}_\mathbf{S}(\bar{\beta})=N_\mathbf{S}\mathbf{I}_d\). ◻

Proof. The proof strategy mirrors Proposition 6, adapted for weighted sums. Let \(\mathcal{E}_{\mathrm{gs}}\) be the good event from Proposition 9: the pooling estimator bound and all taskwise invariant-region noise bounds hold. The pooled event is allocated failure \(\kappa/4\), and the taskwise events are allocated failure \(\kappa/(4m)\) each, so \(\mathbb{P}(\mathcal{E}_{\mathrm{gs}})\ge1-\kappa/2\ge1-\kappa\). Using the pooled whitening notation introduced above, let \[\bar{\hat{\beta}}_G := \boldsymbol{\Sigma}_\mathbf{S}^{1/2}\hat{\beta}_G, \qquad \bar{\hat{\beta}}_{G_\mathbf{S}} := \boldsymbol{\Sigma}_\mathbf{S}^{1/2}\hat{\beta}_{G_\mathbf{S}}.\]

14.3.0.1 Step 1: Strong Convexity.

The aggregate inlier loss is \(F_\mathbf{S}(\beta) = \sum_{j \in \mathbf{S}} \frac{1}{2} \|\mathbf{Y}_j - \mathbf{X}_j \beta\|_2^2\). By Proposition 9, we may set \(\hat{\beta}_{G_\mathbf{S}}=\hat{\beta}_{F_\mathbf{S}}\) on the high-probability event. The Hessian of the transformed inlier loss is \[\nabla^2 \bar{F}_\mathbf{S}(\bar{\beta}) = \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} \left( \sum_{j \in \mathbf{S}} \mathbf{X}_j^\top \mathbf{X}_j \right) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} = \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} (N_\mathbf{S}\boldsymbol{\Sigma}_\mathbf{S}) \boldsymbol{\Sigma}_\mathbf{S}^{-1/2} = N_\mathbf{S}\mathbf{I}_d.\] Thus, \(\bar{F}_\mathbf{S}\) is \(N_\mathbf{S}\)-strongly convex. By Claim 3, this strong convexity holds for \(\bar{G}_\mathbf{S}\) in the relevant neighborhood.

14.3.0.2 Step 2: Lipschitz Constant of Outliers.

We bound the Lipschitz constant of the transformed outlier term \(\bar{L}(\cdot) = \sum_{j \in \mathbf{S}^c} n_j \bar{g}_j(\cdot)\). From Lemma 3, \(\bar{g}_j\) is \(\sqrt{B}\lambda_j\)-Lipschitz. Thus, \(n_j \bar{g}_j\) is \(n_j \sqrt{B} (\lambda/\sqrt{n_j}) = \sqrt{n_j} \sqrt{B} \lambda\)-Lipschitz. Summing over outliers: \[\operatorname{Lip}(\bar{L}) \le \sum_{j \in \mathbf{S}^c} \sqrt{B} \lambda \sqrt{n_j} = \sqrt{B} \lambda \sum_{j \in \mathbf{S}^c} \sqrt{n_j}.\] Using the task-relatedness definition (\(\sum_{\mathbf{S}^c} \sqrt{n_j} \le \varepsilon \frac{N_\mathbf{S}}{\sqrt{n^\star}}\)): \[\operatorname{Lip}(\bar{L}) \le \frac{\varepsilon \sqrt{B} \lambda N_\mathbf{S}}{\sqrt{n^\star}}.\]

14.3.0.3 Step 3: Perturbation Bound.

We apply Lemma 11 with \(\rho = N_\mathbf{S}\) and radius \[r = \frac{\lambda}{2\sqrt{B n^\star}},\] for which Claim 3 guarantees local strong convexity of \(\bar{G}_\mathbf{S}\). The perturbation condition requires \(\operatorname{Lip}(\bar{L}) < \rho r\). Using the bound above, \[\operatorname{Lip}(\bar{L}) \le \frac{\varepsilon \sqrt{B} \lambda N_\mathbf{S}}{\sqrt{n^\star}} = 2\varepsilon B \, \rho r.\] Since \(\varepsilon < \frac{1}{3B}\), the condition holds. The lemma yields \[\|\bar{\hat{\beta}}_G - \bar{\hat{\beta}}_{G_\mathbf{S}}\|_2 \le \frac{\operatorname{Lip}(\bar{L})}{\rho} \le \frac{\varepsilon \sqrt{B} \lambda}{\sqrt{n^\star}}.\] Converting to the \(\boldsymbol{\Sigma}_j\)-norm, \[\|\hat{\beta}_G - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} \le \sqrt{B} \|\hat{\beta}_G - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_\mathbf{S}} \le \frac{\varepsilon B \lambda}{\sqrt{n^\star}}.\]

14.3.0.4 Step 4: Combining Errors.

Combining this with the pooling estimator error (Lemma 6): \[\begin{align} \|\hat{\beta}_G - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} &\le \|\hat{\beta}_G - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} + \|\hat{\beta}_{G_\mathbf{S}} - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \\ &= \|\hat{\beta}_G - \hat{\beta}_{G_\mathbf{S}}\|_{\boldsymbol{\Sigma}_j} + \|\hat{\beta}_{F_\mathbf{S}} - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \\ &\lesssim \frac{\varepsilon B \lambda}{\sqrt{n^\star}} + \alpha \frac{B\delta_0}{\sqrt{n_j}} + \sqrt{\frac{B d \zeta}{N_\mathbf{S}}}. \end{align}\]

14.3.0.5 Step 5: Invariant Region for the Full Estimator.

It remains to justify the zero-seminorm personalization statement for inlier tasks. Proposition 1, with taskwise regularization \(\lambda_j=\lambda/\sqrt{n_j}\), requires \[\frac{\lambda}{\sqrt{n_j}} > \|\hat{\beta}_G-\theta_j^\star\|_{\boldsymbol{\Sigma}_j} + c_0\sqrt{\frac{d\zeta}{n_j}}.\] The deterministic part of the displayed assumption, with \(C_{\mathrm{gs}}\) sufficiently large, controls the terms \(\alpha B\delta_0/\sqrt{n_j}\), \(\sqrt{Bd\zeta/N_\mathbf{S}}\), and \(c_0\sqrt{d\zeta/n_j}\). The perturbation term is absorbed by the small-contamination condition: since \(n_j\le n^\star\), \[\frac{\varepsilon B\lambda}{\sqrt{n^\star}} \le \varepsilon B \frac{\lambda}{\sqrt{n_j}} \le \frac{1}{3}\frac{\lambda}{\sqrt{n_j}}.\] Thus \(\hat{\beta}_G\in R_j\) for every \(j\in\mathbf{S}\), which implies that every task-specific minimizer at center \(\hat{\beta}_G\) satisfies \(\|\hat{\theta}_j-\hat{\beta}_G\|_{\boldsymbol{\Sigma}_j}=0\). Hence the corresponding prediction-seminorm errors coincide, so the pooled-center error bound above also gives the stated bound for \(\hat{\theta}_j\). ◻

Proof. Recall \(\lambda = q\sqrt{d \zeta}\). Since \(q \gtrsim 1\), we may choose the implicit constant so that \(q \ge 8c_0\). Let \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{gs}}\) be the event obtained by applying the personalization bound taskwise with failure probability \(\kappa/(2m)\). Then \(\mathbb{P}(\mathcal{E}_{\mathrm{safe}}^{\mathrm{gs}})\ge1-\kappa/2\), and on this event, for every task \(j\in[m]\), \[\|\hat{\theta}_j-\theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \frac{d\zeta}{n_j}+\frac{\lambda^2}{n_j} \lesssim q^2\frac{d\zeta}{n_j}.\] This proves part (a).

14.4.0.1 Case 1: Proposition 10 Applies.

Because the inequality in Proposition 10 is hardest when \(n_j=n^\star\), it suffices to assume \[\frac{\lambda}{\sqrt{n^\star}} > C_{\mathrm{gs}}\left( \alpha B \frac{\delta_0}{\sqrt{n^\star}} + c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}} + c_0\sqrt{\frac{d \zeta}{n^\star}} \right).\] Indeed, after multiplying both sides by \(\sqrt{n_j}\), the only \(n_j\)-dependent term is the pooling-variance contribution \(c_0\sqrt{n_jBd\zeta/N_\mathbf{S}}\), which is largest at \(n_j=n^\star\). Then Proposition 10 applies. Its proof constructs a transfer event \(\mathcal{E}_{\mathrm{transfer}}^{\mathrm{gs}}\) with \(\mathbb{P}(\mathcal{E}_{\mathrm{transfer}}^{\mathrm{gs}})\ge1-\kappa/2\) under the present good-event budget. On \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{gs}}\cap\mathcal{E}_{\mathrm{transfer}}^{\mathrm{gs}}\), which has probability at least \(1-\kappa\), for every \(j\in \mathbf{S}\), \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \lesssim \alpha\frac{B\delta_0}{\sqrt{n_j}} + \sqrt{\frac{Bd \zeta}{N_\mathbf{S}}} + \frac{\varepsilon B \lambda}{\sqrt{n^\star}}.\] Squaring yields \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \alpha^2B^2\frac{\delta_0^2}{n_j} +\frac{Bd\zeta}{N_\mathbf{S}} +\frac{\varepsilon^2B^2\lambda^2}{n^\star}.\] Moreover, the displayed condition implies \(\alpha B\delta_0 \lesssim \lambda=q\sqrt{d\zeta}\), so the heterogeneity term may be replaced, up to constants, by \[\frac{1}{n_j}\min\left(\alpha^2B^2\delta_0^2,\;q^2d\zeta\right).\] Since \(n_j\le n^\star\), the pooling term satisfies \[\frac{Bd\zeta}{N_\mathbf{S}} \le \frac{n^\star}{n_j}\frac{Bd\zeta}{N_\mathbf{S}}.\] Therefore, on \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{gs}}\cap\mathcal{E}_{\mathrm{transfer}}^{\mathrm{gs}}\), \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \frac{n^\star}{n_j}\frac{Bd\zeta}{N_\mathbf{S}} + \frac{1}{n_j}\min\left(\alpha^2B^2\delta_0^2,\;q^2d\zeta\right) + \frac{\varepsilon^2B^2\lambda^2}{n^\star}.\]

14.4.0.2 Case 2: Proposition 10 Does Not Apply.

If the displayed condition fails, then \[\frac{\lambda}{\sqrt{n^\star}} \le C_{\mathrm{gs}}\left( \alpha B \frac{\delta_0}{\sqrt{n^\star}} + c_0\sqrt{\frac{Bd\zeta}{N_\mathbf{S}}} + c_0\sqrt{\frac{d \zeta}{n^\star}} \right).\] Since \(q \gtrsim 1\) is chosen large enough relative to \(C_{\mathrm{gs}}c_0\), we absorb the last term into the left-hand side and obtain \[\frac{\lambda}{2\sqrt{n^\star}} \lesssim \alpha B \frac{\delta_0}{\sqrt{n^\star}} + \sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] Therefore at least one of the following alternatives must hold: \[\frac{\lambda}{\sqrt{n^\star}} \lesssim \alpha B \frac{\delta_0}{\sqrt{n^\star}} \qquad\text{or}\qquad \frac{\lambda}{\sqrt{n^\star}} \lesssim \sqrt{\frac{Bd\zeta}{N_\mathbf{S}}}.\] The first alternative implies, for every \(j\in\mathbf{S}\), \[q^2\frac{d\zeta}{n_j} = \frac{\lambda^2}{n_j} \lesssim \alpha^2B^2\frac{\delta_0^2}{n_j}.\] The second alternative implies \[q^2 \frac{d \zeta}{n^\star} \lesssim \frac{B d \zeta}{N_\mathbf{S}} \qquad\Longrightarrow\qquad q^2\frac{d\zeta}{n_j} = \frac{n^\star}{n_j}q^2\frac{d\zeta}{n^\star} \lesssim \frac{n^\star}{n_j}\frac{Bd\zeta}{N_\mathbf{S}}.\] Hence, in either subcase, the safety rate obeys \[q^2\frac{d\zeta}{n_j} \lesssim \frac{n^\star}{n_j}\frac{Bd\zeta}{N_\mathbf{S}} + \frac{1}{n_j}\min\left(\alpha^2B^2\delta_0^2,\;q^2d\zeta\right).\]

On the already-defined event \(\mathcal{E}_{\mathrm{safe}}^{\mathrm{gs}}\), the personalization bound with \(\lambda_j=\lambda/\sqrt{n_j}\) gives \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j} \lesssim \lambda_j + \sqrt{\frac{d\zeta}{n_j}} \lesssim q\sqrt{\frac{d\zeta}{n_j}},\] so \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim q^2\frac{d\zeta}{n_j}.\] Combining the last two displays yields \[\|\hat{\theta}_j - \theta_j^\star\|_{\boldsymbol{\Sigma}_j}^2 \lesssim \frac{n^\star}{n_j}\frac{Bd\zeta}{N_\mathbf{S}} + \frac{1}{n_j}\min\left(\alpha^2B^2\delta_0^2,\;q^2d\zeta\right).\] Adding the nonnegative outlier-robustness term \(\varepsilon^2B^2\lambda^2/n^\star\) gives the claimed bound in part (b). ◻

Proof. We again consider the high-probability event \(\mathcal{E}_{\text{samples}}\) where all samples satisfy \(\|x_i\|_2 \le K\). Conditioned on this event, we essentially sample from \(\check{\mathcal{P}}\). By applying a one-sided Matrix Bernstein inequality (or similar concentration bounds for bounded random vectors as in Lemma 7), we obtain: \[\begin{align} \check{\boldsymbol{\Sigma}} \preceq c'\left(\boldsymbol{\Sigma}+ \frac{K^2}{n}\mathbf{I}\right) \end{align}\] for some constant \(c' > 1\). From Lemma 9, we have \(\bar{\boldsymbol{\Sigma}} \preceq 2\check{\boldsymbol{\Sigma}}\). Combining these yields the result. ◻

Proof. Let \(E\) be the indicator event \(\{\|x\|_2 \le K\}\). We can decompose the expectation: \[\begin{align} \bar{\boldsymbol{\Sigma}} = \mathbb{E}[xx^\top] = \mathbb{P}(E)\mathbb{E}[xx^\top \mid E] + \mathbb{E}[xx^\top \mathbb{I}(E^c)] = \mathbb{P}(E)\check{\boldsymbol{\Sigma}} + \Delta. \end{align}\] We bound the residual matrix \(\Delta = \mathbb{E}[xx^\top \mathbb{I}(\|x\|_2 > K)]\) using a peeling argument. Define shells \(S_j := \{ x : K + (j-1)U < \|x\|_2 \le K + jU \}\) for \(j \ge 1\). \[\begin{align} \Delta &\preceq \mathbb{E}[ \|x\|_2^2 \mathbb{I}(\|x\|_2 > K) ] \mathbf{I}\\ &= \sum_{j=1}^\infty \mathbb{E}[ \|x\|_2^2 \mathbb{I}(x \in S_j) ] \mathbf{I}\\ &\preceq \sum_{j=1}^\infty (K + jU)^2 \mathbb{P}(\|x\|_2 > K + (j-1)U) \mathbf{I}. \end{align}\] Using the tail assumption \(\mathbb{P}(\|x\|_2 > tU) \le e^{-ct}\), we have: \[\begin{align} \mathbb{P}(\|x\|_2 > K + (j-1)U) \le \exp\left(-c \frac{K + (j-1)U}{U}\right) = e^{-cK/U} e^{-c(j-1)}. \end{align}\] Substituting this back: \[\begin{align} \|\Delta\|_2 &\le e^{-cK/U} \sum_{j=1}^\infty (K+jU)^2 e^{-c(j-1)}. \end{align}\] Since \(K \gg U\), the term dominates for small \(j\). Recall \(K = A U \log(nU/\gamma)\), so \(e^{-cK/U} = (\frac{\gamma}{nU})^{cA}\). For sufficiently large \(A\), the pre-factor \((\frac{\gamma}{nU})^{cA}\) suppresses the polynomial terms in the sum, ensuring: \[\begin{align} \|\Delta\|_2 \le \frac{\gamma}{2}. \end{align}\] Thus, \(\mathbb{P}(E)\check{\boldsymbol{\Sigma}} \succeq \bar{\boldsymbol{\Sigma}} - \frac{\gamma}{2}\mathbf{I}\succeq \frac{\gamma}{2}\mathbf{I}\). Since \(\mathbb{P}(E) \le 1\), this implies \(\check{\boldsymbol{\Sigma}} \succeq \frac{\gamma}{2}\mathbf{I}\). ◻

Proof. For any \(\theta, \theta' \in \mathbf{C}\) and any \(\varepsilon>0\), there exists \(x_0 \in \mathbf{C}\) such that \[\begin{align} g(\theta') +\varepsilon > f(x_0) + R(\theta'-x_0). \end{align}\] Then we have \[\begin{align} &g(\theta) - g(\theta') \le f(x_0) + R(\theta -x_0) -(f(x_0) + R(\theta' -x_0) - \varepsilon) \\ &\le R(\theta -x_0) -R(\theta' -x_0) +\varepsilon \\ &\le L\|\theta -\theta' \| +\varepsilon. \end{align}\] Since \(\varepsilon\) is arbitrary, we get the wanted result. ◻

Proof. Fix any minimizer \(\hat{x}\in\arg\min_{x\in\mathbf{C}}\{f(x)+g(x)\}\), and set \(t:=\|\hat{x}-x_0\|_2\). If \(t=0\), the claim is immediate. Let \(u:=(\hat{x}-x_0)/t\) and \(s:=\min\{t,r\}\). Since \(\mathbf{C}\) is convex, the segment \(x_0+a u\), \(0\le a\le t\), lies in \(\mathbf{C}\). The one-dimensional function \(\phi(a):=f(x_0+a u)\) is convex on \([0,t]\), and satisfies \(\phi''(a)\ge \rho\) for \(0\le a\le s\). Therefore \(\phi'(t)-\phi'(0)\ge \rho s\), which gives \[\bigl\langle \nabla f(\hat{x})-\nabla f(x_0),\,\hat{x}-x_0\bigr\rangle \ge \rho\min\{t,r\}\,t.\] By the constrained first-order optimality condition for \(\hat{x}\), there exist \(v\in\partial g(\hat{x})\) and \(n\in N_{\mathbf{C}}(\hat{x})\) such that \[\nabla f(\hat{x})+v+n=0.\] Since \(x_0\in\mathbf{C}\), \(\langle n,x_0-\hat{x}\rangle\le0\). Since \(x_0\) minimizes the convex differentiable function \(f\) over \(\mathbf{C}\), we also have \(\langle\nabla f(x_0),\hat{x}-x_0\rangle\ge0\). Thus \[\bigl\langle \nabla f(\hat{x})-\nabla f(x_0),\,\hat{x}-x_0\bigr\rangle \le \langle -v,\hat{x}-x_0\rangle \le \|v\|_2\,t \le \lambda' t,\] where the last inequality uses the \(\lambda'\)-Lipschitz property of \(g\). Combining the two displays and dividing by \(t>0\), we obtain \[\rho\min\{t,r\}\le \lambda'.\] Because \(\lambda'<\rho r\), the minimum cannot be \(r\), and hence \(t<r\). Therefore \(\rho t\le \lambda'\), proving \(t\le \lambda'/\rho\). ◻

Proof. Observe that \[\begin{align} f(y) + \lambda \|x - y\|_2 &\ge f(x) + \bigl\langle \nabla f(x),\,y - x \bigr\rangle + \lambda \|x - y\|_2\\ &\ge f(x) + \bigl(\lambda - \|\nabla f(x)\|_2\bigr)\|x - y\|_2 \;>\;f(x). \end{align}\] ◻

Proof. Fix any \(y\in\mathbf{C}\). By convexity and differentiability of \(f\), \[f(y)\ge f(x)+\langle\nabla f(x),y-x\rangle.\] Therefore \[f(y)+\lambda\|x-y\|_2 \ge f(x)+\langle\nabla f(x),y-x\rangle+\lambda\|x-y\|_2 \ge f(x)+\bigl(\lambda-\|\nabla f(x)\|_2\bigr)\|x-y\|_2.\] If \(y\ne x\), the last term is strictly larger than \(f(x)\) because \(\lambda>\|\nabla f(x)\|_2\). For \(y=x\), the objective value is exactly \(f(x)\). Hence \(x\) is the unique minimizer over \(\mathbf{C}\), and the constrained infimal convolution value at \(x\) equals \(f(x)\). ◻