1 Introduction↩︎

This study investigates efficient causal parameter estimation in a semi-supervised setting. The estimation targets are causal parameters defined via functionals of regression functions, including the average treatment effect (ATE), the average marginal effect (AME), and the average policy effect (APE) as special cases. We aim to construct estimators of these causal parameters with smaller variances by using an auxiliary unlabeled dataset of regressors, in addition to a conventional labeled dataset containing outcomes and regressors. This setup is closely related to the literature on prediction-powered inference [1], but our goal is to estimate causal parameters. Hence, we refer to this framework as prediction-powered causal inference (PPCI). We show that, by using such an auxiliary unlabeled dataset, we can construct estimators whose asymptotic variances are smaller than those of estimators constructed using only the labeled dataset.

Efficient estimation of causal parameters has been a core interest in causal inference. To discuss efficiency, we consider the asymptotic efficiency bound, also called the semiparametric efficiency bound or the Hájek–Le Cam bound [2], [3], which characterizes the theoretically best asymptotic variance among regular estimators. We refer to causal parameter estimators whose asymptotic variances attain the efficiency bound as asymptotically efficient estimators. Asymptotically efficient estimators yield accurate estimation of causal parameters in terms of asymptotic mean squared error and tight confidence intervals. Therefore, constructing asymptotically efficient estimators is a standard goal in causal parameter estimation.

In this study, we consider the semi-supervised setup and develop efficiency bounds and asymptotically efficient estimators for this setup. The efficiency bound has been intensively studied in the standard setup with only labeled data, while it has not been fully studied in settings where unlabeled data can be used for general causal and structural parameters represented as functionals of regression functions. In this study, we show that if such unlabeled data are used appropriately, then we can construct estimators whose asymptotic variances are smaller than those of estimators constructed without using unlabeled data.

For example, consider estimating the effect of a new medicine in clinical trials. If population characteristics of patients without outcomes are available in addition to clinical trial data, then these unlabeled patient characteristics provide information about the regressor distribution over which the causal parameter is averaged. This information reduces the regressor-averaging component of the efficiency bound, even though it does not reduce the conditional outcome-noise component. This result shows that efficiency gains can arise from data that contain no outcome information, provided that such data inform the regressor distribution over which the target functional is averaged. Such approaches can be understood as semi-supervised causal inference or prediction-powered inference for causal inference.

In constructing asymptotically efficient estimators, we propose debiased machine learning (DML)-PPCI. DML is a framework for constructing asymptotically efficient estimators through Neyman orthogonal scores. Several approaches can be used in DML and in more general constructions of asymptotically efficient estimators. In this study, among them, we focus on estimating equations and targeted maximum likelihood (TMLE) [4]–[6]. We refer to DML-PPCI based on the estimating-equation approach as EE-DML-PPCI, and we refer to DML-PPCI based on TMLE as TMLE-DML-PPCI. For these estimators, we show asymptotic efficiency by proving that their asymptotic variances match our derived efficiency bounds.

1.1 Contribution↩︎

We summarize our contributions. First, we formulate PPCI under the two-sample and one-sample scenarios within the automatic DML (ADML) framework. Second, we derive the efficient influence functions and asymptotic efficiency bounds in our semi-supervised setup. Third, we construct asymptotically efficient estimators whose asymptotic variances match the derived efficiency bound. Fourth, we develop a semi-supervised version of generalized Riesz regression for estimating the Riesz representer appearing in the Neyman orthogonal score.

1.1.0.1 Problem formulation.

In the problem formulation, we define our parameters of interest as functionals of regression functions, as in ADML [7]. This framework covers causal and structural parameters that can be written as functionals of regression functions, including ATE, AME, and APE. In addition, we define the data-generating process (DGP) using the two-sample and one-sample scenarios [8], [9]. In the two-sample scenario, we assume that there exist two independent datasets, while in the one-sample scenario, we assume that there exists one dataset and that, from this dataset, we can observe labeled data. The two-sample scenario is closely related to the stratified sampling scheme [10], [11], while the one-sample scenario is closely related to the missing-value literature [12], [13].

1.1.0.2 Efficiency bound.

For this setup, we derive the efficiency bounds, which can be computed from the efficient influence function. The efficiency bounds depend on the DGP. Therefore, for the two-sample and one-sample scenarios, we derive the corresponding efficiency bounds separately [14]. We show that using an auxiliary unlabeled dataset can reduce the asymptotic variance compared with the case where only the labeled dataset is used. The efficiency bound implies the possibility of an efficiency gain, and feasibility is confirmed by constructing estimators whose asymptotic variances match the efficiency bounds.

1.1.0.3 Asymptotically efficient estimators.

We refer to our estimation method as DML-PPCI. We develop an estimating-equation version, called EE-DML-PPCI, and a targeted maximum likelihood version, called TMLE-DML-PPCI. We show that these estimators have asymptotic variances matching the efficiency bounds derived in this study. Therefore, these estimators are asymptotically efficient [15].

1.1.0.4 Semi-supervised generalized Riesz regression.

In DML-PPCI, we need to estimate the efficient influence function. The efficient influence function in our setup depends on two nuisance parameters, the regression function and the Riesz representer. To estimate the Riesz representer, we employ Riesz regression proposed by [7] or its generalization, generalized Riesz regression, proposed by [16]. Note that, under some restrictions on Riesz representer models, Riesz regression is mathematically equivalent to covariate balancing [17], [18]. We extend generalized Riesz regression to the semi-supervised setup so that an unlabeled dataset is also used to estimate the Riesz representer. This extension can be interpreted as providing an implementable nuisance-parameter estimation method for prediction-powered causal inference. For semi-supervised generalized Riesz regression, we provide convergence rates, which cover finite pseudo-dimension classes and deep ReLU sieves, including the Hölder-smooth case, the unbounded-support case, and the approximate low-dimensional manifold case. We then translate these Riesz representer rates into sufficient conditions for the product-rate requirement in DML-PPCI.

1.2 Related Work↩︎

This study is closely related to asymptotic efficiency theory and semi-supervised learning.

Asymptotic efficiency theory aims to construct estimators whose asymptotic variances match the asymptotic efficiency bound, especially in the sense of the semiparametric efficiency bound or the Hájek–Le Cam asymptotic efficiency bound. Various approaches and techniques have been proposed, such as estimating equations, TMLE, and sample splitting [19]. The DML framework organizes these approaches by introducing Neyman orthogonal scores and cross-fitting [20]. The ADML framework is a generalization of DML that deals with many causal and structural parameters written as functionals of a regression function and the corresponding Riesz representer. Under the ADML framework, even if we do not know the closed form of the efficient influence function or the Neyman orthogonal score, we can estimate it using Riesz regression [7], which is related to semiparametric and sieve Riesz modeling [21], [22] and density-ratio estimation [23].

Semi-supervised learning studies how labeled and unlabeled data can be combined to improve learning [24], [25]. Covariate-shift methods are a variant of this setup and use information on a target covariate distribution to improve prediction or estimation under distributional changes [26]. [27] proposes prediction-powered inference as a framework for using machine-learning predictions together with gold-standard data to conduct valid inference. This idea has already been extended to causal inference by studies such as [28] and [1]. However, this study differs from these studies in its focus. We derive the semiparametric efficiency bound for causal and structural regression functionals when the auxiliary observations are unlabeled regressors and then construct ADML-type estimators that attain this bound.

Our study is also related to the literature on semi-supervised regression. [29] studies best linear approximation under misspecification and shows that unlabeled regressors can improve the asymptotic variance of least-squares-type estimators when nonlinear features of the conditional mean interact with the marginal regressor distribution. [30] analyzes semi-supervised regression through minimax theory and shows that unlabeled data do not automatically improve rates, especially for graph-Laplacian regularization. They emphasize that rate improvements require assumptions that connect the regression function and the regressor distribution. We use this insight to state primitive Riesz representer rates only under explicit smoothness, tail, or low-dimensional-structure conditions.

For estimating the Riesz representer, we employ Riesz regression. Note that there is a duality between Riesz regression and covariate balancing [18], [31]–[35], and based on this relationship, [16] develops generalized Riesz regression that formulates Riesz representer estimation as Riesz representer fitting under the Bregman divergence, which incorporates various existing methods such as Riesz regression, calibrated estimation [36], tailored loss minimization [18], density-ratio estimation, and covariate balancing. In this study, we extend generalized Riesz regression to the semi-supervised setting so that we also utilize unlabeled datasets.

The convergence analysis for semi-supervised generalized Riesz regression is closely related to density-ratio estimation under the Bregman divergence [37]–[39]. [37] establishes non-asymptotic error bounds for density-ratio estimation with deep ReLU feedforward neural networks, including minimax-optimal rates up to logarithmic factors under Hölder smoothness, extensions to unbounded support, and rates under approximate low-dimensional manifold structure. We use their result to derive error rates for our proposed semi-supervised generalized Riesz regression.

The stratified sampling scheme plays an important role in our theoretical analysis [10]. When the labeled and unlabeled datasets are independent samples from different strata with fixed sampling proportions, the usual one-sample efficiency bound is not directly applicable. Closely related work includes off-policy evaluation and learning for external validity under covariate shift [14] and DML for covariate shift [40], [41].

This study builds on our previous studies [14], [16], [41]–[43]. In particular, the idea of asymptotic efficiency under stratified sampling is inspired by [14]. An early version of PPCI is presented in [42] for adaptive experiments. Compared with those studies, we consider a general class of regression-functional causal and structural parameters, whereas those studies focus on specific applications, such as policy evaluation or adaptive experimental design under covariate shift.

2 Setup↩︎

Let \(Y \in {\mathbb{R}}\) be an outcome and let \(X,{\widetilde{X}}\in{\mathcal{X}}\) be regressors, where \({\mathcal{X}}\) is the regressor space. We observe the outcome \(Y\) for \(X\), but we do not observe the corresponding outcome for \(\widetilde{X}\). Therefore, we refer to \(W=(X,Y)\) as labeled data and \(\widetilde{X}\) as unlabeled data. In applications, unlabeled data are often test data or evaluation data for which the chosen treatment or policy will be implemented.

2.1 Parameter of Interest↩︎

Our goal is to estimate a parameter of the form \[\begin{align} \theta_0 \mathrel{\vcenter{:}}= {\mathbb{E}}_{V_{0X}}\left[m(X,\gamma_0)\right] \mathrel{\vcenter{:}}= \int m(x,\gamma_0) {\mathrm{d}}V_{0X}(x), \label{eq:target-parameter} \end{align}\tag{1}\] where \(\gamma_0(x)\mathrel{\vcenter{:}}={\mathbb{E}}_{P_0}\left[Y\mid X=x\right]\) and \(m(X,\gamma)\) is a known functional map.

The distribution \(V_{0X}\) is the marginal regressor distribution used to evaluate the target parameter. For simplicity, we assume that \(V_{0X}\) has the density \[\begin{align} v_{0X}(x) \mathrel{\vcenter{:}}= \kappa p_{0X}(x)+(1-\kappa)q_{0X}(x), \qquad \kappa\in(0,1). \label{eq:evaluation-density} \end{align}\tag{2}\] We assume that the constant \(\kappa\) is known. It is distinct from the labeled sampling proportion \(\rho = n/(n + m)\) introduced in Section 3.

We define the parameter of interest as a functional of the regression function, which includes various causal parameters as special cases. We give examples below.

2.2 Observations↩︎

In this study, we consider two different sampling schemes (DGPs): the one-sample scenario and the two-sample scenario.

2.2.0.1 One-sample scenario.

In this scenario, there is a potential complete dataset \[\begin{align} \left\{\left(X^*_k,Y^*_k\right)\right\}^N_{k=1}. \end{align}\] Labeling then occurs. For each \(k=1,2,\dots,N\), let \(S_k\in\{1,0\}\) be a labeling indicator: if \(S_k=1\), then \(Y^*_k\) is observed; otherwise, \(Y^*_k\) is missing. Therefore, we observe the dataset \[\begin{align} \left\{\left(X_k,S_k,Y_k\right)\right\}^N_{k=1}, \end{align}\] where \[\begin{align} Y_k=S_kY^*_k+(1-S_k)\mathrm{NA}, \end{align}\] and \(\mathrm{NA}\) denotes a missing value. In this case, the labeled and unlabeled datasets are \[\begin{align} \text{(labeled dataset)} &\left\{(X_k,Y_k)\mid S_k=1,\;k=1,2,\dots,N\right\},\\ \text{(unlabeled dataset)} &\left\{{\widetilde{X}}_k=X_k\mid S_k=0,\;k=1,2,\dots,N\right\}, \end{align}\] whose sample sizes are \(n=\sum^N_{k=1}S_k\) and \(m=\sum^N_{k=1}(1-S_k)\), respectively.

2.2.0.2 Two-sample scenario.

We observe the following two independent stratified datasets, which we call the labeled dataset and the unlabeled dataset: \[\begin{align} \text{(labeled dataset)} &\left\{(X_i,Y_i)\right\}_{i=1}^n,\\ \text{(unlabeled dataset)} &\left\{{\widetilde{X}}_j\right\}_{j=1}^m. \end{align}\]

2.2.0.3 Comparison.

The one-sample scenario treats labeling as a selection process. In contrast, the two-sample scenario considers a setting in which the usual observations \((X_i,Y_i)\) are available and auxiliary observations \({\widetilde{X}}_j\) are independently observed. This difference leads to slightly different approaches to efficiency analysis and different asymptotic regimes. Therefore, we consider the two cases separately.

2.3 Goal↩︎

Our goal is to estimate the parameter of interest \(\theta_0\) efficiently from the observations. Here, efficiency means that the asymptotic variance of an estimator matches the asymptotic lower bound for regular estimators. This lower bound is called the Hájek–Le Cam asymptotic efficiency bound, and its semiparametric extension is often called the semiparametric efficiency bound.

It is known that if an estimator is regular and asymptotically linear for the efficient influence function, then the estimator is efficient. In many cases, such an estimator is constructed from the efficient influence function by solving an estimating equation or by using TMLE. Therefore, in our methodological and theoretical analysis, we start by deriving the efficiency bound, then propose estimators of \(\theta_0\), and then show their asymptotic efficiency.

In the subsequent sections, since the two-sample scenario is closer to the standard setting in semi-supervised learning, we first introduce the efficiency bound and efficient estimators under the two-sample scenario. Then, we introduce the corresponding efficiency bound and efficient estimators under the one-sample scenario.

2.3.0.1 Notation.

Let \(\sigma^2_0(x)\mathrel{\vcenter{:}}=\mathrm{Var}(Y\mid X=x)\), where the variance is taken over the conditional distribution \(P_0\) given \(X=x\). Let \(\|f\|_{P,2}\mathrel{\vcenter{:}}= \left({\mathbb{E}}_{P_0}\left[f(X)^2\right]\right)^{1/2}\) and \(\|f\|_{Q,2}\mathrel{\vcenter{:}}= \left({\mathbb{E}}_{Q_{0X}}\left[f\left({\widetilde{X}}\right)^2\right]\right)^{1/2}\). For applications with discrete treatments, let \(e_0(d\mid z)\mathrel{\vcenter{:}}= P_0(D=d\mid Z=z)\). Let \(r_{0X}(x)\mathrel{\vcenter{:}}= v_{0X}(x)/p_{0X}(x)\) be the density ratio between the evaluation and labeled regressor distributions for \(X\). When \(X=(D,Z)\), let \(r_{0Z}(z)\mathrel{\vcenter{:}}= v_{0Z}(z)/p_{0Z}(z)\).

3 Efficiency Bound and Efficient Influence Function in the Two-Sample Scenario↩︎

This section derives the semiparametric efficiency bound under the two-sample scenario, where two independent datasets are observed.

3.1 Detailed DGP↩︎

We define the DGP of the two-sample scenario more precisely. We assume that \(W\mathrel{\vcenter{:}}=(X,Y)\) follows a distribution \(P_0\), and let \(P_{0X}\) be the marginal distribution of \(X\). We also assume that \({\widetilde{X}}\) follows a distribution \(Q_{0X}\). For simplicity, we assume that \(P_{0X}\) and \(Q_{0X}\) have probability density functions \(p_{0X}\) and \(q_{0X}\) with respect to a common dominating measure. This assumption can be relaxed by defining the corresponding Radon–Nikodym derivatives with respect to appropriate dominating measures.

3.1.0.1 DGP.

We observe the following two independent stratified datasets, which we call the labeled dataset and the unlabeled dataset: \[\begin{align} \text{(labeled dataset)}\quad &\{W_i\}_{i=1}^n \mathrel{\vcenter{:}}= \left\{(X_i,Y_i)\right\}_{i=1}^n \stackrel{\mathrm{i.i.d}}{\sim} P_0,\\ \text{(unlabeled dataset)}\quad &\left\{{\widetilde{X}}_j\right\}_{j=1}^m \stackrel{\mathrm{i.i.d}}{\sim} Q_{0X}, \end{align}\] where \(W_i=(X_i,Y_i)\) is an independent copy of \(W=(X,Y)\), and \({\widetilde{X}}_j\) is an independent copy of \({\widetilde{X}}\).

Let \(N\mathrel{\vcenter{:}}= n+m\). We denote the sampling proportion of the labeled stratum by \(\rho\mathrel{\vcenter{:}}= n/N\in(0,1)\). In the asymptotic analysis, \(\rho\) is fixed by design. Thus, we consider a deterministic sequence of admissible total sample sizes for which \(n=\rho N\) and \(m=(1-\rho)N\) are integers, and we take the limit \(N\to\infty\) along this sequence.

3.1.0.2 Parameter of interest.

Under this setup, the target parameter can be decomposed as \[\begin{align} \theta_0 = {\mathbb{E}}_{V_0}\left[m(X,\gamma_0)\right] = \kappa\theta_{P,0}+(1-\kappa)\theta_{Q,0}, \end{align}\] where \[\begin{align} \theta_{P,0}\mathrel{\vcenter{:}}= {\mathbb{E}}_{P_{0X}}\left[m(X,\gamma_0)\right], \qquad \theta_{Q,0}\mathrel{\vcenter{:}}= {\mathbb{E}}_{Q_{0X}}\left[m({\widetilde{X}},\gamma_0)\right]. \end{align}\]

3.2 Efficient Influence Function↩︎

This section derives the efficient influence function, from which we obtain both the efficiency bound and the efficient estimators. Since one of the key components in the efficient influence function is the Riesz representer, we first introduce the Riesz representer and then derive the efficient influence function.

3.2.0.1 Riesz representer.

In our parameter of interest, the efficient influence function depends on the Riesz representer, which is obtained by applying the Riesz representation theorem to the parameter functional. Let \(\Gamma\subseteq L^2(P_{0X})\) be the regression space under consideration. Assume that the linear functional \[\begin{align} {\mathcal{L}}_{0,\kappa}(\gamma) \mathrel{\vcenter{:}}= {\mathbb{E}}_{V_0}\left[m(X,\gamma)\right] \end{align}\] is continuous on \(\Gamma\) under the \(L^2(P_{0X})\) norm. Then, by the Riesz representation theorem, there exists a unique \(\alpha_{0,\kappa}\in\overline{\Gamma}\subseteq L^2(P_{0X})\) such that \[\begin{align} {\mathcal{L}}_{0,\kappa}(\gamma) = {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)\gamma(X)\right] \qquad \text{for all } \gamma\in\Gamma . \label{eq:ss-riesz} \end{align}\tag{3}\] This representer generally depends on the evaluation density \(v_{0X}\). We can estimate the Riesz representer without obtaining a closed-form expression. For reference, we present representative Riesz representers below for several cases raised in Example 1.

3.2.0.2 Efficient influence function.

Using the Riesz representer, we derive the efficient influence function.

The following theorem holds. The proof is given in the Appendix.

Because the data are sampled from two strata, the efficient influence function is a pair of stratum-specific influence functions. The first component corresponds to the labeled stratum \(W=(X,Y)\sim P_0\), and the second component corresponds to the unlabeled stratum \({\widetilde{X}}\sim Q_{0X}\).

Both influence functions are centered within their own strata: \[\begin{align} {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)\right]=0, \qquad {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\left({\widetilde{X}}\right)\right]=0. \end{align}\]

3.3 Efficiency Bound↩︎

The stratum-specific efficient influence functions imply the following efficiency bound.

The first term in ?? comes from the outcome residual and is governed by the labeled sample because only labeled observations contain \(Y\). The second and third terms are regressor averaging components. The second term is associated with the labeled regressor distribution, and the third term is associated with the auxiliary regressor distribution. Thus, unlabeled regressors improve efficiency by reducing the noise in the regressor averaging part of the target, but they do not reduce the outcome residual component.

3.4 Asymptotically Efficient Estimators↩︎

Based on the derived efficiency bound and efficient influence functions, we construct estimators whose asymptotic variances match the efficiency bound. The estimators are based on the stratum-specific efficient score \[\begin{align} \psi^{\mathrm{TS}}_0(W) &= \alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right) +\kappa\left(m\big(X,\gamma_0\big)-\theta_{P,0}\right), \\ \widetilde{\psi}^{\mathrm{TS}}_0\left({\widetilde{X}}\right) &=(1-\kappa)\left(m\big({\widetilde{X}},\gamma_0\big)-\theta_{Q,0}\right). \end{align}\] We propose two estimator constructions: an estimating-equation estimator and a TMLE estimator. We refer to the overall method as DML-PPCI and specify these two estimators as EE-DML-PPCI and TMLE-DML-PPCI. Both estimators use the same nuisance estimators: the regression function \(\gamma_0\) and the Riesz representer \(\alpha_{0,\kappa}\). If these nuisance estimators do not satisfy the Donsker condition, then we also apply cross-fitting to control the empirical process terms.

3.4.0.1 EE-DML-PPCI.

Given estimators \(\widehat{\gamma}\) and \(\widehat{\alpha}\) of the nuisance parameters \(\gamma_0\) and \(\alpha_{0,\kappa}\), we construct an estimator as \[\widehat{\theta} \mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1}\left( \widehat{\alpha}(X_i)\left(Y_i-\widehat{\gamma}(X_i)\right) +\kappa m\big(X_i,\widehat{\gamma}\big) \right) +(1-\kappa)\frac{1}{m}\sum^m_{j=1}m\big({\widetilde{X}}_j,\widehat{\gamma}\big).\] This estimator decomposes as \(\widehat{\theta}=\widehat{\theta}_{\mathrm{L}}+\widehat{\theta}_{\mathrm{U}}\), where the labeled and unlabeled parts are \[\begin{align} \widehat{\theta}_{\mathrm{L}} \mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1}\left( \widehat{\alpha}(X_i)\left(Y_i-\widehat{\gamma}(X_i)\right) +\kappa m\big(X_i,\widehat{\gamma}\big) \right), \qquad \widehat{\theta}_{\mathrm{U}} \mathrel{\vcenter{:}}= (1-\kappa)\frac{1}{m}\sum^m_{j=1}m\big({\widetilde{X}}_j,\widehat{\gamma}\big). \end{align}\] The labeled part \(\widehat{\theta}_{\mathrm{L}}\) estimates \(\kappa\theta_{P,0}\) through the labeled estimating equation, and the unlabeled part \(\widehat{\theta}_{\mathrm{U}}\) estimates \((1-\kappa)\theta_{Q,0}\) from the unlabeled regressors. We also refer to this estimator as the augmented Riesz-weighted (ARW) estimator, which is a generalization of the augmented inverse probability weighting (AIPW) estimator.

3.4.0.2 TMLE-DML-PPCI.

Next, we propose the TMLE version of DML-PPCI, which is a regression-adjustment estimator with a bias-corrected regression-function estimator. Let \(\widehat\gamma\) and \(\widehat{\alpha}\) be initial estimators of \(\gamma_0\) and \(\alpha_{0,\kappa}\). TMLE first updates the initial regression-function estimator \(\widehat\gamma\) in the direction of the estimated Riesz representer and then plugs the updated regression-function estimator \(\widehat\gamma^{(1)}\) into the semi-supervised target functional.

In our semi-supervised setup, the targeting step is calibrated by the same evaluation distribution that appears in the target functional. For example, under a Gaussian likelihood, we update the initial estimator as \[\begin{align} \widehat\gamma^{(1)}(x) = \widehat\gamma(x)+\widehat\varepsilon\widehat\alpha(x), \end{align}\] where the fluctuation is given by \[\begin{align} \widehat\varepsilon \mathrel{\vcenter{:}}= \frac{ \frac{1}{n}\sum^n_{i=1} \widehat\alpha(X_i)\left(Y_i-\widehat\gamma(X_i)\right) }{ \kappa \frac{1}{n}\sum^n_{i=1} \widehat\alpha^2\big(X_i\big) +(1-\kappa)\frac{1}{m}\sum^m_{j=1} \widehat\alpha^2\big(\widetilde{X}_j\big)}. \label{eq:tmle-epsilon} \end{align}\tag{4}\] This update follows the same logic as density-ratio-corrected likelihood maximization under covariate shift [26]. The one-dimensional fluctuation is fitted for the regressor law under which the target functional is evaluated. The numerator is computed from the labeled residuals because outcomes are observed only in the labeled sample, and the density-ratio adjustment is already contained in the estimated Riesz representer \(\widehat\alpha\). The denominator estimates the target law average of \(\widehat\alpha^2\).

Then, we obtain the TMLE-DML-PPCI estimator as the following plug-in estimator: \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{TMLE}} &\mathrel{\vcenter{:}}= \kappa \frac{1}{n}\sum^n_{i=1} m\left(X_i,\widehat\gamma^{(1)}\right) +(1-\kappa)\frac{1}{m}\sum^m_{j=1} m\left(\widetilde{X}_j,\widehat\gamma^{(1)}\right). \label{eq:tmle-dml-ppci-estimator} \end{align}\tag{5}\] This construction is a semi-supervised analogue of Auto-TML in [7].

3.4.0.3 Algorithm.

We summarize the procedures for EE-DML-PPCI and TMLE-DML-PPCI in Algorithm 1. In the pseudocode below, we use cross-fitting to apply the proposed framework when Donsker-type conditions are not imposed on the nuisance estimators.

In the pseudocode, we split the datasets into \(K\) subsets, where \(K\ge 2\) is a fixed integer. Partition the labeled indices \([n]\) into \(K\) folds \({\mathcal{I}}_1,\ldots,{\mathcal{I}}_K\) and the unlabeled indices \([m]\) into \(K\) folds \({\mathcal{J}}_1,\ldots,{\mathcal{J}}_K\). We take the folds to be balanced in the sense that \(|{\mathcal{I}}_k|/n=|{\mathcal{J}}_k|/m=1/K+o(1)\). Write \[\begin{align} {\mathcal{I}}_{-k}\mathrel{\vcenter{:}}= [n]\setminus {\mathcal{I}}_k, \qquad {\mathcal{J}}_{-k}\mathrel{\vcenter{:}}= [m]\setminus {\mathcal{J}}_k. \end{align}\] For each fold \(k\), estimate the nuisance functions using only observations in \({\mathcal{I}}_{-k}\) and \({\mathcal{J}}_{-k}\). Denote the resulting estimators by \[\begin{align} \widehat\gamma_k, \qquad \widehat\alpha_k. \end{align}\] The fitted nuisance functions are then evaluated only on the held-out fold. The residual \(Y_i-\widehat\gamma_k(X_i)\) is evaluated only for \(i\in {\mathcal{I}}_k\), whereas the plug-in terms \(m(X_i,\widehat\gamma_k)\) and \(m({\widetilde{X}}_j,\widehat\gamma_k)\) are evaluated on the held-out labeled and unlabeled folds, respectively.

The cross-fitted EE-DML-PPCI estimator is \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{EE}} &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^K_{k=1}\sum_{i\in {\mathcal{I}}_k} \Big( \widehat\alpha_k(X_i)\left(Y_i-\widehat\gamma_k(X_i)\right) +\kappa m\left(X_i,\widehat\gamma_k\right) \Big) +\frac{1-\kappa}{m}\sum^K_{k=1}\sum_{j\in {\mathcal{J}}_k} m\left({\widetilde{X}}_j,\widehat\gamma_k\right). \label{eq:ee-dml-ppci-crossfit} \end{align}\tag{6}\] The cross-fitted TMLE-DML-PPCI estimator uses the fold-specific fluctuation \[\begin{align} \widehat\varepsilon_k \mathrel{\vcenter{:}}= \frac{ \frac{1}{|{\mathcal{I}}_k|}\sum_{i\in {\mathcal{I}}_k} \widehat\alpha_k(X_i)\left(Y_i-\widehat\gamma_k(X_i)\right) }{ \kappa \frac{1}{|{\mathcal{I}}_k|}\sum_{i\in {\mathcal{I}}_k} \widehat\alpha_k^2(X_i) +(1-\kappa)\frac{1}{|{\mathcal{J}}_k|}\sum_{j\in {\mathcal{J}}_k} \widehat\alpha_k^2\left(\widetilde{X}_j\right)}. \label{eq:tmle-epsilon-crossfit} \end{align}\tag{7}\] Using the fluctuation, we set \(\widehat\gamma_k^{(1)}(x)=\widehat\gamma_k(x)+\widehat\varepsilon_k\widehat\alpha_k(x)\), and obtain \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{TMLE}} &\mathrel{\vcenter{:}}= \frac{\kappa}{n}\sum^K_{k=1}\sum_{i\in {\mathcal{I}}_k}m\left(X_i,\widehat\gamma_k^{(1)}\right) + \frac{1-\kappa}{m}\sum^K_{k=1}\sum_{j\in {\mathcal{J}}_k}m\left({\widetilde{X}}_j,\widehat\gamma_k^{(1)}\right). \label{eq:tmle-dml-ppci-crossfit} \end{align}\tag{8}\] If the nuisance classes satisfy the required Donsker-type empirical-process conditions, the estimator can be constructed without sample splitting by estimating the nuisance functions on the full sample and evaluating the same score on the full sample.

3.5 Nuisance Parameter Estimation↩︎

3.5.0.1 Riesz representer estimation.

We estimate the Riesz representer using semi-supervised generalized Riesz regression, which we propose in this study. See Section 5 for details. Here, we briefly describe the fold-specific version used for DML-PPCI with cross-fitting. For a candidate regression function \(\gamma\in\Gamma\), define the training-sample linear functional \[\begin{align} \widehat{{\mathcal{L}}}^{(-k)}_{\kappa}(\gamma) \mathrel{\vcenter{:}}= \frac{\kappa}{|{\mathcal{I}}_{-k}|}\sum_{i\in {\mathcal{I}}_{-k}}m(X_i,\gamma) + \frac{1-\kappa}{|{\mathcal{J}}_{-k}|}\sum_{j\in {\mathcal{J}}_{-k}}m({\widetilde{X}}_j,\gamma). \label{eq:dml-training-linear-functional} \end{align}\tag{9}\] Given a convex Bregman generator \(g\), the fold-specific semi-supervised generalized Riesz regression estimator is \[\begin{align} \widehat\alpha_k \in \arg\min_{\alpha\in{\mathcal{H}}_n} \left\{ \widehat{{\mathcal{Q}}}^{(-k)}_{g,\kappa}(\alpha) +\lambda_{\alpha,n}Reg_{\alpha}(\alpha) \right\}, \label{eq:dml-ssgrr-fold} \end{align}\tag{10}\] where \({\mathcal{H}}_n\) is a user-chosen representer class, \(Reg_{\alpha}\) is a regularization functional, and \[\begin{align} \widehat{{\mathcal{Q}}}^{(-k)}_{g,\kappa}(\alpha) &\mathrel{\vcenter{:}}= \frac{1}{|{\mathcal{I}}_{-k}|}\sum_{i\in {\mathcal{I}}_{-k}} \Big( \partial g\left(\alpha(X_i)\right)\alpha(X_i)-g\left(\alpha(X_i)\right) \Big) - \widehat{{\mathcal{L}}}^{(-k)}_{\kappa}(\partial g\circ\alpha). \label{eq:dml-ssgrr-fold-objective} \end{align}\tag{11}\] The first term in 11 uses labeled regressors because the Riesz inner product is defined under \(P_{0X}\). The second term uses both labeled and unlabeled regressors because it estimates the target functional under \(V_{0X}=\kappa P_{0X}+(1-\kappa)Q_{0X}\).

3.5.0.2 Regression function estimation.

The regression function is identified from labeled observations. For each fold \(k\), estimate \(\gamma_0(x)={\mathbb{E}}\left[Y\mid X=x\right]\) by any estimation method trained on \({\mathcal{I}}_{-k}\). For example, with squared loss, \[\begin{align} \widehat\gamma_k \in \arg\min_{\gamma\in{\mathcal{G}}_n} \left\{ \frac{1}{|{\mathcal{I}}_{-k}|}\sum_{i\in {\mathcal{I}}_{-k}}\big(Y_i-\gamma(X_i)\big)^2 +\lambda_{\gamma,n}Reg_\gamma(\gamma) \right\}, \label{eq:dml-regression-fold} \end{align}\tag{12}\] where \(Reg_\gamma\) is a regularization functional. For binary or bounded outcomes, the squared loss can be replaced by an appropriate negative log-likelihood. The theoretical results for the estimators of \(\theta_0\) below do not require a particular regression method. They only require mean-square convergence rates for \(\widehat\gamma_k\) and \(\widehat\alpha_k\). Unlabeled data can also be incorporated through semi-supervised learning or covariate shift adaptation [26]. We leave the choice of semi-supervised regression method as application-specific, because the appropriate construction depends on the data-generating setting and the target parameter.

4 Theoretical Properties↩︎

For simplicity, we denote both \(\widehat\theta^{\text{TS}}_{\mathrm{EE}}\) and \(\widehat\theta^{\text{TS}}_{\mathrm{TMLE}}\) by \(\widehat{\theta}\). This section establishes the asymptotic properties of estimators \(\widehat{\theta}\) of \(\theta_0\) constructed by the proposed DML-PPCI, covering EE-DML-PPCI and TMLE-DML-PPCI.

4.1 Consistency↩︎

We first show the consistency of \(\widehat{\theta}\).

Note that, since the density ratio \(q_{0X}/p_{0X}\) is bounded under Assumption 1, this assumption also implies \(\|\widehat\gamma-\gamma_0\|_{Q,2} =o_p(1)\).

The following theorem holds.

We do not require that both Assumptions 2 and 3 hold. Such a property is called double robustness [44]. To show asymptotic normality, both Assumptions 2 and 3 need to hold.

4.2 Asymptotic Properties↩︎

In addition to Assumptions 2 and 3, we impose the following assumptions.

The following theorem holds.

4.3 Efficiency Gain↩︎

We now compare the semi-supervised bound with the labeled-only bound in the same-population case. Suppose that \[\begin{align} P_{0X}=Q_{0X}. \end{align}\] Then, \(V_{0X}=P_{0X}\) for every \(\kappa\in[0,1]\), so changing \(\kappa\) does not change the estimand.

Let \(\alpha_{0,P}\) denote the Riesz representer for the ordinary labeled-population functional \[\begin{align} \gamma\mapsto {\mathbb{E}}_{P_0}\left[m(X,\gamma)\right]. \end{align}\] In this same-population case, \(\alpha_{0,\kappa}=\alpha_{0,P}\) and \(\theta_{P,0}=\theta_{Q,0}=\theta_0\). Define \[\begin{align} A_0 \mathrel{\vcenter{:}}= {\mathbb{E}}_{P_0}\left[\alpha_{0,P}(X)^2\sigma_0^2(X)\right], \qquad B_0 \mathrel{\vcenter{:}}= \operatorname{Var}_{P_{0X}}\left(m(X, \gamma_0)\right). \end{align}\] Then, we have \[\begin{align} V^{\mathrm{TS}}_0(\kappa,\rho) = \frac{A_0}{\rho} + \left( \frac{\kappa^2}{\rho} + \frac{(1-\kappa)^2}{1-\rho} \right)B_0 . \label{eq:ss-bound-same-pop} \end{align}\tag{13}\] The value of \(\kappa\) that minimizes 13 is \[\begin{align} \kappa^*=\rho. \end{align}\] At \(\kappa=\rho\), the semi-supervised efficiency bound becomes \[\begin{align} V^{\mathrm{TS}}_0(\rho,\rho) = \frac{A_0}{\rho}+B_0 . \label{eq:ss-bound-kappa-rho} \end{align}\tag{14}\] By contrast, an estimator that uses only the \(n\) labeled observations has the ordinary efficiency bound \(A_0+B_0\) under \(\sqrt n\) normalization in the standard setup of causal inference or DML.

Under the common \(\sqrt N\) normalization and the design identity \(n/N=\rho\), this becomes \[\begin{align} V^{\mathrm{sup}}_0(\rho) = \frac{1}{\rho}(A_0+B_0). \label{eq:ss-supervised-bound} \end{align}\tag{15}\] Therefore, we have \[\begin{align} V^{\mathrm{sup}}_0(\rho)-V^{\mathrm{TS}}_0(\rho,\rho) = \left(\frac{1}{\rho}-1\right) \operatorname{Var}_{P_{0X}}\left(m(X, \gamma_0)\right) \ge 0. \label{eq:ss-eff-gain} \end{align}\tag{16}\] This identity gives the efficiency gain from using the proposed method. Specifically, using unlabeled data reduces the variance stemming from the regressor-averaging component \(\operatorname{Var}_{P_{0X}}\left(m(X, \gamma_0)\right)\), while the residual outcome-noise component \(A_0\) remains inflated by \(1/\rho\) because outcomes are observed only in the labeled stratum.

If \(P_{0X}\ne Q_{0X}\), the parameter \(\kappa\) is part of the estimand because changing \(\kappa\) changes the target regressor distribution. In that case, \(\kappa\) should not be chosen by minimizing ?? unless the scientific target is invariant to \(\kappa\).

5 Semi-Supervised Generalized Riesz Regression↩︎

In this section, we develop a semi-supervised version of generalized Riesz regression, building on [45] and [16]. Riesz regression was proposed by [7] for estimating the Riesz representer \(\alpha_0\). [16] generalizes Riesz regression using Bregman divergence minimization, building on related results in [18] and [17].

5.1 Bregman–Riesz Objective↩︎

Let \({\mathcal{A}}\subset{\mathbb{R}}\) be an interval and let \(g\colon{\mathcal{A}}\to{\mathbb{R}}\) be a differentiable strictly convex function. We write \(\partial g\) for its derivative. For a candidate representer \(\alpha\), we write \(u_\alpha\mathrel{\vcenter{:}}=\partial g\circ\alpha\) and \(b_\alpha\mathrel{\vcenter{:}}= u_\alpha\alpha-g(\alpha)\).

5.1.0.1 Population objective.

For \(a_0,a\in{\mathcal{A}}\), define the Bregman divergence by \[\begin{align} \text{BD}_g^{\dagger}(a_0\mid a) \mathrel{\vcenter{:}}= g(a_0)-g(a)-\partial g(a)(a_0-a). \end{align}\] By ignoring terms irrelevant to the optimization over \(\alpha\), the population semi-supervised Bregman–Riesz objective is \[\begin{align} {\mathcal{R}}_{g,\kappa}(\alpha) &\mathrel{\vcenter{:}}= {\mathbb{E}}_{P_{0X}}\left[\partial g\left(\alpha(X)\right)\alpha(X)-g\left(\alpha(X)\right)\right] - \kappa{\mathbb{E}}_{P_{0X}}\left[m(X,u_\alpha)\right] - (1-\kappa){\mathbb{E}}_{Q_{0X}}\left[m({\widetilde{X}},u_\alpha)\right]. \label{eq:ss-pop-bregman-riesz} \end{align}\tag{17}\]

5.1.0.2 Empirical objective.

The empirical objective is \[\begin{align} \widehat{{\mathcal{R}}}_{g,\kappa}(\alpha) &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1} \big( \partial g\left(\alpha(X_i)\right)\alpha(X_i)-g\left(\alpha(X_i)\right) -\kappa m(X_i,u_\alpha) \big) - \frac{1-\kappa}{m}\sum^m_{j=1} m\left({\widetilde{X}}_j,u_\alpha\right). \label{eq:ss-emp-bregman-riesz} \end{align}\tag{18}\] Here, \(\widehat{{\mathcal{L}}}_\kappa\) is the empirical evaluation functional \[\begin{align} \widehat{{\mathcal{L}}}_\kappa(\gamma) \mathrel{\vcenter{:}}= \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\gamma) + \frac{1-\kappa}{m}\sum^m_{j=1} m({\widetilde{X}}_j,\gamma), \end{align}\] the full-sample analogue of 9 ; in this notation \(\widehat{{\mathcal{R}}}_{g,\kappa}(\alpha)=\frac{1}{n}\sum^n_{i=1} b_\alpha(X_i)-\widehat{{\mathcal{L}}}_\kappa(u_\alpha)\).

Given a hypothesis class \({\mathcal{H}}_n\), we estimate \(\alpha_{0,\kappa}\) by \[\begin{align} \widehat\alpha \in \arg\min_{\alpha\in{\mathcal{H}}_n} \left\{ \widehat{{\mathcal{R}}}_{g,\kappa}(\alpha)+\lambda_n Reg_{\alpha}(\alpha) \right\}, \label{eq:ss-grr-estimator} \end{align}\tag{19}\] where \(Reg_{\alpha}\) is some regularization functional.

5.2 Automatic Regressor Balancing↩︎

The Bregman–Riesz objective automatically produces balancing equations when the dual coordinate is linear in a set of regressors [16]. For a candidate representer \(\alpha\) and a candidate regression function \(\gamma\), define the semi-supervised imbalance gap \[\begin{align} \widehat{\Delta}_{\kappa}(\alpha,\gamma) &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1} \alpha(X_i)\gamma(X_i) - \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\gamma) - \frac{1-\kappa}{m}\sum^m_{j=1} m({\widetilde{X}}_j,\gamma). \label{eq:ss-imbalance-gap} \end{align}\tag{20}\] The population analogue is \[\begin{align} \Delta_{0,\kappa}(\alpha,\gamma) \mathrel{\vcenter{:}}= {\mathbb{E}}_{P_{0X}}\left[\alpha(X)\gamma(X)\right]- {\mathcal{L}}_{0,\kappa}(\gamma). \end{align}\] The Riesz representer is characterized by \(\Delta_{0,\kappa}(\alpha_{0,\kappa},\gamma)=0\) for all valid \(\gamma\).

Let \(\phi(x)=(\phi_1(x),\ldots,\phi_p(x))^\top\) be a dictionary and consider the dual-linear model \[\begin{align} f_\beta(x)=\phi(x)^\top\beta, \qquad \alpha_\beta(x)=(\partial g)^{-1}\left(f_\beta(x)\right). \label{eq:ss-dual-linear-model} \end{align}\tag{21}\] For \(q\in[1,\infty)\), set \(\Omega_q(\beta)=\|\beta\|_q^q/q\). The empirical estimator is \[\begin{align} \widehat\beta \in \arg\min_{\beta\in{\mathbb{R}}^p} \left\{ \frac{1}{n}\sum^n_{i=1} g^*\left(\phi(X_i)^\top\beta\right) - \widehat{{\mathcal{L}}}_\kappa(\phi^\top\beta) + \lambda_n \Omega_q(\beta) \right\}, \qquad \widehat\alpha=\alpha_{\widehat\beta}. \label{eq:ss-dual-linear-estimator} \end{align}\tag{22}\]

Proposition 5 shows that balancing is not an additional constraint imposed after representer estimation. It is the KKT condition of the generalized Riesz regression problem. The functions to be balanced are functions of the full regressor \(X\). In treatment-effect applications with \(X=(D,Z)\), this means that the dictionary should generally contain treatment-specific functions, such as \(D\phi(Z)\) and \((1-D)\phi(Z)\), rather than only common functions of \(Z\).

This balancing property directly controls the deterministic component of the estimated Neyman score. Let \(\varepsilon_i=Y_i-\gamma_0(X_i)\) and define the empirical semi-supervised Neyman error by \[\begin{align} \mathrm{NE}_{n,m}^{\mathrm{TS}}(\widehat\gamma,\widehat\alpha) &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\widehat\gamma(X_i)\right) + \widehat{{\mathcal{L}}}_\kappa(\widehat\gamma) - \widehat{{\mathcal{L}}}_\kappa(\gamma_0). \label{eq:ss-neyman-error-def} \end{align}\tag{23}\] By linearity of \(\widehat{{\mathcal{L}}}_\kappa\), we have \[\begin{align} \mathrm{NE}_{n,m}^{\mathrm{TS}}(\widehat\gamma,\widehat\alpha) = \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\varepsilon_i - \widehat{\Delta}_\kappa\left(\widehat\alpha,\widehat\gamma-\gamma_0\right). \label{eq:ss-neyman-error-decomposition} \end{align}\tag{24}\] Under cross-fitting, the first term is conditionally mean zero given the training sample and the held-out regressors. The second term is the deterministic deviation caused by imperfect regressor balance. If \[\begin{align} \widehat\gamma(x)-\gamma_0(x)=a^\top\phi(x)+r(x), \end{align}\] then Proposition 5 yields the bound \[\begin{align} \left|\widehat{\Delta}_\kappa\left(\widehat\alpha,\widehat\gamma-\gamma_0\right)\right| \le \lambda_n\|a\|_1 + \left|\widehat{\Delta}_\kappa(\widehat\alpha,r)\right| \label{eq:ss-neyman-error-balance-bound} \end{align}\tag{25}\] for the \(\ell_1\)-penalized case. Thus, the represented part of the score-relevant regression error is automatically balanced, and only regularization slack and approximation error remain.

5.3 Choice of the Convex Function↩︎

Different choices of the convex function yield different divergences. The following loss functions are examples.

5.3.0.1 Squared-loss-type Riesz regression.

If we set the convex function to \[\begin{align} g^{\mathrm{SQ}}(a)=\frac{1}{2}(a-C)^2, \end{align}\] then we obtain the squared-loss-type objective in Riesz regression, which is identical to the original Riesz regression [7], [46] and least-squares importance fitting (LSIF) in density-ratio estimation [47]. Note that in this case, the derivative of the convex function is given as \(\partial g^{\mathrm{SQ}}(a)=a-C\). With the affine link \(\alpha_\beta(x)=C+\phi(x)^\top\beta\), the dual coordinate is linear, and the objective becomes the semi-supervised analogue of least-squares Riesz regression.

5.3.0.2 UKL (unnormalized KL)-type Riesz regression.

When \(\alpha_{0,\kappa}\) is nonnegative, or when a signed representer is decomposed into known-sign branches, one can use the positive-branch unnormalized KL (UKL) objective \[\begin{align} g^{\mathrm{UKL}}(a)=a\log a-a, \qquad \partial g^{\mathrm{UKL}}(a)=\log a, \qquad a>0. \end{align}\] The link \(\alpha_\beta(x)=\exp\left(\phi(x)^\top\beta\right)\) is compatible with automatic regressor balancing. In addition, this choice yields an entropy-balancing or KLIEP-type objective as its dual.

5.3.0.3 Other loss functions.

We can also use other convex functions in the Bregman divergence with the automatic regressor balancing property. Binary KL-type (BKL) losses connect to logistic likelihoods, Basu-power losses interpolate between squared-loss and KL-type behavior, and PU (positive and unlabeled)-type losses are useful when the representer is bounded in an interval.

5.4 Candidate Hypothesis Classes↩︎

Several hypothesis classes can be used for \(\alpha_{0,\kappa}\).

5.4.0.1 Series and sieve classes.

A basic choice is the finite-dimensional dual-linear class \[\begin{align} {\mathcal{F}}_n^{\mathrm{ser}} = \{x\mapsto \phi_n(x)^\top\beta\colon\beta\in{\mathbb{R}}^{p_n}\}, \end{align}\] where \(\phi_n\) contains splines, polynomials, interactions, or other researcher-chosen features. In causal applications with \(X=(D,Z)\), treatment-specific dictionaries are recommended because the score-relevant regression error is generally a function of the full regressor.

5.4.0.2 RKHS and random-feature classes.

Kernel methods can be used by taking \({\mathcal{F}}_n\) to be an RKHS ball or a finite-dimensional approximation obtained by random Fourier features or Nyström features. This gives functional balance over a rich class without manually specifying every interaction.

5.4.0.3 Tree, forest, matching, and neural classes.

Tree leaf indicators and random-forest leaf encodings produce sparse dictionaries that adapt to heterogeneous regions of the regressor space. Nearest-neighbor catchment bases give matching-type representer estimators under squared loss [48]. Neural networks can be used either as frozen embeddings followed by a convex dual-linear Riesz fit or as a fully end-to-end nonconvex class.

5.5 Error Analysis↩︎

We now provide error bounds for the semi-supervised generalized Riesz regression estimator. The analysis follows the same structure as error analyses for density-ratio estimation and Riesz regression. First, the population objective is identified with a Bregman divergence. Second, the empirical process is decomposed into labeled and unlabeled parts. Third, strong convexity converts excess Bregman risk into \(L^2(P_{0X})\) error. The first result is a general oracle inequality. Later results specialize this inequality to finite pseudo-dimension classes and deep ReLU sieves.

For \(\alpha\in{\mathcal{H}}_n\), define \(c_\alpha(x)\mathrel{\vcenter{:}}= m(x,u_\alpha)\) and \(a_\alpha(x)\mathrel{\vcenter{:}}= b_\alpha(x)-\kappa c_\alpha(x)\). Also define \({\mathcal{A}}_n\mathrel{\vcenter{:}}= \{a_\alpha\colon\alpha\in{\mathcal{H}}_n\}\) and \({\mathcal{C}}_n\mathrel{\vcenter{:}}= \{c_\alpha\colon\alpha\in{\mathcal{H}}_n\}\). For a probability law \(R\) and a function class \({\mathcal{F}}\), let \[\begin{align} \mathfrak R_{r,R}({\mathcal{F}}) \mathrel{\vcenter{:}}= {\mathbb{E}}\left[ \sup_{f\in{\mathcal{F}}}|\frac{1}{r}\sum_{i=1}^r \xi_i f(S_i)| \right] \end{align}\] be the Rademacher complexity, where \(S_i\sim R\) and \(\xi_i\) are independent Rademacher variables. Define \[\begin{align} \mathfrak E_{n,m}(\delta) &\mathrel{\vcenter{:}}= 2\mathfrak R_{n,P_{0X}}({\mathcal{A}}_n) +2(1-\kappa)\mathfrak R_{m,Q_{0X}}({\mathcal{C}}_n) \nonumber\\ &\quad +B_A\sqrt{\frac{\log(4/\delta)}{2n}} +(1-\kappa)B_C\sqrt{\frac{\log(4/\delta)}{2m}}, \label{eq:ss-emp-process-bound} \end{align}\tag{26}\] where \(B_A\) and \(B_C\) are envelopes for \({\mathcal{A}}_n\) and \({\mathcal{C}}_n\).

Theorem 6 is a general distribution-free oracle inequality. It is useful for stability diagnostics and for abstract DML conditions. Under a localized empirical-process condition, the usual fast-rate version is obtained. Let \(V_n\) denote a dimension or pseudo-dimension upper bound for the dual class \({\mathcal{F}}_n=\{u_\alpha\colon\alpha\in{\mathcal{H}}_n\}\), and let \[\begin{align} r_{n,m}^2(\delta) \mathrel{\vcenter{:}}= \left\{V_n\log(n+m)+\log(1/\delta)\right\} \left(\frac{1}{n}+\frac{1}{m}\right). \end{align}\]

We next provide primitive rates for deep ReLU sieves. These results make the abstract rate condition in Assumption 4 verifiable from smoothness assumptions on the dual Riesz oracle. They also show precisely where density-ratio analysis enters the semi-supervised generalized Riesz regression problem.

Let \[\begin{align} N_{\min}\mathrel{\vcenter{:}}= n\wedge m, \qquad f_0\mathrel{\vcenter{:}}= \partial g\circ\alpha_{0,\kappa}. \end{align}\] Let \({\mathcal{F}}_n\) be a class of clipped ReLU feedforward neural networks, and define \(\widehat\alpha=(\partial g)^{-1}\circ\widehat f\), where \(\widehat f\) minimizes the dual objective in ?? . For \(s>0\), let \({\mathcal{H}}^s([0,1]^d,M_f)\) denote the Hölder ball with smoothness \(s\) and radius \(M_f\).

The rate in Theorem 7 is the semi-supervised analogue of density-ratio estimators based on neural networks. The only structural difference is that the objective contains two empirical processes: the labeled one for the \(P_{0X}\) inner product and the unlabeled one for the \(Q_{0X}\) part of the target functional. Since \(N_{\min}\) controls both empirical processes, the rate is governed by the smaller sample size.

Proposition 8 gives the exact point at which the semi-supervised generalized Riesz regression objective reduces to density-ratio estimation under the Bregman divergence. It justifies the use of theoretical results developed in density-ratio estimation.

Theorems 7, 9, and 10 should be interpreted together with the minimax message of [30]. Unlabeled regressors do not automatically improve a nonparametric rate merely because they are available. In the rate analysis above, they have two precise roles. First, they change the target functional and the efficiency bound through \(V_{0X}\). Second, they enter the Riesz objective through \(\widehat{{\mathcal{L}}}_\kappa\). The rates above require smoothness of the dual Riesz oracle, tail control, or low-dimensional structure. Without such structure, Assumption 4 is the appropriate abstract condition.

5.6 Automatic Neyman Orthogonalization↩︎

The balancing property also clarifies the relation between the Riesz-weighted estimator and the ARW estimator. Suppose that exact balance holds for the true regression function, \[\begin{align} \widehat{\Delta}_\kappa(\widehat\alpha,\gamma_0)=0. \end{align}\] Then, we have \[\begin{align} \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)Y_i &= \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\gamma_0(X_i) + \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\gamma_0(X_i)\right) \nonumber\\ &= \widehat{{\mathcal{L}}}_\kappa(\gamma_0) + \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\gamma_0(X_i)\right). \label{eq:ss-rw-as-infeasible-arw} \end{align}\tag{27}\] Thus, the Riesz-weighted estimator behaves as the infeasible semi-supervised ARW estimator that uses the true regression function.

6 Efficiency Bound and Efficient Influence Function in the One-Sample Scenario↩︎

Next, we consider the one-sample scenario, where we observe one sample and labels are observed with certain probabilities.

6.1 Setup↩︎

In this scenario, a complete population observation is \((X,Y^*)\), but the outcome is observed only when \(S=1\). The observed data are \[\begin{align} O=(X,S,Y), \qquad Y=SY^*+(1-S)\mathrm{NA}. \end{align}\] Let \(P_{0X}\) be the marginal distribution of \(X\), let \(\pi_0(X)\mathrel{\vcenter{:}}=\mathbb{P}\left[(\right]S=1\mid X)\), and let \(\gamma_0(X)\mathrel{\vcenter{:}}={\mathbb{E}}\left[Y^*\mid X\right]\). We assume the missing-at-random condition \[\begin{align} Y^*\perp S\mid X \end{align}\] and the overlap condition that \(\pi_0(X)\) is bounded away from zero. In the one-sample scenario, the full set of regressors is observed from \(P_{0X}\). Therefore, the target parameter is \[\begin{align} \theta^{\mathrm{OS}}_0 \mathrel{\vcenter{:}}= {\mathbb{E}}_{P_{0X}}\left[m(X,\gamma_0)\right]. \label{eq:os-target} \end{align}\tag{28}\]

6.2 Efficient Influence Function↩︎

Let \(\alpha^{\mathrm{OS}}_0\) be the Riesz representer of the one-sample functional, so that \[\begin{align} {\mathbb{E}}_{P_{0X}}\left[m(X,\gamma)\right] = {\mathbb{E}}_{P_{0X}}\left[\alpha^{\mathrm{OS}}_0(X)\gamma(X)\right] \qquad \text{for all }\gamma\in\Gamma. \label{eq:os-riesz} \end{align}\tag{29}\] We derive the efficient influence function. First, we impose the following assumption.

Then, we obtain the following theorem.

6.3 Efficiency Bound↩︎

The efficiency bound is given as follows.

The first term in ?? depends on \(1/\pi_0(X)\) because outcomes are observed only for labeled units. The second term does not depend on \(1/\pi_0(X)\) because \(X\) is observed for every unit.

6.4 Efficient Estimators↩︎

Based on the efficient influence function in Theorem 11, we construct one-sample analogues of EE-DML-PPCI and TMLE-DML-PPCI. Let \(\widehat\gamma\), \(\widehat\alpha\), and \(\widehat\pi\) be estimators of \(\gamma_0\), \(\alpha^{\mathrm{OS}}_0\), and \(\pi_0\), respectively. As in the two-sample scenario, if the Donsker condition does not hold for the estimators, we employ cross-fitting to control the empirical process term in the asymptotic analysis.

6.4.0.1 EE-DML-PPCI.

The one-sample EE-DML-PPCI estimator is \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} \mathrel{\vcenter{:}}= \frac{1}{N}\sum^N_{i=1} \left( \frac{S_i}{\widehat\pi(X_i)} \widehat\alpha(X_i) \left(Y_i-\widehat\gamma(X_i)\right) + m\left(X_i,\widehat\gamma\right) \right). \label{eq:os-ee-estimator} \end{align}\tag{30}\] Here, the product \(S_i\left(Y_i-\widehat\gamma(X_i)\right)\) is interpreted as \(S_i\left(Y_i^*-\widehat\gamma(X_i)\right)\), which is observed.

6.4.0.2 TMLE-DML-PPCI.

The one-sample TMLE-DML-PPCI estimator first updates \(\widehat\gamma\) in the direction of \(\widehat\alpha\). For example, if we consider TMLE under the Gaussian likelihood, we update the initial estimator as \[\begin{align} \widehat\gamma^{(1)}(x) \mathrel{\vcenter{:}}= \widehat\gamma(x) + \widehat\varepsilon^{\mathrm{OS}}\widehat\alpha(x), \label{eq:os-tmle-update} \end{align}\tag{31}\] where \(\widehat\varepsilon^{\mathrm{OS}}\) is the perturbation defined as \[\begin{align} \widehat\varepsilon^{\mathrm{OS}} &\mathrel{\vcenter{:}}= \frac{\widehat A^{\mathrm{OS}}_N}{\widehat D^{\mathrm{OS}}_N}, \label{eq:os-tmle-epsilon} \\ \widehat A^{\mathrm{OS}}_N &\mathrel{\vcenter{:}}= \frac{1}{N}\sum^N_{i=1} \frac{S_i}{\widehat\pi(X_i)} \widehat\alpha(X_i) \left(Y_i-\widehat\gamma(X_i)\right), \qquad \widehat D^{\mathrm{OS}}_N \mathrel{\vcenter{:}}= \frac{1}{N}\sum^N_{i=1} \widehat\alpha^2\left(X_i\right). \end{align}\tag{32}\] Then, the one-sample TMLE-DML-PPCI estimator is given as \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} \mathrel{\vcenter{:}}= \frac{1}{N}\sum^N_{i=1} m\left(X_i,\widehat\gamma^{(1)}\right). \label{eq:os-tmle-estimator} \end{align}\tag{33}\] The numerator in 32 uses labeled residuals corrected by \(S_i/\widehat\pi(X_i)\), because outcomes are observed only when \(S_i=1\). The denominator estimates the target-law average of \(\widehat\alpha^2\) in the one-sample scenario, where the regressor distribution is observed through \(\left\{X_i\right\}^N_{i=1}\).

6.5 Asymptotic Properties↩︎

We first establish consistency and then show asymptotic linearity and efficiency. The assumptions are stated for generic nuisance estimators \(\widehat\gamma\), \(\widehat\alpha\), and \(\widehat\pi\), in the same way as in the two-sample scenario.

The following theorem holds.

To show asymptotic normality, both consistency conditions need to hold. In addition, we impose the following rate and empirical-process conditions.

The following theorem holds.

7 Conclusion↩︎

This study develops PPCI for causal and structural regression functionals with labeled observations and unlabeled auxiliary regressors. We derive the efficient influence functions and efficiency bounds under two different DGPs, or sampling schemes, and develop an estimation method called DML-PPCI. In particular, we present EE-DML-PPCI and TMLE-DML-PPCI as special implementations of DML-PPCI. We also develop semi-supervised generalized Riesz regression for estimating the Riesz representer in the Neyman orthogonal score. Using these methods, we show that we can construct estimators whose asymptotic variances are smaller than those of estimators constructed using only the labeled dataset. These results clarify when unlabeled regressors improve efficiency.

8 Proofs for Section 3↩︎

8.1 Proof of Theorem 1↩︎

We prove the result by verifying the pathwise derivative in every regular parametric submodel and then specifying the canonical gradient in the stratified tangent space.

Let \(\left\{P_t\colon t\in(-\eta,\eta)\right\}\) be a regular parametric submodel for the labeled distribution and let \(\left\{Q_t\colon t\in(-\eta,\eta)\right\}\) be a regular parametric submodel for the unlabeled regressor distribution. Both submodels pass through the true laws at \(t=0\). Define their scores as \[\begin{align} s_{\mathrm L}\big(W\big) = \frac{\partial}{\partial t}\log p_t(W)|_{t=0}, \qquad s_{\mathrm U}\big({\widetilde{X}}\big) = \frac{\partial}{\partial t}\log q_t\big({\widetilde{X}}\big)|_{t=0}. \end{align}\] The scores are normalized to have mean zero: \[\begin{align} {\mathbb{E}}_{P_0}\left[s_{\mathrm L}(W)\right]=0, \qquad {\mathbb{E}}_{Q_{0X}}\left[s_{\mathrm U}\big({\widetilde{X}}\big)\right]=0. \end{align}\] Under the labeled law, we decompose the score into the following two elements: \[\begin{align} s_X(X) \mathrel{\vcenter{:}}= {\mathbb{E}}_{P_0}\left[s_{\mathrm L}(W)\mid X\right], \qquad s_{Y\mid X}(W) \mathrel{\vcenter{:}}= s_{\mathrm L}(W)-s_X(X). \end{align}\] They also satisfy \[\begin{align} {\mathbb{E}}_{P_0}\left[s_X(X)\right]=0, \qquad {\mathbb{E}}_{P_0}\left[s_{Y\mid X}(W)\mid X\right]=0. \end{align}\] The first equality follows by taking expectations of \(s_X(X)\), and the second follows directly from the definition.

Let us define \[\begin{align} \gamma_t(x)\mathrel{\vcenter{:}}= {\mathbb{E}}_{P_t}\left[Y\mid X=x\right]. \end{align}\] We first compute the derivative of \(\gamma_t\). From the conditional likelihood, for each fixed \(x\), the conditional score for \(Y\) given \(X=x\) is \(s_{Y\mid X}(Y,x)\). Hence, we have \[\begin{align} \dot{\gamma}_0(x) &\mathrel{\vcenter{:}}= \frac{\partial}{\partial t}\gamma_t(x)|_{t=0} \nonumber\\ &= \frac{\partial}{\partial t} \int y p_t(y\mid x){\mathrm{d}}y |_{t=0} \nonumber\\ &= \int y s_{Y\mid X}(y,x)p_0(y\mid x){\mathrm{d}}y. \end{align}\] Since \({\mathbb{E}}_{P_0}\left[s_{Y\mid X}(Y,X)\mid X=x\right]=0\) holds, subtracting \(\gamma_0(x)\) from \(y\) gives \[\begin{align} \dot{\gamma}_0(x) = {\mathbb{E}}_{P_0}\left[\left(Y-\gamma_0(X)\right)s_{Y\mid X}(W)\mid X=x\right]. \label{eq:app-ts-gamma-derivative} \end{align}\tag{34}\]

Along the submodel, the parameter is \[\begin{align} \theta_t = \kappa{\mathbb{E}}_{P_{tX}}\left[m\big(X,\gamma_t\big)\right] + (1-\kappa){\mathbb{E}}_{Q_{tX}}\left[m\big({\widetilde{X}},\gamma_t\big)\right]. \label{eq:app-ts-param-submodel} \end{align}\tag{35}\] Because \(\gamma\mapsto m(\cdot,\gamma)\) is linear, differentiating 35 at \(t=0\) yields three terms: \[\begin{align} \frac{\partial}{\partial t}\theta_t|_{t=0} \label{eq:app-ts-derivative-decomposition} = \kappa{\mathbb{E}}_{P_{0X}}\left[m(X,\gamma_0)s_X(X)\right] + (1-\kappa){\mathbb{E}}_{Q_{0X}}\left[m({\widetilde{X}},\gamma_0)s_{\mathrm U}\big({\widetilde{X}}\big)\right] + {\mathcal{L}}_{0,\kappa}(\dot{\gamma}_0). \end{align}\tag{36}\] The first term is derived from the perturbation of the labeled regressor marginal distribution, the second term is derived from the perturbation of the unlabeled regressor distribution, and the last term is derived from the perturbation of the conditional outcome law.

We now rewrite the last term in 36 . From the Riesz representer theorem 3 , we have \[\begin{align} {\mathcal{L}}_{0,\kappa}(\dot{\gamma}_0) = {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)\dot{\gamma}_0(X)\right]. \end{align}\] Substituting 34 gives \[\begin{align} {\mathcal{L}}_{0,\kappa}(\dot{\gamma}_0) &= {\mathbb{E}}_{P_0}\left[ \alpha_{0,\kappa}(X) \left(Y-\gamma_0(X)\right) s_{Y\mid X}(W) \right]. \label{eq:app-ts-residual-conditional-score} \end{align}\tag{37}\] Because \(s_{\mathrm L}=s_X+s_{Y\mid X}\) and \({\mathbb{E}}_{P_0}\left[Y-\gamma_0(X)\mid X\right]=0\), we have \[\begin{align} {\mathbb{E}}_{P_0}\left[ \alpha_{0,\kappa}(X) \left(Y-\gamma_0(X)\right) s_X(X) \right] =0. \end{align}\] Therefore, 37 is equivalently \[\begin{align} {\mathcal{L}}_{0,\kappa}(\dot{\gamma}_0) = {\mathbb{E}}_{P_0}\left[ \alpha_{0,\kappa}(X) \left(Y-\gamma_0(X)\right) s_{\mathrm L}(W) \right]. \label{eq:app-ts-residual-score} \end{align}\tag{38}\]

Next, rewrite the marginal-distribution terms. Since \(s_{\mathrm L}=s_X+s_{Y\mid X}\) and \(m(X,\gamma_0)\) is a function of \(X\), \[\begin{align} {\mathbb{E}}_{P_0}\left[m(X,\gamma_0)s_{Y\mid X}(W)\right] = {\mathbb{E}}_{P_0}\left[m(X,\gamma_0){\mathbb{E}}_{P_0}\left[s_{Y\mid X}(W)\mid X\right]\right] =0. \end{align}\] Also, \({\mathbb{E}}_{P_0}\left[s_X(X)\right]=0\). Hence, \[\begin{align} \kappa{\mathbb{E}}_{P_{0X}}\left[m(X,\gamma_0)s_X(X)\right] &= {\mathbb{E}}_{P_0}\left[ \kappa\left(m(X,\gamma_0)-\theta_{P,0}\right)s_{\mathrm L}(W) \right]. \label{eq:app-ts-p-term} \end{align}\tag{39}\] Similarly, using \({\mathbb{E}}_{Q_{0X}}[s_{\mathrm U}\big({\widetilde{X}}\big)]=0\), \[\begin{align} (1-\kappa){\mathbb{E}}_{Q_{0X}}\left[m({\widetilde{X}},\gamma_0)s_{\mathrm U}\big({\widetilde{X}}\big)\right] = {\mathbb{E}}_{Q_{0X}}\left[ (1-\kappa)\{m({\widetilde{X}},\gamma_0)-\theta_{Q,0}\}s_{\mathrm U}\big({\widetilde{X}}\big) \right]. \label{eq:app-ts-q-term} \end{align}\tag{40}\] Combining 36 , 38 , 39 , and 40 , it holds that \[\begin{align} \frac{\partial}{\partial t}\theta_t|_{t=0} = {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)s_{\mathrm L}(W)\right] + {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)s_{\mathrm U}\big({\widetilde{X}}\big)\right]. \label{eq:app-ts-gradient-identity} \end{align}\tag{41}\] Thus, the pair \((\psi^{\mathrm{TS}}_0,\widetilde{\psi}^{\mathrm{TS}}_0)\) is a gradient for the parameter in the product tangent space.

It remains to show that this gradient is the efficient one. Under the fully nonparametric model, the tangent space for the labeled stratum is the full space of mean-zero square-integrable functions of \(W\), and the tangent space for the unlabeled stratum is the full space of mean-zero square-integrable functions of \({\widetilde{X}}\). The labeled component satisfies \[\begin{align} {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)\right] &= {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right)\right] + \kappa{\mathbb{E}}_{P_{0X}}\left[m(X,\gamma_0)-\theta_{P,0}\right] =0, \end{align}\] and the unlabeled component satisfies \[\begin{align} {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)\right]=0. \end{align}\] Therefore, the gradient above itself belongs to the tangent space. Since the canonical gradient is the orthogonal projection of any gradient onto the tangent space, and this gradient is already in the tangent space, it is the canonical gradient. This proves Theorem 1.

8.2 Proof of Theorem 2↩︎

We derive the lower bound from the fixed-proportion stratified experiment. The stratum sizes are part of the sampling design and are not random variables. Along the admissible deterministic sequence of total sample sizes, the observed block is \[\begin{align} O_N=(W_1,\ldots,W_n,{\widetilde{X}}_1,\ldots,{\widetilde{X}}_m), \end{align}\] where the design fixes \[\begin{align} n=\rho N, \qquad m=(1-\rho)N. \end{align}\] Thus, \(\rho\) is not introduced as an asymptotic limit of \(n/N\); the equality \(n/N=\rho\) is imposed by the design along the sequence. The law of the block is \(P_0^n\times Q_{0X}^m\).

Consider regular finite-dimensional submodels \(\{P_a:a\in\mathbb{R}^{d_L}\}\) for the labeled stratum and \(\{Q_b:b\in\mathbb{R}^{d_U}\}\) for the unlabeled stratum, both passing through the true laws at \(a=0\) and \(b=0\). Let the corresponding scores be \[\begin{align} S_L(W)=\frac{\partial}{\partial a}\log p_a(W)|_{a=0}, \qquad S_U\big({\widetilde{X}}\big)=\frac{\partial}{\partial b}\log q_b\big({\widetilde{X}}\big)|_{b=0}, \end{align}\] where \(S_L\) and \(S_U\) are column vectors with mean zero. Define the stratum-specific information matrices \[\begin{align} I_L={\mathbb{E}}_{P_0}\left[S_L(W)S_L(W)^\top\right], \qquad I_U={\mathbb{E}}_{Q_{0X}}\left[S_U\big({\widetilde{X}}\big)S_U\big({\widetilde{X}}\big)^\top\right]. \end{align}\] The joint log-likelihood of the block is \[\begin{align} \sum^n_{i=1} \log p_a(W_i)+\sum^m_{j=1}\log q_b({\widetilde{X}}_j). \end{align}\] Therefore, the block score for \((a,b)\) is \[\begin{align} \left( \sum^n_{i=1} S_L(W_i)^\top, \sum^m_{j=1} S_U({\widetilde{X}}_j)^\top \right)^\top, \end{align}\] and the block Fisher information matrix is \[\begin{align} I_N= \begin{pmatrix} n I_L & 0\\ 0 & m I_U \end{pmatrix}. \end{align}\] The off-diagonal blocks are zero because the two strata are sampled independently and because both stratum-specific scores are centered.

Let \(\theta(a,b)\) be the value of the target parameter under \((P_a,Q_b)\). From the pathwise derivative calculation in the proof of Theorem 1, \[\begin{align} \frac{\partial}{\partial a}\theta(a,b)|_{a=b=0} = {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)S_L(W)\right], \qquad \frac{\partial}{\partial b}\theta(a,b)|_{a=b=0} = {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)S_U\big({\widetilde{X}}\big)\right]. \end{align}\] For this finite-dimensional submodel, the Cramér–Rao lower bound for an estimator whose error is normalized by \(\sqrt N\) is \[\begin{align} &N \begin{pmatrix} \partial_a\theta(0,0)\\ \partial_b\theta(0,0) \end{pmatrix}^{\top} I_N^{-1} \begin{pmatrix} \partial_a\theta(0,0)\\ \partial_b\theta(0,0) \end{pmatrix} \nonumber\\ &\quad= \frac{1}{\rho} \partial_a\theta(0,0)^\top I_L^{-1}\partial_a\theta(0,0) + \frac{1}{1-\rho} \partial_b\theta(0,0)^\top I_U^{-1}\partial_b\theta(0,0). \label{eq:app-ts-finite-cr} \end{align}\tag{42}\] The factors \(\rho^{-1}\) and \((1-\rho)^{-1}\) appear because the information contributed by each stratum is proportional to its fixed sample size.

We now pass from finite-dimensional submodels to the nonparametric model. Under the fully nonparametric model, the labeled tangent space is \({\mathcal{T}}_L=L^2_0(P_0)\), and the unlabeled tangent space is \({\mathcal{T}}_U=L^2_0(Q_{0X})\). For arbitrary score directions \((s_L,s_U)\in{\mathcal{T}}_L\times{\mathcal{T}}_U\), the derivative is \[\begin{align} \frac{\partial}{\partial t}\theta_t|_{t=0}(s_L,s_U) = {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)s_L(W)\right] + {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)s_U\big({\widetilde{X}}\big)\right]. \label{eq:app-ts-derivative-score-pair} \end{align}\tag{43}\] The local log-likelihood ratio along the path \((P_{t/\sqrt N},Q_{t/\sqrt N})\) is \[\begin{align} &\log\frac{{\mathrm{d}}P_{t/\sqrt N}^{n}{\mathrm{d}}Q_{t/\sqrt N}^{m}}{{\mathrm{d}}P_0^{n}{\mathrm{d}}Q_{0X}^{m}}(O_N) \nonumber\\ &\quad= \frac{t}{\sqrt N} \left( \sum^n_{i=1} s_L(W_i)+\sum^m_{j=1} s_U({\widetilde{X}}_j) \right) -\frac{t^2}{2} \left( \rho{\mathbb{E}}_{P_0}\left[s_L(W)^2\right] +(1-\rho){\mathbb{E}}_{Q_{0X}}\left[s_U\big({\widetilde{X}}\big)^2\right] \right) +o_p(1), \label{eq:app-ts-lan-fixed-rho} \end{align}\tag{44}\] where the design equalities \(n=\rho N\) and \(m=(1-\rho)N\) are used. Hence the local information in direction \((s_L,s_U)\) under \(\sqrt N\) normalization is \[\begin{align} I_\rho(s_L,s_U) = \rho{\mathbb{E}}_{P_0}\left[s_L(W)^2\right] +(1-\rho){\mathbb{E}}_{Q_{0X}}\left[s_U\big({\widetilde{X}}\big)^2\right]. \end{align}\] Introduce the weighted inner product \[\begin{align} \langle (a_L,a_U),(b_L,b_U)\rangle_\rho = \rho{\mathbb{E}}_{P_0}\left[a_L(W)b_L(W)\right] +(1-\rho){\mathbb{E}}_{Q_{0X}}\left[a_U\big({\widetilde{X}}\big)b_U\big({\widetilde{X}}\big)\right]. \end{align}\] Then, 43 can be written as \[\begin{align} \frac{\partial}{\partial t}\theta_t|_{t=0}(s_L,s_U) = \left\langle \left(\frac{\psi^{\mathrm{TS}}_0}{\rho}, \frac{\widetilde{\psi}^{\mathrm{TS}}_0}{1-\rho}\right), (s_L,s_U) \right\rangle_\rho. \label{eq:app-ts-weighted-gradient-correct} \end{align}\tag{45}\] Thus, the canonical gradient for the \(\sqrt N\)-normalized fixed-proportion stratified experiment is the weighted pair \[\begin{align} \left(\frac{\psi^{\mathrm{TS}}_0}{\rho}, \frac{\widetilde{\psi}^{\mathrm{TS}}_0}{1-\rho}\right). \end{align}\] Its squared norm in the weighted tangent space is \[\begin{align} \left\| \left(\frac{\psi^{\mathrm{TS}}_0}{\rho}, \frac{\widetilde{\psi}^{\mathrm{TS}}_0}{1-\rho}\right) \right\|_\rho^2 &= \frac{1}{\rho}{\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)^2\right] + \frac{1}{1-\rho}{\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)^2\right]. \label{eq:app-ts-bound-from-canonical-gradient} \end{align}\tag{46}\] Equivalently, the block-level influence function is \[\begin{align} \Phi_N(O_N) = \frac{1}{n}\sum^n_{i=1}\psi^{\mathrm{TS}}_0(W_i) + \frac{1}{m}\sum^m_{j=1}\widetilde{\psi}^{\mathrm{TS}}_0({\widetilde{X}}_j), \end{align}\] and \(\sqrt N\Phi_N(O_N)\) has variance equal to the right-hand side of 46 . Moreover, for every local score pair \((s_L,s_U)\), the covariance between \(\sqrt N\Phi_N(O_N)\) and the local block score in 44 equals \(\frac{\partial}{\partial t}\theta_t\mid_{t=0}(s_L,s_U)\). Therefore, \(\Phi_N\) is the canonical block influence function, and the lower bound is ?? .

It remains to expand the two second moments. For the labeled component, \[\begin{align} \psi^{\mathrm{TS}}_0(W) = \alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right) + \kappa\left(m(X,\gamma_0)-\theta_{P,0}\right). \end{align}\] The cross term is zero because \[\begin{align} &{\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right) \left(m(X,\gamma_0)-\theta_{P,0}\right)\right] \nonumber\\ &\quad= {\mathbb{E}}_{P_{0X}}\left[\alpha_{0,\kappa}(X)\left(m(X,\gamma_0)-\theta_{P,0}\right) {\mathbb{E}}_{P_0}\left[Y-\gamma_0(X)\mid X\right]\right] =0. \end{align}\] Therefore, we have \[\begin{align} {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)^2\right] = {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)^2\sigma_0^2(X)\right] + \kappa^2\operatorname{Var}_{P_{0X}}\left(m(X,\gamma_0)\right). \label{eq:app-ts-labeled-variance-updated} \end{align}\tag{47}\] For the unlabeled component, \[\begin{align} {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\big({\widetilde{X}}\big)^2\right] = (1-\kappa)^2\operatorname{Var}_{Q_{0X}}\left(m({\widetilde{X}},\gamma_0)\right). \label{eq:app-ts-unlabeled-variance-updated} \end{align}\tag{48}\] Substituting 47 and 48 into ?? gives ?? . This completes the proof.

9 Proofs for Section 4↩︎

9.1 Proof of the Efficiency Gain↩︎

Assume \(P_{0X}=Q_{0X}\). Then, \(V_{0X}=P_{0X}\) for every \(\kappa\in[0,1]\), and hence the target parameter is invariant to \(\kappa\). Also, \[\begin{align} \alpha_{0,\kappa}=\alpha_{0,P}, \qquad \theta_{P,0}=\theta_{Q,0}=\theta_0, \end{align}\] where \(\alpha_{0,P}\) is the Riesz representer of \(\gamma\mapsto {\mathbb{E}}_{P_0}\left[m(X,\gamma)\right]\). Define \[\begin{align} A_0 = {\mathbb{E}}_{P_0}\left[\alpha_{0,P}(X)^2\sigma_0^2(X)\right], \qquad B_0 = \operatorname{Var}_{P_{0X}}\left(m(X, \gamma_0)\right). \end{align}\] The semi-supervised bound in Theorem 2 reduces to \[\begin{align} V^{\mathrm{TS}}_0(\kappa,\rho) = \frac{A_0}{\rho} + \left( \frac{\kappa^2}{\rho} + \frac{(1-\kappa)^2}{1-\rho} \right)B_0. \end{align}\] The bracketed term is minimized by differentiating with respect to \(\kappa\): \[\begin{align} \frac{\partial}{\partial\kappa} \left( \frac{\kappa^2}{\rho} + \frac{(1-\kappa)^2}{1-\rho} \right) = \frac{2\kappa}{\rho} - \frac{2(1-\kappa)}{1-\rho}. \end{align}\] Setting this equal to zero gives \(\kappa=\rho\). At \(\kappa=\rho\), we have \[\begin{align} \frac{\kappa^2}{\rho} + \frac{(1-\kappa)^2}{1-\rho} = \rho+(1-\rho)=1. \end{align}\] Therefore, we have \[\begin{align} V^{\mathrm{TS}}_0(\rho,\rho) = \frac{A_0}{\rho}+B_0. \end{align}\] The labeled-only efficiency bound under \(\sqrt n\) normalization is \(A_0+B_0\). Since the present comparison uses \(\sqrt N\) normalization and \(n/N=\rho\), the corresponding labeled-only bound is \[\begin{align} V^{\mathrm{sup}}_0(\rho)=\frac{A_0+B_0}{\rho}. \end{align}\] Therefore, it holds that \[\begin{align} V^{\mathrm{sup}}_0(\rho)-V^{\mathrm{TS}}_0(\rho,\rho) = \frac{A_0+B_0}{\rho} - \left(\frac{A_0}{\rho}+B_0\right) = \left(\frac{1}{\rho}-1\right)B_0, \end{align}\] which is 16 . This proves the efficiency gain.

9.2 Proof of Theorem 3↩︎

We give the proof for EE-DML-PPCI. The TMLE-DML-PPCI case follows under the corresponding targeting calibration condition. By the decomposition used in the proof of Theorem 4, the population drift is \[\begin{align} {\mathbb{E}}_{P_0}\left[\left(\alpha_{0,\kappa}(X)-\widehat\alpha(X)\right)\left(\widehat\gamma(X)-\gamma_0(X)\right)\right]. \end{align}\] If Assumption 2 holds and \(\widehat\alpha\) is bounded in \(L^2(P_{0X})\), this term is \(o_p(1)\) by Cauchy–Schwarz. If Assumption 3 holds and \(\widehat\gamma\) is bounded in \(L^2(P_{0X})\), the same conclusion follows. The empirical terms are \(o_p(1)\) by the law of large numbers under the stated moment conditions. Therefore, \(\widehat\theta\xrightarrow{{\mathrm{p}}}\theta_0\).

9.3 Proof of Theorem 4 for EE-DML-PPCI↩︎

We give the proof for the cross-fitted version. The proof without cross-fitting is the same after replacing the conditional empirical-process argument by the corresponding Donsker-type stochastic equicontinuity condition.

For fixed \((\gamma,\alpha)\), define \[\begin{align} f_{\gamma,\alpha}(W) \mathrel{\vcenter{:}}= \alpha(X)\left(Y-\gamma(X)\right)+\kappa m(X,\gamma), \qquad u_\gamma\big({\widetilde{X}}\big) \mathrel{\vcenter{:}}= (1-\kappa)m({\widetilde{X}},\gamma). \end{align}\] Let \[\begin{align} f_0(W)\mathrel{\vcenter{:}}= f_{\gamma_0,\alpha_{0,\kappa}}(W), \qquad u_0\big({\widetilde{X}}\big)\mathrel{\vcenter{:}}= u_{\gamma_0}\big({\widetilde{X}}\big). \end{align}\] Then, we have \[\begin{align} {\mathbb{E}}_{P_0}\left[f_0(W)\right]+{\mathbb{E}}_{Q_{0X}}\left[u_0\big({\widetilde{X}}\big)\right]=\theta_0. \end{align}\] Moreover, we have \[\begin{align} \frac{1}{n}\sum^n_{i=1}\left(f_0(W_i)-{\mathbb{E}}_{P_0}\left[f_0(W)\right]\right) &= \frac{1}{n}\sum^n_{i=1}\psi^{\mathrm{TS}}_0(W_i), \tag{49} \\ \frac{1}{m}\sum^m_{j=1}\left(u_0({\widetilde{X}}_j)-{\mathbb{E}}_{Q_{0X}}[u_0\big({\widetilde{X}}\big)]\right) &= \frac{1}{m}\sum^m_{j=1}\widetilde{\psi}^{\mathrm{TS}}_0({\widetilde{X}}_j). \tag{50} \end{align}\] For each fold \(k\), write \[\begin{align} \widehat f_k(W)\mathrel{\vcenter{:}}= f_{\widehat\gamma_k,\widehat\alpha_k}(W), \qquad \widehat u_k\big({\widetilde{X}}\big)\mathrel{\vcenter{:}}= u_{\widehat\gamma_k}\big({\widetilde{X}}\big). \end{align}\] By the definition of \(\widehat\theta^{\text{TS}}_{\mathrm{EE}}\), we have \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{EE}}-\theta_0 &= \frac{1}{n}\sum^K_{k=1}\sum_{i\in {\mathcal{I}}_k}\widehat f_k(W_i) + \frac{1}{m}\sum^K_{k=1}\sum_{j\in {\mathcal{J}}_k}\widehat u_k({\widetilde{X}}_j) -\theta_0 \nonumber\\ &= \frac{1}{n}\sum^n_{i=1}\psi^{\mathrm{TS}}_0(W_i) + \frac{1}{m}\sum^m_{j=1}\widetilde{\psi}^{\mathrm{TS}}_0({\widetilde{X}}_j) +R_{1N}+R_{2N}, \label{eq:app-ee-decomposition} \end{align}\tag{51}\] where \[\begin{align} R_{1N} &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^K_{k=1}\sum_{i\in {\mathcal{I}}_k} \left[ \left(\widehat f_k(W_i)-f_0(W_i)\right) - {\mathbb{E}}_{P_0}\left[\widehat f_k(W)-f_0(W)\mid {\mathcal{T}}_k\right] \right] \nonumber\\ &\quad + \frac{1}{m}\sum^K_{k=1}\sum_{j\in {\mathcal{J}}_k} \left[ \left(\widehat u_k({\widetilde{X}}_j)-u_0({\widetilde{X}}_j)\right) - {\mathbb{E}}_{Q_{0X}}\left[\widehat u_k\big({\widetilde{X}}\big)-u_0\big({\widetilde{X}}\big)\mid {\mathcal{T}}_k\right] \right], \label{eq:app-ee-R1} \end{align}\tag{52}\] and \[\begin{align} R_{2N} &\mathrel{\vcenter{:}}= \frac{1}{n}\sum^K_{k=1} |{\mathcal{I}}_k| {\mathbb{E}}_{P_0}\left[\widehat f_k(W)-f_0(W)\mid {\mathcal{T}}_k\right] + \frac{1}{m}\sum^K_{k=1} |{\mathcal{J}}_k| {\mathbb{E}}_{Q_{0X}}\left[\widehat u_k\big({\widetilde{X}}\big)-u_0\big({\widetilde{X}}\big)\mid {\mathcal{T}}_k\right]. \label{eq:app-ee-R2} \end{align}\tag{53}\] Here \({\mathcal{T}}_k\) denotes the training sample used to construct \(\widehat\gamma_k\) and \(\widehat\alpha_k\). Because of cross-fitting, \({\mathcal{T}}_k\) is independent of the held-out observations in \({\mathcal{I}}_k\) and \({\mathcal{J}}_k\).

We first show that \(R_{2N}=o_p(N^{-1/2})\). Conditional on \({\mathcal{T}}_k\), the population deviation in fold \(k\) is \[\begin{align} &{\mathbb{E}}_{P_0}\left[\widehat\alpha_k(X)\left(Y-\widehat\gamma_k(X)\right)+\kappa m(X,\widehat\gamma_k) -\alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right)-\kappa m(X,\gamma_0)\mid {\mathcal{T}}_k\right] \nonumber\\ &\quad +(1-\kappa){\mathbb{E}}_{Q_{0X}}\left[m({\widetilde{X}},\widehat\gamma_k)-m({\widetilde{X}},\gamma_0)\mid {\mathcal{T}}_k\right]. \end{align}\] The residual term satisfies \[\begin{align} {\mathbb{E}}_{P_0}\left[\widehat\alpha_k(X)\left(Y-\widehat\gamma_k(X)\right)\mid X,{\mathcal{T}}_k\right] = \widehat\alpha_k(X)\{\gamma_0(X)-\widehat\gamma_k(X)\}. \end{align}\] Using the Riesz representer theorem and the linearity of \(m\), the deviation becomes \[\begin{align} &{\mathbb{E}}_{P_0}\left[\widehat\alpha_k(X)\{\gamma_0(X)-\widehat\gamma_k(X)\}\mid {\mathcal{T}}_k\right] + {\mathcal{L}}_{0,\kappa}(\widehat\gamma_k-\gamma_0) \nonumber\\ &= {\mathbb{E}}_{P_0}\left[\left(\alpha_{0,\kappa}(X)-\widehat\alpha_k(X)\right) \left(\widehat\gamma_k(X)-\gamma_0(X)\right)\mid {\mathcal{T}}_k\right]. \label{eq:app-ee-drift-product} \end{align}\tag{54}\] Therefore, by Cauchy–Schwarz, \[\begin{align} |R_{2N}| &\le \max_{1\le k\le K} \|\widehat\alpha_k-\alpha_{0,\kappa}\|_{P,2} \|\widehat\gamma_k-\gamma_0\|_{P,2} =o_p(N^{-1/2}). \label{eq:app-ee-R2-rate} \end{align}\tag{55}\]

We next show that \(R_{1N}=o_p(N^{-1/2})\). Conditional on the training samples, each summand in 52 has mean zero. Its conditional variance is bounded by \[\begin{align} &\frac{C}{n} \max_k{\mathbb{E}}_{P_0}\left[\{\widehat f_k(W)-f_0(W)\}^2\mid{\mathcal{T}}_k\right] + \frac{C}{m} \max_k{\mathbb{E}}_{Q_{0X}}\left[\{\widehat u_k\big({\widetilde{X}}\big)-u_0\big({\widetilde{X}}\big)\}^2\mid{\mathcal{T}}_k\right], \label{eq:app-ee-R1-var} \end{align}\tag{56}\] where \(C\) is finite because \(K\) is fixed and the folds are balanced. The first conditional second moment is \(o_p(1)\) by Assumption 4, the bounded moment condition, and the mean-square consistency of \(\widehat\alpha_k\) and \(\widehat\gamma_k\). The second conditional second moment is \(o_p(1)\) by the consistency of \(m(\cdot,\widehat\gamma_k)\) under \(Q_{0X}\). Since \(n=\rho N\) and \(m=(1-\rho)N\), the conditional variance in 56 is \(o_p(N^{-1})\). Conditional Chebyshev’s inequality gives \(R_{1N}=o_p(N^{-1/2})\).

Substituting the bounds for \(R_{1N}\) and \(R_{2N}\) into 51 proves ?? . The two leading sums are independent. Their variances satisfy \[\begin{align} N\mathrm{Var}\left( \frac{1}{n}\sum^n_{i=1}\psi^{\mathrm{TS}}_0(W_i) + \frac{1}{m}\sum^m_{j=1}\widetilde{\psi}^{\mathrm{TS}}_0({\widetilde{X}}_j) \right) = V^{\mathrm{TS}}_0(\kappa,\rho), \end{align}\] using \(n=\rho N\) and \(m=(1-\rho)N\). The Lindeberg condition follows from the uniformly bounded \((2+\delta)\)-moment assumption. Hence the central limit theorem for independent triangular arrays yields ?? .

9.4 Proof of the Equivalence between TMLE-DML-PPCI and EE-DML-PPCI↩︎

We prove the claim under Assumption 6 and the product-rate condition in Assumption 4. For simplicity of notation, write the proof for one held-out fold and then average over folds. We first record why \(\widehat\varepsilon\) takes the form in 4 . Let \[\begin{align} A_N \mathrel{\vcenter{:}}= \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\widehat\gamma(X_i)\right), \qquad D_N \mathrel{\vcenter{:}}= \kappa\frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)^2 +(1-\kappa)\frac{1}{m}\sum^m_{j=1}\widehat\alpha({\widetilde{X}}_j)^2 . \label{eq:app-tmle-A-D} \end{align}\tag{57}\] The numerator \(A_N\) is the residual score for the fluctuation direction \(\widehat\alpha\). It is computed only from labeled observations, because only those observations contain outcomes. The density-ratio adjustment required to evaluate the score under the target regressor law is already included in \(\widehat\alpha\). The denominator \(D_N\) estimates the target-law second moment of \(\widehat\alpha\), and the target law is available through the empirical mixture of labeled and unlabeled regressors. Equivalently, \(\widehat\varepsilon\) is obtained by maximizing the local importance-weighted Gaussian working criterion \[\begin{align} \varepsilon\mapsto \varepsilon A_N-\frac{1}{2}\varepsilon^2D_N. \label{eq:app-tmle-local-likelihood} \end{align}\tag{58}\] The first-order condition for 58 is \(A_N-\varepsilon D_N=0\), and therefore \(\widehat\varepsilon=A_N/D_N\), which is exactly 4 . Because \(m(X,\gamma)\) is linear in \(\gamma\), the targeted update \(\widehat\gamma^{(1)}=\widehat\gamma+\widehat\varepsilon\widehat\alpha\) implies \[\begin{align} m(X,\widehat\gamma^{(1)}) = m(X,\widehat\gamma)+\widehat\varepsilon m(X,\widehat\alpha), \qquad m({\widetilde{X}},\widehat\gamma^{(1)}) = m({\widetilde{X}},\widehat\gamma)+\widehat\varepsilon m({\widetilde{X}},\widehat\alpha). \end{align}\] Therefore, the plug-in change caused by targeting is \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{TMLE}}-\widehat\theta_{\mathrm{plug}} = \widehat\varepsilon \left( \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\widehat\alpha) + \frac{1-\kappa}{m}\sum^m_{j=1} m({\widetilde{X}}_j,\widehat\alpha) \right), \label{eq:app-tmle-plugin-change} \end{align}\tag{59}\] where \(\widehat\theta_{\mathrm{plug}}\) is the same semi-supervised plug-in estimator based on \(\widehat\gamma\). By the definition of \(\widehat\varepsilon\) in 4 and 57 , \(\widehat\varepsilon=A_N/D_N\). The EE estimator is \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{EE}} = \widehat\theta_{\mathrm{plug}}+A_N. \end{align}\] Combining this identity with 59 gives \[\begin{align} \widehat\theta^{\text{TS}}_{\mathrm{TMLE}}-\widehat\theta^{\text{TS}}_{\mathrm{EE}} = A_N \left( \frac{L_N}{D_N}-1 \right), \label{eq:app-tmle-ee-difference} \end{align}\tag{60}\] where \[\begin{align} L_N=\frac{\kappa}{n}\sum^n_{i=1} m(X_i,\widehat\alpha) + \frac{1-\kappa}{m}\sum^m_{j=1} m({\widetilde{X}}_j,\widehat\alpha). \end{align}\] The TMLE assumptions imply that \(D_N\) is bounded away from zero and \(L_N-D_N=o_p(1)\). This condition connects the importance-weighted update to the orthogonal score, because \(D_N\) is the empirical second moment under the density-ratio-corrected regressor distribution while \(L_N\) is the plug-in derivative of the target along \(\widehat\alpha\). The same assumptions imply \(A_N=O_p(N^{-1/2})\). Therefore, 60 is \(o_p(N^{-1/2})\). Applying this fold by fold and using that \(K\) is fixed proves the equivalence between TMLE-DML-PPCI and EE-DML-PPCI.

10 Proofs for Section 5↩︎

10.1 Proof of the Error Product Property in the Influence Function↩︎

For any candidate \((\gamma,\alpha)\), the population score error is \[\begin{align} &{\mathbb{E}}_{P_0}\left[\alpha(X)\left(Y-\gamma(X)\right)\right]+ {\mathcal{L}}_{0,\kappa}(\gamma)-\theta_0 \nonumber\\ &= {\mathbb{E}}_{P_0}\left[\alpha(X)\left(\gamma_0(X)-\gamma(X)\right)\right] + {\mathcal{L}}_{0,\kappa}(\gamma-\gamma_0). \end{align}\] From the Riesz representer theorem, we have \[\begin{align} {\mathcal{L}}_{0,\kappa}(\gamma-\gamma_0) = {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)\left(\gamma(X)-\gamma_0(X)\right)\right]. \end{align}\] Combining the two equations gives \[\begin{align} {\mathbb{E}}_{P_0}\left[\alpha(X)\left(Y-\gamma(X)\right)\right] + {\mathcal{L}}_{0,\kappa}(\gamma)-\theta_0 = {\mathbb{E}}_{P_0}\left[\left(\alpha_{0,\kappa}(X)-\alpha(X)\right)\left(\gamma(X)-\gamma_0(X)\right)\right], \end{align}\] which is ?? .

10.2 Proof of the Bregman–Riesz Identity↩︎

For any \(\alpha\), recall that \(u_\alpha=\partial g\circ\alpha\) and \(b_\alpha=u_\alpha\alpha-g(\alpha)\). From the Riesz representer theorem, we have \[\begin{align} {\mathcal{L}}_{0,\kappa}(u_\alpha)={\mathbb{E}}_{P_{0X}}\left[\alpha_{0,\kappa}(X)u_\alpha(X)\right]. \end{align}\] Therefore, \[\begin{align} {\mathcal{R}}_{g,\kappa}(\alpha) &= {\mathbb{E}}_{P_{0X}}\left[u_\alpha(X)\alpha(X)-g\left(\alpha(X)\right)-\alpha_{0,\kappa}(X)u_\alpha(X)\right]. \end{align}\] At \(\alpha=\alpha_{0,\kappa}\), this becomes \[\begin{align} {\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) = {\mathbb{E}}_{P_{0X}}\left[-g\{\alpha_{0,\kappa}(X)\}\right]. \end{align}\] Subtracting the two equations gives \[\begin{align} {\mathcal{R}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) &= {\mathbb{E}}_{P_{0X}}\left[ g\{\alpha_{0,\kappa}(X)\} -g\{\alpha(X)\} -\partial g\{\alpha(X)\}\{\alpha_{0,\kappa}(X)-\alpha(X)\} \right] \\ &= {\mathbb{E}}_{P_{0X}}\left[\text{BD}_g^\dagger\left(\alpha_{0,\kappa}(X)\mid\alpha(X)\right)\right]. \end{align}\] This proves ?? . Since \(g\) is convex, the right-hand side is nonnegative. If \(g\) is strictly convex, it equals zero only when \(\alpha(X)=\alpha_{0,\kappa}(X)\) almost surely.

10.3 Proof of Proposition 5↩︎

Let \[\begin{align} Q_n(\beta) = \frac{1}{n}\sum^n_{i=1} g^*\big(\phi(X_i)^\top\beta\big) - \widehat{{\mathcal{L}}}_\kappa(\phi^\top\beta) + \lambda_n\Omega_q(\beta). \end{align}\] For the first term, by the chain rule and the equation \(\partial g^*=(\partial g)^{-1}\), we have \[\begin{align} \frac{\partial}{\partial\beta_j} \frac{1}{n}\sum^n_{i=1} g^*\big(\phi(X_i)^\top\beta\big) = \frac{1}{n}\sum^n_{i=1} \alpha_\beta(X_i)\phi_j(X_i). \end{align}\] For the linear functional term, linearity gives \[\begin{align} \frac{\partial}{\partial\beta_j}\widehat{{\mathcal{L}}}_\kappa(\phi^\top\beta) &= \widehat{{\mathcal{L}}}_\kappa(\phi_j) \\ &= \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\phi_j) + \frac{1-\kappa}{m}\sum_{j'=1}^m m({\widetilde{X}}_{j'},\phi_j). \end{align}\] Thus, any subgradient first-order condition at \(\widehat\beta\) gives \[\begin{align} \frac{1}{n}\sum^n_{i=1} \widehat\alpha(X_i)\phi_j(X_i) - \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\phi_j) - \frac{1-\kappa}{m}\sum_{j'=1}^m m({\widetilde{X}}_{j'},\phi_j) + \lambda_n s_j =0. \end{align}\] The first three terms are exactly \(\widehat\Delta_\kappa(\widehat\alpha,\phi_j)\). Hence, ?? holds. The claims for \(\lambda_n=0\), \(q=1\), and \(q>1\) follow from the subgradient of \(|\beta_j|^q/q\).

10.4 Proof of the Neyman-Error Decomposition↩︎

By definition, \[\begin{align} \mathrm{NE}_{n,m}^{\mathrm{TS}}(\widehat\gamma,\widehat\alpha) &= \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\widehat\gamma(X_i)\right) +\widehat{{\mathcal{L}}}_\kappa(\widehat\gamma)-\widehat{{\mathcal{L}}}_\kappa(\gamma_0). \end{align}\] Since \(Y_i=\gamma_0(X_i)+\varepsilon_i\), \[\begin{align} \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\left(Y_i-\widehat\gamma(X_i)\right) &= \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\varepsilon_i - \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\{\widehat\gamma(X_i)-\gamma_0(X_i)\}. \end{align}\] By linearity of \(\widehat{{\mathcal{L}}}_\kappa\), \[\begin{align} \widehat{{\mathcal{L}}}_\kappa(\widehat\gamma)-\widehat{{\mathcal{L}}}_\kappa(\gamma_0) = \widehat{{\mathcal{L}}}_\kappa(\widehat\gamma-\gamma_0). \end{align}\] Combining the last two equations and using the definition of \(\widehat\Delta_\kappa\) yields \[\begin{align} \mathrm{NE}_{n,m}^{\mathrm{TS}}(\widehat\gamma,\widehat\alpha) = \frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)\varepsilon_i - \widehat\Delta_\kappa\left(\widehat\alpha,\widehat\gamma-\gamma_0\right). \end{align}\] If \(\widehat\gamma-\gamma_0=a^\top\phi+r\), then linearity of \(\widehat\Delta_\kappa\) gives \[\begin{align} \widehat\Delta_\kappa\left(\widehat\alpha,\widehat\gamma-\gamma_0\right) = \sum_{j=1}^p a_j\widehat\Delta_\kappa(\widehat\alpha,\phi_j) + \widehat\Delta_\kappa(\widehat\alpha,r). \end{align}\] Under the \(\ell_1\) KKT bound \(|\widehat\Delta_\kappa(\widehat\alpha,\phi_j)|\le\lambda_n\), it holds that \[\begin{align} \left|\widehat\Delta_\kappa\left(\widehat\alpha,\widehat\gamma-\gamma_0\right)\right| \le \lambda_n\|a\|_1+|\widehat\Delta_\kappa(\widehat\alpha,r)|. \end{align}\] This proves 24 and 25 .

10.5 Proof of Theorem 6↩︎

Define the empirical-process deviation \[\begin{align} \mathbb{G}_{n,m}(\alpha) &\mathrel{\vcenter{:}}= \widehat{{\mathcal{R}}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha). \end{align}\] Using 17 and 18 , and using the definitions \(a_\alpha=b_\alpha-\kappa c_\alpha\) and \(c_\alpha=m(\cdot,u_\alpha)\), we have \[\begin{align} \mathbb{G}_{n,m}(\alpha) &= \left(\frac{1}{n}\sum^n_{i=1} a_\alpha(X_i)-{\mathbb{E}}_{P_{0X}}\left[a_\alpha(X)\right]\right) -(1-\kappa) \left(\frac{1}{m}\sum^m_{j=1} c_\alpha({\widetilde{X}}_j)-{\mathbb{E}}_{Q_{0X}}\left[c_\alpha\big({\widetilde{X}}\big)\right]\right). \label{eq:app-grr-emp-process} \end{align}\tag{61}\] By the definition of \(\widehat\alpha\), for every \(\alpha\in{\mathcal{H}}_n\), \[\begin{align} \widehat{{\mathcal{R}}}_{g,\kappa}(\widehat\alpha)+\lambda_nReg_\alpha(\widehat\alpha) \le \widehat{{\mathcal{R}}}_{g,\kappa}(\alpha)+\lambda_nReg_\alpha(\alpha). \end{align}\] Add and subtract the population risks. Since \(Reg_\alpha(\widehat\alpha)\ge0\), it follows that \[\begin{align} {\mathcal{R}}_{g,\kappa}(\widehat\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) &\le {\mathcal{R}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) +\lambda_nReg_\alpha(\alpha) +\mathbb{G}_{n,m}(\alpha)-\mathbb{G}_{n,m}(\widehat\alpha) \nonumber\\ &\le {\mathcal{R}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) +\lambda_nReg_\alpha(\alpha) +2\sup_{\bar\alpha\in{\mathcal{H}}_n}|\mathbb{G}_{n,m}(\bar\alpha)|. \label{eq:app-grr-basic-ineq} \end{align}\tag{62}\] This is the basic inequality.

We now control the supremum in 62 . For the labeled part in 61 , symmetrization gives \[\begin{align} {\mathbb{E}}\left[ \sup_{\alpha\in{\mathcal{H}}_n} \left|\frac{1}{n}\sum^n_{i=1} a_\alpha(X_i)-{\mathbb{E}}_{P_{0X}}a_\alpha(X)\right| \right] \le 2\mathfrak R_{n,P_{0X}}({\mathcal{A}}_n). \end{align}\] The bounded-difference inequality with envelope \(B_A\) gives that, with probability at least \(1-\delta/2\), \[\begin{align} \sup_{\alpha\in{\mathcal{H}}_n} \left|\frac{1}{n}\sum^n_{i=1} a_\alpha(X_i)-{\mathbb{E}}_{P_{0X}}a_\alpha(X)\right| \le 2\mathfrak R_{n,P_{0X}}({\mathcal{A}}_n)+B_A\sqrt{\frac{\log(4/\delta)}{2n}}. \end{align}\] The same argument applied to the unlabeled part gives that, with probability at least \(1-\delta/2\), \[\begin{align} \sup_{\alpha\in{\mathcal{H}}_n} \left|\frac{1}{m}\sum^m_{j=1} c_\alpha({\widetilde{X}}_j)-{\mathbb{E}}_{Q_{0X}}c_\alpha\big({\widetilde{X}}\big)\right| \le 2\mathfrak R_{m,Q_{0X}}({\mathcal{C}}_n)+B_C\sqrt{\frac{\log(4/\delta)}{2m}}. \end{align}\] Taking a union bound and multiplying the unlabeled display by \((1-\kappa)\) yields \[\begin{align} \sup_{\alpha\in{\mathcal{H}}_n}|\mathbb{G}_{n,m}(\alpha)| \le \mathfrak E_{n,m}(\delta) \end{align}\] with probability at least \(1-\delta\).

Substitute this bound into 62 and take the infimum over \(\alpha\in{\mathcal{H}}_n\). From the Bregman representer theorem ?? , \[\begin{align} {\mathbb{E}}_{P_{0X}}\left[\text{BD}_g^\dagger\left(\alpha_{0,\kappa}(X)\mid\widehat\alpha(X)\right)\right] &\le \inf_{\alpha\in{\mathcal{H}}_n} \left\{ {\mathbb{E}}_{P_{0X}}\left[\text{BD}_g^\dagger\{\alpha_{0,\kappa}(X)\mid\alpha(X)\}\right] +\lambda_nReg_\alpha(\alpha) \right\} +2\mathfrak E_{n,m}(\delta). \end{align}\] Finally, the lower strong-convexity bound on \(g\) implies \[\begin{align} \text{BD}_g^\dagger(a_0\mid a) \ge \frac{\underline g}{2}(a-a_0)^2 \qquad \text{for all } a_0,a\in{\mathcal{A}}. \end{align}\] Combining the last two equations proves ?? . If \(\alpha_{0,\kappa}\in{\mathcal{H}}_n\), setting \(\alpha=\alpha_{0,\kappa}\) in the infimum gives ?? .

10.6 Proof of Corollary 1↩︎

Let \(\alpha_n^*\) be a near-best approximation in \({\mathcal{H}}_n\). Define the excess risk \[\begin{align} \mathcal{E}(\alpha) \mathrel{\vcenter{:}}= {\mathcal{R}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}). \end{align}\] From the Bregman representer theorem and the smoothness and strong-convexity bounds, \(\mathcal{E}(\alpha)\) is equivalent to \(\|\alpha-\alpha_{0,\kappa}\|_{P,2}^2\) on \({\mathcal{A}}\). The basic inequality centered at \(\alpha_n^*\) gives \[\begin{align} \mathcal{E}(\widehat\alpha) &\le \mathcal{E}(\alpha_n^*)+\lambda_nReg_\alpha(\alpha_n^*) + \{ \mathbb{G}_{n,m}(\alpha_n^*)- \mathbb{G}_{n,m}(\widehat\alpha) \}. \label{eq:app-fast-basic} \end{align}\tag{63}\] The localized Bernstein condition states that the variance of each centered loss difference is bounded by a constant multiple of \(\mathcal{E}(\alpha)+\mathcal{E}(\alpha_n^*)\). The finite pseudo-dimension condition implies the following localized empirical-process inequality: with probability at least \(1-\delta\), uniformly over \(\alpha\in{\mathcal{H}}_n\), \[\begin{align} |\mathbb{G}_{n,m}(\alpha)-\mathbb{G}_{n,m}(\alpha_n^*)| &\le C_1 r_{n,m}(\delta) \sqrt{\mathcal{E}(\alpha)+\mathcal{E}(\alpha_n^*)} +C_2 r_{n,m}^2(\delta), \label{eq:app-local-emp-process} \end{align}\tag{64}\] where \(C_1\) and \(C_2\) depend only on the envelope and Bernstein constants. This inequality follows by peeling the class according to the dyadic shells \(2^{r-1}r_{n,m}^2(\delta)<\mathcal{E}(\alpha)+\mathcal{E}(\alpha_n^*)\le2^r r_{n,m}^2(\delta)\), applying the pseudo-dimension entropy bound on each shell, and summing the resulting exponential probabilities. The labeled and unlabeled empirical processes are handled separately and are then combined by a union bound.

Applying 64 to \(\widehat\alpha\) in 63 yields \[\begin{align} \mathcal{E}(\widehat\alpha) &\le A_n +C_1 r_{n,m}(\delta) \sqrt{\mathcal{E}(\widehat\alpha)+\mathcal{E}(\alpha_n^*)} +C_2 r_{n,m}^2(\delta), \label{eq:app-fast-before-absorb} \end{align}\tag{65}\] where \(A_n=\mathcal{E}(\alpha_n^*)+\lambda_nReg_\alpha(\alpha_n^*)\). Since \(\mathcal{E}(\alpha_n^*)\le A_n\), the square-root term in 65 is bounded by \[\begin{align} C_1 r_{n,m}(\delta)\sqrt{\mathcal{E}(\widehat\alpha)+A_n}. \end{align}\] Using \(ab\le a^2/(4C)+Cb^2\) with a sufficiently large constant, this term can be absorbed into the left-hand side. Hence, \[\begin{align} \mathcal{E}(\widehat\alpha) \le C\{A_n+r_{n,m}^2(\delta)\}. \end{align}\] These bounds give \(\mathcal{E}(\alpha_n^*)\le C\|\alpha_n^*-\alpha_{0,\kappa}\|_{P,2}^2\) and \(\|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2}^2\le C\mathcal{E}(\widehat\alpha)\). Combining these inequalities gives ?? .

10.7 Proof of Theorem 7↩︎

Let \(M_N=N_{\min}\). Under the Hölder condition on \(f_0\), the chosen clipped ReLU class contains a function \(f_N^*\) satisfying \[\begin{align} \|f_N^*-f_0\|_\infty^2 \le C_A M_N^{-2s/(d+2s)}\log^{c_A}M_N \end{align}\] for constants \(C_A\) and \(c_A\). Let \(\alpha_N^*=(\partial g)^{-1}\circ f_N^*\). Since \((\partial g)^{-1}\) is Lipschitz on the compact dual range, \[\begin{align} \|\alpha_N^*-\alpha_{0,\kappa}\|_{P,2}^2 \le C\|f_N^*-f_0\|_\infty^2 \le C M_N^{-2s/(d+2s)}\log^{c_A}M_N. \label{eq:app-relu-approx-alpha} \end{align}\tag{66}\] The architecture is chosen so that its pseudo-dimension satisfies \[\begin{align} V_n\log(n+m)\left(\frac{1}{n}+\frac{1}{m}\right) \le C_E M_N^{-2s/(d+2s)}\log^{c_E}M_N \label{eq:app-relu-estimation-term} \end{align}\tag{67}\] for constants \(C_E\) and \(c_E\). This is the bias-variance balance: the approximation error in 66 and the finite pseudo-dimension term in 67 have the same polynomial order. Substituting 66 and 67 into Corollary 1 gives \[\begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2}^2 = O_p\left(M_N^{-2s/(d+2s)}\log^c M_N+\lambda_nReg_\alpha(\alpha_N^*)\right) \end{align}\] for \(c\ge\max\left\{c_A,c_E\right\}\). Since \(M_N=N_{\min}\), this proves ?? . If \(n=\rho N\) and \(m=(1-\rho)N\), then \(N_{\min}=N\min\left\{\rho,1-\rho\right\}\), so the same polynomial rate can be written in terms of \(N\). Taking square roots gives ?? .

10.8 Proof of Proposition 8↩︎

When \(m(X,\gamma)=\gamma(X)\) and \(\kappa=0\), from the Riesz representer theorem, we have \[\begin{align} {\mathbb{E}}_{Q_{0X}}\left[\gamma\big({\widetilde{X}}\big)\right] = {\mathbb{E}}_{P_{0X}}\left[\alpha_{0,0}(X)\gamma(X)\right] \qquad \text{for all }\gamma\in\Gamma. \end{align}\] Therefore, \(\alpha_{0,0}(x)=q_{0X}(x)/p_{0X}(x)\). Let \(D_0=\log\alpha_{0,0}\). In the dual parametrization \(\alpha_D=\exp(D)\), the population objective becomes \[\begin{align} {\mathbb{E}}_{P_{0X}}\left[\partial g\{\exp(D(X))\}\exp(D(X))-g\{\exp(D(X))\}\right] - {\mathbb{E}}_{Q_{0X}}\left[\partial g\{\exp(D\big({\widetilde{X}}\big))\}\right], \end{align}\] which is exactly the Bregman objective for the log density ratio. The empirical objective has the same two-sample form with sample sizes \(n\) and \(m\). Applying the same localized empirical-process argument as in the proof of Theorem 7, with the sharper logarithmic entropy calculation for the log-density-ratio loss, gives \[\begin{align} \|\widehat D-D_0\|_{L^2(P_{0X})}^2 = O_p\left(N_{\min}^{-2s/(d+2s)}\log^3N_{\min}\right). \end{align}\] Because \(D_0\) and \(\widehat D\) are clipped to a bounded interval, the exponential map is Lipschitz on that interval. Hence, \[\begin{align} \|\exp(\widehat D)-\exp(D_0)\|_{L^2(P_{0X})}^2 \le C\|\widehat D-D_0\|_{L^2(P_{0X})}^2, \end{align}\] which proves ?? . For \(\kappa\in(0,1)\), the mean-type representer is \(\alpha_{0,\kappa}=\kappa+(1-\kappa)\exp(D_0)\), and the estimator \(\widehat\alpha_\kappa=\kappa+(1-\kappa)\exp(\widehat D)\) satisfies \[\begin{align} \|\widehat\alpha_\kappa-\alpha_{0,\kappa}\|_{P,2}^2 = (1-\kappa)^2\|\exp(\widehat D)-\exp(D_0)\|_{P,2}^2. \end{align}\] This gives the stated extension.

10.9 Proof of Theorem 9↩︎

Let \({\mathcal{X}}_N=\{x\colon\|x\|_\infty\le C_1\log N_{\min}\}\). Decompose the squared error as \[\begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2}^2 = {\mathbb{E}}_{P_{0X}}\left[(\widehat\alpha(X)-\alpha_{0,\kappa}(X))^2\mathbb{1}[X\in{\mathcal{X}}_N]\right] + {\mathbb{E}}_{P_{0X}}\left[(\widehat\alpha(X)-\alpha_{0,\kappa}(X))^2\mathbb{1}[X\notin{\mathcal{X}}_N]\right]. \end{align}\] On \({\mathcal{X}}_N\), rescale covariates to \([0,1]^d\). The Hölder norm after rescaling grows only by a logarithmic factor because the side length of \({\mathcal{X}}_N\) is of order \(\log N_{\min}\). Therefore, the approximation argument in Theorem 7 gives the same polynomial order with a larger logarithmic exponent. On \({\mathcal{X}}_N^c\), clipping gives a bounded envelope for \(\widehat\alpha-\alpha_{0,\kappa}\), and the tail assumption ?? gives \[\begin{align} {\mathbb{E}}_{P_{0X}}\left[(\widehat\alpha(X)-\alpha_{0,\kappa}(X))^2\mathbb{1}[X\notin{\mathcal{X}}_N]\right] \le C N_{\min}^{-2s/(d+2s)}. \end{align}\] The same tail bound controls the unlabeled empirical-process part over \(Q_{0X}\). Substituting the truncated approximation bound, the pseudo-dimension bound, and the tail bound into Corollary 1 gives ?? .

10.10 Proof of Theorem 10↩︎

Under the approximate-manifold condition, the regressor distribution is concentrated on a \(\varrho\)-neighborhood of a compact \(d_M\)-dimensional manifold. A random or deterministic embedding with distortion \(\delta\) maps this neighborhood into a Euclidean space of dimension \(d_\delta\), preserving distances up to the prescribed relative error on the relevant support. Since \(f_0\) is Hölder, the embedded function remains Hölder with constants changed only by the distortion and manifold regularity constants. A ReLU network on the embedded coordinates therefore attains \[\begin{align} \|f_N^*-f_0\|_\infty^2 \le C N_{\min}^{-2s/(d_\delta+2s)}\log^{c_1}N_{\min}. \end{align}\] The neighborhood radius condition ensures that the error from replacing the original support by the embedded approximation is no larger than the approximation scale above. The pseudo-dimension of the resulting network is balanced so that \[\begin{align} V_n\log(n+m)\left(\frac{1}{n}+\frac{1}{m}\right) \le C N_{\min}^{-2s/(d_\delta+2s)}\log^{c_2}N_{\min}. \end{align}\] The Lipschitz property of \((\partial g)^{-1}\) transfers the approximation bound from \(f_0\) to \(\alpha_{0,\kappa}\). Substitution into Corollary 1 gives ?? .

10.11 Proof of Corollary 2↩︎

The two nuisance rates above imply \[\begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2} \|\widehat\gamma-\gamma_0\|_{P,2} = O_p\left( N^{-a}\log^{c_\alpha+c_\gamma}N \right), \end{align}\] where \[\begin{align} a= \frac{s_\alpha}{d_\alpha+2s_\alpha} + \frac{s_\gamma}{d_\gamma+2s_\gamma}. \end{align}\] If ?? holds, then \(a>1/2\). Therefore, for a sufficiently small \(\eta>0\), it holds that \(a\ge1/2+\eta\). Polynomial decay \(N^{-\eta}\) dominates the logarithmic factor, and thus \[\begin{align} N^{1/2} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2} \|\widehat\gamma-\gamma_0\|_{P,2} =o_p(1). \end{align}\] This is exactly the product-rate condition in Assumption 4.

11 Proofs for Section 6↩︎

11.1 Proof of Theorem 11↩︎

We work directly with the observed-data model. The observed data are \(O=(X,S,Y)\), where \(Y=SY^*+(1-S)\mathrm{NA}\). Under missing at random, the observed-data law can be decomposed into the marginal law of \(X\), the labeling law of \(S\) given \(X\), and the conditional law of \(Y^*\) given \(X\), which is observed only when \(S=1\). Along a regular parametric submodel, write the observed-data score as \[\begin{align} s_O(O)=s_X(X)+s_{S\mid X}(S,X)+S s_{Y\mid X}(Y^*,X), \end{align}\] where \[\begin{align} {\mathbb{E}}\left[s_X(X)\right]=0, \qquad {\mathbb{E}}\left[s_{S\mid X}(S,X)\mid X\right]=0, \qquad {\mathbb{E}}\left[s_{Y\mid X}(Y^*,X)\mid X\right]=0. \end{align}\] The term \(S s_{Y\mid X}(Y^*,X)\) is observable because it is zero when \(S=0\), and when \(S=1\) the observed outcome satisfies \(Y=Y^*\). This is the observed-data score decomposition associated with the censoring setting.

Let \(\gamma_t(x)={\mathbb{E}}_t[Y^*\mid X=x]\). The labeling law does not enter this conditional mean. Differentiating the conditional mean under the conditional outcome score gives \[\begin{align} \dot{\gamma}_0(x) &\mathrel{\vcenter{:}}= \frac{\partial}{\partial t}\gamma_t(x)|_{t=0} = {\mathbb{E}}\left[\left(Y^*-\gamma_0(X)\right)s_{Y\mid X}(Y^*,X)\mid X=x\right]. \label{eq:app-os-gamma-derivative-updated} \end{align}\tag{68}\] The target along the submodel is \[\begin{align} \theta_t^{\mathrm{OS}}={\mathbb{E}}_{P_{tX}}\left[m(X,\gamma_t)\right]. \end{align}\] Because \(\gamma\mapsto m(X,\gamma)\) is linear, differentiating at zero yields \[\begin{align} \frac{\partial}{\partial t}\theta^{\mathrm{OS}}_t\mid_{t=0} = {\mathbb{E}}\left[m(X,\gamma_0)s_X(X)\right] + {\mathbb{E}}\left[\alpha_0^{\mathrm{OS}}(X)\dot{\gamma}_0(X)\right], \label{eq:app-os-param-derivative-updated} \end{align}\tag{69}\] where the second term follows from the Riesz representer theorem 29 .

We now verify that \(\psi_0^{\mathrm{OS}}\) represents this derivative. Since \(m(X,\gamma_0)-\theta^{\mathrm{OS}}_0\) is a function of \(X\), \[\begin{align} &{\mathbb{E}}\left[\left(m(X,\gamma_0)-\theta^{\mathrm{OS}}_0\right)s_O(O)\right] \nonumber\\ &\quad= {\mathbb{E}}\left[\left(m(X,\gamma_0)-\theta^{\mathrm{OS}}_0\right)s_X(X)\right] = {\mathbb{E}}\left[m(X,\gamma_0)s_X(X)\right]. \label{eq:app-os-x-score-updated} \end{align}\tag{70}\] The terms involving \(s_{S\mid X}\) and \(s_{Y\mid X}\) vanish after conditioning on \(X\).

Next, consider the inverse-labeling residual term. Since \({\mathbb{E}}\left[Y^*-\gamma_0(X)\mid X\right]=0\), the marginal covariate score contributes zero. The labeling score also contributes zero. Indeed, by missing at random, conditional on \(X\), the residual \(Y^*-\gamma_0(X)\) is independent of the labeling score component, and therefore \[\begin{align} {\mathbb{E}}\left[\frac{S}{\pi_0(X)}\alpha_0^{\mathrm{OS}}(X)\left(Y^*-\gamma_0(X)\right)s_{S\mid X}(S,X)\mid X\right] = \frac{\alpha_0^{\mathrm{OS}}(X)}{\pi_0(X)}{\mathbb{E}}\left[S s_{S\mid X}(S,X)\mid X\right]{\mathbb{E}}\left[Y^*-\gamma_0(X)\mid X\right] =0. \end{align}\] Thus, it holds that \[\begin{align} &{\mathbb{E}}\left[\frac{S}{\pi_0(X)}\alpha^{\mathrm{OS}}_0(X)\left(Y-\gamma_0(X)\right)s_O(O)\right] \nonumber\\ &\quad= {\mathbb{E}}\left[\frac{S}{\pi_0(X)}\alpha^{\mathrm{OS}}_0(X)\left(Y^*-\gamma_0(X)\right)S s_{Y\mid X}(Y^*,X)\right] \nonumber\\ &\quad= {\mathbb{E}}\left[\alpha^{\mathrm{OS}}_0(X)\left(Y^*-\gamma_0(X)\right)s_{Y\mid X}(Y^*,X)\right] \nonumber\\ &\quad= {\mathbb{E}}\left[\alpha^{\mathrm{OS}}_0(X)\dot{\gamma}_0(X)\right]. \label{eq:app-os-outcome-score-updated} \end{align}\tag{71}\] The second equality uses \(S^2=S\), \({\mathbb{E}}\left[S\mid X\right]=\pi_0(X)\), and missing at random.

Combining 69 , 70 , and 71 , it holds that \[\begin{align} \frac{\partial}{\partial t}\theta^{\mathrm{OS}}_t\mid_{t=0} = {\mathbb{E}}\left[\psi_0^{\mathrm{OS}}(O)s_O(O)\right]. \end{align}\] The proposed influence function is centered because \[\begin{align} {\mathbb{E}}\left[\frac{S}{\pi_0(X)}\alpha_0^{\mathrm{OS}}(X)\left(Y-\gamma_0(X)\right)\mid X\right]=0 \end{align}\] and \({\mathbb{E}}\left[m(X,\gamma_0)-\theta_0^{\mathrm{OS}}\right]=0\). The observed-data tangent space under the nonparametric missing-at-random model is generated by sums of the three score components given above. The proposed function belongs to this tangent space: its covariate part is \(m(X,\gamma_0)-\theta_0^{\mathrm{OS}}\), its labeling-score component is zero, and its conditional-outcome component is represented by \(S\alpha_0^{\mathrm{OS}}(X)\left(Y^*-\gamma_0(X)\right)/\pi_0(X)\), whose conditional mean given \(X\) is zero. Since it represents the pathwise derivative for every regular score and lies in the tangent space, it is the canonical gradient. This proves Theorem 11.

11.2 Proof of Theorem 12↩︎

By Theorem 11, the semiparametric efficiency bound under \(\sqrt N\) normalization is the second moment of the canonical gradient. Write \[\begin{align} \psi_0^{\mathrm{OS}}(O)=A(O)+B(X), \end{align}\] where \[\begin{align} A(O)=\frac{S}{\pi_0(X)}\alpha_0^{\mathrm{OS}}(X)\left(Y-\gamma_0(X)\right), \qquad B(X)=m(X,\gamma_0)-\theta_0^{\mathrm{OS}}. \end{align}\] Since \(S\left(Y-\gamma_0(X)\right)=S\left(Y^*-\gamma_0(X)\right)\), \[\begin{align} {\mathbb{E}}\left[A(O)\mid X\right] = \frac{\alpha_0^{\mathrm{OS}}(X)}{\pi_0(X)} {\mathbb{E}}\left[S\left(Y^*-\gamma_0(X)\right)\mid X\right] =0. \end{align}\] Thus, the cross term between \(A(O)\) and \(B(X)\) is zero, and \[\begin{align} {\mathbb{E}}\left[\psi_0^{\mathrm{OS}}(O)^2\right] = {\mathbb{E}}\left[A(O)^2\right]+{\mathbb{E}}\left[B(X)^2\right]. \end{align}\] For the first term, we have \[\begin{align} {\mathbb{E}}\left[A(O)^2\mid X\right] &= \frac{\left(\alpha_0^{\mathrm{OS}}(X)\right)^2}{\pi_0(X)^2} {\mathbb{E}}\left[S\left(Y^*-\gamma_0(X)\right)^2\mid X\right] \nonumber\\ &= \frac{\left(\alpha_0^{\mathrm{OS}}(X)\right)^2}{\pi_0(X)^2} \pi_0(X)\sigma_0^2(X) = \frac{\left(\alpha_0^{\mathrm{OS}}(X)\right)^2\sigma_0^2(X)}{\pi_0(X)}. \end{align}\] For the second term, we have \[\begin{align} {\mathbb{E}}\left[B(X)^2\right]= \operatorname{Var}_{P_{0X}}\left(m(X,\gamma_0)\right). \end{align}\] Combining these equations proves ?? .

11.3 Proof of Theorem 13↩︎

For fixed nuisance functions, the population drift of the one-sample EE-DML-PPCI score is \[\begin{align} {\mathbb{E}}_{P_0}\left[ \left( \alpha^{\mathrm{OS}}_0(X) - \frac{\pi_0(X)}{\widehat\pi(X)} \widehat\alpha(X) \right) \left(\widehat\gamma(X)-\gamma_0(X)\right) \right]. \end{align}\] If Assumption 10 holds and the remaining nuisance components are bounded in the required \(L^2(P_{0X})\) norms, this drift is \(o_p(1)\). If Assumption 11 holds and \(\widehat\gamma\) is bounded in \(L^2(P_{0X})\), the same conclusion follows. The empirical average converges to its population counterpart by the law of large numbers under the stated moment and overlap conditions. Hence, \(\widehat\theta^{\mathrm{OS}}_{\mathrm{EE}}\xrightarrow{{\mathrm{p}}}\theta^{\mathrm{OS}}_0\).

11.4 Proof of Theorem 14↩︎

We give the proof for the cross-fitted implementation. This implementation verifies the empirical-process requirement in Assumption 13 without imposing Donsker-type stochastic equicontinuity conditions.

Let \(K\ge 2\) be fixed. Partition \([N]\) into folds \({\mathcal{I}}_1,\ldots,{\mathcal{I}}_K\). For each fold \(k\), let \({\mathcal{I}}_{-k}\mathrel{\vcenter{:}}= [N]\setminus{\mathcal{I}}_k\), and let \({\mathcal{T}}_k\) denote the observations indexed by \({\mathcal{I}}_{-k}\). Let \(\widehat\gamma_k\), \(\widehat\alpha_k\), and \(\widehat\pi_k\) be nuisance estimators constructed from \({\mathcal{T}}_k\). The fold-specific score is evaluated only on observations in \({\mathcal{I}}_k\). Under this implementation, the EE-DML-PPCI estimator is \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} \mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} \left( \frac{S_i}{\widehat\pi_k(X_i)} \widehat\alpha_k(X_i) \left(Y_i-\widehat\gamma_k(X_i)\right) + m\left(X_i,\widehat\gamma_k\right) \right). \label{eq:app-os-ee-crossfit} \end{align}\tag{72}\] Here, \(S_i\left(Y_i-\widehat\gamma_k(X_i)\right)\) is interpreted as \(S_i\left(Y_i^*-\widehat\gamma_k(X_i)\right)\), which is observed.

For fixed nuisance functions \(\gamma\), \(\alpha\), and \(\pi\), define \[\begin{align} \varphi^{\mathrm{OS}}(O;\gamma,\alpha,\pi) \mathrel{\vcenter{:}}= \frac{S}{\pi(X)} \alpha(X) \left(Y-\gamma(X)\right) + m\left(X,\gamma\right). \end{align}\] For each fold \(k\), define \[\begin{align} \widehat\varphi^{\mathrm{OS}}_k(O) &\mathrel{\vcenter{:}}= \varphi^{\mathrm{OS}}\left(O;\widehat\gamma_k,\widehat\alpha_k,\widehat\pi_k\right), \\ \varphi^{\mathrm{OS}}_0(O) &\mathrel{\vcenter{:}}= \varphi^{\mathrm{OS}}\left(O;\gamma_0,\alpha^{\mathrm{OS}}_0,\pi_0\right). \end{align}\] Then, \[\begin{align} \varphi^{\mathrm{OS}}_0(O)-\theta^{\mathrm{OS}}_0 = \psi^{\mathrm{OS}}_0(O). \end{align}\] By adding and subtracting \(\varphi^{\mathrm{OS}}_0(O_i)\) and the conditional mean of the score difference on each fold, we obtain \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} - \theta^{\mathrm{OS}}_0 &= \frac{1}{N}\sum^N_{i=1} \psi^{\mathrm{OS}}_0(O_i) + R^{\mathrm{OS}}_{1N} + R^{\mathrm{OS}}_{2N}, \label{eq:app-os-ee-expansion} \end{align}\tag{73}\] where \[\begin{align} R^{\mathrm{OS}}_{1N} &\mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} \left( \widehat\varphi^{\mathrm{OS}}_k(O_i) - \varphi^{\mathrm{OS}}_0(O_i) - {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \mid {\mathcal{T}}_k \right] \right), \tag{74} \\ R^{\mathrm{OS}}_{2N} &\mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1} |{\mathcal{I}}_k| {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \mid {\mathcal{T}}_k \right]. \tag{75} \end{align}\]

We first control \(R^{\mathrm{OS}}_{2N}\). Conditional on \({\mathcal{T}}_k\), the nuisance estimators are fixed. Since \(Y=Y^*\) when \(S=1\) and \(Y\) appears only through \(S\left(Y-\widehat\gamma_k(X)\right)\), the missing-at-random condition gives \[\begin{align} {\mathbb{E}}_{P_0}\left[ \frac{S}{\widehat\pi_k(X)} \widehat\alpha_k(X) \left(Y-\widehat\gamma_k(X)\right) \mid X,{\mathcal{T}}_k \right] = \frac{\pi_0(X)}{\widehat\pi_k(X)} \widehat\alpha_k(X) \left(\gamma_0(X)-\widehat\gamma_k(X)\right). \label{eq:app-os-cond-mean} \end{align}\tag{76}\] Therefore, \[\begin{align} & {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \mid {\mathcal{T}}_k \right] \nonumber\\ &= {\mathbb{E}}_{P_0}\left[ \frac{\pi_0(X)}{\widehat\pi_k(X)} \widehat\alpha_k(X) \left(\gamma_0(X)-\widehat\gamma_k(X)\right) + m\left(X,\widehat\gamma_k\right) - m\left(X,\gamma_0\right) \mid {\mathcal{T}}_k \right]. \label{eq:app-os-drift-1} \end{align}\tag{77}\] By the Riesz representation for the one-sample target, \[\begin{align} {\mathbb{E}}_{P_0}\left[ m\left(X,\widehat\gamma_k\right) - m\left(X,\gamma_0\right) \mid {\mathcal{T}}_k \right] = {\mathbb{E}}_{P_0}\left[ \alpha^{\mathrm{OS}}_0(X) \left(\widehat\gamma_k(X)-\gamma_0(X)\right) \mid {\mathcal{T}}_k \right]. \label{eq:app-os-riesz} \end{align}\tag{78}\] Substituting 78 into 77 yields \[\begin{align} & {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \mid {\mathcal{T}}_k \right] \nonumber\\ &= {\mathbb{E}}_{P_0}\left[ \left( \alpha^{\mathrm{OS}}_0(X) - \frac{\pi_0(X)}{\widehat\pi_k(X)} \widehat\alpha_k(X) \right) \left( \widehat\gamma_k(X)-\gamma_0(X) \right) \mid {\mathcal{T}}_k \right]. \label{eq:app-os-drift-2} \end{align}\tag{79}\] By Cauchy–Schwarz, \[\begin{align} & \left| {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \mid {\mathcal{T}}_k \right] \right| \nonumber\\ &\le \left\| \alpha^{\mathrm{OS}}_0 - \frac{\pi_0\widehat\alpha_k}{\widehat\pi_k} \right\|_{P,2} \|\widehat\gamma_k-\gamma_0\|_{P,2}. \label{eq:app-os-drift-bound} \end{align}\tag{80}\] By the one-sample convergence rate condition, the right-hand side of 80 is \(o_p(N^{-1/2})\) uniformly over \(k\). Since \(K\) is fixed and the folds are balanced, it follows that \[\begin{align} R^{\mathrm{OS}}_{2N} = o_p(N^{-1/2}). \label{eq:app-os-r2-bound} \end{align}\tag{81}\]

We next control \(R^{\mathrm{OS}}_{1N}\). Conditional on \({\mathcal{T}}_1,\ldots,{\mathcal{T}}_K\), the observations in each held-out fold are independent of the nuisance estimators used on that fold, and each summand in 74 has conditional mean zero. Hence, \[\begin{align} & {\mathbb{E}}_{P_0}\left[ \left(R^{\mathrm{OS}}_{1N}\right)^2 \mid {\mathcal{T}}_1,\ldots,{\mathcal{T}}_K \right] \nonumber\\ &\le \frac{C}{N} \max_{1\le k\le K} {\mathbb{E}}_{P_0}\left[ \left( \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \right)^2 \mid {\mathcal{T}}_k \right], \label{eq:app-os-r1-variance} \end{align}\tag{82}\] where \(C\) is finite because \(K\) is fixed and the folds are balanced.

It remains to show that the conditional second moment in 82 is \(o_p(1)\) uniformly over \(k\). We decompose \[\begin{align} & \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \nonumber\\ &= S \left( \frac{\widehat\alpha_k(X)}{\widehat\pi_k(X)} - \frac{\alpha^{\mathrm{OS}}_0(X)}{\pi_0(X)} \right) \left(Y-\gamma_0(X)\right) \nonumber\\ &\quad - \frac{S}{\widehat\pi_k(X)} \widehat\alpha_k(X) \left(\widehat\gamma_k(X)-\gamma_0(X)\right) + m\left(X,\widehat\gamma_k\right) - m\left(X,\gamma_0\right). \label{eq:app-os-score-diff} \end{align}\tag{83}\] The first term in 83 is \(o_p(1)\) in conditional \(L^2(P_0)\) uniformly over \(k\), because \[\begin{align} \left\| \frac{\widehat\alpha_k}{\widehat\pi_k} - \frac{\alpha^{\mathrm{OS}}_0}{\pi_0} \right\|_{P,2} \le C \left\| \frac{\pi_0\widehat\alpha_k}{\widehat\pi_k} - \alpha^{\mathrm{OS}}_0 \right\|_{P,2} \end{align}\] with probability approaching one, and the residual has a uniformly bounded conditional moment. The second term in 83 is \(o_p(1)\) in conditional \(L^2(P_0)\), because \(\widehat\alpha_k/\widehat\pi_k\) has bounded second moment uniformly over folds and \(\|\widehat\gamma_k-\gamma_0\|_{P,2}=o_p(1)\). The third term is \(o_p(1)\) in \(L^2(P_0)\) by the convergence condition for \(m\left(\cdot,\widehat\gamma_k\right)\). Thus, \[\begin{align} \max_{1\le k\le K} {\mathbb{E}}_{P_0}\left[ \left( \widehat\varphi^{\mathrm{OS}}_k(O) - \varphi^{\mathrm{OS}}_0(O) \right)^2 \mid {\mathcal{T}}_k \right] = o_p(1). \label{eq:app-os-score-l2} \end{align}\tag{84}\] Combining 82 and 84 , conditional Chebyshev’s inequality gives \[\begin{align} R^{\mathrm{OS}}_{1N} = o_p(N^{-1/2}). \label{eq:app-os-r1-bound} \end{align}\tag{85}\]

Substituting 85 and 81 into 73 , we obtain \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} - \theta^{\mathrm{OS}}_0 = \frac{1}{N}\sum^N_{i=1} \psi^{\mathrm{OS}}_0(O_i) + o_p(N^{-1/2}). \label{eq:app-os-ee-al} \end{align}\tag{86}\] The observations \(O_1,\ldots,O_N\) are i.i.d. in the one-sample scenario. Moreover, \[\begin{align} {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{OS}}_0(O)\right] = 0, \qquad {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{OS}}_0(O)^2\right] = V^{\mathrm{OS}}_0 < \infty. \end{align}\] Therefore, the central limit theorem yields \[\begin{align} \sqrt N \left( \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} - \theta^{\mathrm{OS}}_0 \right) \xrightarrow{d} N\left(0,V^{\mathrm{OS}}_0\right). \label{eq:app-os-ee-clt} \end{align}\tag{87}\] This proves the assertion for EE-DML-PPCI.

We next prove the assertion for TMLE-DML-PPCI. Under the cross-fitted implementation, define \[\begin{align} \widehat A^{\mathrm{OS}}_N &\mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} \frac{S_i}{\widehat\pi_k(X_i)} \widehat\alpha_k(X_i) \left(Y_i-\widehat\gamma_k(X_i)\right), \tag{88} \\ \widehat D^{\mathrm{OS}}_N &\mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} \widehat\alpha_k^2\left(X_i\right), \tag{89} \\ \widehat\varepsilon^{\mathrm{OS}} &\mathrel{\vcenter{:}}= \frac{\widehat A^{\mathrm{OS}}_N}{\widehat D^{\mathrm{OS}}_N}. \tag{90} \end{align}\] The updated regression function on fold \(k\) is \[\begin{align} \widehat\gamma^{(1)}_k(x) \mathrel{\vcenter{:}}= \widehat\gamma_k(x) + \widehat\varepsilon^{\mathrm{OS}}\widehat\alpha_k(x). \label{eq:app-os-tmle-update} \end{align}\tag{91}\] The corresponding TMLE-DML-PPCI estimator is \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} \mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} m\left(X_i,\widehat\gamma^{(1)}_k\right). \label{eq:app-os-tmle-estimator} \end{align}\tag{92}\] Because \(\gamma\mapsto m(X,\gamma)\) is linear, \[\begin{align} m\left(X_i,\widehat\gamma^{(1)}_k\right) = m\left(X_i,\widehat\gamma_k\right) + \widehat\varepsilon^{\mathrm{OS}} m\left(X_i,\widehat\alpha_k\right). \label{eq:app-os-tmle-linearity} \end{align}\tag{93}\] Therefore, \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} &= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} m\left(X_i,\widehat\gamma_k\right) + \frac{\widehat A^{\mathrm{OS}}_N}{\widehat D^{\mathrm{OS}}_N} \widehat L^{\mathrm{OS}}_N, \label{eq:app-os-tmle-expansion} \end{align}\tag{94}\] where \[\begin{align} \widehat L^{\mathrm{OS}}_N \mathrel{\vcenter{:}}= \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} m\left(X_i,\widehat\alpha_k\right). \label{eq:app-os-tmle-L} \end{align}\tag{95}\] On the other hand, by 72 , \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} = \widehat A^{\mathrm{OS}}_N + \frac{1}{N}\sum^K_{k=1}\sum_{i\in{\mathcal{I}}_k} m\left(X_i,\widehat\gamma_k\right). \label{eq:app-os-ee-for-tmle} \end{align}\tag{96}\] Subtracting 96 from 94 gives \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} &= \widehat A^{\mathrm{OS}}_N \left( \frac{\widehat L^{\mathrm{OS}}_N}{\widehat D^{\mathrm{OS}}_N} - 1 \right) \nonumber\\ &= \widehat A^{\mathrm{OS}}_N \frac{ \widehat L^{\mathrm{OS}}_N - \widehat D^{\mathrm{OS}}_N }{ \widehat D^{\mathrm{OS}}_N }. \label{eq:app-os-tmle-ee-diff} \end{align}\tag{97}\] By the one-sample targeting calibration condition, \(\widehat A^{\mathrm{OS}}_N=O_p(N^{-1/2})\), \(\widehat L^{\mathrm{OS}}_N-\widehat D^{\mathrm{OS}}_N=o_p(1)\), and \(\widehat D^{\mathrm{OS}}_N\) is bounded away from zero with probability approaching one. Hence, \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} = o_p(N^{-1/2}). \label{eq:app-os-tmle-ee-equivalence} \end{align}\tag{98}\] Combining 86 and 98 , we have \[\begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \theta^{\mathrm{OS}}_0 = \frac{1}{N}\sum^N_{i=1} \psi^{\mathrm{OS}}_0(O_i) + o_p(N^{-1/2}). \label{eq:app-os-tmle-al} \end{align}\tag{99}\] The central limit theorem again gives \[\begin{align} \sqrt N \left( \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \theta^{\mathrm{OS}}_0 \right) \xrightarrow{d} N\left(0,V^{\mathrm{OS}}_0\right). \label{eq:app-os-tmle-clt} \end{align}\tag{100}\] This completes the proof of Theorem 14.

References↩︎

---

1. Email: mkato-csecon@g.ecc.u-tokyo.ac.jp↩︎