¹

1 Introduction↩︎

1.1 Overview↩︎

Quantile regression [1], [2] offers a powerful statistical methodology for modeling conditional quantiles for scalar response variables. Recent attention in the literature is to extend quantile regression to vector response variables, which is, however, not straightforward because of the lack of natural ordering in multidimensional space [3]. Optimal transport (OT) provides a promising approach to extending the quantile function to vector variables.² For a vector response variable \(Y \in \mathbb{R}^{d_y}\) and a given absolutely continuous reference distribution \(\mu\) on \(\mathbb{R}^{d_y}\) (such as \(\mu = \mathrm{Unif} ([0,1]^{d_y})\) or \(\mu=\mathcal{N}(0,I_{d_y})\)), several authors proposed using the Brenier map [6] transporting \(\mu\) onto the law of \(Y\) as a vector quantile function for \(Y\) [7]–[9].

In the presence of covariates \(X \in \mathbb{R}^{d_x}\), [7] considered the conditional Brenier map \(\Phi: \mathbb{R}^{d_y+d_x} \to \mathbb{R}^{d_y}\), transporting \(\mu\) onto the conditional distribution of \(Y\) given \(X\), as a conditional vector quantile function, which satisfies the distributional relation \[(X,Y) \stackrel{d}{=} (X,\Phi(U,X)), \;U \mid X \sim \mu,\] where \(\stackrel{d}{=}\) denotes equality in distribution. The joint distribution \(\pi^o\) of \((U,X,\tilde{Y})\) with \(\tilde{Y}=\Phi(U,X)\) and \(U \mid X \sim \mu\) can be characterized as an optimal solution to the OT problem with an independence constraint, \[\inf_{\pi \in \Pi(\mu,\nu)}\big \{ \mathbb{E}[\|U-\tilde{Y}\|^2] :(U,\tilde{X},\tilde{Y}) \sim \pi, \;U \mathpalette{\independenT}{\perp}\tilde{X} \big \}, \label{eq:32vqr32indep}\tag{1}\] where \(\nu\) denotes the distribution of \((X,Y)\) and \(\Pi(\mu,\nu)\) denotes the collection of couplings for \((\mu,\nu)\). Recall that any coupling \(\pi \in \Pi(\mu,\nu)\) is a joint distribution on \(\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\) with marginals \(\mu,\nu\). In addition, [7] considered modeling the conditional vector quantile function \(\Phi(u,x)\) as an affine function in \(x\), i.e., \[\Phi(u,x) = b_0(u) + b_1(u)^\top x \label{eq:32affine}\tag{2}\] for suitable mappings \(b_0: \mathbb{R}^{d_y} \to \mathbb{R}^{d_y}\) and \(b_1:\mathbb{R}^{d_y} \to \mathbb{R}^{d_x \times d_y}\). When \(\Phi\) is of the form (2 ), under technical conditions, the corresponding coupling \(\pi^o\) is optimal for the relaxed problem \[\inf_{\pi \in \Pi(\mu,\nu)} \big \{\mathbb{E}[\|U-\tilde{Y}\|^2] : (U,\tilde{X},\tilde{Y}) \sim \pi, \, \mathbb{E}[\tilde{X} \mid U]=\mathbb{E}[X] \big \}, \label{eq:32vqr32init320}\tag{3}\] which is an OT problem subject to a mean-independence constraint [7]. Following [7], we shall call (3 ) the vector quantile regression (VQR) problem, which extends (linear) quantile regression to vector response variables ([7]; see also their follow-up work [10]).

For standard OT, entropic regularization offers various computational and analytical advantages, which has prompted extensive research activities on entropic OT [11], [12]. The popularity of entropic OT stems from the fact that it is amenable to efficient computation via a dual block coordinate ascent algorithm, called Sinkhorn’s algorithm, for which rigorous convergence guarantees have been developed [13]–[17]. Dual potentials for entropic OT, which are completely characterized by the system of functional equations, called the Schrödinger system, are smooth provided that the ground cost is smooth and marginals have sufficiently light tails, which enables faster sample complexity rates than unregularized OT [18]–[22]. In addition, as the regularization parameter tends to zero, various objects of entropic OT converge to those of unregularized OT [23]–[28]

For VQR, one can consider its entropic variant by adding an entropic penalty to the primal objective as \[\inf_{\pi \in \Pi(\mu,\nu)} \big \{\mathbb{E}[\|U-\tilde{Y}\|^2/2] + \varepsilon \mathsf{KL}(\pi \,\|\, \mu\otimes \nu ) : (U,\tilde{X},\tilde{Y}) \sim \pi, \, \mathbb{E}[\tilde{X} \mid U]=\mathbb{E}[X] \big \}, \label{eq:32evqr}\tag{4}\] where \(\varepsilon > 0\) is a regularization parameter and \(\mathsf{KL}\) denotes the Kullback-Leibler (KL) divergence (or relative entropy) defined by \[\mathsf{KL}(P \,\|\, Q ) := \begin{cases} \int \log \frac{dP}{dQ}\, dP & \text{if P \ll Q}, \\ \infty & \text{otherwise}. \end{cases}\] We shall call (4 ) the entropic VQR problem. Without the presence of covariates, entropic VQR reduces to standard entropic OT. The entropic VQR problem (4 ) admits a unique optimal solution under mild conditions.

Entropic VQR was previously considered by [29] for discrete marginals as a practical means to approximately solve the VQR problem (3 ). However, to the authors’ best knowledge, the fundamental structural and approximation properties of entropic VQR are still much less understood than entropic OT. The goal of this paper is to address some of these gaps. Our contributions are summarized as follows. First, we conduct an in-depth study of duality theory for entropic VQR. At least formally (e.g., by considering the fully discrete case), the dual problem for entropic VQR can be seen as \[\sup_{(f,g,h)} \int f \,d\mu + \int h \, d\nu - \varepsilon \int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} e^{ \frac{1}{\varepsilon} \left(f(u) + \langle g(u), x\rangle + h(x,y) - \|u-y\|^2/2 \right) } \,d\mu(u) d\nu(x,y) + \varepsilon,\] where the supremum is taken over a suitable class of functions \(f: \mathbb{R}^{d_y} \to \mathbb{R}, g: \mathbb{R}^{d_y} \to \mathbb{R}^{d_x}\), and \(h: \mathbb{R}^{d_x+d_y} \to \mathbb{R}\). As the first main result, we establish strong duality and dual attainment for entropic VQR when the supremum above is taken over \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\). Importantly, our result allows for both \(\mu\) and \(\nu\) to have unbounded supports. The proof shows that the optimal coupling admits a density of the form \[\frac{d\pi}{d(\mu \otimes \nu)}(u,x,y) = e^{\frac{1}{\varepsilon}\big(f(u)+\langle g(u),x\rangle +h(x,y)-\|u-y\|^2/2\big)},\] for suitable triplet of functions \((f,g,h)\), which yields strong duality and dual attainment. Our proof adapts various techniques from duality theory for entropic OT that originate from [30]–[32] (see also [12]). One obstacle in our proof is that (unconditional) moment constraints approximating the mean-independence constraint need not be continuous in total variation unless \(X\) is compactly supported. We bypass this difficulty by imposing coercivity of the KL-divergence in the \(1\)-Wasserstein topology with respect to a modified Euclidean metric, which appears to be new and can be adapted to handle different conditional moment constraints. In addition, we show that the preceding coercivity assumption holds under a reasonably mild moment condition on \(X\). As another difficulty compared with standard entropic OT, one function \(g\) appears in the dual problem through the interaction term, \(\langle g(u),x \rangle\), between \(u\) (the “input" variable) and \(x\) (part of the”output” variables). As such, a more careful measure-theoretic argument is needed to separately construct dual potentials in a measurable way; see the discussion above Lemma 5.

Similar to entropic OT, dual potentials for entropic VQR (i.e., optimal solutions to the dual problem) are characterized by the system of functional equations that are akin to the Schrödinger system. Our next goal is to study regularity of dual potentials via the said system of functional equations. In contrast to standard entropic OT, one dual potential \(g\) is characterized only through an implicit fixed point equation involving other potentials, which poses a significant challenge to study regularity of dual potentials. To overcome the said obstacle, we employ theory of exponential families [33], [34] to establish real analyticity of dual potentials when \(\mu,\nu\) are compactly supported. The argument herein is new in the OT literature and may be of independent interest.

Finally, as an important test case, we consider the setting where \(\mu\) and \(\nu\) are Gaussian. Building on our duality theory, we find a closed-form expression for the optimal coupling \(\pi^\varepsilon\) for entropic VQR, which is again Gaussian. The weak limit \(\pi^o\) of the optimal coupling \(\pi^\varepsilon\) when \(\varepsilon \to 0+\) solves the unregularized VQR problem (3 ) (and indeed (1 ) because of Gaussianity), and we establish the precise approximation rate of \(\pi^\varepsilon\) toward \(\pi^o\) in the 2-Wasserstein distance.

1.2 Related literature↩︎

The literature related to this paper is broad, so we confine ourselves to references directly related to our work, other than those already discussed. We refer the reader to [35], [36] as excellent reviews on statistical OT, which has seen extensive research activities in recent years.

VQR is a special case of weak OT with moment constraints considered in the recent preprint [37], which establishes strong duality and dual attainment in their general framework, including entropic variants. Importantly, however, [37] require marginals to be compactly supported, and their proofs do not seem to carry over to the unbounded setting, at least directly. Their proof of strong duality rests on a Fenchel-Rockafellar argument, and their proof of dual attainment rests on taking a maximizing sequence for the dual objective and establishing a subsequential limit. These arguments largely differ from our proof. In particular, allowing for unbounded supports brings a major obstacle in our proof of strong duality and dual attainment and is needed to cover Gaussian marginals studied in the later section. Finally, regularity of dual potentials is not studied in [37]. As such, we view our work and [37] as complementary.

Another related work is [38], where the authors studied duality theory for weak OT under fairly general settings. Their results can be used to establish primal attainment for entropic VQR (see the proof of Proposition 1), but our intended duality results do not seem to follow from their general results, at least directly, because it seems nontrivial to verify their Condition (B) in our context.

The structure of the mean-independence constraint in VQR is reminiscent of martingale OT considered in the mathematical finance literature (see, e.g., [39], [40]), at least formally. For an entropic variant of martingale OT, [41] studied its duality theory in detail for two marginals defined on the real line. However, the setting and analysis of VQR differ substantially from martingale OT. In particular, in VQR, the mean-independence constraint is imposed on the auxiliary variable \(\tilde{X}\) that does not enter the ground cost \(\|U-\tilde{Y}\|^2\), and two marginals \(\mu,\nu\) are defined on spaces with different dimensions.

Gaussian distributions serve as an important test case for OT, as they often allow for closed-form expressions for OT costs and couplings; see [42], [43] for the case of standard entropic OT with quadratic cost.

1.3 Organization↩︎

The rest of the paper is organized as follows. Section 2 formally sets up VQR and entropic VQR, establishes primal attainment, and presents the results for duality theory. Section 3 presents the results under Gaussian marginals. Sections 4 and 5 collect proofs for Sections 2 and 3, respectively.

1.4 Notation↩︎

On a Euclidean space, let \(\| \cdot \|\) and \(\langle \cdot, \cdot \rangle\) denote the standard Euclidean norm and inner product, respectively. For any Polish metric space \((M,d)\), we use \(\mathcal{B}(M)\) to denote its Borel \(\sigma\)-field. Let \(\mathcal{P}(M)\) denote the collection of all Borel probability measures on \(M\). When \(M\) is a finite-dimensional Euclidean space, we consider the standard Euclidean metric, unless otherwise stated. For any \(p \in [1,\infty)\) and any fixed \(x_0 \in M\), let \(\mathcal{P}_p(M) := \{ \mu \in \mathcal{P}(M): \int d^p(x,x_0) \, d\mu(x) < \infty \}\). For \(\rho_0,\rho_1 \in \mathcal{P}_p(M)\), the \(p\)-Wasserstein distance is \[\mathsf{W}_p(\rho_0,\rho_1) := \left ( \inf_{\pi \in \Pi(\rho_0,\rho_1)} \int d^p(x,y) \, d\pi(x,y) \right )^{1/p},\] which defines a metric on \(\mathcal{P}_p(M)\). For any \(\mu \in \mathcal{P}(M), p \in [1,\infty]\), and \(d \in \mathbb{N}\), let \(L^p(\mu; \mathbb{R}^d)\) denote the \(L^p(\mu)\)-space of Borel measurable mappings \(M \to \mathbb{R}^{d}\). We write \(L^p(\mu) = L^p(\mu;\mathbb{R})\). Finally, for \(a \in \mathbb{R}\), let \(a^+=\max \{ a,0 \}\) and \(a^- = \max \{ -a, 0 \}\); for \(a,b \in \mathbb{R}\), we use the notation \(a \wedge b = \min \{ a,b \}\).

2 Duality theory for entropic VQR↩︎

2.1 VQR and entropic VQR↩︎

We first fix notation. Let \((X,Y) \in \mathbb{R}^{d_x+d_y}\) be a pair of vectors of covariates \(X \in \mathbb{R}^{d_x}\) and response variables \(Y \in \mathbb{R}^{d_y}\). Let \(\nu \in \mathcal{P}(\mathbb{R}^{d_x+d_y})\) denote the joint distribution of \((X,Y)\), and let \(\nu_x\) denote the conditional distribution of \(Y\) given \(X\). We assume throughout the paper that \[\mathbb{E}\big[\|X\|^2 + \|Y\|^2\big] < \infty \quad \text{and} \quad \mathbb{E}[X]=0.\] The latter condition, \(\mathbb{E}[X] = 0\), is for normalization and does not lose generality. Let \(\mu \in \mathcal{P}(\mathbb{R}^{d_y})\) be a reference measure with a finite second moment. We do not assume that \(\mu\) is absolutely continuous.

For notational convenience, in what follows, we set \[c(u,y) := \frac{1}{2}\|u-y\|^2\] and write \(\int c \, d\pi = \int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} c(u,y) \, d\pi(u,x,y)\). For any coupling \(\pi \in \Pi(\mu,\nu)\), let \(\pi_u\) denote the conditional distribution of \((\tilde{X},\tilde{Y})\) given \(U\) when \((U,\tilde{X},\tilde{Y}) \sim \pi\). That is, for any nonnegative measurable function \(\varphi: \mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y} \to [0,\infty]\), \[\int \varphi \, d\pi = \int_{\mathbb{R}^{d_y}}\left ( \int_{\mathbb{R}^{d_x+d_y}} \varphi (u,x,y) \, d\pi_u(x,y)\right ) \, d\mu(u). \label{eq:32rcd}\tag{5}\] See Chapter 10.2 in [44] for (regular) conditional distributions. With this notation, the VQR problem (3 ) reads as \[\inf_{\pi \in \Pi(\mu, \nu)} \int c \, d\pi\;\; \text{subject to} \;\int_{\mathbb{R}^{d_x+d_y}} x \, d\pi_u(x,y) = 0 \;\text{\mu-a.e. u}, \label{eq:32vqr}\tag{6}\] and the entropic VQR problem (4 ) reads as \[\label{eq:32entropicPrimal} \begin{align} \mathsf{T}^\varepsilon(\mu,\nu)&:= \inf_{\pi \in \Pi(\mu, \nu)} \left( \int c \, d\pi + \varepsilon \mathsf{KL}(\pi \,\|\, \mu\otimes\nu ) \right) \\ &\qquad \text{subject to} \;\int_{\mathbb{R}^{d_x+d_y}} x \, d\pi_u(x,y) = 0 \;\text{\mu-a.e. u}. \end{align}\tag{7}\]

Entropic VQR can be seen as the entropic projection of a modified reference measure onto the feasible set \[\mathcal{Q}:= \left \{ \pi \in \Pi(\mu,\nu) : \int_{\mathbb{R}^{d_x + d_y}} x \, d\pi_u(x,y) =0 \;\text{\mu-a.e. u} \right \}. \label{eq:32feasible32set}\tag{8}\] Indeed, defining \[d\tilde{R} := \alpha^{-1}e^{-c/\varepsilon} dR \quad \text{with} \quad \alpha := \int e^{-c/\varepsilon} \,dR \in (0,\infty), \label{eq:32reference}\tag{9}\] we see that \[\int c \, d\pi + \varepsilon \mathsf{KL}(\pi \,\|\, R ) = \varepsilon \mathsf{KL}(\pi \,\|\, \tilde{R} ) - \varepsilon \log \alpha.\] Hence, the primal problem (7 ) is equivalent to minimizing \(\mathsf{KL}(\pi \,\|\, \tilde{R} )\) over \(\mathcal{Q}\). Observe that the feasible set \(\mathcal{Q}\) is nonempty (as \(\mu \otimes \nu \in \mathcal{Q}\)) and convex.

We first verify below that the entropic VQR problem (7 ) admits a unique optimal solution \(\pi^\varepsilon\).

In the rest of this section, we fix \(\varepsilon > 0\) and study duality theory for entropic VQR.

2.2 Duality↩︎

Let \(R:=\mu \otimes \nu\). Define the dual objective as \[D^\varepsilon(f,g,h) := \int f \,d\mu + \int h \, d\nu - \iota^\varepsilon (f,g,h)\] with \[\iota^\varepsilon (f,g,h) := \varepsilon \int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} e^{ \frac{1}{\varepsilon} \left(f(u) + \langle g(u), x\rangle + h(x,y) - c(u,y) \right) } \,dR(u,x,y) - \varepsilon.\] The dual problem reads as \[\mathsf{D}^\varepsilon (\mu,\nu) := \sup_{(f,g,h)} D^\varepsilon(f,g,h), \label{eq:32dual}\tag{10}\] where the supremum is taken over \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\).

We first establish weak duality. The proof is standard except for one place where we verify that \(\langle g(u),x \rangle \in L^1(\pi)\) for any \(\pi \in \mathcal{Q}\) with \(\mathsf{KL}(\pi \,\|\, R ) < \infty\), provided that \(\iota^\varepsilon (f,g,h) < \infty\).

For strong duality and dual attainment, we will make the following additional assumption. Recall that \(\tilde{R}\) is a probability measure on \(\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\) defined in (9 ). We consider the \(1\)-Wasserstein distance for a modified metric on \(\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\) different from the Euclidean one. Define \[\tilde{\mathsf{d}}\big((u,x,y),(u',x',y')\big) := \|u-u'\| \wedge 1 + \|x-x'\| + \| y-y' \| \wedge 1\] for \((u,x,y), (u',x',y') \in \mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\). The metric \(\tilde{\mathsf{d}}\) induces the same topology as the Euclidean one. We denote by \(\tilde{\mathsf{W}}_1\) the corresponding \(1\)-Wasserstein distance, \[\tilde{\mathsf{W}}_1 (P,Q) := \inf_{\gamma \in \Pi(P,Q)} \int \tilde{\mathsf{d}} \, d\gamma,\] which defines a metric on \[\tilde{\mathcal{P}}_1 := \left \{ P \in \mathcal{P}(\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}) : \int \tilde{\mathsf{d}} (\cdot,0) \, dP < \infty \right \}. \label{eq:32modified32P1}\tag{11}\] By definition, \(P \in \tilde{\mathcal{P}}_1\) if and only if \(\mathbb{E}[\|\tilde{X}\|] < \infty\) for \((U,\tilde{X},\tilde{Y}) \sim P\). Observe that \(P_n \to P\) in \(\tilde{\mathsf{W}}_1\) if and only if, for any continuous function \(\varphi: \mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y} \to \mathbb{R}\) with \(|\varphi(u,x,y)| \le K(1+\|x\|)\) for \((u,x,y) \in \mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\) for some finite constant \(K\), it holds that \(\int \varphi \, dP_n \to \int \varphi dP\). See Theorem 6.9 in [4]. In particular, the topology induced by \(\tilde{\mathsf{W}}_1\) is stronger than the weak topology.

Condition (i) guarantees that \(X\) is not concentrated on any affine hyperplane in \(\mathbb{R}^{d_x}\). The function \(g\) enters the dual objective \(D(f,g,h)\) only through \(\langle g(u),x \rangle\), and the said condition guarantees to recover \(g\) from \(\langle g(u),x \rangle\). Condition (ii) is a high-level condition that guarantees the existence of the entropic projection onto the set of probability measures satisfying a finite number of unconditional moment constraints that approximate the feasible set \(\mathcal{Q}\) in (8 ). Since the support of \(X\) may be unbounded, these unconditional moment constraints need not be continuous in total variation, but continuous in \(\tilde{\mathsf{W}}_1\); see Lemma 7 below. Coercivity of \(\mathsf{KL}(\cdot \, || \, \tilde{R})\) in \(\tilde{\mathsf{W}}_1\) ensures the existence of the said entropic projection. Condition (ii) is satisfied under a suitable moment condition on \(X\).

Now, we state the first main theorem of this section.

We shall call any triplet of functions \((f,g,h)\) attaining the supremum in the dual problem (10 ) dual potentials. With respect to the modified reference measure \(\tilde{R}\), against which we take the entropic projection, the optimal coupling \(\pi^\varepsilon\) has a density of the form \[\frac{d\pi^{\varepsilon}}{d\tilde{R}} (u,x,y) = e^{\frac{1}{\varepsilon}(f^{\varepsilon}(u) + \langle g^\varepsilon(u), x\rangle +h^{\varepsilon}(x,y)) + \log \alpha}.\] In contrast to standard entropic OT, there is an interaction term between \(u\) and \(x\) via \(\langle g^\varepsilon(u),x \rangle\) due to the mean-independence constraint, so the log density does not factor into a tensor sum of two separate functions of \(u\) and \((x,y)\) only.

The next proposition addresses uniqueness of dual potentials. It shows that dual potentials are unique up to an affine shift.

The following proposition is a converse to Theorem 1 (ii).

Combining Theorem 1 (ii) and the preceding proposition, we obtain the following characterization of dual potentials, which is akin to the Schrödinger system in entropic OT.

Indeed, let \(\pi\) be a Borel measure on \(\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}\) of the form (?? ). Equations (14 ) and (16 ) are equivalent to \(\pi\) being a coupling for \((\mu,\nu)\). Equation (15 ) is equivalent to \(\pi\) satisfying the mean-independence constraint in (7 ). Hence, the conclusion follows from Theorem 1 (ii) and Proposition 4.

2.3 Regularity of dual potentials↩︎

In this section, we assume the existence of dual potentials satisfying the Schrödinger-like system (14 )–(16 ) and study their regularity. Throughout this section, we choose versions of dual potentials \((f^\varepsilon,g^\varepsilon,h^\varepsilon)\) so that (14 ) and (16 ) hold for all \(u \in \mathbb{R}^{d_y}\) and \((x,y) \in \mathbb{R}^{d_x+d_y}\), respectively; see Remark 2. Furthermore, let \(\mathcal{U}\subset \mathbb{R}^{d_y}\) denote the support of \(\mu\), and let \(\mathcal{X}\subset \mathbb{R}^{d_x},\mathcal{Y}\subset \mathbb{R}^{d_y}\) denote the supports of \(X,Y\), respectively. We will assume throughout this section that \(\mathcal{U}\) is compact.

The first two propositions concern regularity of \(h^\varepsilon\).

For the rest of this section, we will assume that \(\mathcal{X}\) and \(\mathcal{Y}\) are both compact, in addition to compactness of \(\mathcal{U}\). The next proposition shows that one can choose a version of \(g^\varepsilon\) so that (15 ) holds for all \(u\). The proof relies on theory of exponential families (cf. Chapter 3 of [33] and [34]).

The preceding proposition guarantees that, given \(h^\varepsilon\), one can choose a version of \(g^\varepsilon\) such that it is everywhere finite and (15 ) holds for all \(u \in \mathbb{R}^{d_y}\). It is not difficult to verify that \(f^\varepsilon\) and \(h^\varepsilon\) are everywhere finite. Furthermore, the next theorem establishes that \((f^\varepsilon,g^\varepsilon,h^\varepsilon)\) are real analytic. We refer the reader to [45] as an excellent reference on theory of real analytic functions.

Arguably, the most challenging part is the analysis of \(g^{\varepsilon}\). Assume \(\varepsilon = 1\) and drop the superscript \(\varepsilon\) for simplicity. In view of Proposition 7, \(g(u)\) is the unique vector in \(\mathbb{R}^{d_x}\) such that \(\nabla_\theta A(g(u),u) = 0\), where \[A(\theta,u) = \int e^{\langle \theta,x\rangle + \langle u,y \rangle } \, d\tilde{\nu}(x,y) \quad \text{with} \quad d\tilde{\nu}(x,y) = e^{h(x,y)-\|y\|^2/2} \, d\nu(x,y).\] Since \(A(\theta,u)\) is the Laplace transform of the base measure \(\tilde{\nu}\), it is analytic by [34]. Real analyticity of \(g\) follows from applying the real analytic version of the implicit function theorem [45].

3 Gaussian case↩︎

3.1 Closed-form solution↩︎

In this section, we derive closed-form solutions for the entropic VQR problem under Gaussian marginals.

In the absence of covariates \(X\) (in which case entropic VQR reduces to standard entropic OT with quadratic cost), our results are consistent with those of [42]. The proof uses a key observation from our duality theory (note that Gaussian distributions satisfy Assumption 1 (ii) by Lemma 1) that, the cross term between \(u\) and \(y\) in the log (Lebesgue) density of \(\pi^\varepsilon\) is \(u^\top y/\varepsilon\), which implies that the corresponding block in the precision matrix \(\Gamma_\varepsilon^{-1}\) is proportional to \(I_{d_y}\). Using this, we derive a matrix Riccati-type equation for \(\Lambda_\varepsilon\) (see (33 ) below), solving which yields (i). Part (ii) follows by comparing two density expressions.

3.2 Convergence of entropic VQR↩︎

In this section, using the closed-form expression of the optimal coupling \(\pi^{\varepsilon}\), we study convergence of \(\pi^\varepsilon\) when \(\varepsilon \to 0+\). The next result is straightforward.

Furthermore, we obtain the asymptotic expansion of the \(2\)-Wasserstein distance between \(\pi^\varepsilon\) and \(\pi^o\).

The proposition implies that the precise order of \(\mathsf{W}_2^2(\pi^\varepsilon,\pi^o)\) is \(\varepsilon\) under Gaussian marginals. It is worth pointing out that, for standard entropic OT, the recent paper by [28] establishes that the approximation rate in \(\mathsf{W}_2^2\) of the optimal entropic coupling under quadratic cost is precisely \(\varepsilon\), for absolutely continuous marginals that admit a Lipschitz continuous Brenier map; see Theorem 3.9 in [28]. The above result is consistent with theirs.

4 Proofs for Section 2↩︎

We recall some properties of the KL divergence. Let \(M\) be a Polish metric space. For \(P,Q \in \mathcal{P}(M)\), \(\mathsf{KL}(P \,\|\, Q ) \in [0,\infty]\) and \(\mathsf{KL}(P \,\|\, Q )=0\) if and only if \(P=Q\). In addition, the mapping \(P \mapsto \mathsf{KL}(P \,\|\, Q )\) is convex, lower semicontinuous for the weak topology (and any stronger topology), and strictly convex on its domain. The lower semicontinuity follows by the Donsker-Varadhan variational representation (cf. Theorem 4.6 in [46]).

4.1 Proof of Proposition 1↩︎

Uniqueness follows from strict convexity of \(\mathsf{KL}(\cdot \,\|\, \mu\otimes \nu )\) on its domain, so we prove the existence. In fact, it is not very hard to verify that \(\mathcal{Q}\) is weakly compact (as each \(\pi \in \mathcal{Q}\) has fixed marginals and \(X\) has finite expectation), which yields the existence.

We provide an alternative argument by rewriting the primal problem (7 ) as a weak OT problem [38]. Since \(\mathsf{KL}(\pi \,\|\, \mu \otimes \nu ) = \int_{\mathbb{R}^{d_y}} \mathsf{KL}(\pi_u \,\|\, \nu )\, d\mu(u)\) [46], we have \[\int c \, d\pi + \varepsilon \mathsf{KL}(\pi \,\|\, \mu\otimes\nu ) = \int_{\mathbb{R}^{d_y}} \left(\int_{\mathbb{R}^{d_x+d_y}} c(u,y) \, d\pi_u(x,y) + \varepsilon \mathsf{KL}(\pi_u \,\|\, \nu ) \right)\, d\mu(u).\] Let \[\mathcal{C}:= \left \{\rho \in \mathcal{P}_2(\mathbb{R}^{d_x+d_y}): \int_{\mathbb{R}^{d_x+d_y}} x\, d\rho(x,y) = 0 \right\},\] and define the convex indicator \[\chi_{\mathcal{C}}(\rho) := \begin{cases}0, & \text{if \rho\in \mathcal{C},} \\ \infty, & \text{otherwise.} \end{cases}\] The mean-independence constraint in (7 ) is equivalent to \(\int \chi_{\mathcal{C}}(\pi_u) \, d\mu(u) = 0.\) Hence, the primal problem (7 ) is equivalent to the following weak OT problem: \[\label{weakOTprimal} \inf_{\pi \in \Pi(\mu, \nu)} \int_{\mathbb{R}^{d_y}} C(u, \pi_u) \, d\mu(u),\tag{18}\] where the cost \(C: \mathbb{R}^{d_y} \times \mathcal{P}_2(\mathbb{R}^{d_x+d_y}) \to [0,\infty]\) is given by \[C(u, \rho) := \int_{\mathbb{R}^{d_x+d_y}} c(u,y) \, d\rho(x,y) + \varepsilon \mathsf{KL}(\rho \,\|\, \nu ) + \chi_\mathcal{C}(\rho). \label{eq:32weak32cost}\tag{19}\] We will apply Theorem 2.2 (i) in [38] to establish the desired claim. Endow \(\mathcal{P}_2(\mathbb{R}^{d_x+d_y})\) with the \(2\)-Wasserstein distance. We need to verify that the cost \(C\) is Borel, and that \(\rho \mapsto C(u,\rho)\) is convex and lower semicontinuous. The mapping \((u,\rho) \mapsto \int_{\mathbb{R}^{d_x+d_y}} c(u,y) \, d\rho(x,y)\) is jointly continuous and linear in \(\rho\), and \(\rho \mapsto \mathsf{KL}(\rho \,\|\, \nu )\) is convex and lower semicontinuous. The set \(\mathcal{C}\) is convex and closed in \(\mathcal{P}_2(\mathbb{R}^{d_x+d_y})\), so \(\chi_\mathcal{C}\) is convex and lower semicontinuous. As such, applying Theorem 2.2 (i) in [38], we obtain the desired claim. 0◻

4.2 Proof of Proposition 2↩︎

It suffices to show that \(\mathsf{T}^\varepsilon(\mu,\nu) \ge D^\varepsilon(f,g,h)\) for all \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\). We may assume that \(\iota^\varepsilon(f, g, h) < \infty\); otherwise, \(D(f, g, h) = -\infty\) and the conclusion trivially holds.

Pick any feasible coupling \(\pi \in \mathcal{Q}\) with \(\mathsf{KL}(\pi \,\|\, R ) < \infty\), and let \(p_\pi:= \frac{d\pi}{dR}\). Then \(\int p_\pi \, dR = \int d\pi = 1\). Let \(F(u,x,y):= f(u) + \langle g(u), x\rangle + h(x,y)\). Observe that \[\int e^{\frac{1}{\varepsilon}(F - c)}\,dR = \frac{1}{\varepsilon} \iota^\varepsilon(f, g, h) + 1 < \infty.\] Apply the inequality \[\alpha\log\alpha - \alpha \ge \beta \alpha -e^{\beta}, \;\alpha \ge 0, \beta \in \mathbb{R},\] which follows from Fenchel’s inequality, with \(\alpha = p_\pi\) and \(\beta = (F-c)/\varepsilon\), to obtain \[\varepsilon (p_\pi \log p_\pi - p_\pi) \ge (F-c)p_\pi - \varepsilon e^{\frac{1}{\varepsilon}(F-c)}. \label{eq:32fenchel}\tag{20}\]

We shall verify that \(\langle g(u),x \rangle \in L^1(\pi)\). Since the left-hand side and the second term on the right-hand side of (20 ) are integrable under \(R\), we have \((F-c)^+p_\pi \in L^1(R)\), i.e., \((F-c)^+ \in L^1(\pi)\). Observe that \[(\langle g(u),x \rangle)^+ \le (F(u,x,y)-c(u,y))^{+} + (f(u)+h(x,y)-c(u,y))^-.\] The right-hand side is integrable under \(\pi\), so that \((\langle g(u),x \rangle)^+ \in L^1(\pi)\). Now, since \(\pi\) is a coupling for \((\mu,\nu)\), \(\| x \|\) is integrable under \(\pi\), so that (cf. (5 )) \[\int_{\mathbb{R}^{d_x+d_y}} \|x\| \, d\pi_u(x,y) < \infty \;\text{\mu-a.e. u.}\] Pick any \(u\) for which the integral on the left-hand side is finite. Then \[\int_{\mathbb{R}^{d_x+d_y}} \langle g(u),x\rangle \, d\pi_u(x,y) = g(u)^\top \int_{\mathbb{R}^{d_x+d_y}} x \, d\pi_u(x,y) = 0.\] This implies that \[\int_{\mathbb{R}^{d_x+d_y}} (\langle g(u),x\rangle)^- \, d\pi_u(x,y) = \int_{\mathbb{R}^{d_x+d_y}} (\langle g(u),x\rangle)^+ \, d\pi_u(x,y).\] Hence, \[\begin{align} \int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} (\langle g(u),x\rangle)^{-} \, d\pi(u,x,y) &= \int_{\mathbb{R}^{d_y}} \int_{\mathbb{R}^{d_x+d_y}} (\langle g(u),x\rangle)^- \, d\pi_u(x,y) \, d\mu(u) \\ &= \int_{\mathbb{R}^{d_y}} \int_{\mathbb{R}^{d_x+d_y}} (\langle g(u),x\rangle)^+ \, d\pi_u(x,y) \, d\mu(u) \\ &< \infty. \end{align}\] Conclude that \(\langle g(u),x\rangle \in L^1(\pi)\), which ensures that, for \((U,\tilde{X},\tilde{Y}) \sim \pi\), \[\int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} \langle g(u),x \rangle \, d\pi(u,x,y) = \mathbb{E}\big[ \langle g(U),\tilde{X} \rangle \big] = \mathbb{E}\big[\langle g(U),\mathbb{E}[\tilde{X} \mid U] \rangle \big] = 0.\]

Finally, using inequality (20 ) again, we obtain \[\varepsilon\int (p_\pi\log p_\pi - p_\pi)\,dR \ge \int (F - c)p_\pi\, dR - \varepsilon \int e^{\frac{1}{\varepsilon}(F - c)}\,dR,\] that is, \[\varepsilon \mathsf{KL}(\pi \,\|\, R ) - \varepsilon \ge \left( \int f \,d\mu + \int h \,d\nu - \int c \,d\pi \right) - \iota^\varepsilon(f, g, h) - \varepsilon.\] Rearranging terms, we obtain the desired claim. 0◻

4.3 Proof of Lemma 1↩︎

Lemma 1 directly follows from the next lemma and the fact that \(e^{-c/\varepsilon}\) is bounded.

There is a partial converse to Lemma 2; see Appendix 6.

4.4 Proof of Theorem 1↩︎

4.4.1 Preliminary lemmas↩︎

Before the proof, we shall recall the following (standard) result. We provide its proof for completeness.

We will also use the following lemma, which is a small modification to (a special case of) Theorem 3.1 in [30].

The following lemma is inspired by Lemma 2.10 in [12]. In the proof of Theorem 1 below, we will construct dual potentials from the \(R\)-a.e. limit of \(f_n(u)+\langle g_n(u),x\rangle + h_n(x,y)\) for suitable measurable functions \((f_n,g_n,h_n)\). Because of the presence of the interaction term \(\langle g_n(u),x \rangle\), special care is needed to construct dual potentials in such a way that they are separately measurable, i.e., one has to choose “canonical” points more carefully than standard entropic OT.

We are now ready to prove Theorem 1.

4.4.2 Proof of Theorem 1↩︎

We split the proof into a few steps.

Step 1. We first show that there exist measurable functions \(f: \mathbb{R}^{d_y} \to \mathbb{R}, g: \mathbb{R}^{d_y} \to \mathbb{R}^{d_x}\), and \(h: \mathbb{R}^{d_x+d_y} \to \mathbb{R}\) such that \[\log \frac{d\pi^\varepsilon}{d\tilde{R}}(u,x,y) = f(u) + \langle g(u),x \rangle + h(x,y). \label{eq:32density32R}\tag{24}\] We shall translate the feasible set into countably many unconditional moment constraints. To this end, we first verify the following lemma. Recall the definitions of \(\mathcal{Q}\) in (8 ) and \(\tilde{\mathcal{P}}_1\) in (11 ). For a metric space \(M\), let \(BL_1(M)\) denote the set of all \(1\)-Lipschitz functions \(M \to [-1,1]\). Observe that, when \(M\) is \(\sigma\)-compact, \(BL_1(M)\) is separable for the topology of uniform convergence on compacta.

For each \(n\), define the set \[\mathcal{Q}_n := \left\{\pi \in \tilde{\mathcal{P}}_1 : \text{(\ref{eq:32moment32condition}) holds all i \le n} \right\}.\] Each \(\mathcal{Q}_n\) is nonempty, convex, closed in \(\tilde{\mathsf{W}}_1\) (cf. Theorem 6.9 in [4]), and nonincreasing in \(n\) with \(\bigcap_{n=1}^\infty \mathcal{Q}_n = \mathcal{Q}\). Now, since \(Q \mapsto \mathsf{KL}(Q \,\|\, \tilde{R} )\) is strictly convex (on its domain) and coercive in \(\tilde{\mathsf{W}}_1\) (the coercivity is guaranteed by Assumption 1 (ii)), by Lemma 3, there exists a (unique) \(\pi_n \in \mathcal{Q}_n\) such that \[\mathsf{KL}(\pi_n \,\|\, \tilde{R} ) = \inf_{\pi \in \mathcal{Q}_n} \mathsf{KL}(\pi \,\|\, \tilde{R} ).\] As Condition (23 ) above holds for \(\mathcal{Q}= \mathcal{Q}_n, R=\tilde{R}\), \(Q=\pi_n\), and \(\phi_i\)’s suitably chosen, we can apply Lemma 4 to conclude that there exist \((f_n,g_n,h_n) \in L^\infty(\mu) \times L^\infty(\mu;\mathbb{R}^{d_x}) \times L^\infty(\nu)\) such that \[\log \frac{d\pi_n}{d\tilde{R}} (u,x,y) = f_n(u) + \langle g_n(u),x \rangle + h_n(x,y).\] These are constructed as follows: \(f_n\) is a linear combination of \(\{ 1,\mathfrak{f}_1,\dots,\mathfrak{f}_n \}\), and \(g_n,h_n\) are linear combinations of \(\{ \mathfrak{g}_i : i \le n\}\) and \(\{ \mathfrak{h}_i : i \le n \}\), respectively.

Arguing exactly as in the proof of [12], we have \[\begin{align} &\pi_n \to \pi^\varepsilon \;\text{in total variation}, \;\text{i.e.}, \;\frac{d\pi_n}{d\tilde{R}} \to \frac{d\pi^\varepsilon}{d\tilde{R}} \;\text{in L^1(\tilde{R})}, \tag{26} \\ &\log \frac{d\pi_n}{d\tilde{R}} \to \log \frac{d\pi^\varepsilon}{d\tilde{R}} \;\text{in L^1(\pi^\varepsilon)}. \tag{27} \end{align}\] In [12], \(\mathcal{Q}_n\) is assumed to be closed in total variation, which does not hold unless \(X\) is bounded in our case, but the said assumption is used to guarantee the existence of entropic projection \(\pi_n\) of \(\tilde{R}\) onto \(\mathcal{Q}_n\), which was established via a different route in our case. Given the existence of \(\pi_n\), the argument in the proof of [12] goes through verbatim.

Now, by (26 ), after passing to a subsequence if necessary and using \(\tilde{R} \sim R\), we conclude \[F(u,x,y):=\log \frac{d\pi^\varepsilon}{d\tilde{R}}(u,x,y) = \lim_{n \to \infty} \big( f_n(u) + \langle g_n(u),x \rangle + h_n(x,y) \big) \in [-\infty,\infty) \label{eq:32limit}\tag{28}\] for \(R\)-a.e. \((u,x,y)\). Recall that the feasible set \(\mathcal{Q}\) is convex and that \(\pi^\varepsilon\) minimizes \(\mathsf{KL}(\pi \,\|\, \tilde{R} )\) over all \(\pi \in \mathcal{Q}\). Since \(\mathsf{KL}(R \,\|\, \tilde{R} ) < \infty\), Corollary 1.13 of [12] yields that \(F \in L^1(R)\), which implies that \(F\) is finite \(R\)-a.e. Let \(S \subset \mathbb{R}^{d_x+d_y}\) be the set of full \(R\)-measure on which the limit (in \(\mathbb{R}\)) on the right-hand side of (28 ) exists.

We invoke Lemma 5 above to find sets \(\mathcal{U}_0 \subset \mathbb{R}^{d_y}, \mathcal{Z}_0 \subset \mathbb{R}^{d_x+d_y}\) with \(\mu(\mathcal{U}_0)=\nu(\mathcal{Z}_0)=1\) and points \(u^* \in \mathbb{R}^{d_y}, (x_i^*,y_i^*) \in \mathbb{R}^{d_x+d_y} \;(i \in \{ 0,1,\dots,d_x \})\) for which (i) and (ii) in Lemma 5 hold. Recall \(S_0 = (\mathcal{U}_0 \times \mathcal{Z}_0) \cap S\). If we define \[\begin{align} &\tilde{f}_n(u) = f_n(u) - f_n(u^*), \\ &\tilde{g}_n(u) = g_n(u) - g_n(u^*), \\ &\tilde{h}_n(x,y) = h_n(x,y) + f_n(u^*) + \langle g_n(u^*),x \rangle, \end{align}\] then \(\tilde{f}_n(u^*)=0, \tilde{g}_n(u^*)=0\), and \[F_n(u,x,y):=f_n(u)+\langle g_n(u),x \rangle + h_n(x,y) \\ =\tilde{f}_n(u)+\langle \tilde{g}_n(u),x \rangle + \tilde{h}_n(x,y).\] For notational convenience, relabel \((\tilde{f}_n,\tilde{g}_n,\tilde{h}_n)\) as \((f_n,g_n,h_n)\). Set \(\mathcal{U}_{00} := \{ u \in \mathcal{U}_0 : (u,x,y) \in S_0 \;\text{for some} \;(x,y) \in \mathcal{Z}_0 \}\) and \(\mathcal{Z}_{00} := \{ (x,y) \in \mathcal{Z}_0 : (u,x,y) \in S_0 \;\text{for some} \;u \in \mathcal{U}_0 \}\). These sets need not be Borel measurable, but are analytic and hence \(\mu\)- and \(\nu\)-completion measurable, respectively (cf. Chapter 13 in [44]). Observe that \(\bar{\mu}(\mathcal{U}_{00}) = \bar{\nu}(\mathcal{Z}_{00})=1\), with \(\bar{\mu}, \bar{\nu}\) denoting the completions of \(\mu, \nu\), respectively.

Pick any \((x,y) \in \mathcal{Z}_{00}\). Since \((u^*, x,y) \in S_0\), we have \[F(u^*, x,y) = \lim_{n\to\infty} F_n(u^*, x,y) = \lim_{n \to \infty} h_n(x,y).\] So let us define \[h(x,y) := \begin{cases} F(u^*, x,y) & \text{if (x,y) \in \mathcal{Z}_{00},} \\ 0 & \text{otherwise}, \end{cases}\] and \(G(u,x,y) := F(u,x,y) - h(x,y)\). On \(S_0\), one has \[G(u, x,y) = \lim_{n\to\infty} (F_n(u, x,y) - h_n(x,y)) = \lim_{n\to\infty} \big ( f_n(u) + \langle g_n(u),x \rangle \big ).\] Let \(H_n(u) := (f_n(u), g_n(u)^{\top})^{\top}\) be a vector-valued mapping \(H_n: \mathbb{R}^{d_y} \to \mathbb{R}^{d_x+1}\), and let \(v(x,y) := (1,x^\top)^\top\) be another vector-valued mapping \(v: \mathbb{R}^{d_x+d_y} \to \mathbb{R}^{d_x+1}\). We have \[G(u,x,y) = \lim_{n \to \infty}\langle H_n(u),v(x,y) \rangle.\]

Pick any \(u \in \mathcal{U}_{00}\). Observe that \((u,x_i^*,y_i^*) \in S_0\) for all \(i \in \{0,1,\dots,d_x \}\). Let \(V\) be the invertible \((d_x+1) \times (d_x+1)\) matrix whose rows are \(v(x_i^*,y_i^*)^\top\), and define \[\varphi_n(u) := \begin{pmatrix} \langle H_n(u), v(x_0^*,y_0^*) \rangle \\ \vdots \\ \langle H_n(u), v(x_{d_x}^*,y_{d_x}^*) \rangle \end{pmatrix} = V H_n(u).\] By construction, the limit of \(\varphi_n(u)\) exists for \(u \in \mathcal{U}_{00}\), so define \[\varphi(u) := \lim_{n\to\infty} \varphi_n(u).\] Finally, define \[\big(f(u), g(u)^\top\big)^\top:= \lim_{n\to\infty} H_n(u) = \lim_{n\to\infty} V^{-1} \varphi_n(u) = V^{-1} \varphi(u), \;u \in \mathcal{U}_{00},\] and set \((f(u), g(u)^\top)^\top = 0\) outside \(\mathcal{U}_{00}\). The functions \(f\) and \(g\), as constructed, are \(\mu\)-completion measurable. They need not be Borel measurable, but one can find Borel measurable \(\tilde{f}: \mathbb{R}^{d_y} \to \mathbb{R}\) and \(\tilde{g}: \mathbb{R}^{d_y} \to \mathbb{R}^{d_x}\) such that \(f\) and \(g\) differ from \(\tilde{f}\) and \(\tilde{g}\), respectively, only on \(\mu\)-null sets. Likewise, one can find Borel measurable \(\tilde{h}: \mathbb{R}^{d_x+d_y} \to \mathbb{R}\) that differs from \(h\) only on a \(\nu\)-null set. For notational convenience, relabel \((\tilde{f},\tilde{g},\tilde{h})\) as \((f,g,h)\). We have shown that \[\log \frac{d\pi^\varepsilon}{d\tilde{R}}(u,x,y) = F(u,x,y) = f(u) + \langle g(u),x \rangle + h(x,y)\] for \(R\)-a.e. \((u,x,y)\).

Step 2. Next, we shall show that \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\). Recall that \(F \in L^1(R)\). By Fubini’s theorem, the function \(\tilde{F}(x, y) := \int_{\mathbb{R}^{d_y}} F(u, x, y)\, d\mu(u)\) is in \(L^1(\nu)\). Since \[\begin{align} \tilde{F}(x, y) &= \int_{\mathbb{R}^{d_y}} \big(f(u) + \langle g(u),x \rangle \big) \,d\mu(u) + h(x, y) \end{align}\] we see that \(K(x):=\int_{\mathbb{R}^{d_y}} \big(f(u) + \langle g(u),x \rangle \big) \,d\mu(u)\) is finite for all \((x,y)\) on a set of full \(\nu\)-measure.

Now, by Lemma 6, one can find \(d_x+1\) points \(x_0, x_1, \dots, x_{d_x}\) such that \(K(x_i)\) are finite for all \(i \in \{0,\dots,d_x\}\), and that \(v_i := x_i - x_0\) (\(i \in \{ 1,\dots,d_x\}\)) form a basis of \(\mathbb{R}^{d_x}\). Since \(K(x_i) - K(x_0) = \int_{\mathbb{R}^{d_y}} \langle g(u),v_i \rangle \, d\mu(u)\), we see that \(\langle g(\cdot),v_i \rangle \in L^1(\mu)\) for all \(i \in \{1,\dots,d_x\}\). This implies that \(g \in L^1(\mu;\mathbb{R}^{d_x})\) and \(f \in L^1(\mu)\). Finally, since \(|h(x,y)| \le |\tilde{F}(x,y)| + \| f \|_{L^1(\mu)} + \| g \|_{L^1(\mu)} \| x \|\), \(\tilde{F} \in L^1(\nu)\), and \(\mathbb{E}[\|X\|] < \infty\), we conclude that \(h \in L^1(\nu)\). In addition, since \(|\langle g(u),x\rangle| \le |F(u,x,y)| + |f(u)| + |h(x,y)|\) and \(F \in L^1(\pi^\varepsilon)\) by (27 ), we have that \(\langle g(u),x\rangle\) is integrable under \(\pi^\varepsilon\).

We have shown that there exist \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\) such that (24 ) holds. Since \[\begin{align} \log \frac{d\pi^\varepsilon}{dR}(u,x,y) &= \log \frac{d\pi^\varepsilon}{d\tilde{R}}(u,x,y) + \log \frac{d\tilde{R}}{dR}(u,x,y) \\ &= f(u) + \langle g(u),x \rangle + h(x,y) - \log \alpha - \frac{1}{\varepsilon}c(u,y), \end{align}\] the expression (13 ) holds with \(f^\varepsilon = \varepsilon(f-\log \alpha), g^\varepsilon = \varepsilon g\), and \(h^\varepsilon = \varepsilon h\).

Step 3. Finally, we establish the conclusion of the theorem. Observe that \[\begin{align} \varepsilon \mathsf{KL}(\pi^{\varepsilon} \,\|\, R ) &= \varepsilon \int \log\left(\frac{d\pi^{\varepsilon}}{dR}\right) \,d\pi^{\varepsilon} \\ &= \int \big ( f^\varepsilon (u) + \langle g^\varepsilon (u),x \rangle + h^\varepsilon(x,y) -c(u,y) \big)\, d\pi^{\varepsilon}(u,x,y) \\ &=\int f^\varepsilon \, d\mu + \int h^\varepsilon \, d\nu - \int c \, d\pi^\varepsilon. \end{align}\] For the third equality, we used the fact that, for \((U,\tilde{X},\tilde{Y}) \sim \pi^\varepsilon\), we have \(\mathbb{E}[|\langle g^\varepsilon (U),\tilde{X}\rangle|] < \infty\) as we have verified before and \[\int \langle g^\varepsilon (u),x \rangle \, d\pi^\varepsilon(u,x,y)=\mathbb{E}\big[\langle g^\varepsilon (U),\tilde{X}\rangle\big] =\mathbb{E}\big[\langle g^\varepsilon (U), \mathbb{E}[\tilde{X} \mid U] \rangle\big] = 0.\] Observe that \(\iota^\varepsilon(f^\varepsilon,g^\varepsilon,h^\varepsilon)=0\) as \(\pi^\varepsilon\) is a probability measure. Conclude that \[\begin{align} \mathsf{T}^\varepsilon (\mu,\nu) &= \int c \,d\pi^\varepsilon + \varepsilon\mathsf{KL}(\pi^\varepsilon \,\|\, R ) \\ &= \int f^\varepsilon \, d\mu + \int h^\varepsilon \, d\nu \\ &= D^\varepsilon (f^\varepsilon,g^\varepsilon,h^\varepsilon) \le \mathsf{D}^\varepsilon(\mu,\nu). \end{align}\] Combining the weak duality from Proposition 2, the strong duality holds, and \((f^\varepsilon,g^\varepsilon,h^\varepsilon)\) solve the dual problem. This completes the proof. 0◻

4.5 Proof of Proposition 3↩︎

Observe that the dual objective can be written as \[\begin{align} D^\varepsilon (f,g,h) &= \int \big(f(u)+\langle g(u),x\rangle + h(x,y) \big) \, dR(u,x,y) \\ &\quad - \varepsilon \int e^{\frac{1}{\varepsilon} (f(u)+\langle g(u),x\rangle + h(x,y) -c(u,y))} \, dR(u,x,y) + \varepsilon \end{align}\] for \((f,g,h) \in L^1(\mu) \times L^1(\mu;\mathbb{R}^{d_x}) \times L^1(\nu)\). If \((f,g,h), (\tilde{f},\tilde{g},\tilde{h})\) both maximize the dual objective, then by strict convexity of the exponential function, we have \[f(u) + \langle g(u), x\rangle + h(x,y) = \tilde{f}(u) + \langle \tilde{g}(u),x\rangle + \tilde{h}(x,y) \quad \text{R-a.e. (u,x,y)}.\] Define \[\mathfrak{f}(u) = \tilde{f}(u) - f(u), \quad \mathfrak{g}(u) = \tilde{g}(u) - g(u), \quad \mathfrak{h}(x,y) = \tilde{h}(x,y) - h(x,y).\] We have \[\mathfrak{h}(x,y) = - \left( \mathfrak{f}(u) + \langle \mathfrak{g}(u), x \rangle \right) \quad \text{R-a.e. (u,x,y)}.\] Since the left-hand side does not depend on \(u\), there exists \(u_0 \in \mathbb{R}^{d_y}\) such that \[\mathfrak{h}(x,y)=-a-\langle v,x \rangle \quad \text{\nu-a.e. (x,y),}\] where \(a=\mathfrak{f}(u_0)\) and \(v = \mathfrak{g}(u_0)\). This implies that \[(\mathfrak{f}(u) - a) + \langle \mathfrak{g}(u)- v,x \rangle = 0 \quad \text{R-a.e. (u,x,y)}.\] Under Assumption 1 (i) (cf. Lemma 6), one has \(\mathfrak{f}- a = 0\) and \(\mathfrak{g}- v = 0\) \(\mu\)-a.e. 0◻

4.6 Proof of Proposition 4↩︎

Observe that \[\int \left(\log \frac{d\pi}{dR}\right)^- \, d\pi = \int \left(\log \frac{d\pi}{dR}\right)^- \frac{d\pi}{dR} \, dR < \infty\] because \(a\log a \ge -e^{-1}\) for \(a \ge 0\). Since \[(\langle g(u),x\rangle)^- \le \varepsilon \left(\log \frac{d\pi}{dR}\right)^- + \left(f(u) + h(x,y) - c(u,y)\right)^+,\] it follows that \((\langle g(u),x\rangle)^- \in L^1(\pi)\). Arguing as in the proof of Proposition 2, we see that \(\langle g(u),x\rangle \in L^1(\pi)\), which ensures \(\int_{\mathbb{R}^{d_y} \times \mathbb{R}^{d_x+d_y}} \langle g(u),x\rangle d\pi(u,x,y) = 0\) as \(\pi \in \mathcal{Q}\). The rest follows from weak duality.

0◻

4.7 Proofs for Section 2.3↩︎

Let \(\kappa\) denote the marginal distribution of \(X\), so that \(d\nu = d\nu_x d\kappa\) (recall that \(\nu_x\) denotes the conditional distribution of \(Y\) given \(X\)).

It remains to prove Lemma 8 that was used in the proof of Proposition 7.

5 Proofs for Section 3↩︎

5.1 Proof of Theorem 3 (i)↩︎

We first verify that the optimal coupling must be Gaussian.

5.2 Proof of Theorem 3 (ii)↩︎

To find the dual potentials, we first need to find \(\Theta\) explicitly. We have already obtained \(\Theta_{UY} = - \frac{1}{\varepsilon}I_{d_y}\) and \(\Theta_{XU} = \frac{1}{\varepsilon} \Sigma_{XX}^{-1} \Sigma_{XY}\).

From the block matrix inversion formula applied to \(\Theta = \Gamma_\varepsilon^{-1}\), we obtain \[\begin{pmatrix} \Theta_{UX} & \Theta_{UY} \end{pmatrix} = - \Theta_{UU} \begin{pmatrix} O & \Lambda_\varepsilon \end{pmatrix} \begin{pmatrix} \Omega_{XX} & \Omega_{XY} \\ \Omega_{YX} & \Omega_{YY} \end{pmatrix}.\] Since \(\Theta_{UY} = - \frac{1}{\varepsilon}I_{d_y}\), we deduce that \(\Theta_{UU} = \frac{1}{\varepsilon} (\Lambda_{\varepsilon} \Omega_{YY})^{-1}\). But according to equation (33 ), \(\Lambda_{\varepsilon} \Omega_{YY} (\Lambda_{\varepsilon} + \varepsilon I_{d_y}) = I_{d_y}\), which implies \((\Lambda_{\varepsilon} \Omega_{YY})^{-1} = \Lambda_{\varepsilon} + \varepsilon I_{d_y}\). Conclude that \[\Theta_{UU} = \frac{1}{\varepsilon} (\Lambda_{\varepsilon} + \varepsilon I_{d_y}) = \frac{1}{\varepsilon} \Lambda_{\varepsilon} + I_{d_y}.\]

It remains to find \(\Theta_{ZZ} = (\Sigma - B^\top B)^{-1}\). Recall the partition (32 ). From the Sherman-Morrison-Woodbury formula, we have \[(\Sigma - B^\top B)^{-1} = \Sigma^{-1} + \Sigma^{-1}B^\top (I_{d_y} - B\Sigma^{-1}B^\top)^{-1}B\Sigma^{-1}\] Recalling that \(\Lambda_\varepsilon\) is symmetric, we observe that \[\Sigma^{-1}B^\top = \begin{pmatrix} \Omega_{XY} \Lambda_{\varepsilon} \\ \Omega_{YY} \Lambda_{\varepsilon} \end{pmatrix} \quad \text{and} \quad I_{d_y}- B\Sigma^{-1}B^\top = I_{d_y} - \Lambda_{\varepsilon}\Omega_{YY}\Lambda_{\varepsilon}^\top.\] Since, by equation (33 ), \(I_{d_y} - \Lambda_{\varepsilon}\Omega_{YY}\Lambda_{\varepsilon} = \varepsilon \Lambda_{\varepsilon}\Omega_{YY}\), we see that \[(I - B\Sigma^{-1}B^\top )^{-1} = \frac{1}{\varepsilon}\Omega_{YY}^{-1}\Lambda_{\varepsilon}^{-1}.\] Combining these yields \[\Theta_{ZZ} = \Omega + \frac{1}{\varepsilon} \begin{pmatrix} \Omega_{XY} \Lambda_{\varepsilon} \\ \Omega_{YY} \Lambda_{\varepsilon} \end{pmatrix} (\Omega_{YY}^{-1} \Lambda_{\varepsilon}^{-1}) \begin{pmatrix} \Lambda_{\varepsilon} \Omega_{YX} & \Lambda_{\varepsilon} \Omega_{YY} \end{pmatrix}.\] The expression can be further simplified. From \(\Sigma \Omega = I_{d_x+d_y}\) we have \(\Sigma_{XX} \Omega_{XY} + \Sigma_{XY} \Omega_{YY} = O\), or \(\Omega_{XY} = G\Omega_{YY}\) with \(G = -\Sigma_{XX}^{-1} \Sigma_{XY}\). Plugging this, we have \[\begin{align} \Theta_{ZZ} &= \Omega + \frac{1}{\varepsilon} \left( \begin{pmatrix} G \\ I_{d_y} \end{pmatrix} \Omega_{YY} \Lambda_{\varepsilon} \right) (\Omega_{YY}^{-1} \Lambda_{\varepsilon}^{-1}) \left( \Lambda_{\varepsilon} \Omega_{YY} \begin{pmatrix} G^\top & I_{d_y} \end{pmatrix} \right) \\ &= \Omega + \frac{1}{\varepsilon} \begin{pmatrix} G \\ I_{d_y} \end{pmatrix} (\Lambda_{\varepsilon} \Omega_{YY}) \begin{pmatrix} G^\top & I_{d_y} \end{pmatrix} \\ &= \Omega + \frac{1}{\varepsilon} \begin{pmatrix} G \Psi_\varepsilon G^\top & G \Psi_\varepsilon \\ \Psi_\varepsilon G^\top & \Psi_\varepsilon \end{pmatrix} \end{align}\] where \(\Psi_\varepsilon = \Lambda_{\varepsilon}\Omega_{YY}\).

In conclusion, denoting by \(\Theta_0 := \begin{pmatrix} I_{d_y} & O \\ O & \Omega \end{pmatrix}\) the precision matrix for the reference measure \(\mu \otimes \nu\), we have \[\Delta \Theta := \Theta - \Theta_0 = \frac{1}{\varepsilon} \begin{pmatrix} \Lambda_{\varepsilon} & -G^\top & -I_{d_y} \\ -G & G \Psi_\varepsilon G^\top & G \Psi_\varepsilon \\ -I_{d_y} & \Psi_\varepsilon G^\top & \Psi_\varepsilon \end{pmatrix}.\] Since \(\pi^\varepsilon\) is multivariate Gaussian, for \(w = (u^\top,x^\top,y^\top)^\top\), \[\begin{align} \log\frac{d\pi}{d(\mu\otimes \nu)}(w) &= -\frac{1}{2}(w-m)^\top \Theta (w -m) + \frac{1}{2}(w-m)^\top \Theta_0(w-m) + \frac{1}{2}\log\frac{\det (\Theta)}{\det (\Theta_0)} \\ \end{align}\]

We shall find the constant \(\frac{1}{2}\log\frac{\det (\Theta)}{\det (\Theta_0)}\). Observe that \(\det(\Theta) = \det(\Gamma_\varepsilon)^{-1}\) and \(\det(\Theta_0) = \det(\Sigma)^{-1}\). From the partition (32 ), Schur’s formula yields \[\begin{align} \det(\Gamma_\varepsilon) &= \det(\Sigma) \det(I_{d_y} - \Lambda_{\varepsilon} \Omega_{YY} \Lambda_{\varepsilon}) \\ &= \det(\Sigma) \det(\varepsilon \Lambda_{\varepsilon} \Omega_{YY}) \end{align}\] where the last step follows from equation (33 ). Therefore, \[\frac{\det (\Theta)}{\det (\Theta_0)} = \frac{\det(\Sigma)^{-1} \det(\varepsilon \Lambda_{\varepsilon} \Omega_{YY})^{-1}}{\det(\Sigma)^{-1}} = \det(\varepsilon \Lambda_{\varepsilon} \Omega_{YY})^{-1}.\]

It remains to find explicit formulas for \((f^\varepsilon, g^\varepsilon, h^\varepsilon)\) so that \[\begin{align} &\frac{1}{\varepsilon}\left(f^\varepsilon(u) + \langle g^\varepsilon(u), x\rangle + h^\varepsilon(x,y) - \frac{1}{2}\|u-y\|^2\right) \\ &\quad = -\frac{1}{2}(w-m)^\top \Delta \Theta (w-m) - \frac{1}{2}\log \det(\varepsilon \Lambda_{\varepsilon} \Omega_{YY}). \end{align} \label{eq:32density32gaussian}\tag{34}\] By direct calculation, \[\begin{align} \varepsilon(w-m)^\top \Delta \Theta (w-m) &= u^\top \Lambda_{\varepsilon} u + x^\top G\Psi_\varepsilon G^\top x + (y-m_Y)^\top \Psi_\varepsilon(y-m_Y) \\ &\qquad -2u^\top G^\top x -2u^\top (y-m_Y) + 2x^\top G\Psi_\varepsilon(y-m_Y). \end{align}\] It is straightforward to verify that choosing \((f^\varepsilon,g^\varepsilon,h^\varepsilon)\) as in (17 ) satisfies (34 ). 0◻

5.3 Proof of Proposition 8↩︎

The weak convergence follows directly from the closed-form expression of \(\pi^\varepsilon\). For \((U,\tilde{X},\tilde{Y}) \sim \pi^o\), the conditional distribution of \(\tilde{Y}\) given \((U,\tilde{X})\) is a multivariate Gaussian distribution with mean \[m_Y + \begin{pmatrix} \Lambda_o & \Sigma_{YX} \end{pmatrix} \begin{pmatrix} I_{d_y} & O \\ O & \Sigma_{XX}^{-1} \end{pmatrix} \begin{pmatrix} U \\ \tilde{X} \end{pmatrix} =m_Y + \Lambda_o U + \Sigma_{YX}\Sigma_{XX}^{-1}\tilde{X},\] and covariance matrix \[\Sigma_{YY}-\begin{pmatrix} \Lambda_o & \Sigma_{YX} \end{pmatrix} \begin{pmatrix} I_{d_y} & O \\ O & \Sigma_{XX}^{-1} \end{pmatrix} \begin{pmatrix} \Lambda_o \\ \Sigma_{YX} \end{pmatrix} =\Sigma_{YY}-\Lambda_o^2-\Sigma_{YX}\Sigma_{XX}^{-1}\Sigma_{XY} = O,\] that is, \(\tilde{Y} = m_Y + \Lambda_o U + \Sigma_{YX}\Sigma_{XX}^{-1}\tilde{X}\) a.s. In view of Example 2.1 in [7], \(\pi^o\) is optimal for (1 ). 0◻

5.4 Proof of Proposition 9↩︎

Assume without loss of generality that \(m_Y = 0\). We write \(w=(u,x,y) \in \mathbb{R}^{N}\) with \(N:=d_x+2d_y\) instead of \(w=(u^\top,x^\top,y^\top)^\top\) for notational convenience. Observe that \(\pi^o\) is supported on the vector subspace \[\mathcal{S} := \left\{(u,x,y) \in \mathbb{R}^{N} :y = \Lambda_o u -G^\top x \right\}.\] Let \(\mathcal{S}^\bot\) denote the orthocomplement of \(\mathcal{S}\) and \(\mathsf{P}_{\mathcal{S}^\bot}\) denote the projection matrix onto \(\mathcal{S}^\bot\) with \(\mathsf{P}_{\mathcal{S}}:=I_{N}-\mathsf{P}_{\mathcal{S}^\bot}\). We observe that for any \(W^\varepsilon \sim \pi^\varepsilon\) and \(W^o \sim \pi^o\), \[\| W^\varepsilon-W^o \|^2 = \| \mathsf{P}_{\mathcal{S}^\bot}W^\varepsilon\|^2 + \| \mathsf{P}_{\mathcal{S}}W^\varepsilon-W^o\|^2.\] This implies \[\mathsf{W}_2^2(\pi^\varepsilon,\pi^o) = \mathbb{E}[\| \mathsf{P}_{\mathcal{S}^\bot}W^\varepsilon\|^2] + \mathsf{W}_2^2(\pi^\varepsilon \circ \mathsf{P}_{\mathcal{S}}^{-1},\pi^o).\] The first term on the right-hand side depends only on the marginal \(\pi^\varepsilon\). We split the rest of the proof into two steps.

Step 1. We first establish that \[\mathbb{E}[\| \mathsf{P}_{\mathcal{S}^\bot}W^\varepsilon\|^2] = \varepsilon \mathop{\mathrm{tr}}(L^{-1}\Lambda_o) + O(\varepsilon^{2}).\] Let \(U \sim \mathcal{N}(0,I_{d_y})\) and \(X \sim \mathcal{N}(0,\Sigma_{XX})\) be independent and generate \(Y^\varepsilon\) as \[Y^\varepsilon = \Lambda_\varepsilon U -G^\top X + \eta_\varepsilon, \quad \eta_\varepsilon \sim \mathcal{N}(0,\varepsilon \Lambda_\varepsilon), \;\eta_\varepsilon \mathpalette{\independenT}{\perp}(U,X).\] It is not difficult to see that \(W^\varepsilon := (U,X,Y^\varepsilon) \sim \pi^\varepsilon\). We have \[W^\varepsilon = \underbrace{(U,X,\Lambda_o U - G^\top X)}_{\in \mathcal{S}} + (0,0,(\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon),\] so that \(\mathsf{P}_{\mathcal{S}^\bot}W^\varepsilon = \mathsf{P}_{\mathcal{S}^\bot}(0,0,(\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon)\). We shall verify the following lemma.

Now, we have \[\begin{align} \| \mathsf{P}_{\mathcal{S}^\bot}(0,0,(\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon)\|^2 &= \big((\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon\big)^\top L^{-1}\big((\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon\big) \\ &=\mathop{\mathrm{tr}}\Big( L^{-1} \big((\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon\big)\big((\Lambda_\varepsilon-\Lambda_o)U+\eta_\varepsilon\big)^\top \Big). \end{align}\] Taking expectation, we have \[\begin{align} \mathbb{E}[\| \mathsf{P}_{\mathcal{S}^\bot}W^\varepsilon\|^2] &= \mathop{\mathrm{tr}}\Big ( L^{-1} \big ( (\Lambda_\varepsilon - \Lambda_o)^2 + \varepsilon \Lambda_\varepsilon \big) \Big) \\ &=\varepsilon \mathop{\mathrm{tr}}(L^{-1}\Lambda_o) + \mathop{\mathrm{tr}}\big ( L^{-1} (\Lambda_\varepsilon - \Lambda_o)^2 \big ) + \varepsilon \mathop{\mathrm{tr}}\big(L^{-1}(\Lambda_\varepsilon - \Lambda_o)\big). \end{align}\] Using spectral expansions, one can see that \(\Lambda_\varepsilon - \Lambda_o = O(\varepsilon)\), so that the last two terms on the right-hand side are \(O(\varepsilon^2)\). This finishes the proof of Step 1.

Step 2. In this step, we shall show that \[\mathsf{W}_2^2(\pi^\varepsilon \circ \mathsf{P}_{\mathcal{S}}^{-1},\pi^o) = O(\varepsilon^2),\] which, combined with Step 1, leads to the conclusion of the proposition.

Let \(\mathsf{Q} \in \mathbb{R}^{N \times(d_x + d_y)}\) be a matrix whose columns consist of orthonormal vectors spanning \(\mathcal{S}\), so that \(\mathsf{P}_{\mathcal{S}} = \mathsf{Q} \mathsf{Q}^\top\). Observe that \[\mathsf{W}_2^2(\pi^\varepsilon \circ \mathsf{P}_{\mathcal{S}}^{-1},\pi^o) = \inf_{W^\varepsilon \sim \pi^\varepsilon,W^o \sim \pi^o} \mathbb{E}[\| \mathsf{Q}^\top W^\varepsilon - \mathsf{Q}^\top W^o\|^2].\] We have \(\mathsf{Q}^\top W^\varepsilon \sim \mathcal{N}(0,\mathsf{Q}^\top \Gamma_\varepsilon \mathsf{Q})\) and \(\mathsf{Q}^\top W^o \sim \mathcal{N}(0,\mathsf{Q}^\top \Gamma_o \mathsf{Q})\). Now, [36] yields \[\inf_{W^\varepsilon \sim \pi^\varepsilon,W^o \sim \pi^o} \mathbb{E}[\| \mathsf{Q}^\top W^\varepsilon - \mathsf{Q}^\top W^o\|^2] \le \| (\mathsf{Q}^\top\Gamma_\varepsilon\mathsf{Q})^{1/2} - (\mathsf{Q}^\top\Gamma_o \mathsf{Q})^{1/2}\|_{\text{F}}^2,\] where \(\| \cdot \|_{\mathrm{F}}\) denotes the Frobenius norm. Observe that \(\| \mathsf{Q}^\top(\Gamma_\varepsilon - \Gamma_o) \mathsf{Q}\|_{\text{F}} \leq \|\Gamma_\varepsilon - \Gamma_o\|_{\text{F}}\). Recalling the explicit formulas for \(\Gamma_\varepsilon\) and \(\Gamma_o\), we have \(\|\Gamma_\varepsilon - \Gamma_o\|_{\text{F}}^2 = 2\|\Lambda_\varepsilon -\Lambda_o\|_{\text{F}}^2 \le d_y\varepsilon^2 / 2\). Since \(\lambda_{\min} ((\mathsf{Q}^\top \Gamma_o \mathsf{Q})^{1/2}) = \sqrt{\lambda_{\min} (\mathsf{Q}^\top \Gamma_o \mathsf{Q})} > 0\) (with \(\lambda_{\min}(\cdot)\) denoting the minimum eigenvalue), [54] yields \[\| (\mathsf{Q}^\top\Gamma_\varepsilon\mathsf{Q})^{1/2} - (\mathsf{Q}^\top\Gamma_o \mathsf{Q})^{1/2}\|_{\text{F}} \le \frac{1}{\sqrt{\lambda_{\min} (\mathsf{Q}^\top \Gamma_o \mathsf{Q})}} \| \mathsf{Q}^\top(\Gamma_\varepsilon - \Gamma_o) \mathsf{Q}\|_{\text{F}}.\] Conclude that \[\mathsf{W}_2^2(\pi^\varepsilon \circ \mathsf{P}_{\mathcal{S}}^{-1},\pi^o) \le \frac{1}{2\lambda_{\min} (\mathsf{Q}^\top \Gamma_o \mathsf{Q})} d_y \varepsilon^2 = O(\varepsilon^2),\] completing the proof. 0◻

6 Partial converse of Lemma 2↩︎

The following lemma provides a partial converse of Lemma 2.

References↩︎

---

1. K. Kato is partially supported by NSF grant DMS-2413405.↩︎

2. We refer the reader to [4], [5] as standard references on OT theory.↩︎

3. Weak precompactness of \(\mathcal{L}_a\) also follows from the (classical) Sanov theorem (cf. Theorem 3.2.17 in [49]).↩︎