Abstract: This paper primarily considers the robust estimation problem under Wasserstein distance constraints on the parameter and noise distributions in the linear measurement model with additive noise, which can be formulated as an infinite-dimensional nonconvex minimax problem. We prove that the existence of a saddle point for this problem is equivalent to that for a finite-dimensional minimax problem, and give a counterexample demonstrating that the saddle point may not exist. Motivated by this observation, we present a verifiable necessary and sufficient condition whose parameters can be derived from a convex problem and its dual. Additionally, we also introduce a simplified sufficient condition, which intuitively indicates that when the Wasserstein radii are small enough, the saddle point always exists. In the absence of the saddle point, we solve an finite-dimensional nonconvex minimax problem, obtained by restricting the estimator to be linear. Its optimal value establishes an upper bound on the robust estimation problem, while its optimal solution yields a robust linear estimator. Numerical experiments are also provided to validate our theoretical results.

Keywords: Robust estimation, Wasserstein distance, Saddle point

1 Introduction↩︎

Robust estimation is a classical methodology widely employed in statistics [1]–[3] and engineering [4]–[6] to deal with parameter uncertainty arising from limited observable data, noise, outliers, and measurement errors. Within this framework, decision makers are typically assumed to have only partial information about uncertain parameters. Based on this partial information, an appropriate uncertainty set can be constructed: for deterministic parameters, the set contains all possible values [7]; for random parameters, it can be defined by the family of admissible distributions or by the structured uncertainties about some statistics [8]. By minimizing the worst-case risk over the uncertainty set, the estimator can be derived that exhibit relative insensitivity to deviations of the actual model from the assumed model [9], [10]. However, it also implies that the performance of robust estimator depends heavily on the characterization of uncertainties.

An important research direction in robust estimation for random parameters, known as distributionally robust estimation, is designed to minimize the worst-case risk within a specified distributional uncertainty set. A common assumption in this framework is that the unknown true distribution is proximate to a predefined nominal distribution in some sense, where the nominal distribution captures known characteristics and can often be chosen as the empirical distribution derived from observed samples. Numerous metrics are employed to quantify this proximity, including Kullback-Leibler (KL) divergence [11], Wasserstein distance [12], \(\phi\)-divergence [13], moments-based similarity [14], and \(\epsilon\)-contamination [15].

Under this assumption, the distributionally robust estimation problem can be formulated as an infinite-dimensional minimax problem. Existing work has explored related topics under various models. In [11], the authors derive the optimal least-squares estimator for the least-favorable distribution within a KL-divergence ball centered on a given nominal joint distribution of parameter and observation. This result has been later applied to state-space models to obtain robust Kalman filtering, yielding significant practical implications [16], [17]. Following this, [18] extends this result to the multi-sensor setting by imposing the same KL divergence constraint on each marginal joint distribution of the parameter and the single-sensor observation. Additionally, [19] considers the linear observation model with additive noise and obtains the minimum mean square error estimator corresponding to the least favorable distribution in a KL-divergence ball on the parameter distribution. Typically, these studies identify an optimal solution to a finite-dimensional auxiliary problem and establish its role as a saddle point solution to the infinite-dimensional minimax problem. On the other hand, for the Wasserstein ambiguity sets,[20] considers the distributionally robust estimation problem under a Wasserstein distance ball on the joint distribution, reformulating the minimax problem as an semidefinite programming (SDP) problem via its Nash equilibrium. Subsequently, [13]–[15] considered similar distributionally robust estimation problems defined by more types of metrics in the state space model.

However, as the model complexity increases, such as the linear measurement model with additive noise by assuming separate uncertainties in both the parameter and the noise, which is a common scenario in practical dynamic systems, the distributionally robust estimation problem becomes nonconvex and the existence of saddle points is not always guaranteed. In this paper, we mainly focus on a Wasserstein distributionally robust estimation (WDRE) problem in the linear measurement model with additive noise, where the parameter and the noise are mutually independent and their true distributions are unknown, and each constrained in a Wasserstein-distance ball. This problem has been considered in [12] and the authors prove that it is equivalent to a finite-dimensional SDP problem when a saddle point exists. However, an example is proposed in this paper to illustrate that the saddle point solution to this problem does not necessarily exist, motivating the persuit for the conditions that allow for its existence. On the other hand, if the saddle point is absent, the optimal value derived from the SDP problem can merely provide a lower bound for the optimal value of the WDRE problem. Consequently, we must either tackle the original infinite-dimensional nonconvex robust estimation problem directly or establish an upper bound by relaxing the minimax problem.

Our main contributions are summarized as follows:

• We first prove that the saddle point solution of the WDRE problem exists if and only if that of a finite-dimensional minimax problem exists. This finite-dimensional problem is formulated by restricting the distribution to be Gaussian and the estimator to be linear. Then we establish a connection among the optimal values of this finite-dimensional problem, the WDRE problem, and their corresponding problems obtained by interchanging the minimization and maximization. Furthermore, we provide an example to illustrate that the saddle point solution of the WDRE problem may not exist.

• We present a necessary and sufficient condition for the existence of the saddle point solution for the WDRE problem, which can be precisely characterized by determining its parameters through the resolution of a convex problem and its dual. Moreover, to simplify the assessment of the existence of the saddle point solution, we also provide a straightforward sufficient condition, which indicates that when the Wasserstein radii of the uncertainty sets are small enough, the saddle point always exists.

• In the absence of the saddle point, we focus on a robust estimation problem with the linear estimator, which can be formulated as a finite-dimensional nonconvex problem. The optimal value of this problem, which serves as an upper bound for the WDRE problem, can be demonstrated to be equivalent to the optimal value of a SDP problem. Furthermore, leveraging the primal and dual optimal solutions of this SDP problem, we construct the optimal solution to the finite-dimensional nonconvex problem, which yields a robust linear estimator.

The rest of this paper is organized as follows. Section 2 formally presents the WDRE problem. Section 3 provides a theoretical analysis for the existence of the saddle point solution to the WDRE problem. In cases where the saddle point is absent, Section 4 addresses an upper bound problem that restricts the estimator to a linear function, which provides a robust linear estimator. Subsequently, Section 5 presents numerical experiments designed to validate the theoretical results. Finally, Section 6 concludes this paper.

Notation: Let \(\mathbb{R}^n\) be the \(n\) dimensional real vector space and \(\mathbb{R}^{n \times m}\) be the \(n \times m\) dimensional real matrix space. The notation \(I_n\) stands for the identity matrix in \(\mathbb{R}^{n \times n}\) and \(\boldsymbol{0}\) denotes the zero matrix of proper dimension. For any \(A \in \mathbb{R}^{n \times n}\), we use \(A^-\) and \(A^\dagger\) to denote the generalized inverse and the Moore-Penrose pseudo-inverse of the matrix \(A\). The notation \(Tr(A)\) denotes the trace of the matrix \(A\) and the null space of \(A\) is denoted by \(Null(A)\). For any \(A, B \in \mathbb{R}^{n \times n}\), the inner product of \(A\) and \(B\) is denoted by \(\left\langle A,B \right\rangle =Tr(AB)\) and \(A \perp B\) means \(\left\langle A,B \right\rangle =0\). Let \(\mathbb{S}^n\) be the space of symmetric matrices in \(\mathbb{R}^{n \times n}\) and \(\mathbb{S}^n_+\) be the cone of positive semidefinite matrices in \(\mathbb{S}^n\). For any \(A \in \mathbb{S}^n\), we use \(\lambda_{\min}(A)\) and \(\lambda_{\max}(A)\) to denote the minimum and maximum eigenvalues of \(A\), respectively. For any \(A \in \mathbb{S}^n_+\), the notation \(A^{\frac{1}{2}}\) denotes the unique positive semidefinite square root of \(A\). For any \(A,B \in \mathbb{S}^n\), the relation \(A \succ B\) \((A \succeq B)\) means that \(A-B\) is positive definite (semidefinite). For a measurable function of two variables \(f(x,y)\), we use \(\bigtriangledown_x f (x,y)\) to denote the partial derivative of \(f(x,y)\) with respect to \(x\). The notation \(\| \cdot \|\) denotes the Euclidean norm. A normal distribution with mean vector \(\mu\) and covariance matrix \(\Sigma\) is denoted by \(\mathcal{N}(\mu,\Sigma)\).

2 Problem Formulation↩︎

Consider the linear measurement model with additive noise \[y=H x +w,\] where \(x \in \mathbb{R}^n\) is the unknown parameter, \(y \in \mathbb{R}^m\) is the observation, \(H \in \mathbb{R}^{m \times n}\) is the known observation matrix and the noise \(w \in \mathbb{R}^m\) is independent of \(x\). Let a Lebesgue measurable function \(f:\mathbb{R}^m \to \mathbb{R}^n\) be an estimator that estimates parameter \(x\) from the noisy observation \(y\) and \[\mathcal{F} \triangleq \left\{ f \left| f:\mathbb{R}^m \to \mathbb{R}^n \text{ is a Lebesgue measurable function} \right\} \right. \label{fc}\tag{1}\] be the set of all estimators. Moreover, let \(\mathcal{P}_d\) denote the set of probability distributions of a random variable on \(\mathbb{R}^d\) with finite second order moments. Then for a given joint distribution of \(x\) and \(w\) denoted by \(P \in \mathcal{P}_{n+m}\), the mean square error (MSE) obtained by the estimator \(f\) is denoted by \[mse(f,P) \triangleq \int_{\mathbb{R}^n \times \mathbb{R}^m} \left\| f(Hx+w)-x\right\| ^2 \mathrm{d} P(x,w). \label{mse}\tag{2}\] Furthermore, the minimum mean square error (MMSE) of the distribution \(P\), as determined by the optimal estimator corresponding to \(P\), is provided by \[mmse(P) \triangleq \inf_{f \in \mathcal{F}} \int_{\mathbb{R}^n \times \mathbb{R}^m} \left\| f(Hx+w)-x\right\| ^2 \mathrm{d} P(x,w).\]

Similar to [12], assume that the marginal distributions \(P_x\) of the parameter \(x\) and \(P_w\) of the noise \(w\) are unknown. However, given the nominal distributions \(\hat{P}_x\) and \(\hat{P}_w\), the uncertainty about \(P_x\) and \(P_w\) can be quantified by their corresponding bounded Wasserstein distances from \(\hat{P}_x\) and \(\hat{P}_w\). In this paper, we focus exclusively on the type-2 Wasserstain distance, which is defined for two distributions \(P\) and \(\hat{P}\) on \(\mathbb{R}^d\) as follows: \[W_2\left( P,\hat{P} \right) \triangleq \inf_{\pi \in \Pi \left( P, \hat{P} \right) } \left( \int_{\mathbb{R}^d \times \mathbb{R}^d} \left\| \xi_1-\xi_2\right\| ^2 \pi(\mathrm{d}\xi_1,\mathrm{d}\xi_2) \right) ^\frac{1}{2},\] where \(\Pi \left( P, \hat{P} \right)\) denotes the set of all joint distributions of the random variables \(\xi_1\) and \(\xi_2\) that have marginal distributions \(P\) and \(\hat{P}\), respectively. Specifically, for the given nominal Gaussian marginal distributions \(\hat{P}_x\) and \(\hat{P}_w\), due to the independence of \(x\) and \(w\), the nominal joint distribution of \(x\) and \(w\) is \(\hat{P}=\hat{P}_x \times \hat{P}_w\), where the product of two distributions means that for any \((x,w)\in \mathbb{R}^n \times \mathbb{R}^m\), \(\hat{P}(x,w)=\hat{P}_x(x) \hat{P}_w(w)\). Taking the Wasserstain radii \(\rho_x \geq 0\) and \(\rho_w \geq 0\), we assume that the true joint distribution \(P\) belongs to the following set \[\mathbb{B}(\hat{P})\triangleq \left\{ P_x \times P_w \in \mathcal{P}_n \times \mathcal{P}_m \left| \begin{align} &W_2\left( P_x,\hat{P}_x \right) \leq \rho_x, \; \hat{P}_x=\mathcal{N}\left(\hat{\mu}_x,\hat{\Sigma}_x \right) \\ &W_2\left( P_w,\hat{P}_w \right) \leq \rho_w,\; \hat{P}_w=\mathcal{N}\left( \hat{\mu}_w,\hat{\Sigma}_w \right) \end{align} \right\} \right., \label{set}\tag{3}\] where \(\mathcal{N}(\mu,\Sigma)\) denotes a normal distribution with mean vector \(\mu\) and covariance matrix \(\Sigma\). We intend to find the optimal estimator of the least favorable distribution in set (3 ), i.e., considering the following robust estimation problem \[\inf_{f \in \mathcal{F}} \sup_{P \in \mathbb{B}(\hat{P})} mse \left( f,P \right), \label{minmax} \tag{4}\] where the objective function is given by (2 ), the constraint sets are defined by (1 ) and (3 ), respectively, and the following assumption will be made in this paper.

Assumption 1.

i) The Wasserstein radii \(\rho_x>0\) and \(\rho_w>0\).

ii) The nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\) are positive semidefinite.

3 The Existence of the Saddle Point Solution for (4 )↩︎

In this section, we devote to a theoretical analysis on the existence of the saddle point solution for the robust estimation problem (4 ). To begin with, we demonstrate that the saddle point solution for (4 ) exists if and only if a finite-dimensional minimax problem has a saddle point solution. Subsequently, we show that the saddle point solution does not always exist for (4 ) by means of a counterexample. Following this, we present a necessary and sufficient condition for the existence of the saddle point solution. Furthermore, we propose a straightforward sufficient condition, which does not require complex calculations and allows us to ascertain the existence of the saddle point solution when it is satisfied.

3.1 An Finite-dimensional Problem Related to the Existence of the Saddle Point Solution for (4 )↩︎

We first glance at problem (4 ) as an optimization problem with infinite-dimensional variables being the distribution \(P\) and the estimator \(f\). Notice that the objective function of (4 ) is linear in \(P \in \mathbb{B}(\hat{P})\) for a given \(f\), and convex in \(f \in \mathcal{F}\) for a given \(P\). However, according to the definition of \(\mathbb{B}(\hat{P})\) in (3 ), the elements in the constraint set of the joint distribution \(\mathbb{B}(\hat{P})\) depends on the elements in the two convex Wasserstein balls, which is generally not convex. Consequently, it is hard to apply current theoretical results (such as Sion’s minimax theorem) to directly deduce the existence of the saddle point solution for (4 ).

In order to discuss the existence of the saddle point solution for (4 ), we first formulate a finite-dimensional minimax problem. Specifically, we restrict the distribution to the set consisting only of Gaussian distributions \[\mathbb{B}_\mathcal{N}\left( \hat{P}\right) \triangleq \left\lbrace P_x \times P_w \in \mathcal{P}_n \times \mathcal{P}_m \left| \begin{align} &W_2\left( P_x,\hat{P}_x \right) \leq \rho_x, \; \hat{P}_x=\mathcal{N}\left(\hat{\mu}_x,\hat{\Sigma}_x \right), P_x=\mathcal{N}\left(\mu_x,\Sigma_x \right) \\ &W_2\left( P_w,\hat{P}_w \right) \leq \rho_w,\; \hat{P}_w=\mathcal{N}\left( \hat{\mu}_w,\hat{\Sigma}_w \right), P_w=\mathcal{N}\left(\mu_w,\Sigma_w \right) \end{align} \right\rbrace. \right. \label{Guass32class}\tag{5}\] Correspondingly, the estimator is limited to the set of linear functions \[\mathcal{F_L} \triangleq \left\lbrace f \in \mathcal{F} \left| \undefined A \in \mathbb{R}^{n \times m}, b \in \mathbb{R}^nwithf(y)=Ay+b, \;\undefined y \in \mathbb{R}^m \right\rbrace. \right. \label{linear32class}\tag{6}\] Then we introduce an auxiliary problem \[\inf_{f \in \mathcal{F_L}} \sup_{P \in \mathbb{B}_\mathcal{N}\left( \hat{P}\right) } mse \left( f,P \right), \label{minmaxG} \tag{7}\] where the objective function is given by (2 ), and the constraint sets are defined by (6 ) and (5 ), respectively. Essentially, (7 ) is a finite-dimensional robust estimation problem, because the linear estimator \(f\) can be parameterized by the matrix \(A\) and the vector \(b\), and the Gaussian distributions can also be fully described by their mean vectors and covariance matrices. Next, we shall explore the equivalence between the existence of the saddle point solution in (4 ) and that in (7 ).

First, we review a property of Wasserstain distance.

Lemma 1 shows that for any distribution in \(\mathbb{B}(\hat{P})\) , the Gaussian distribution defined by its mean vector and covariance matrix is still in \(\mathbb{B}(\hat{P})\). Then we have the following lemma.

With the help of Lemma 2, we can now establish the equivalence between the existence of the saddle point solution in (4 ) and that in (7 ).

The above theorem shows that if a saddle point solution exists for (4 ), there must be a saddle point solution consisting of a linear estimator and a Gaussian distribution, which is also a saddle point solution for (7 ). Unfortunately, we have constructed a counterexample in which the saddle point solution of (7 ) does not exist.

Counterexample: Consider the scalar case, i.e., \(m=n=1\). Let the nominal means and variances be \(\hat{\mu}_x=\hat{\mu}_w=0\) and \(\hat{\Sigma}_x=\hat{\Sigma}_w=1\), respectively. Take the Wasserstein radii \(\rho_x=\rho_w=2\) and the observation matrix \(H=1\). Consider the problem \[\sup_{P \in \mathbb{B}_\mathcal{N}\left( \hat{P}\right) } \inf_{f \in \mathcal{F_L}} \text{mse}\left( f,P\right) .\] It can be parameterized as follows [12] \[\begin{align} \sup_{\mu_x,\mu_w,\Sigma_x,\Sigma_w} &\Sigma_x-\Sigma_x \left(\Sigma_x+\Sigma_w \right) ^{-1} \Sigma_x \\ s.t. \;&\mu_x^2 + \Sigma_x+1-2 \sqrt{\Sigma_x} \leq 4, \;\Sigma_x \geq 0 ,\\ & \mu_w^2+ \Sigma_w+1-2 \sqrt{\Sigma_w} \leq 4, \;\Sigma_w \geq 0 . \label{example} \end{align}\tag{11}\] Then the objective function of (11 ) can be expressed as \[\begin{align} \Sigma_x-\Sigma_x \left( \Sigma_x+\Sigma_w \right) ^{-1} \Sigma_x &=\Sigma_x \left( \Sigma_x+\Sigma_w \right) ^{-1} \left( \Sigma_x+\Sigma_w \right) -\Sigma_x \left( \Sigma_x+\Sigma_w \right) ^{-1} \Sigma_x \\ &=\Sigma_x \left( \Sigma_x+\Sigma_w \right) ^{-1} \Sigma_w \\ &=\left(\Sigma_w^{-1}+\Sigma_x^{-1} \right) ^{-1} , \end{align}\] which is monotonically increasing with \(\Sigma_x\) and \(\Sigma_w\). Therefore, (11 ) has a unique optimal solution given by \(\mu_x^*=\mu_w^*=0\), \(\Sigma_x^*=\Sigma_w^*=9\), i.e., the least favorable distribution \(P^*=\mathcal{N}(0,9) \times \mathcal{N}(0,9)\). Then the corresponding optimal estimator is given by \[f^*(y)= \frac{H \Sigma_x^*}{H^2 \Sigma_x^*+\Sigma_w^*}y=\frac{1}{2}y,\] and the mean square error is \[mse(f^*,P^*)=\Sigma_x^*-\Sigma_x^* \left(\Sigma_x^*+\Sigma_w^* \right) ^{-1} \Sigma_x^*=\frac{9}{2}.\] But in fact, for \(\tilde{P} = \mathcal{N}\left( \sqrt{3},4\right) \times \mathcal{N}\left( -\sqrt{3},4\right)\), the type-2 Wasserstain Distance between \(\tilde{P}\) and the nominal distribution is given by \[W_2 \left(\mathcal{N}(\sqrt{3},4), \mathcal{N}(0,1)\right) = \sqrt{\|\sqrt{3} - 0\|^2 + \left[ 4+1 - 2\times 4^\frac{1}{2}\right] } = 2,\] and \[W_2 \left(\mathcal{N}(-\sqrt{3},4), \mathcal{N}(0,1)\right) = \sqrt{\|-\sqrt{3}- 0\|^2 + \left[ 4 + 1 - 2 \times 4^\frac{1}{2}\right] } = 2.\] It implies that \(\tilde{P} \in \mathbb{B}_\mathcal{N}( \hat{P}) \subseteq \mathbb{B}(\hat{P})\). And we have \[\begin{align} mse( f^*,\tilde{P}) &=\int_{\mathbb{R} \times \mathbb{R}} \left\| \frac{1}{2} (x+w) - x \right\| ^2 \mathrm{d} \tilde{P} \\ &=\frac{1}{4} \Sigma_x +\frac{1}{4} \Sigma_w +\left( \frac{1}{2} \mu_x - \frac{1}{2}\mu_w \right) ^2 \\ &=5> mse\left( f^*,P^*\right) . \end{align}\] Hence, for the given \(f^*\), the corresponding least favorable distribution is not \(P^*\), which indicates that the unique solution to the maximin problem \(\left( f^*,P^*\right)\) does not form a saddle point solution of (7 ).

3.2 A Necessary and Sufficient Condition for the Existence of the Saddle Point Solution for (4 )↩︎

When a saddle point solution exists, it is easy to deduce from Theorem 1 that solving the infinite-dimensional robust estimation problem (4 ) is equivalent to solving the finite-dimensional minimax problem (7 ). Under such scenario, all inequalities in Lemma 2 become equalities. Consequently, as detailed in [12], the optimal solution to (4 ) can be effectively obtained by solving the tractable problem \[\sup_{P \in \mathbb{B}_\mathcal{N}\left( \hat{P}\right) } \inf_{f \in \mathcal{F_L}} mse\left( f,P\right) . \label{maxmina}\tag{12}\] Conversely, in the absence of the saddle point, the optimal value of (12 ) serves merely as a strict lower bound for the optimal value of (4 ). It complicates the resolution of (4 ) and motivates us to establish a necessary and sufficient condition for its existence in this case.

To identitify this condition, we commence our analysis with (12 ). Since the distribution and the estimator are restricted to be a Gaussian distribution in (5 ) and a linear estimator in (6 ) respectively, (12 ) can be reformulated as a finite-dimensional optimization problem, where MSE can be naturally parameterized as the following objective function by its definition, and the Wasserstein distance degenerates into the Gelbrich distance [23]: \[\begin{align} \sup_{\mu_x,\mu_w,\Sigma_x,\Sigma_w} \inf_{A,b} \;\; &Tr\left[ \left( AH-I_n\right) \Sigma_x \left( AH-I_n\right) ^\top +A\Sigma_w A^\top \right] +\left[ \left( AH-I_n \right) \mu_x+A\mu_w+b\right] ^\top \left[ \left( AH-I_n \right) \mu_x+A\mu_w+b\right] \nonumber \\ s.t. \; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \nonumber \\ & \left\| \mu_x-\hat{\mu}_x \right\| ^2+ Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_x^2, \nonumber \\[0.5ex] & \left\| \mu_w-\hat{\mu}_w \right\| ^2+ Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_w^2. \label{maxminap} \end{align}\tag{13}\] For fixed \(\mu_x\), \(\mu_w\), \(\Sigma_x\), \(\Sigma_w\) and \(A\), the optimal solution to the inner minimization problem is given by \(b^* = (I_n - A H)\mu_x - A \mu_w\), which minimizes the objective function by eliminating the quadratic term with respect to \(b\). By substituting \(b^*\) into the above equation, problem (13 ) becomes \[\begin{align} \sup \limits_{\mu_x,\mu_w,\Sigma_{x},\Sigma_w} \inf \limits_A \;\; &Tr\left[ \left( AH-I_n \right) \Sigma_x \left( AH-I_n \right) ^\top +A\Sigma_w A^\top \right]\\ s.t. \; &\Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \\&\left\| \mu_x-\hat{\mu}_x\right\| ^2+Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_x^2, \\&\left\| \mu_w-\hat{\mu}_w\right\| ^2+Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_w^2. \\\end{align}\] It is straightforward to observe that the objective function is independent of \(\mu_x\) and \(\mu_w\). Consequently, setting \(\mu_x^*=\hat{\mu}_x\) and \(\mu_w^*=\hat{\mu}_w\) can result in the largest feasible set of \(\Sigma_x\) and \(\Sigma_w\), which leads to the largest objective function value. It follows that the optimal solution must satisfy \(\mu_x^*=\hat{\mu}_x\) and \(\mu_w^*=\hat{\mu}_w\). Accordingly, the above problem can be further simplified to \[\begin{align} \sup \limits_{\Sigma_{x},\Sigma_w} \inf \limits_A \;\; &Tr\left[ (AH-I_n)\Sigma_x(AH-I_n)^\top +A\Sigma_w A^\top \right]\\ s.t. \; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \\& Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_x^2, \\& Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_w^2. \\\end{align} \label{maxmin1}\tag{14}\]

For the sake of simplicity of notations, we denote the constraint set in (14 ) by \[\mathbb{B}_\Sigma=\left\lbrace \left( \Sigma_x,\Sigma_w \right) \left| \begin{align} &\Sigma_x \succeq{\boldsymbol{0}}, \; Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_x^2 \\ &\Sigma_x \succeq{\boldsymbol{0}}, \; Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_w^2 \end{align} \right. \right\rbrace .\] Since the objective function of (14 ) is convex in \(A\) for each \((\Sigma_x,\Sigma_w)\), concave in \((\Sigma_x,\Sigma_w)\) for each \(A\), and \(\mathbb{B}_\Sigma\) is convex and compact [21], we can apply Sion’s minimax theorem [24] to conclude that \[\begin{align} &\sup_{(\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma} \inf_A \quad \left\langle \left( I_n-AH\right) ^\top \left( I_n-AH\right) ,\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle \tag{15} \\ &=\inf_A \sup_{(\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma} \left\langle \left( I_n-AH \right) ^\top \left( I_n-AH \right) ,\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle, \tag{16} \end{align}\] where (15 ) is a reformulation of (14 ) with simplified notations. Furthermore, since \(\mathbb{B}_\Sigma\) is compact, and there exists a pair of positive definite matrices \((\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma\) such that the objective function is coercive in \(A\), then the above problems admit the saddle point solution [25].

Then problem (15 ) can be transformed into \[\begin{align} &\sup_{(\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma} \inf_A \quad \left\langle \left( I_n-AH\right) ^\top \left( I_n-AH\right) ,\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle \notag \\ &\begin{aligned} =\sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } \; &\left[ \inf_A \quad \left\langle \left( I_n-AH\right) ^\top \left( I_n-AH\right) ,\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle \right]+ \notag \\ &\gamma_x \left\{ \rho_x^2- Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\}+ \gamma_w \left\{ \rho_w^2- Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\} \end{aligned} \\ &\begin{align} =\sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} \inf_A \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } \quad &\left\langle \left(I_n-AH\right) ^\top \left(I_n-AH\right) ,\Sigma_{x} \right\rangle +\gamma_x \left\{ \rho_x^2- Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\} \\ &+\left\langle A^\top A,\Sigma_w \right\rangle +\gamma_w \left\{ \rho_w^2- Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\}, \label{maxminL} \end{align} \end{align}\tag{17}\] where the first equality arises from reformulating the constraints as penalty terms in the objective function. Specifically, when \((\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma\), the corresponding optimal multiplier \((\gamma_x,\gamma_w)\) satisfies \[\gamma_x \left\lbrace \rho_x^2-Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\rbrace =0\] and \[\gamma_w \left\lbrace \rho_w^2-Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\rbrace =0,\] thereby keeping the objective function value unchanged; otherwise, the multiplier \(\gamma_x\) or \(\gamma_w\) tends to \(+\infty\), driving the objective function to \(-\infty\).

On the other hand, for any fixed \(A\), consider the inner maximization problem in (16 ) \[\sup_{(\Sigma_x,\Sigma_w) \in \mathbb{B}_\Sigma} \left\langle \left( I_n-AH\right) ^\top \left( I_n-AH \right) ,\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle. \label{maxminis32in}\tag{18}\] Problem (18 ) maximizes a continuous concave function on a convex and compact set, and thus it has a finite optimal value. Moreover, since the cartesian set of two cones of positive semidefinite matrices is convex, the objective function and the Gelbrich distance are also convex in \((\Sigma_x,\Sigma_w)\), then strong duality holds under Slater condition, i.e., (18 ) is equivalent to its Lagrangian dual problem \[\begin{align} \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } \sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} \quad &\left\langle \left( I_n-AH\right) ^\top \left( I_n-AH\right) ,\Sigma_{x} \right\rangle +\gamma_x \left\{ \rho_x^2- Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\} \\ &+\left\langle A^\top A,\Sigma_w \right\rangle +\gamma_w \left\{ \rho_w^2- Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\}, \end{align} \label{maxminis32in32dual}\tag{19}\] and the set of optimal dual solutions is nonempty [26]. Thus, problem (16 ) is equivalent to \[\begin{align} \inf_A \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } \sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} \quad &\left\langle \left( I_n-AH\right) ^\top \left( I_n-AH\right) ,\Sigma_{x} \right\rangle +\gamma_x \left\{ \rho_x^2- Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\} \\ &+\left\langle A^\top A,\Sigma_w \right\rangle +\gamma_w \left\{ \rho_w^2- Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right) ^{\frac{1}{2}} \right] \right\}. \label{maxminLe} \end{align}\tag{20}\] We denote the objective function of (20 ), which is also the objective function of (17 ), as \[\begin{align} G(\Sigma_x,\Sigma_w,A,\gamma_x,\gamma_w) \triangleq &\left\langle \left( I_n-AH \right) ^\top \left( I_n-AH \right) ,\Sigma_{x} \right\rangle +\gamma_x \left\{ \rho_x^2- Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \right\} \notag \\ &+\left\langle A^\top A,\Sigma_w \right\rangle +\gamma_w \left\{ \rho_w^2- Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}} \right] \right\}. \label{G} \end{align}\tag{21}\] Then since the optimal values of problems (15 ) and (16 ) are equal, it follows that the optimal values of (17 ) and (20 ) are also equal, i.e., \[\sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} \inf_A \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } G(\Sigma_x,\Sigma_w,A,\gamma_x,\gamma_w)= \inf_A \inf_{\substack {\gamma_x \geq 0 \\ \gamma_w \geq 0} } \sup_{\substack{\Sigma_x\succeq{\boldsymbol{0}}\\ \Sigma_w\succeq{\boldsymbol{0}}}} G(\Sigma_x,\Sigma_w,A,\gamma_x,\gamma_w).\]

Furthermore, we shall give the relationship between saddle point solutions to problems (15 ) and (17 ).

Subsequently, the saddle point solution of problem (17 ) enables us to derive a necessary and sufficient condition for the existence of a saddle point solution for the robust estimation problem (4 ).

Nevertheless, the condition presented in Lemma 4 not easily checked in certain cases. Specifically, if the matrix \(H\Sigma_x^* H^\top + \Sigma_w^*\) is not positive definite, the uniqueness of the optimal solution \(A^*\) for (14 ) is no longer guaranteed and it becomes challenging to identify which solution can form a saddle point solution for (17 ). However, Section 4 proposes that for this case, the existence of a saddle point can be determined numerically by solving two SDPs. Therefore, we shall only focus on a simplified case under a mild assumption below.

Assumption 2.

i) The Wasserstein radii \(\rho_x>0\) and \(\rho_w>0\).

ii) The nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\) are positive definite.

Refocusing on problem (14 ), it is well known that for fixed \(\Sigma_x\) and \(\Sigma_w\), the optimal solution to the inner minimization problem is given by \(A^* = \Sigma_x H^\top (H \Sigma_x H^\top + \Sigma_w)^{-}\). By substituting \(A^*\) into (14 ), we can derive its equivalent problem as follows: \[\begin{array}{cl} \sup \limits_{\Sigma_{x},\Sigma_w} & Tr\left[\Sigma_x - \Sigma_x H^\top \left( H \Sigma_x H^\top + \Sigma_w \right)^{-} H \Sigma_x\right] \\ s.t. & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \\& Tr\left[\Sigma_x + \hat{\Sigma}_x - 2 \left( \hat{\Sigma}_x^{\frac{1}{2}}\Sigma_x\hat{\Sigma}_x^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_x^2, \\[0.5ex] & Tr\left[\Sigma_w + \hat{\Sigma}_w - 2 \left( \hat{\Sigma}_w^{\frac{1}{2}}\Sigma_w\hat{\Sigma}_w^{\frac{1}{2}} \right)^{\frac{1}{2}}\right] \leq \rho_w^2. \\[0.5ex] \end{array} \label{maxmin}\tag{34}\]

The objective function of (34 ), i.e., the inner minimization problem of (14 ) \[\inf_A \quad \left\langle \left(I_n-AH\right)^\top \left(I_n-AH\right),\Sigma_{x} \right\rangle +\left\langle A^\top A,\Sigma_w \right\rangle\] can be interpreted as minimizing a set of linear functions over the variable \((\Sigma_x,\Sigma_w)\), which implies that it is concave in \((\Sigma_x,\Sigma_w)\) [29]. Consequently, (34 ) is a convex optimization problem in which strong duality holds under Slater condition. As detailed in Corollary 3.1 of [12], problem (34 ) can be transformed into the following SDP problem \[\begin{align} \sup \limits_{\Sigma_x,\Sigma_w,V_x,V_w,U} &Tr(\Sigma_x)-Tr(U) \\ s.t.\;\;\;\; &\Sigma_x \succeq {\boldsymbol{0}}, \Sigma_w \succeq {\boldsymbol{0}}, \;V_x \succeq{\boldsymbol{0}}, \;V_w \succeq{\boldsymbol{0}},\\ &\left[\begin{array} {cc} \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}} & V_x \\ V_x &I_n \end{array} \right]\succeq {\boldsymbol{0}}, \quad \left[\begin{array} {cc} \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}} & V_w \\ V_w &I_m \end{array} \right]\succeq {\boldsymbol{0}},\\ &Tr\left( \Sigma_x+\hat{\Sigma}_x-2V_x \right) \succeq {\boldsymbol{0}}, \quad Tr\left( \Sigma_w+\hat{\Sigma}_w-2V_w \right) \succeq {\boldsymbol{0}},\\ &\begin{bmatrix} U & \Sigma_x H^\top \\ H \Sigma & H \Sigma_x H^\top+\Sigma_w \end{bmatrix} \succeq {\boldsymbol{0}}. \end{align} \label{maxmin32SDP}\tag{35}\] In the following theorem, we will demonstrate that under a mild condition, the condition in Lemma 4 can be simplified, with all parameters determined through the convex problem (34 ) and its dual.

3.3 A Sufficient Condition for the Existence of the Saddle Point Solution for (4 )↩︎

Section 3.2 establishes a necessary and sufficient condition for the existence of the saddle point solution in (4 ). However, verifying this condition requires determining both the primal and dual optimal solutions of (34 ), which may be computationally intractable for large-scale problems. In this section, we will present a more direct and computationally simple sufficient condition for the existence of a saddle point solution to (4 ), which intuitively indicates that when the Wasserstein radii \(\rho_x\) and \(\rho_w\) are small enough, the saddle point always exists.

4 The Robust Linear Estimator↩︎

In Section 3.1, Theorem 1 shows that the absence of the saddle point solution in the finite-dimensional optimization problem (7 ) implies the absence of the saddle point solution in the infinite-dimensional optimization problem (4 ). In this case, although the exact optimal solution of (4 ) is unavailable by solving (7 ), the optimal value of (7 ) can still provide an upper bound on that of (4 ), as detailed in Lemma 2. Furthermore, due to (10 ), the optimal solution to (7 ) is also the optimal solution to \[\label{infLsup} \inf_{f \in \mathcal{F_L}} \sup_{P \in \mathbb{B}(\hat{P})} mse(f,P),\tag{48}\] which indicates that this solution yields an optimal robust estimator in the class of linear estimators.

Consequently, the focus of this section is on problem (7 ). Specifically, we demonstrate that (7 ) is equivalent to a convex relaxation problem, and thus equivalent to an SDP problem. Furthermore, based on the primal and dual optimal solutions of the SDP problem, we construct an optimal solution to (7 ), which is also the optimal solution to (48 ) and thus provides a robust linear estimator.

4.1 A Tight Convex Relaxation of (7 )↩︎

In light of the above discussion, we now focus on (7 ). Analogous to parameterizing (12 ) into (13 ) in Section 3.2, problem (7 ) can be parameterized into a finite-dimensional form as follows: \[\begin{align} \inf_{A,b} \sup_{\mu_x,\mu_w,\Sigma_x,\Sigma_w} &Tr\left[ \left( AH-I_n\right) \Sigma_x\left( AH-I_n\right) ^\top+A\Sigma_w A^\top \right]+\left[ \left( AH-I_n\right) \mu_x+A\mu_w+b\right] ^\top \left[ \left( AH-I_n\right) \mu_x+A\mu_w+b \right] \notag \\ s.t.\; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \notag \\ &\left\| \mu_x-\hat{\mu}_x \right\| ^2+Tr \left[ \Sigma_x+\hat{\Sigma}_x- 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_x^2, \label{minmaxGG} \\ &\left\| \mu_w-\hat{\mu}_w\right\| ^2+Tr \left[ \Sigma_w+\hat{\Sigma}_w- 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_w^2. \notag \end{align}\tag{49}\] For given \(A\) and \(b\), the objective function of (49 ) is convex in \((\mu_x,\mu_w)\). Consequently, the inner maximization problem over \((\mu_x,\mu_w)\) is non-convex, which brings a significant challenge in identifying the optimal solution. For ease of notations, we denote \(\tilde{\mu}_x \triangleq \mu_x-\hat{\mu}_x\), \(\tilde{\mu}_w \triangleq \mu_w-\hat{\mu}_w\) and \(\tilde{b} \triangleq b+(AH-I_n)\hat{\mu}_x+A\hat{\mu}_w\), respectively. Then problem (49 ) can be reformulated as follows \[\begin{align} \inf_{A,\tilde{b}} \; \sup_{\tilde{\mu}_x,\tilde{\mu}_w,\Sigma_x,\Sigma_w} &Tr\left[ \left( AH-I_n\right) \Sigma_x\left( AH-I_n\right) ^\top+A\Sigma_w A^\top \right] +\left[\left( AH-I_n\right) \tilde{\mu}_x+A\tilde{\mu}_w+\tilde{b}\right] ^\top \left[\left( AH-I_n \right) \tilde{\mu}_x+A\tilde{\mu}_w+\tilde{b}\right] \notag \\ s.t.\; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \notag \\ &\left\| \tilde{\mu}_x\right\| ^2+Tr \left[ \Sigma_x+\hat{\Sigma}_x- 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_x^2, \label{3minimax} \\ &\left\| \tilde{\mu}_w\right\| ^2+Tr \left[ \Sigma_w+\hat{\Sigma}_w- 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_w^2. \notag \end{align}\tag{50}\] Notice that for given \(A\), \(\tilde{b}\), \(\Sigma_x\) and \(\Sigma_w\), the inner maximization problem over \((\tilde{\mu}_x,\tilde{\mu}_w)\) in (50 ) is a nonhomogeneous QCQP problem with two homogeneous constraints. If we directly apply the SDP relaxation method, we can not assert that the relaxed problem is equivalent to problem (50 ) [27], [32]. To address this difficulty, we first give a special structure of the optimal solution to (50 ) in the following theorem.

With the help of Theorem 4, (50 ) can be equivalently transformed into the following problem: \[\begin{align} \inf_{A} \sup_{\tilde{\mu}_x,\tilde{\mu}_w,\Sigma_x,\Sigma_w} &Tr\left[ \left( AH-I_n\right) \Sigma_x\left( AH-I_n\right) ^\top+A\Sigma_w A^\top \right] +\left[\left( AH-I_n\right) \tilde{\mu}_x+A\tilde{\mu}_w \right] ^\top \left[\left( AH-I_n\right) \tilde{\mu}_x+A\tilde{\mu}_w \right] \notag \\ s.t.\; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \notag \\ &\left\| \tilde{\mu}_x\right\| ^2+Tr \left[ \Sigma_x+\hat{\Sigma}_x- 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_x^2, \label{2minimax} \\ &\left\| \tilde{\mu}_w\right\| ^2+Tr \left[ \Sigma_w+\hat{\Sigma}_w- 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}}\right) ^{\frac{1}{2}} \right] \leq \rho_w^2. \notag \end{align}\tag{51}\] Notice that for given \(A\), \(\Sigma_x\) and \(\Sigma_w\), the inner maximization problem over \((\tilde{\mu}_x,\tilde{\mu}_w)\) in (51 ) is a homogeneous QCQP problem with two homogeneous constraints, which can be be equivalently reformulated as a convex problem through the SDP relaxation method as described in the following theorem.

4.2 An Optimal Solution to (7 ) and (48 )↩︎

Section 4.1 shows that the optimal value of (7 ) is equal to that of an SDP problem (?? ). However, in practical applications, we often need to obtain the optimal solution of (7 ), specifically the robust linear estimator and the corresponding least favorable distribution. When the saddle point solution does not exist, the robust linear estimator is not the optimal estimator corresponding to its least favorable distribution, and thus it can not be directly calculated.

Notice that problem (7 ) is equivalent to (52 ), while the SDP problem (?? ) is equivalent to (53 ) obtained by exchanging minimization and maximization in problem (52 ). Therefore, if the saddle point solution of (53 ) can be constructed by the primal and dual optimal solutions of (?? ), then this saddle point solution is also an optimal solution to (52 ), and thus the optimal solution of (7 ) can be further obtained.

Therefore, we first consider the Lagrangian function of (?? ) denoted by \[\begin{align} &\mathfrak{L}(Q,S,\Sigma_x,\Sigma_w,V_x,V_w,G_S,G_x,G_w,G_{vx},G_{vw},W,T_x,T_w,\alpha_x,\alpha_w)\\ &\begin{aligned} =&-Tr(Q)-Tr(G_S^\top S)-Tr(G_x^\top \Sigma_{x}) -Tr(G_w^\top \Sigma_w)-Tr(G_{vx}^\top V_x)-Tr(G_{vw}^\top V_w)\\ &-Tr\left[ W^\top \left( \begin{bmatrix} I_n & {\boldsymbol{0}} \\H & I_m \end{bmatrix} \left( \begin{bmatrix} \Sigma_x & {\boldsymbol{0}} \\ {\boldsymbol{0}} & \Sigma_w \end{bmatrix}+S \right) \begin{bmatrix} I_n & H^\top \\ {\boldsymbol{0}} & I_m \end{bmatrix}-\begin{bmatrix} Q & {\boldsymbol{0}} \\ {\boldsymbol{0}} & {\boldsymbol{0}} \end{bmatrix} \right) \right] \\ &-Tr\left(T_x^\top \begin{bmatrix} \hat{\Sigma}^\frac{1}{2}_x \Sigma_x \hat{\Sigma}^\frac{1}{2}_x & V_x \\ V_x & I_n \end{bmatrix} \right) -Tr\left(T_w^\top \begin{bmatrix} \hat{\Sigma}^\frac{1}{2}_w \Sigma_w \hat{\Sigma}^\frac{1}{2}_w & V_w \\ V_w & I_m \end{bmatrix} \right)\\ &+\alpha_x \left[Tr \left(\begin{bmatrix} I_n & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{bmatrix} S \right) +Tr\left( \Sigma_x + \hat{\Sigma}_x - 2 V_x \right) -\rho_x^2 \right]\\ &+\alpha_w \left[ Tr \left(\begin{bmatrix} \boldsymbol{0} & \boldsymbol{0} \\ \boldsymbol{0} &I_m \end{bmatrix} S \right) +Tr\left(\Sigma_w + \hat{\Sigma}_w - 2 V_w\right) -\rho_w^2 \right], \end{aligned} \end{align}\] where the dual variables \(G_x,G_{vx} \in \mathbb{S}_+^n\); \(G_w,G_{vw} \in \mathbb{S}_+^m\); \(G_S,W \in \mathbb{S}_+^{m+n}\); \(T_x \in \mathbb{S}_+^{2n}\); \(T_w \in \mathbb{S}_+^{2m}\) and \(\alpha_x,\alpha_w \in \mathbb{R}_+\). Since (?? ) is a convex optimization problem, its optimal solution must stasify KKT system, i.e., \[\begin{align} &\bigtriangledown \mathfrak{L}_Q=-I_n+W_{11}={\boldsymbol{0}},\tag{57} \\ &\bigtriangledown \mathfrak{L}_S=-G_S-\begin{bmatrix} I_n & H^\top \\ {\boldsymbol{0}} & I_m \end{bmatrix} W \begin{bmatrix} I_n & {\boldsymbol{0}} \\ H & I_m \end{bmatrix}+\alpha_x \begin{bmatrix} I_n & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{bmatrix}+\alpha_w \begin{bmatrix} \boldsymbol{0} & \boldsymbol{0} \\ \boldsymbol{0} &I_m \end{bmatrix}={\boldsymbol{0}}, \tag{58}\\ &\bigtriangledown \mathfrak{L}_{\Sigma_x}=-G_x-\left( W_{11}+W_{12} H+H^\top W_{12}^\top+H^\top W_{22} H\right) - \hat{\Sigma}_x^\frac{1}{2} T^{11}_x \hat{\Sigma}_x^\frac{1}{2}+ \alpha_x I_n={\boldsymbol{0}}, \tag{59}\\ &\bigtriangledown \mathfrak{L}_{\Sigma_w}= -G_w-W_{22}-\hat{\Sigma}_w^\frac{1}{2} T^{11}_w \hat{\Sigma}_w^\frac{1}{2}+\alpha_w I_m={\boldsymbol{0}},\tag{60}\\ &\bigtriangledown \mathfrak{L}_{V_x}= -G_{vx}-T^{12}_x-\left( T^{12}_x\right) ^\top-2\alpha_x I_n={\boldsymbol{0}},\tag{61}\\ &\bigtriangledown \mathfrak{L}_{V_w}= -G_{vw}-T^{12}_w-\left( T^{12}_w\right) ^\top-2\alpha_w I_m={\boldsymbol{0}},\tag{62} \\ &{\boldsymbol{0}} \preceq G_S \perp S \succeq {\boldsymbol{0}}, \tag{63} \\ &{\boldsymbol{0}} \preceq G_x \perp \Sigma_x \succeq {\boldsymbol{0}}, \tag{64} \\ &{\boldsymbol{0}} \preceq G_w \perp \Sigma_w \succeq {\boldsymbol{0}}, \tag{65} \\ &{\boldsymbol{0}} \preceq G_{vx} \perp V_x \succeq {\boldsymbol{0}},\tag{66} \\ &{\boldsymbol{0}} \preceq G_{vw} \perp V_w \succeq {\boldsymbol{0}},\tag{67} \\ &{\boldsymbol{0}} \preceq W \perp \left( \begin{bmatrix} I_n & {\boldsymbol{0}} \\H & I_m \end{bmatrix} \left( \begin{bmatrix} \Sigma_x & {\boldsymbol{0}} \\ {\boldsymbol{0}} & \Sigma_w \end{bmatrix}+S \right) \begin{bmatrix} I_n & H^\top \\ {\boldsymbol{0}} & I_m \end{bmatrix}-\begin{bmatrix} Q & {\boldsymbol{0}} \\ {\boldsymbol{0}} & {\boldsymbol{0}} \end{bmatrix} \right) \succeq {\boldsymbol{0}}, \tag{68}\\ &{\boldsymbol{0}} \preceq T_x \perp \begin{bmatrix} \hat{\Sigma}^\frac{1}{2}_x \Sigma_x \hat{\Sigma}^\frac{1}{2}_x & V_x \\ V_x & I_n \end{bmatrix} \succeq {\boldsymbol{0}}, \tag{69} \\ &{\boldsymbol{0}} \preceq T_w \perp \begin{bmatrix} \hat{\Sigma}^\frac{1}{2}_w \Sigma_w \hat{\Sigma}^\frac{1}{2}_w & V_w \\ V_w & I_m \end{bmatrix} \succeq {\boldsymbol{0}}, \tag{70} \\ &{\boldsymbol{0}} \preceq \alpha_x \perp \left[\rho_x^2-Tr \left(\begin{bmatrix} I_n & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{bmatrix} S \right) -Tr\left( \Sigma_x + \hat{\Sigma}_x - 2 V_x \right) \right] \succeq {\boldsymbol{0}}, \tag{71} \\ &{\boldsymbol{0}} \preceq \alpha_w \perp \left[\rho_w^2-Tr \left(\begin{bmatrix} \boldsymbol{0} & \boldsymbol{0} \\ \boldsymbol{0} &I_m \end{bmatrix} S \right) -Tr\left(\Sigma_w + \hat{\Sigma}_w - 2 V_w\right) \right] \succeq {\boldsymbol{0}}, \tag{72} \end{align} \tag{73}\] where some dual variables are appropriately partitioned as \[W=\begin{bmatrix} W_{11} & W_{12} \\ W_{12}^\top & W_{22} \end{bmatrix}, T_x=\begin{bmatrix} T^{11}_x & T^{12}_x \\ (T^{12}_x)^\top & T^{22}_x \end{bmatrix}, T_w=\begin{bmatrix} T^{11}_w & T^{12}_w \\ (T^{12}_w)^\top & T^{22}_w \end{bmatrix},\] with the first diagonal submatrices \(W_{11}, T^{11}_x \in \mathbb{S}_+^n\), \(T^{11}_w \in \mathbb{S}_+^m\). Then we can formulate the saddle point solution of (53 ) in the following theorem.

For given \(A^*=-W_{12}^*\), \(\Sigma_{x}^*\) and \(\Sigma_{w}^*\), we can obtain a rank-one optimal solution \(S^*=\begin{bmatrix} \tilde{\mu}_x^* \\ \tilde{\mu}_w^* \end{bmatrix} \begin{bmatrix} (\tilde{\mu}_x^*)^\top & (\tilde{\mu}_w^*)^\top \end{bmatrix}\) to the inner maximization problem over \(S\) in (52 ) as outlined in the proof of Theorem 2.5 in [27]. Then \((A^*,\tilde{\mu}^*_x,\tilde{\mu}^*_w,\Sigma^*_x,\Sigma^*_w)\) is an optimal solution to (51 ), which further implies that \((A^*,-(A^*H-I_n)\hat{\mu}_x-A^* \hat{\mu}_w,\tilde{\mu}^*_x+\hat{\mu}_x,\tilde{\mu}^*_w+\hat{\mu}_w,\Sigma^*_x,\Sigma^*_w)\) is an optimal solution to (49 ). That is, \(f^*(y)=A^*y-(A^*H-I_n)\hat{\mu}_x-A^* \hat{\mu}_w\) and \(P^*=\mathcal{N}(\tilde{\mu}^*_x+\hat{\mu}_x,\Sigma^*_x) \times \mathcal{N}(\tilde{\mu}^*_w+\hat{\mu}_w,\Sigma^*_w)\) constitute an optimal solution to (7 ) and (48 ).

5 Simulation↩︎

In this section, we intend to verify the effectiveness of our theory through numerical experiments. All experiments are implemented in MATLAB R2024a on a PC with AMD Ryzen 7 9800X3D processors (4.7GHz) and 64 GB of RAM. In all experiments, the SDP problems are numerically solved by SDPT3 solver through CVX interface [35], and all parameters are set to default values when solving optimization problems.

5.1 The Nonexistence of the Saddle Point↩︎

In the first experiment, we aim to illustrate that the saddle point may not exist in high-dimensional case when the parameter and noise dimensions are fixed to \(n=m=d\). We take the nominal mean vectors to be \(\hat{\mu}_x=\hat{\mu}_w={\boldsymbol{0}}\) and draw the elements of the observation matrix \(H\) independently from the standard Gaussian distribution. The nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\) are constructed as follows: first, we sample the elements of matrices \(Q_x\) and \(Q_w\) independently from the standard Gaussian distribution and denote \(R_x\) and \(R_w\) the orthogonal matrices whose columns are the orthogonal eigenvectors of \(Q_x+Q_x^\top\) and \(Q_w+Q_w^\top\), respectively. Then we define \(\hat{\Sigma}_x=R_x \Lambda_x R_x^\top\) and \(\hat{\Sigma}_w=R_w \Lambda_w R_w^\top\), where \(\Lambda_x\) and \(\Lambda_w\) are diagonal with entries sampled uniformly from [1,5] and [1,2], respectively. Finally, we set the Wasserstein radiu of the parameter distribution to \(\rho_x=3\) and vary the Wasserstein radiu of the noise distribution \(\rho_w\) across the interval [1,10] with a stepsize of 0.5.

For a given \(\rho_w\), we first calculate the optimal solution to (12 ), which is equal to that of the SDP problem (35 ). From the proof of Theorem 2, the optimal solution to (12 ) denoted by \((f^*,P^*)\) is unique due to the positive definiteness of the nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\). Subsequently, the least favourable distribution corresponding to \(f^*\) will be calculated by solving the problem \[\sup_P mse(f^*,P), \label{least}\tag{97}\] where \(f^*=A^* y+b^*\) and \(b^*= (I_n-A^*H)\hat{\mu}_x - A^* \hat{\mu}_w={\boldsymbol{0}}\). Then (97 ) can be parameterized as \[\label{least322} \begin{align} \sup_{\mu_x,\mu_w,\Sigma_x,\Sigma_w} &Tr\left[\left( A^*H-I_n\right) \Sigma_x\left( A^*H-I_n\right) ^\top+A^*\Sigma_w \left( A^*\right) ^\top \right]+\left[ \left( A^*H-I_n\right) \mu_x+A^*\mu_w\right] ^\top \left[ \left( A^*H-I_n\right) \mu_x+A^*\mu_w \right] \\ s.t.\; & \Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \\ &\left\| \mu_x \right\| ^2+Tr \left[ \Sigma_x+\hat{\Sigma}_x- 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}}\right) ^{\frac{1}{2}}\right] \leq \rho_x^2, \\ &\left\| \mu_w\right\| ^2+Tr \left[ \Sigma_w+\hat{\Sigma}_w- 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}}\right) ^{\frac{1}{2}}\right] \leq \rho_w^2. \end{align}\tag{98}\] For given \(\Sigma_x\) and \(\Sigma_w\), the maximization problem over \((\mu_x,\mu_w)\) is a QCQP in which the two constraint functions and the objective function are all homogeneous quadratic. Then the SDP relaxation of problem (98 ) is \[\begin{align} \begin{array}{cl} \sup \limits_{S,\Sigma_x,\Sigma_w} &Tr\left[\left(A^*H-I_n\right)\Sigma_x\left(A^*H-I_n\right)^\top +A^*\Sigma_w \left( A^*\right) ^\top\right]+ Tr \left(\begin{bmatrix} (A^*H-I_n)^\top \\ (A^*)^\top \end{bmatrix} \begin{bmatrix} A^*H-I_n & A^* \end{bmatrix} S \right) \\ s.t. &\Sigma_x \succeq{\boldsymbol{0}}, \quad \Sigma_w \succeq {\boldsymbol{0}}, \\ &Tr \left(\begin{bmatrix} I_n & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{bmatrix} S \right) +Tr \left[ \Sigma_x+\hat{\Sigma}_x- 2 \left( \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}}\right) ^{\frac{1}{2}}\right] \leq \rho_x^2, \\ &Tr \left(\begin{bmatrix} \boldsymbol{0} & \boldsymbol{0} \\ \boldsymbol{0} &I_m \end{bmatrix} S \right) +Tr \left[ \Sigma_w+\hat{\Sigma}_w- 2 \left( \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}}\right) ^{\frac{1}{2}}\right] \leq \rho_w^2. \end{array} \label{least32SDP1} \end{align}\tag{99}\] According to Theorem 2.5 in [27], problem (99 ) has a rank-one optimal solution, which implies that the SDP relaxation is tight. Thus, combined with Proposition 2 in [34], introducing auxiliary variables \(V_x \in \mathbb{S}_+^n\) and \(V_w \in \mathbb{S}_+^m\), problem (98 ) is equivalent to the following SDP problem \[\begin{align} \label{least32SDP} \begin{array}{cl} \sup \limits_{S,\Sigma_x,\Sigma_w,V_x,V_w} &Tr\left[\left(A^*H-I_n\right)\Sigma_x\left(A^*H-I_n\right)^\top +A^*\Sigma_w \left( A^*\right) ^\top\right]+ Tr \left(\begin{bmatrix} (A^*H-I_n)^\top \\ (A^*)^\top \end{bmatrix} \begin{bmatrix} A^*H-I_n & A^* \end{bmatrix} S \right) \\ s.t.& S \succeq{\boldsymbol{0}}, \;\Sigma_x \succeq{\boldsymbol{0}}, \;\Sigma_w \succeq{\boldsymbol{0}}, \;V_x \succeq{\boldsymbol{0}}, \;V_w \succeq{\boldsymbol{0}},\\ &\left[\begin{array} {cc} \hat{\Sigma}_x^{\frac{1}{2}} \Sigma_x \hat{\Sigma}_x^{\frac{1}{2}} & V_x \\ V_x &I_n \end{array} \right]\succeq {\boldsymbol{0}}, \quad \left[\begin{array} {cc} \hat{\Sigma}_w^{\frac{1}{2}} \Sigma_w \hat{\Sigma}_w^{\frac{1}{2}} & V_w \\ V_w &I_m \end{array} \right]\succeq {\boldsymbol{0}},\\ & Tr \left(\begin{bmatrix} I_n & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0} \end{bmatrix} S \right) +Tr\left( \Sigma_x + \hat{\Sigma}_x - 2 V_x \right) \leq \rho_x^2 ,\\ &Tr \left(\begin{bmatrix} \boldsymbol{0} & \boldsymbol{0} \\ \boldsymbol{0} &I_m \end{bmatrix} S \right) +Tr\left(\Sigma_w + \hat{\Sigma}_w - 2 V_w\right) \leq \rho_w^2. \end{array} \end{align}\tag{100}\] We denote the least favorable distribution corresponding to \(f^*\) obtained by (100 ) as \(\tilde{P}\), and then compare \(mse(f^*,P^*)\) and \(mse(f^*,\tilde{P})\) which are the optimal values of (35 ) and (100 ), respectively. If \(mse(f^*,P^*) < mse(f^*,\tilde{P})\), it can be inferred that \(P^*\) is not the least favorable distribution corresponding to \(f^*\), which indicates that the saddle point solution of (4 ) does not exist.

In Figure 5.1, \(mse(f^*,P^*)\), \(mse(f^*,\tilde{P})\) and the optimal value of (?? ), which is also the optimal value of the minimax problem (48 ), are plotted versus the quantile of the radiu \(\rho_w\) for \(d=20\). It is evident that the saddle point exists when \(\rho_w\) is sufficiently small. However, as \(\rho_w\) gradually increases, \(mse(f^*,P^*)\) is smaller than \(mse(f^*,\tilde{P})\), which indicates that \(P^*\) is no longer the least favorable distribution corresponding to \(f^*\). Consequently, the saddle point solution of (4 ) may not exist. Moreover, irrespective of the existence of the saddle point, the optimal value of (48 ) is always greater than \(mse(f^*,P^*)\) but less than \(mse(f^*,\tilde{P})\), which is consistent with the theoretical result. Furthermore, the optimal value of (48 ) and \(mse(f^*,P^*)\) provide upper and lower bounds on the optimal value of the original problem (4 ), respectively.

5.2 The Validity of Sufficient Condition↩︎

In the second experiment, we aim to verify the validity of the sufficient condition by comparing the lower bound on the existence of the saddle point determined by the sufficient condition in Theorem 3 with the actual bound determined by the necessary and sufficient condition in Theorem 2. The nominal mean vectors \(\hat{\mu}_x\) and \(\hat{\mu}_w\), nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\), and observation matrix \(H\) are generated in the same way as in the first experiment. We vary the Wasserstein radiu of the parameter distribution \(\rho_x\) across the interval [1,10] with a stepsize of 0.1 and calculate the lower bound determined by the sufficient condition in Theorem 3 through \[\rho_w^L=\frac{\lambda_{\min}^{\frac{1}{2}}\left( \hat{\Sigma}_x\right) \lambda_{\min}^{\frac{1}{2}}\left(\hat{\Sigma}_w\right) }{\rho_x}.\] In addition, for a given \(\rho_x\), to determine the actual bound of the existence of the saddle point, we vary the Wasserstein radiu of the noise distribution \(\rho_w\) across the interval [1,10] with a stepsize of 0.02. For given \(\rho_x\) and \(\rho_w\), we slove (35 ) and verify whether the matrix in Theorem 2 is negative semidefinite. If the largest eigenvalue of the matrix is larger than 0, we assert that the saddle point solution of (4 ) does not exist by Theorem 2 and record the smallest \(\rho_w^*\) that satisfies this condition for each \(\rho_x\). Then the actual bound of the existence of the saddle point can be given by \(\rho_w^A=\rho_w^*-0.02\).

In Figure 5.2, the lower bound calculated by the sufficient condition \(\rho_w^L\) and the actual bound obtained by the traversal algorithm \(\rho_w^A\) are plotted versus the quantile of the radiu \(\rho_x\) for \(d=20\). It is evident that the lower bound is consistently inferior to the actual bound. As \(\rho_x\) increases, the discrepancy between the two bounds decreases.

5.3 The Robustness of the Robust Linear Estimator↩︎

In the third experiment, we aim to verify the robustness of the linear estimator obtained from the upper bound problem (48 ) in Section 4. In this experiment, we assume that the parameter and noise dimensions are equal, denoted by \(n=m=d\). Furthermore, we take the observation matrix to be \(H=I_d\). Without loss of generality, the true mean vectors and the nominal mean vectors are all set to be zeros, i.e., \(\mu_x=\mu_w=\hat{\mu}_x=\hat{\mu}_w={\boldsymbol{0}}\). The experiment comprises 3000 simulation runs. In each run, the true covariance matrices \(\Sigma_x\) and \(\Sigma_w\) is randomly generated in the same way as in the first experiment. Then the nominal covariance matrices \(\hat{\Sigma}_x\) and \(\hat{\Sigma}_w\) are defined as the sample covariance matrices corresponding to 100 independent samples from \(\mathcal{N}({\boldsymbol{0}},\Sigma_x)\) and \(\mathcal{N}({\boldsymbol{0}},\Sigma_w)\). Finally, we design the Wasserstein radii of the uncertainty sets \(\rho_x\) and \(\rho_w\). The simulation is repeated 1,000 times, with 100 samples drawn from the true distributions on each occasion. The Wasserstein distances between the sample covariance matrices and the true covariance matrices are then calculated and recorded. The Wasserstein distances obtained from the 1,000 simulations are sorted in ascending order, and the 0.95-quantile is taken as the Wasserstein radii \(\rho_x\) and \(\rho_w\) to ensure that the sample covariance matrices can be included in the uncertainty sets in most cases.

In this framework, two distributions are considered: the true distribution \(P=\mathcal{N}({\boldsymbol{0}},\Sigma_x) \times \mathcal{N}({\boldsymbol{0}},\Sigma_w)\) and the nominal distribution \(\hat{P}=\mathcal{N}({\boldsymbol{0}},\hat{\Sigma}_x) \times \mathcal{N}({\boldsymbol{0}},\hat{\Sigma}_w)\). This allows us to obtain three estimators: the optimal estimator of the nominal distribution \(f^*(\hat{P})\), the optimal estimator of the true distribution \(f^*(P)\), and the robust linear estimator \(\tilde{f}^*\) which is proposed in Section 4. To verify the robustness of the estimator \(\tilde{f}^*\), we can compare the relative mean square errors (RMSE) \(mse(\tilde{f}^*,P)-mse(f^*(P),P)\) and \(mse(f^*(\hat{P}),P)-mse(f^*(P),P)\). Figure 5.3 presents the frequency histograms of the relative mean square errors of the robust linear estimator and the nominal optimal estimator in 3,000 repeated experiments for \(d=10\) and \(d=20\), respectively. It is evident that the relative mean square error of the robust linear estimator is superior to that of the optimal estimator of the nominal distribution, and this advantage becomes increasingly apparent as the dimensions of the parameter and noise increase.

6 Conclusion↩︎

In this paper, we consider a robust estimation problem in the linear measurement model with additive noise, where the parameter and noise are constrained by bounded Wasserstein-distance balls, respectively. This robust estimation problem can be formulated as an infinite-dimensional nonconvex minimax problem whose saddle point may not exist. By transforming the existence of its saddle point to that in a finite-dimensional minimax problem, we provide a verifiable necessary and sufficient condition and a simplified sufficient condition. When a saddle point exists, the original infinite-dimensional minimax problem reduces to a SDP problem. Conversely, when the saddle point is absent, the problem becomes intractable. This fact motivates us to consider an upper-bound problem where the estimator is restricted to be linear. By demonstrating the tightness of the SDP relaxation for the upper-bound problem, we prove that its optimal value coincides with that of a SDP problem. Furthermore, the optimal solution of this upper-bound problem is constructed and yields a robust linear estimator.

7 Acknowledgements↩︎

The authors would like to thank Prof. Zhi-Quan (Tom) Luo from The Chinese University of Hong Kong, Shenzhen, for the helpful discussions on this work.

References↩︎