1 Introduction↩︎

Deception is a mechanism defenders often use in adversarial environments, particularly as a tool to mislead their opponents into adopting oblivious policies. For example, adversarial entities can employ deception in cyber-physical settings as a means to render their attacks unnoticed [1], [2], but also, in other contexts such as robotics [3], autonomous vehicles [4], and warfare [5]. Despite the heterogeneity of these applications, a common theme among them is that of information asymmetry; that is, to be able to deceive the opponent, one must have an information advantage against them, and the opponent should usually be oblivious to this advantage.

From a control-theoretic perspective, deception in dynamical systems can be formulated within a broader optimal control and game-theoretic framework. In such settings, one agent strategically perturbs information or system inputs in order to influence the policy computed or learned by one or more other agents [6]. This leads naturally to bilevel optimal control problems in which one decision maker anticipates and shapes the optimization or learning process of another.

In this paper, we consider deception in the context of data-driven linear-quadratic control. This is a recently emerged field in control theory that aims to design optimal controllers for systems whose dynamics are either uncertain or completely unknown. Its underlying idea is to use state and/or input histories of the system at hand to synthesize the desired control policy, instead of solving model-based equations that require strong prior empirical knowledge [7], [8]. Yet, a major backdoor in data-driven control is its sensitivity to perturbations often present in the measured data. Such perturbations can be benign when they conform to specific random patterns or are sufficiently small [9], [10]. However, this is not the case when they are specifically crafted by an opponent with an information advantage as a means for deception. Crafting such deception is the focus of this paper.

More specifically, we consider a setting where an adversary wants to compute the optimal linear quadratic attack against a dynamical system. However, being an outsider, the adversary has no prior knowledge regarding the system’s dynamics. It thus needs to either perform a direct data-driven method to learn the optimal attack or employ system identification and then learn the optimal attack indirectly. On the other end, the defender of the system is aware of the adversary’s malicious objective and is motivated to perform deception to mislead them towards learning a more benign attack. To that end, the defender exploits its information advantage with respect to the system’s dynamics and injects deceptive feedback into the system to distort the data the adversary is using. A key characteristic of such feedback is that it has the same effect on the attack that the adversary will ultimately learn, irrespective of the adversary’s actual learning algorithm.

To obtain the defender’s deceptive input, we cast its objective as a constrained optimization problem balancing two objectives: i) the desire to steer the adversary’s learned policy towards an a priori chosen benign gain; and ii) the desire to use as small of deceptive feedback as possible to maintain the nominal closed-loop characteristics and stability. We mathematically specify the conditions under which the defender’s optimization is well-defined and characterize its solution as the root of a coupled algebraic Riccati equation and a Lyapunov equation. Subsequently, since these coupled equations are difficult to analytically solve, we tackle them numerically with a successive over-relaxation iterative algorithm. In addition, we prove this algorithm’s convergence under certain conditions. We finally note that the proposed deception design also applies to the easier problem of deceiving minimizing data-driven linear-quadratic regulators.

Related work↩︎

Despite data-driven control being a relatively young field, relevant studies regarding its deception have recently appeared in the literature. For example, related to the philosophy of this paper is the concept of deception in data-driven Nash equilibrium seeking [11]. It considers a setting where an agent engaged in a multi-player game has an information advantage and uses this advantage to design a deceptive input that steers the other agents’ policies toward a deceptive equilibrium. Nevertheless, the setting considered in [11] is tailored to repeated static games.

The concept of deceiving data-driven methods is also often related to poisoning such methods, owing to the data corruption that implicitly or explicitly takes place as the means for deception. This idea of poisoning has attracted much interest in the machine learning community [12]–[16] and a few studies have also investigated it from a systems and control theoretic perspective. For instance, relevant examples include poisoning virtual reference feedback controllers [17], [18] and data predictive control [19]. Nevertheless, most of these techniques revolve around distantly related setups with discrete-time systems, different objectives, and sometimes without theoretical convergence guarantees. Relevant research initiatives have also focused on increasing resilience against poisoning attacks [20]–[23]. However, the problem of explicitly deceiving a learning-based optimal control algorithm, so that it converges to a prespecified policy, has not been studied.

Moving target defense is another mechanism used to deal with adversaries [24]–[26]. Its main idea is that to prevent an adversary from stealthily attacking a dynamical system, it is in the interest of the defender to constantly alter the system’s dynamics to reveal or hinder the attack. However, moving target defense does not exploit the defender’s information advantage to explicitly deceive the adversary’s policy. This is the case in the deception method we develop, where one can prescribe how the defender’s deceptive input will affect the learned adversarial policy and hence steer it towards an a priori chosen benign target. While worst-case min-max approaches as in [27]–[29] aim to protect against fully-informed adversaries, we exploit the defender’s advantage to strategically control the adversary’s policy.

A preliminary version of this paper appeared in [30], which studied the problem of deception against minimizing data-driven linear quadratic regulators. However, [30] only provided simulation results to support its algorithm’s ability in extracting the optimal deception policy. On the other hand, hereon, we adopt a stricter approach and theoretically prove the algorithm’s convergence properties. In addition, we shed light on the problem of deceiving maximizing regulators. This is a much more challenging problem owing to the associated value function possibly becoming unbounded and thus undefined [27].

The contributions of the paper are summarized as follows.

1. We consider deception in a setting where a defender, who has knowledge of the system’s matrices, injects an additional state-feedback input to strategically influence the learning dynamics of a data-driven attacker, with the goal of steering it toward a suboptimal attack policy. Unlike classical LQR game formulations [27]–[29], where all players have model access and the equilibrium is uniquely determined by the problem parameters, our setting exploits the fact that the adversary must learn its policy from data, opening the door to shaping what it converges to. This also distinguishes our approach from moving target defense [24]–[26], which does not explicitly target the adversary’s learned policy, and from deception in Nash equilibrium seeking [11], which considers static game settings.

2. We characterize the optimal deception gain as the solution of a coupled algebraic Riccati and Lyapunov equation, and develop a block successive over-relaxation algorithm with provable convergence guarantees. This provides theoretical foundations for this specific problem that are absent from related data poisoning approaches, such as those on virtual reference feedback controllers [17], [18] and data predictive control [19], as well as from the preliminary version of this paper [30].

3. We analyze robustness to uncertainty in the adversary’s cost matrices and validate the framework on an aircraft model, demonstrating substantial attack suppression.

Structure: The remainder of the paper is organized as follows. Section 2 presents preliminaries, Section 3 formulates the deception problem, and Section 4 outlines the proposed solution within an algorithmic framework. Section 5 establishes convergence guarantees for the algorithm, while Section 6 discusses robustness to parametric uncertainty and relaxed assumptions. Finally, Section 7 presents simulation results, and Section 8 concludes the paper.

2 Preliminaries↩︎

Notation: We denote as \(\mathbb{R}\) the set of real numbers, and as \(\mathbb{N}\) the set of natural numbers including zero. The vector \(e_i\in\mathbb{R}^n\) denotes a unit vector with a unity located in the \(i\)-th row. For any matrix \(X\), \(\left\lVert X\right\rVert_{\mathrm{F}}\) denotes its Frobenius norm, whereas \(\mathrm{Ran}(X)\) denotes its range (i.e., its column space). In addition, \([X]_{ij}\) denotes the entry in its \(i\)-th row and \(j\)-th column. Moreover, we follow the matrix differentiation convention \(\left[\frac{\mathrm{d}}{\mathrm{d}X}\right]_{ij}=\frac{\mathrm{d}}{\mathrm{d}[X]_{ij}}\). For a square matrix \(X\), \(\alpha(X)\) denotes its spectral abscissa, and \(\textrm{tr}(X)\) its trace. For a symmetric matrix \(X\), \(\lambda_{\mathrm{min}}(X)\) and \(\lambda_{\mathrm{max}}(X)\) denote its minimum and maximum eigenvalue, respectively. We denote \(X\succ0\) (respectively, \(X\succeq0)\) if \(X\) is positive definite (respectively, positive semidefinite). We denote as \(\textrm{vec}(X)\) the vectorization of \(X\). We use \(I_n\) to denote the identity matrix of order \(n\) (we omit the subscript when the order is obvious). We denote the Kronecker delta function as \(\delta_{i,j}\), so that \(\delta_{i,j}=1\) if \(i=j\), and \(\delta_{i,j}=0\) else.

3 Problem Formulation↩︎

3.1 Closed-Loop System↩︎

Consider the continuous-time, closed-loop system for all \(t\geq0\): \[\label{eq:sys} \dot{x}(t)=Ax(t),\tag{1}\] where \(x(t)\in\mathbb{R}^n\) is the state of the system, and \(A\in\mathbb{R}^{n\times n}\) is the closed-loop state matrix. By closed loop, we mean that \(A=A_o+B_uF\), where \(A_o\in\mathbb{R}^{n\times n}\) is the open-loop state matrix, \(B_u\in\mathbb{R}^{n\times m_u}\) is the input matrix, and \(F\in\mathbb{R}^{m_u\times n}\) is a stabilizing gain such that \(A\) is Hurwitz. For example, \(F\) could be a gain obtained from a pole-placement procedure that moves the eigenvalues of \(A\) to the open left-half plane. For such a gain to be feasible, \((A, B_u)\) must also be controllable.

3.2 The Adversary’s Model and Objectives↩︎

We assume that an adversary can launch an adversarial attack on system 1 . In the presence of such an attack, system 1 would take the form \[\label{eq:sysa} \dot{x}(t)=Ax(t)+B_aa(t),\tag{2}\] with \(B_a\in\mathbb{R}^{n\times m_a}\) being the adversary’s input matrix, and \(a(t)\in\mathbb{R}^{m_a}\) being the attack. The adversary’s purpose is to then design the attack to balance two objectives: i) disturbing the stabilization of 2 ; and ii) moderating the attack’s magnitude. We capture this dual objective with the maximizing linear-quadratic control problem: \[\begin{align} \nonumber \max_{K\in\mathbb{R}^{ m_a\times n}} ~&\int_0^\infty(x^\mathrm{T}(\tau)Qx(\tau)-a^\mathrm{T}(\tau)Ra(\tau))\mathrm{d}\tau,\\ \textrm{s.t.} \qquad &\dot{x}(t)=Ax(t)+B_aa(t),\label{eq:Jnom}\\ & a(t)=Kx(t),\nonumber \end{align}\tag{3}\] where \(Q,~R\succ0\) are weighting matrices and \(K\) is the adversary’s attack gain that is to be optimized. Provided that \(\lambda_{\textrm{min}}(R)\) is above a threshold, it then follows from standard linear systems theory [27] that the solution to this problem is given by \[\label{eq:K} K^\star=R^{-1}B_a^\mathrm{T}P,\tag{4}\] where \(P\in\mathbb{S}_+\) is the minimal positive definite solution of the algebraic Riccati equation (ARE) \[\label{eq:are} A^\mathrm{T}P+PA+Q+PB_aR^{-1}B_a^\mathrm{T}P=0.\tag{5}\]

We now present the information structure of the adversary.

The main implication of the adversary’s information structure is that it impedes them from directly solving the ARE 5 , and hence from obtaining the optimal attack gain \(K^\star\) in 4 . In that regard, the adversary is forced to gather measurements from the system over a given window and use them to either solve 3 in a direct data-driven manner, or to perform system identification and then solve 3 .

3.3 The Defender’s Model and Objectives↩︎

We assume that the defender of the system is aware that a potential adversary might be gathering information from the system with the aim of synthesizing the optimal attack gain \(K^\star\) in 4 . In response, the defender may temporarily modify the system’s dynamics to induce the adversary to learn a prespecified gain different from \(K^\star\). Specifically, during this window, the system’s dynamics take the form \[\label{eq:sysua} \dot{x}(t)=Ax(t)+B_uu(t)+B_aa(t),\tag{6}\] where \(u(t)\in\mathbb{R}^{ m_u}\) is the defender’s deceiving input added to spoof the system dynamics and deceive the adversary, forcing them to learn a gain \(\bar{K}\in\mathbb{R}^{m_a\times n}\) that is different from \(K^\star\).

We now present the information structure of the defender.

The fact that the defender does not know the learning algorithm of the adversary, though realistic, makes it difficult to gauge how the deceiving input \(u(t)\) will affect where that algorithm will converge. An exception is if the defender chooses \(u\) as \[u(t)=\Lambda x(t),\] where \(\Lambda\in \mathbb{R}^{ m_u\times n}\) is an appropriately chosen deception gain. Such a strategy simply redefines the system’s internal dynamics, and hence will generate the same effect to the learned adversarial gain irrespective of the underlying learning dynamics. Hence, if the adversary employs a learning-based scheme to estimate the optimal gain \(K^\star\) that solves 3 while under the proposed deceptive measure, the adversary will instead learn the gain that solves \[\begin{align} \nonumber \max_{K\in\mathbb{R}^{m_a\times n}} ~&\int_0^\infty(x^\mathrm{T}(\tau)Qx(\tau)-a^\mathrm{T}(\tau)Ra(\tau))\mathrm{d}\tau,\\ \textrm{s.t.} \qquad &\dot{x}(t)=(A+B_u\Lambda)x(t)+B_aa(t),\label{eq:Jper}\\ &a(t)=Kx(t),\nonumber \end{align}\tag{7}\] where we note that the trajectories in this maximization problem are over the spoofed dynamics, whereas the trajectories in 3 were over the dynamics of the original system. Following again standard linear systems theory, one can show that the optimal solution to 7 is \[\label{eq:KL} K_u(\Lambda)=R^{-1}B_a^\mathrm{T}P_u(\Lambda),\tag{8}\] where \(P_u(\Lambda)\in\mathbb{R}^{n\times n}\in\mathbb{S}_+\) is now the minimal positive definite solution of the “spoofed" ARE: \[\begin{gather} \label{eq:RicL} (A+B_u\Lambda)^\mathrm{T}P_u+P_u(A+B_u\Lambda)+Q\\+P_uB_aR^{-1}B_a^\mathrm{T}P_u=0. \end{gather}\tag{9}\] Figure 1 depicts this deceptive process.

The above equations imply that the defender can predict the adversarial gain \(K_u(\Lambda)\) that the enemy will ultimately learn, particularly by exploiting its information advantage with respect to the system’s dynamics. In addition, the defender can perform this prediction without using any data. Nevertheless, choosing the deception gain \(\Lambda\) to control \(K_u(\Lambda)\) is far from being a trivial task, because the dependence of \(K_u(\Lambda)\) on \(\Lambda\) goes through the minimal solution of the ARE 9 . In that regard, if the defender wants, for example, to suppress the adversarial attack by forcing \(K_u(\Lambda)\) to be very close to zero, then it is not clear how it should select \(\Lambda\) to achieve this suppression optimally. Motivated by this issue, we formalize the problem we investigate in the remainder of the paper as follows.

4 Deception Design↩︎

4.1 Deception Design using a Distance-Based Metric↩︎

One method the defender could use to design its deception gain \(\Lambda\), is so that \(K_u(\Lambda)\) is as close as possible—in the matrix norm sense—to \(\bar{K}\). In other words, the defender could select its gain \(\Lambda\) so that it minimizes \(\left\lVert K_u(\Lambda)-\bar{K}\right\rVert_\mathrm{F}^2\). Under this objective, the optimal deception gain would be derived as the solution of the constrained optimization \[\begin{align} \nonumber &\inf_{\Lambda\in\mathbb{R}^{m_u\times n},P_u\in\mathbb{R}^{n\times n}}J(\Lambda,P_u)=\left\lVert K_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}^2+\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda),\\ &~\quad\textrm{s.t.} \qquad\quad \eqref{eq:KL}, \eqref{eq:RicL}, \label{eq:opt1}\\ &\qquad\qquad\quad~ P_u\in\mathbb{S}_+, \nonumber \end{align}\tag{10}\] where \(\bar{K}\) is as in Problem [pr:1], \(\Gamma\succ0\), and \(\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda)\) is a regulation term. This regulation term is necessary to ensure that the minimum exists, but also to penalize how large the deception gain \(\Lambda\) can be to minimize deviation from nominal closed-loop performance and maintain stability⁵. As an alternative, the distance of the target gain \(\bar{K}\) from the original gain \(K^\star\) should be sufficiently small, assuming \(K^\star\) exists. We provide an explicit – though conservative – condition on the distance \(\left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}\) as well as the regulation weight \(\Gamma\), for ensuring closed-loop stability and the feasibility of 10 , in the following Lemma. The Lemma’s proof, as well as the proofs of all technical results, are postponed to the Appendix.

We note that the condition of Lemma 1, though sufficient, is not necessary for maintaining stability and feasibility. It can often prove to be quite conservative, and merely having a sufficiently large regulation matrix \(\Gamma\) in terms of eigenvalues should suffice both in practice and theory. Moreover, simulation results and an additional analysis in Section 6.2 show that even this condition is often redundant, as for certain reasonable choices of \(\bar{K}\) the regulation matrix \(\Gamma\) can be arbitrarily small; see Section 6.1 for further details.

To proceed, we use the Karush–Kuhn–Tucker conditions to characterize the solution to the optimization problem 10 , which yields the optimal deception policy for the defender.

According to Theorem 1, we can extract the optimal solution to the deception design problem 10 by solving the matrix equations ?? ?? . However, a close inspection of these equations reveals that they are not easy to solve analytically, even in the scalar case. This is mainly due to their coupled and nonlinear nature, arising from the cross-term \(P_uB_u\Lambda\) in ?? , and the cross-terms \(P_u\Pi\) and \(\Lambda\Pi\) in ?? .

A potential solution to the aforementioned bottleneck could be to employ off-the-shelf numerical solvers for nonlinear systems of equations on ?? ?? , combined with standard heuristics that may aid their chances of success. However, not only would such a choice be computationally expensive since the number of unknowns scales quadratically with \(n\) in ?? ?? , but it would also have no theoretical guarantees of success. A more thoughtful numerical approach is thus clearly required.

4.2 Proposed Numerical Solution↩︎

To be able to numerically attack equations ?? ?? in an efficient manner, it is important to first understand their structure. Even though as a whole this system of equations does not seem to fall within any known class of systems, each of its equations – when viewed independently – has a relatively familiar structure. More specifically, equation ?? is an ARE when viewed with \(P_u\) being its sole unknown variable, and such AREs are straightforward to solve for their minimal positive definite solution. In addition, equation ?? is a Lyapunov equation (LE) when viewed with \(\Pi\) being its sole unknown variable, and hence has a unique solution – as long as \(P_u\) is minimal – that we can extract even analytically. Finally, provided that both \(P_u\) and \(\Pi\) are known a priori, ?? is a very simple computation that yields the deception gain \(\Lambda\).

In view of the above, a potential structured numerical method to solve ?? ?? is the following: one can proceed to begin with an arbitrary gain \(\Lambda\), then solve ?? for its minimal positive definite solution \(P_u\) (provided it exists), then solve ?? for its symmetric solution (provided it exists), then update \(\Lambda\) according to ?? , and then repeat this process iteratively until a fixed point is found. Such a fixed point will necessarily be a solution to ?? ?? , and hence qualify for being a stationary point of 10 .

The aforementioned method is known as the Gauss-Seidel method in numerical linear algebra [31]. We present a variation of it in Algorithm 2, known as block successive over-relaxation, wherein the update of the gain \(\Lambda\) at each iteration goes through a low-pass filter, and where the iterates [eq:it1]-[eq:it3] imitate the structure of equations ?? ?? . Unlike the Gauss-Seidel and the Jacobi method (Chapter 7 in [31]), which may diverge depending on the problem’s parameters, successive over-relaxation has provable convergence guarantees as we show in Section 5.

5 Convergence Analysis↩︎

We first presented Algorithm 2 in the preliminary study [30], which designed poisoning actuation attacks against defenders learning the minimizing linear-quadratic control of an unknown system. While simulation results in [30] showcased that Algorithm 2 performs well in a variety of settings, it is difficult to use such empirical results to argue about convergence properties, mainly because fixed-point iteration schemes are notorious for often being divergent. For this reason, here we will adopt a theoretically strict approach, and mathematically prove that Algorithm 2 leads to a stationary point of the deception problem 10 . In addition, we will show that this stationary point, under certain conditions, corresponds to a minimum.

Towards this end, notice that in the optimization problem 10 , the ARE constraint 9 completely dictates the choice of the matrix \(P_u\). That is, once we select a specific gain \(\Lambda\), then we subsequently obtain the matrix \(P_u\) by solving the equality constraint 9 for its minimal positive definite solution. Taking this into consideration, it is possible to define the cost function of 10 uniquely with respect to \(\Lambda\) as \[\label{eq:JL} \tilde{J}(\Lambda)=J(\Lambda,P_u(\Lambda)),\tag{11}\] where, with a slight abuse of notation, \(P_u(\cdot)\) is now a function with unknown analytical form, which maps \(\Lambda\) to the minimal positive definite solution of the ARE \[\begin{gather} \label{eq:PL} (A+B_u\Lambda)^\mathrm{T}P_u(\Lambda)+P_u(\Lambda)(A+B_u\Lambda)\\+Q+P_u(\Lambda)B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda)=0. \end{gather}\tag{12}\] In light of this definition, in what follows we prove the following central result: one can interpret the fixed point iteration presented in Algorithm 2 as a gradient iteration on the cost function \(\tilde{J}(\Lambda)\), scaled by \(\Gamma^{-1}\).

According to the notation and results of Lemma 2, from lines 9-10 of Algorithm 2, we have \[\begin{align} \omega(\Lambda^\mathrm{GS}-\Lambda^i)&=\omega(-\Gamma^{-1}B_u^\mathrm{T}P_u^{i+1}\Pi^{i+1}-\Lambda^i)\\&=\omega(-\Gamma^{-1}B_u^\mathrm{T}P_u(\Lambda^i)\Pi(\Lambda^i)-\Lambda^i)\\&=-\frac{1}{2}\omega\Gamma^{-1}\cdot2(B_u^\mathrm{T}P_u(\Lambda^i)\Pi(\Lambda^i)+\Gamma\Lambda^i)\\&=-\frac{1}{2}\omega\Gamma^{-1}\frac{\mathrm{d}\tilde{J}(\Lambda^i)}{\mathrm{d}\Lambda^i}. \end{align}\] Hence, we can interpret the iterative update of the deception gain \(\Lambda^i\) in line 10 of Algorithm 2 as \[\label{eq:grad95update} \Lambda^{i+1}\leftarrow \Lambda^{i}-\frac{1}{2}\omega\Gamma^{-1}\frac{\mathrm{d}\tilde{J}(\Lambda^i)}{\mathrm{d}\Lambda^i},\tag{13}\] where we recall that \(\tilde{J}(\Lambda)\) is the cost function in 10 when \(P_u\) is viewed implicitly as a function of \(\Lambda\) (see 11 ). This means that Algorithm 2 is a first-order method on the “generalized" cost 11 and should, in principle, converge to a stationary point, which would then correspond to a stationary point of problem 10 . However, a couple of issues must be considered.

The first issue is that the domain of \(\tilde{J}\) is not equal to all of \(\mathbb{R}^{m_u\times n}\). This is because \(\Lambda\) must be such that a solution \(P_u(\Lambda)\in\mathbb{S}_+\) for 12 exists, and this is clearly not the case for all \(\Lambda\) in \(\mathbb{R}^{m_u\times n}\). Accordingly, if Algorithm 2 happens to abruptly exit the domain of \(\tilde{J}\) at some iteration, it will terminate prematurely with an error while attempting to solve [eq:it1]. The second issue is that even if the iterates \(\Lambda^i\) do not violate the domain of \(\tilde{J}\) throughout the execution of Algorithm 2, convergence to a stationary point can be assured only as long as the gradient of \(\tilde{J}\) has a bounded Lipschitz constant [32]. Fortunately, in what follows, we will show that neither of those two issues will become an obstacle to convergence, under certain conditions.

Towards this end, let us define the set \[\label{eq:S0} \mathcal{S}_0=\{\Lambda\in\mathbb{R}^{m_u\times n}~|~\exists P_{u}(\Lambda)\in\mathbb{S}_+~\textrm{and}~\tilde{J}(\Lambda)\le \tilde{J}(0)\},\tag{14}\] where we observe that \(\Lambda^0\), i.e., the starting point of Algorithm 2, lies inside \(\mathcal{S}_0\). The following auxiliary lemma shows that both \(P_{u}(\Lambda)\) and \(\Pi(\Lambda)\) are bounded uniformly in \(\mathcal{S}_0\), and that \(A+B_u\Lambda+B_aK_u(\Lambda)\) remains strictly stable uniformly in \(\mathcal{S}_0\).

The practical interpretation of Lemma 3 is that, by continuity, \(P_u(\Lambda)\in\mathbb{S}_+\) exists not only everywhere in \(\mathcal{S}_0\) but also in a neighborhood around it. Therefore, the negated gradient \(-\frac{\mathrm{d}\tilde{J}(\Lambda)}{\mathrm{d}\Lambda}\) points in a direction towards which \(P_u(\Lambda)\in\mathbb{S}_+\) exists, and hence this direction is in the interior of \(\mathcal{S}_0\). Accordingly, Algorithm 2 will always remain within a set where both equations [eq:it1] and [eq:it2] admit unique solutions and will not terminate prematurely. This effectively deals with one of the two issues raised in the preceding paragraphs.

Next, we focus on the second issue raised in the preceding discussion; that the gradient of \(\tilde{J}\) must admit a global Lipschitz constant on \(\mathcal{S}_0\) for Algorithm 2 to converge to a stationary point of 10 . Generally speaking, such a conclusion cannot be directly obtained by inspection because \(P_u(\Lambda)\) and \(\Pi(\Lambda)\) in ?? are derived as solutions of nonlinear equations, and hence can depend sublinearly on \(\Lambda\). Nevertheless, using the uniform boundedness result asserted in the second part of Lemma 3, we are able to establish such property as follows.

Finally, we combine the previous Lemmas to obtain the main result of this subsection, which characterizes the convergence properties of Algorithm 2. In particular, we show that Algorithm 1 converges to a stationary point of 10 , and that this point is a minimum under a stricter condition on the regulation matrix \(\Gamma\).

6 Discussion↩︎

In this section, we discuss how we can extend the results of the preceding sections to tackle more general cases of deception in linear-quadratic control.

6.1 On the Condition of Lemma 1↩︎

A sufficient condition for guaranteeing the existence of a minimizer to 10 is inequality ?? , requiring that the attack gain \(\bar{K}\) the defender wants to induce be sufficiently close to the original gain \(K^\star\). As stated in the preceding sections, such a condition can be conservative because we derived it without making any a priori assumptions regarding the deceptive target gain \(\bar{K}\). On the other hand, if we consider the possibility that \(\bar{K}\) will be better for the defender in terms of closed-loop performance when compared to the original gain \(K^\star\), then 10 is unnecessary. We illustrate this in the following example, wherein the solution to the ARE 5 that yields the nominal attack \(K^\star\) does not even exist, because \(\lambda_{\textrm{min}}(R)\) is small and hence \(K^\star\) is destabilizing.

The example above showcases that even when ?? does not hold, 10 and its ARE constraint 9 can still be well-defined. The main idea is that it is not that the target \(\bar{K}\) needs to be close to \(K^\star\) per se, as ?? requires; rather, that it needs to be equal to an attack gain that is milder than \(K^\star\), or at least not much worse than \(K^\star\). Accordingly, in these cases, one should select the initialization \(\Lambda^0\) in Algorithm \(1\) to render \(A+B_u\Lambda^0\) sufficiently stable, i.e., with a sufficiently small spectral absissca⁶. In this way, one can guarantee that the first ARE iterate [eq:it1] will admit a well-defined stabilizing solution, and that [eq:it2] will admit a unique solution, too.

We capture the intuition that ?? does not need to hold when \(\bar{K}\) is milder than \(K^\star\) in the technical result below, for the case \(\bar{K} = 0\).

6.2 On the Assumption of Known \(Q\) and \(R\) Matrices↩︎

In Assumption 2, we assumed the defender does not know the learning algorithm used by the adversary, but knows its objective, i.e., its matrices \(Q\) and \(R\). A natural question thus regards how well the deception scheme will work when the actual adversary matrices are different.

To investigate this question, suppose that the actual weighing matrices in the adversary’s objective are equal to \(\hat{Q}\ne Q\) and \(\hat{R}\ne R\). In addition, consider the cost function 10 , but under these actual weighing matrices \(\hat{Q},~\hat{R}\) of the adversary: \[\begin{align} \label{eq:hatL} &\hat{J}(\Lambda)=\left\lVert\hat{K}_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}^2+\mathrm{tr} (\Lambda^\mathrm{T}\Gamma\Lambda), \end{align}\tag{15}\] where \(\hat{K}_u(\Lambda)=\hat{R}^{-1}B_a^\mathrm{T}\hat{P}_u(\Lambda)\) and \[\begin{gather} \label{eq:Phat} (A+B_u\Lambda)^\mathrm{T}\hat{P}_u(\Lambda)+\hat{P}_u(\Lambda)(A+B_u\Lambda)+\hat{Q}\\+\hat{P}_u(\Lambda)B_a\hat{R}^{-1}B_a^\mathrm{T}\hat{P}_u(\Lambda)=0, ~\hat{P}_u(\Lambda)\in\mathbb{S}_+. \end{gather}\tag{16}\] In the following proposition, we show that when \(\bar{K}=0\), the gain the defender derives by optimizing 10 is still efficient in deceiving the adversary towards \(\bar{K}=0\), despite the incorrectly assumed matrices \(Q\) and \(R\) in 10 . In particular, we show that i) in the case the actual matrices of the adversary satisfy \(\hat{Q}\prec Q\) and \(\hat{R}\succ R\), then the optimal value of 10 provides an upper bound for 15 ; and ii) in the general case, the optimal value of 10 provides an approximate upper bound for 15 .

6.3 On the Deception against Minimizing Data-Driven Linear-Quadratic Regulators↩︎

A dual setup in data-driven linear quadratic control is one where a defender wants to learn the solution to the minimization problem \[\begin{align} \nonumber \min_{N\in\mathbb{R}^{m_u\times n}} ~&\int_0^\infty(x^\mathrm{T}(\tau)Qx(\tau)+u^\mathrm{T}(\tau)Mu(\tau))\mathrm{d}\tau,\\ \textrm{s.t.} \qquad &\dot{x}(t)=Ax(t)+B_uu(t),\label{eq:Jper95u}\\ &u(t)=Nx(t),\nonumber \end{align}\tag{17}\] with \(Q,~M\succ0\). Assuming \((A,B_u)\) is stabilizable, then we know the solution to this problem is \[\label{eq:K2} N^\star=-M^{-1}B_u^\mathrm{T}Z,\tag{18}\] where \(Z\succ0\) is the unique positive definite solution of the ARE \[\label{eq:are95u} A^\mathrm{T}Z+ZA+Q-ZB_uR^{-1}B_u^\mathrm{T}Z=0.\tag{19}\] Accordingly, a dual setup in deceiving such a defender would be to inject an adversarial perturbation \(a(t)=Lx(t)\) in the system, forcing the defender to converge to the incorrect linear-quadratic gain \[\label{eq:KL2} N_a(L)=-M^{-1}B_u^\mathrm{T}Z_a(L),\tag{20}\] where \(Z_a(L)\succ0\) is obtained by solving the perturbed ARE \[\begin{gather} \label{eq:are95u2} (A+B_aL)^\mathrm{T}Z_a+Z_a(A+B_aL)\\+Q-Z_aB_uR^{-1}B_u^\mathrm{T}Z_a=0. \end{gather}\tag{21}\] Note that 19 is much easier to analyze than 9 ; as long as \(\mathrm{Ran}(B_a)\subseteq\mathrm{Ran}(B_u)\), i.e., as long as the attacker does not have more input channels than the defender, then the positive definite solution to 21 always exists (see [30]). Therefore, it is straightforward to verify that the dual deception problem \[\begin{align} \nonumber &\inf_{L\in\mathbb{R}^{m_a\times n},Z_a\in\mathbb{R}^{n\times n}}\left\lVert N_a(L)-\bar{N}\right\rVert_{\mathrm{F}}^2+\mathrm{tr}(L^\mathrm{T}\Gamma L),\\ &~\quad\textrm{s.t.} \qquad\quad \eqref{eq:KL2}, \eqref{eq:are95u2}, \label{eq:opt2}\\ &\qquad\qquad\quad~ Z_a\succ0, \nonumber \end{align}\tag{22}\] admits a global minimizer for any choice of \(\Gamma\succ0\) and \(\bar{N}\in\mathbb{R}^{m_u\times n}\). With minor modifications, one may then adjust Algorithm 2 to solve 22 numerically (see [30]). The guarantees of convergence will follow the same line as those of Section 5, but will not be constrained by the condition of Lemma 1 since 22 is well-defined for any \(L\in\mathbb{R}^{m_a\times n}\).

7 Simulations↩︎

We consider a linearized model of the ADMIRE benchmark aircraft, whose open-loop state and input matrices \(A_o, B_u\) are given in [33]. Using these, we define the closed-loop state matrix of the aircraft as \(A=A_o+B_uF\), where the gain \(F\) originates from a linear-quadratic regulation problem with identity weighting matrices.

7.1 Case 1: Deception against a Performance-Degrading Attack↩︎

In the first case, we consider that an adversary has access to the first and the third actuator of the aircraft. Since each actuator corresponds to a column of \(B_u\), this means that \[\small B_a^\textrm{T}=\begin{bmatrix}-0.0062 & -0.0072 & 1.2456 & 2.7172 & -0.7497 \\ -0.0709 & 0.0039 & -10.6058 & -2.4724 & -0.4923 \\\end{bmatrix}.\] The adversary then wants to use these actuators to disturb the stabilization of the aircraft. To that end, it seeks to learn the attack that solves the optimization problem 3 with \(Q=I_5\) and \(R=0.6I_2\).

On the other side, the defender who is aware of the adversary’s objective wants to deceive them to converge as closely as possible to the attack gain \(\bar{K}=0\). To that end, the defender plugs a deceptive gain in the system to corrupt the attacker’s learning data. This gain is obtained by performing Algorithm \(1\) with \(\omega=0.001\), so as to numerically solve the deception optimization 10 with \(\Gamma=10^{-3}I_7\).

The evolutions of the deception gain sequence \(\Lambda^i\) generated by Algorithm 2, as well as of the deception cost 10 , are shown in Figures 3-4. From Figure 3, we notice that Algorithm 2 reaches a stationary point of the deception problem 10 after about 3500 iterations, which validates Theorem 2. In addition, Figure 4 shows that the deception cost 10 is decreasing at each iteration of Algorithm 2. This is in line with the observation made in Lemma 2, regarding the Algorithm’s equivalence to a first-order gradient method. It indicates that, at each iteration \(i\), the gain \(\Lambda^i\) the defender computes becomes more and more efficient in deceiving the adversary.

Table ¿tbl:tab:ratio? additionally shows the ratio between the optimal attack \(K^\star\) the attacker would have learned in the absence of deception, and the attack \(K_u(\Lambda^\star)\) they were forced to learn instead. Clearly, the latter has been suppressed substantially and is thus more benign in disturbing closed-loop performance. Note that \(K_u(\Lambda^\star)\) does not become exactly zero because of the regularization term that penalizes large values of \(\Lambda\) in 10 .

To validate this prediction at the level of the attacker’s learning dynamics, Figure 5 illustrates how the deception gain \(\Lambda^\star\) affects the evolution of two learning algorithms from the literature [34], [35]. One assumes unknown \(A\) and \(B_a\) matrices [35], whereas the other assumes unknown \(A\) and known \(B_a\) [34]. In both cases, we observe the learning process converges to the suboptimal attack \(K_u(\Lambda^\star)\) that the defender had predicted.

To further assess the impact at the level of closed-loop performance, Table 1 reports the resulting state energy \(\mathcal{E}_x = \int_{0}^{\infty} \left\lVert x(t)\right\rVert^2 dt\) under nominal and deceived attacks for different values of \(R\). We observe that, for small values of \(R\), the nominal attack may even destabilize the system (see next subsection), whereas the deceived attack consistently yields finite and smaller energy.

Finally, we showcase the robustness of the derived deception gain when the adversary’s weighing matrices are different than those assumed. Specifically, we take the deception gain we previously derived under \(Q=I_5\) and \(R=0.6I_2\), and show that even if the adversary’s weighing matrices are equal to \(\hat{Q}=5I_6\ne Q\) and \(\hat{R}=2I_2\ne R\), then the learned adversarial attack is still suppressed. We show the ratio between the learned adversarial attack \(\hat{K}_u(\Lambda^\star)\) and the nominal attack \(\hat{K}^\star\) under \(\hat{Q},~\hat{R}\) in Table ¿tbl:tab:ratio2?, validating this conjecture along with the analysis we carried out in Section 6.2.

7.2 Case 2: Deception against a Destabilizing Attack↩︎

In Case 2, we consider the same setup for the adversary, but where \(R=0.1I_2\) in their optimization problem 3 . In this case, the optimal attack \(K^\star\) is destabilizing. To find a deception gain that will force the adversary to learn a more benign attack instead, we re-employ the same scheme as in Case 1 for the defender. However, following the discussion in Section 6, we select \(\Lambda^0\) in Algorithm \(1\) so that it makes the eigenvalues of \(A+B_u\Lambda^0\) sufficiently negative; in particular, equal to \(-100\).

The evolution of the deception gain sequence \(\Lambda^i\) and the deception cost 10 during Algorithm 2 is shown in Figures 6-7. While we still observe a decreasing deception cost and a convergent deception gain, the number of iterations needed to achieve the same level of tolerance increased significantly. This was due to the conservative initialization of \(\Lambda^0\) very far from the optimal deception gain \(\Lambda^\star\). Meanwhile, the eigenvalues of the state matrix in closed-loop with the deceptive attack gain are equal to \(-17.6911, -7.7253, -2.5628 + 0.4642i, -2.5628 - 0.4642i, -1.6568\). This means that the learned optimal attack under deception is not destabilizing, as opposed to the nominal optimal attack \(K^\star\).

8 Conclusion↩︎

We study the problem of deception in the context of data-driven linear-quadratic control. Given an adversary (defender) who wants to learn the optimal linear-quadratic attack (control) for a system using data, we design a deceptive input that optimally corrupts this data and forces the learner to converge to a prespecified suboptimal policy. We characterize the deceptive input as the solution of coupled matrix equations, which we subsequently solve using fixed-point iteration and with convergence guarantees.

Future work includes an extension of the presented setup to the deception of nonlinear data-driven optimal control. While such a problem is significantly more challenging, both computationally and mathematically, one can tackle it using partial differential equations to characterize the optimal control, in lieu of a Riccati equation.

The following lemma presents matrix differentiation formulas from [36]. Note that since [36] follows the convention \(\left[\frac{\mathrm{d}}{\mathrm{d}X}\right]_{ij}=\frac{\mathrm{d}}{\mathrm{d}[X]_{ji}}\), these formulas have been transposed.

The proofs of the main results follow next.

Proof of Lemma 1: Let \(\{\hat{\Lambda}^n,\hat{P}_u^n\}_{n\in\mathbb{N}}\) be a minimizing sequence of 10 , i.e., a sequence that satisfies the constraints of 10 , and for which \(J(\hat{\Lambda}^n,\hat{P}_u^n)\) converges to the infimum of 10 as \(n\rightarrow\infty\). Note that \(\hat{\Lambda}^n\) and \(K_u(\hat{\Lambda}^n)\) must be bounded uniformly; for if they are not, then \(\lim_{n\rightarrow\infty}J(\hat{\Lambda}^n,\hat{P}_u^n)=\infty\) owing to the fact that \(J(\hat{\Lambda}^n,\hat{P}_u^n)\ge\left\lVert K_u(\hat{\Lambda}^n)-\bar{K}\right\rVert_{\mathrm{F}}^2+\lambda_{\textrm{min}}(\Gamma)\left\lVert\hat{\Lambda}^n\right\rVert^2_{\mathrm{F}}\), whereas \(J(0,P)<\infty\), which contradicts optimality. Therefore, one can extract from \(\{\hat{\Lambda}^n,\hat{P}_u^n\}_{n\in\mathbb{N}}\) a subsequence \(\{\bar{\Lambda}^n,\bar{P}_u^n\}_{n\in\mathbb{N}}\) such that \(\lim_{n\rightarrow\infty}\bar{\Lambda}^n=\bar{\Lambda}\in\mathbb{R}^{m_u\times n}\) and \(\lim_{n\rightarrow\infty}K_u(\bar{\Lambda}^n)=\bar{K}_u\in\mathbb{R}^{m_a\times n}\) [Theorem 3.6 in [37]]. Using these limits and ?? , we have \[\begin{align} (A&+B_u\bar{\Lambda}+B_a\bar{K}_u)^\mathrm{T}S+S(A+B_u\bar{\Lambda}+B_a\bar{K}_u)\\=&(A+B_aK^\star)^\mathrm{T}S+S(A+B_aK^\star)+SB_u\bar{\Lambda} +(SB_u\bar{\Lambda} )^\mathrm{T}\\&+SB_a(\bar{K}_u-K^\star)+(SB_a(\bar{K}_u-K^\star))^\mathrm{T}\\=&-2I+SB_u\bar{\Lambda} +(SB_u\bar{\Lambda} )^\mathrm{T}\\&+SB_a(\bar{K}_u-K^\star)+(SB_a(\bar{K}_u-K^\star))^\mathrm{T}. \end{align}\] Based on the Lyapunov theorem and the equality above, \(A+B_u\bar{\Lambda}+B_a\bar{K}_u\) will be strictly stable if \[\begin{gather} \|SB_u\bar{\Lambda} +(SB_u\bar{\Lambda} )^\mathrm{T}+SB_a(\bar{K}_u-K^\star)\\+(SB_a(\bar{K}_u-K^\star))^\mathrm{T}\|_{\mathrm{F}}<2. \end{gather}\] This holds true if \[\label{eq:ineq951f} \left\lVert SB_u\right\rVert_{\mathrm{F}}\left\lVert\bar{\Lambda}\right\rVert_{\mathrm{F}}+\left\lVert SB_a\right\rVert_{\mathrm{F}}\left\lVert\bar{K}_u-K^\star\right\rVert_{\mathrm{F}}<1.\tag{23}\] So we need to obtain a condition that will allow 23 to hold. To that end, note that \(\Lambda=0\) and \(P_u=P\) from 5 are feasible solutions to 10 . As a result, by optimality of the limit of the minimizing sequence, we have: \[\begin{align} &\lim_{n\rightarrow\infty}J(\bar{\Lambda}^n,\bar{P}_u^n)\le J(0,P) \Longrightarrow\\&\lim_{n\rightarrow\infty}\left\lVert K_u(\bar{\Lambda}^n){-}K^\star\right\rVert_{\mathrm{F}}^2{+} \lim_{n\rightarrow\infty}\mathrm{tr}(\bar{\Lambda}^{n\mathrm{T}}\Gamma\bar{\Lambda}^n)\le \left\lVert K^\star{-}\bar{K}\right\rVert_{\mathrm{F}}^2 \\&{\Longrightarrow} \left\lVert\bar{K}_u{-}K^\star\right\rVert_{\mathrm{F}}^2+ \mathrm{tr}(\bar{\Lambda}^{\mathrm{T}}\Gamma\bar{\Lambda})\le \left\lVert K^\star{-}\bar{K}\right\rVert_{\mathrm{F}}^2. \end{align}\] This implies that \[\label{eq:ineq952f} \begin{align} &\left\lVert\bar{K}_u-K^\star\right\rVert_{\mathrm{F}}\le \left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}},\\ &\left\lVert\bar{\Lambda}\right\rVert_{\mathrm{F}}\le \frac{1}{\sqrt{\lambda_{\textrm{min}}(\Gamma)}}\left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}. \end{align}\tag{24}\] Hence, using 24 , 23 will hold by imposing the condition \[\begin{gather} \label{eq:cond95stab95proof} \frac{1}{\sqrt{\lambda_{\textrm{min}}(\Gamma)}}\left\lVert SB_u\right\rVert_{\mathrm{F}}\left\lVert K^\star{-}\bar{K}\right\rVert_{\mathrm{F}}+\left\lVert SB_a\right\rVert_{\mathrm{F}}\left\lVert K^\star{-}\bar{K}\right\rVert_{\mathrm{F}}<1 \\\Longrightarrow \left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}< \frac{\sqrt{\lambda_{\textrm{min}}(\Gamma)}}{\sqrt{\lambda_{\textrm{min}}(\Gamma)}\left\lVert SB_a\right\rVert_{\mathrm{F}}+\left\lVert SB_u\right\rVert_{\mathrm{F}}}. \end{gather}\tag{25}\] Therefore, under 25 , \(A+B_u\bar{\Lambda}+B_a\bar{K}_u\) is strictly stable. Accordingly, the following LE admits a unique symmetric solution \(\bar{P}_u\in\mathbb{R}^{n\times n}\): \[\begin{gather} \label{eq:PulimLE} (A+B_u\bar{\Lambda}+B_a\bar{K}_u)^\mathrm{T}\bar{P}_u+\bar{P}_u(A+B_u\bar{\Lambda}+B_a\bar{K}_u)\\+Q-\bar{K}_u^\mathrm{T}R\bar{K}_u=0. \end{gather}\tag{26}\] Subtracting 26 from 9 for \(\Lambda=\bar{\Lambda}^n\) and \(P_u=\bar{P}_u^n\), and defining \(\tilde{P}_u^n=\bar{P}_u^n-\bar{P}_u\), \(\tilde{\Lambda}_u^n=\bar{\Lambda}^n-\bar{\Lambda}\), \(\tilde{K}_u^n=K_u(\bar{\Lambda}^n)-\bar{K}_u\) gives: \[\begin{align} \nonumber (A+&B_u\bar{\Lambda}^n{+}B_aK_u(\bar{\Lambda}^n))^\mathrm{T}\tilde{P}_u^n+\tilde{P}_u^n(A{+}B_u\bar{\Lambda}^n{+}B_aK_u(\bar{\Lambda}^n))\\\nonumber&+\tilde{\Lambda}^{n\mathrm{T}}B_u^\mathrm{T}\bar{P}_u+\bar{P}_uB_u\tilde{\Lambda}^n+\tilde{K}_u^{n\mathrm{T}}B_a^\mathrm{T}\bar{P}_u+\bar{P}_uB_a\tilde{K}_u^n \\&-{K}_u^{\mathrm{T}}(\bar{\Lambda}^n)R{K}_u(\bar{\Lambda}^n) +\bar{K}_u^\mathrm{T}R\bar{K}_u=0.\label{eq:tildep} \end{align}\tag{27}\] Note that \(A_n:=A+B_u\bar{\Lambda}^n+B_aK_u(\bar{\Lambda}^n)\) must remain strictly stable uniformly and it converges to the strictly stable matrix \(A_{\infty}:=A+B_u\bar{\Lambda}+B_a\bar{K}_u\) as \(n\rightarrow\infty\). In addition, for each \(n\) 27 is an LE for \(\tilde{P}_u^n\), in which the last six terms satisfy: \[\begin{gather} S_n:=\tilde{\Lambda}^{n\mathrm{T}}B_u^\mathrm{T}\bar{P}_u+\bar{P}_uB_u\tilde{\Lambda}^n+\tilde{K}_u^{n\mathrm{T}}B_a^\mathrm{T}\bar{P}_u+\bar{P}_uB_a\tilde{K}_u^n \\-{K}_u^{\mathrm{T}}(\bar{\Lambda}^n)R{K}_u(\bar{\Lambda}^n) +\bar{K}_u^\mathrm{T}R\bar{K}_u\rightarrow0 \end{gather}\] as \(n\rightarrow\infty\) because \(\lim_{n\rightarrow\infty}\bar{\Lambda}^n=\bar{\Lambda}\) and \(\lim_{n\rightarrow\infty}K_u(\bar{\Lambda}^n)=\bar{K}_u\). Therefore, by solving 27 : \[\tilde{P}_u^n=\int_0^{\infty}\mathrm{e}^{A_n^\textrm{T}\tau}S_n\mathrm{e}^{A_n\tau}\textrm{d}\tau\rightarrow\int_0^{\infty}\mathrm{e}^{A_\infty^\textrm{T}\tau}\cdot0\cdot\mathrm{e}^{A_\infty\tau}\textrm{d}\tau=0\] and thus \(\bar{P}_u^n\rightarrow\bar{P}_u\) as \(n\rightarrow\infty\). In addition, note that since \(\bar{P}_u^n\) are positive definite for all \(n\), \(\bar{P}_u\) must be positive semi-definite, and in fact positive definite owing to Theorem 6.23(ii) in [27]. At the same time, it is straightforward to see that \(\bar{K}_u=R^{-1}B_a^\mathrm{T}\bar{P}_u\) since \(\bar{K}_u=\lim_{n\rightarrow\infty}K_u(\bar{\Lambda}^n)=\lim_{n\rightarrow\infty}R^{-1}B_a^\mathrm{T}\bar{P}_u^n=R^{-1}B_a^\mathrm{T}\bar{P}_u\), thus \(\bar{P}_u\) is the stabilizing solution to 9 for \(\Lambda=\bar{\Lambda}\) and hence its minimal one because of Theorem 6.23(vi) in [27], i.e., \(\bar{P}_u\in\mathbb{S}_+\). Thus \((\bar{\Lambda},\bar{P}_u)\) is a feasible solution of 10 that is also minimizing, hence 10 admits a minimizer. The second part follows from the fact that \(A+B_u\bar{\Lambda}+B_a\bar{K}_u\) was proved to be strictly stable.

Proof of Theorem 1: Define the Lagrangian of the optimization 10 as: \[\begin{align} \nonumber \mathcal{L}(\Lambda,P_u,\Pi)=&\left\lVert R^{-1}B_a^\mathrm{T}P_u-\bar{K}\right\rVert_{\mathrm{F}}^2+\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda)\\\nonumber&+\mathrm{tr}\Big(\Pi ((A+B_u\Lambda)^\mathrm{T}P_u+P_u(A+B_u\Lambda)\\&+Q+P_uB_aR^{-1}B_a^\mathrm{T}P_u) \Big),\nonumber \end{align}\] where \(\Pi\in\mathbb{R}^{n\times n}\) denotes the Lagrange multiplier. Note that because the left-hand side of the ARE 9 is symmetric, \(\Pi\) is symmetric as well. Applying the first of the three necessary conditions for optimality, namely \(\frac{\partial \mathcal{L}}{\partial {\Pi}}=0\), we get \[\begin{gather} \frac{\partial}{\partial {\Pi}}\mathrm{tr}\Big(\Pi ((A+B_u\Lambda)^\mathrm{T}P_u+P_u(A+B_u\Lambda)\\ \qquad\qquad+Q+P_uB_aR^{-1}B_a^\mathrm{T}P_u) \Big)=0 \stackrel{\eqref{eq:AX}}{\Longrightarrow} \\ (A+B_u\Lambda)^\mathrm{T}P_u{+}P_u(A+B_u\Lambda){+}Q{+}P_uB_aR^{-1}B_a^\mathrm{T}P_u=0, \end{gather}\] which yields ?? . Next, by applying the second of the three necessary conditions for optimality, namely \(\frac{\partial \mathcal{L}}{\partial {\Lambda}}=0\), and the fact that \(\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda)=\mathrm{tr}(\Gamma\Lambda\Lambda^\mathrm{T})\) we get: \[\begin{gather} \frac{\partial }{\partial {\Lambda}} \left(\mathrm{tr}\Big(\Lambda^\mathrm{T}\Gamma\Lambda{+}\Pi ((A+B_u\Lambda)^\mathrm{T}P_u{+}P_u(A+B_u\Lambda)) \right)=0 \\\stackrel{\eqref{eq:AXBXt},\eqref{eq:AXtB},\eqref{eq:AX}}{\Longrightarrow} 2\Gamma\Lambda+2B_u^\mathrm{T}P_u\Pi =0 \Longrightarrow \Lambda=-\Gamma^{-1}B_u^\mathrm{T}P_u\Pi, \end{gather}\] where we used the fact that \(\Pi\) and \(P_u\) are symmetric, hence we obtain ?? . Finally, applying the third of the three necessary conditions for optimality, namely \(\frac{\partial \mathcal{L}}{\partial {P_u}}=0\), and using the fact that \(\left\lVert R^{-1}B_a^\mathrm{T}P_u-\bar{K}\right\rVert_{\mathrm{F}}^2=\mathrm{tr}((R^{-1}B_a^\mathrm{T}P_u-\bar{K})(R^{-1}B_a^\mathrm{T}P_u-\bar{K})^\mathrm{T})\), we require: \[\begin{gather} \frac{\partial }{\partial {P_u}}\Big(\mathrm{tr}\Big(R^{-1}B_a^\mathrm{T}P_u^2B_aR^{-1}- R^{-1}B_a^\mathrm{T}P_u\bar{K}^\mathrm{T}\\-\bar{K}P_uB_aR^{-1}\Big) +\mathrm{tr}\Big(\Pi ((A+B_u\Lambda)^\mathrm{T}P_u\\+P_u(A+B_u\Lambda)+P_uB_aR^{-1}B_a^\mathrm{T}P_u) \Big)=0. \end{gather}\] Using properties ?? , ?? , ?? , ?? , we obtain \[\begin{gather} \nonumber (A+B_u\Lambda)\Pi+\Pi(A+B_u\Lambda)^\mathrm{T}+\Pi P_uB_aR^{-1}B_a^\mathrm{T}\\+B_aR^{-1}B_a^\mathrm{T}P_u\Pi-\bar{K}^\mathrm{T}R^{-1}B_a^\mathrm{T}-B_aR^{-1}\bar{K}\\+P_uB_aR^{-2}B_a^\mathrm{T}+B_aR^{-2}B_a^\mathrm{T}P_u=0, \end{gather}\] which, after grouping together the terms containing \(\Pi\), is equivalent to ?? .

Proof of Lemma 2: Note that we can write the cost as \(\tilde{J}(\Lambda)=\tilde{J}_1(\Lambda)+\tilde{J}_2(\Lambda)\), where \(\tilde{J}_1(\Lambda)=\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda)\) and \(\tilde{J}_2(\Lambda)=\left\lVert K_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}^2\). In that respect, on the one hand, we have: \[\label{eq:dJ1} \frac{\mathrm{d}\tilde{J}_1(\Lambda)}{\mathrm{d}\Lambda}=\frac{\mathrm{d}}{\mathrm{d}\Lambda}\mathrm{tr}(\Lambda^\mathrm{T}\Gamma\Lambda)=\frac{\mathrm{d}}{\mathrm{d}\Lambda}\mathrm{tr}(\Gamma\Lambda\Lambda^\mathrm{T})\stackrel{\eqref{eq:AXBXt}}{=}2\Gamma\Lambda.\tag{28}\] On the other hand, using the definition of the Frobenius norm we can write \(\tilde{J}_2(\Lambda)=\mathrm{tr}((R^{-1}B_a^\mathrm{T}P_u(\Lambda)-\bar{K})(R^{-1}B_a^\mathrm{T}P_u(\Lambda)-\bar{K})^\mathrm{T})\). In that respect, denoting \(\Lambda_{ij}=[\Lambda]_{ij}\) and using the commutative property of the derivative operator and the trace, we get: \[\begin{align} \nonumber \frac{\mathrm{d}\tilde{J}_2(\Lambda)}{\mathrm{d}\Lambda_{ij}}&=\frac{\mathrm{d}}{\mathrm{d}\Lambda_{ij}}\Big(\mathrm{tr}\Big(R^{-1}B_a^\mathrm{T}P_u^2B_aR^{-1}- R^{-1}B_a^\mathrm{T}P_u\bar{K}^\mathrm{T}\\\nonumber&\quad-\bar{K}P_uB_aR^{-1}+\bar{K}^\mathrm{T}\bar{K}\Big)\\\nonumber&=\mathrm{tr}\Big(R^{-1}B_a^\mathrm{T}P_u\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}B_aR^{-1}- R^{-1}B_a^\mathrm{T}\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}\bar{K}^\mathrm{T}\\\nonumber&\quad+R^{-1}B_a^\mathrm{T}\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}P_uB_aR^{-1}{-}\bar{K}\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}B_aR^{-1}\Big)\\&\nonumber=\mathrm{tr}\Big(\Big(-\bar{K}^\mathrm{T}R^{-1}B_a^\mathrm{T}-B_aR^{-1}\bar{K}\\&\quad+ P_uB_aR^{-2}B_a^\mathrm{T}{+}B_aR^{-2}B_a^\mathrm{T}P_u \Big)\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}\Big),\label{eq:dJ2t} \end{align}\tag{29}\] where we also applied the cyclic property of the trace. Next, note that since 12 is an equality constraint, we can take the total derivative with respect to \(\Lambda_{ij}\) on both sides and obtain: \[\begin{align} 0=&~\frac{\mathrm{d}}{\mathrm{d}\Lambda_{ij}}\Big((A+B_u\Lambda)^\mathrm{T}P_u(\Lambda)+P_u(\Lambda)(A+B_u\Lambda)\\&+Q+P_u(\Lambda)B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda)\Big)\\=&~e_je_i^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda)+(A+B_u\Lambda)^\mathrm{T}\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}+P_u(\Lambda)B_ue_ie_j^\mathrm{T}\\&+\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}(A+B_u\Lambda)+\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda)\\&+P_u(\Lambda)B_aR^{-1}B_a^\mathrm{T}\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}, \end{align}\] where we used the fact that \(\frac{\mathrm{d}\Lambda}{\mathrm{d}\Lambda_{ij}}=e_ie_j^\mathrm{T}\). Grouping together the differential terms yields the LE: \[\label{eq:LE95dPij} \tilde{A}^\mathrm{T}(\Lambda)\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}+\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}\tilde{A}(\Lambda)+\tilde{Q}(\Lambda)=0\tag{30}\] where \[\begin{align} \tag{31} \tilde{A}(\Lambda)&=A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda),\\\tag{32} \tilde{Q}(\Lambda)&=e_je_i^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda)+P_u(\Lambda)B_ue_ie_j^\mathrm{T}. \end{align}\] Note that since \(P_u(\Lambda)\in\mathbb{S}_+\), \(\tilde{A}(\Lambda)\) is Hurwitz. Therefore, the LE 30 admits the following unique symmetric solution: \[\label{eq:dPij} \frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{ij}}=\int_0^\infty \mathrm{e}^{\tilde{A}^\mathrm{T}(\Lambda)\tau}\tilde{Q}(\Lambda)\mathrm{e}^{\tilde{A}(\Lambda)\tau}\mathrm{d}\tau.\tag{33}\] Combining 33 with 29 , and using the trace cyclic property, yields \[\begin{align} \nonumber \frac{\mathrm{d}\tilde{J}_2(\Lambda)}{\mathrm{d}\Lambda_{ij}}&=\mathrm{tr}\Big(\Big(-\bar{K}^\mathrm{T}R^{-1}B_a^\mathrm{T}-B_aR^{-1}\bar{K}+ P_uB_aR^{-2}B_a^\mathrm{T}\\\nonumber&\quad+B_aR^{-2}B_a^\mathrm{T}P_u \Big)\int_0^\infty \mathrm{e}^{\tilde{A}^\mathrm{T}(\Lambda)\tau}\tilde{Q}(\Lambda)\mathrm{e}^{\tilde{A}(\Lambda)\tau}\mathrm{d}\tau\Big)\\\nonumber&=\mathrm{tr}\Big(\tilde{Q}(\Lambda)\int_0^\infty \mathrm{e}^{\tilde{A}(\Lambda)\tau}\Big(-\bar{K}^\mathrm{T}R^{-1}B_a^\mathrm{T}-B_aR^{-1}\bar{K}\\&\quad{+} P_uB_aR^{-2}B_a^\mathrm{T}{+}B_aR^{-2}B_a^\mathrm{T}P_u \Big)\mathrm{e}^{\tilde{A}^\mathrm{T}(\Lambda)\tau}\mathrm{d}\tau\Big).\label{eq:dJ295b} \end{align}\tag{34}\] Note that the integral inside the trace expression above is exactly the solution of the LE ?? . Hence, in light also of the formula 32 for \(\tilde{Q}\), 34 turns into: \[\begin{align} \frac{\mathrm{d}\tilde{J}_2(\Lambda)}{\mathrm{d}\Lambda_{ij}}&=\mathrm{tr}\Big((e_je_i^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda)+P_u(\Lambda)B_ue_ie_j^\mathrm{T})\Pi(\Lambda)\Big)\\&=2e_i^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda)\Pi(\Lambda)e_j=2[B_u^\mathrm{T}P_u(\Lambda)\Pi(\Lambda)]_{ij}. \end{align}\] Therefore \[\label{eq:dJ2} \frac{\mathrm{d}\tilde{J}_2(\Lambda)}{\mathrm{d}\Lambda}=2B_u^\mathrm{T}P_u(\Lambda)\Pi(\Lambda).\tag{35}\] Since we defined \(\tilde{J}(\Lambda)=\tilde{J}_1(\Lambda)+\tilde{J}_2(\Lambda)\), the results follows by 28 and 35 .

Proof of Lemma 3: Based on ?? , we have: \[\begin{align} \nonumber &(A+B_u\Lambda+B_aK_u(\Lambda))^\mathrm{T}S+S(A+B_u\Lambda+B_aK_u(\Lambda))\\&\nonumber=(A{+}B_aK^\star)^\mathrm{T}S+S(A{+}B_aK^\star)+(K_u(\Lambda){-}K^\star)^\mathrm{T}B_a^\mathrm{T}S\\&\nonumber\quad+SB_a(K_u(\Lambda)-K^\star)+\Lambda B_u^\mathrm{T}S+SB_u\Lambda\\\nonumber&=-2I+(K_u(\Lambda)-K^\star)^\mathrm{T}B_a^\mathrm{T}S+SB_a(K_u(\Lambda)-K^\star)\\&\quad+\Lambda^\mathrm{T}B_u^\mathrm{T}S+SB_u\Lambda.\label{eq:bounded1} \end{align}\tag{36}\] However: \[\begin{gather} \label{eq:bounded2} \|(K_u(\Lambda)-K^\star)^\mathrm{T}B_a^\mathrm{T}S+SB_a(K_u(\Lambda)-K^\star)\\+\Lambda^\mathrm{T}B_u^\mathrm{T}S+SB_u\Lambda\|_{\mathrm{F}}\\\le 2\left\lVert SB_a\right\rVert_{\mathrm{F}}\left\lVert K_u(\Lambda)-K^\star\right\rVert_{\mathrm{F}}+2\left\lVert SB_u\right\rVert_{\mathrm{F}}\left\lVert\Lambda\right\rVert_{\mathrm{F}}\\\le 2\left\lVert SB_a\right\rVert_{\mathrm{F}}\left\lVert K_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}+2\left\lVert SB_u\right\rVert_{\mathrm{F}}\left\lVert\Lambda\right\rVert_{\mathrm{F}}\\+2\left\lVert SB_a\right\rVert_{\mathrm{F}}\left\lVert\bar{K}-{K}^\star\right\rVert_{\mathrm{F}}. \end{gather}\tag{37}\] In addition, if \(\Lambda \in\mathcal{S}_0\) then \[\begin{align} \nonumber &\tilde{J}(\Lambda)\le \tilde{J}(0) \\&\nonumber\Longrightarrow\left\lVert K_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}^2+ \mathrm{tr}(\Lambda^{\mathrm{T}}\Gamma\Lambda)\le \left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}^2\\&\Longrightarrow \left\lVert K_u(\Lambda)-\bar{K}\right\rVert_{\mathrm{F}}^2+\lambda_{\textrm{min}}(\Gamma)\left\lVert\Lambda\right\rVert_{\mathrm{F}}^2\le \left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}^2.\label{eq:bounded3} \end{align}\tag{38}\] Combining 37 38 yields \[\begin{gather} \nonumber \|(K_u(\Lambda)-K^\star)^\mathrm{T}B_a^\mathrm{T}S+SB_a(K_u(\Lambda)-K^\star)+\Lambda^\mathrm{T}B_u^\mathrm{T}S+SB_u\Lambda\|_{\mathrm{F}}\\\le 4(\left\lVert SB_a\right\rVert_{\mathrm{F}}+\lambda_{\textrm{min}}^{-1/2}(\Gamma)\left\lVert SB_u\right\rVert_{\mathrm{F}})\left\lVert K^\star-\bar{K}\right\rVert_{\mathrm{F}}. \end{gather}\] Using ?? , we conclude: \[\begin{gather} \label{eq:2-e} \|(K_u(\Lambda)-K^\star)^\mathrm{T}B_a^\mathrm{T}S+SB_a(K_u(\Lambda)-K^\star)\\+\Lambda^\mathrm{T}B_u^\mathrm{T}S+SB_u\Lambda\|_{\mathrm{F}}\le2\epsilon. \end{gather}\tag{39}\] Combining 39 with 36 , we obtain \[\begin{gather} \label{eq:ineq95unif95stab} (A+B_u\Lambda+B_aK_u(\Lambda))^\mathrm{T}S\\+S(A+B_u\Lambda+B_aK_u(\Lambda))\preceq -2(1-\epsilon) I. \end{gather}\tag{40}\] Since \(S\) is constant, this implies that \(A+B_u\Lambda+B_aK_u(\Lambda)\) remains strictly stable uniformly in \(\mathcal{S}_0\) and that the energy \(\int_0^\infty x^\textrm{T}(t)x(t)\textrm{d}t\) is bounded for every \(x_0\) uniformly in \(\mathcal{S}_0\). In light of this and the fact that \(K_u(\Lambda)\) is bounded uniformly in \(\mathcal{S}_0\), it follows that the value matrix \(P_u(\Lambda)\) is also bounded uniformly in \(\mathcal{S}_0\), i.e., \(\textrm{sup}_{\Lambda\in\mathcal{S}_0}\left\lVert P_u(\Lambda)\right\rVert_\mathrm{F}\le \bar{P}\) for some finite \(\bar{P}>0\). Because of this, the cost matrix of the LE ?? , i.e., its last four terms, is uniformly bounded, so \(\Pi(\Lambda)\) also remains bounded uniformly, i.e., \(\textrm{sup}_{\Lambda\in\mathcal{S}_0}\left\lVert\Pi(\Lambda)\right\rVert_\mathrm{F}\le \bar{\Pi}\) for some finite \(\bar{\Pi}>0\).

Proof of Lemma 4: Let \(\Lambda_1,\Lambda_2\in\mathcal{S}_0\). Denote \(\tilde{\Lambda}=\Lambda_1-\Lambda_2\), \(\tilde{P_u}=P_u(\Lambda_1)-P_u(\Lambda_2)\), \(\tilde{\Pi}=\Pi(\Lambda_1)-\Pi(\Lambda_2)\). Then, using the formula ?? : \[\begin{align} \frac{\mathrm{d}\tilde{J}(\Lambda_1)}{d\Lambda}&-\frac{\mathrm{d}\tilde{J}(\Lambda_2)}{d\Lambda}=2\Gamma(\Lambda_1-\Lambda_2)\\&+2B_u^\mathrm{T}(P_u(\Lambda_1)\Pi(\Lambda_1)-P_u(\Lambda_2)\Pi(\Lambda_2))\\&=2\Gamma\tilde{\Lambda}+2B_u^\mathrm{T}(\tilde{P_u}\Pi(\Lambda_1)+P_u(\Lambda_2)\tilde{\Pi}). \end{align}\] Using Lemma 3, this equation implies that: \[\begin{gather} \label{eq:dJ1dJ2} \left\lVert\frac{\mathrm{d}\tilde{J}(\Lambda_1)}{d\Lambda}-\frac{\mathrm{d}\tilde{J}(\Lambda_2)}{d\Lambda}\right\rVert_{\mathrm{F}}\le 2\left\lVert\Gamma\right\rVert_{\mathrm{F}}\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}\\+2\bar{\Pi}\left\lVert B_u^\mathrm{T}\right\rVert_{\mathrm{F}}\left\lVert\tilde{P}_u\right\rVert_{\mathrm{F}}+2\bar{P}\left\lVert B_u^\mathrm{T}\right\rVert_{\mathrm{F}}\left\lVert\tilde{\Pi}\right\rVert_{\mathrm{F}}. \end{gather}\tag{41}\] Hence, to get a Lipschitz continuity result, one needs to relate the errors \(\left\lVert\tilde{P}_u\right\rVert_{\mathrm{F}}\) and \(\left\lVert\tilde{\Pi}\right\rVert_{\mathrm{F}}\) back to \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}\). To that end, recall that \(P_u(\Lambda_1)\) and \(P_u(\Lambda_2)\) are the minimal positive definite solutions of the AREs: \[\begin{align} \nonumber &(A+B_u\Lambda_1)^\mathrm{T}P_u(\Lambda_1)+P_u(\Lambda_1)(A+B_u\Lambda_1)\\&\qquad\qquad\qquad\qquad+Q+P_u(\Lambda_1)B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda_1)=0,\\ &(A+B_u\Lambda_2)^\mathrm{T}P_u(\Lambda_2)+P_u(\Lambda_2)(A+B_u\Lambda_2)\\&\qquad\qquad\qquad\qquad+Q+P_u(\Lambda_2)B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda_2)=0. \end{align}\] Subtracting these equations yields the following quadratic matrix equation for the error \(\tilde{P}_u\): \[\begin{gather} \label{eq:temp95lips1} \left(A+B_u\Lambda_1+B_aK_u(\Lambda_1)-\frac{1}{2}B_aR^{-1}B_a^\mathrm{T}\tilde{P}_u\right)^{\mathrm{T}}\tilde{P}_u\\+\tilde{P}_u\left(A+B_u\Lambda_1+B_aK_u(\Lambda_1)-\frac{1}{2}B_aR^{-1}B_a^\mathrm{T}\tilde{P}_u\right)\\ +P_u(\Lambda_2)B_u\tilde{\Lambda}+\tilde{\Lambda}^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda_2)=0. \end{gather}\tag{42}\] Note that this is an equation that yields more than one solutions. Nevertheless, the solution of interest has the property that \(\tilde{P}_u\rightarrow0\) as \(\tilde{\Lambda}\rightarrow0\). For this solution, since \(A+B_u\Lambda_1+B_aK_u(\Lambda_1)\) is Hurwitz as \(P_1\in\mathbb{S}_+\), it follows that \(\tilde{A}_1(\tilde{P}_u)=A+B_u\Lambda_1+B_aK_u(\Lambda_1)-\frac{1}{2}B_aR^{-1}B_a^\mathrm{T}\tilde{P}_u\) is also Hurwitz for some small \(\epsilon>0\) such that \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}<\epsilon\). Hence, for this solution \(\tilde{P}_u\) and for \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}<\epsilon\), we get from 42 the implicit formula \[\tilde{P}_u=\int_0^\infty \mathrm{e}^{\tilde{A}_1^\mathrm{T}(\tilde{P}_u)\tau}(P_u(\Lambda_2)B_u\tilde{\Lambda}+\tilde{\Lambda}^\mathrm{T}B_u^\mathrm{T}P_u(\Lambda_2)) \mathrm{e}^{\tilde{A}_1(\tilde{P}_u)\tau} \mathrm{d}\tau.\] Therefore, \[\label{eq:temp95lips2} \left\lVert\tilde{P}_u\right\rVert_{\textrm{F}}\le 2\bar{P}\left\lVert B_u\right\rVert_{\textrm{F}}\left\lVert\tilde{\Lambda}\right\rVert_{\textrm{F}}\int_0^\infty \left\lVert\mathrm{e}^{\tilde{A}_1(\tilde{P}_u)\tau}\right\rVert^2_{\textrm{F}}\mathrm{d}\tau.\tag{43}\] Recall that \(\tilde{A}_1(\tilde{P}_u)\rightarrow A+B_u\Lambda_1+B_aK_u(\Lambda_1)\) as \(\tilde{\Lambda}\rightarrow0\), and \(A+B_u\Lambda_1+B_aK_u(\Lambda_1)\) is strictly stable uniformly for all \(\Lambda_1\in\mathcal{S}_0\) owing to Lemma 3. Hence, the real parts of the eigenvalues of \(\tilde{A}_1(\tilde{P}_u)\) remain in the left half plane for some small \(\epsilon>0\) such that \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}<\epsilon\). Hence, for all \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}<\epsilon\) there exists a finite constant \(c_1>0\) such that \(\int_0^\infty \left\lVert\mathrm{e}^{\tilde{A}_1(\tilde{P}_u)\tau}\right\rVert^2_{\mathrm{F}}\mathrm{d}\tau<c_1\). Putting this in 43 we get the following inequality, locally for \(\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}<\epsilon\): \[\label{eq:temp95lips3} \left\lVert\tilde{P}_u\right\rVert_{\mathrm{F}}\le 2c_1\bar{P}\left\lVert B_u\right\rVert_{\mathrm{F}}\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}.\tag{44}\] Subsequently, we proceed to derive a similar bound for \(\tilde{\Pi}\). Note that subtracting ?? for \(\Lambda=\Lambda_1\) and \(\Lambda=\Lambda_2\) yields \[\begin{gather} \nonumber (A+B_u\Lambda_1+B_aK_u(\Lambda_1))\tilde{\Pi}+\tilde{\Pi}(A+B_u\Lambda_1{+}B_aK_u(\Lambda_1))^\mathrm{T}\label{agmkpxhb}\\+\tilde{P_u}B_aR^{-2}B_a^\mathrm{T}+B_aR^{-2}B_a^\mathrm{T}\tilde{P_u}+B_u\tilde{\Lambda}\Pi(\Lambda_2)+\Pi(\Lambda_2)\tilde{\Lambda}^\mathrm{T}B_u^\mathrm{T}\\+B_aR^{-1}B_a^\mathrm{T}\tilde{P}_u\Pi(\Lambda_2)+\Pi(\Lambda_2)\tilde{P}_uB_aR^{-1}B_a^\mathrm{T}=0.\nonumber \end{gather}\tag{45}\] Denote \(\tilde{A}(\Lambda_1)=A+B_u\Lambda_1+B_aK_u(\Lambda_1)\) and \(\tilde{Q}_1=\tilde{P_u}B_aR^{-2}B_a^\mathrm{T}+B_aR^{-2}B_a^\mathrm{T}\tilde{P_u}+B_u\tilde{\Lambda}\Pi(\Lambda_2)+\Pi(\Lambda_2)\tilde{\Lambda}^\mathrm{T}B_u^\mathrm{T}\\+B_aR^{-1}B_a^\mathrm{T}\tilde{P}_u\Pi(\Lambda_2)+\Pi(\Lambda_2)\tilde{P} _uB_aR^{-1}B_a^\mathrm{T}\). Since \(\tilde{A}(\Lambda_1)\) is Hurwitz, we obtain the explicit formula \(\tilde{\Pi}=\int_0^\infty \mathrm{e}^{\tilde{A}(\Lambda_1)\tau}\tilde{Q}_1\mathrm{e}^{\tilde{A}^\mathrm{T}(\Lambda_1)\tau}\mathrm{d}\tau\), and thus \[\label{eq:temp95lipsPi1} \left\lVert\tilde{\Pi}\right\rVert_{\mathrm{F}}\le \left\lVert\tilde{Q}_1\right\rVert_{\mathrm{F}}\int_0^\infty \left\lVert\mathrm{e}^{\tilde{A}(\Lambda_1)\tau}\right\rVert_{\mathrm{F}}^2\mathrm{d}\tau\le c_2\left\lVert\tilde{Q}_1\right\rVert_{\mathrm{F}},\tag{46}\] where \(c_2=\sup_{\Lambda\in\mathcal{S}_0}\int_0^\infty \left\lVert\mathrm{e}^{\tilde{A}_1(\Lambda)\tau}\right\rVert_{\mathrm{F}}^2\mathrm{d}\tau>0\) is finite since \(\tilde{A}(\Lambda)\) is Hurwitz uniformly in \(\mathcal{S}_0\) by Lemma 3. On the other hand, we have \[\begin{gather} \label{eq:temp95lipsPi2} \left\lVert\tilde{Q}_1\right\rVert_{\mathrm{F}}\le 2(\left\lVert B_aR^{-2}B_a^\mathrm{T}\right\rVert_\mathrm{F}+\left\lVert B_aR^{-1}B_a^\mathrm{T}\right\rVert_\mathrm{F}\bar{\Pi})\left\lVert\tilde{P}_u\right\rVert_{\mathrm{F}}\\+2\bar{\Pi}\left\lVert B_u\right\rVert_{\mathrm{F}}\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}. \end{gather}\tag{47}\] Combining 44 , 46 and 47 , we conclude \[\begin{gather} \label{eq:Ptildeineq} \left\lVert\tilde{\Pi}\right\rVert_{\mathrm{F}}\le c_2\Big(4c_1\bar{P}\left\lVert B_u\right\rVert_{\mathrm{F}}\big(\left\lVert B_aR^{-2}B_a^\mathrm{T}\right\rVert_\mathrm{F}\\+\left\lVert B_aR^{-1}B_a^\mathrm{T}\right\rVert_\mathrm{F}\bar{\Pi}\big)+2\bar{\Pi}\left\lVert B_u\right\rVert_{\mathrm{F}}\Big)\left\lVert\tilde{\Lambda}\right\rVert_{\mathrm{F}}. \end{gather}\tag{48}\] Therefore, in view of 41 , 44 and 48 , we conclude that \(\frac{\mathrm{d}\tilde{J}(\Lambda)}{\mathrm{d}\Lambda}\) is locally Lipschitz on \(\mathcal{S}_0\).

Next, note that the set \(\mathcal{S}_0\) is bounded; for if \(\Lambda\in\mathcal{S}_0\) and \(\left\lVert\Lambda\right\rVert_{\mathrm{F}}^2>\lambda_{\textrm{max}}(\Gamma)\left\lVert K^\star-\bar{K}\right\rVert_\mathrm{F}^2\), then \(\tilde{J}(\Lambda)>\tilde{J}(0)\) by definition, and thus \(\Lambda\notin\mathcal{S}_0\), which is contradicting. In addition, the set \(\mathcal{S}_0\) is closed. To prove this, let \(\{\Lambda_n\}_{n\in\mathbb{N}}\in\mathcal{S}_0\) be a sequence such that \(\Lambda_n\rightarrow{\Lambda}_{\infty}\in\mathbb{R}^{m_u\times n}\). Then, based on Lemma 3, the eigenvalues of the matrix sequence \(A+B_u\Lambda+B_aK_u(\Lambda_n)\) remain in the left half plane uniformly on \(n\in\mathbb{N}\) while satisfying ?? . Therefore, the sequence of minimal positive definite solutions \(P_u(\Lambda_n)\) is bounded, and following [37] one can extract a subsequence \(\{\Lambda_{n_k}\}_{k\in\mathbb{N}}\) such that \(P_u(\Lambda_{n_k})\rightarrow P_u^\infty\) and \(K_u(\Lambda_{n_k})\rightarrow R^{-1}B_a^\textrm{T}P_u^{\infty}=K_u^\infty\) for some positive semi-definite \(P_u^\infty\in\mathbb{R}^{n\times n}\). Note that \(A+B_u\Lambda_{\infty}+B_aK_u^\infty\) will be strictly stable, because \(A+B_u\Lambda_{n_k}+B_aK_u(\Lambda_{n_k})\) are strictly stable uniformly in the sense of ?? and converge to \(A+B_u\Lambda_\infty+B_aK_u^\infty\). Thus, plugging \(\Lambda=\Lambda_{n_k}\) in 12 and taking the limit as \(k\rightarrow\infty\), we obtain: \[\begin{gather} \nonumber (A+B_u\Lambda_{\infty})^\mathrm{T}P_u^\infty+P_u^\infty(A+B_u\Lambda_{\infty})\\+Q+P_u^\infty B_aR^{-1}B_a^\mathrm{T}P_u^\infty=0, \end{gather}\] meaning that \(P(\Lambda_\infty)\) exists and is equal to \(P_u^\infty\), and is also in \(\mathbb{S}_+\) since \(A+B_u\Lambda_{\infty}+B_aK_u^\infty\) was proved to be strictly stable. Given this, the continuity of \(\tilde{J}\) implies \(\tilde{J}(0)\ge \tilde{J}(\Lambda_{n_k}) \Longrightarrow \tilde{J}(0)\ge \lim_{k\rightarrow\infty}\tilde{J}(\Lambda_{n_k})= \tilde{J}\left(\lim_{k\rightarrow\infty}\Lambda_{n_k}\right)=\tilde{J}(\Lambda_{\infty}).\) Hence \(\Lambda_{\infty}\) is such that \(\exists P_{u}(\Lambda_{\infty})\in\mathbb{S}_+\) and \(\tilde{J}(0)\ge \tilde{J}(\Lambda_{\infty})\), hence \(\Lambda_{\infty}\in\mathcal{S}_0\), which means \(\mathcal{S}_0\) is closed. Since \(\mathcal{S}_0\) is a finite-dimensional set that is closed and bounded, it is compact. Therefore, since \(\frac{\mathrm{d}\tilde{J}(\Lambda)}{\mathrm{d}\Lambda}\) is locally Lipschitz on \(\mathcal{S}_0\), it is also globally Lipschitz on \(\mathcal{S}_0\).

Proof of Theorem 2: To prove item 1, it suffices to show that \(\Lambda^i\in\mathcal{S}_0\) for all \(i\in\mathbb{N}\), because in this case both the ARE [eq:it1] and the LE [eq:it2] admit the solutions of interest. We will use induction for that purpose.

- For \(i=0\) we have \(\Lambda^i\in\mathcal{S}_0\) by design.

- For a general \(i\in\mathbb{N}\), suppose that \(\Lambda^i\in\mathcal{S}_0\). We want to show that \(\Lambda^{i+1}\in\mathcal{S}_0\). To that end, suppose in seek of contradiction that \(\Lambda^{i+1}\notin\mathcal{S}_0\). Define \(\hat{\Lambda}(a)=a\Lambda^{i+1}+(1-a)\Lambda^{i}\). Then, there must exist some \(a^\star\in(0,1]\) for which the ARE 9 does not admit a strictly stabilizing solution for \(\Lambda=\hat{\Lambda}(a^\star)\), whereas it does for \(\Lambda=\hat{\Lambda}(a)\) and for all \(a\in[0,a^\star)\). For the latter, it follows that \(\tilde{J}(\hat{\Lambda}(a))\le\tilde{J}(\Lambda^i)\) because \(\hat{\Lambda}(a)\) is in the direction of the negation of the gradient and \(\omega<\frac{2}{L}\frac{\lambda_{\mathrm{min}}(\Gamma^{-1})}{\lambda_{\mathrm{max}}^2(\Gamma^{-1})}\)⁷. Therefore, since \(\Lambda^i\in\mathcal{S}_0\) we obtain \(\tilde{J}(\hat{\Lambda}(a))\le\tilde{J}(0)\), and thus \(\hat{\Lambda}(a)\in\mathcal{S}_0\) for all \(a\in[0,a^\star)\). Using Lemma 3 it follows that \(A+B_u\hat{\Lambda}(a)+B_aK_u(\hat{\Lambda}(a))\) remains strictly stable uniformly for all \(a\in[0,a^\star)\). Therefore, following continuity arguments similar to those of Lemma 4, it follows that 9 admits a stabilizing solution also for \(\Lambda=\hat{\Lambda}(a^\star)\), which is contradicting. Hence, \(\Lambda^{i+1}\in\mathcal{S}_0\). This completes the induction and shows equations [eq:it1]-[eq:it3] admit solutions for all \(i\in\mathbb{N}\).

To prove items 2-3, note that they follow from i) standard arguments regarding the convergence of gradient descent and the selection of the step size \(\omega\) [32]; ii) the fact that the iteration of Algorithm 1 on \(\Lambda^i\) is equivalent to 13 ; and iii) the fact that the sequence of iterates \(\Lambda^i\) always remains inside \(\mathcal{S}_0\) for all \(i\in\mathbb{N}\).

To prove item 4, using Lemma 2 we have \[\begin{align} \nonumber \frac{1}{2}\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{ij}\mathrm{d}\Lambda_{kl}}&=\frac{1}{2}\frac{\mathrm{d}}{\mathrm{d}\Lambda_{kl}}\left[\frac{\mathrm{d}\tilde{J}(\Lambda)}{\mathrm{d}\Lambda}\right]_{ij} \\\nonumber&=\frac{\mathrm{d}}{\mathrm{d}\Lambda_{kl}}e_i^\textrm{T}(\Gamma\Lambda+B_u^\mathrm{T}P_u(\Lambda)\Pi(\Lambda))e_j\\\nonumber&=e_i^\textrm{T}\Gamma\frac{\mathrm{d}\Lambda}{\mathrm{d}\Lambda_{kl}}e_j+e_i^\mathrm{T}B_u^\mathrm{T}\Big(\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\Pi(\Lambda)\\&\qquad\qquad\qquad\qquad+P_u(\Lambda)\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\Big)e_j.\label{eq:hessian1} \end{align}\tag{49}\] In addition, \(e_i^\textrm{T}\Gamma\frac{\mathrm{d}\Lambda}{\mathrm{d}\Lambda_{kl}}e_j=e_i^\textrm{T}\Gamma e_ke_l^\textrm{T}e_j=[\Gamma]_{ik}\delta_{l,j}\). Thus, 49 becomes \[\begin{gather} \label{eq:hessian2} \frac{1}{2}\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{ij}\mathrm{d}\Lambda_{kl}}=[\Gamma]_{ik}\delta_{l,j}\\+e_i^\mathrm{T}B_u^\mathrm{T}\Big(\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\Pi(\Lambda)+P_u(\Lambda)\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\Big)e_j. \end{gather}\tag{50}\] Using the bounds of Lemma 3, which are independent of \(\Gamma\), we get from 50 the inequality: \[\begin{gather} \label{eq:hessian3} \frac{1}{2}\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{ij}\mathrm{d}\Lambda_{kl}}\ge[\Gamma]_{ik}\delta_{l,j} \\-\left\lVert B_u^\mathrm{T}\right\rVert_{\mathrm{F}}\Big(\bar{\Pi}\left\lVert\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}+\bar{P}\left\lVert\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}\Big). \end{gather}\tag{51}\]

The equation above implies the possibility that if \(\Gamma\) is sufficiently large in terms of eigenvalues, then \(\tilde{J}\) is strictly convex on \(\mathcal{S}_0\). However, it first needs to be shown that \(\left\lVert\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}\) and \(\left\lVert\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}\) remain bounded uniformly as \(\lambda_{\textrm{min}}(\Gamma)\) increases. To that end, from 33 , we get \[\nonumber \left\lVert\frac{\mathrm{d}P_u}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}{\le}2\left\lVert B_u\right\rVert_{\mathrm{F}}\bar{P}\int_0^\infty\left\lVert\mathrm{e}^{(A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda))\tau}\right\rVert^2_{\mathrm{F}}\mathrm{d}\tau.\] But by Lemma 3, \(A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda)\) is strictly stable uniformly on \(\mathcal{S}_0\) and independently of \(\Gamma\) (see ?? ), so there exists a finite constant \(c_1\) independent of \(\Gamma\) and \(\Lambda\) such that \(\int_0^\infty\left\lVert\mathrm{e}^{(A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda))\tau}\right\rVert_{\mathrm{F}}^2\mathrm{d}\tau<c_1\), and thus \[\label{eq:DPbound} \left\lVert\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}\le2\left\lVert B_u\right\rVert_{\mathrm{F}}\bar{P}c_1.\tag{52}\]

On the other hand, a differentiation of ?? yields: \[\begin{align} \nonumber &(A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda))\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\\&+\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}(A+B_u\Lambda{+}B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda))^\mathrm{T}\label{viqsmcpb}\\&+\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}B_aR^{-2}B_a^\mathrm{T}+B_aR^{-2}B_a^\mathrm{T}\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\\&+B_ue_ie_j^\textrm{T}\Pi(\Lambda)+\Pi(\Lambda)e_je_i^\textrm{T}B_u^\textrm{T} \\&+B_aR^{-1}B_a^\mathrm{T}\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}\Pi(\Lambda)+\Pi(\Lambda)\frac{\mathrm{d}P_u(\Lambda)}{\mathrm{d}\Lambda_{kl}}B_aR^{-1}B_a^\mathrm{T}=0. \end{align}\tag{53}\] This is an LE for \(\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\), the plant matrix of which, i.e., \(A+B_u\Lambda+B_aR^{-1}B_a^\mathrm{T}P_u(\Lambda)\), is uniformly strictly stable owing to Lemma 3, and the cost matrix of which, i.e., its last six terms, is uniformly bounded owing to Lemma 3 and 52 . Hence, following the line of proof of the previous paragraph, there exists a finite constant \(c_2>0\) such that for all \(\Lambda\in\mathcal{S}_0\): \[\label{eq:DPibound} \left\lVert\frac{\mathrm{d}\Pi(\Lambda)}{\mathrm{d}\Lambda_{kl}}\right\rVert_{\mathrm{F}}\le c_2.\tag{54}\] Combining 51 with 52 54 we obtain \[\begin{gather} \label{eq:hessianineq1} \frac{1}{2}\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{ij}\mathrm{d}\Lambda_{kl}}\ge[\Gamma]_{ik}\delta_{l,j} \\-\left\lVert B_u^\mathrm{T}\right\rVert_{\mathrm{F}}\Big(2c_1\left\lVert B_u\right\rVert_{\mathrm{F}}\bar{P}\bar{\Pi}+\bar{P}c_2\Big). \end{gather}\tag{55}\] To proceed, define the Hessian matrix: \[\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\textrm{vec}(\Lambda)^2}=\begin{bmatrix} \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{11}^2} & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{11}\mathrm{d}\Lambda_{21}} & \ldots & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{11}\mathrm{d}\Lambda_{m_u,n}} \\ \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{21}\mathrm{d}\Lambda_{11}} & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{21}^2} & \ldots & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{21}\mathrm{d}\Lambda_{m_u,n}} \\\vdots & \vdots & \ddots & \vdots \\ \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{m_u,n}\mathrm{d}\Lambda_{11}} & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{m_u,n}\mathrm{d}\Lambda_{21}} & \ldots & \frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\Lambda_{m_u,n}^2} \end{bmatrix}.\] Based on the inequality 55 , this Hessian becomes diagonally dominant for sufficiently large values of \([\Gamma]_{ii}\), \(i=1,\ldots,m_u\), while keeping \([\Gamma]_{ij}\) constant for \(i\ne j\), \(i,j=1,\ldots,m_u\). Therefore, there exists \(\gamma^\star>0\) such that if \(\Gamma\succ\gamma^\star I\) then \(\frac{\mathrm{d}^2\tilde{J}(\Lambda)}{\mathrm{d}\textrm{vec}(\Lambda)^2}\) is positive definite uniformly. Therefore, if \(\Gamma\succ\gamma^\star I\) then \(\tilde{J}\), when viewed as a function of \(\textrm{vec}(\Lambda)\), is strictly convex, and thus any stationary point is a minimum. Therefore, by item 3, Algorithm \(1\) converges to a minimum of \(\tilde{J}\), and thus of 10 .

Proof of Proposition [pr:1]: Since \(\mathrm{Ran}(B_a)\subseteq\mathrm{Ran}(B_u)\), there exists a matrix \(E\in\mathbb{R}^{m_u\times m_a}\) such that \(B_a=B_uE\). In light of this, 9 becomes \((A+B_u{\Lambda})^\mathrm{T}P_u+P_u(A+B_u{\Lambda})+Q+P_uB_uER^{-1}E^\textrm{T}B_u^\mathrm{T}P_u=0\). Choosing \(\Lambda=\bar{\Lambda}=-\frac{1}{2}R_u^{-1}B_u^\mathrm{T}P_u\) for some positive definite matrix \(R_u\succ0\) to be selected, this equation becomes \[\label{eq:temprob1} A^\mathrm{T}P_u+P_uA+Q+P_uB_u(ER^{-1}E^\textrm{T}-R_u^{-1})B_u^\mathrm{T}P_u=0.\tag{56}\] Selecting \(R_u=(aI+ER^{-1}E^\textrm{T})^{-1}\) for some positive \(a>0\), we have \(R_u^{-1}=aI+ER^{-1}E^\textrm{T}\). Therefore, 56 yields \[\label{eq:temprob2} A^\mathrm{T}P_u+P_uA+Q-aP_uB_uB_u^\mathrm{T}P_u=0.\tag{57}\] Since \(A\) is Hurwitz, we can define a constant matrix \(V\) as the unique positive definite solution of the LE \[\label{eq:tempV} A^\mathrm{T}V+VA+Q=0.\tag{58}\] Subtracting 57 from 58 , we obtain \(A^\mathrm{T}(V-P_u)+(V-P_u)A+aP_uB_uB_u^\mathrm{T}P_u=0\). Since \(A\) is Hurwitz and \(aP_uB_uB_u^\mathrm{T}P_u\succeq0\), we obtain \(P_u\preceq V\) by the Lyapunov theorem. Since \(P_u\) is positive semidefinite, this implies \(\left\lVert P_u\right\rVert_\mathrm{F}\le \left\lVert V\right\rVert_\mathrm{F}\). Next, by applying norms to 57 : \[\begin{gather} a\left\lVert P_uB_uB_u^\mathrm{T}P_u\right\rVert_\mathrm{F}=\left\lVert A^\mathrm{T}P_u+P_uA+Q\right\rVert_{\mathrm{F}}\\\le 2\left\lVert A\right\rVert_\mathrm{F}\left\lVert P_u\right\rVert_\mathrm{F}+\left\lVert Q\right\rVert_\mathrm{F}\le 2\left\lVert A\right\rVert_\mathrm{F}\left\lVert V\right\rVert_\mathrm{F}+\left\lVert Q\right\rVert_\mathrm{F}. \end{gather}\] Since \(A, V, Q\) are constant matrices, as \(a\rightarrow\infty\) we have \(\left\lVert P_uB_uB_u^\mathrm{T}P_u\right\rVert_\mathrm{F}\rightarrow0\), hence \(\left\lVert B_u^\mathrm{T}P_u\right\rVert_\mathrm{F}\rightarrow0\), and hence, \(\left\lVert K_u(\bar{\Lambda})\right\rVert_\mathrm{F}=\left\lVert R^{-1}B_a^\mathrm{T}P_u(\bar{\Lambda})\right\rVert_\mathrm{F}=\left\lVert R^{-1}E^\mathrm{T}B_u^\mathrm{T}P_u(\bar{\Lambda})\right\rVert_\mathrm{F}\rightarrow0\). Therefore, for every \(\epsilon>0\) there exists \(\Lambda\in\mathbb{R}^{m_u\times n}\) such that \(\left\lVert K_u(\Lambda)\right\rVert_{\mathrm{F}}^2<\frac{\epsilon}{2}\). In addition, for this \(\Lambda\), one can find \(\gamma_1>0\) such that if \(\Gamma\prec\gamma_1I\) then \(\textrm{tr}(\Lambda^\textrm{T}\Gamma\Lambda)<\frac{\epsilon}{2}\), and hence \(\tilde{J}(\Lambda)<\epsilon\) in total. Summarizing, we conclude for every \(\epsilon>0\) there exists \(\gamma_1>0\) such that if \(\Gamma\prec\gamma_1 I\) then \(\inf_{\Lambda} \tilde{J}(\Lambda)<\epsilon\).

Next, let \(\{{\Lambda}^n,{P}_u^n\}_{n\in\mathbb{N}}\) be a minimizing sequence of 10 , and \(K^n_u=R^{-1}B_a^\textrm{T}P_u^n\). From the constraints of 10 we get: \[\begin{gather} \label{eq:tempabove} (A+B_u\Lambda^n)^\mathrm{T}P_u^n+P_u^n(A+B_u\Lambda^n)+Q\\+K_u^{n\textrm{T}}RK_u^n=0. \end{gather}\tag{59}\] Since \(P_u^n\) is constrained to be positive definite and \(Q\succ0\), it follows from 59 that \(A+B_u\Lambda^n\) is strictly stable for all \(n\in\mathbb{N}\). Assume now, in seek for contradiction, that \(\sup_{n\in\mathbb{N}}\alpha(A+B_u\Lambda^n)=0\). Then, for every \(a>0\) there exists \(n^\star>0\) such that if \(n\ge n^\star\) then \(\alpha(A+B_u\Lambda^n)\ge-a\). In addition, if we choose \(a\) sufficiently close to \(0\), then this spectral absissa corresponds to an (without loss of generality simple) eigenvalue, denoted as \(\lambda_n\), that is not controllable by \(B_a\); otherwise, \(A+B_u\Lambda^n+B_aK_u^n\) would be unstable for some finite \(n\) (because \(K_u^n\) is the optimal attack gain against \(A+B_u\Lambda^n\), and \(A+B_u\Lambda^n\) is bounded), which contradicts \(P_u^n\in\mathbb{S}_+\). Given this, denote the uncontrollable eigenvector of \((A+B_u\Lambda^n, B_a)\) as \(v_n\), and let us decompose \(\Lambda^n=\Lambda^nv_nv_n^\dagger+\Lambda^n(I-v_nv_n^\dagger )\). Note, \(\Lambda^nv_nv_n^\dagger\ne0\) uniformly; otherwise, the eigenvalue \(\lambda^n\) corresponding to \(v_n\) would not tend to \(0\) as \(n\rightarrow\infty\), since it is not controllable by \(B_a\). Next, define \(\hat{\Lambda}^n=\Lambda^n(I-v_n v_n^\dagger)\), which satisfies \(\textrm{tr}(\hat{\Lambda}^{n\textrm{T}}\Gamma\hat{\Lambda}^n)<\textrm{tr}({\Lambda}^{n\textrm{T}}\Gamma{\Lambda}^n)\) uniformly. Since the difference between \(\hat{\Lambda}^n\) and \({\Lambda}^n\) is only over the space that is uncontrollable by \(B_a\), it follows that \(K_u(\hat{\Lambda}^n)=K_u(\Lambda^n)\). Therefore, \(\tilde{J}(\hat{\Lambda}^n)<\tilde{J}({\Lambda}^n)\) uniformly, i.e., \({\Lambda}^n\) is not a minimizing sequence. Therefore, we arrive at a contradiction, and \(\sup_{n\in\mathbb{N}}\alpha(A+B_u\Lambda^n)<0\).

Matrices \(\Lambda^n\), \(K_u^n\) are uniformly bounded because they are a minimizing sequence of 10 , so there exist subsequences of \(\Lambda^{n_k}\), \(K_u^{n_k}\), \(P_u^{n_k}\) of \(\Lambda^{n}\), \(K_u^{n}\), \(P_u^n\) such that \(\Lambda^{n_k}\rightarrow\bar{\Lambda}, K_u^{n_k}\rightarrow\bar{K}_u\) for some bounded limits \(\bar{\Lambda},~\bar{K}_u\). Since \(\sup_{n\in\mathbb{N}}\alpha(A+B_u\Lambda^n)<0\), \(A+B_u\bar{\Lambda}\) is strictly stable, hence there is a strictly positive definite matrix \(\bar{P}_u\succ0\) such that \[\label{eq:subtractalt} (A+B_u\bar{\Lambda})^\mathrm{T}\bar{P}_u+\bar{P}_u(A+B_u\bar{\Lambda})+Q+\bar{K}_u^\textrm{T}R\bar{K}_u=0.\tag{60}\] Defining \(\tilde{P}_u^{n_k}=P_u^{n_k}-\bar{P}_u\), \(\tilde{\Lambda}^{n_k}=\Lambda^{n_k}-\bar{\Lambda}\), subtracting 60 from 59 for the subsequence we get \[\begin{gather} \label{eq:subtractalt2} (A+B_u\bar{\Lambda})^\mathrm{T}\tilde{P}_u^{{n_k}}+\tilde{P}_u^{{n_k}}(A+B_u\bar{\Lambda})-\bar{K}_u^\textrm{T}R\bar{K}_u\\+K_u^{{n_k}\textrm{T}}RK_u^{n_k}+P_u^{n_k}B_u\tilde{\Lambda}^{n_k}+\tilde{\Lambda}^{\textrm{T}{n_k}}B_u^\textrm{T}P_u^{n_k}=0. \end{gather}\tag{61}\] Note, \(K_u^{n_k}\rightarrow\bar{K}_u\) and \(\tilde{\Lambda}^{n_k}\rightarrow0\). Moreover, \(P_u^{n_k}\) is bounded uniformly because \(A+B_u\Lambda^{n_k}\) is bounded and strictly stable uniformly and because \(Q+K_u^{n\textrm{T}}RK_u^n\) is bounded uniformly. Hence, 61 implies \(\tilde{P}_u^{n_k}\rightarrow0\) and thus \(P_u^{n_k}\rightarrow\bar{P}_u\). Finally, since this implies \(\bar{K}_u=R^{-1}B_a^\textrm{T}\bar{P}_u,\) rearranging 60 yields \[\begin{gather} \nonumber (A+B_u\bar{\Lambda}+B_a\bar{K}_u)^\mathrm{T}\bar{P}_u+\bar{P}_u(A+B_u\bar{\Lambda}+B_a\bar{K}_u)\\+Q-\bar{K}_u^\textrm{T}R\bar{K}_u=0. \end{gather}\] From the first paragraph, for every \(\epsilon\) there is \(\gamma^\star>0\) such that if \(\Gamma\prec\gamma^\star I\) then \(\left\lVert\bar{K}_u\right\rVert_{\mathrm{F}}<\epsilon\) and hence \(Q-\bar{K}_u^\textrm{T}R\bar{K}_u\succ0\) for small enough \(\epsilon\), so the above equation implies \(A+B_u\bar{\Lambda}+B_a\bar{K}_u\) is strictly stable. Thus \((\bar{\Lambda},\bar{P}_u)\) is a feasible solution of 10 that is also minimizing, hence 10 admits a minimizer.

Proof of Proposition [pr:2]: Let \(\Lambda\) be in the domain of \(\hat{J}\) and \(\tilde{J}\). Using the value of the optimal control problem 7 and the assumption \(\hat{Q}\preceq Q\), \(\hat{R}\succeq R\), we have \[\begin{gather} x_0^\mathrm{T}\hat{P}_u(\Lambda)x_0=\max_K\int_0^\infty(x^\mathrm{T}(\tau)\hat{Q}x(\tau)-a^\mathrm{T}(\tau)\hat{R}a(\tau))\mathrm{d}\tau\\\le \max_K\int_0^\infty(x^\mathrm{T}(\tau){Q}x(\tau)-a^\mathrm{T}(\tau){R}a(\tau))\mathrm{d}\tau=x_0^\mathrm{T}{P}_u(\Lambda)x_0. \end{gather}\] Since this holds for every \(x_0\), it follows \(\hat{P}_u(\Lambda)\preceq P_u(\Lambda)\). Accordingly, by combining this with the fact that \(\hat{R}\succeq R\): \[\begin{gather} \left\lVert\hat{R}^{-1}B_a^\mathrm{T}\hat{P}_u(\Lambda)\right\rVert_\textrm{F}^2\le\left\lVert{R}^{-1}B_a^\mathrm{T}{P}_u(\Lambda)\right\rVert_\textrm{F}^2 \\ \Longrightarrow \left\lVert\hat{K}_u(\Lambda)-\bar{K}\right\rVert_\textrm{F}^2\le\left\lVert{K}_u(\Lambda)-\bar{K}\right\rVert_\textrm{F}^2 \Longrightarrow \hat{J}(\Lambda)\le\tilde{J}(\Lambda), \end{gather}\] which yields \(\hat{J}(\Lambda^\star)\le \tilde{J}(\Lambda^\star)\) and proves item 1.

To prove item 2, there exists \(\lambda\ge1\) so that \(\hat{R}=\frac{1}{\lambda} \bar{R}\), \(\hat{Q}=\lambda \bar{Q}\), \(\bar{R}\succ R\), \(\bar{Q}\prec Q\). Given this, let \(\bar{P}_u(\Lambda^\star)\in\mathbb{S}_+\) solve \[\begin{gather} \label{eq:pert1} (A+B_u{\Lambda}^\star)^\mathrm{T}\bar{P}_u(\Lambda^\star)+\bar{P}_u(\Lambda^\star)(A+B_u{\Lambda}^\star)\\+\bar{Q}+\bar{P}_u(\Lambda^\star)B_a\bar{R}^{-1}B_a^\mathrm{T}\bar{P}_u(\Lambda^\star)=0, \end{gather}\tag{62}\] and denote as \(\bar{J}(\Lambda^\star)\) the cost with matrices \(\bar{Q}\), \(\bar{R}\).

In Proposition [pr:1] it was shown that for every \(\epsilon>0\) there exists \(\gamma>0\) such that if \(\Gamma\prec \gamma I\) then \(\tilde{J}(\Lambda^\star)<\epsilon\). In item 1, we also proved that \(\bar{J}(\Lambda^\star)\le\tilde{J}(\Lambda^\star)\). Combining these, for every \(\epsilon>0\) there exists \(\gamma>0\) such that if \(\Gamma\prec \gamma I\) then \(\bar{J}(\Lambda^\star)<\epsilon\). Hence, \(B_a^\textrm{T}\bar{P}_u(\Lambda^\star)\rightarrow0\) and thus \(\bar{P}_u(\Lambda^\star)B_a\bar{R}^{-1}B_a^\textrm{T}\bar{P}_u(\Lambda^\star)\rightarrow0\) as \(\gamma\rightarrow0\). Moreover, note that from 16 : \[\begin{gather} \label{eq:pert2} (A+B_u\Lambda^\star)^\mathrm{T}\hat{P}_u(\Lambda^\star)+\hat{P}_u(\Lambda^\star)(A+B_u\Lambda^\star)\\+\lambda(\bar{Q}+\hat{P}_u(\Lambda^\star)B_a\bar{R}^{-1}B_a^\mathrm{T}\hat{P}_u(\Lambda^\star))=0. \end{gather}\tag{63}\] Comparing 62 63 , since \(\bar{P}_u(\Lambda^\star)B_a\bar{R}^{-1}B_a^\textrm{T}\bar{P}_u(\Lambda^\star)\rightarrow0\) as \(\gamma\rightarrow0\), it follows that \(\hat{P}_u(\Lambda^\star)\approx \lambda\bar{P}_u(\Lambda^\star)\), where the approximation becomes exact as \(\gamma\rightarrow0\). Hence, as \(\gamma\rightarrow0\), \(B_a^\textrm{T}\hat{P}_u(\Lambda^\star)\rightarrow\lambda B_a^\textrm{T}\bar{P}_u(\Lambda^\star)\rightarrow0\) and thus \(\hat{K}_u(\Lambda^\star)\rightarrow0\). Therefore, for every \(\epsilon>0\) there exists \(\gamma^\star>0\) such that if \(\Gamma\prec \gamma^\star I\) then \(\left\lVert\hat{K}_u(\Lambda^\star)\right\rVert_\textrm{F}^2=\left\lVert\hat{R}^{-1}B_a^\textrm{T}\hat{P}_u(\Lambda^\star)\right\rVert_\textrm{F}^2<\frac{\epsilon}{2}\), and thus also \(\hat{J}(\Lambda^\star)<\epsilon\), which yields the required result.

References↩︎

---

1. \(^1\) F. Fotiadis and U. Topcu are with the Oden Institute for Computational Engineering & Sciences, University of Texas at Austin, Austin, TX, USA. Email: {ffotiadis, utopcu}utexas.edu?.↩︎

2. \(^2\) A. Kanellopoulos is with the Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden, e-mail: arisk@kth.se.↩︎

3. \(^3\)K. G. Vamvoudakis is with the School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA. Email: kyriakos@gatech.edu.↩︎

4. This work was supported in part, by ARO under grant No. W\(911\)NF-\(24-1-0174\), and by NSF under grant Nos. CAREER CPS-\(1851588\), SLES-\(2415479\), SATC-\(2231651\), CPS-\(2227185\), and CPS-\(2227153\).↩︎

5. This is for the case where \(\bar{K}\) is not stabilizing. For the case where \(\bar{K}=0\), regularization is not needed; see Section 6.1.↩︎

6. This implicitly requires that \((A,B_u)\) be controllable.↩︎

7. This follows from standard argument regarding the convergence of gradient descent; see [32].↩︎