1 Introduction↩︎

Two equivalent major results in multivariate prediction theory by H. Helson and D. Lowdenslager [1] and by N. Wiener and P. Masani [2] motivate new research efforts in matrix theory and multivariate prediction theory in the present paper. The goal is an extension of these results to a wider class of matrix sets to get a deeper insight into these aspects of multivariate prediction theory and to strengthen its ties with matrix theory.

Generalizing the Szegő infimum theorem, H. Helson and D. Lowdenslager proved the following remarkable result, cf. [1]. The result was announced in an earlier paper [3] by V. N. Zasukhin, a younger colleague of A. N. Kolmogorov. The paper [4] deals with an extension to certain \(L^p\)-spaces, and the paper [5] contains its further extension to the infinite-dimensional case. We denote the set of all complex-valued type \(n \times n\) matrices of a certain fixed dimension \(n\) by \(\mathcal{M}\).

By the Lebesgue-Radon-Nikodym theorem \(W\) can be described by a Radon-Nikodym derivative of the absolutely continuous part of \(M\). Considering \(M\) as a (non-stochastic) spectral measure of an \(n\)-variate weakly stationary sequence \({\mathbf{X}}=(X(s) = (x_1(s), \ldots , x_n(s))^\top )_{s \in \mathbb{Z}}\) and taking into account a multivariate version of Kolmogorov’s isomorphism, Theorem 1 can be given a probabilistic interpretation. Indeed, Theorem 1 is equivalent to the assertion that the distance of the set \(\{ AX(0) : A \in {\mathcal{A}}_1 \}\) to the \(\mathcal{M}\)-linear space spanned by \(\{ X(s): s \in -{\mathbb{N}} \}\) is equal to \(n \cdot {\rm exp} \left( \frac{1}{n} \int_{0}^{2 \pi} {\rm log} ({\rm det} (W))\, {\mathrm d} L \right)\), cf. [1] for a precise formulation. The assertion of Theorem 1 is equivalent to Main Theorem I of another celebrated paper of that time, cf. [2] Subsection 7.10. To state it denote the one-step-ahead prediction error matrix of \(\mathbf{X}\) by \(\Delta\), \(\Delta = \inf \{ \int_{0}^{2 \pi} (I-P) \, {\mathrm d} M \, (I-P)^* : P \in {\mathcal{P}}_0 \}\), where \(I\) denotes the identity matrix of \(\mathcal{M}\), and the infimum is taken with respect to Loewner’s semiordering.

Since the infimum ?? is equal to \[\begin{align} \begin{aligned} &\inf \left\{ {\rm tr} \left(A \int\limits_{0}^{2 \pi} (I-A^{-1}P) \, {\mathrm d}M \, (I-A^{-1}P)^*A^*\right) : A \in {\mathcal{A}}_1, P \in {\mathcal{P}}_0 \right\} \\ & \quad \quad \quad \quad \quad = \inf \left\{ {\rm tr} \left(A \int\limits_{0}^{2 \pi} (I-P) \, {\mathrm d}M \, (I-P)^*A^*\right) : A \in {\mathcal{A}}_1, P \in {\mathcal{P}}_0 \right\} \\ & \quad \quad \quad \quad \quad = \inf \{ {\rm tr} (A \Delta A^*) : A \in {\mathcal{A}}_1 \} \\ & \quad \quad \quad \quad \quad = n \, {\rm det}(\Delta^{1/n}) \, , \end{aligned} \end{align}\] cf. [6] for the last equality, the equivalence of Theorems 1 and 2 is obvious.

Generalizing the problem of H. Helson and D. Lowdenslager one could ask for the infimum \[\label{f1462} \inf \left\{ {\rm tr} \left(\int\limits_{0}^{2 \pi} (A-P) \, {\mathrm d}M \, (A-P)^* \right) : A \in {\mathcal{A}}, P \in {\mathcal{P}}({\mathcal{O}}) \right\} \, ,\tag{1}\] where \(\mathcal{A}\) is a non-empty subset of \(\mathcal{M}\), the symbol \(\mathcal{O}\) denotes a non-empty subset of \({\mathbb{Z}} \setminus \{ 0 \}\), which is interpreted as observation set of the sequence \({\mathbf{X}}\), and \({\mathcal{P}}({\mathcal{O}})\) is the set of all \(\mathcal{M}\)-valued trigonometric polynomials of the form \(\sum_{s \in {\mathcal{O}}} A_s e^{{\boldsymbol{i}}s(\cdot)}\). In Section 6 we formulate this problem, which we will denote by Problem (HL), as a problem of prediction theory and call it “multivariate prediction problem of Helson-Lowdenslager type”. If for given \(\mathcal{O}\) some information on the corresponding prediction error matrix \(\Delta ({\mathcal{O}}) = \inf \{ \int_{0}^{2 \pi} (I-P) \, {\mathrm d} M \, (I-P)^* : P \in {\mathcal{P}}({\mathcal{O}}) \}\) is known, one can try to determine the infimum 1 following the proof of Theorem 1 by Theorem 2 as sketched above. In this case, a main goal will be the computation of the infimum \[\label{f1463} \iota (\Delta ; {\mathcal{A}}) = \inf \{ {\rm tr} (A \Delta A^*): A \in {\mathcal{A}} \}\tag{2}\] for any \(\Delta \in {\mathcal{M}}^\geq\). A large part of our paper deals with this problem. Although probably the computation of \(\iota (\Delta ; {\mathcal{A}} )\) has little prediction-theoretical interest for many particular choices of the set \(\mathcal{A}\), we think this kind of problem (and, of course, also the corresponding problem of determining \(\sup \{ {\rm tr} (A \Delta A^*): A \in {\mathcal{A}} \}\)) is of interest from the point of view of linear algebra and matrix theory. In particular, we are interested in cases where the infimum is attained. Moreover, as will be pointed out in Section 5, these problems deserve investigation since one could derive new inequalities from their solutions. In places we indicate open problems.

In Section 2 some general and auxilary results are summarized. We also recall the simple variant of computing the infimum 2 in case the set \(\mathcal{A}\) is a hyperplane of \(\mathcal{M}\). In Section 6 we demonstrate that its solution is of significant importance in multivariate prediction theory despite the simplicity of that particular case.

Section 3 deals with the infimum of 2 if \(\mathcal{A}\) is a unitarily invariant subset of \(\mathcal{M}\). In this situation the value \(\iota (\Delta ; {\mathcal{A}})\) can be expressed by a permutation invariant function of the eigenvalues of \(\Delta\), so the theory of majorization is helpful, cf. [7] for a comprehensive treatise of it. For some observation sets \(\mathcal{O}\) that are important in practice the matrix \(\Delta({\mathcal{O}})\) cannot be calculated precisely. However, this matrix is at least known to be non-invertible for certain sets of spectral measures \(M\). Consequently, we are searching for sets \(\mathcal{A}\) such that \(\iota(\Delta; {\mathcal{A}})\) can be described. In particular, sets \(\mathcal{A}\) forcing \(\iota(\Delta; {\mathcal{A}})=0\) are of special interest. What is more, Corollary 25 describes a large class of sets \(\mathcal{A}\) such that for any \(\Delta \in {\mathcal{M}}^\geq\) the infimum \(\iota (\Delta ; {\mathcal{A}})\) is equal to the smallest eigenvalue of \(\Delta\).

In Sections 4 and 5 we deal with more concrete examples. Section 4 is mainly devoted to the case that \(\mathcal{A}\) is a unit sphere of a unitarily invariant norm, but we also obtain a result if \(\mathcal{A}\) is a unit sphere of a \(C\)-numerical radius. Section 5 contains further examples, particularly, a generalization of the case \({\mathcal{A}}={\mathcal{A}}_1\) is given, shown at the beginning of this introduction. Most results of Sections 4 and 5 provide examples of permutation invariant and concave, hence, Schur concave functions on \([0,+\infty)^n\). As indicated in Section 5, this observation opens the way to prove new inequalities.

Section 6 shows applications to prediction theory of multivariate weakly stationary sequences. Beside the interpretation given by H. Helson and D. Lowdenslager ([1]) we give some further interpretations and indicate additional consequences. In particular, we obtain results concerning the multivariate prediction problem of Yu. A. Rozanov ([8]), where the value \(x_1(0)\) should be predicted not only on the basis of the realizations of \(\mathbf{X}\) at the points of the observation set \(\mathcal{O}\), but additionally, on the basis of the values \(\{ x_j(0) : j \in \{ 2,\ldots,n\} \}\).

The approach described in the present paper is applicable only if the prediction error matrix \(\Delta({\mathcal{O}})\) itself or at least some information on it are known. So, we recall several investigations of practical interest from the literature where such information is available:

• ‘classical’ prediction: \({\mathcal{O}}= - {\mathbb{N}}\), cf. Theorems 1 and 2 and the corresponding papers [1], [2] as well as the monograph [8];

• interpolation problem: \({\mathcal{O}} = (- {\mathbb{N}}) \cup {\mathbb{N}}\), cf. [9];

• Nakazi’s problem: \({\mathcal{O}} = (- {\mathbb{N}}) \cup \{ 1,\ldots,k\}\) for some \(k \in {\mathbb{N}}\), cf. [10];

• Pourahmadi’s problem: \({\mathcal{O}} = (- {\mathbb{N}}) \setminus \{ -k \}\) for some \(k \in \mathbb{N}\), cf. [10];

• periodic observations: \({\mathcal{O}} = \{ ks+m : s \in {\mathbb{Z}} \}\) for a certain \(k \in {\mathbb{N}} \setminus \{ 1 \}\) and selected \(m \in \{ 1,\ldots,(k-1) \}\), cf. [11].

2 Preliminaries and the hyperplane case↩︎

In this section we fix denotations and recall some basic results used in the sequel. We consider \(m \times n\) matrices as linear operators between complex finite-dimensional vector spaces, i.e. from \({\mathbb{C}}^n\) to \({\mathbb{C}}^m\), where some orthonormal bases of these spaces are fixed to get a unique matrix representation of each linear operator. Vectors are imagined as column vectors, however, for the sake of a reasonable text reading, we write them as row vectors with a transposed sign at the end. For any \(m \times n\) matrix \(A\) with complex entries denote \(A^{\top}\), \(A^*\), \(A^\dagger\), \({\mathcal{R}}(A)\), \({\mathcal{N}}(A)\), and \({\rm rk}(A)\) its transpose, adjoint, Moore–Penrose inverse, range, kernel, and rank, respectively. If \(A\) is a quadratic matrix, then \({\rm tr}(A)\) and \({\rm det}(A)\) denote its trace and determinant values, respectively. If \(A\) is a positive matrix \(A^{1/2}\) denotes the unique positive matrix such that \((A^{1/2})^2=A\). Throughout the present paper, let \(n\) be a positive integer, where the symbol \(\mathcal{M}\) stands for the algebra of complex \(n \times n\) matrices, \(\mathcal{I}\) for its open subset of all invertible matrices, \(\mathcal{U}\) for its compact subset of all unitary matrices, and \({\mathcal{M}}^{\geq}\) for its convex cone of all non-negative Hermitian matrices.

For \({\mathcal{A}} \subseteq {\mathcal{M}}\) being a hyperplane, we want to resume the solution of the problem of computing the infimum of the value \({\rm tr}(A \Delta A^*)\) \[\label{f2460} \iota(\Delta;{\mathcal{A}}) := \inf \left \{ {\rm tr}( A \Delta A^*) : A\in {\mathcal{A}} \right \}\tag{3}\] for any \(\Delta\in {\mathcal{M}}^{\geq}\).

In the following, the singular values of a matrix \(A \in {\mathcal{M}}\) are denoted by \(\sigma_j(A)\), where the dependence on \(A\) will not be indicated in unambiguous cases, i.e. we write \(\sigma_j\) in short instead of \(\sigma_j(A)\). However, we emphasize that the singular values of \(A\) are always assumed to be ordered decreasingly throughout this paper, i.e. \[\label{f2461} \sigma_j \geq \sigma_{j+1} \, .\tag{4}\]

Let \(\|A\|_2\) be the Schatten \(2\)-norm (or Euclidean or Frobenius norm) of \(A\), what implies \[\|A\|_2 ^2= {\rm tr}(A^* A) = \sum_{k,l} |a_{k l}|^2 = \sum_{j} \sigma_j^2 \, ,\] where \(A=(a_{k l})\) is a quadratic matrix with complex entries \(a_{k l}\), \(A \in {\mathcal{M}}\) of arbitrary size \(n\). The symbol \(\Delta\) always stands for a matrix of \({\mathcal{M}}^{\geq}\). Let \(\lambda_1,\ldots,\lambda_n\) be the eigenvalues of \(\Delta\), where \[\lambda_j \geq \lambda_{j+1}, \quad j\in\{1,\ldots,n-1\},\] in accordance with 4 . Moreover, let \({\mathcal{U}}_{\Delta}\) denote the set of all \(U\in {\mathcal{U}}\) such that \[\label{f2463} \Delta = U \, {\rm diag}(\lambda_n,\ldots,\lambda_1) \, U^*,\tag{5}\] where \({\rm diag}(\lambda_n,\ldots,\lambda_1)\) stands abbreviated for the diagonal matrix in \(\mathcal{M}\) with entries on the principal diagonal \(\lambda_n,\ldots,\lambda_1\) incipient top left. Note that, therefore, the eigenvalues on the principal diagonal at the right-hand side of 5 are ordered increasingly, which simplifies our presentation slightly. We denote the identity matrix by \(I\).

Recall that a real-valued function \(f\) on a convex cone \(\mathcal{K}\) of an \(\mathbb{R}\)-linear space is called positively homogeneous, concave, and superadditive if \[f(ax) = af(x),\,\,\, f\bigl(bx+(1-b)y\bigr) \geq bf(x)+(1-b)f(y),\,\,\, and\,\,\, f(x+y\bigr) \geq f(x)+f(y),\] respectively, for all \(a\in (0,+\infty)\), \(b\in[0,1]\), \(x,y\in {\mathcal{K}}\). Moreover, if \(0\in {\mathcal{K}}\), we suppose that \(f(0)=0\). Note that, here and henceforth, the notation \(0\) is used for the null element in the corresponding space.

The set of all non-negative and positively homogeneous functions on \(\mathcal{M}\), which are different from the zero function, is denoted by \(\mathcal{F}\). The proofs of the following facts are omitted.

In the following, \(\mathcal{A}\) always denotes a non-empty subset of \(\mathcal{M}\), and we use the notation given in 4 based on a matrix \(\Delta\). If the set \(\mathcal{A}\) is characterized by a certain family of conditions \(\mathcal{C}\) we shall write \(\iota(\Delta;{\mathcal{C}})\) for short. For example, if \({\mathcal{A}}=\{A\in {\mathcal{M}} : {\rm tr}(A) = 1\}\), we write \(\iota(\Delta; {\rm tr}(A) = 1)\) instead of \(\iota(\Delta;\{A\in {\mathcal{M}} : {\rm tr}(A) = 1\})\). If the infimum stated in 3 is attained, we call it a minimum and write the lowercase letter \(m\) in place of \(\iota\). Moreover, the set of all \(A\in {\mathcal{A}}\), where the minimum is attained, will be denoted by \({\mathcal{A}}(\Delta)\). In particular, \({\mathcal{A}}(\Delta)=\emptyset\) if and only if the infimum is not attained.

Note that the function \(\iota(\,\cdot\,; {\mathcal{A}})\) on the convex cone \({\mathcal{M}}^{\geq}\) is non-negative, positively homogeneous, concave, and superadditive, hence, increasing with respect to Loewner’s semiordering. From concavity follows that \(\iota(\,\cdot\,; {\mathcal{A}})\) is continuous at the set \({\mathcal{M}}^{\geq} \cap {\mathcal{I}}\) of inner points of \({\mathcal{M}}^{\geq}\). As an infimum of continuous functions, it is upper semicontinuous on \({\mathcal{M}}^{\geq}\). If \({\mathcal{A}}\) is a bounded set, there exists a constant \(c\in(0,+\infty)\) such that \({\rm tr}( A \Delta A^*)\leq c \cdot {\rm tr}(\Delta)\) for all \(A\in {\mathcal{A}}\), hence, \(\iota(\,\cdot\,;{\mathcal{A}})\) is continuous. In general, \(\iota(\,\cdot\,;{\mathcal{A}})\) is not necessarily continuous at boundary points of \({\mathcal{M}}^{\geq}\).

For future reference the following lemma summarizes further properties of \(\iota(\Delta;{\mathcal{A}})\) for non-empty subsets \(\mathcal{A}\) of \(\mathcal{M}\), whose easy verifications are omitted.

The next lemma reveals that for many sets \(\mathcal{A}\) it is sufficient to deal with invertible matrices \(\Delta\), i.e. with matrices \(\Delta\) such that \({\mathcal{R}}(\Delta)\) is the entire space.

The second assertion of the preceding lemma has a partial extension to invertible matrices \(\Delta\).

Now, we resort to a first concrete setting. If the set \(\mathcal{A}\) is a hyperplane, the behaviour of \(\iota(\,\cdot\,;{\mathcal{A}})\) is known and easily to describe, cf. [12] and [6] for a related result. Despite the simplicity, the hyperplane case is of considerable importance in the prediction theory. For convenience of the reader we give a full proof and add some details and examples, which will be applied to multivariate prediction problems in Section 6.

As an immediate conclusion we get the following.

Let \(r:={\rm rk}(X)\). According to the orthogonal decomposition \({\mathbb{C}}^n = {\mathcal{R}}(X^*) \oplus {\mathcal{N}}(X)\) the matrix has a block representation \(X = (X_1 \,\, 0)\) with an \(n\times r\) matrix \(X_1\) of rank \(r\). If \(r < n\), let \[\Delta = \left(\begin{array}{cc} \Delta_{11} & \Delta_{12} \\ \Delta_{12}^{*} & \Delta_{22} \end{array}\right)\] be the corresponding partition of \(\Delta\) with \(r\times r\) matrix \(\Delta_{11}\) and let \(\Delta/\Delta_{22}:=\Delta_{11} - \Delta_{12}\Delta_{22}^{\dagger}\Delta_{12}^*\) be its generalized Schur complement. If \(r = n\), for harmonization, let \(\Delta/\Delta_{22}:=\Delta\). Some consequences of Proposition 8 can be stated in terms of \(\Delta/\Delta_{22}\).

We also note the following consequence of Proposition 8 and Corollary 10.

One can also derive a well-known generalization of Bergström’s inequality, cf. [14]. For this, let \(\Delta_k\) denote the right lower \((n-k) \times (n-k)\) corner of \(\Delta\).

3 Unitary invariance↩︎

The most valuable results on \(\iota(\,\cdot\,;{\mathcal{A}}) = \inf \{ {\rm tr}(A \Delta A^*) : A \in {\mathcal{A}} \}\) can be established if \({\mathcal{A}}\) has various properties of unitary invariance. A non-empty subset \(\mathcal{A}\) of \(\mathcal{M}\) is called right unitarily invariant, unitary similarity invariant or unitarily invariant if \({\mathcal{A}}U = {\mathcal{A}}\), \(U^*{\mathcal{A} }U = {\mathcal{A}}\) or \(V{\mathcal{A}}U={\mathcal{A}}\), respectively, for all \(U,V \in {\mathcal{U}}\). If \(\mathcal{A}\) is right unitarily invariant or unitary similarity invariant then it is called weakly unitarily invariant. Note, that our choice of notions differs from the notion of weakly unitarily invariant (wui) norms in [15]. Analogously to the denotation for diagonal matrices, we denote a block-diagonal matrix with a sequence of quadratic blocks \((A_j)\), \(j=1,\ldots,n\), of possibly different sizes as \({\rm diag}(A_1, \ldots , A_n)\).

The next theorem is partially already known. We define a set \(\mathcal{D}\) by \[\label{f3462} {\mathcal{D}} = \{ (x_1, \ldots ,x_n)^\top \in [0,+\infty)^n : x_1 \geq \ldots \geq x_n \}\tag{9}\] and its subset \({\mathcal{D}}_+\) by \({\mathcal{D}}_+ = {\mathcal{D}} \cap (0,+\infty)^n\). Note, that according to our convention 4 the vector consisting of the singular values of a matrix \(A \in \mathcal{M}\) belongs to \(\mathcal{D}\).

The relation ?? is a classical result and the main assertion of this theorem, cf. [7]. To prove ?? we use an auxiliary result:

The next consequence of Theorem 15 plays a key role in the present paper.

Corollary 18 shows that the number \(\iota(\,\Delta \,;{\mathcal{A}})\) is uniquely defined by the ordered set of eigenvalues of \(\Delta\) as long as \({\mathcal{A}}\) is weakly unitarily invariant. Thus, it is natural to define a function \(g_{\mathcal{A}}\) on \(\mathcal{D}\) by \[g_{\mathcal{A}}(x_1, \ldots , x_n) = \iota({\rm diag} (x_1, \ldots , x_n); {\mathcal{A}}) \, , \,\, (x_1, \ldots , x_n)^\top \in {\mathcal{D}} \, .\]

Given \(g_{\mathcal{A}}\) we introduce its permutation invariant extension to \([0,+\infty)^n\) denoted by \(({g}_{\mathcal{A}})\tilde{\,}\). We would like to describe the set \(\tilde{{\mathcal{G}}} = \{ ({g}_{\mathcal{A}})\tilde{\,}: {\mathcal{A}} \,\, {\rm is} \, {\rm unitarily} \, {\rm invariant} \}\). In particular, \(({g}_{\mathcal{A}})\tilde{\,}\) can be considered as the restriction of \(\iota( \cdot ; {\mathcal{A}})\) to the set of non-negative Hermitian diagonal matrices. Therefore, the function \(({g}_{\mathcal{A}})\tilde{\,}\) is non-negative, positively homogeneous, superadditive, continuous, permutation invariant and concave, so in particular, Schur concave, cf. [7]. As a question, does a function belong to \(\tilde{{\mathcal{G}}}\) provided it has the named properties? The following example shows that the property of Schur concavity instead of concavity does not warrant that a respective function will be an element of \(\tilde{{\mathcal{G}}}\).

The next assertion is a first result on the structure of \(\tilde{{\mathcal{G}}}\), cf. Proposition 43 below for its multiplicative analogue.

Let \(\phi\) be an element of the set \(\mathcal{F}\) of non-negative and positively homogeneous functions on \(\mathcal{M}\) different from the zero function. We are going to discuss \(\iota( \cdot; {\mathcal{S}}_\phi)\) for the ‘unit sphere’ \({\mathcal{S}}_\phi = \{ A \in {\mathcal{M}}: \phi(A)=1 \}\) of \(\phi\). The following facts are repeatedly used in the sequel without explicit arguments:

We will also use an analogue of Proposition 7:

A function \(\phi\) on \(\mathcal{M}\) is called right unitarily invariant, unitary similarity invariant or unitarily invariant if \(\phi(AU)=\phi(A)\), \(\phi(U^*AU) = \phi(A)\) or \(\phi(VAU) = \phi(A)\), respectively, for all \(A \in \mathcal{M}\), any \(U,V \in \mathcal{U}\). If \(\phi\) is right unitarily invariant or unitary similarity invariant it is called weakly unitarily invariant. Clearly, \(\phi\) has one of these properties if and only if its unit sphere \({\mathcal{S}}_\phi\) has the corresponding property. In accordance with a common normalization of unitarily invariant norms we will often assume additionally that \[\label{f34612} \phi(E)=1\tag{13}\] for the minimal projection \(E = {\rm diag}(1,0,\ldots,0)\).

In contrast to the preceding corollary there exist unitarily invariant functions \(\phi \in \mathcal{F}\) satisfying the conditions 11 , 13 , and nevertheless \(m( \Delta ; {\mathcal{S}}_\phi ) < \lambda_n\) for all invertible \(\Delta\), cf. Proposition 29 below. Even more, we do not know any other function \(\phi\) different from \(\| \cdot \|_2\) that satisfies \[\label{f34613} \iota(\Delta ; {\mathcal{S}}_\phi ) = \lambda_n\tag{14}\] for any \(\Delta \in {\mathcal{I}}\). We give an example where 14 appears for some invertible \(\Delta\).

By Lemma 6 one obtains a large class of functions \(\phi \in \mathcal{F}\) satisfying \(m(\Delta ; {\mathcal{S}}_\phi )=0\) if \(\Delta\) is not invertible.

If ?? is not satisfied the value \(m(\Delta ; {\mathcal{S}}_\phi )\) can be strictly greater than zero for any \(\Delta \in ({\mathcal{M}}^\geq \setminus \{ 0 \} )\). If, e.g., \(\phi(A)=\sigma_n\) for \(A \in \mathcal{M}\), then \(m(\Delta ; {\mathcal{S}}_\phi ) = \sum_{j=1}^n \lambda_j\) by Corollary 18. Note, however, that the realization of condition ?? is not necessary for the equality \(\iota(\Delta ; {\mathcal{S}}_\phi ) = 0\) to hold for any non-invertible \(\Delta\), cf. Proposition 38 below.

4 Properties of \(m( \cdot ; {\mathcal{S}}_{\| \cdot \|})\) as an important particular case↩︎

We are going to deal with the calculation of the minimum \(m( \cdot ; {\mathcal{S}}_\phi)\) if \(\phi( \cdot ) = \| \cdot \|\) is a norm on \(\mathcal{M}\) which has one of the three properties of unitary invariance under consideration. Taking into account Lemma 27 we can and shall assume throughout this section that \(\Delta\) is invertible to compute \(m( \Delta ; {\mathcal{S}}_{\| \cdot \|})\).

A unitarily invariant norm \(\| \cdot \|\) on \(\mathcal{M}\) is called a Q-norm if there exists a unitarily invariant norm \(\| \cdot \| \widehat{\,\,}\) such that \(\| A \|^2=\| A^*A \| \widehat{\,\,}\) for \(A \in \mathcal{M}\), cf. [15]. Let \(\| \cdot \|_1\) denote the Schatten 1-norm, i.e. \(\| A \|_1 = \sum_{j=1}^n \sigma_j\). Since for a Q-norm \(\| \cdot \|\) normalised by 13 , the chain of inequalities \(\| A \|^2 = \| A^*A \| \widehat{\,\,} \leq \| A^*A \|_1 = \| A \|_2^2\) is true, we derive the next assertion by Corollary 25:

Let us mention that there exists a unitarily invariant norm \(\| \cdot \|\) satisfying \(\| \cdot \| \leq \| \cdot \|_2\) and \(\| E \| = 1\), which is not a Q-norm. For example, if the norm defined by \(\| A \| = 1/2 \cdot (\| A \|_2 + \sigma_1(A))\) would be a Q-norm, there would exist a symmetric gauge function \(g\) such that \(g(x_1,\ldots,x_n)=1/4 \cdot ((\sum_{j=1}^n x_j)^{1/2} + x_1)^2\) for \((x_1,\ldots,x_n)^\top \in \mathcal{D}\). Elementary calculations reveal that \(g\) does not satisfy the triangle inequality and, thus, \(g\) cannot be a symmetric gauge function.

For \({\mathbf{t}}=(t_1,\ldots,t_n)^\top \in [0,+\infty)^n \setminus \{ 0 \}\) and \(p \in (0,+\infty)\) define a unitarily invariant function \(\phi_{{\mathbf{t}},p} \in \mathcal{F}\) by \[\phi_{{\mathbf{t}},p}(A) = \left( \sum_{j=1}^n t_j\sigma_j^p \right)^{1/p} \, , \,\, A \in \mathcal{M} \, .\]

If \({\mathbf{t}} \in \mathcal{D}\) and \(p \in [1,+\infty)\), then \(\phi_{{\mathbf{t}},p}\) is a norm, cf. [15] for a more general result. Conversely, if \(\phi_{{\mathbf{t}},p}\) is a norm, its restriction to the set of non-negative Hermitian diagonal matrices can be identified with a convex and permutation invariant function on \([0,+\infty)^n\). By [7] this function is Schur convex, and [7] shows \({\mathbf{t}} \in \mathcal{D}\) and \(p \in [1,+\infty)\). To simplify denotation we use \({\mathcal{S}}_{{\mathbf{t}},p}\) instead of \({\mathcal{S}}_{\phi_{{\mathbf{t}},p}}\) in the sequel.

From the proof of assertion (ii) of the preceding proposition we see that formula ?? is true for some \(\Delta\) as soon as the chain of inequalities 18 is valid, even if \({\mathbf{t}} \not\in {\mathcal{D}}\). Note that the condition \({\mathbf{t}} \in {\mathcal{D}}\) is equivalent to the fact that 18 is true for all \(\Delta \in {\mathcal{I}}\). This observation allows to give the assertion of Proposition 29(ii) a slightly more general expression.

For \(\mathbf{t}\) as in the corollary, a conclusion analogous to that one of Proposition 29(i) fails for \(p \in [2,+\infty)\). Indeed, to prove the inequality \(\phi( \cdot ) \leq \| \cdot \|_2\) we have to verify the inequality \(\left( \sum_{\ell=1}^k t_{n_\ell} \sigma_{n_\ell}^p \right)^{1/p} \leq \left(\sum_{\ell=1}^k (n_\ell-n_{\ell-1}) \sigma_{n_\ell}^2\right)^{1/2}\) which cannot be obtained, in general, replacing \(\sigma_{n_l}^2\) by the particular values \((\sigma_{n_\ell}')^2 =(n_\ell-n_{\ell-1}) \sigma_{n_\ell}^2\), since the order of \(\sigma_{n_\ell}'\) might be destroyed. Only a weaker extension of Proposition 29(i) can be stated.

An example shows that for general \(\mathbf{t}\) one cannot expect to describe \(m(\Delta ; {\mathcal{S}}_{{\mathbf{t}}, p})\) by a single formula.

Recall, that for \(C \in \mathcal{M}\) the \(C\)-numerical radius \(w_C(A)\) of a matrix \(A\) is defined by \[\label{f44610} w_C(A) = \max \{ | {\rm tr}(UCU^*A) | : U \in {\mathcal{U}} \} \, .\tag{20}\] The function \(w_C\) is a unitary similarity invariant seminorm on \(\mathcal{M}\) and a norm if and only if \({\rm tr}(C) \not= 0\) and \(C\) is not a scalar multiple of \(I\), cf. [15]. To simplify denotation set \({\mathcal{S}}_{w_C} = {\mathcal{S}}_C\).

Let \(\| \cdot \|\) be a unitarily invariant norm. Recall, that its dual norm \(\| \cdot \|^D\) is defined by \(\| A \|^D = \max \{ | {\rm tr}(AC^*)| : \| C \| = 1 \}\), cf. [16]. Another consequence of Proposition 29 is an alternative extremal representation of \(m( \cdot ; {\mathcal{S}}_{\| \cdot \|})\) in terms of the dual norm of \(\| \cdot \|\).

We finish this section giving a positive lower bound for \(m(\Delta ; {\mathcal{S}}_{\| \cdot \|})\), \(\Delta \in \mathcal{I}\). Let \(g\) denote the symmetric gauge function of a unitarily invariant norm \(\| \cdot \|\). Recall, that the Ky Fan \(k\)-norm is defined as the sum of the first \(k\) singular values of a given \(m \times n\) matrix \(A\), i.e. \(\|A\|_{(k)} = \sum_{i=1}^k s_i(A)\) with \(k \leq \min \{ m,n\}\), where the singular values are assumed to be ordered in decreasing order, cf. [15], [17]–[19].

Because of the preceding proposition one might guess that for all unitarily invariant norms the inequality ?? should be an equality either for any \(\Delta \in {\mathcal{I}}\) or for no \(\Delta \in {\mathcal{I}}\). We give an example where ?? is an equality for a non-trivial subset of \({\mathcal{M}}^\geq \cap \mathcal{I}\). The problem of a suitable description of the set of all unitarily invariant norms such that the inequality ?? is an equality for all or for no \(\Delta \in \mathcal{I}\), respectively, remains to be open.

5 Further examples and related inequalities↩︎

First, we present an amplification of a result by Helson and Lowdenslager [1] (cf. Thm. 1 above). For \({\mathbf{t}}=(t_1,\ldots,t_n)^\top \in (0,+\infty)^n\) set \(t = \sum_{j=1}^n t_j\) and \({\mathcal{S}}_{\mathbf{t}} = \{ A \in {\mathcal{M}} : \prod_{j=1}^n \sigma_j^{t_j} = 1 \}\).

To prepare the proof we need the following lemma to apply Corollary 18:

Using Proposition 38 and the following lemma a multiplicative analogue of Proposition 21 can be proved.

Similarly to Remark 41, the assertion of Proposition 29 can be extended to negative exponents in two ways.

For \(k \in \{ 1,\ldots,n \}\) and \(r \in \{ 1,\ldots,k \}\) let \(E_{r,k}\) be the \(r\)th elementary symmetric harmonic polynomial of \(k\) variables, i.e. \(E_{r,k}(x_1,\ldots,x_k) = \sum_{j \in J} x_{j_1} \ldots x_{j_r}\) with \((x_1,\ldots,x_k)^\top \in [0,+\infty)^k\) and \(J\) the (index) set of all strictly increasing \(r\)-tuples \((j_1,\ldots,j_r)\) selected from \(\{ 1,\ldots,k \}\). Since \(m(\Delta ; E_{1,n}(\sigma_1^2,\ldots,\sigma_n^2)=1)=\lambda_n\) by Proposition 29(i) and since \(m(\Delta ; E_{n,n}(\sigma_1^2,\ldots,\sigma_n^2)=1) = n \cdot {\rm det}(\Delta^{1/n})\) by Proposition 38(ii) one could try to find a formula for \(m(\Delta ; E_{r,n}(\sigma_1^2,\ldots,\sigma_n^2)=1)\), \(r \in \{ 2,\ldots,n-1\}\). We do not know a general solution. However, Proposition 46 below gives an answer in case \(r=2\). Again, we start with an auxiliary extremal problem. Let \((a_k)_{k \in \mathbb{N}}\) be an increasing sequence of positive real numbers. For \(n \geq 2\) let \(f_n(x_1,\ldots,x_n)=\sum_{j=1}^n a_jx_j\) with \((x_1,\ldots,x_n)^\top \in [0,+\infty)^n\). Consider the following extremal problem \({\mathcal{E}}_n\): \[\label{f54616} {\rm Find} \,\, \mu_n = \min f_n(x_1,\ldots,x_n) \,\, {\rm under} \, {\rm the} \, {\rm condition} \,\, E_{2,n}(x_1,\ldots,x_n)=1 \, .\tag{32}\] Since the problem \({\mathcal{E}}_{n-1}\) can be considered as the problem \({\mathcal{E}}_n\) restricted to the part \(\{ (x_1,\ldots,x_{n-1},0)^\top : x_1,\ldots,x_{n-1} \in [0,+\infty)^{n-1} \}\) of its boundary we get \(\mu_n \leq \mu_{n-1}\) for any \(n\). Moreover, some computations show that \({\mathcal{E}}_2\) has exactly one stationary point \((x_1^{(2)},x_2^{(2)})\) satisfying \(\mu_2 = f_2(x_1^{(2)},x_2^{(2)}) = 2 \sqrt{a_1a_2}\).

For \(n \geq 2\) set \({\mathcal{A}} = \{ A \in {\mathcal{M}}: E_{2,n}(\sigma_1^2, \ldots , \sigma_n^2)=1 \}\).

We enrich this collection of examples giving another application of Proposition 29. Set \(f_{r,k}(A) = E_{r,k} (a_{11},\ldots,a_{kk})\) and \(\phi_{r,k}(A) = \max \{ f_{r,k} (U^*AU) : U \in {\mathcal{U}} \}\) for \({\mathcal{A}} \in {\mathcal{M}}^\geq\).

It is well-known that extremal representations of matrix-valued functions are a rich source of inequalities. We want to sketch ways to obtain inequalities from the results above illustrating them by examples.

A first set of inequalities can be immediately derived from concavity and superadditivity of the function \(\iota( \cdot : {\mathcal{A}})\). For example, selecting \({\mathbf{t}}=(1,\ldots,1)^\top\) and \(p=1\) in Proposition 29(ii) one obtains \[\begin{align} \label{f54625} & & \\ \frac{E_{n,n}(\lambda_1(\Delta_1+\Delta_2),\ldots,\lambda_n(\Delta_1+\Delta_2))}{E_{n-1,n}(\lambda_1(\Delta_1+\Delta_2),\ldots,\lambda_n(\Delta_1+\Delta_2))} & \geq & \frac{E_{n,n}(\lambda_1(\Delta_1),\ldots,\lambda_n(\Delta_1))}{E_{n-1,n}(\lambda_1(\Delta_1),\ldots,\lambda_n(\Delta_1))} \nonumber \\ & & + \frac{E_{n,n}(\lambda_1(\Delta_2),\ldots,\lambda_n(\Delta_2))}{E_{n-1,n}(\lambda_1(\Delta_2),\ldots,\lambda_n(\Delta_2))} \nonumber \end{align}\tag{39}\] for \(\Delta_1,\Delta_2 \in {\mathcal{M}}^\geq \cap \mathcal{I}\). In particular, if \(\Delta_1={\rm diag}(x_1,\ldots,x_n)\), \(\Delta_2 = {\rm diag}(y_1,\ldots,y_n)\) we obtain superadditivity of harmonic mean or the equivalent inequality \[\label{f54626} \frac{E_{n,n}(x_1+y_1,\ldots,x_n+y_n)}{E_{n-1,n}(x_1+y_1,\ldots,x_n+y_n)} \geq \frac{E_{n,n}(x_1,\ldots,x_n)}{E_{n-1,n}(x_1,\ldots,x_n)} + \frac{E_{n,n}(y_1,\ldots,y_n)}{E_{n-1,n}(y_1,\ldots,y_n)}\tag{40}\] for \((x_1,\ldots,x_n)^\top, (y_1,\ldots,y_n)^\top \in (0,+\infty)^n\)

Conversely, if the inequality 40 is true the function \(\frac{E_{n,n}}{E_{n-1,n}}\) is concave, and hence, Schur concave on \((0,+\infty)^n\), cf. [7]. Since the vector \((\lambda_1(\Delta_1+\Delta_2),\ldots,\lambda_n(\Delta_1+\Delta_2))^\top\) is majorized by the vector \((\lambda_1(\Delta_1)+\lambda_1(\Delta_2),\ldots,\lambda_n(\Delta_1)+\lambda_n(\Delta_2))^\top\), 40 implies 39 . Marcus and Lopez proved by induction on \(r\) in [21] that 40 is still true if \(\frac{E_n,n}{E_{n-1,n}}\) is replaced by \(\frac{E_r,n}{E_{r-1,n}}\) for any \(r \in \{1,\ldots,n\}\). To obtain such a result applying the method which was described above, we would have to give a respective extremal representation of any of the functions \(\frac{E_r,n}{E_{r-1,n}}\), \(r \in \{ 1,\ldots,n-1 \}\). We do not know such a representation. Might be, multilinear algebra cannot be applied here, compare with remarks at the end of the present section.

By Proposition 38 and Lemma 3 we get a generalization of Minkowski’s determinantal inequality. For \({\mathbf{t}}=(t_1,\ldots,t_n)^\top \in(0,+\infty)^n\) define a function \(\psi_{{\mathbf{t}}}\) by \(\psi_{{\mathbf{t}}}(\Delta)=\prod_{j=1}^n \lambda_{n-j+1}^{t_j}\) for \(\Delta \in {\mathcal{M}}^\geq\), together with a function \({g}_{{\mathbf{t}}}\) by \({g}_{{\mathbf{t}}}(x_1,\ldots,x_n) = \prod_{j=1}^n x_j^{t_j}\) for \((x_1,\ldots,x_n)^\top \in {\mathcal{D}}\). Let \(({g}_{{\mathbf{t}}})\tilde{\,\,}\) be the permutation invariant extension of \({g}_{{\mathbf{t}}}\) to \([0,+\infty)^n\).

If \({\mathbf{t}} \in (0,+\infty)^n \setminus {\mathcal{D}}_+\) the function \(({g}_{{\mathbf{t}}})\tilde{\,\,}\) is known to be not concave, cf. [7]. So it is not Schur concave. We do not know a description of the set of all \({\mathbf{t}} \in (0,+\infty)^n\) such that functions \(\psi_{{\mathbf{t}}}\) or \(({g}_{{\mathbf{t}}})\tilde{\,\,}\), respectively, are superadditive. In case \(n=2\) easy calculations show that these functions can be superadditive for some \({\mathbf{t}} \not\in {\mathcal{D}}_+\).

Let \(\phi \in {\mathcal{F}}\). The definition of \(\iota( \cdot ; {\mathcal{S}}_\phi)\) implies that \({\rm tr}(A \Delta A^*) \geq \iota(\Delta ; {\mathcal{A}}) = \iota(\Delta ; {\mathcal{A}})(\phi(A))^2\) for any \(A \in {\mathcal{S}}_\phi\). Since \((\phi(aA))^2 = a^2 (\phi(A))^2\) for \(a \in [0,+\infty)\), \(A \in {\mathcal{M}}\), the inequality \({\rm tr}(A \Delta A^*) \geq \iota(\Delta ; {\mathcal{A}})(\phi(A))^2\) is true for any \(A \in \mathcal{M}\) and any \(\Delta\). Consequently, a lower bound for \({\rm tr}(A \Delta A^*)\) is found for any given \(\Delta\), any \(A \in \mathcal{M}\). In particular, if \(\Delta = I\) then a lower bound for the Schatten \(2\)-norm of \(A\) can be derived. Another method is based on Lemma 22(ii). For example, if \({\mathbf{t}}= (1,\ldots,1)^\top\) and \(q= p / (p-2)\), Proposition 29 implies that for given \((x_1,\ldots,x_n)^\top \in (0,+\infty)^n\), the monotonicity of the function \(p \in (0,2) \to (\sum_{j=1}^n x_j^p)^{1/p}\) implies the monotonicity of the function \(q \in (-\infty,0) \to (\sum_{j=1}^n x_j^q)^{1/q}\).

Assertions concerning superadditivity of certain functions can also be derived by methods of multilinear algebra. Let \(P\) denote the orthogonal projection of the tensor product \({\mathbb{C}}^n \otimes {\mathbb{C}}^n\) onto the antisymmetric tensor product \({\mathbb{C}}^n \wedge {\mathbb{C}}^n\). For linear operators \(A,B \in {\mathbb{C}}^n\) define \(A \wedge B = P (A \otimes B) P\). Recall that \({\mathbb{C}}^n \wedge {\mathbb{C}}^n\) is an invariant subspace of \(A \wedge A\) and that the eigenvalues of the restriction of \(A \otimes A\) to \({\mathbb{C}}^n \wedge {\mathbb{C}}^n\) are exactly the numbers \(\{ \lambda_i (A)\lambda_k (A) : j,k \in\{ 1,\ldots,n \} \, , \,\, j < k \}\). Since \(\Delta_1 \otimes \Delta_2\) is non-negative Hermitian for \(\Delta_1,\Delta_2 \in {\mathcal{M}}^\geq\), the operator \(\Delta_1 \wedge \Delta_2\) is non-negative Hermitian as well.

Thus, \({\rm tr}(\tilde{A} ((\Delta_1+\Delta_2) \wedge (\Delta_1+\Delta_2)) \tilde{A}^* ) \geq {\rm tr}(\tilde{A} (\Delta_1 \wedge \Delta_1) \tilde{A}^*) + {\rm tr}(\tilde{A} (\Delta_2 \wedge \Delta_2) \tilde{A}^*)\) for any \(n^2 \times n^2\) matrix \(\tilde{A}\) implying the superadditivity of the function \(\Delta \in {\mathcal{M}}^\geq \to \inf \{ {\rm tr}(\tilde{A} (\Delta \wedge \Delta) \tilde{A}^*): \tilde{A} \in {\mathcal{A}} \}\) for arbitrary subsets \(\tilde{{\mathcal{A}}}\) of the set of \(n^2 \times n^2\) matrices. In particular, for the set \(\tilde{{\mathcal{A}}}\) of all \(n^2 \times n^2\) matrices satisfying the equality \({\rm tr}(P (A \otimes A) P) = {\rm tr}(A \wedge A)=1\), the function \(\frac{E_{n,n}}{E_{n-2,n}}\) is superadditive by Proposition 8(i) or by Proposition 29(ii). This result is weaker than the superadditivity of the function \(\Big( \frac{E_{n,n}}{E_{n-2,n}} \Big)^{1/2}\) established by Bullen and Marcus in [22]. Our method can be extended to the \(k\)-th antisymmetric tensor power or even to general symmetric classes, cf. [23]. The consequence of this observation has still to be explored.

6 Prediction of multivariate stationary sequences↩︎

Let \(K\) be a complex Hilbert space with inner product \((\cdot,\cdot)\) and corresponding norm \(\| \cdot \|\). For a matrix \(A=(a_{ik}) \in \mathcal{M}\) and vectors \(X=(x_1,\ldots,x_n)^\top\), \(Y=(y_1,\ldots,y_n)\) in \(K^n\) define the multiplication by \(AX=(\sum_{j=1}^n a_{1j}x_j,\ldots, \sum_{j=1}^n a_{nj} x_j)^\top\) and the Grammian \(\mathcal{M}\)-valued inner product \(G\) by \(G(X,Y)=((x_j,y_k))_{j,k \in \{ 1,\ldots,n \}}\) so that \(K^n\) has the structure of a left Hilbert \(\mathcal{M}\)-module. Recall that \[\label{f6461} G(AX,BY) = A G(X,Y) B^* \,\, {\rm for} \,\,A,B \in {\mathcal{M}}, \, X,Y \in K^n \, .\tag{41}\] Set \((X,Y)_n = {\rm tr}(G(X,Y)) = \sum_{j=1}^n (x_j,y_j)\) and denote by \(\| \cdot \|_n\) the norm defined by the scalar inner product \((\cdot,\cdot)_n\). The notions of a subspace or a submodule of \(K\) and \(K^n\), respectively, mean always norm-closedness of the selected linear subspaces. For simplicity \(\|x-y\|^2\) is called the distance of \(x,y \in K\), although the notion ‘squared distance’ would be correct. Similarly, \(\| X-Y \|_n^2\) is called the distance of \(X,Y \in K^n\).

Let \({\mathbf{X}} = (X(s)=(x_1(s),\ldots,x_n(s))^\top )_{s \in {\mathbb{Z}}}\) be an \(n\)-variate stationary sequence with values in \(K^n\). The subspace \(H={\rm c.l.h.}\{ x_j(s) : j=1,\ldots,n , \, s \in {\mathbb{Z}} \}\) is called the time domain of \(\mathbf{X}\). The submodule \(H^n\) is the closed \(\mathcal{M}\)-linear hull of all \(X(s)\) with \(s \in \mathbb{Z}\), and it is also called the time domain of \(\mathbf{X}\). The probabilistic interpretation of these notions can be seen if \(K\) is assumed to be the Hilbert space of square-integrable random variables with expectation \(0\) over a certain probability space.

For a non-empty subset \(\mathcal{O}\) of \({\mathbb{Z}} \setminus \{ 0 \}\) let \(H({\mathcal{O}})= {\rm c.l.h.} \{ x_j(s) : j= 1,\ldots,n, \, s \in {\mathcal{O}} \}\) be the observation domain based on observations at all points of the observation set \(\mathcal{O}\). Let \(P({\mathcal{O}})=P\) be the orthogonal projection of \(H\) onto \(H({\mathcal{O}})\), and let \({\mathbf{P}}({\mathcal{O}})={\mathbf{P}}\) be its \(n\)-fold inflation, \({\mathbf{P}}((x_1,\ldots,x_n)^\top) = (P(x_1),\ldots,P(x_n))^\top \in H^n\) for \((x_1, \ldots , x_n)^\top \in H^n\). Recall, that \(\mathbf{P}\) is the \(\mathcal{M}\)-linear Grammian orthogonal projection in \(H^n\) onto \(H({\mathcal{O}})^n\), i.e. \(G(X-{\mathbf{P}}(X),Y)=0\) for any \(X \in H^n\), \(Y \in H({\mathcal{O}})^n\). Therefore, \[\label{f6462} G(X-{\mathbf{P}}(X),X-{\mathbf{P}}(X) ) = \min \{ G(X-Y,X-Y) : Y \in H({\mathcal{O}})^n \}\tag{42}\] with respect to Loewner’s semiordering, what implies \[\| X-{\mathbf{P}}(X) \|_n^2 = \sum_{j=1}^n \| x_j-P(x_j) \|^2 = \min \left\{ \sum_{j=1}^n \| x_j-y_j \|^2 ; y \in H({\mathcal{O}}), j = 1,\ldots,n \right\}\] for any \(X=(x_1,\ldots,x_n)^\top \in H^n\).

A main goal of multivariate prediction is to determine the prediction error matrix \[\label{f6463} \Delta({\mathcal{O}}) = \Delta = G(X(0)-{\mathbf{P}}(X(0)), X(0)-{\mathbf{P}}(X(0))) \, .\tag{43}\] From 43 we can find that the distance \[\label{f6464} d_j({\mathcal{O}}) = d_j = \| x_j(0)-P(x_j(0)) \|^2\tag{44}\] of the element \(x_j(0)\) to the observation domain is the \(j\)-th element of the principal diagonal of \(\Delta\).

In practice one would sometimes be interested to predict a certain functional of \(X(0)\) instead of \(X(0)\) itself, and to compute the corresponding prediction error, cf. [24], [25]. The simplest problem of this type is the prediction of a linear combination of the elements of \(X(0)\) or, slightly more general, an \(\mathcal{M}\)-linear combination of these elements. From 41 , 42 , 43 and the \(\mathcal{M}\)-linearity of \(\mathbf{P}\) follows that \[\label{f6465} \begin{align} A \Delta A^* & = G(AX(0)-{\mathbf{P}}(A X(0)), AX(0)-{\mathbf{P}}(A X(0)) ) =\\ & \quad \quad \quad = \min \{ G(A X(0)-Y,A X(0)-Y) : Y \in H({\mathcal{O}})^n \}\\ \end{align}\tag{45}\] for \(A \in \mathcal{M}\) with respect to Loewner’s semiordering. However, recall that for arbitrary subsets \({\mathcal{A}} \subseteq \mathcal{M}\) an infimum of the set \(\{ A \Delta A^* : A \in {\mathcal{A}} \}\) might not exist with respect to Loewner’s semiordering. Therefore, motivated by the paper [1] we formulate the following prediction problem of Helson-Lowdenslager type:

Since \(d({\mathcal{O}} ; {\mathcal{A}}) = \inf \{ {\rm tr}(A \Delta({\mathcal{O}}) A^*): A \in {\mathcal{A}} \} = \iota ( \Delta({\mathcal{O}}) ; {\mathcal{A}})\) by 45 , one is lead to apply the results of the preceding sections to the problem (HL) and to give them a prediction theoretic meaning. In the sequel we discuss mainly consequences that are of particular interest in multivariate prediction theory. A convenient way is to partition a matrix \(A\) into its row vectors \[A = \left( \begin{array}{c} {\mathbf{a}}^{(1)} \\ \vdots \\ {\mathbf{a}}^{(n)} \end{array} \right) \in {\mathcal{M}} \, .\] Then \({\rm tr}(A \Delta A^*) = \sum_{j=1}^n \| {\mathbf{a}}^{(j)} X(0) - P({\mathbf{a}}^{(j)} X(0) )\|^2\) is the sum of distances of linear combinations of \({\mathbf{a}}^{(j)}X(0)\), \(j\in \{ 0,\ldots,n \}\), to the observation domain, and \(d\) is the minimum of these sums if \(A\) runs through the set \(\mathcal{A}\). A similar interpretation was given by Helson and Lowdenslager [1]. A more concrete result follows if we apply Proposition 7 with \(p=2\).

Setting \(k=1\) in the preceding proposition we get that the distance of the set \(\{ {\mathbf{a}}^\top X(0) : \| {\mathbf{a}}\|_2=1 \}\) to the observation domain equals to \(\lambda_n\). Since all vectors of the canonical orthonormal basis have Euclidean length \(1\) we have \(\lambda_n \leq \min \{ d_j : j \in \{ 1,\ldots,n \} \}\) what leads to the well-known fact that the smallest element on the principal diagonal of a non-negative Hermitian matrix is not smaller than the minimal eigenvalue of that matrix. It would be interesting to determine the distance of the set \(\{ {\mathbf{a}}^\top X(0) : {\mathbf{a}} \in {\mathcal{A}}_1 \}\) for other subsets \({\mathcal{A}}_1 \subseteq \mathcal{M}\). For instance, we do not know a solution if \({\mathcal{A}}_1\) is the set of all vectors whose \(l_p\)-norm equals to \(1\) for \(p \not= 2\). If in Proposition 49 the condition \(\sum_{j=1}^k \| {\mathbf{a}} \|^2=1\) is replaced by the condition \(\sum_{j=1}^k \| {\mathbf{a}}_j \|^2=k\) the corresponding distance \(d\) equals to \(k \lambda_n\). It is instructive to compare this fact with Ky Fan’s result, which claims that the minimal sum of the distances of \(\{ {\mathbf{a}}_j^\top X(0) : j \in \{ 1,\ldots,k \} \}\) equals to \(\sum_{j=1}^k \lambda_{n-j+1}\) if the set \(\{ {\mathbf{a}}_1,\ldots,{\mathbf{a}}_k \}\) covers all orthonormal systems of \(k\) vectors, cf. [7], [17].

To apply Proposition 8 and Example 11 note, that in case \({\mathcal{A}}_1\) is a subset of \({\mathbb{C}}^n\) and \({\mathcal{A}}_2\) is a subset of the set of complex-valued \((n-1) \times n\) matrices such that the zero matrix belongs to \({\mathcal{A}}_2\), then for \({\mathcal{A}} = \left\{ \left( \begin{array}{c} {\mathbf{a}}^\top \\ A_2 \end{array} \right) : {\mathbf{a}} \in {\mathcal{A}}_1, {\mathcal{A}}_2 \in {\mathcal{A}}_2 \right\}\) the prediction error \(d({\mathcal{O}} ; {\mathcal{A}})\) coincides with the distance of the set \(\{ {\mathbf{a}}^\top X(0) : {\mathbf{a}} \in {\mathcal{A}}_1 \}\) to \(H({\mathcal{O}})\). In particular, if \({\mathcal{A}}_1\) is the singleton \(\{ e_1 \}\) then \(d=d_1\). Furthermore, if \(r \in \{ 1,\ldots,n \}\) and \({\mathcal{A}}_1\) is the set of all vectors \({\mathbf{a}} = (a_1 ,\ldots, a_n ) \in {\mathbb{C}}^n\) with \[\label{f6466} \sum_{j=1}^r a_j = 1\tag{46}\] then \(d\) can be considered as the distance of the set of all linear combinations \(\sum_{j=1}^r a_j x_j(0)\) with coefficients satisfying 46 to the set \(H({\mathcal{O}}) + {\rm l.h.} \{ x_j(0) : j \in \{ r+1,\ldots,n \}\}\). The case \(r=1\) is of particular interest. Remembering Rozanov’s minimality definition [24] (cf. [26] for an overview on minimality notions of multivariate stationary sequences), we call the distances \[\label{f6467} \tilde{d}_k({\mathcal{O}}) = \tilde{d}_k = \inf \{ \| x_k(0)-y \|^2 : y \in H({\mathcal{O}}) + {\rm l.h.} \{ x_j(0) : j \not=k \}\} \, , \,\, k \in \{ 1,\ldots,n\}\tag{47}\] the Rozanov distances. Using \(\tilde{d}_k\) we can give a practicable invertibility criterion for the prediction error matrix.

Now, we can give Examples 11(i),(ii) the prediction theoretical form. Let \(r \in \{ 1,\ldots,n \}\) and recall that \(\delta_{jk}^{(\dagger)}\) denotes the \((j,k)\)-th element of \(\Delta^\dagger\).

Because \(\mu_r=0\) if \(\tilde{d}_j=0\) for some \(j \in \{ 1,\ldots,n \}\), we can derive a generalization of Proposition 50 from Proposition 51(i):

If \(\tilde{d}_j > 0\) for any \(j \in \{ 1,\ldots,r \}\) then we can show that \(\mu_r=(\sum_{j=1}^r \tilde{d}_j^{-1})^{-1}\) leading to another corollary of Proposition 50(i):

If \(r=1\) one has \(\tilde{d}_1=(\delta_{11}^{(\dagger)})^{-1}\) if \(e_1 \in {\mathcal{R}}(\Delta)\), and \(\tilde{d}_1=0\) otherwise. Thus, if \(\Delta \in {\mathcal{I}}\) we can say that \[\label{f6468} \tilde{d}_j = (\delta_{jj}^{(\dagger)})^{-1} = \frac{{\rm det}(\Delta)}{{\rm det}(\Delta_j)} \, ,\tag{48}\] where \(\Delta_j\) is obtained from \(\Delta\) deleting the \(j\)-th row and column from the matrix representation of \(\Delta\), and \(j\) takes any value of \(\{ 1,\ldots,n \}\). The interlacing theorem for eigenvalues of Hermitian matrices yields \[\lambda_n \leq \tilde{d}_j \leq \lambda_1 \, .\] Moreover, since \(\tilde{d}_j\) equals the \(j\)-th element of the principal diagonal of \(\Delta^{-1}\) we get the equality \(\sum_{j=1}^n \tilde{d}_j^{-1} = {\rm tr}(\Delta^{-1}) = \sum_{j=1}^n \lambda_j^{-1}\). So, there exist a certain index number \(k\) such that \(\tilde{d}_k\) is not larger than the harmonic mean of the eigenvalues of \(\Delta\).

The next result describes conditions under which \(\tilde{d}_j\) is equal to one of the extremal values \(\lambda_1\) or \(\lambda_n\):

The preceding proposition reveals that \(\tilde{d}_1\) is extremal if and only if knowledge of \(x_2(0),\ldots,x_n(0)\) does not improve the prediction. Let us mention, however, that the condition ?? does not mean that the sequence \((x_1(s))_{s \in {\mathbb{Z}}}\) is orthogonal to any of the sequences \((x_j(s))_{s \in {\mathbb{Z}}}\), \(j\in \{ 2,\ldots,n \}\), in general.

We finish the present section with another interpretation of \(d({\mathcal{O}};{\mathcal{A}})\). Let \(M\) be the (non-stochastic) spectral measure of the sequence \(\mathbf{X}\). Let \({\mathcal{T}}({\mathcal{O}})\) be the set of all \(\mathcal{M}\)-valued trigonometric polynomials of the form \(\sum_k A_k e^{{\mathbf{i}} s_k ( \cdot )}\) for \(A_k \in \mathcal{M}\), \(s_k \in \mathcal{O}\). According to the multivariate version of Kolmogorov’s isomorphism theorem we get \(d({\mathcal{O}},{\mathcal{A}}) = \inf \left\{ {\rm tr} \left( \int_0^{2 \pi} (A-F(\theta)) \, {\mathrm d}(M(\theta))(A-F(\theta))^* \right) : F \in {\mathcal{T}}({\mathcal{O}}), A \in {\mathcal{A}} \right\}\). If \(A\) is invertible then \(\inf \left\{ {\rm tr} \left( \int_0^{2 \pi} (A-F(\theta)) \, {\mathrm d}(M(\theta))(A-F(\theta))^* \right) : F \in {\mathcal{T}}({\mathcal{O}}) \right\} = \inf \left\{ {\rm tr} \left( \int_0^{2 \pi} (I-F(\theta)) \, A\, {\mathrm d}(M(\theta)) \, A^* (I-F(\theta))^* \right) : F \in {\mathcal{T}}({\mathcal{O}}), A \in {\mathcal{A}} \right\}\), and hence, \(d({\mathcal{O}} ; {\mathcal{A}}) = \inf \{ {\rm tr}(\Delta_A({\mathcal{O}})) : A \in ({\mathcal{A}} \cap {\mathcal{I}} ) \}\) in case \(({\mathcal{A}} \cap {\mathcal{I}})\) is dense in \(\mathcal{A}\) and \(\Delta_A({\mathcal{O}})\) denotes the prediction error matrix of the sequence \((A X(s))_{s \in {\mathbb{Z}}}\), according to Lemma 5(ii). Since in applications the modelling stationary sequence is estimated from observations (cf. [27]) there is some freedom of choice. Therefore, using the preceding results one could try to choose from a set \(\{ A X(s) \}_{s \in {\mathbb{Z}}}\), where \(A\) exhaustes a certain subset \(\mathcal{A}\) of \(\mathcal{M}\) in such a way that \(d({\mathcal{O}},{\mathcal{A}})\) is as small as possible.

References↩︎