1 Introduction↩︎

The Blahut–Arimoto (BA) algorithm [4], [5] computes rate-distortion functions and channel capacities via an iterative Gibbs-type update. Classical analysis  [4]–[6] proves convergence via monotone decrease of the free energy, without identifying the underlying dissipation mechanism. Nakagawa et al [7] established sharp \(O(1/n)\) rates for the discrete iteration. Hayashi [1]–[3] reformulated each BA step as a Bregman EM update, obtaining global convergence guarantees within an elegant information-geometric framework.

Despite this progress, two questions remain open. First, existing convergence constants are non-constructive: they depend on abstract geometric quantities and cannot be directly computed from the channel and temperature parameter \(\beta\). Second, the precise spectral mechanism governing the per-step Lyapunov decrease—and in particular its dependence on the non-idempotence of the underlying operator—has not been identified.

The present paper closes both gaps around a single operator:

\[\mathcal{G}= \mathbb{E}_p\bigl[K^*_X \otimes K^*_X\bigr]\]

the relaxation kernel, constructed from the equilibrium Gibbs conditionals \(K^*_x(y) = e^{-\beta d(x,y)}/Z^*_x\). This operator governs, within one unified framework, the continuous-time entropy production, the linearised discrete dynamics, the free-energy curvature, and the per-step Lyapunov dissipation.

1.1 Main Contributions↩︎

1. Exact \(\chi^2\)-dissipation identity (Section 3). Along the continuous-time BA flow \(\dot{q} = \mathcal{T}(q)-q\), \[\frac{d}{dt}F_\beta(q_t) = -\mathcal{D}(q_t), \qquad \mathcal{D}(q) = \chi^2(\mathcal{T}q \| q) = \sum_y \frac{(\mathcal{T}q(y)-q(y))^2}{q(y)}.\] This is an exact equality, not merely a bound.

2. Threefold identity of \(\mathcal{G}\) (Section 4). At an interior fixed point \(q^*\), \[\begin{align} \mathcal{G}&= \mathbb{E}_p[K^*_X \otimes K^*_X] && \text{(Gram / correlation matrix)},\\ DV(q^*) &= -\mathcal{G}&& \text{(linearised generator)},\\ \nabla^2_{\mathrm{FR}}F_\beta(q^*) &= \mathcal{G}&& \text{(Fisher--Rao Hessian)}. \end{align}\]

3. Exact one-step Lyapunov dissipation (Section 5). For the discrete BA iteration, \[\label{eq:dlambda95intro} \Delta\mathcal{L}^{(2)} = \sum_i c_i^2\, d(\lambda_i), \qquad d(\lambda)= -\lambda + \tfrac{3}{2}\lambda^2 - \tfrac{1}{2}\lambda^3,\tag{1}\] where \(\lambda_i\) are eigenvalues of \(\mathcal{G}\) and \(c_i = \langle h,\, e_i\rangle_{q^*}\). The non-idempotence of \(\mathcal{G}\) (i.e.\(\mathcal{G}^2 \neq \mathcal{G}\)) is what drives \(d(\lambda)<0\) for all \(\lambda\in(0,1)\); without non-idempotence the dissipation would vanish identically.

4. Two-stage global convergence (Section 7). \(\chi^2\)-dissipation drives finite-time entry into a neighbourhood of \(q^*\); thereafter, exponential contraction at rate \(\lambda_*\) takes over.

5. Explicit KL convergence factor (Section 8). For the discrete iterates \(q_{n+1} = \mathcal{T}(q_n)\), writing \(v_n = q_n - q^*\in T\): \[D_{\mathrm{KL}}(q^*\|q_{n+1}) \le (1-\lambda_*)^2\,D_{\mathrm{KL}}(q^*\|q_n) + O(\|v_n\|_*^3),\] with explicit \(\gamma = \lambda_*(2-\lambda_*)\), computable for every non-degenerate fixed point. Equality holds asymptotically when \(v_n\) is aligned with the \(\lambda_*\)-eigendirection of \(\mathcal{G}\).

1.2 Relation to Prior Work↩︎

1.2.0.1 Bregman and EM perspectives.

Hayashi [1]–[3] developed an influential framework in which each BA step is viewed as an alternating \(e\)/\(m\)-projection in a Bregman divergence system. This yields global monotone decrease of the free energy, an \(O(1/n)\) global convergence rate, and, locally, exponential convergence once the iterate enters a neighbourhood of the fixed point—though the exponential rate constant is not expressed in closed form. The present paper addresses complementary questions: it identifies the relaxation kernel \(\mathcal{G}\) as the central spectral object, derives the exact per-step Lyapunov dissipation formula \(d(\lambda)\), and makes the local convergence factor \(\gamma = \lambda_*(2-\lambda_*)\) explicit and computable from the channel and temperature. The two frameworks are therefore complementary: Hayashi’s provides global guarantees from any starting point; the spectral theory developed here provides fine-grained quantitative information in the local phase.

1.2.0.2 Classical convergence analysis.

Csiszár [6] characterised each BA step as an alternating \(I\)-projection, providing a discrete geometric picture. Nakagawa et al [7] proved sharp \(O(1/n)\) rates. Our continuous-time framework complements these results by providing the exact entropy production rate and an explicit local relaxation time.

1.2.0.3 Replicator and vector-field formulations.

Beretta and Pelillo [8] proposed a continuous vector flow for channel capacity and proved qualitative exponential convergence via a replicator structure. The present analysis refines their picture by identifying the explicit spectral relaxation rate \(\lambda_*\).

1.2.0.4 KL monotonicity.

Ramakrishnan et al [9] proved that \(D_{\mathrm{KL}}(q^*\|q_{n+1}) \le D_{\mathrm{KL}}(q^*\|q_n)\) for the quantum BA algorithm; our Corollary 4 upgrades this qualitative monotonicity to a quantitative formula with explicit factor \((1-\lambda_*)^2\).

1.3 Organisation↩︎

Section 2 defines the BA flow and weighted Hilbert space. Section 3 proves the \(\chi^2\)-dissipation identity. Section 4 establishes the threefold identity of \(\mathcal{G}\). Section 5 derives the exact one-step dissipation, analyses \(d(\lambda)\), and reports numerical validation. Section 6 proves local exponential convergence. Section 7 assembles the two-stage global theorem, including the uniform moment bound required for the energy argument. Section 8 gives the explicit KL convergence factor. Section 9 specialises to Gaussian sources. Section 10 gives exact solutions for discrete models. Section 11 concludes. Appendix 12 provides the complete proof of two-stage global exponential convergence with explicit entry time and convergence factor. Appendix 13 gives the full proof of the Gaussian attractor theorem, including moment control, Hermite spectral decay, weak convergence, and upgrade to total variation.

2 Setup↩︎

This section establishes the objects that appear throughout the paper. We define the BA operator and its continuous-time flow, identify the class of interior fixed points and the dual identity they satisfy, and introduce the weighted Hilbert space in which all subsequent analysis takes place. Readers familiar with the BA algorithm may wish to skim Sections 2.1–2.2 and focus on Section 2.3, which introduces the conditional correlation operator \(\mathcal{G}\) whose threefold identity is the central object of Section 4.

2.1 BA Operator and BA Flow↩︎

Let \(\mathcal{X},\mathcal{Y}\) be finite sets, \(|\mathcal{X}|=M\), \(|\mathcal{Y}|=N\), \(p\) a strictly positive probability distribution on \(\mathcal{X}\), \(d:\mathcal{X}\times\mathcal{Y}\to\mathbb{R}\) a distortion function, and \(\beta>0\) an inverse temperature. Throughout, \(\Delta(\mathcal{Y})\) denotes the probability simplex on \(\mathcal{Y}\) and \(\operatorname{int}\Delta(\mathcal{Y})\) its relative interior (all coordinates positive). The BA algorithm seeks to minimise the free energy \(F_\beta:\Delta(\mathcal{Y})\to\mathbb{R}\) defined in Section 3 over \(q\in\Delta(\mathcal{Y})\); its iterations act on the output marginal \(q\in\Delta(\mathcal{Y})\), while the joint channel is implicitly optimised at each step via the Gibbs kernel.

2.1.0.1 From the discrete iteration to the continuous flow.

The classical BA algorithm is the discrete iteration \(q_{n+1} = \mathcal{T}(q_n)\). Its fixed points \(\mathcal{T}(q^*) = q^*\) are precisely the KKT stationarity conditions for the rate-distortion optimisation problem \(\min_{q\in\Delta(\mathcal{Y})}F_\beta(q)\): setting the Lagrangian gradient to zero and eliminating the multiplier yields exactly the equation \((\mathcal{T}q)(y) = q(y)\) for all \(y\) [4], [10]. Thus the fixed-point condition is not an auxiliary construct but the optimality condition of the underlying variational problem.

The continuous-time flow studied in this paper is the natural continuous relaxation of the discrete iteration: \[\label{eq:BAflow} \dot{q}_t = \mathcal{T}(q_t) - q_t, \qquad q_0\in\operatorname{int}\Delta(\mathcal{Y}).\tag{3}\] This ODE has the same fixed points as the discrete map and can be understood as the Euler discretisation \(q_{n+1} - q_n = \mathcal{T}(q_n) - q_n\) with unit step size, or equivalently as the replicator-type gradient flow of \(F_\beta\) with respect to the Fisher–Rao metric on \(\operatorname{int}\Delta(\mathcal{Y})\) [8]. The continuous-time formulation is adopted here because it admits exact differential-equation tools—in particular the \(\chi^2\)-dissipation identity of Section 3—that are not directly available for the discrete iteration. All qualitative convergence conclusions transfer back to the discrete map via the free-energy monotonicity \(F_\beta(q_{n+1})\le F_\beta(q_n)\), which holds for both flows [4].

The simplex \(\Delta(\mathcal{Y})\) is forward-invariant under 3 : \(\sum_y\dot{q}_t(y) = \sum_y(\mathcal{T}q_t)(y) - \sum_y q_t(y) = 0\) preserves the sum-to-one constraint, and \(q_t(y)=0\) implies \((\mathcal{T}q_t)(y)\ge 0\), so no coordinate can become negative. The interior \(\operatorname{int}\Delta(\mathcal{Y})\) is likewise forward-invariant since \((\mathcal{T}q)(y)>0\) for all \(q\in\operatorname{int}\Delta(\mathcal{Y})\).

2.2 Fixed Points and the Dual Identity↩︎

2.3 Weighted Hilbert Space and the Relaxation Kernel↩︎

All perturbation analysis takes place in the tangent space to the simplex. Let \[T = \bigl\{u\in\mathbb{R}^\mathcal{Y}: \textstyle\sum_{y\in\mathcal{Y}} u(y)=0\bigr\}\] be the \((N-1)\)-dimensional subspace of zero-sum vectors, which is the tangent space to \(\Delta(\mathcal{Y})\) at any interior point.

We now introduce the three auxiliary quantities that enter the definition of \(\mathcal{G}\). Fix an interior fixed point \(q^*\) and a tangent perturbation \(h\in T\). Define:

• the log-perturbation \(\phi:\mathcal{Y}\to\mathbb{R}\) by \(\phi(y) = h(y)/q^*(y)\);

• the conditional mean \(m:\mathcal{X}\to\mathbb{R}\) by \(m(x) = \mathbb{E}_{K^*_x}[\phi] = \sum_{y\in\mathcal{Y}} K^*_x(y)\,\phi(y)\);

• the conditional residual \(r:\mathcal{Y}\times\mathcal{X}\to\mathbb{R}\) by \(r(y,x) = \phi(y) - m(x)\), so that \(\mathbb{E}_{K^*_x}[r(\cdot,x)]=0\) for every \(x\).

The quantity \(m(x)\) is the component of \(\phi\) visible to source symbol \(x\) through the equilibrium channel \(K^*_x\), and \(r(\cdot,x)\) is the residual invisible to \(x\).

Expanding the right-hand side of ?? , the \((y,y')\)-entry of \(\mathcal{G}\) as a matrix acting on \(\mathbb{R}^\mathcal{Y}\) is \[\label{eq:G95matrix} \mathcal{G}_{yy'} = \sum_{x\in\mathcal{X}} p(x)\,K^*_x(y)\,K^*_x(y'),\tag{5}\] which is the covariance of the random vector \(K^*_X\in\mathbb{R}^\mathcal{Y}\) under the source distribution \(p\). Equivalently, \(\mathcal{G}= \mathbb{E}_p[K^*_X \otimes K^*_X]\) where \(K^*_X\) is viewed as a \(\mathbb{R}^\mathcal{Y}\)-valued random variable. This representation immediately shows that \(\mathcal{G}\) is symmetric (\(\mathcal{G}_{yy'} = \mathcal{G}_{y'y}\)) and positive semidefinite (\(u^\top\mathcal{G}u = \sum_x p(x)\langle K^*_x, u\rangle^2\ge 0\)).

In the language of alternating projections [6], \(\mathcal{G}= \Pi^*\Pi\) where \(\Pi: \mathcal{H}_{q^*}\to \ell^2(\mathcal{X}, p)\) is the conditional expectation operator \((\Pi h)(x) = m(x) = \mathbb{E}_{K^*_x}[\phi]\), mapping tangent vectors \(h\in T\) to functions on \(\mathcal{X}\), and \(\Pi^*\) is its adjoint with respect to the inner products \(\langle\cdot,\cdot\rangle_*\) and \(\langle\cdot,\cdot\rangle_p\). The eigenvalues of \(\mathcal{G}\) therefore lie in \([0,1]\), and \(\mathcal{G}|_T\) has the same eigenvalues as \(\Pi^*\Pi|_T\).

For finite \(\mathcal{Y}\) these are automatic.

3 The Exact \(\chi^2\)-Dissipation Identity↩︎

The classical monotonicity \(F_\beta(q_{k+1})\le F_\beta(q_k)\) is well known, but the mechanism behind it has not been identified precisely. This section shows that the rate of decrease of the free energy along the continuous-time flow equals, exactly and at every instant, the Pearson \(\chi^2\)-divergence between \(\mathcal{T}q_t\) and \(q_t\). The derivation is short and rests on the envelope theorem; the result is stronger than a bound. We also record a corollary that will serve as the energy estimate in the global phase of the two-stage convergence argument (Section 7).

3.1 The Free Energy↩︎

The BA free energy is \[\label{eq:F} F_\beta(q) = \sum_x p(x)\log Z_q(x) + \frac{1}{\beta}\sum_y q(y)\log q(y).\tag{6}\] This equals \(\min_P \mathcal{F}_\beta(P;q)\) where \(\mathcal{F}_\beta(P;q) = I_P(X;\hat{X}) + \beta\mathbb{E}_P[d(X,\hat{X})]\) is the standard rate-distortion auxiliary functional, and the minimum is achieved at \(P(\hat{x}|x) = K_q(x,\hat{x})\).

4 The Relaxation Kernel: Threefold Identity↩︎

With the dissipation mechanism in hand, we turn to the operator that governs it. The relaxation kernel \(\mathcal{G}= \mathbb{E}_p[K^*_X\otimes K^*_X]\) was introduced in Definition 4 as a purely algebraic object. This section shows that it in fact plays three distinct and individually natural roles: it is simultaneously the Gram matrix of equilibrium correlations, the linearised generator of the BA flow, and the Fisher–Rao Hessian of the free energy. The identification of these three roles in a single operator is what makes \(\mathcal{G}\) the natural centre of the spectral theory developed in the subsequent sections.

4.1 Linearisation at Equilibrium↩︎

Define the BA vector field \(V:\operatorname{int}\Delta(\mathcal{Y})\to T\) by \(V(q) = \mathcal{T}(q) - q\), so that the BA flow 3 reads \(\dot{q}_t = V(q_t)\). Since \(\mathcal{T}\) maps \(\operatorname{int}\Delta(\mathcal{Y})\) into itself and \(\sum_y(\mathcal{T}q)(y) = \sum_y q(y) = 1\), the difference \(V(q)\) lies in the tangent space \(T\) for every \(q\). The Fréchet derivative \(DV(q^*): T\to T\) is a linear map from \(T\) to itself.

4.2 Gram Structure↩︎

The representation \(\mathcal{G}= \mathbb{E}_p[K^*_X\otimes K^*_X]\) in 5 identifies \(\mathcal{G}\) as the second moment (Gram) matrix of the \(\mathbb{R}^\mathcal{Y}\)-valued random vector \(K^*_X\) under the source distribution \(p\). Here \(K^*_X\otimes K^*_X\) denotes the \(N\times N\) matrix with \((y,y')\)-entry \(K^*_X(y)\,K^*_X(y')\), and the expectation is taken coordinatewise. This representation shows that \(\mathcal{G}\) encodes the equilibrium correlation structure among output symbols: \(\mathcal{G}_{yy'}\) is the covariance (under \(p\)) between the Gibbs probability of outputting \(y\) and the Gibbs probability of outputting \(y'\), when the source symbol is drawn from \(p\).

4.3 Identification with the Fisher–Rao Hessian↩︎

The Fisher–Rao Hessian of \(F_\beta\) at \(q\) is the symmetric bilinear form \[\nabla^2_{\mathrm{FR}}F_\beta(q)[h_1,h_2] := \left.\frac{d^2}{d\varepsilon\,d\eta}\right|_{\varepsilon=\eta=0} F_\beta(q+\varepsilon h_1 + \eta h_2).\]

Theorem 5 completes the threefold identity: \(\mathcal{G}\) is (1) Gram matrix of equilibrium correlations, (2) linearised generator of the BA flow, and (3) Fisher–Rao Hessian of the free energy.

4.4 Temperature Asymptotics of the Spectral Gap↩︎

5 One-Step Lyapunov Dissipation and Non-Idempotence↩︎

We now turn to the discrete BA iteration and ask how much the Lyapunov functional decreases in a single step. The answer is given by a spectral formula involving the function \(d(\lambda) = -\lambda + \frac{3}{2}\lambda^2 - \frac{1}{2}\lambda^3\), which encodes the effect of the non-idempotence of \(\mathcal{G}\) on the per-step dissipation. Two features of \(d\) deserve attention before the formal analysis: first, \(d(\lambda)<0\) for all \(\lambda\in(0,1)\), confirming strict decrease; second, \(d\) vanishes at both endpoints \(\lambda=0\) and \(\lambda=1\), revealing the double-bottleneck structure discussed in Section 5.4. The section closes with a numerical example on a binary channel that illustrates the spectral dissipation formula concretely.

We now turn to the discrete BA iteration and analyse the one-step decrease of the Lyapunov functional \[\label{eq:Lyap} \mathcal{L}(q) = \sum_x p(x)\,D_{\mathrm{KL}}\bigl(K^*(\cdot|x)\;\|\;K_q(\cdot|x)\bigr).\tag{7}\] This functional satisfies \(\mathcal{L}(q)\ge 0\) with equality iff \(q=q^*\), and decreases along the BA iteration [4].

5.1 Taylor Expansion of the Lyapunov Functional↩︎

Fix an interior fixed point \(q^*\) and write \(q = q^*+h\) with \(h\in T\) small. For each \(x\in\mathcal{X}\), the Gibbs kernel at \(q\) satisfies \[K_q(x,y) = K^*_x(y)\,\frac{1 + \varepsilon r(y,x) + \tfrac12\varepsilon^2(r(y,x)^2 - \sigma^2(x)) + O(\varepsilon^3)}{1},\] where \(\varepsilon\sim\|h\|_*\), \(r(y,x) = \phi(y) - m(x)\) is the conditional residual introduced in Section 2 (a centred function of \(y\) under \(K^*_x\)), \(\sigma^2(x) = \mathbb{E}_{K^*_x}[r(\cdot,x)^2]\) is its conditional variance, and \(\mu_3(x) = \mathbb{E}_{K^*_x}[r(\cdot,x)^3]\) is the conditional skewness. Substituting into \(\mathcal{L}(q) = -\sum_x p(x)\mathbb{E}_{K^*_x}[\log(K_q(x,\cdot)/K^*_x(\cdot))]\) and expanding the logarithm, one finds: \[\label{eq:L95expand} \mathcal{L}(q^*+h) = \underbrace{\tfrac12\mathbb{E}_p[\sigma^2(x;h)]}_{\mathcal{L}^{(2)}} + \underbrace{\tfrac16\mathbb{E}_p[\mu_3(x;h)]}_{\mathcal{L}^{(3)}} + \underbrace{\tfrac{1}{24}\mathbb{E}_p[\kappa_4(x;h)]}_{\mathcal{L}^{(4)}} + O(\|h\|^5),\tag{8}\] where \(\kappa_4(x;h)=\mathbb{E}_{K^*_x}[r(\cdot,x)^4]-3\sigma^4(x;h)\) is the conditional excess kurtosis, and all expectations are over \(y\sim K^*_x\). The notation \(\sigma^2(x;h)\) makes explicit the dependence on \(h\) through \(r(y,x) = h(y)/q^*(y) - m(x;h)\).

The quadratic leading term can be expressed directly in terms of \(\mathcal{G}\): \[\label{eq:L295Fisher} \mathcal{L}^{(2)}(h) = \tfrac12\mathbb{E}_p[\sigma^2(x;h)] = \tfrac12\bigl(\|h\|_*^2 - \langle h,\, \mathcal{G}h\rangle_{q^*}\bigr) = \tfrac12\langle h,\, (I-\mathcal{G})h\rangle_{q^*}.\tag{9}\] To see this, note \(\|h\|_*^2 = \sum_y h(y)^2/q^*(y) = \mathbb{E}_{q^*}[\phi^2]\) and \(\langle h,\, \mathcal{G}h\rangle_{q^*} = \mathbb{E}_p[m(x)^2] = \mathbb{E}_p[\mathbb{E}_{K^*_x}[\phi]^2]\); then \(\mathbb{E}_p[\sigma^2] = \mathbb{E}_{q^*}[\phi^2] - \mathbb{E}_p[m^2]\) is the law of total variance.

5.2 Exact Second-Order Dissipation Formula↩︎

Write the image of \(q^*+h\) under \(\mathcal{T}\) as \[\mathcal{T}(q^*+h) = q^* + h', \quad h' = h^{(1)} + h^{(2)} + O(\|h\|^3),\] where the terms are determined by expanding \(\mathcal{T}\) around \(q^*\):

• \(h^{(1)} = D\mathcal{T}(q^*)\,h = (I-\mathcal{G})h\) is the first-order term (Theorem 3);

• \(h^{(2)} = \tfrac{1}{2}D^2\mathcal{T}(q^*)[h,h] \in T\) is the second-order term arising from the curvature of \(\mathcal{T}\) at \(q^*\); it is a symmetric bilinear function of \(h\) with itself, and lies in \(T\) since \(\mathcal{T}\) maps into \(\Delta(\mathcal{Y})\).

The second-order one-step Lyapunov decrement is \[\Delta\mathcal{L}^{(2)} := \mathcal{L}^{(2)}(h') - \mathcal{L}^{(2)}(h),\] which measures the change in the quadratic part of \(\mathcal{L}\) after one BA step. Through order \(\|h\|^2\), only \(h^{(1)}\) contributes.

5.3 Spectral Decomposition and the Function \(d(\lambda)\)↩︎

Since \(\mathcal{G}:\mathcal{H}_{q^*}\to\mathcal{H}_{q^*}\) is a self-adjoint operator on the \((N-1)\)-dimensional Hilbert space \((T,\langle\cdot,\cdot\rangle_*)\), the spectral theorem provides an orthonormal eigenbasis \(\{e_i\}_{i=1}^{N-1}\subset T\) with \(\langle e_i, e_j\rangle_* = \delta_{ij}\) and corresponding eigenvalues \(\lambda_i\in[0,1]\) satisfying \(\mathcal{G}e_i = \lambda_i e_i\). The eigenvalues lie in \([0,1]\) because \(\mathcal{G}= \Pi^*\Pi\) implies \(\langle u, \mathcal{G}u\rangle_* = \|\Pi u\|_p^2 \ge 0\) and \(\langle u, \mathcal{G}u\rangle_* \le \|u\|_*^2\) (since \(\Pi\) is a contraction). Any \(h\in T\) decomposes as \(h = \sum_i c_i e_i\) with \(c_i = \langle h, e_i\rangle_*\).

5.4 Properties of \(d(\lambda)\) and Non-Idempotence Correction↩︎

This correction reflects the curvature of \(\mathcal{T}\) at \(q^*\): the non-idempotence of \(\mathcal{G}\) means that the image of the linearised step \((I-\mathcal{G})h\) has strictly smaller Fisher–Rao norm than predicted by the first-order eigenvalue alone.

5.5 A Concrete Numerical Example↩︎

We illustrate the spectral dissipation formula on the simplest non-trivial instance: a binary channel with asymmetric source. All quantities are computed exactly and can be verified by hand.

5.5.0.1 Setup.

Let \(\mathcal{X}= \mathcal{Y}= \{0,1\}\), \(p = (0.6,\,0.4)\), Hamming distortion \(d(x,y) = \mathbf{1}_{\{x\ne y\}}\), and \(\beta = 1\). Set \(r = e^{-\beta} = e^{-1} \approx 0.3679\).

5.5.0.2 Fixed point.

The BA operator acts on \(q = (q_0, q_1)\in\operatorname{int}\Delta(\mathcal{Y})\) via \[(\mathcal{T}q)(0) = p_0\,\frac{q_0}{q_0 + r q_1} + p_1\,\frac{r q_0}{r q_0 + q_1}.\] Setting \((\mathcal{T}q^*)(0) = q^*_0\) and solving gives \[q^* = (0.7163953414,\; 0.2836046586),\] which satisfies \((\mathcal{T}q^*)(0) = q^*_0 = 0.7163953414\) exactly.

5.5.0.3 Equilibrium Gibbs kernels.

With partition functions \(Z^*_0 = q^*_0 + r q^*_1\) and \(Z^*_1 = r q^*_0 + q^*_1\): \[K^*_{x=0} = \bigl(0.8728782667,\; 0.1271217333\bigr), \qquad K^*_{x=1} = \bigl(0.4816709534,\; 0.5183290466\bigr).\]

5.5.0.4 Eigenvalue of \(\mathcal{G}\).

Since \(|\mathcal{Y}|=2\), the tangent space \(T\) is one-dimensional, spanned by \(h = (1,-1)\). The log-perturbation is \(\phi = h/q^*\), and the conditional means are \[m(x=0) = \mathbb{E}_{K^*_{x=0}}[\phi] \approx 0.7702, \qquad m(x=1) = \mathbb{E}_{K^*_{x=1}}[\phi] \approx -1.1553.\] Computing \(\langle h,\mathcal{G}h\rangle_* = 0.8898\) and \(\|h\|_*^2 = 4.9219\) via Definition 4, the unique eigenvalue of \(\mathcal{G}|_T\) is \[\lambda = \frac{\langle h,\mathcal{G}h\rangle_*}{\|h\|_*^2} = \frac{0.8898}{4.9219} = 0.1808.\] Note that this is not the off-diagonal matrix element \(\mathcal{G}_{01}\), but the Rayleigh quotient of \(\mathcal{G}\) in the Fisher–Rao metric.

5.5.0.5 Spectral dissipation.

The exact formula of Corollary 3 gives \[d(\lambda) = -\lambda + \tfrac{3}{2}\lambda^2 - \tfrac{1}{2}\lambda^3 = -0.1808 + 1.5\times 0.0327 - 0.5\times 0.0059 = -0.1347.\] The correction due to non-idempotence (Proposition 12) accounts for \(0.1808 - 0.1347 = 0.0461\), which is \(\frac{1}{2}\lambda^2(3-\lambda) = 0.5\times 0.0327\times 2.8192 \approx 0.0461\), consistent with the analytic formula. The KL convergence factor for this channel is \(\gamma = \lambda_*(2-\lambda_*) = 0.1808\times 1.8192 \approx 0.329\), meaning approximately \(33\%\) of the KL gap is closed per iteration in the local phase.

5.6 Third-Order Term and Cubic Cancellation↩︎

The third-order one-step decrement is \[\label{eq:DeltaL3} \Delta\mathcal{L}^{(3)} = 2B(h^{(1)}, h^{(2)}) + \bigl[C(h^{(1)}) - C(h)\bigr],\tag{10}\] where \(B\) is the bilinear form of \(\mathcal{L}^{(2)}\) and \(C(h) = \frac{1}{6}\mathbb{E}_p[\mu_3(h)]\). If \(\mu_3(x;h)=0\) for all \(x\) (conditional symmetry), then \(C(\cdot)\equiv 0\) and the bracket in 10 vanishes, but the term \(2B(h^{(1)},h^{(2)})\) survives because \(h^{(2)}\) encodes conditional variances, not skewness. Thus:

This contrasts with a different notion of “cubic cancellation”—the vanishing of the third-order jet \(\langle h, D^2\mathcal{T}^*(h,h)\rangle_*\) when \(\mu_3=0\) [14]—which refers to the map expansion rather than the Lyapunov decrement.

6 Spectral Gap and Local Exponential Convergence↩︎

The spectral dissipation formula of Section 5 shows that convergence is governed, in the slowest directions, by the smallest eigenvalue of \(\mathcal{G}\) on the tangent space. This section makes that connection precise by proving local exponential contraction in a neighbourhood of a non-degenerate fixed point. The rate is exactly \(\lambda_*= \lambda_{\min}(\mathcal{G}|_T)\), and the proof is a direct application of Gronwall’s inequality to the Lyapunov function \(\mathcal{E} = \frac{1}{2}\|q - q^*\|_*^2\). These local results are the second ingredient in the two-stage global argument assembled in Section 7.

The variational form 11 is transparent: \(\lambda_*\) is the minimum variance of the projections \(\langle K^*_x,u\rangle\) over all unit tangent directions. Non-degeneracy means the equilibrium kernels \(\{K^*_x\}\) span the full tangent hyperplane.

7 Two-Stage Global Convergence↩︎

The \(\chi^2\)-dissipation identity and the local spectral contraction are individually useful but each has a limited scope: the dissipation identity is global but does not itself imply exponential decay, while the spectral contraction is exponential but only local. This section joins the two by a finite-entry argument: the dissipation identity guarantees that the BA residual \(\|\mathcal{T}q_t - q_t\|_1\) becomes small in finite time \(T_*\), after which the bi-Lipschitz equivalence of Theorem 14(A) places the trajectory in the exponential basin. The complete proof with explicit constants for the entry time and convergence factor is given in Appendix 12.

The argument requires that the free energy \(F_\beta\) is bounded below along the trajectory. For finite \(\mathcal{Y}\) this is immediate from continuity and compactness of \(\Delta(\mathcal{Y})\). For continuous alphabets with quadratic distortion, the following lemma provides the necessary control.

8 Explicit KL Convergence Factor↩︎

The two-stage theorem establishes exponential convergence, but leaves the convergence factor implicit. This section makes it explicit for the discrete BA iteration \(q_{n+1} = \mathcal{T}(q_n)\), \(n = 0, 1, 2, \ldots\) . Write \(v_n = q_n - q^* \in T\) for the signed deviation of the \(n\)-th iterate from the fixed point; this is a well-defined element of the tangent space \(T\) whenever \(q_n \in \operatorname{int}\Delta(\mathcal{Y})\). In the local phase, the KL divergence \(D_{\mathrm{KL}}(q^*\|q_n)\) contracts by a factor \((1-\lambda_*)^2\) per iteration, up to cubic corrections in \(\|v_n\|_*\). The improvement factor \(\gamma = \lambda_*(2-\lambda_*)\) is a simple function of the spectral gap and can be computed directly from the channel and temperature. This upgrades the qualitative monotonicity of Ramakrishnan et al [9] to a quantitative rate.

9 Gaussian Sources: Exact Solution↩︎

The abstract theory of the preceding sections takes its sharpest form when specialised to Gaussian sources with quadratic distortion. In this case the spectral gap is \(\lambda_*= 1/(2\beta\sigma^2)\), the Jacobian is diagonalised by Hermite polynomials, and the convergence factor \(\gamma = \lambda_*(2-\lambda_*)\) is a simple explicit function of the inverse temperature and source variance. The section also identifies critical slowing down as \(\beta\) approaches the rate-distortion threshold \(1/(2\sigma^2)\), a phenomenon that emerges naturally from the vanishing of \(\lambda_*\).

When the source is Gaussian and distortion is quadratic, the general theory achieves its sharpest quantitative form.

9.1 Setup↩︎

Let \(X\sim\mathcal{N}(0,\sigma^2)\) and \(d(x,\hat{x})=(x-\hat{x})^2\). The BA operator becomes \[\label{eq:Gauss95BA} \mathcal{T}(q)(\hat{x}) = \int \frac{e^{-\beta(x-\hat{x})^2}q(\hat{x})}{\int e^{-\beta(x-\hat{y})^2}q(\hat{y})\,d\hat{y}}\, \frac{e^{-x^2/(2\sigma^2)}}{\sqrt{2\pi\sigma^2}}\,dx.\tag{13}\] For a Gaussian initial distribution \(q_0 = \mathcal{N}(0,s_0)\), \(q_t\) remains Gaussian under the BA flow, and its variance \(s(t) = \mathbb{E}_{q_t}[\hat{X}^2]\) satisfies a closed ODE.

9.2 Hermite Spectral Decomposition↩︎

9.3 Spectral Dissipation for the Gaussian Case↩︎

At the Gaussian fixed point, the \(\mathcal{G}\)-eigenvalues are \(\lambda_n = 1-\alpha^n\in(0,1)\). The spectral function \(d(\lambda_n) = -\lambda_n + \frac{3}{2}\lambda_n^2 - \frac{1}{2}\lambda_n^3\) is negative for all \(n\ge 1\), confirming strict dissipation. The optimal dissipation occurs at the eigenvalue closest to \(\lambda_{\mathrm{opt}}\approx 0.423\). Since \(\lambda_n = 1-\alpha^n\) is increasing in \(n\), the optimal dissipation index is \(n^* = \lfloor \log(1-\lambda_{\mathrm{opt}})/ \log\alpha\rfloor\).

The KL convergence factor is explicitly \[\gamma = \lambda_*(2-\lambda_*) = \frac{1}{2\beta\sigma^2}\!\left(2 - \frac{1}{2\beta\sigma^2}\right),\] which increases from \(0\) (at threshold) toward \(2-1 = 1\) (as \(\beta\to\infty\)).

9.4 Gaussian as Dynamical Attractor↩︎

The Gaussian distribution emerges here as a dynamical consequence, not a variational assumption. The BA flow is a spectral filter suppressing non-Gaussian cumulants hierarchically.

10 Low-Dimensional Exact Models↩︎

The two-point and three-cluster models below serve a dual purpose. They provide closed-form expressions for \(\lambda_*\), making the spectral gap directly verifiable, and they illustrate the two-stage convergence structure in settings simple enough to be analysed by hand. Both examples confirm the general theory and offer concrete parameter dependence that may guide intuition for larger channels.

10.1 Two-Point Source↩︎

Let \(\mathcal{X}=\mathcal{Y}=\{0,1\}\), \(p = (p_0,p_1)\), and \(d(x,y)=\mathbf{1}_{x\ne y}\). The BA operator with \(\beta>0\) reduces to a scalar equation in \(q = q(1)\in(0,1)\). The fixed point satisfies \[q^* = \frac{p_0 e^{-\beta\cdot 0} \cdot q^*}{Z_0(q^*)} + \frac{p_1 e^{-\beta\cdot 1}\cdot q^*}{Z_1(q^*)},\] and the spectral gap is \[\lambda_*= p_0 K^*_0(0)K^*_0(1) + p_1 K^*_1(0)K^*_1(1)\cdot(-1)^2\] (sum of squared equilibrium kernels evaluated on the single tangent direction). This gives \(\lambda_*\) as an explicit function of \(p\) and \(\beta\), with \(\lambda_*\to 0\) at the hard-decision boundary \(\beta\to\infty\).

10.2 Three-Cluster Source↩︎

Let \(\mathcal{X}= \{1,2,3\}\), \(\mathcal{Y}= \{1,2,3\}\), uniform \(p\), and distortion \(d(x,y) = 0\) if \(x=y\), \(d(x,y) = 1\) otherwise. This symmetric model has a uniform fixed point \(q^* = (1/3,1/3,1/3)\). By symmetry, the spectral gap of the two-dimensional tangent space is \[\lambda_*= \frac{2 e^{-\beta}}{(2e^{-\beta}+1)^2} = \frac{2r}{(2r+1)^2}, \qquad r = e^{-\beta}.\] For this model, the two-stage structure is clearly visible in simulation: from a highly asymmetric initial distribution, the global \(\chi^2\)-phase redistributes mass over a timescale \(O(1/\delta^2)\) (where \(\delta\) is the initial asymmetry), after which exponential contraction at rate \(\lambda_*\) takes over.

11 Conclusion↩︎

We have developed a unified spectral theory for BA dynamics, centred on the relaxation kernel \(\mathcal{G}= \mathbb{E}_p[K^*_X\otimes K^*_X]\).

Summary of results.

1. The exact \(\chi^2\)-dissipation identity ?? reveals the precise entropy production mechanism of the continuous-time flow.

2. The threefold identity (Theorems 3 and 5) unifies statistical, operator-theoretic, and information-geometric views of \(\mathcal{G}\).

3. The spectral dissipation formula ?? with \(d(\lambda) = -\lambda+\frac{3}{2}\lambda^2-\frac{1}{2}\lambda^3\) quantifies the exact one-step Lyapunov decrease and reveals the double bottleneck.

4. The two-stage global convergence theorem combines finite-time basin entry (from \(\chi^2\)-dissipation) with local exponential contraction (from \(\lambda_*\)).

5. The explicit KL convergence factor \(\gamma = \lambda_*(2-\lambda_*)\) upgrades Ramakrishnan et al’s monotonicity to a constructive rate.

Relation to prior work. The results of this paper are intended as a complement to the frameworks of Hayashi [1]–[3] and Nakagawa et al [7]. Hayashi’s Bregman-EM analysis establishes elegant global convergence guarantees and an \(O(1/n)\) rate from any initial condition; the present contribution identifies the relaxation kernel \(\mathcal{G}\) as the central spectral object and makes the local convergence factor \(\gamma = \lambda_*(2-\lambda_*)\) explicit and computable from the channel and temperature. Nakagawa et al’s sharp \(O(1/n)\) rates govern the transient phase before the local exponential phase described here. Together, these results give a more complete quantitative picture of the full BA trajectory: global sublinear approach, finite-time entry, and explicit exponential contraction.

Algorithmic implications. The spectral theory developed here has direct consequences for the design and acceleration of BA-type algorithms. The double-bottleneck structure of \(d(\lambda)\), the explicit dependence of the convergence factor on \(\lambda_*\), and the two-stage entry mechanism all suggest concrete strategies for improving the iteration in practice. These algorithmic questions—including spectral preconditioning, momentum-based acceleration targeting the \(\lambda\approx 0\) directions, and online estimation of the spectral gap during the global phase—are taken up in my companion paper currently in preparation.

Acknowledgements↩︎

This research received no formal funding and was conducted based on the author’s independent academic interest.

12 Two-Stage Global Exponential Convergence↩︎

This appendix provides the complete proof of the two-stage global exponential convergence theorem for the discrete BA iteration, with all constants made explicit. The argument assembles three ingredients from the main text: the \(\chi^2\)-dissipation identity (Theorem 1), the uniform moment bound (Lemma 3), the bi-Lipschitz equivalence of BA residual and distance to the fixed point (Theorem 14(A)), and the spectral contraction (Theorem 14(B)).

Statement↩︎

Global Phase: Finite Entry Time↩︎

Local Phase: Spectral Contraction with Remainder Absorption↩︎

For \(n \ge N_*\), write \(v_n = q_n - q^* \in T\). The BA iteration gives \[v_{n+1} = \mathcal{T}(q^*+v_n) - q^* = (I - \mathcal{G})v_n + R(v_n),\] where \(R(v_n) = \mathcal{T}(q^*+v_n) - q^* - D\mathcal{T}(q^*)v_n \in T\) is the nonlinear remainder satisfying \(\|R(v_n)\|_* \le L\|v_n\|_*^2\). The linear term \((I-\mathcal{G})v_n\) satisfies \[\|(I-\mathcal{G})v_n\|_* \le \|(I-\mathcal{G})|_T\|_{\mathrm{op}}\|v_n\|_* = (1-\lambda_*)\|v_n\|_*,\] because the operator norm of \((I-\mathcal{G})\) on \((T,\langle\cdot,\cdot\rangle_*)\) equals \(1 - \lambda_{\min}(\mathcal{G}|_T) = 1-\lambda_*\) (since eigenvalues of \(I-\mathcal{G}\) are \(1-\lambda_i\in[0,1-\lambda_*]\)).

By induction, for all \(n \ge N_*\): \[\label{eq:vn95decay} \|v_n\|_* \le \left(1 - \frac{3\lambda_*}{4}\right)^{n-N_*}\|v_{N_*}\|_*.\tag{14}\]

KL Bound via Fisher–Rao Expansion↩︎

Assembly and Proof of Theorem 26↩︎

Combining 14 and Lemma 6: \[\begin{align} D_{\mathrm{KL}}(q^*\|q_n) &\le \frac{1+M\rho}{2}\|v_n\|_*^2 \le \frac{1+M\rho}{2} \left(1-\frac{3\lambda_*}{4}\right)^{2(n-N_*)}\|v_{N_*}\|_*^2. \end{align}\] Similarly, \(D_{\mathrm{KL}}(q^*\|q_{N_*}) \ge \frac{1-M\rho}{2}\|v_{N_*}\|_*^2\) (lower bound from the same Taylor expansion), so \[\|v_{N_*}\|_*^2 \le \frac{2}{1-M\rho}\,D_{\mathrm{KL}}(q^*\|q_{N_*}).\] Setting \[C(q_0) = \frac{1+M\rho}{1-M\rho}\,D_{\mathrm{KL}}(q^*\|q_{N_*}),\] which is finite since \(M\rho < 1\) for \(\rho = \lambda_*/(4L)\) sufficiently small, completes the proof. 0◻

Relation to Hayashi’s Global Result↩︎

Hayashi [2] establishes global \(O(1/n)\) convergence for the discrete BA iteration via the Bregman-EM framework, a result that is both powerful and applicable from the very first iteration. Theorem 26 provides a complementary perspective: after the finite entry time \(N_*\), the iteration converges exponentially with an explicit rate determined by \(\lambda_*\).

The two frameworks address complementary aspects of the convergence problem: Hayashi’s [2] global \(O(1/n)\) result governs the full trajectory from any initial condition, while Theorem 26 provides an explicit exponential rate for the asymptotic phase \(n \ge N_*\). The entry time \(N_*\) and convergence factor \((1-3\lambda_*/4)^2\) are both determined by \(\lambda_*\) and the initial free energy gap, giving a complete and quantitative picture of the trajectory.

Explicit Constants for Gaussian Sources↩︎

For \(X\sim\mathcal{N}(0,\sigma^2)\) with quadratic distortion and \(\beta > 1/(2\sigma^2)\), the spectral gap is \(\lambda_*= 1/(2\beta\sigma^2)\) (Theorem 23). The convergence factor is \[\left(1-\frac{3\lambda_*}{4}\right)^2 = \left(1 - \frac{3}{8\beta\sigma^2}\right)^2,\] and the entry time bound becomes \[N_*(q_0) \le \left\lceil\frac{128\,\beta^2\sigma^4 L^2 (F_\beta(q_0)-F_\beta(q^*))}{c_1^2}\right\rceil.\] As \(\beta\to 1/(2\sigma^2)^+\), the factor \((1-3\lambda_*/4)^2\to 1\) and the entry time diverges, consistent with critical slowing down at the rate-distortion threshold (Remark following Theorem 23).

13 Proof of the Gaussian Attractor Theorem↩︎

This appendix provides a complete proof of Theorem 25. Throughout, we consider the continuous-time BA flow under quadratic distortion \(d(x,\hat{x})=|x-\hat{x}|^2\), Gaussian source \(X\sim\mathcal{N}(0,\sigma^2)\), and inverse temperature \(\beta>1/(2\sigma^2)\). We write \(q_t\) for the evolving reproduction law and \(q_* = \mathcal{N}(0,s^*)\) for the Gaussian fixed point of Theorem 20. The goal is to prove \(\|q_t - q_*\|_{\mathrm{TV}}\to 0\) as \(t\to\infty\).

Uniform Moment Bound↩︎

Hence the family \(\{q_t\}_{t\ge 0}\) is tight in \(\mathcal{P}(\mathbb{R})\).

Hermite Expansion↩︎

Fix the equilibrium measure \(q_* = \mathcal{N}(0,s^*)\) and let \(\{H_n\}_{n\ge 0}\) denote the orthonormal Hermite polynomial basis in \(L^2(q_*)\). Since \(q_t \ll q_*\) for all \(t > 0\), define the density ratio \(f_t = dq_t/dq_*\) and expand \[f_t = 1 + \sum_{n=1}^{\infty} a_n(t)\,H_n, \qquad a_n(t) = \langle f_t - 1,\, H_n\rangle_{L^2(q_*)}.\]

Spectral Decay of Hermite Modes↩︎

Convergence of Mean and Variance↩︎

Because the source is centred and the distortion is symmetric, the BA flow preserves zero mean: \(\int \hat{x}\,q_t(d\hat{x}) = 0\) for all \(t\), so the \(n=1\) mode vanishes identically.

Hence the \(n=2\) Hermite component also converges to equilibrium.

Weak Convergence to the Gaussian Fixed Point↩︎

Upgrade to Total Variation↩︎

It remains to verify \(D_{\mathrm{KL}}(q_t\|q_*)\to 0\).

Entropy Convergence↩︎

The free energy \(F_\beta\) is a strict Lyapunov functional (Theorem 1) with \(q_* = \arg\min F_\beta\), so \(F_\beta(q_t)\downarrow F_\beta(q_*)\). By Theorem 5, near \(q^*\) there is local quadratic equivalence \(F_\beta(q) - F_\beta(q_*) \asymp D_{\mathrm{KL}}(q\|q_*)\). Since \(q_t \Rightarrow q_*\) by Lemma 10, eventually \(q_t\) lies in this local neighbourhood, and therefore \(D_{\mathrm{KL}}(q_t\|q_*)\to 0\).

Conclusion↩︎

Combining Lemmas 7–11 yields \(q_t \Rightarrow q_*\) and \(D_{\mathrm{KL}}(q_t\|q_*)\to 0\), hence \(\|q_t - q_*\|_{\mathrm{TV}}\to 0\) as \(t\to\infty\). This completes the proof of Theorem 25. 0◻

References↩︎

---

1. Q. Wang is with the School of Information Science and Engineering, and the School of Economics and Management, Southeast University, Nanjing, China. E-mail: qiaowang@seu.edu.cn↩︎