1 Introduction↩︎

Global objectives, such as Kullback-Leibler (KL) and the evidence lower bound (ELBO), are standard in Bayesian inference [1]–[4]. However, global closeness does not necessarily determine how probability mass behaves near a specified parameter value. Bayesian asymptotic theory has long made this local issue explicit through prior mass and small-ball conditions for posterior contraction, while sparse, singular, and constrained Bayesian models provide common settings in which the behaviour of mass near special parameter values is substantively important [5]–[7]. Such questions are not fully captured by a single global divergence value. A sequence of distributions can converge in KL divergence while still exhibiting different small-ball scaling around a fixed point. This paper develops a local-mass framework for Bayesian inference. The basic object of study is the small-ball probability \(p(B_r(\boldsymbol{\theta}))\), \(r \downarrow 0\), where \(B_r(\boldsymbol{\theta})\) denotes the ball of radius \(r\) centred at a parameter value \(\boldsymbol{\theta}\). The asymptotic behaviour of this quantity records how probability mass accumulates around \(\boldsymbol{\theta}\).

We use the term Mass Index for a concentration-oriented parametrisation of local small-ball behaviour. The object of interest is a probability measure near a fixed parameter value, and the notion is closely related to the local dimension theory of a measure [8]–[10]. Specifically, suppose that, as \(r\downarrow 0\), \(\mu(B_r(\boldsymbol{\theta})) \asymp r^a\). Here \(d\) is the ambient dimension, and \(a\) is the usual local scaling exponent. The power component of the Mass Index records the normalised reciprocal scale \(d/a\). Thus it reverses the usual local-dimension scale and expresses it as a concentration scale. The logarithmic component records the first log-order correction, when the power scale is well defined. This plays a role analogous to slowly varying corrections in regular variation [11]–[13]. The notation therefore distinguishes measures with the same leading polynomial order but different logarithmic local mass behaviour, such as smooth priors and horseshoe-type priors near sparse parameter values.

To compare local masses, we use regularised extended KL (RE-KL), a set-localised regularised \(f\)-divergence [14]–[16]. Its finite recession slope permits an extension to possibly singular measures by separating absolutely continuous and singular contributions. We therefore use RE-KL as a local \(f\)-divergence-based tool for comparing small-ball masses, not as a new global divergence. In the absolutely continuous case, it relates to Tsallis-type divergences, includes squared Hellinger distance at \(\alpha=1/2\), and converges to KL as \(\alpha\uparrow1\).

We prove that Bayesian updating preserves both power and logarithmic local mass scales, if the likelihood is locally regular around the parameter value. Thus Bayesian updating can be decomposed into a prior local scale and a likelihood-induced local correction. When the support is parameter-dependent, the likelihood may contain a hard support constraint that removes part of the prior mass near the parameter value. We prove that now local mass is governed by the fraction of prior mass that survives the constraint in shrinking neighbourhoods. If this surviving fraction decays at a polynomial or logarithmic rate, the posterior Mass Index shifts by the corresponding amount. We also show that smoothing hard constraints may not preserve this local structure: the zero-temperature limit of a softened likelihood and the small-neighbourhood limit need not commute.

We then study local discrepancy between two probability measures. RE-KL gives both absolute and relative bounds on local probability masses. These bounds imply directional sufficient conditions for comparing Mass Indices. The approximation-to-target direction is rigid: sufficiently small normalised local RE-KL forces equality of the local power scale, and also of the logarithmic scale when defined. The target-to-approximation direction is weaker: bounded local divergence prevents the approximation from being locally thinner than the target, but still allows extra local concentration. This expresses the usual asymmetry between KL directions directly in terms of local small-ball mass.

Numerical experiments illustrate the theory. Synthetic examples display regular mass, depletion, cusps, logarithmic corrections, and atoms. A UCI Bayesian logistic-regression experiment shows that regular Bayesian updating can greatly increase posterior mass near the Laplace mean while preserving the leading local power order. A final toy example demonstrates the directionality of local RE-KL, with one direction remaining bounded and the other diverging on shrinking neighbourhoods.

2 Related work↩︎

Variational inference and mode coverage. Divergence choice is known to affect the behaviour of variational approximations [2], [3]. Inclusive-KL and related \(f\)-divergence methods often encourage broader mass coverage than reverse-KL objectives [3], [14], [16]–[18]. These approaches are mainly global. Our use of local RE-KL is closer in spirit to the local analysis of Csiszár \(\phi\)-divergences by Avlogiaris et al. [15]. We study a narrower question: how Bayesian updating and variational approximation affect small-ball probabilities \(p(B_r(\boldsymbol{\theta}))\) as \(r\downarrow0\). This is also related to mode-coverage questions, where global criteria may fail to capture set-level or pointwise coverage [19], [20].

2.0.0.1 Regular variation and local asymptotic scales.

Our notation draws on two classical sources. The first is regular variation, which provides the language for separating a leading power law from slowly varying, including logarithmic, corrections [11]–[13]. The second is the local, or pointwise, dimension theory of measures, where the small-ball behaviour of \(p(B_r(\boldsymbol{\theta}))\) is summarised through limits of \(\log p(B_r(\boldsymbol{\theta}))/\log r\) [8]–[10]. We use these ideas only as a local small-ball bookkeeping device: for a fixed parameter value \(\boldsymbol{\theta}\), \(p(B_r(\boldsymbol{\theta}))\) records the leading power scale and, when present, the first logarithmic correction of the probability mass.

Bayesian pruning. Bayesian sparsification methods use priors to control how mass is allocated across parameter regions while avoiding excessive shrinkage of large signals [21]–[23]. These works motivate our local perspective, but they typically evaluate architecture size, contraction, objective value, or prediction [6], [7], [24]. We instead isolate a measure-level object: the asymptotic scale of local small-ball mass, independent of a particular architecture or optimisation procedure.

3 Preliminaries↩︎

3.0.0.1 Basics.

For non-negative functions \(f\) and \(g\), with respect to the limit under consideration, \(f=O(g)\), \(f=\Omega(g)\), and \(f\asymp g\) denote upper, lower, and two-sided comparison up to positive constants, respectively. For probability measures \(p\) and \(q\), \(p\ll q\) means that \(q(E)=0\Rightarrow p(E)=0\) for every measurable set \(E\). We write \(p\perp q\) if there exists measurable \(E\) such that \(p(\mathbb{R}^d\setminus E)=0\) and \(q(E)=0\). These notions are referred to as absolute continuity and mutual singularity, respectively.

3.0.0.2 Problem formulation.

Let \(\Theta={\mathbb{R}^d}\) be the parameter space, and let \(\mathcal{D}=(z_1,\ldots,z_n)\in\mathcal{Z}^n\) denote the observed data, where \(z_i\sim P_{\boldsymbol{\theta}},\,i=1,\ldots,n.\) The likelihood of the observed data under parameter \(\boldsymbol{\theta}\) is denoted by \(L(\boldsymbol{\theta}\mid\mathcal{D}).\) Let \(\pi_0\) be a prior probability measure on \(\Theta\). Whenever the marginal \(Z_{\mathcal{D}} := \int_{\Theta} L(\boldsymbol{\theta}\mid\mathcal{D})\,\pi_0(\mathrm d\boldsymbol{\theta})\in(0,\infty)\), the posterior is defined by \(\pi_{\mathcal{D}}(E) =Z_{\mathcal{D}}^{-1} {\int_E L(\boldsymbol{\theta}\mid\mathcal{D})\,\pi_0(\mathrm d\boldsymbol{\theta})} .\) This map from the prior \(\pi_0\) to the posterior \(\pi_{\mathcal{D}}\) is called the Bayesian update.

In this paper, Bayesian inference refers to the construction and representation of the posterior distribution induced by the prior and the observed data. The Bayesian update gives the target posterior \(\pi_{\mathcal{D}}\). Since this posterior is often analytically or computationally intractable, one may further replace it by an approximating measure \(q\) chosen from a tractable variational family \(\mathcal{Q}\subseteq\mathcal{P}(\Theta)\).

The variational approximation problem is to find \(q^*\) such that \(q^{*}\in \mathrm{argmin}_{q\in\mathcal{Q}} D_{\mathrm{KL}}(q\|\pi_{\mathcal{D}}).\) Or equivalently, maximising the evidence lower bound \(\mathcal{L}_{\mathcal{D}}(q) := \int_{\Theta}\log L(\boldsymbol{\theta}\mid\mathcal{D})\,q(\mathrm d\boldsymbol{\theta}) - D_{\mathrm{KL}}(q\|\pi_0).\)

4 The local mass framework↩︎

We now introduce the local mass framework. Throughout this paper, \(p\) and \(q\) denote two arbitrary probability measures on \(\Theta\).

4.1 Mass Index: On local small-ball scale↩︎

We now introduce the Mass Index as an order-scale summary of the local small-ball mass \(p(B_r(\boldsymbol{\theta})),r\downarrow0.\) The power component records the reciprocal of the sharp local power scale, normalised by the ambient dimension \(d\). The logarithmic component is defined only after the power scale is fixed, and records the first log-power correction. This construction is deliberately weaker than regular variation: it uses upper and lower power gauges, and does not require an exact asymptotic representation or a slowly varying factor.

We compare \((\mathrm{MI}_\mathrm{pow}(p,\boldsymbol{\theta}),\mathrm{MI}_\mathrm{log}(p,\boldsymbol{\theta}))\) using the lexicographic order. Throughout the paper, we denote \(\mathrm{MI}_{\mathrm{pow}}^{-1}(p,\boldsymbol{\theta}) := \frac{1}{\mathrm{MI}_{\mathrm{pow}}(p,\boldsymbol{\theta})}\) whenever \(\mathrm{MI}_{\mathrm{pow}}(p,\boldsymbol{\theta})\in(0,\infty)\).

4.1.1 Basic calibrations↩︎

The following proposition records the basic local calibrations of the Mass Index.

Interpretation. Proposition 1 gives basic local small-ball calibrations of the Mass Index. A local hole, regular continuous mass, and an atom correspond to \(\mathrm{MI}_{\mathrm{pow}}=0,1,\infty\), respectively. The factor \(d\) normalises the reciprocal exponent against the ordinary full-dimensional scaling \(r^d\).

Table 1 gives representative one-dimensional examples at \(\boldsymbol{\theta}=0\).

4pt

4.1.2 Scope of the definition↩︎

Although the Horseshoe prior and the Gaussian prior both have \(\mathrm{MI}_{\mathrm{pow}}=1\), the Horseshoe prior is widely used in Bayesian compression [22], and its behaviour near zero is closely connected with an additional logarithmic-order factor. Thus, a logarithmic refinement is needed if one wants to distinguish these two priors beyond their leading power order.

The cases \(\mathrm{MI}_{\mathrm{pow}}=0\) and \(\mathrm{MI}_{\mathrm{pow}}=\infty\) correspond to behaviours outside the polynomial scale, such as super-polynomial or logarithmic-type decay. Thus, the Power Mass Index should be viewed as a modest power-scale calibration. It captures the dominant polynomial order when meaningful, but it does not distinguish all finer sub-polynomial or super-polynomial behaviours.

4.2 RE-KL: A set-localised divergence↩︎

Avlogiaris et al. [15] introduced local divergences based on Csiszár \(\phi\)-divergences, while Póczos et al. [28] studied global Tsallis-\(\alpha\) divergences and their nonparametric estimation. Our construction is related to both: we use a Tsallis-type generator within the extended \(f\)-divergence framework [14], [16] and localise it to a measurable set \(E\). The resulting quantity separates absolutely continuous and singular contributions and relates them to small-ball mass behaviour.

4.2.1 Relation to standard divergences↩︎

On the whole space \(\Theta\) with \(q\ll p\):

(1) KL divergence. When \(\alpha\uparrow 1\), \(D_\alpha^\Theta(q\|p)\uparrow D_{\mathrm{KL}}(q\|p)\).

(2) Hellinger distance \(\mathcal{H}(q,p)\) [29]. When \(\alpha=1/2\), \(D_{1/2}^\Theta(q\|p)=2\mathcal{H}^2(q,p)=2\bigl(1-\int_\Theta \sqrt{\mathrm dq/\mathrm dp}\,\mathrm dp\bigr)\).

(3) Tsallis-\(\alpha\) divergence \(T_\alpha\) [28]. When \(\alpha\in(0,1)\), \(D_{\alpha}^\Theta(q\|p)=T_\alpha(q\|p) = (\alpha-1)^{-1} \left( \int_{\Theta} q^\alpha(x)p^{1-\alpha}(x)\,\mathrm dx -1 \right).\)

Thus, the present definition can be viewed as a mild localisation of the standard extended \(f\)-divergence, isolating the contribution of a measurable set \(E\) while keeping track of the corresponding singular part.

4.3 Why global KL is not enough↩︎

Global KL divergence need not determine local probability mass. Related but different concerns appear in the GAN literature under mode collapse and incomplete mode coverage [19], [20]. We use this connection only as motivation for separating global closeness from local small-ball behaviour. The result below formulates this distinction through the Power Mass Index and shows that global KL convergence alone need not determine the local power order at \(\boldsymbol{\theta}\).

Interpretation. Theorem 2 gives a cautionary example for variational approximation. Even when \(D_{\mathrm{KL}}(q_n\|p)\to0\), global KL convergence alone need not preserve the local small-ball power order near \(\boldsymbol{\theta}\). KL-small perturbations may still change the Power Mass Index at \(\boldsymbol{\theta}\). This does not mean that larger variational families necessarily cause local distortion. It shows only that, without additional structural restrictions, such stability cannot be derived from the KL objective alone.

Proof sketch. Modify \(p\) only on \(B_{\rho_n}(\boldsymbol{\theta})\), where \(\rho_n\downarrow0\), and keep \(q_n=p\) outside. Since \(p\) has no atom at \(\boldsymbol{\theta}\), the modified mass tends to zero. Redistribute this mass so that the local power index at \(\boldsymbol{\theta}\) equals \(k\). Then \(\mathrm{MI}_{\mathrm{pow}}(q_n,\boldsymbol{\theta})=k\) for all \(n\), while the KL cost is confined to a set of vanishing \(p\)-mass, so \(D_{\mathrm{KL}}(q_n\|p)\to0\).

5 Local mass preservation theory↩︎

Using the two local tools developed in Section 4, we analyse the mathematical structure of local probability mass and local discrepancy under Bayesian update and variational approximation.

5.1 Bayesian updating and local mass scaling↩︎

We first study how Bayesian updating changes local mass.

5.1.1 Stability under Bayesian updating↩︎

We prove a result that characterises how the likelihood affects the preservation of local mass under Bayesian update. The analysis is based on the following two assumptions.

Assumption 1 ensures that the posterior exists and is well-defined as a probability measure.

Assumption 2 rules out scale-wise oscillations of small-ball probabilities by requiring the upper and lower Power Mass Indices to coincide.

Example 1 shows that the MI may fail to exist. It is deliberately artificial: it builds infinite oscillations between two local power laws. Such behaviour is absent from standard priors with atoms, regular densities, finite mixtures, or regularly varying small-ball masses. Hence Assumption 2 rules out pathological oscillations, not ordinary Bayesian models.

Interpretation. Theorem 3 describes Bayesian updating as a local reweighting of the prior by the likelihood. If the likelihood is locally of power-log order, then the polynomial factor shifts the posterior Power Mass Index, while the logarithmic factor shifts only the posterior Logarithmic Mass Index.

In the regular case, where the likelihood is bounded above and below by positive constants near \(\boldsymbol{\theta}\), we have \(\gamma=\beta=0\). So Bayesian updating preserves both local mass scales. Thus local mass distortion under exact Bayesian updating arises only from non-regular local likelihood behaviour, such as zeros, singularities, logarithmic corrections, or support constraints.

Proof sketch. By Bayes’ formula, the problem reduces to estimating the likelihood-weighted prior mass of \(B_r(\boldsymbol{\theta})\). The upper estimates are obtained by decomposing \(B_r(\boldsymbol{\theta})\) into a sequence of shrinking annuli: on each annulus, the likelihood is controlled according to its distance scale, and the upper Mass Index bounds are applied to the prior mass. For the lower estimates, we restrict the integral to an outer annulus. On these annuli, the likelihood has a uniform lower bound, while the discarded inner part is negligible, yielding the matching lower order.

5.1.2 Parameter-dependent support and surviving local mass↩︎

In economics, parameter-dependent support arises in certain parametric auction models, search models, among others [30]. In these models, the boundary of the support of the observed data depends on some parameters of interest and on regressor variables. The likelihood can then be written as \[L(\boldsymbol{x}\mid \mathcal{D}) = g(\boldsymbol{x})\mathbf{1}_E(\boldsymbol{x}),\,\text{with}\,g(\boldsymbol{x})\asymp 1,\boldsymbol{x}\to \boldsymbol{\theta}.\label{eq:g1}\tag{1}\] The indicator \(\mathbf{1}_E\) may remove part, or all, of the local prior mass near \(\boldsymbol{\theta}\). The following theorem characterises the effect of such removal on local-mass preservation.

Interpretation. The quantity \(A_{E,\boldsymbol{\theta}}(r)\) measures the fraction of the local prior mass around \(\boldsymbol{\theta}\) that is not removed by the hard constraint \(\mathbf{1}_E\). Since \(g(\boldsymbol{x})\asymp 1\) near \(\boldsymbol{\theta}\), the smooth part of the likelihood does not change the local mass scale. Hence the only local effect of the hard constraint is the multiplicative factor \(A_{E,\boldsymbol{\theta}}(r) \asymp r^\delta(-\log r)^\lambda.\) Thus the hard constraint adds \(\delta\) to the local power exponent and \(\lambda\) to the local logarithmic exponent.

Proof sketch. The proof is a direct analogue of Theorem 3. Since \(g(\boldsymbol{x})\asymp 1\) near \(\boldsymbol{\theta}\), Bayes’ formula gives \[\pi_{\mathcal{D}}(B_r(\boldsymbol{\theta})) \asymp \pi_0(B_r(\boldsymbol{\theta})\cap E) = A_{E,\boldsymbol{\theta}}(r)\pi_0(B_r(\boldsymbol{\theta})).\] Thus \(A_{E,\boldsymbol{\theta}}(r)\asymp r^\delta(-\log r)^\lambda\) shifts the local power and logarithmic exponents by \(\delta\) and \(\lambda\), respectively. The condition on \(\delta\) guarantees that the posterior power exponent remains positive. The identities follow.

In practice, hard constraints, indicator factors, and parameter-dependent supports are often smoothed for optimisation or sampling. Such softening is not locally innocuous: it can change small-ball mass scales and hence the local mass concentration induced by the hard formulation.

Interpretation. This corollary shows that the local limit \(r\to0\) and the zero-temperature limit \(\tau\downarrow0\) need not commute. For fixed \(\tau>0\), the soft factor \(s_\tau\) is locally positive near \(\boldsymbol{\theta}\), so the local MI is preserved. In the hard limit, the constraint may retain only a fraction \(A_{E,\boldsymbol{\theta}}(r)\asymp r^\delta(-\log r)^\lambda\) of the local prior mass, and therefore changes the local mass scale unless \(\delta=\lambda=0\). Thus softening is not locally innocuous: it may fail to reproduce the local mass-concentration structure of the hard-constrained model.

5.2 Local RE-KL analysis for Mass Indices↩︎

MI provides a way to characterise the local mass of a probability distribution. The local RE-KL bounds below give local bounds and sufficient conditions for comparing the mass behaviour of two measures.

5.2.0.1 Local RE-KL bounds.

We first show how the absolutely continuous component and the singular component jointly contribute to the absolute error.

Interpretation. Theorem 5 gives a local absolute error control for the mass assigned to a measurable region \(E\). Small local RE–KL discrepancy on \(E\) forces \(q(E)\) and \(p(E)\) to be close in additive mass. Importantly, the bound remains meaningful even when \(q\) has a \(p\)-singular component: singular mass does not make the right-hand side automatically infinite, but is charged through the finite recession penalty in \(D_\alpha^E(q\|p)\).

Proof sketch. Decompose \(q\) into its \(p\)-absolutely continuous and \(p\)-singular parts. For the former, rewrite the local mass error in Hellinger form and apply Cauchy–Schwarz. The RE–KL term dominates the resulting quantity up to \(C_\alpha\). The singular part is already charged by the recession term in \(D_\alpha^E(q\|p)\). Combining these two estimates yields the bound.

Theorem 5 leads to Corollary 2 directly.

We next derive a relative bound for the local mass ratio \(q(E)/p(E)\).

Interpretation. Theorem 6 upgrades absolute error control to relative error control. This is needed for MI: when \(p(E)\) is itself small, absolute error is not enough to prevent \(q(E)\) from deviating too much in relative terms, whereas controlling \(q(E)/p(E)\) prevents asymptotic local under- or over-coverage.

Proof sketch. Jensen’s inequality controls the absolutely continuous part, while the recession slope of \(f_\alpha\) absorbs the singular part. Then a scalar lower bound for \(f_\alpha\) near its minimum at \(1\) converts this control into a quadratic inequality for \(|1-q(E)/p(E)|\), whose solution gives the displayed relative error bound.

Theorem 6 leads to Corollary 3 directly.

5.2.0.2 Directional local RE–KL implications for Mass Indices.

We next use Theorem 6 to compare the local small-ball scales of two measures. In variational approximation, \(p=\pi_{\mathcal{D}}\) denotes the posterior and \(q\) denotes the approximation. The comparison of interest is the one-sided relation \(\mathrm{MI}_{\mathrm{pow}}(q,\boldsymbol{\theta})\ge\mathrm{MI}_{\mathrm{pow}}(p,\boldsymbol{\theta})\). The following result gives directional sufficient conditions for this relation, and for equality in the stronger case.

Interpretation. The two parts describe different strengths of local information. The \(q\|p\) direction is rigid: a merely finite value of \(\overline{D}_\alpha^{B_r(\boldsymbol{\theta})}(q\|p)\) is not enough at the power scale, and one needs the subcritical condition \(\limsup_{r\to0}\overline{D}_\alpha^{B_r(\boldsymbol{\theta})}(q\|p)<1\). Once this condition is imposed, there is no remaining freedom at the leading small-ball scale: \(q\) and \(p\) have the same power index, and also the same logarithmic index whenever the latter is defined for \(p\).

The \(p\|q\) direction is weaker but more permissive. Finiteness of \(\limsup_{r\to0}\overline{D}_\alpha^{B_r(\boldsymbol{\theta})}(p\|q)\) is enough only for a one-sided comparison: it precludes \(q\) from being locally thinner than \(p\) at the polynomial scale, but it still allows \(q\) to assign more mass near \(\boldsymbol{\theta}\). Thus this direction leaves room for local over-concentration of the approximation. This asymmetry is consistent with the usual distinction between KL directions in variational approximation [3], [17], [18]. These are conditional local implications, not a characterisation of minimisers of the global variational objective.

Proof sketch. Apply the local RE–KL mass bound to \(B_r(\boldsymbol{\theta})\). In the \(q\|p\) direction, the subcritical bound forces equality of the power scale, and then of the logarithmic scale. In the \(p\|q\) direction, finite normalised divergence only rules out polynomially smaller \(q(B_r(\boldsymbol{\theta}))\), giving the one-sided Power Mass Index bound. When the power scales coincide, the logarithmic comparison gives the final claim.

Example 2 illustrates that the two directions encode genuinely different local comparisons. The condition in the \(p\|q\) direction permits the conclusion that \(q\) has at least as large a local power mass index as \(p\), while the reverse condition is stronger here and is not satisfied.

6 Experiments↩︎

We give three small-scale checks of the proposed local quantities. These experiments are not intended as scalable diagnostics for large neural networks, but as controlled sanity checks. For the real-data experiment, we use three UCI binary tasks: Breast Cancer, Iris \(0\) vs \(1\), and Wine \(0\) vs \(1\). Each model is Bayesian logistic regression with four PCA covariates and an intercept, so \(d=5\). We use a Gaussian prior with scale \(2\), approximate the posterior by a Laplace Gaussian, set \(\boldsymbol{\theta}_0\) to the Laplace posterior mean, and estimate Euclidean small-ball masses by Sobol quadrature over \(r\in[0.03,0.45]\). UCI results are averaged over five seeds. Full experimental details are given in Appendix 10.

6.0.0.1 Experiment 1: Synthetic calibration.

Figure 1 shows the basic regimes used in the theory. A regular Gaussian follows the reference law \(p(B_r(\boldsymbol{\theta}_0))\asymp r^d\). The power-law examples \(p(B_r(\boldsymbol{\theta}_0))\asymp r^{d+\beta}\) show how the parameter \(\beta\) changes the local small-ball mass order. The log-singular and spike-slab examples illustrate logarithmic correction and atomic mass. The exact curves used in this calibration are listed in Appendix 10.1.

6.0.0.2 Experiment 2: UCI Bayesian sanity check.

Figure 2 shows that posterior small-ball mass around the Laplace mean is much larger than prior mass. As shown in Table 2, at the smallest radii, both prior and posterior slopes remain close to \(d=5\). This matches the regular case: Bayesian updating changes the local density level, but not the leading local power order. The posterior slope decrease at larger radii is a finite-radius effect. Dataset preprocessing, posterior fitting, quadrature, and slope computation are described in Appendix 10.2.

3pt

6.0.0.3 Experiment 3: Directionality of local RE-KL.

Figure 3 illustrates that local RE-KL control is directional. On the same shrinking neighbourhoods, the \(p\|q\) direction stays bounded while the \(q\|p\) direction diverges. Thus the order of the two arguments determines which local mismatch is detected. The explicit construction is given in Appendix 10.3.

7 Conclusion↩︎

This paper studied local small-ball mass \(p(B_r(\boldsymbol{\theta}))\) as a measure-level quantity in Bayesian inference. The aim was not to propose a new global divergence or a scalable diagnostic method, but to make precise how local mass scales can change under Bayesian updating and approximation.

We made three concrete contributions. First, we used the Mass Index to record the leading power order and the first logarithmic correction of local small-ball probabilities. Second, we derived local scale identities for Bayesian updating, showing how regular likelihoods preserve the local scale, how power-log likelihood factors shift it, and how parameter-dependent supports act through the surviving fraction of prior mass. Third, we used local RE-KL to prove absolute, relative, and directional bounds for comparing small-ball masses, yielding sufficient conditions for equality or one-sided comparison of Mass Indices under the two KL directions.

These results should be read as local asymptotic scale comparisons rather than a characterisation of global variational minimisers. The experiments provide finite-radius illustrations of the quantities and regimes appearing in the theory.

8 Examples and Counterexamples↩︎

9 Proofs↩︎

Proof of Proposition 1↩︎

Proof of the RE-KL relations↩︎

Proof of Theorem 2↩︎

Proof of Theorem 3↩︎

Proof of Theorem 4↩︎

Proof of Corollary 1↩︎

Proof of Theorem 5↩︎

Proof of Corollary 2↩︎

Proof of Theorem 6↩︎

Proof of Corollary 3↩︎

Proof of Theorem 7↩︎

10 Additional experimental details↩︎

This appendix gives the concrete settings used for the experiments in Section 6. All experiments are finite-radius illustrations. We do not estimate asymptotic Mass Indices directly.

10.1 Experiment 1: synthetic calibration↩︎

Experiment 1 plots closed-form small-ball curves over \[r\in[10^{-3},0.6].\] The purpose is to visualize representative local regimes rather than to fit a statistical model.

For the regular reference case, we use the one-dimensional Gaussian small-ball mass \[p_{\mathrm{Gauss}}(B_r(0)) = \operatorname{erf}\left(\frac{r}{\sqrt{2}}\right).\] This behaves as \(r\) for small \(r\).

For the local-depletion example, we plot \[p_{\mathrm{dep}}(B_r(0))=r^2.\] This has faster small-radius decay than the regular reference.

For the cusp example, we plot \[p_{\mathrm{cusp}}(B_r(0))=\sqrt r.\] This has slower small-radius decay than the regular reference.

For the logarithmically corrected example, we plot \[p_{\log}(B_r(0))=r(1-\log r).\] This has the same leading power order as the regular reference, but differs by a logarithmic factor.

For the spike-slab example, we use spike weight \(0.25\) and slab scale \(0.35\), giving \[p_{\mathrm{spike}}(B_r(0)) = 0.25 + 0.75\, \operatorname{erf}\left( \frac{r}{0.35\sqrt{2}} \right).\] The nonzero spike produces a small-ball mass that does not vanish as \(r\to0\).

10.2 Experiment 2: UCI Bayesian logistic-regression sanity check↩︎

Experiment 2 uses three binary classification datasets: Breast Cancer, Iris \(0\) vs \(1\), and Wine \(0\) vs \(1\). For Iris and Wine, only classes \(0\) and \(1\) are retained. Each dataset is repeated over five random seeds, \[\{0,1,2,3,4\}.\] The seed controls the stratified train-test split. Conditional on the training set, preprocessing, Laplace optimisation, and quadrature settings are fixed. For each dataset and seed, we use a stratified train-test split with test fraction \(0.30\). Only the training set is used for the Bayesian logistic-regression fit.

The covariates are standardized on the training set and then projected to four principal components using PCA. An intercept is added, so the parameter dimension is \[d=5.\] The model is Bayesian logistic regression. Given design matrix \(X\), labels \(y_i\in\{0,1\}\), and parameter \(\boldsymbol{\theta}\in\mathbb{R}^5\), the negative log posterior objective is \[\sum_i \left[ \log(1+\exp(x_i^\top\boldsymbol{\theta})) - y_i x_i^\top\boldsymbol{\theta} \right] + \frac{1}{2\sigma_0^2}\|\boldsymbol{\theta}\|^2, \qquad \sigma_0=2.\] Equivalently, the prior is \[\boldsymbol{\theta}\sim N(0,4I_5).\]

The posterior is approximated by a Laplace Gaussian approximation. The posterior mean \(\hat{\boldsymbol{\theta}}\) is obtained by minimizing the negative log posterior. We use a trust-region Newton optimizer with analytic gradient and Hessian. If this optimizer fails, we use an L-BFGS-B fallback. The Laplace covariance is the inverse Hessian at \(\hat{\boldsymbol{\theta}}\), with eigenvalues clipped below \(10^{-10}\) for numerical stability. We set \[\boldsymbol{\theta}_0=\hat{\boldsymbol{\theta}}.\]

For each run, prior and posterior Euclidean small-ball masses are estimated for \[B_r(\boldsymbol{\theta}_0) = \{\boldsymbol{\theta}:\|\boldsymbol{\theta}-\boldsymbol{\theta}_0\|_2\leq r\}\] over a geometric grid of \(24\) radii in \[[0.03,0.45].\] The prior covariance is \(4I_5\), while the posterior covariance is the Laplace covariance. The prior mean is \(0\), and the posterior mean is \(\boldsymbol{\theta}_0\).

The small-ball masses are estimated by Sobol quadrature over the Euclidean unit ball. The default number of Sobol points is \[2^{15}=32768\] per run. The unit-ball Sobol points are generated in dimension \(d+1\). The first \(d\) coordinates are transformed into Gaussian directions and normalized to obtain directions on the sphere. The final coordinate is transformed as \(U^{1/d}\) to obtain the radial component. For a radius \(r\), the quadrature points are \[\boldsymbol{\theta}_0+r z_j, \qquad z_j\in B_1(0).\]

For a Gaussian distribution \(N(m,\Sigma)\), the estimated log small-ball mass is \[\log |B_1(0)| + d\log r + \log\left[ \frac{1}{M} \sum_{j=1}^{M} \varphi_{\Sigma}(\boldsymbol{\theta}_0+r z_j-m) \right],\] where \(M=32768\), and \(\varphi_{\Sigma}\) denotes the Gaussian density with covariance \(\Sigma\).

Finite-radius slopes are computed by first differences on the log-log scale: \[s_k = \frac{ \log p(B_{r_{k+1}}(\boldsymbol{\theta}_0)) - \log p(B_{r_k}(\boldsymbol{\theta}_0)) }{ \log r_{k+1}-\log r_k }.\] The slope location is reported at the geometric midpoint \[(r_k r_{k+1})^{1/2}.\] The main-text table reports small-radius slopes fitted over \(r\leq0.06\).

Across datasets and seeds, the plotted aggregate curves use means and standard errors over all runs. In total, the aggregate UCI figure uses \[3\times5=15\] runs.

10.3 Experiment 3: local RE-KL directionality↩︎

Experiment 3 is a one-dimensional toy example designed to show that local RE-KL control is directional. On \((-1,1)\), define \[p(x)=\frac{1}{2}, \qquad q(x)=\frac{1}{4} |x|^{-1/2}.\] Both are probability densities on \((-1,1)\). Around \(\boldsymbol{\theta}_0=0\), their small-ball masses are \[p(B_r(0))=r, \qquad q(B_r(0))=\sqrt r.\]

The plotted local RE-KL quantities use mass-ratio weighted conditional terms. The conditional constants are \[C_{p\|q}=\log 2-\frac{1}{2}, \qquad C_{q\|p}=1-\log 2.\] The plotted finite-radius quantities are \[D_r(p\|q) = \sqrt r \left(\log 2-\frac{1}{2}\right),\] and \[D_r(q\|p) = \frac{1-\log 2}{\sqrt r}.\] Therefore, \[D_r(p\|q)\to0, \qquad D_r(q\|p)\to\infty \qquad (r\to0).\] This example shows that the two directions can behave differently on the same shrinking neighbourhoods. Hence the direction in a local RE-KL assumption is substantive.

References↩︎