Denoising Distances in Metric Measure Spaces


Abstract

Recent work studied the problem of finding clusters and denoising pairwise distances from noisy distances of points sampled on a manifold. We study the same problems in more general metric measure spaces under lower \(\phi\)-regularity. We give an algorithm that extracts large localized clusters around every sampled point and uses them to denoise distances to any fixed accuracy, with near-linear running time in the dense fixed-accuracy regime. We also show how to achieve much higher accuracy with a non-efficient algorithm. This suggests that unlike the Riemannian case, denoising to higher accuracy in more general metric spaces has a statistical-computational gap.

1 Introduction↩︎

A general model for noisy distance data.↩︎

A common way to model noisy metric data is through latent-distance observations. There are unobserved points \(X_1,\ldots,X_n\) in a metric space \((M,{\rm d})\), sampled according to a probability measure \(\mu\). Instead of observing the distances \({\rm d}(X_u,X_v)\) directly, we observe noisy pairwise measurements \(Z_{u,v}\) between vertices \(u\) and \(v\). The distribution of these measurements depends on the latent distance, typically through a monotone link function. This viewpoint includes latent-space network models, random geometric graphs, and latent-distance graph models [1][3]. The statistical goal is to use these noisy pairwise observations to recover the latent metric, or at least to estimate latent distances up to a prescribed accuracy.

Lower \(\phi\)-regularity.  A standard assumption that makes local metric recovery statistically meaningful is a lower bound on the mass of small balls. Conditions of this type appear in nearest-neighbor analysis, nonparametric estimation, random geometric graphs, graph-based manifold learning, and analysis on metric measure spaces, under names such as lower mass conditions, small-ball assumptions, minimal mass assumptions, or lower volume growth [2], [4][9]. At a high level, they ensure that local neighborhoods contain enough probability mass for statistical estimates to concentrate uniformly across the space.

In our notation, lower \(\phi\)-regularity means that there is a nondecreasing lower-regularity function \(\phi\) such that \[\mu(B(x,r))\ge \phi(r)\] for every support point \(x\) and every relevant small radius \(r\), where \(B(x,r)\) denotes the open ball of radius \(r\) centered at \(x\). In a \(d\)-dimensional space, one can think of \(\phi(r)\) as proportional to \(r^d\), meaning that every radius-\(r\) ball has mass at least a constant times \(r^d\).

Local link assumption.  The second structural assumption concerns the link between latent distance and observed edge statistics. By link function, we mean a function \(\mathrm{p}\) such that our noisy observation \(Z_{u,v}\) has expectation \(\mathbb{E}[Z_{u,v}\mid X_u,X_v] = \mathrm{p}({\rm d}({X_u},{X_v}))\). Monotonicity says that the observations are ordered by distance: in similarity graphs, nearby points tend to produce stronger or more frequent edges, while in noisy distance models the expected observation increases with distance. Monotonicity alone, however, is not enough for metric recovery at small scales. Close points may become statistically indistinguishable from one another.

The local link assumption is the following local identifiability condition: the link function is bi-Lipschitz on a neighborhood \([0,r_\mathrm{p}]\) of zero. We refer to this neighborhood as the local bi-Lipschitz window. This means that, for small distances, a change in latent distance of order \(r\) produces a change in the expected observation of order \(r\).

The assumption is deliberately local. We do not require the link to remain informative at large distances; it may flatten, saturate, or otherwise lose metric information outside the local bi-Lipschitz window. This is natural in similarity graphs, where faraway points may all have nearly indistinguishable connection probabilities.

Latent-distance observation model.  We work with a general latent-distance observation model. The latent points \(X_v\) are sampled independently from a metric probability space \((M,{\rm d},\mu)\) satisfying the lower \(\phi\)-regularity condition above. For each pair \(u\neq v\), there is a noisy weighted measurement whose mean depends only on the latent distance \({\rm d}(X_u,X_v)\). Informally, \[Z_{u,v} = B_{u,v} \tilde{Z}_{u,v},\] where \(\tilde{Z}_{u,v}\) is a subgaussian random variable with mean \(\mathrm{p}({\rm d}(X_u,X_v))\), and \(B_{u,v}\) is a Bernoulli random variable with parameter \(\mathsf{s}\), independent of \(\tilde{Z}_{u,v}\), which models the sparsity of the observed weighted graph (\(\mathsf{s}\) can be \(n\) dependent). Meaning that with probability \(\mathsf{s}\), the noisy measurement is observed, and with probability \(1-\mathsf{s}\), it is \(0\). The link function \(\mathrm{p}\) is a monotone link function satisfying the local link assumption described above, and bounded on \([0, {\rm diam}(M)]\). Further, we assume \(Z_{u,v}\) are independent across pairs up to symmetry \(Z_{u,v}=Z_{v,u}\), conditional on the latent points. The precise formulation is given in Definition 8 and Assumptions 13, 14, and 15.

For terminology, we view the observed data as an observed weighted graph: a weighted graph \(G\) on vertex set \(\{1,\ldots,n\}\), where the weight of the edge \((u,v)\) is \(Z_{u,v}\). The latent points \(X_v\) are unobserved. This leads to the central question of the paper:

Under these two assumptions, and without any knowledge of the structure of the latent metric space, how much can we recover latent distance between the latent points from the observed weighted graph?

Examples↩︎

Before stating our results, we discuss several examples that fit into this framework.

Noisy distance data on a manifold.  The first motivating example is noisy distance data on a manifold. Suppose that \(M\) is a \(d\)-dimensional Riemannian manifold, the latent points \(X_1,\ldots,X_n\) are sampled from a regular probability measure on \(M\), and we observe \[Z_{u,v} = q({\rm d}(X_u,X_v))+\xi_{u,v},\] where the raw mean link is monotone increasing and the noises \(\xi_{u,v}\) are centered subgaussian random variables. For instance, the natural raw link is \(q(t)=t\). The usual regularity assumptions on a \(d\)-dimensional manifold imply that small balls have mass at least a constant times \(r^d\), so in our notation one may take \(\phi(r)\gtrsim r^d\).

Soft random geometric graphs.  The Bernoulli soft random geometric graph is a basic example [2], [10]. In this model, \(Z_{u,v}\in\{0,1\}\), and \[\mathbb{P}(Z_{u,v}=1\mid X_u,X_v) = \mathsf{s}\,\mathrm{p}({\rm d}(X_u,X_v)),\] where \(\mathrm{p}:\mathbb{R}_+\to[0,1]\) is non-increasing. Thus nearby latent points are more likely to be connected by an edge. This captures, for instance, models in which two devices are more likely to communicate when their latent distance is small, or more generally similarity graphs in which edge probabilities decrease with distance.

These two examples are the main objects in recent work on manifold distance denoising and random geometric graphs. Huang, Jiradilok, and Mossel [11][13] and Fefferman, Marty, and Ren [14] study closely related problems for points sampled from \(d\)-dimensional Riemannian manifolds or similar geometric spaces, under regularity assumptions that imply lower \(\phi\)-regularity of the form \[\mu(B(x,r))\gtrsim r^d.\] Their algorithms recover latent distances at polynomial rates \(n^{-\Theta(1/d)}\), where \(n\) is the number of sampled points and \(d\) is the intrinsic dimension. Latent-distance estimation in random geometric graphs has also been studied through spectral and graph-distance methods in Euclidean or spherical settings [3], [15], though these methods usually assume concrete knowledge of the latent geometry but in the hard disc setting, where the link function is an indicator of whether the distance is below a threshold.

Beyond manifolds.  Beyond manifold models from [11][14], our framework also covers a variety of non-Euclidean latent geometries, provided the sampling measure satisfies appropriate lower \(\phi\)-regularity.

This includes more singular metric measure spaces, such as compact metric trees, finite metric graphs, stratified spaces, or Ahlfors-regular fractal sets, the Heisenberg group with the Carnot–Carathéodory metric as well as subsets of Euclidean spaces with rough boundaries. In such examples, the lower-regularity function \(\phi\) records the relevant local mass growth; for instance, an Ahlfors-regular fractal with dimension \(d_f\) has \(\phi(r)\) comparable to \(r^{d_f}\), where \(d_f\) need not be an integer.

Stochastic block models.  Here we illustrate an example at the opposite extreme. Let \(M=\{a_1,\ldots,a_k\}\) be a finite metric space with \[{\rm d}(a_i,a_i)=0, \qquad {\rm d}(a_i,a_j)=1 \quad (i\neq j),\] and let \(\mu\) be a probability distribution on \(M\). Sampling \(X_v\sim\mu\) assigns each vertex to one of \(k\) latent classes. If \[\mathbb{P}(Z_{u,v}=1\mid X_u,X_v) = \mathsf{s}\,\mathrm{p}({\rm d}(X_u,X_v)),\] then vertices in the same class connect with probability \(\mathsf{s}\,\mathrm{p}(0),\) while vertices in different classes connect with probability \(\mathsf{s}\,\mathrm{p}(1).\) Thus the model reduces to a stochastic block model with \(k\) blocks, within block probability \(\mathsf{s} \, \mathrm{p}(0)\), and between-block probability \(\mathsf{s} \, \mathrm{p}(1)\). This connects the finite-metric special case to the classical stochastic block model [16], [17]. In this finite example, lower \(\phi\)-regularity is simply a lower bound on the smallest block mass. This example illustrates that the metric measure framework is not restricted to smooth spaces; it also includes highly discrete latent geometries.

Monotonicity convention.  The preceding examples illustrate the two possible orientations: noisy-distance observations naturally have a non-decreasing mean link, while similarity graphs naturally have a non-increasing link. For real-valued observations this orientation is immaterial. If the raw observations have conditional mean \[\mathbb{E}[Z_{u,v}\mid X_u,X_v] = q({\rm d}({X_u},{X_v}))\] with \(q\) non-decreasing, we replace \(Z_{u,v}\) by \(-Z_{u,v}\) and use the link function \(\mathrm{p}=-q\). Then \(\mathrm{p}\) is non-increasing, with the same local bi-Lipschitz constants up to sign. In all formal statements and proofs below, we adopt this non-increasing similarity convention, relabeling the transformed observations as \(Z_{u,v}\).

Results↩︎

From distance recovery to local cluster extraction.  A natural way to estimate latent distances is to first construct local neighborhoods around the sampled points. Suppose, informally, that for each vertex \(v\) we could find (by examining the observed weighted graph) a large set \(U_v\), whose latent points all lie within a small radius \(r\) of \(X_v\). Then \(U_v\) can be used as a local statistical proxy for \(X_v\). For two vertices \(v,w\), the average observed weight between \(U_v\) and \(U_w\) concentrates around its conditional expectation; because the clusters are localized, this expectation is close to \(\mathrm{p}({\rm d}(X_v,X_w))\). Thus, once such local clusters are available, distance recovery can be reduced to averaging edge weights between clusters and inverting the link function if it is known.

Our main technical result is a cluster-extraction theorem: under lower \(\phi\)-regularity and the local link assumption, we construct large localized clusters around all target vertices. The corresponding distance-denoising statements then follow as consequences.

In the context below, we always assume any algorithm has access to the observed weighted graph, the parameters of \(\mathrm{p}\) (bi-Lipschitz constant, the radius where \(\mathrm{p}\) is bi-Lipschitz, and the \(\max_{t \in [0,\operatorname{diam}(M)]} |\mathrm{p}(t)|\)), the lower-regularity function \(\phi\), and the sparsity parameter \(\mathsf{s}\), but not the latent points, any other structural information about the latent metric space, or the link function \(\mathrm{p}\) beyond the stated assumptions.

Theorem 1 (Informal statement of Theorem 47). There exists an algorithm that takes as input the observed weighted graph and a target scale \(r\), and outputs, with high probability, for every target vertex \(v\) a set \(U_v\) satisfying \[v\in U_v, \qquad X_{U_v}\subseteq B(X_v,Cr), \qquad |U_v|\ge c\,\phi(r)n,\] provided \(0<\sqrt r\le c\min\{1,r_\mu,r_\mathrm{p}\}\) and \[\mathsf{s}n\,\phi(r)r^2 \ge C\Lambda \log n\,.\] The algorithm has running time \[\exp\left( C \log^2\!\left(\frac{3}{r\phi(r)}\right) \frac{1}{\mathsf{s}r^2}\right)n\Lambda\log n\,.\] Further, the collection \[\{U_v\}_{v=1}^n\] without counting multiplicity is bounded by \[\exp\left( C \log^2\!\left(\frac{3}{r\phi(r)}\right) \frac{1}{\mathsf{s}r^2}\right).\] The constants \(C,c\), and the required lower bound on \(\Lambda\), depend only on the model parameters.

Usually, a cluster result of this type can be translated into distance estimates by considering the average observed edge weight between two clusters. These ideas are standard in the literature on clustering and community detection, and they also appear in recent work on distance denoising under manifold assumptions [11][14]. Here we state a reader-friendly version of this translation.

Corollary 2 (Informal distance recovery from extracted clusters). Assume the hypotheses of the cluster-extraction theorem.

  1. If \(\mathrm{p}\) is known and bi-Lipschitz on \([0,\operatorname{diam}(M)]\), then inverting cluster averages recovers all pairwise distances \({\rm d}({X_v},{X_w})\) with error \(O(r)\).

  2. If \(\mathrm{p}\) is known but bi-Lipschitz only on \([0,r_\mathrm{p}]\), then the same argument recovers distances below the local bi-Lipschitz window with error \(O(r)\). Large distances recovery is in general not possible. (See Figure 1) However, if \(M\) is a geodesic space, then distance recovery still holds for all distances.

Remark 3. For the second case, the geodesic assumption can be relaxed; see Corollary 51. It is enough to have chains whose consecutive distances are below \(r_\mathrm{p}\) and whose total length approximates the original distance; the approximation error is reflected in the final recovery error.

Remark 4 (Unknown link). When the link function \(\mathrm{p}\) is unknown, one can still adapt the bisection/calibration strategy of [14] after the localized clusters have been extracted. This would recover the metric up to an unknown global dilation factor, with the corresponding logarithmic loss. We do not pursue this direction in the present manuscript.

Efficiency at fixed accuracy.↩︎

In general, our algorithm is inefficient because it includes an exhaustive search. However, we still have an efficient algorithm when we only aim for a fixed accuracy in the dense regime.

Corollary 5 (Efficiency Statement). When \(r\) is fixed and in the dense regime \(\mathsf{s}\) is a constant, the cluster-extraction algorithm runs in time \[n \, \log(n),\] and the same running time applies to distance recovery when \(\mathrm{p}\) is known and bi-Lipschitz on the full distance range. The additional running time for distance recovery is polynomial in the number of distinct extracted clusters, which is bounded by \(\exp(C\log^2(1/\phi(r)))\) and is therefore constant when \(r\) is fixed. The distance recovery step only computes averages between pairs of these distinct clusters, and vertices in the same cluster have latent distance at most \(O(r)\), which is already within the recovery error scale.

Information-theoretic bounds.↩︎

In the case where \(\mathrm{p}\) is Bi-Lipschitz on the full distance range, or say the underlying metric space is geodesic, the cluster-extraction algorithm from Theorem 47 together with cluster-to-distance recovery Corollary 2 can be used to recover all pairwise distances with error \(O(r)\) when \[\mathsf{s}n\,\phi(r)r^2 \gtrsim \log n\,.\] Below we provide a near-matching lower bound.

Lemma 1 (Lower bound). Let \(\phi\) be nondecreasing, let \(\mathsf{s}\in(0,1]\), let \(r_0\in(0,1]\), and assume \(0<\phi(r_0)\le1/3\). There are universal constants \(C_0,c_0,c_1>0\) such that if \[n\phi(r_0)> C_0 \qquad\text{and}\qquad \mathsf{s}n\phi(r_0)r_0^2<c_0,\] then there is a three-point metric probability space that is lower-\(\phi\)-regular up to scale \(r_0\), together with a locally bi-Lipschitz observation model satisfying the standing assumptions with sparsity parameter \(\mathsf{s}\), for which every estimator \(\widehat d\) satisfies \[\mathbb{P}\!\left( \max_{u,v}|\widehat d(u,v)-{\rm d}({X_u},{X_v})|\ge r_0/2 \right) \ge c_1.\]

The proof is given in Section 10.

Figure 2: A folding obstruction at the volumetric scale, illustrated for \phi(r)=r. Both panels show the same space: a circle with a duplicated cap of length r, equipped with the intrinsic metric. The only displayed change is the location of y: in panel (a), y lies in the red copy A_-; in panel (b), it lies in the matched blue copy A_+. Every point outside A_-\cup A_+ has the same distance to these two matched locations. Therefore the two configurations are distinguishable only through interactions between the two duplicated caps. There are about n\phi(r) sampled points in each cap, and cross-cap edge probabilities change by O(\mathsf{s}r), giving the heuristic obstruction O(\mathsf{s}n\phi(r)r^2). The same idea extends to duplicated caps in S^d, where \phi(r)\asymp r^d, and to analogous spaces adapted to a prescribed \phi.

Related work↩︎

The closest works to ours are the recent distance-denoising results of Huang, Jiradilok, and Mossel [11][13] and Fefferman, Marty, and Ren [14]. These works study closely related latent-distance observation models under geometric assumptions, most prominently when the latent points are sampled from a \(d\)-dimensional Riemannian manifold. In such settings, the sampling measure satisfies lower \(\phi\)-regularity of the form \[\mu(B(x,r))\gtrsim r^d\] at small scales, and the intrinsic dimension \(d\) governs the achievable distance-denoising rates.

The work of Fefferman, Marty, and Ren [14] also treats settings beyond smooth manifolds. They do so by abstracting the conditions needed for their manifold estimation algorithm. However, the results of [14] do not apply for fractal or discrete settings and many other spaces where our results hold.

Proof ideas↩︎

Here in this paper we simply assume the link function \(\mathrm{p}\) is monotone non-increasing, as in similarity graphs. The other case is similar after \(Z_{u,v}\) by \(-Z_{u,v}\) and \(\mathrm{p}\) by \(-\mathrm{p}\). Let \(\mathbf{V}\) denote the vertex set of the observed weighted graph, so \(\mathbf{V}\) is the index set of the latent points. For a subset \(U\subseteq\mathbf{V}\), we define the latent link average \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(\cdot)}\hypersetup{linkcolor=blue}:M\to\mathbb{R}\) by \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} := \frac{1}{|U|} \sum_{u\in U} \mathrm{p}({\rm d}(X_u,y)), \qquad y\in M.\] When the argument is a vertex \(v\), we use the convention \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}:=\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(X_v)}\hypersetup{linkcolor=blue}\). For a vertex \(v \notin U\), the observable counterpart is the empirical link average \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}:= \frac{1}{\mathsf{s}|U|} \sum_{u\in U}Z_{u,v}.\] Conditionally on the latent positions \(X_U\) and \(X_v\), \[\mathbb{E}\bigl[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\mid X_U,X_v\bigr] = \hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}.\] Thus \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) is a noisy empirical evaluation of the latent link average at \(v\). When \(U\) is a cluster with center \(x\in M\), we expect \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue}\) to be a good approximation of \(\mathrm{p}({\rm d}({x},{y}))\): averaging over points \(X_u\) near \(x\) should nearly produce the link value from \(x\) to \(y\). So if \(U\) is a cluster around \(X_w\), then \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) is a good proxy for \(\mathrm{p}({\rm d}({X_w},{X_v}))\). This is the key to distance recovery from clusters.

The algorithm is built from two ingredients. The first produces many internal-average candidates by exhaustive search. The second refines an internal-average candidate into a cleaner cluster by thresholding its edge averages to fresh vertices (see Definitions 17 and 18, and Proposition 12).

The starting point is the monotonicity of the link function. Since \(\mathrm{p}\) is monotone decreasing, a set of vertices with unusually large internal average edge weight should correspond to latent points that are mostly close to one another. Conversely, by lower \(\phi\)-regularity, every small latent ball contains many sampled vertices, and those vertices form a set with high internal average; this uses Assumptions 13 and 14, together with the lower-occupancy estimate in Lemma 2.

Fix a scale \(\rho\). When \(n\) is large enough, we expect for each latent point \(x\) that the ball \(B(x,\rho)\) contains about \(n\phi(\rho)\) sampled points, let us denote it by \(U_x\) for the moment. For any pair of points \(u,v\in U_x\), \(\mathrm{p}({\rm d}({X_u},{X_v}))\) is at least \(\mathrm{p}(0) - L_\mathrm{p}\rho\). Thus, we can search for subsets \(U\) of with \[\overline{N}\!\left(U\right) := \frac{1}{\mathsf{s}}\frac{1}{|U|^2}\sum_{u,v\in U}Z_{u,v}\] whose expected value is at least \(\mathrm{p}(0) - L_\mathrm{p}\rho\). (The \(\mathrm{p}(0)\) is not known, but can be well estimated from the maximum internal average of subsets of size about \(n\phi(\rho)\), see Lemma 7.)

In order for \(\overline{N}\!\left(U\right)\) to be realistically close to its expectation, we need its fluctuation to be smaller than \(\rho\) for every \(U\) of size about \(n\phi(\rho)\). This requires a search-scale condition of the form \[\mathsf{s}n \phi(\rho)\rho^2\gtrsim \log n.\] In the final two-round construction, the internal search scale is chosen so that this reduces to the main sample-size condition at target scale \(r\).

This exhaustive search produces a candidate family which we denote by \({\mathcal{C}}_\rho\). (See Proposition 36, with the estimate of \(\mathrm{p}(0)\) supplied by Lemma 7.) Then, this candidate family has two useful properties.

  • First, it covers the latent space: every ball of radius \(\rho\) in the latent space contributes at least one internal-average candidate, as in Proposition 36. That is, for every point \(x\in M\), there is a candidate \(U\in{\mathcal{C}}_\rho\) such that all latent points in \(U\) lie within distance \(\rho\) of \(x\).

  • Second, every such candidate localizes after Markov. Namely, a candidate \(U\) has a representative point \(\theta_U \in M\) such that, for larger radii \(R\), all but an \(O(\rho/R)\)-fraction of the latent points in \(U\) lie within distance \(R\) of \(\theta_U\), as long as \(R\) is within the local bi-Lipschitz window of the link function \(\mathrm{p}\); see Lemma 6 and Remark 28.

Simplified when \(\mathrm{p}\) is bi-Lipschitz on the full distance range.  If \(\mathrm{p}\) were bi-Lipschitz on the full distance range, then the second property would be enough for distance recovery: Each candidate set \(U\) has a good representative point \(\theta_U\) such that \[|\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} - \mathrm{p}({\rm d}(\theta_U,y))| = O(\rho)\,.\]

At a higher level, this is already enough to give a distance-recovery procedure, ignoring some non-essential technicalities. The final statement in our main theorem that each \(v\) has a no-error cluster \(U_v\) with \(X_{U_v} \subseteq B(X_v,Cr)\) is more than what is needed for distance recovery.

So our main algorithm resolves two issues: 1. handle the fact that the link function \(\mathrm{p}\) is only bi-Lipschitz on a local bi-Lipschitz window, and 2. avoid running an exhaustive search on the whole vertex set when \(n\) is larger than the minimum assumption \(\mathsf{s}n \phi(\rho)\rho^2\gtrsim \log n\), improving the running time.

Presence of a local bi-Lipschitz window.  In the case that \(\mathrm{p}\) is only bi-Lipschitz on a local bi-Lipschitz window, one can treat each \(U\) as a \((R,c\rho/R)\) cluster in the sense of Definition 17, meaning that it is mostly contained in a ball of radius \(R\) around \(\theta_U\), except for a fraction of \(c\rho/R\) of its points. Such a cluster is in general insufficient for distance recovery, as a portion of its points may be far from the representative point \(\theta_U\) with distance much larger than the local bi-Lipschitz window of the link function \(\mathrm{p}\). Still, for a fresh vertex \(v\), the empirical link average \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) concentrates around \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\) by Lemma 3 (and uniformly by Lemma 4). For each \(U \in {\mathcal{C}}_\rho\), \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\) therefore serves as a proxy for \(\mathrm{p}({\rm d}(\theta_U,X_v))\) with error \(O(\sqrt{\rho})\), instead of \(O(\rho)\) in the “global” bi-Lipschitz case; see Remark 34. In short, there is a tradeoff between the score error and the fraction of points that are far from \(\theta_U\).

Regime Applicable \(y\)’s Error bound
Global bi-Lipschitz all \(y\in M\) \(O(\rho)\)
Local bi-Lipschitz window \({\rm d}({\theta_U},{y})+O(\sqrt{\rho})\le r_\mathrm{p}\) \(O(\sqrt{\rho})\)

The second ingredient is link-average threshold refinement, which cleans up a seed set using fresh vertices. Thresholding the observed averages \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) selects vertices whose latent points are near the seed representative; the abstract refinement statement is Proposition 12.

This refinement step turns a seed into a cleaner cluster. A first refinement may still leave a small exceptional fraction of mistakes, but a second refinement can be made strong enough to give exact inclusions: all vertices in an inner ball are selected, and no vertices outside a larger ball are selected. The needed link-average gap is packaged in the unified Proposition 31, whose two cases are the oracle route and the local route.

Algorithm outline↩︎

Vertex Partition.  To keep the concentration arguments independent, we apply the two ingredients on separate vertex blocks. For the basic three-block procedure, we split the vertices into \[V_1,\quad V_2,\quad V_3.\] The exhaustive search and candidate net construction are performed in \(V_1\). The selected cluster candidates (a subset of \({\mathcal{C}}_\rho\)) are then refined into \(V_2\), producing large intermediate clusters. These intermediate clusters are refined once more into \(V_3\), which is the final output block. This is formalized in Theorem 42 and Remark 43; the candidate net construction is the unified Proposition 38, with route-specific proofs for the oracle route and the local route. At a high level, \[\text{internal-average search} \longrightarrow \text{candidate net} \longrightarrow \text{intermediate refinement} \longrightarrow \text{exact refinement}.\]

The reason for using disjoint blocks is simple: the edges used to construct a cluster candidate are independent of the edges used to test that cluster candidate against fresh vertices. This lets us condition on the cluster candidate and apply concentration to the next block, which allows us to exploit conditional independence; this is the conditioning setup used in Proposition 12.

Accuracy/efficiency tradeoffs by choosing smaller subsets.  Although the theorem is stated with three ambient blocks of size \(n\), the first two stages do not need to use all vertices in their blocks. The expensive exhaustive search is performed only on a smaller working subset \[V_1'\subseteq V_1,\qquad |V_1'|=n_1,\] where \(n_1\) is chosen just large enough for the internal-average search and the first refinement-net construction to succeed. In the local route this internal scale is \(\rho\asymp r^2\), while in the oracle route it is \(\rho\asymp r\), up to constants depending only on the model parameters. The working size \(n_1\) is chosen near the smallest value for which the first-block occupancy and pair-average concentration events hold, with the additional mild lower bound \(n_1\ge\log\log n\).

Similarly, the first refinement is performed only into a smaller working subset \[V_2'\subseteq V_2,\qquad |V_2'|=n_2.\] The role of \(V_2'\) is to produce intermediate clusters large enough to seed the exact refinement into the full output block \(V_3\). Thus it is enough to choose \(n_2\) so that \[\mathsf{s}n_2\phi(cr)r^2 \gtrsim \Lambda\log n.\] The final refinement is still performed into the full block \(V_3\), so the output clusters have size of order \(n\phi(r)\).

This separation of working sizes is useful computationally. In the dense fixed-scale regime, where \(\mathsf{s}\asymp1\), \(r\asymp1\), and \(\phi(cr)\asymp1\), one may take \[n_1\asymp \log\log n, \qquad n_2\asymp \Lambda\log n.\] Then the exhaustive search over \(V_1'\) is only polylogarithmic in \(n\), and the refinement stages cost \(n\,\operatorname{polylog}(n)\). Thus the same statistical construction yields a near-linear-time algorithm for fixed-accuracy recovery in dense regimes.

The local-window difficulty.  If the link function were bi-Lipschitz on all of \([0,\operatorname{diam}(M)]\), the internal-average candidates would behave like ordinary small-radius clusters. However, we only assume that \(\mathrm{p}\) is bi-Lipschitz on a local bi-Lipschitz window \([0,r_\mathrm{p}]\), as in Assumption 14. Outside this window, the link may flatten or lose metric information. As a result, an internal-average candidate is not automatically safe for all comparisons.

This is where the window-safe idea enters. A set is window-safe if its latent diameter lies safely inside the local bi-Lipschitz window (Definition 24). Within such a set, the local bi-Lipschitz assumption behaves like a global one. If a fuzzy window oracle is available on the first block (Definition [def:fuzzy-window-oracle]), we restrict the exhaustive search to candidates certified by the oracle. This removes the square-root loss that appears in the local route and allows the candidate search to operate at the target scale \(r\), as reflected in the two routes of Theorem 42.

Two-round bootstrap.  The final algorithm constructs the fuzzy window oracle itself; this is the content of Theorem 45. It uses six blocks. First, we run the local route of the three-block extraction at the coarser scale \(r_0=\sqrt r\) on the first three blocks. This produces exact coarse clusters. These coarse clusters are then used as local probes: by averaging their edges to vertices in the fourth block, we certify which vertices lie in a common local bi-Lipschitz window. This gives a fuzzy window oracle on the fourth block, via Lemma 15. Then we run the oracle route of the three-block extraction at the target scale \(r\) on the last three blocks (see Theorem 42). The local-route run at scale \(\sqrt r\) and the oracle-route run at scale \(r\) both use internal candidate scales comparable to \(r\), so their statistical requirements reduce to \[\mathsf{s}n\phi(c r)r^2\gtrsim \log n\] for a model-dependent constant \(c>0\). The final repacking argument removes this constant inside \(\phi\) in the main theorem. Thus the square-root loss is paid only in the preliminary oracle-construction round, not in the final resolution.

Open questions↩︎

We close with several questions left open by the present work.

  1. Can the efficient regime be improved beyond the fixed-accuracy dense setting considered here?

  2. Is there a genuine statistical-computational gap for high-accuracy distance denoising in general metric measure spaces?

  3. It seems the result could be extended to the case where \(\mathrm{p}\) is not necessarily bi-Lipschitz, but only satisfies some weaker regularity condition. For example, if \(\mathrm{p}\) satisfies a local Hölder condition of the form \[\ell_\mathrm{p}|t-t'|^\alpha \le |\mathrm{p}(t)-\mathrm{p}(t')| \le L_\mathrm{p}|t-t'|^\alpha\] for some \(\alpha \in (0,1]\), it seems that the same higher level strategy might still work both on the upper and lower bounds, with a different transition.

Acknowledgments↩︎

The authors were partially supported by Vannevar Bush Faculty Fellowship ONR-N00014- 20-1-2826 and by Simon Investigator award (622132). E.M. was also partially supported by ARO MURI W911NF1910217, NSF DMS-2031883, and NSF award CCF 1918421.

Disclosure of AI-assisted work. H.H. used OpenAI Codex and ChatGPT as auxiliary tools while preparing this manuscript from an older draft, where H.H. provided an older statement or a proof sketch and asked Codex to carry out some of the details. All statements, arguments, and proofs were verified and revised by the authors through multiple rounds of editing.

2 Graph model and standing assumptions↩︎

Definition 6 (Metric measure space / metric probability space). A metric measure space* is a triple \((M,{\rm d},\mu)\), where \((M,{\rm d})\) is a metric space and \(\mu\) is a Borel measure on \(M\).*

If in addition \[\mu(M)=1,\] then \((M,{\rm d},\mu)\) is called a metric probability space.

Throughout, balls are open: \[B(x,r):=\{y\in M:{\rm d}(x,y)< r\}.\]

Definition 7 (Centered subgaussian norm). The centered subgaussian norm* of a random variable \(X\) is defined as \[\|X\|_{\psi_2} := \inf \left\{ \sigma > 0 \;:\; \mathbb{E}\left[\exp\left(\lambda (X - \mathbb{E}X)\right)\right] \le \exp\left(\frac{\sigma^2 \lambda^2}{2}\right) \;\; \text{for all } \lambda \in \mathbb{R} \right\}.\]*

For \(K\in[0,\infty)\), we say that \(X\) is \(K\)-subgaussian* if \[\|X\|_{\psi_2}\le K.\]*

Definition 8 (Random graph model). Let \((M,{\rm d},\mu)\) be a metric probability space, and consider a non-increasing function \[\mathrm{p}:[0,\infty)\to\mathbb{R}.\] We formulate the model in the non-increasing, similarity-oriented convention; non-decreasing real-valued observation models are reduced to this case by multiplying all observations and the link by \(-1\). Let \(\mathbf{V}\) be a finite vertex set of \(n\) vertices, and let \(\mathsf{s}= \mathsf{s}_n \in(0,1]\) be a sparsity parameter.

First, let \(\{X_v\}_{v\in\mathbf{V}}\) be i.i.d.samples of latent points in \(M\) according to \(\mu\). For each unordered pair \(\{u,v\}\subseteq\mathbf{V}\) with \(u\neq v\), let \(\mathcal{U}_{u,v}\sim \mathrm{Unif}[0,1]\), independently over unordered pairs and independently of \(\{X_v\}_{v\in\mathbf{V}}\), and set \(\mathcal{U}_{v,u}:=\mathcal{U}_{u,v}\).

Assume there is a measurable function \[\mathbf{F}:[0,\infty)\times[0,1]\to\mathbb{R}\] such that for every \(t\ge0\), \[\int_0^1 \mathbf{F}(t,s)\,ds=\mathrm{p}(t),\] and such that \[\mathbf{F}(t,\mathcal{U})-\mathrm{p}(t)\] is \({\rm K}_{\mathrm{sg}}\)-subgaussian uniformly in \(t\), where \(\mathcal{U}\sim\mathrm{Unif}[0,1]\). Equivalently, \[\|\mathbf{F}(t,\mathcal{U})-\mathrm{p}(t)\|_{\psi_2}\le {\rm K}_{\mathrm{sg}} \qquad\text{for every }t\ge0.\] Thus, for \(u\neq v\), \[\widetilde{Z}_{u,v} := \mathbf{F}\!\bigl({\rm d}(X_u,X_v),\mathcal{U}_{u,v}\bigr)\] has conditional mean \(\mathrm{p}({\rm d}(X_u,X_v))\) and conditional subgaussian norm at most \({\rm K}_{\mathrm{sg}}\).

For each unordered pair \(\{u,v\}\subseteq\mathbf{V}\) with \(u\neq v\), let \(B_{u,v}\sim\mathrm{Bernoulli}(\mathsf{s})\), independently over \(u<v\), and independently of \(\{X_v\}_{v\in\mathbf{V}}\) and \(\{\mathcal{U}_{u,v}\}_{u<v}\). Set \[B_{v,u}:=B_{u,v}, \qquad Z_{u,v}:=B_{u,v}\widetilde{Z}_{u,v}.\] Finally, set \(Z_{u,u}=0\).

Under this model, for \(U\subseteq\mathbf{V}\), write \[X_U:=(X_u)_{u\in U}\] for the corresponding latent point family. All counts of sampled points are understood with multiplicity; for example, \[|\{u\in U:X_u\in A\}|\] counts vertices whose latent points lie in \(A\).

Remark 9. We refer to \(Z_{u,v}\) as the observed edge weight between \(u\) and \(v\). The collection of these weights is the observed weighted graph. In the Bernoulli soft random geometric graph special case, this observed edge weight is the edge indicator.

Remark 10. When \(0\le \mathrm{p}\le 1\) and \(\mathbf{F}(t,s) = {\boldsymbol{1}}\{s \le \mathrm{p}(t)\}\), the model reduces to a sparse soft random geometric graph with edge probability \(\mathsf{s}\mathrm{p}({\rm d}(X_u,X_v))\). The formulation above allows more general weighted subgaussian observations.

Definition 11 (Lower \(\phi\)-regularity). Let \(\mu\) be a Borel probability measure on a metric space \((M,{\rm d})\), and define \[\mu_{\min}(r) := \inf_{p \in \operatorname{supp}(\mu)} \mu(B(p,r)).\] Let \(I\) be either \([0,r_*)\) for some \(r_*>0\), or \((0,\infty)\), and let \(\phi:I\to[0,\infty)\) be nonnegative and nondecreasing. We say that \((M,{\rm d},\mu)\) satisfies lower \(\phi\)-regularity on \(I\) if \[\mu_{\min}(r)\ge \phi(r) \qquad\text{for all } r\in I.\] When \(I=(0,r_\mu]\), we also say that \((M,{\rm d},\mu)\) is lower \(\phi\)-regularity up to scale \(r_\mu\). When \(I=(0,\infty)\), we simply say that it is lower \(\phi\)-regular. Equivalently, throughout the extraction argument one may replace \(M\) by \(\operatorname{supp}(\mu)\).

Assumption 12 (Support convention). We assume, after replacing the ambient metric space by \(\operatorname{supp}(\mu)\) if necessary, that \[M=\operatorname{supp}(\mu).\]

Assumption 13 (Lower regularity of the measure). Let \(\mu\) be a Borel probability measure on a metric space \((M,{\rm d})\). We assume that there exist \(r_{\mu}>0\) and a nonnegative nondecreasing function \(\phi:(0,r_\mu]\to[0,1]\) such that \((M,{\rm d},\mu)\) is lower \(\phi\)-regularity up to scale \(r_\mu\).

Assumption 15 (Standing model assumptions). Throughout this paper, we consider the random graph model defined in Definition 8 with link function \(\mathrm{p}\) satisfying Assumption 14 and measure \(\mu\) satisfying Assumptions 12 and 13. We may consider the same model with different vertex-set sizes and sparsity parameters \(\mathsf{s}\).

3 Basic definitions, events, and concentration↩︎

Here we define several basic events and estimates that will be used repeatedly in the main construction. These estimates are standard consequences of occupancy bounds, Chernoff–Bernstein inequalities, subgaussian concentration, and union bounds. To keep the main construction readable, their proofs are postponed to Appendix 13.

Here we introduce a global large parameter \[\label{def:global-large-parameter} \Lambda \ge 1,\tag{1}\] whose value will be determined by the needs of the main construction. The parameter \(\Lambda\) is used to control the probability of failure of various events. Unless explicitly stated otherwise, throughout the rest of this paper we work under Assumption 15. In particular, the latent points \((X_v)_{v\in\mathbf{V}}\) are sampled from a metric probability space \((M,{\rm d},\mu)\) satisfying lower \(\phi\)-regularity with \(M=\operatorname{supp}(\mu)\), the observed edge weights \(Z_{u,v}\) are generated by Definition 8, and the link function \(\mathrm{p}\) satisfies the local link assumption of Assumption 14.

3.1 Occupancy event↩︎

The first event is a uniform lower occupancy event, which ensures that every latent ball contains a sufficiently large number of sampled points. This is a consequence of the lower \(\phi\)-regularity assumption and standard occupancy bounds.

Definition 16 (Lower occupancy event). \[\label{eq:def-lower-occupancy-event} \hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,r)}\hypersetup{linkcolor=blue} := \left\{ |\{v\in W:X_v\in B(x,r)\}| \ge \frac{|W|\phi(r/3)}{2} \text{ for every }x\in M \right\}.\qquad{(1)}\] We call \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,r)}\hypersetup{linkcolor=blue}\) the lower occupancy event.

Lemma 2 (Uniform lower occupancy). Let \((M,{\rm d},\mu)\) satisfy lower \(\phi\)-regularity up to scale \(r_\mu\). Thus \(\phi\) is nonnegative and nondecreasing, as in Definition 11. Let \(W\) be a finite vertex set, assume \(n_\star:=|W| \ge 2\), and let \(\{X_v\}_{v\in W}\) be i.i.d.samples from \(\mu\). If \[r\in(0,r_\mu], \qquad \phi(r/6)\ge \Lambda \frac{\log n_\star}{n_\star},\] then \[\mathbb{P}\bigl(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,r)}\hypersetup{linkcolor=blue}\bigr)\ge 1-\exp\left( - \tfrac{1}{16}\Lambda \log(n_\star) \right)\,,\] provided \(\Lambda\) is greater than some universal constant.

3.2 Clusters and empirical averages↩︎

We first record the basic cluster notion and the empirical averages used to test clusters against vertices.

Definition 17 (\((r,\eta)\)-cluster). Assume the graph model of Definition 8. Let \(r>0\) and \(\eta\in[0,1]\). A nonempty subset \(U\subseteq\mathbf{V}\) is called an \((r,\eta)\)-cluster with center \(x\in M\)* if \[\left|\left\{u\in U:X_u\in B(x,r)\right\}\right| \ge (1-\eta)|U|.\]*

Definition 18 (Latent and empirical link averages). For a nonempty set \(U\subseteq\mathbf{V}\), define its latent link average \[\label{eq:def-latent-link-average} \hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} := \frac{1}{|U|} \sum_{u\in U}\mathrm{p}\bigl({\rm d}(X_u,y)\bigr), \qquad y\in M.\qquad{(2)}\] When the argument is a vertex \(v\), we write, by abuse of notation, \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}:=\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(X_v)}\hypersetup{linkcolor=blue}.\] For \(v\in\mathbf{V}\setminus U\), define the empirical link average \[\label{eq:def-normalized-average} \hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue} := \frac{\sum_{u\in U}Z_{u,v}}{\mathsf{s}|U|},\qquad{(3)}\] so that \[\mathbb{E}\bigl[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\mid X_v,X_U\bigr]=\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}.\] Thus \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) is the empirical link average from \(U\) to \(v\), and \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\) is its conditional mean.

The key object is the empirical link average \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) from a set \(U\) to a vertex \(v\). The link-average concentration events ensure that \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) concentrates around its conditional mean \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\). Thus, if \(U\) is a cluster with center \(x\), then \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) is a good proxy for \(\mathrm{p}({\rm d}(x,X_v))\), which is the key to distance recovery from clusters.

Lemma 3 (Fluctuation of empirical link averages for a fixed vertex). Consider the graph model of Assumption 15. Let \(U\subseteq\mathbf{V}\) be nonempty, let \(v\in\mathbf{V}\setminus U\), and set \(m:=|U|\). There exist constants \(c_{\rm link},C_{\rm link}>0\), depending only on \(M_\mathrm{p}\) and \({\rm K}_{\mathrm{sg}}\), such that for every fixed realization \(X_U=x_U\) and \(X_v=x_v\), \[\mathbb{P}\!\left( \left|\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\right|>t \;\middle|\;X_U=x_U,\;X_v=x_v \right) \le C_{\rm link}\exp\!\bigl(-c_{\rm link}\,\mathsf{s}m\,\min\{t^2,1\}\bigr) \qquad\text{for all }t>0.\]

Definition 19 (Link-average concentration event). We associate to \(\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\) the fluctuation scale \[\label{eq:def-fluctuation-scale} \hypersetup{linkcolor=black}\hyperref[eq:def-fluctuation-scale]{\varepsilon_{U}(n_\star)}\hypersetup{linkcolor=blue} := \sqrt{\Lambda \frac{\log n_\star}{\mathsf{s}|U|}}, \qquad n_\star\ge2,\qquad{(4)}\] where \(n_\star\) is a reference size parameter for logarithmic union bounds.

\[\label{eq:def-navigation-event} \hypersetup{linkcolor=black}\hyperref[eq:def-navigation-event]{\mathcal{E}_{\rm link}(U,V;n_\star)}\hypersetup{linkcolor=blue} := \left\{ \forall v\in V: \left|\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\right| \le \hypersetup{linkcolor=black}\hyperref[eq:def-fluctuation-scale]{\varepsilon_{U}(n_\star)}\hypersetup{linkcolor=blue} \right\}.\qquad{(5)}\] We call \(\hypersetup{linkcolor=black}\hyperref[eq:def-navigation-event]{\mathcal{E}_{\rm link}(U,V;n_\star)}\hypersetup{linkcolor=blue}\) the link-average concentration event.

Lemma 4 (Uniform link-average concentration). Consider the graph model of Assumption 15. Let \(U,V\subseteq\mathbf{V}\) be disjoint, with \(U\) nonempty. Let \(n_\star\ge2\) be a reference size parameter. Assume \[\mathsf{s}|U|\ge \Lambda \log n_\star, \qquad |V|\le n_\star\,.\] Then for all fixed realizations \(X_U=x_U\) and \(X_V=x_V\), \[\mathbb{P}\!\left( \hypersetup{linkcolor=black}\hyperref[eq:def-navigation-event]{\mathcal{E}_{\rm link}(U,V;n_\star)}\hypersetup{linkcolor=blue}^{\,c} \;\middle|\;X_U=x_U,\;X_V=x_V \right) \le \exp( - \tfrac{1}{2}c_{\rm link}\,\Lambda \log n_\star)\,,\] provided that \(\Lambda\) is greater than some universal constant depending on \(C_{\rm link}, c_{\rm link}\) (and thus depends on \(M_\mathrm{p}\) and \({\rm K}_{\mathrm{sg}}\)).

3.4 Pair averages↩︎

The next definition generalizes the empirical link average: it averages observed edge weights over pairs of vertices in two sets \(U\) and \(V\). The corresponding expected value is an average of the link function over pairs of latent points in \(X_U\) and \(X_V\).

Definition 20 (Pair averages). For subsets \(U,V\subseteq\mathbf{V}\), define the ordered off-diagonal pair set \[\mathcal{D}(U,V) := \{(u,v)\in U\times V:\;u\neq v\}.\] Then \[|\mathcal{D}(U,V)|=|U||V|-|U\cap V|.\] When \(|\mathcal{D}(U,V)|>0\), define \[\overline{N}\!\left(U,V\right) := \frac{1}{\mathsf{s}|\mathcal{D}(U,V)|} \sum_{(u,v)\in\mathcal{D}(U,V)}Z_{u,v}, \quadand\quad \overline{\mathrm{p}}\!\left(U,V\right) := \frac{1}{|\mathcal{D}(U,V)|} \sum_{(u,v)\in\mathcal{D}(U,V)} \mathrm{p}\!\bigl({\rm d}({X_u},{X_v})\bigr).\] With this convention, \[\mathbb{E}\!\left[\overline{N}\!\left(U,V\right)\mid X_{\mathbf{V}}\right]=\overline{\mathrm{p}}\!\left(U,V\right).\] For \(U=V\) with \(|U|\ge2\), write \[\overline{N}\!\left(U\right):=\overline{N}\!\left(U,U\right), \qquad \overline{\mathrm{p}}\!\left(U\right):=\overline{\mathrm{p}}\!\left(U,U\right).\] We call \(\overline{N}\!\left(U,V\right)\) the pair average between \(U\) and \(V\). Since \(\mathcal{D}(U,U)\) consists of ordered off-diagonal pairs, each unordered pair is counted twice. This agrees with the usual unordered average because \(Z_{u,v}=Z_{v,u}\) and \({\rm d}({X_u},{X_v})={\rm d}({X_v},{X_u})\).

Definition 21 (Uniform pair-average event). \[\label{eq:def-pair-average-event} \hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue} := \left\{ \begin{array}{l} \text{for all }U_1,U_2\subseteq W\text{ with }|U_1|\ge m,\;|U_2|\ge m,\\[0.3ex] \left|\overline{N}\!\left(U_1,U_2\right)-\overline{\mathrm{p}}\!\left(U_1,U_2\right)\right|\le\lambda \end{array} \right\}.\qquad{(6)}\] We call \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}\) the uniform pair-average event.

Remark 22. The event \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}\) allows \(U_1\) and \(U_2\) to overlap. The diagonal is omitted by \(\mathcal{D}(U_1,U_2)\), and the appendix proof handles the resulting duplicate appearances of unordered edges by assigning multiplicities in \(\{0,1,2\}\). The union bound is taken over all cardinalities \(|U_1|=a\), \(|U_2|=b\) with \(a,b\ge m\).

Lemma 5 (Uniform concentration of pair averages). Let \(W\) be a vertex set, and set \(n_\star:=|W|\). Fix \(\varphi\in(0,1)\), set \[m:=\lceil\varphi n_\star\rceil,\] and assume \(m\ge2\). There exist constants \(c_{\text{\tiny{\ref{lem:uniform-pair-average}}}},C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}>0\), depending only on \({\rm K}_{\mathrm{sg}}\) and \(M_\mathrm{p}\), such that for every \(\lambda\in(0,1)\), if \[\mathsf{s}n_\star\varphi\lambda^2 \ge C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}\log(e/\varphi),\] then, conditionally on \(X_W\), \[\mathbb{P}\!\left( \hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}^{\,c} \;\middle|\;X_W \right) \le \exp\{-c_{\text{\tiny{\ref{lem:uniform-pair-average}}}}\varphi n_\star\log(e/\varphi)\}\]

Remark 23. Later we will often verify the stronger sufficient condition \[\mathsf{s}\frac{n_\star}{\Lambda\log n_\star}\varphi\lambda^2 \ge C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}.\] Indeed, since \(\Lambda\ge1\), this implies the same condition with \(\Lambda\) removed from the denominator. For \(a,b\ge e\) and \(k>0\), \[a\ge b\log^k a \quad\Longrightarrow\quad a\ge b\log^k b,\] because \(\log(a)\ge1\) first gives \(a\ge b\), and hence \(\log(a)\ge\log(b)\). Applying this with \[a=n_\star, \qquad b=\frac{C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}}{\mathsf{s}\varphi\lambda^2}, \qquad k=1,\] we obtain \[\mathsf{s}n_\star\varphi\lambda^2 \ge C_{\text{\tiny{\ref{lem:uniform-pair-average}}}} \log\!\left( \frac{C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}}{\mathsf{s}\varphi\lambda^2} \right) \ge C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}\log(e/\varphi).\] The last inequality follows after increasing \(C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}\) if necessary, using \(\mathsf{s}\le1\) and \(\lambda\in(0,1)\).

3.5 Window-safe sets and fuzzy window oracles↩︎

The next definition is the notion of window-safe sets, which are sets whose latent diameter lies safely inside the local bi-Lipschitz window. This ensures that the local bi-Lipschitz assumption behaves like a global one on such sets. The second part of the definition is the notion of a fuzzy window oracle, which is an oracle that certifies pairs of vertices whose latent points lie within a smaller window. This allows us to restrict the candidate search to certified sets, which removes the square-root loss in the main construction.

Definition 24 (Window-safe sets and fuzzy window oracles). For a possibly multiset \(A\subseteq M\), define the latent diameter \[\operatorname{diam}(A) := \sup_{p,p'\in A}{\rm d}({p},{p'}).\] For \(\lambda_{\mathrm{win}}\in(0,1]\), we say a multiset \(A\) of \(M\) is \(\lambda_{\mathrm{win}}\)-window-safe if \[\operatorname{diam}(A)\le \lambda_{\mathrm{win}}r_\mathrm{p}.\] By abuse of notation, a vertex set \(U\subseteq\mathbf{V}\) is \(\lambda_{\mathrm{win}}\)-window-safe when the corresponding latent multiset \(X_U\) has that property. Similarly, \((U,y)\) is \(\lambda_{\mathrm{win}}\)-window-safe when \(X_U\cup\{y\}\) has that property. The case \(\lambda_{\mathrm{win}}=1\) means that the relevant latent diameter lies inside the local bi-Lipschitz window.

For \(0<\alpha_{\mathrm{win}}<\lambda_{\mathrm{win}}<1\), a map \[\mathcal{O}_{\mathrm{win}}:\mathbf{V}\times\mathbf{V}\to\{0,1\}\] is called an \((\alpha_{\mathrm{win}},\lambda_{\mathrm{win}})\)-fuzzy window oracle if, for all \(u,v\in\mathbf{V}\), \[{\rm d}({X_u},{X_v})\le \alpha_{\mathrm{win}}r_\mathrm{p} \quad\Longrightarrow\quad \mathcal{O}_{\mathrm{win}}(u,v)=1,\] and \[{\rm d}({X_u},{X_v})>\lambda_{\mathrm{win}}r_\mathrm{p} \quad\Longrightarrow\quad \mathcal{O}_{\mathrm{win}}(u,v)=0.\] We say that the oracle certifies a set \(U\subseteq\mathbf{V}\) if \[\mathcal{O}_{\mathrm{win}}(u,v)=1 \qquad\text{for every }u,v\in U.\]

Remark 25. If a fuzzy window oracle certifies \(U\), then \(U\) is \(\lambda_{\mathrm{win}}\)-window-safe. Conversely, if \[\operatorname{diam}(X_U)\le \alpha_{\mathrm{win}}r_\mathrm{p},\] then the oracle certifies \(U\). Similarly, if \[\operatorname{diam}(X_U\cup X_V)\le \alpha_{\mathrm{win}}r_\mathrm{p},\] then all pairs between \(U\) and \(V\) are certified by the oracle.

4 Internal-average candidates and link-average separation↩︎

We first introduce the notion of an internal-average candidate, which is a set \(U\) whose internal pair average \(\overline{\mathrm{p}}\!\left(U\right)\) is close to the maximum possible value \(\mathrm{p}(0)\). These sets can be extracted from the observed graph, and the key point is that such a set behaves like a cluster with a center \(\theta_U\).

We then introduce link-average separation, the deterministic condition used by the refinement step to separate points close to the center \(\theta_U\) from points that are far away.

Definition 26 (Internal-average candidate). Let \(U\subseteq\mathbf{V}\) and \(\Delta\ge0\). We say that \(U\) is a \(\Delta\)-internal-average candidate if \[\overline{\mathrm{p}}\!\left(U\right)\ge \mathrm{p}(0)-\Delta.\]

Definition 27 (Average distance). For a nonempty set \(U\subseteq\mathbf{V}\) and a point \(x\in M\), define \[\bar d_U(x) := \frac{1}{|U|}\sum_{u\in U}{\rm d}({X_u},{x}).\]

Lemma 6 (Internal average gap gives a center representative). Let \(U\subseteq\mathbf{V}\) with \(|U|\ge2\), and suppose \[\overline{\mathrm{p}}\!\left(U\right)\ge \mathrm{p}(0)-\Delta.\] Then there exists \(u_U \in U\) such that \(\theta_U := X_{u_U} \in M\) satisfies \[\tfrac{1}{|U|}\sum_{u \in U}\mathrm{p}({\rm d}({X_u},{\theta_U})) \ge \mathrm{p}(0)-\Delta,\] and \[\left|\{u\in U:X_u\notin B(\theta_U,R)\}\right| \le \frac{\Delta}{\ell_\mathrm{p}R}|U| \qquad\text{for every } 0 < R \le r_\mathrm{p}.\] In addition, if \(U\) is \(1\)-window-safe, then the same tail bound holds for every \(R>0\), and moreover \[\bar d_U(\theta_U)\le \frac{\Delta}{\ell_\mathrm{p}}.\]

Proof. Good Center.  Set \(m:=|U|\). Since \(\overline{\mathrm{p}}\!\left(U\right)\) is the ordered off-diagonal average, \[\mathrm{p}(0)-\overline{\mathrm{p}}\!\left(U\right) = \frac{1}{m(m-1)} \sum_{\substack{u,v\in U\\u\neq v}} \left[ \mathrm{p}(0)-\mathrm{p}\!\bigl({\rm d}({X_u},{X_v})\bigr) \right].\] Averaging over the first coordinate, there exists \(u_U\in U\) such that \[\begin{align} \frac{1}{m-1}\sum_{\substack{v\in U\\v\neq u_U}} \left[\mathrm{p}(0)-\mathrm{p}\!\bigl({\rm d}({X_{u_U}},{X_v})\bigr)\right] \le \mathrm{p}(0)-\overline{\mathrm{p}}\!\left(U\right) \le \Delta. \end{align}\] Let us simply set \(\theta_U:=X_{u_U}\), so that the above average is taken over the distances from \(\theta_U\) to the other points in \(U\). Since the term \(u=u_U\) contributes \(\mathrm{p}(0)\), this also gives \[\frac{1}{m}\sum_{u\in U}\mathrm{p}({\rm d}({X_u},{\theta_U})) \ge \mathrm{p}(0)-\Delta.\] Now relying on the lower Lipschitz bound, we have \[\mathrm{p}(0)-\mathrm{p}\!\bigl({\rm d}({\theta_U},{X_v})\bigr) \ge \mathrm{p}(0) - \mathrm{p}\bigl( \min\{{\rm d}({\theta_U},{X_v}), r_\mathrm{p}\}\bigr) \ge \ell_\mathrm{p}\min\{{\rm d}({\theta_U},{X_v}), r_\mathrm{p}\}\,.\] And hence, \[\begin{align} \label{eq:truncated-average} \frac{1}{m-1}\sum_{\substack{v\in U\\v\neq u_U}} \min\{{\rm d}({\theta_U},{X_v}), r_\mathrm{p}\} \le \tfrac{\Delta}{\ell_\mathrm{p}}\,. \end{align}\tag{2}\]

Markov’s inequality.  From the above truncated average, we can deduce a tail bound as long as \(R\le r_\mathrm{p}\) via Markov’s inequality: \[\left|\{u\in U:X_u\notin B(\theta_U,R)\}\right|R \le \sum_{u\in U}\min\{{\rm d}({X_u},{\theta_U}),r_\mathrm{p}\} \le \sum_{\substack{u\in U\\u\neq u_U}}\min\{{\rm d}({X_u},{\theta_U}),r_\mathrm{p}\} \le |U| \cdot \frac{\Delta}{\ell_\mathrm{p}}.\]

Average radius under \(1\)-window-safety.  Assume that \(U\) is \(1\)-window-safe. Then every distance \({\rm d}({\theta_U},{X_v})\) lies in \([0,r_\mathrm{p}]\), so the truncated average bound 2 is actually an average radius bound. Therefore \[\bar d_U(\theta_U) = \frac{1}{m}\sum_{v\in U}{\rm d}({X_v},{\theta_U}) \le \frac{m-1}{m}\frac{\Delta}{\ell_\mathrm{p}} \le \frac{\Delta}{\ell_\mathrm{p}}.\] ◻

Remark 28. Lemma 6 implies that every \(\Delta\)-internal-average candidate \(U\) is an \[\left(R,\frac{\Delta}{\ell_\mathrm{p}R}\right)\text{-cluster}\] with center \(\theta_U\), for every \(0<R\le r_\mathrm{p}\).

The refinement step only needs one deterministic property: the latent link average \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(\cdot)}\hypersetup{linkcolor=blue}\) must separate an inner ball from the complement of a larger ball. We package that property directly.

For an internal-average candidate \(U\), the drop of \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue}\) from its near-maximal value provides a statistic for how far \(y\) lies from the representative point \(\theta_U\).

Remark 30 (Oracle Route and Local Route). The Oracle Route assumes that a fuzzy window oracle is available and uses it to restrict the candidate search to oracle-certified, window-safe candidates. The Local Route uses no oracle; it treats internal-average candidates as approximate localized clusters before applying the same refinement mechanism. We use \({\rm(O)}\) and \(\mathrm{ora}\) for the former, and \({\rm(L)}\) and \(\mathrm{loc}\) for the latter.

Below we give a unified statement of link-average separation for the two routes.

Proof. This is just the combination of the two route-specific propositions below (Propositions 32 and [prop:localized-cluster-separated-seed]). Choose \(A_{\text{\tiny{\ref{prop:link-average-separation}}}}\) so that \[A_{\text{\tiny{\ref{prop:link-average-separation}}}} \ge \max\left\{ L_\mathrm{p}+\frac{L_\mathrm{p}}{\ell_\mathrm{p}}+2, 2(L_\mathrm{p}+M_\mathrm{p})+2 \right\}.\] The lower bound above on \(K\) implies the \(K\)-requirements in Propositions 32 and [prop:localized-cluster-separated-seed] when their \(A\)-parameters are set equal to \(A_{\text{\tiny{\ref{prop:link-average-separation}}}}\). The scale assumption \(Kr\le\tfrac12\min\{1,r_\mathrm{p}\}\) implies the scale assumption in either route-specific proposition.

In the Oracle Route case, apply Proposition 32. In the Local Route case, apply Proposition [prop:localized-cluster-separated-seed] with \(\tau=r\). In both cases, use the same \(A\)-parameter \(A_{\text{\tiny{\ref{prop:link-average-separation}}}}\) and the same outer-radius parameter \(K\). This gives the claimed link-average separation. ◻

4.1 \(1\)-window-safe upgrade and window-safe separation↩︎

Proposition 32 (Window-safe internal-average candidates give link-average separation). Assume \(\lambda_{\mathrm{win}}<1/4\). Let \[A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}} \ge L_\mathrm{p}+\frac{L_\mathrm{p}}{\ell_\mathrm{p}}+2 \quadand\quad K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}} \ge \frac{2A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}}{\ell_\mathrm{p}} \ge 1,\] and let \(r>0\) be small enough so that \[K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r \le \tfrac{1}{2}r_\mathrm{p}.\] Let \(U\subseteq\mathbf{V}\) be a \(\lambda_{\mathrm{win}}\)-window-safe, \(r\)-internal-average candidate with representative \(\theta_U\). Suppose that \(\widehat{\mathrm{p}}(0)\in\mathbb{R}\) satisfies \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le r.\] Then \(U\) is \[\bigl(\theta_U,r,K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r, \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r,r\bigr) \text{-link-average separated}.\]

Remark 33. The condition \(2A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}/\ell_\mathrm{p}\ge1\) is automatic since \(L_\mathrm{p}\ge\ell_\mathrm{p}\). We include it to make clear that \(K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r\ge r\).

Proof. Since \(U\) is \(\lambda_{\mathrm{win}}\)-window-safe and \(\theta_U\in X_U\), every \(u\in U\) satisfies \[{\rm d}({X_u},{\theta_U})\le \lambda_{\mathrm{win}}r_\mathrm{p}.\] Also, \(U\) is \(1\)-window-safe, so Lemma 6 gives \[\bar d_U(\theta_U)\le \frac{r}{\ell_\mathrm{p}}.\]

If \({\rm d}({y},{\theta_U})\le r\), then for every \(u\in U\), \[{\rm d}({X_u},{y}) \le {\rm d}({X_u},{\theta_U})+{\rm d}({\theta_U},{y}) \le \lambda_{\mathrm{win}}r_\mathrm{p}+r \le r_\mathrm{p}.\] The local Lipschitz bound and the triangle inequality give \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \ge \mathrm{p}({\rm d}({\theta_U},{y}))-L_\mathrm{p}\bar d_U(\theta_U) \ge \mathrm{p}(0)-L_\mathrm{p}r-\frac{L_\mathrm{p}}{\ell_\mathrm{p}}r \ge \widehat{\mathrm{p}}(0)-L_\mathrm{p}r-\frac{L_\mathrm{p}}{\ell_\mathrm{p}}r-r \ge \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r+r.\]

Now suppose \({\rm d}({y},{\theta_U})\ge K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r\). If \({\rm d}({y},{\theta_U})\le (1-\lambda_{\mathrm{win}})r_\mathrm{p}\), then the same link-average approximation gives \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le \mathrm{p}({\rm d}({\theta_U},{y}))+L_\mathrm{p}\bar d_U(\theta_U) \le \mathrm{p}(0)-\ell_\mathrm{p}K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r +\frac{L_\mathrm{p}}{\ell_\mathrm{p}}r \le \widehat{\mathrm{p}}(0) -\ell_\mathrm{p}K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r +\frac{L_\mathrm{p}}{\ell_\mathrm{p}}r +r.\] If instead \({\rm d}({y},{\theta_U})>(1-\lambda_{\mathrm{win}})r_\mathrm{p}\), then every \(u\in U\) satisfies \[{\rm d}({X_u},{y}) \ge {\rm d}({y},{\theta_U})-{\rm d}({X_u},{\theta_U}) > (1-2\lambda_{\mathrm{win}})r_\mathrm{p}.\] By monotonicity and the local lower Lipschitz bound, \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le \mathrm{p}(0)-\ell_\mathrm{p}(1-2\lambda_{\mathrm{win}})r_\mathrm{p} \le \mathrm{p}(0)-\ell_\mathrm{p}K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r.\] Since \(\mathrm{p}(0)\le\widehat{\mathrm{p}}(0)+r\), both cases give \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le \widehat{\mathrm{p}}(0)-\ell_\mathrm{p}K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r +\frac{L_\mathrm{p}}{\ell_\mathrm{p}}r+r.\] The choices of \(A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}\) and \(K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}\) imply \[\ell_\mathrm{p}K_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}} \ge 2A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}} \ge A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}} +\frac{L_\mathrm{p}}{\ell_\mathrm{p}}+2,\] and hence \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:window-safe-internal-average-separated-seed}}}}r-r.\] This is exactly the claimed link-average separation. ◻

4.2 Local bi-Lipschitzness case↩︎

Remark 34 (From internal average to \((\tau,\tau)\)-localization). Let \(U\) be a \(\Delta\)-internal-average candidate, and let \(\theta_U\) be the representative point from Lemma 6. If \[\sqrt{\frac{\Delta}{\ell_\mathrm{p}}}\le \tau\le \min\{1,r_\mathrm{p}\},\] then \(U\) is a \((\tau,\tau)\)-cluster with center \(\theta_U\). Indeed, Remark 28 gives \[\frac{|\{u\in U:X_u\notin B(\theta_U,\tau)\}|}{|U|} \le \frac{\Delta}{\ell_\mathrm{p}\tau} \le \tau.\] Equivalently, for every \(C\ge1\), a \(\Delta\)-internal-average candidate is a \[\left( C\sqrt{\frac{\Delta}{\ell_\mathrm{p}}}, C\sqrt{\frac{\Delta}{\ell_\mathrm{p}}} \right)\text{-cluster}\] provided \(C\sqrt{\Delta/\ell_\mathrm{p}}\le \min\{1,r_\mathrm{p}\}\). Thus the estimates below, which use the local bi-Lipschitz window, may be applied to internal-average candidates at the cost of passing to the square-root scale. If an estimate also uses an outer radius \(K\tau\), one must additionally require \((K-1)\tau\le r_\mathrm{p}\).

We now record the link-average separation estimate for genuine \((\tau,\tau)\)-clusters in the same threshold form as the window-safe case. The corresponding pair-average estimate is used in Section 6.

Proposition 35 (Approximate localized clusters give link-average separation). Let \[A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}} \ge 2(L_\mathrm{p}+M_\mathrm{p})+2, \qquad K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}} \ge 1+\frac{2(A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}+2)}{\ell_\mathrm{p}}.\] Let \(U\subseteq\mathbf{V}\) be a \((\tau,\tau)\)-cluster with center \(x\), where \[K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau \le \frac{1}{2}\min\{1,r_\mathrm{p}\}.\] Let \(\widehat{\mathrm{p}}(0)\in\mathbb{R}\) satisfy \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le \tau.\] Then \(U\) is \[\bigl(x,\tau,K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau, \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau,\tau\bigr) \text{-link-average separated}.\]

Proof. Choose \(G\subseteq U\) such that \[|G|\ge(1-\tau)|U|, \qquad X_u\in B(x,\tau)\quad\text{for all }u\in G.\] Let \(C_0:=L_\mathrm{p}+M_\mathrm{p}\). If \({\rm d}({y},{x})\le\tau\), then for every \(u\in G\), \[{\rm d}({X_u},{y})\le2\tau\le r_\mathrm{p},\] and hence \[\mathrm{p}({\rm d}({X_u},{y}))\ge \mathrm{p}(0)-2L_\mathrm{p}\tau.\] The remaining vertices contribute at least \(-M_\mathrm{p}\), while \(\mathrm{p}(0)\le M_\mathrm{p}\). Therefore \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \ge (1-\tau)(\mathrm{p}(0)-2L_\mathrm{p}\tau)-\tau M_\mathrm{p} \ge \mathrm{p}(0)-2C_0\tau \ge \widehat{\mathrm{p}}(0)-(2C_0+1)\tau \ge \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau+\tau.\]

Now suppose \({\rm d}({y},{x})\ge K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau\). For every \(u\in G\), \[{\rm d}({X_u},{y})\ge (K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau.\] Since \(\mathrm{p}\) is non-increasing and bi-Lipschitz on \([0,r_\mathrm{p}]\), \[\mathrm{p}({\rm d}({X_u},{y})) \le \mathrm{p}((K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau) \le \mathrm{p}(0)-\ell_\mathrm{p}(K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau.\] The remaining vertices contribute at most \(\mathrm{p}(0)\). Hence \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le (1-\tau)\bigl(\mathrm{p}(0)-\ell_\mathrm{p}(K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau\bigr) + \tau\mathrm{p}(0) = \mathrm{p}(0)-(1-\tau)\ell_\mathrm{p}(K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau.\] Since \(\tau\le1/2\) and \[K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}} \ge 1+\frac{2(A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}+2)}{\ell_\mathrm{p}},\] we have \[(1-\tau)\ell_\mathrm{p}(K_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}-1)\tau \ge (A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}+2)\tau.\] \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(y)}\hypersetup{linkcolor=blue} \le \mathrm{p}(0)-(A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}+2)\tau \le \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:localized-cluster-separated-seed}}}}\tau-\tau.\] This is exactly the claimed link-average separation. ◻

5 Internal-average search and candidate families↩︎

We now turn the internal-average primitives into a concrete candidate family. The output of this section is only \[\bigl(\widehat{\mathrm{p}}(0),\mathcal{C}_\rho\bigr):\] an observable estimate of the top link value and a large family of raw internal-average candidates covering the latent space. The subsequent sparsification of \(\mathcal{C}_\rho\) is handled in Section 6.

Lemma 7 (Estimating \(\mathrm{p}(0)\) from the maximal internal average). Let \(W\subseteq\mathbf{V}\) be a vertex set, and set \(n_\star:=|W|\). Fix \[\rho\in(0,\min\{r_\mu,r_\mathrm{p}/2\}],\] and set \[m:=\left\lfloor\frac{\phi(\rho/3)}{2}n_\star\right\rfloor, \qquad \widehat{\mathrm{p}}(0):=\max\{\overline{N}\!\left(U\right):U\subseteq W,\;|U|=m\}.\] Assume \(m\ge2\). On the event \[\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,\rho)}\hypersetup{linkcolor=blue}\cap\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m)}\hypersetup{linkcolor=blue},\] we have \[\mathrm{p}(0)-3L_\mathrm{p}\rho \le \widehat{\mathrm{p}}(0) \le \mathrm{p}(0)+L_\mathrm{p}\rho.\] In particular, \(|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le3L_\mathrm{p}\rho\).

Proof. Fix \(x\in M\). On \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,\rho)}\hypersetup{linkcolor=blue}\), there exists \(U_x\subseteq W\) with \(|U_x|=m\) such that \[X_u\in B(x,\rho) \qquad\text{for every }u\in U_x.\] Thus \({\rm d}({X_u},{X_v})<2\rho\) for all distinct \(u,v\in U_x\), and hence \[\overline{\mathrm{p}}\!\left(U_x\right)\ge \mathrm{p}(2\rho).\] Using \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m)}\hypersetup{linkcolor=blue}\), \[\widehat{\mathrm{p}}(0)\ge\overline{N}\!\left(U_x\right)\ge\overline{\mathrm{p}}\!\left(U_x\right)-L_\mathrm{p}\rho \ge \mathrm{p}(2\rho)-L_\mathrm{p}\rho.\] Since \(2\rho\le r_\mathrm{p}\) and \(\mathrm{p}\) is \(L_\mathrm{p}\)-Lipschitz on \([0,r_\mathrm{p}]\), \[\mathrm{p}(2\rho)\ge \mathrm{p}(0)-2L_\mathrm{p}\rho,\] which gives the lower bound.

For the upper bound, let \(U\subseteq W\) have \(|U|=m\). Since \(\mathrm{p}\) is non-increasing, \(\overline{\mathrm{p}}\!\left(U\right)\le\mathrm{p}(0)\). Again using \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m)}\hypersetup{linkcolor=blue}\), \[\overline{N}\!\left(U\right)\le\overline{\mathrm{p}}\!\left(U\right)+L_\mathrm{p}\rho\le\mathrm{p}(0)+L_\mathrm{p}\rho.\] Taking the maximum over all such \(U\) proves the claim. ◻

Proposition 36 (Internal-average search produces candidate families). Let \(W\subseteq\mathbf{V}\) be a vertex set, and set \(n_\star:=|W|\). Let \[C_*\ge \frac{48L_\mathrm{p}}{\ell_\mathrm{p}}, \qquad 0<\rho\le \min\{r_\mu,r_\mathrm{p}/C_*\}.\] Set \[m_\rho:=\left\lfloor\frac{\phi(\rho/3)}{2}n_\star\right\rfloor, \qquad \Delta_\rho:=4L_\mathrm{p}\rho,\] assume \(m_\rho\ge2\), and define \[\widehat{\mathrm{p}}(0):=\max\{\overline{N}\!\left(U\right):U\subseteq W,\;|U|=m_\rho\},\] \[\mathcal{C}_\rho := \{U\subseteq W:\;|U|=m_\rho,\;\overline{N}\!\left(U\right)\ge \widehat{\mathrm{p}}(0)-\Delta_\rho\}.\] On the good event \[\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,\rho)}\hypersetup{linkcolor=blue}\cap\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m_\rho)}\hypersetup{linkcolor=blue},\] the following hold.

  1. The maximal internal average satisfies \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le 3L_\mathrm{p}\rho.\]

  2. For every \(x\in M\), there exists \(U_x\in\mathcal{C}_\rho\) such that \[X_u\in B(x,\rho)\qquad\text{for every }u\in U_x.\]

  3. Every \(U\in\mathcal{C}_\rho\) is a \(8L_\mathrm{p}\rho\)-internal-average candidate; namely, for every such \(U\), \[\overline{\mathrm{p}}\!\left(U\right)\ge \mathrm{p}(0)-8L_\mathrm{p}\rho.\]

Proof. By Lemma 7, \[\mathrm{p}(0)-3L_\mathrm{p}\rho \le \widehat{\mathrm{p}}(0) \le \mathrm{p}(0)+L_\mathrm{p}\rho.\] This proves the first assertion.

For every \(x\in M\), the event \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,\rho)}\hypersetup{linkcolor=blue}\) gives a set \(U_x\subseteq W\) with \(|U_x|=m_\rho\) and \(X_u\in B(x,\rho)\) for all \(u\in U_x\). As in the proof of Lemma 7, \[\overline{N}\!\left(U_x\right) \ge \mathrm{p}(2\rho)-L_\mathrm{p}\rho \ge \mathrm{p}(0)-3L_\mathrm{p}\rho.\] Since \[\widehat{\mathrm{p}}(0)\le \mathrm{p}(0)+L_\mathrm{p}\rho,\] we have \[\overline{N}\!\left(U_x\right) \ge \widehat{\mathrm{p}}(0)-4L_\mathrm{p}\rho = \widehat{\mathrm{p}}(0)-\Delta_\rho.\] Thus \(U_x\in\mathcal{C}_\rho\), proving the second assertion.

Now let \(U\in\mathcal{C}_\rho\). Then \[\overline{N}\!\left(U\right)\ge \widehat{\mathrm{p}}(0)-\Delta_\rho.\] Using the lower bound \[\widehat{\mathrm{p}}(0)\ge \mathrm{p}(0)-3L_\mathrm{p}\rho\] and the event \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m_\rho)}\hypersetup{linkcolor=blue}\), we get \[\overline{\mathrm{p}}\!\left(U\right) \ge \overline{N}\!\left(U\right)-L_\mathrm{p}\rho \ge \mathrm{p}(0)-8L_\mathrm{p}\rho.\] This proves the third assertion. ◻

6 Refinement nets from candidate families↩︎

The raw family \(\mathcal{C}_\rho\) from Section 5 is too large to use directly. This section extracts a small subfamily whose representatives cover \(M\), using the link-average separation properties from Section 4. The resulting object is called a refinement net. As in the previous section, the main result is a unified statement, Proposition 38, covering both the oracle route and the local route.

Definition 37 (Refinement net). Let \(R_{\rm net},R_{\rm in},R_{\rm out},\gamma>0\), and let \(M_{\rm net}\ge1\). Let \(\Theta\in\mathbb{R}\). A family \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] is a refinement net with parameters \[\Theta,\quad R_{\rm net},\quad R_{\rm in},\quad R_{\rm out},\quad \gamma,\quad M_{\rm net},\] if:

  1. the centers \(\{\theta_U:U\in\mathcal{N}\}\) form an \(R_{\rm net}\)-net of \(M\);

  2. for every \(U\in\mathcal{N}\), \(U\) is \[(\theta_U,R_{\rm in},R_{\rm out},\Theta,\gamma)\text{-link-average separated};\]

  3. \(|\mathcal{N}|\le M_{\rm net}\).

Proposition 38 (Candidate families produce refinement nets). There exists a constant \[K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}>0\] depending only on \(L_\mathrm{p},\ell_\mathrm{p},M_\mathrm{p}\) such that the following holds. Work in the setup of Proposition 36, and assume the good event there. Let \(K\ge K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\).

Consider either of the following two cases:

  • Oracle Route: Let \(\mathcal{O}_{\mathrm{win}}\) be an \((\alpha_{\mathrm{win}},\lambda_{\mathrm{win}})\)-fuzzy window oracle with \[\lambda_{\mathrm{win}}\le \frac{1}{8}, \qquad \rho\le \frac{\alpha_{\mathrm{win}}}{2} r_\mathrm{p}.\] Let \(r_{\rm ref}>0\) satisfy \[K\rho \le r_{\rm ref}.\]

  • Local Route: Let \(r_{\rm ref}>0\) satisfy \[K\sqrt{\rho} \le r_{\rm ref}.\]

Suppose in addition that \[K^3 r_{\rm ref} \le \min\{1,r_\mathrm{p}\}.\] Then there is a procedure, using pair average comparisons between candidates and the oracle in the Oracle Route case, that outputs a subfamily \(\mathcal{N}\). For suitable analysis representatives \(\theta_U\), the family \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] is a refinement net with parameters \[\begin{gather} \Theta=\widehat{\mathrm{p}}(0) -A_{\text{\tiny{\ref{prop:link-average-separation}}}}r_{\rm ref},\qquad R_{\rm net}= \tfrac13 Kr_{\rm ref},\\ R_{\rm in}=r_{\rm ref},\qquad R_{\rm out}=Kr_{\rm ref},\qquad \gamma=r_{\rm ref},\qquad M_{\rm net}=|W|, \end{gather}\] where \(W\) is the vertex set used in Proposition 36.

Proof. This is the combination of the two route-specific propositions below. Choose \(K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\) larger than \(4\), larger than \(48L_\mathrm{p}/\ell_\mathrm{p}\), larger than \(8L_\mathrm{p}\), large enough compared with the constants in Propositions 39 and 40, and large enough that \[K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}} \ge 1+\frac{2(A_{\text{\tiny{\ref{prop:link-average-separation}}}}+2)}{\ell_\mathrm{p}}.\] Let \(K\ge K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\). The scale condition above then implies the scale assumptions in the corresponding route proposition, including \[Kr_{\rm ref} \le \tfrac12\min\{1,r_\mathrm{p}\}.\] In either case, apply the corresponding route-specific proposition. By the choice of \(K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\), the route-specific outer radius is \(Kr_{\rm ref}\), and the route-specific net radius in either case is \[\tfrac13Kr_{\rm ref}.\] ◻

Both route constructions use the same selection mechanism. One first builds a comparison graph \(H\) on a candidate family: in the Oracle Route case the vertices are the oracle-certified candidates, while in the Local Route case the vertices are all candidates in \(\mathcal{C}_\rho\). Edges are determined by observable information, namely the oracle filter when available and empirical pair average comparisons. A maximal independent set of this graph keeps only well-separated representatives, while maximality preserves coverage. The representatives \(\theta_U\) are analysis witnesses; the procedure itself only uses the candidate family, the oracle in the Oracle Route case, the empirical pair averages, and \(\widehat{\mathrm{p}}(0)\). The next lemma isolates this deterministic selection step.

Lemma 8 (Comparison graph selection). Let \(\mathcal{C}\) be a candidate family with representatives \(\theta_U\), and let \(H\) be any graph on \(\mathcal{C}\). Write \(U\stackrel{H}{\sim} W\) when \(U,W\) are adjacent in \(H\). Suppose there are \(\delta,\rho_{\rm cov}>0\) and \(K_{\rm cmp}\ge1\) such that \[{\rm d}({\theta_U},{\theta_W})\le \delta \quad\Longrightarrow\quad U\stackrel{H}{\sim} W,\] \[U\stackrel{H}{\sim} W \quad\Longrightarrow\quad {\rm d}({\theta_U},{\theta_W})\le K_{\rm cmp}\delta,\] and for every \(x\in M\) there exists \(U_x\in\mathcal{C}\) with \[{\rm d}({x},{\theta_{U_x}})\le \rho_{\rm cov}.\] Then every maximal independent set \(\mathcal{N}\subseteq\mathcal{C}\) satisfies \[U\neq W\in\mathcal{N} \quad\Longrightarrow\quad {\rm d}({\theta_U},{\theta_W})>\delta,\] and its representatives form a \((\rho_{\rm cov}+K_{\rm cmp}\delta)\)-net of \(M\).

Proof. The first implication gives separation: if two distinct vertices of \(\mathcal{N}\) had representative distance at most \(\delta\), they would be adjacent. For covering, fix \(x\in M\) and choose \(U_x\). By maximality, either \(U_x\in\mathcal{N}\), or \(U_x\stackrel{H}{\sim} U\) for some \(U\in\mathcal{N}\). In the latter case, \[{\rm d}({x},{\theta_U}) \le {\rm d}({x},{\theta_{U_x}})+{\rm d}({\theta_{U_x}},{\theta_U}) \le \rho_{\rm cov}+K_{\rm cmp}\delta.\] ◻

6.1 Oracle route↩︎

Proposition 39 (Oracle route produces a refinement net). There exists a constant \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}>0\), depending only on \(L_\mathrm{p},\ell_\mathrm{p},M_\mathrm{p}\), such that the following holds. Work in the setup of Proposition 36, and assume the good event there. Let \(\mathcal{O}_{\mathrm{win}}\) be an \((\alpha_{\mathrm{win}},\lambda_{\mathrm{win}})\)-fuzzy window oracle with \[\lambda_{\mathrm{win}}\le \frac{1}{8}, \qquad \rho\le \frac{\alpha_{\mathrm{win}}}{2} r_\mathrm{p}.\] For any \(K\ge K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) and any \(r_{\rm ref}>0\) satisfying \[K\rho \le r_{\rm ref}, \qquad K^3r_{\rm ref} \le \min\{1,r_\mathrm{p}\},\] define the oracle-filtered candidate family \[\mathcal{C}_\rho^{\mathrm{ora}} := \left\{ U\in\mathcal{C}_\rho: \mathcal{O}_{\mathrm{win}}(u,v)=1 \text{ for every }u,v\in U \right\}.\] For each \(U\in\mathcal{C}_\rho^{\mathrm{ora}}\), let \(\theta_U=X_{u_U}\) be the representative obtained from Lemma 6, applied with \(\Delta=8L_\mathrm{p}\rho\). Define a graph \(H_{\mathrm{ora}}\) on vertex set \(\mathcal{C}_\rho^{\mathrm{ora}}\) by joining \(U_1,U_2\) whenever \[\overline{N}\!\left(U_1,U_2\right) \ge \widehat{\mathrm{p}}(0) -2L_\mathrm{p}\sqrt{K}r_{\rm ref}.\] Let \(\mathcal{N}_{\mathrm{ora}}\subseteq\mathcal{C}_\rho^{\mathrm{ora}}\) be any maximal independent set of \(H_{\mathrm{ora}}\). Then the family \[\mathfrak N_{\mathrm{ora}} := \{(U,\theta_U): U\in\mathcal{N}_{\mathrm{ora}}\}\] is a refinement net with parameters \[\begin{gather} \Theta=\widehat{\mathrm{p}}(0) -A_{\text{\tiny{\ref{prop:link-average-separation}}}}r_{\rm ref},\qquad R_{\rm net}= \tfrac{1}{3}Kr_{\rm ref},\\ R_{\rm in}=r_{\rm ref},\qquad R_{\rm out}=Kr_{\rm ref},\qquad \gamma=r_{\rm ref},\qquad M_{\rm net}=|W|. \end{gather}\]

We prove Proposition 39. The comparison graph \(H_{\mathrm{ora}}\) is built from empirical pair averages. To apply Lemma 8, we need two deterministic pair average facts: one approximates pair averages while all relevant points stay inside the local link window, and the other forces a pair average drop when two window-safe representatives are beyond that window.

Proof. For every \((u_1,u_2)\in\mathcal{D}(U_1,U_2)\), the assumed diameter bound puts both \({\rm d}({X_{u_1}},{X_{u_2}})\) and \({\rm d}({x_1},{x_2})\) inside \([0,r_\mathrm{p}]\). Therefore the local \(L_\mathrm{p}\)-Lipschitz bound applies: \[\begin{align} \left|\overline{\mathrm{p}}\!\left(U_1,U_2\right) - \mathrm{p}\!\bigl({\rm d}({x_1},{x_2})\bigr)\right| \le& \frac{1}{|\mathcal{D}(U_1,U_2)|} \sum_{(u_1,u_2) \in \mathcal{D}(U_1,U_2)} L_\mathrm{p}\left|{\rm d}({X_{u_1}},{X_{u_2}}) - {\rm d}({x_1},{x_2})\right|\\ \le& \frac{1}{|\mathcal{D}(U_1,U_2)|} \sum_{(u_1,u_2) \in \mathcal{D}(U_1,U_2)} L_\mathrm{p}({\rm d}({X_{u_1}},{x_1})+{\rm d}({X_{u_2}},{x_2}))\\ \le& \frac{1}{|\mathcal{D}(U_1,U_2)|} \sum_{u_1 \in U_1,u_2 \in U_2} L_\mathrm{p}({\rm d}({X_{u_1}},{x_1})+{\rm d}({X_{u_2}},{x_2}))\\ \le& \frac{|U_1||U_2|}{|\mathcal{D}(U_1,U_2)|}\,2L_\mathrm{p}R \le 4L_\mathrm{p}R, \end{align}\] where in the last step we used the fact that \[|\mathcal{D}(U_1,U_2)| \ge |U_1||U_2| - \min\{|U_1|,|U_2|\} \ge m(m-1)\,.\] ◻

Lemma 10 (Far representatives force a pair-average drop). Assume \(\lambda_{\mathrm{win}}<1/4\). Let \(U_1,U_2\subseteq\mathbf{V}\), with \(|\mathcal{D}(U_1,U_2)|>0\), and let \(x_i\in M\), \(i=1,2\). Suppose \[{\rm d}({X_u},{x_i})\le \lambda_{\mathrm{win}}r_\mathrm{p} \qquad (u\in U_i,\;i=1,2).\] If \[{\rm d}({x_1},{x_2})\ge (1-2\lambda_{\mathrm{win}})r_\mathrm{p},\] then \[\overline{\mathrm{p}}\!\left(U_1,U_2\right) \le \mathrm{p}((1-4\lambda_{\mathrm{win}})r_\mathrm{p}) \le \mathrm{p}(0)-\ell_\mathrm{p}(1-4\lambda_{\mathrm{win}})r_\mathrm{p}.\]

Proof. For \(u_i\in U_i\), \[{\rm d}({X_{u_1}},{X_{u_2}}) \ge {\rm d}({x_1},{x_2})-{\rm d}({X_{u_1}},{x_1})-{\rm d}({X_{u_2}},{x_2}) \ge (1-4\lambda_{\mathrm{win}})r_\mathrm{p}.\] Since \(\mathrm{p}\) is non-increasing, \[\mathrm{p}({\rm d}({X_{u_1}},{X_{u_2}})) \le \mathrm{p}((1-4\lambda_{\mathrm{win}})r_\mathrm{p}).\] Averaging gives the first inequality. Since \((1-4\lambda_{\mathrm{win}})r_\mathrm{p}\in[0,r_\mathrm{p}]\), the local lower bi-Lipschitz bound gives the second inequality. ◻

Proof of Proposition 39. Set \[\rho_{\rm sep}:= \sqrt{K}r_{\rm ref},\] which is the separation scale for the comparison graph \(H_{\mathrm{ora}}\). Taking \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) large enough, the assumptions on \(r_{\rm ref}\) imply \[8L_\mathrm{p}\rho\le r_{\rm ref}.\]

The oracle-filtered family still covers.  For every \(x\in M\), Proposition 36 gives \(U_x\in\mathcal{C}_\rho\) such that \[X_u\in B(x,\rho) \qquad\text{for every }u\in U_x.\] Thus, for \(u,v\in U_x\), \[{\rm d}({X_u},{X_v})<2\rho\le \alpha_{\mathrm{win}}r_\mathrm{p},\] by the assumption on \(\alpha_{\mathrm{win}}\). The oracle therefore certifies every pair in \(U_x\), so \[U_x\in\mathcal{C}_\rho^{\mathrm{ora}}.\]

Average-radius control.  If \(U\in\mathcal{C}_\rho^{\mathrm{ora}}\), then the oracle implication \(\mathcal{O}_{\mathrm{win}}(u,v)=1\Rightarrow {\rm d}({X_u},{X_v})\le \lambda_{\mathrm{win}}r_\mathrm{p}\) shows that \(U\) is \(\lambda_{\mathrm{win}}\)-window-safe and hence \(1\)-window-safe. Since Proposition 36 gives \[\overline{\mathrm{p}}\!\left(U\right)\ge \mathrm{p}(0)-8L_\mathrm{p}\rho,\] Lemma 6 gives \[\begin{align} \label{eq:oracle-candidate-average-radius-control} \bar d_U(\theta_U) \le \frac{8L_\mathrm{p}}{\ell_\mathrm{p}}\rho \qquad\text{for every }U\in\mathcal{C}_\rho^{\mathrm{ora}}. \end{align}\tag{3}\] Moreover, since \(\theta_U\in X_U\), every \(u\in U\) satisfies \[{\rm d}({X_u},{\theta_U})\le \lambda_{\mathrm{win}}r_\mathrm{p}.\]

Close representatives are adjacent.  Let \(U_1,U_2\in\mathcal{C}_\rho^{\mathrm{ora}}\), and write \[d_{12}:={\rm d}({\theta_{U_1}},{\theta_{U_2}}).\] If \(d_{12}\le\rho_{\rm sep}\), then for every \(u_i\in U_i\), \[{\rm d}({X_{u_1}},{X_{u_2}}) \le {\rm d}({X_{u_1}},{\theta_{U_1}})+d_{12}+{\rm d}({\theta_{U_2}},{X_{u_2}}) \le \lambda_{\mathrm{win}}r_\mathrm{p}+ \rho_{\rm sep} + \lambda_{\mathrm{win}}r_\mathrm{p} \le \tfrac{1}{4}r_\mathrm{p}+ \rho_{\rm sep} \le r_\mathrm{p},\] where the last inequality follows from \(\rho_{\rm sep}=\sqrt{K}r_{\rm ref} \le K^{-2.5}\min\{1,r_\mathrm{p}\}\). Since it holds for every \(u_i\in U_i\), we have \[\operatorname{diam}(X_{U_1}\cup X_{U_2}\cup \{\theta_{U_1},\theta_{U_2}\}) \le r_\mathrm{p}.\] Lemma 9 with \(\bar d_U(\theta_U) \le \frac{8L_\mathrm{p}}{\ell_\mathrm{p}}\rho\) from 3 and the event \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m_\rho)}\hypersetup{linkcolor=blue}\) give \[\left| \overline{N}\!\left(U_1,U_2\right) - \mathrm{p}(d_{12}) \right| \le \left| \overline{N}\!\left(U_1,U_2\right) - \overline{\mathrm{p}}\!\left(U_1,U_2\right) \right| + \left| \overline{\mathrm{p}}\!\left(U_1,U_2\right) - \mathrm{p}(d_{12}) \right| \le 4L_\mathrm{p}\cdot \frac{8L_\mathrm{p}}{\ell_\mathrm{p}}\rho + L_\mathrm{p}\rho \le 0.1L_\mathrm{p}\rho_{\rm sep},\] provided \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) is large enough. Therefore \[\overline{N}\!\left(U_1,U_2\right) \ge \mathrm{p}(d_{12})-0.1 L_\mathrm{p}\rho_{\rm sep} \ge \mathrm{p}(0)-L_\mathrm{p}\rho_{\rm sep}-0.1 L_\mathrm{p}\rho_{\rm sep} \ge \widehat{\mathrm{p}}(0)- 3L_\mathrm{p}\rho -L_\mathrm{p}\rho_{\rm sep}-0.1 L_\mathrm{p}\rho_{\rm sep} \ge \widehat{\mathrm{p}}(0)- 2L_\mathrm{p}\rho_{\rm sep}\] where the last inequality holds when \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) is large enough. Thus \(U_1\stackrel{H_{\mathrm{ora}}}{\sim}U_2\).

Adjacent representatives are not too far.  Assume now that \[d_{12}>K_{\rm cover}\rho_{\rm sep},\] for some \(K_{\rm cover}\) to be chosen later. We show that \(U_1\) and \(U_2\) are not adjacent. First suppose \[d_{12}\le (1-2\lambda_{\mathrm{win}})r_\mathrm{p}.\] Then the same triangle-inequality check as above gives \[\operatorname{diam}(X_{U_1}\cup X_{U_2}\cup \{\theta_{U_1},\theta_{U_2}\}) \le r_\mathrm{p},\] and the same approximation bound gives \[\overline{N}\!\left(U_1,U_2\right) \le \mathrm{p}(d_{12})+0.1 L_\mathrm{p}\rho_{\rm sep} \le \mathrm{p}(0)-\ell_\mathrm{p}d_{12}+0.1 L_\mathrm{p}\rho_{\rm sep} \le \mathrm{p}(0)-\ell_\mathrm{p}K_{\rm cover}\rho_{\rm sep}+0.1 L_\mathrm{p}\rho_{\rm sep} \le \widehat{\mathrm{p}}(0)-2L_\mathrm{p}\rho_{\rm sep},\] where \(K_{\rm cover}\) is chosen large enough depending only on \(L_\mathrm{p}\) and \(\ell_\mathrm{p}\), for example, \[K_{\rm cover} = \tfrac{1}{\ell_\mathrm{p}} \left( 2L_\mathrm{p}+ 0.1 L_\mathrm{p}+ 3L_\mathrm{p} \right)\] suffices, due to \(\rho \le \rho_{\rm sep}\).

Thus \(U_1\) and \(U_2\) are not adjacent.

It remains to consider \[d_{12}>(1-2\lambda_{\mathrm{win}})r_\mathrm{p}.\] Lemma 10, applied with \(x_i=\theta_{U_i}\), gives \[\overline{\mathrm{p}}\!\left(U_1,U_2\right) \le \mathrm{p}(0)-\ell_\mathrm{p}(1-4\lambda_{\mathrm{win}})r_\mathrm{p}.\] Together with \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m_\rho)}\hypersetup{linkcolor=blue}\), this yields \[\overline{N}\!\left(U_1,U_2\right) \le \mathrm{p}(0)-\ell_\mathrm{p}(1-4\lambda_{\mathrm{win}})r_\mathrm{p}+L_\mathrm{p}\rho < \widehat{\mathrm{p}}(0)-2L_\mathrm{p}\rho_{\rm sep},\] provided \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) is large enough. Thus, in all cases, adjacency implies \[{\rm d}({\theta_{U_1}},{\theta_{U_2}})\le K_{\rm cover}\rho_{\rm sep}.\]

Apply the comparison graph selection lemma.  The oracle-filtered covering step gives, for every \(x\in M\), a candidate \(U_x\in\mathcal{C}_\rho^{\mathrm{ora}}\) with \[{\rm d}({x},{\theta_{U_x}})\le\rho.\] The preceding two steps verify the hypotheses of Lemma 8 with \[\delta=\rho_{\rm sep}, \qquad \rho_{\rm cov}=\rho, \qquad K_{\rm cmp}=K_{\rm cover}.\] Thus the selected representatives are \(\rho_{\rm sep}\)-separated and form a \((\rho+K_{\rm cover}\rho_{\rm sep})\)-net. Since \(\rho\le\rho_{\rm sep}\), this net radius is at most \[(1+K_{\rm cover})\rho_{\rm sep}.\] By increasing \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) if necessary, this is at most \(\tfrac{K}{3} r_{\rm ref}\). The separation also implies that the representatives \(\theta_U=X_{u_U}\), \(U\in\mathcal{N}_{\mathrm{ora}}\), are all distinct. Hence the map \(U\mapsto u_U\) injects \(\mathcal{N}_{\mathrm{ora}}\) into \(W\), and \(|\mathcal{N}_{\mathrm{ora}}|\le |W|\).

Separated-seed certificates.  For every selected \(U\), Proposition 36 gives \[\overline{\mathrm{p}}\!\left(U\right)\ge \mathrm{p}(0)-8L_\mathrm{p}\rho,\] and the oracle filter gives \(\lambda_{\mathrm{win}}\)-window-safe. Also \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le 3L_\mathrm{p}\rho\le r_{\rm ref}.\] Since \(8L_\mathrm{p}\rho\le r_{\rm ref}\), \(U\) is an \(r_{\rm ref}\)-internal-average candidate. The choice of \(K_{\text{\tiny{\ref{prop:oracle-candidate-net}}}}\) and the assumption \(K^3r_{\rm ref}\le \min\{1,r_\mathrm{p}\}\) give the \(K\)-requirements in Proposition 31. Therefore Proposition 31, in the Oracle Route case, applied with \(r=r_{\rm ref}\), the estimate for \(\widehat{\mathrm{p}}(0)\), and this value of \(K\), gives the required refinement-net certificate. ◻

6.2 Local route↩︎

In the local route, each \(U\in\mathcal{C}_\rho\) is viewed as an \((r,r)\)-cluster with center \(\theta_U\), at the cost of passing from \(r\asymp \rho\) to \(r\asymp \sqrt{\rho}\), as explained in Remark 34.

We prove Proposition 40. Here all candidates are first viewed as approximate localized clusters. The only additional ingredient needed for Lemma 8 is the following pair-average estimate, which relates the empirical comparison graph \(H_{\mathrm{loc}}\) to distances between the cluster centers.

Lemma 11 (Pair averages for approximate local clusters). Let \(U_1,U_2\subseteq\mathbf{V}\) with \[|U_1|=|U_2|=:m\ge2.\] For \(i=1,2\), suppose that \(U_i\) is a \((\tau,\tau)\)-cluster with center \(x_i\in M\), where \(0<\tau\le1/2\). Then \[\mathrm{p}\!\bigl({\rm d}({x_1},{x_2})+2\tau\bigr)-8M_\mathrm{p}\tau \le \overline{\mathrm{p}}\!\left(U_1,U_2\right) \le \mathrm{p}\!\bigl(({\rm d}({x_1},{x_2})-2\tau)_+\bigr)+8M_\mathrm{p}\tau.\]

Proof. For \(i=1,2\), choose \(G_i\subseteq U_i\) such that \[|G_i|\ge(1-\tau)m, \qquad X_u\in B(x_i,\tau)\quad\text{for all }u\in G_i,\] and set \(B_i:=U_i\setminus G_i\). Let \[d_0:={\rm d}({x_1},{x_2}), \qquad \underline A:=\mathrm{p}(d_0+2\tau), \qquad \overline{A}:=\mathrm{p}((d_0-2\tau)_+), \qquad D_{12}:=|\mathcal{D}(U_1,U_2)|.\] For every \((u_1,u_2)\in G_1\times G_2\), \(u_1\neq u_2\), the triangle inequality gives \[(d_0-2\tau)_+\le {\rm d}({X_{u_1}},{X_{u_2}})\le d_0+2\tau.\] Hence the corresponding \(\mathrm{p}\)-values lie between \(\underline A\) and \(\overline{A}\). Let \(\mathcal{B}\subseteq\mathcal{D}(U_1,U_2)\) be the set of ordered pairs for which at least one endpoint lies in \(B_1\cup B_2\). Then \[|\mathcal{B}|\le |B_1|m+|B_2|m\le 2\tau m^2, \qquad D_{12}\ge m(m-1),\] and therefore, since \(m\ge2\) and \(\tau\le1/2\), \[\frac{|\mathcal{B}|}{D_{12}}\le 4\tau.\] Because all \(\mathrm{p}\)-values have absolute value at most \(M_\mathrm{p}\), replacing the good-pair bounds by arbitrary bad-pair values can change the average by at most \(2M_\mathrm{p}|\mathcal{B}|/D_{12}\). Consequently, \[\overline{\mathrm{p}}\!\left(U_1,U_2\right)\ge \underline A-2M_\mathrm{p}\frac{|\mathcal{B}|}{D_{12}} \ge \underline A-8M_\mathrm{p}\tau\] and \[\overline{\mathrm{p}}\!\left(U_1,U_2\right)\le \overline{A}+2M_\mathrm{p}\frac{|\mathcal{B}|}{D_{12}} \le \overline{A}+8M_\mathrm{p}\tau.\] ◻

Proof of Proposition 40. Set \[\rho_{\rm sep}:= \sqrt{K}r_{\rm ref}.\] By Proposition 36 and Remark 34, every \(U\in\mathcal{C}_\rho\) is a \((r_{\rm ref},r_{\rm ref})\)-cluster with center \(\theta_U\). Here we use the scale assumptions above and take \(K_{\text{\tiny{\ref{prop:local-link-candidate-net}}}}\) large enough so that \[\sqrt{\frac{8L_\mathrm{p}\rho}{\ell_\mathrm{p}}}\le r_{\rm ref}, \qquad 3L_\mathrm{p}\rho\le r_{\rm ref}.\]

Let \(U_1,U_2\in\mathcal{C}_\rho\), and write \[d_{12}:={\rm d}({\theta_{U_1}},{\theta_{U_2}}).\] By Lemma 11 and \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,L_\mathrm{p}\rho,m_\rho)}\hypersetup{linkcolor=blue}\), \[\overline{N}\!\left(U_1,U_2\right) \ge \mathrm{p}(d_{12}+2r_{\rm ref})-8M_\mathrm{p}r_{\rm ref}-L_\mathrm{p}\rho\] and \[\overline{N}\!\left(U_1,U_2\right) \le \mathrm{p}((d_{12}-2r_{\rm ref})_+)+8M_\mathrm{p}r_{\rm ref}+L_\mathrm{p}\rho.\] Also, Proposition 36 gives \[\mathrm{p}(0)-3L_\mathrm{p}\rho \le \widehat{\mathrm{p}}(0) \le \mathrm{p}(0)+L_\mathrm{p}\rho.\] By choosing \(K_{\text{\tiny{\ref{prop:local-link-candidate-net}}}}\) large enough, the scale assumptions ensure \[L_\mathrm{p}\rho_{\rm sep} \ge 2L_\mathrm{p}r_{\rm ref}+8M_\mathrm{p}r_{\rm ref}+2L_\mathrm{p}\rho, \qquad \ell_\mathrm{p}\rho_{\rm sep} \ge 8M_\mathrm{p}r_{\rm ref}+5L_\mathrm{p}\rho, \qquad \rho_{\rm sep}\ge \max\{r_{\rm ref},\rho\}.\] If \[d_{12}\le \rho_{\rm sep},\] then \(d_{12}+2r_{\rm ref}\le 3\rho_{\rm sep}\le r_\mathrm{p}\), and the Lipschitz bound gives \[\overline{N}\!\left(U_1,U_2\right) \ge \mathrm{p}(0)-L_\mathrm{p}(d_{12}+2r_{\rm ref})-8M_\mathrm{p}r_{\rm ref}-L_\mathrm{p}\rho \ge \widehat{\mathrm{p}}(0)-2L_\mathrm{p}\rho_{\rm sep}.\] Thus \(U_1\stackrel{H_{\mathrm{loc}}}{\sim}U_2\).

If \[d_{12}> 2r_{\rm ref}+ \left(1+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep},\] then \(d_{12}-2r_{\rm ref}>(1+2L_\mathrm{p}/\ell_\mathrm{p})\rho_{\rm sep}\). Since \((1+2L_\mathrm{p}/\ell_\mathrm{p})\rho_{\rm sep}\le r_\mathrm{p}\), monotonicity and the lower Lipschitz bound give \[\begin{align} \overline{N}\!\left(U_1,U_2\right) &\le \mathrm{p}\!\left(\left(1+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep}\right) +8M_\mathrm{p}r_{\rm ref}+L_\mathrm{p}\rho\\ &\le \mathrm{p}(0)-\ell_\mathrm{p}\rho_{\rm sep}-2L_\mathrm{p}\rho_{\rm sep} +8M_\mathrm{p}r_{\rm ref}+L_\mathrm{p}\rho\\ &< \widehat{\mathrm{p}}(0)-2L_\mathrm{p}\rho_{\rm sep}. \end{align}\] Thus \(U_1\) and \(U_2\) are not adjacent. Therefore adjacency implies \[\begin{align} \label{eq:local-candidate-adjacent-representatives-close} d_{12}\le 2r_{\rm ref}+\left(1+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep}. \end{align}\tag{4}\] Apply the comparison graph selection lemma.  Proposition 36 supplies, for every \(x\in M\), a candidate \(U_x\in\mathcal{C}_\rho\) with \[{\rm d}({x},{\theta_{U_x}})\le\rho.\] Since \(r_{\rm ref}\le\rho_{\rm sep}\), 4 gives \[U_1\stackrel{H_{\mathrm{loc}}}{\sim}U_2 \quad\Longrightarrow\quad d_{12}\le \left(3+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep}.\] Lemma 8, applied with \[\delta=\rho_{\rm sep}, \qquad \rho_{\rm cov}=\rho, \qquad K_{\rm cmp}=3+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}},\] shows that the selected representatives are \(\rho_{\rm sep}\)-separated and form a net of radius at most \[\rho+\left(3+2\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep} \le 4\left(1+\frac{L_\mathrm{p}}{\ell_\mathrm{p}}\right)\rho_{\rm sep}.\] By increasing \(K_{\text{\tiny{\ref{prop:local-link-candidate-net}}}}\) if necessary, this is at most \(\tfrac13 Kr_{\rm ref}\). The separation also implies that the representatives \(\theta_U=X_{u_U}\), \(U\in\mathcal{N}_{\mathrm{loc}}\), are all distinct. Hence the map \(U\mapsto u_U\) injects \(\mathcal{N}_{\mathrm{loc}}\) into \(W\), and \(|\mathcal{N}_{\mathrm{loc}}|\le |W|\).

Separated-seed certificates.  Each selected \(U\) is a \((r_{\rm ref},r_{\rm ref})\)-cluster with center \(\theta_U\). Furthermore, \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le 3L_\mathrm{p}\rho\le r_{\rm ref}.\] By the choice of \(K_{\text{\tiny{\ref{prop:local-link-candidate-net}}}}\), and since \(K^3r_{\rm ref}\le\min\{1,r_\mathrm{p}\}\), the \(K\)-requirements in Proposition 31 hold. Proposition 31, in the Local Route case, applied with \(r=r_{\rm ref}\), the estimate for \(\widehat{\mathrm{p}}(0)\), and this value of \(K\), gives the required refinement-net certificate. ◻

7 Link-average threshold refinement↩︎

The refinement step consumes a link-average separated set and tests it against a fresh vertex block. Thresholding at the associated level \(t\) keeps the inner ball and rejects vertices outside the outer ball, up to the empirical link-average fluctuation. The main statement below is the form used later: it gives readable sufficient conditions for applying the refinement step simultaneously over a whole refinement net.

Proposition 41 (Simultaneous threshold refinement). Let \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] be a refinement net with parameters \[(\Theta,\quad R_{\rm net},\quad R_{\rm in},\quad R_{\rm out},\quad \gamma,\quad M_{\rm net}).\] Suppose \(|U|\ge m\) for every \(U\in\mathcal{N}\). Let \(V\subseteq\mathbf{V}\) be disjoint from every seed in \(\mathcal{N}\), and set \(n_\star:=|V|\ge2\). Assume \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V,R_{\rm in})}\hypersetup{linkcolor=blue}\). Let \(L_{\rm fail}>0\), and define \[\varphi_{\rm in}:=\phi(R_{\rm in}/3).\] Assume \(R_{\rm out}\le1\), \(\varphi_{\rm in}>0\), and \[\begin{align} \label{eq:sim-refinement-failure-budget} L_{\rm fail} &\le c_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} R_{\rm out}\varphi_{\rm in}n_\star, \\ \label{eq:sim-refinement-approx-concentration} \mathsf{s}m\min\{\gamma^2,1\} &\ge C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} \log\!\left(\frac{e}{R_{\rm out}\varphi_{\rm in}}\right), \end{align}\] {#eq: sublabel=eq:eq:sim-refinement-failure-budget,eq:eq:sim-refinement-approx-concentration} where the constants \[c_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}}, C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}}>0\] depend only on \(c_{\rm link},C_{\rm link}\). For each \(U\in\mathcal{N}\), define \[\widehat U:=\{v\in V:\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\ge \Theta\}.\] Then, conditionally on all latent positions of the seeds and of \(V\), with probability at least \(1-M_{\rm net}\exp(-L_{\rm fail})\), every \(\widehat U\), \(U\in\mathcal{N}\), satisfies \[|\widehat U| \ge \frac{\phi(R_{\rm in}/3)}{4}n_\star\] and is an \((R_{\rm out},R_{\rm out})\)-cluster with center \(\theta_U\). If, in addition, \[\begin{align} \label{eq:sim-refinement-exact-concentration} \mathsf{s}m\min\{\gamma^2,1\} &\ge C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} \bigl(L_{\rm fail}+\log n_\star\bigr), \end{align}\qquad{(7)}\] then, with the same probability bound, every \(U\in\mathcal{N}\) also satisfies \[V\cap B(\theta_U,R_{\rm in}) \subseteq \widehat U \subseteq V\cap B(\theta_U,R_{\rm out}).\]

For \(m\ge1\) and \(\gamma>0\), define \[q_{\rm ref}(m,\gamma) := C_{\rm link} \exp\!\left( -c_{\rm link} \mathsf{s}m\min\{\gamma^2/4,1\} \right),\] the fixed-vertex tail bound from Lemma 3 with deviation level \(\gamma/2\). The following lemma is the technical input for 41.

Lemma 12 (link-average threshold refinement). Let \(U,V\subseteq\mathbf{V}\) be disjoint, with \(m:=|U|\) and \(n_\star:=|V|\ge2\). Fix realizations \(X_U=x_U\) and \(X_V=x_V\). Suppose that \(U\) is \[(x_0,R_{\rm in},R_{\rm out},t,\gamma)\text{-link-average separated}\] and that \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V,R_{\rm in})}\hypersetup{linkcolor=blue}\) holds. Define \[\widehat U:=\{v\in V:\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\ge t\}.\] Let \(L_{\rm fail}>0\), and assume \[\begin{align} \label{eq:single-refinement-bad-budget} b_{\rm ref}:=2\bigl(q_{\rm ref}(m,\gamma)n_\star+L_{\rm fail}\bigr) &\le \frac{\phi(R_{\rm in}/3)}{4}n_\star. \end{align}\qquad{(8)}\] Then, conditionally on \(X_U=x_U\) and \(X_V=x_V\), with probability at least \(1-\exp(-L_{\rm fail})\), the following holds:

  • \[|\widehat U| \ge \frac{\phi(R_{\rm in}/3)}{4}n_\star.\]

  • \(\widehat U\) is an \[\left( R_{\rm out}, \frac{4b_{\rm ref}}{\phi(R_{\rm in}/3)n_\star} \right)\text{-cluster}\] with center \(x_0\).

  • If, in addition, \[\begin{align} \label{eq:single-refinement-exact-condition} n_\star q_{\rm ref}(m,\gamma)\le \exp(-L_{\rm fail}), \end{align}\qquad{(9)}\] then \[V\cap B(x_0,R_{\rm in}) \subseteq \widehat U \subseteq V\cap B(x_0,R_{\rm out}).\] In particular, \(\widehat U\) is an \((R_{\rm out},0)\)-cluster with center \(x_0\).

Proof. Define \[V_{\rm in}:=\{v\in V:X_v\in B(x_0,R_{\rm in})\}, \qquad V_{\rm out}:=\{v\in V:X_v\notin B(x_0,R_{\rm out})\},\] and \[\mathcal{I}_{\rm ref} := \left\{ v\in V: |\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}|>\gamma/2 \right\}.\] Set \(q:=q_{\rm ref}(m,\gamma)\) and \(b:=b_{\rm ref}\). Conditionally on \(X_U=x_U\) and \(X_V=x_V\), the indicators \(\mathbf{1}_{\{v\in\mathcal{I}_{\rm ref}\}}\), \(v\in V\), are independent, and Lemma 3 bounds each success probability by \(q\). Since \[\left(\sqrt{qn_\star}+\sqrt{L_{\rm fail}}\right)^2\le b,\] Remark 52 gives \[\mathbb{P}\left( |\mathcal{I}_{\rm ref}|>b \;\middle|\;X_U=x_U,\;X_V=x_V \right) \le \exp(-L_{\rm fail}).\] Work on the event \(|\mathcal{I}_{\rm ref}|\le b\).

If \(v\in V_{\rm in}\setminus\mathcal{I}_{\rm ref}\), then \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\ge t+\gamma\), and hence \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\ge t+\gamma-\gamma/2>t.\] Thus \(V_{\rm in}\setminus\mathcal{I}_{\rm ref}\subseteq\widehat U\). Using \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V,R_{\rm in})}\hypersetup{linkcolor=blue}\), \[|\widehat U\cap V_{\rm in}| \ge \frac{\phi(R_{\rm in}/3)}{2}n_\star-b.\] Similarly, if \(v\in V_{\rm out}\setminus\mathcal{I}_{\rm ref}\), then \(\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\le t-\gamma\), and \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}\le t-\gamma+\gamma/2<t,\] so \(v\notin\widehat U\). Therefore \[\widehat U\cap V_{\rm out}\subseteq\mathcal{I}_{\rm ref}, \qquad |\widehat U\cap V_{\rm out}|\le b.\] The bound ?? gives \[|\widehat U| \ge \frac{\phi(R_{\rm in}/3)}{4}n_\star,\] and then \[\frac{|\widehat U\cap V_{\rm out}|}{|\widehat U|} \le \frac{4b_{\rm ref}}{\phi(R_{\rm in}/3)n_\star}.\] This is exactly the asserted cluster bound.

For the exact-inclusion clause, ?? and the union bound give \[\mathbb{P}\left( \mathcal{I}_{\rm ref}\neq\varnothing \;\middle|\;X_U=x_U,\;X_V=x_V \right) \le \exp(-L_{\rm fail}).\] On \(\mathcal{I}_{\rm ref}=\varnothing\), the two pointwise inclusions become \[V_{\rm in}\subseteq\widehat U \qquad\text{and}\qquad \widehat U\cap V_{\rm out}=\varnothing,\] which is the desired exact inclusion. ◻

Proof of Proposition 41. Since \[\min\{\gamma^2/4,1\}\ge \frac{1}{4}\min\{\gamma^2,1\},\] assumption ?? implies, after increasing \(C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}}\), that \[q_{\rm ref}(m,\gamma) \le \frac{1}{16}R_{\rm out}\varphi_{\rm in}.\] Assumption ?? gives \[L_{\rm fail}\le \frac{1}{16}R_{\rm out}\varphi_{\rm in}n_\star\] after decreasing \(c_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}}\). Hence \[\begin{align} \label{eq:sim-refinement-bad-budget} b_{\rm ref}:= 2\bigl(q_{\rm ref}(m,\gamma)n_\star+L_{\rm fail}\bigr) &\le \frac{R_{\rm out}\varphi_{\rm in}}{4} n_\star \le \frac{\varphi_{\rm in}}{4} n_\star. \end{align}\tag{5}\] For each \(U\in\mathcal{N}\), apply Proposition 12 with \[x_0=\theta_U,\qquad t=\Theta.\] Since \(|U|\ge m\), monotonicity of \(q_{\rm ref}\) in the first argument gives \(q_{\rm ref}(|U|,\gamma)\le q_{\rm ref}(m,\gamma)\). Hence the corresponding single-seed value \[b_U:=2\bigl(q_{\rm ref}(|U|,\gamma)n_\star+L_{\rm fail}\bigr)\] satisfies \(b_U\le b_{\rm ref}\). The bound 5 implies the size conclusion and the cluster error bound \[\frac{4b_U}{\varphi_{\rm in}n_\star}\le R_{\rm out}\] for every seed. Union bounding over \(|\mathcal{N}|\le M_{\rm net}\) gives the claimed probability.

For the exact-inclusion conclusion, assumption ?? gives \[q_{\rm ref}(m,\gamma)\le n_\star^{-1}e^{-L_{\rm fail}}\] after increasing \(C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}}\) again. Thus, again using \(|U|\ge m\), \[n_\star q_{\rm ref}(m,\gamma)\le e^{-L_{\rm fail}},\] so the exact-inclusion clause of Proposition 12 applies to every seed. The same union bound over the refinement net proves the simultaneous exact-inclusion claim. ◻

8 Three-block extraction↩︎

The extraction statement has two routes. With a fuzzy window oracle, the first block uses the oracle-filtered candidate net at internal scale \(\rho_\circ\asymp r\). Without such an oracle, the first block uses the smaller internal-average scale \(\rho_\circ\asymp r^2\) and converts internal-average candidates into localized clusters. In both routes, \(\rho_\circ\) is chosen so that the unified candidate net statement produces a first-block refinement net at the theorem scale \(r\). After that, the two routes pass through the same two refinement blocks.

Theorem 42 (Three-block extraction). There exist constants \[C_{\rm ext},\Lambda_0>0\] depending only on \(L_\mathrm{p},\ell_\mathrm{p},M_\mathrm{p}\), such that the following holds. Let \(K_\star\) be a parameter satisfying \[K_\star \ge \max\{3,2K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\}\] and let \(\Lambda\ge \Lambda_0\). Assume the vertex set \(\mathbf{V}\) has three disjoint blocks \(V_1,V_2,V_3\) of equal size \(n\). Let \(r>0\). Assume one of the following two cases.

  • Oracle Route: Suppose an \((\alpha_{\mathrm{win}},\lambda_{\mathrm{win}})\)-fuzzy window oracle \(\mathcal{O}_{\mathrm{win}}\) is available on the first block \(V_1\), with \[0<\alpha_{\mathrm{win}}<\lambda_{\mathrm{win}}\le 1/8, \qquad \rho_\circ:=K_\star^{-1}r, \qquad \rho_\circ\le \frac{\alpha_{\mathrm{win}}}{2} r_\mathrm{p}.\]

  • Local Route: Suppose no fuzzy window oracle is used, and set \[\rho_\circ:=K_\star^{-2}r^2.\]

Assume \[0<r\le K_\star^{-3}\min\{1,r_\mu,r_\mathrm{p}\}, \label{eq:uniform-extraction-range-refactored}\qquad{(10)}\] and \[\mathsf{s}n\,\phi(\rho_\circ/3)\rho_\circ^2 \ge C_{\rm ext}\Lambda\log n. \label{eq:uniform-extraction-scale-condition-refactored}\qquad{(11)}\] Then there is a three-block procedure which, with probability \(1-o(1)\), outputs

  • for every \(v\in V_3\), a set \(v \in U_v\subseteq V_3\) such that \[|U_v|\ge \frac{1}{2}\phi(r)n \quad and\quad U_v\text{ is a }(K_\star^2r,0)\text{-cluster with center }X_v,\]

  • an estimator \(\widehat{\mathrm{p}}(0)\) satisfying \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le r\,;\]

Moreover, for a constant \(C>0\) depending only on the model parameters, the number of distinct sets in \(\{U_v\}_{v\in V_3}\) is at most \[\exp\left\{ C\, \log^2\!\left(\frac{3}{r\phi(\rho_\circ/3)}\right) \frac{1}{\mathsf{s}\rho_\circ^2} \right\},\] meaning that many \(U_v=U_{v'}\) may coincide. Finally, the running time is bounded by \[\exp\left\{ C\, \log^2\!\left(\frac{3}{r\phi(\rho_\circ/3)}\right) \frac{1}{\mathsf{s}\rho_\circ^2} \right\} n\Lambda\log n.\]

Remark 43 (Procedure behind Theorem 42). The statistical statement uses three ambient blocks of size \(n\), but the implementation may work with smaller internal subsets. In the proof we choose \(V_1'\subseteq V_1\) and \(V_2'\subseteq V_2\). The first-stage candidate search and candidate net construction are performed only inside \(V_1'\). The first refinement is performed into \(V_2'\). The final refinement and the final output use the full block \(V_3\).

The first block produces a small net of seed sets. In the oracle route this net is built from candidates certified by the oracle at internal scale \(\rho_\circ=K_\star^{-1}r\). In the local route the net is built from internal-average candidates at \(\rho_\circ\asymp r^2\), which are then viewed as localized clusters. The important point is that \(r\) is not the raw search scale in both routes: \(\rho_\circ\) is the internal candidate-search scale, while \(r\) is the first usable refinement scale. We choose \(\rho_\circ\) so that the unified candidate net statement can be applied with \(r_{\rm ref}=r\). The selected seeds are refined once into \(V_2'\), producing intermediate clusters, and then refined again into \(V_3\), producing exact clusters.

Working scales and constant choices↩︎

Set \[r_1:=r, \qquad r_2:=K_\star r, \qquad r_{\rm net}:=\tfrac13K_\star r, \qquad r_3:=\tfrac12K_\star r_2.\] Choose the auxiliary block sizes by \[\begin{align} \tag{6} n_1 &:= \min\left\{ N\in\mathbb{N}: \begin{array}{l} \lceil\log\log n\rceil\le N\le n,\\ \mathsf{s}N\phi(\rho_\circ/3)\rho_\circ^2 \ge C_{\rm ext}\Lambda\log N \end{array} \right\}, \\ \tag{7} n_2 &:= \min\left\{ N\in\mathbb{N}: \begin{array}{l} 2\le N\le n,\\ \mathsf{s}N\phi(r_1/3)r_1^2 \ge C_{\rm ext}\Lambda\log n \end{array} \right\}. \end{align}\] The sets in 6 and 7 are nonempty under the scale assumption ?? from 42: \(N=n\) is admissible in both definitions.

Lemma 13 (First-block construction of a refinement net). Under the assumptions of Theorem 42, there is a first-block procedure using a subset \(V_1'\subseteq V_1\) of size \(n_1\). Within the event \[\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_1',\rho_\circ)}\hypersetup{linkcolor=blue} \cap \hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(V_1',L_\mathrm{p}\rho_\circ,m_1)}\hypersetup{linkcolor=blue} withm_1:=\left\lfloor\frac{\phi(\rho_\circ/3)}{2}n_1\right\rfloor,\] which holds with probability at least \(1-n_1^{-c\Lambda}\), the procedure outputs an estimator \(\widehat{\mathrm{p}}(0)\), a threshold \(\Theta_2\), and a refinement net \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] such that \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le r,\] and \(\mathfrak N\) is a refinement net with parameters \[\begin{gather} \Theta=\Theta_2,\qquad R_{\rm net}=r_{\rm net},\qquad R_{\rm in}=r_1,\\ R_{\rm out}=r_2,\qquad \gamma=r_1,\qquad M_{\rm net}=n_1. \end{gather}\]

Proof. Let \(V_1'\subseteq V_1\) have size \(n_1\), where \(n_1\) is chosen in 6 .

Probability of the first-block event.  The Uniform lower-occupancy Lemma 2 gives \[\mathbb{P}\left(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_1',\rho_\circ)}\hypersetup{linkcolor=blue}\right) \ge 1-n_1^{- \tfrac{1}{16}\Lambda}.\] Also, we apply the Uniform concentration of pair averages Lemma 5 with Remark 23 with \[\lambda = \rho_\circ, \qquad m = m_1, \qquad n = n_1, \qquad \phi = \tfrac{\phi(\rho_\circ/3)}{2},\] together with the assumption that \(C_{\rm ext}\) is large enough. The defining inequality 6 gives the condition required by the lemma: \[\mathsf{s}\tfrac{n_1}{\Lambda \log n_1} \cdot \tfrac{\phi(\rho_\circ/3)}{2} \cdot \rho_\circ^2 \ge C_{\text{\tiny{\ref{lem:uniform-pair-average}}}}.\] Then, we have \[\mathbb{P}\left(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(V_1',L_\mathrm{p}\rho_\circ,m_1)}\hypersetup{linkcolor=blue}\, \vert\, X_{V_1'}\right) \ge 1 - \exp \left\{ - c_{\text{\tiny{\ref{lem:uniform-pair-average}}}} \phi(\rho_\circ/3) n_1 \log( e/ \phi(\rho_\circ/3) ) \right\} \ge 1 - n_1^{-c_{\text{\tiny{\ref{lem:uniform-pair-average}}}} \Lambda},\] where the last inequality holds from our assumption on \(n_1\), \(\mathsf{s}\le 1\), and \(\log(e/\phi(\rho_\circ/3)) \ge \log e =1\). Therefore, we conclude that for \(C_{\rm ext}\) and \(\Lambda\) greater than some constants depending on the model parameters, the event holds with \[1-n_1^{-c\Lambda} = 1-o(1).\]

Construction of the refinement net.  Within the event, Proposition 36 gives \[\widehat{\mathrm{p}}(0):=\max\{\overline{N}\!\left(U\right):U\subseteq V_1',\;|U|=m_1\}, \qquad |\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le 3L_{\mathrm{p}}\rho_\circ\le r,\] where the last inequality follows from the definition of \(\rho_\circ\), the range assumption, and the lower bound on \(K_\star\). The same proposition also gives the candidate family \[\mathcal{C}_{\rho_\circ}:= \{U\subseteq V_1':\;|U|=m_1,\;\overline{N}\!\left(U\right)\ge \widehat{\mathrm{p}}(0)-4L_{\mathrm{p}}\rho_\circ\}.\]

We now select a refinement net from \(\mathcal{C}_{\rho_\circ}\). Apply Proposition 38 in the appropriate route with \(K=K_\star\). The range assumption and the lower bound on \(K_\star\) imply that \[\rho_\circ\le \min\{r_\mu,r_\mathrm{p}/C_*\}, \qquad K_\star^3r \le \min\{1,r_\mathrm{p}\}.\] In the Oracle Route case, the theorem assumes \[\rho_\circ\le \frac{\alpha_{\mathrm{win}}}{2}r_\mathrm{p},\] and \[K_\star\rho_\circ\le r.\] In the Local Route case, \[K_\star\sqrt{\rho_\circ}\le r.\] Thus the hypotheses of Proposition 38 hold with \(r_{\rm ref}=r\). This gives a refinement net \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] with threshold \[\Theta_2= \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:link-average-separation}}}}r\] and parameters \[\begin{gather} \Theta= \widehat{\mathrm{p}}(0)-A_{\text{\tiny{\ref{prop:link-average-separation}}}}r,\qquad R_{\rm net}=r_{\rm net},\qquad R_{\rm in}=r_1,\\ R_{\rm out}=r_2,\qquad \gamma=r_1,\qquad M_{\rm net}=n_1. \end{gather}\] ◻

Lemma 14 (Two-step refinement from a refinement net). Let \[\mathfrak N=\{(U,\theta_U):U\in\mathcal{N}\}\] be the refinement net from Lemma 13, with threshold \(\Theta_2\), and let \(\widehat{\mathrm{p}}(0)\) be the estimator produced in the same first-block step. Recall that every first-block seed has size \[m_1=\left\lfloor\frac{\phi(\rho_\circ/3)}{2}n_1\right\rfloor.\] Let \(V_2'\subseteq V_2\) have size \(n_2\), where \(n_2\) is chosen in 7 . Then, with probability at least \(1-n_2^{-c\Lambda}-n^{-\Omega(\Lambda)}\), for every \(U\in\mathcal{N}\) the two refinement rounds produce sets \[U^{(2)}\subseteq V_2', \qquad U^{(3)}\subseteq V_3,\] such that \(U^{(2)}\) is an \((r_2,r_2)\)-cluster with center \(\theta_U\), and \[V_3\cap B(\theta_U,r_2) \subseteq U^{(3)} \subseteq V_3\cap B(\theta_U,r_3).\]

Proof. For the first refinement, 7 gives \[\frac{\Lambda\log n}{n_2} \le \frac{\mathsf{s}}{C_{\rm ext}}\phi(r_1/3)r_1^2.\] Since \(\mathsf{s}\le1\), \(r_1\le1\), and \(r_1\le r_2\), increasing \(C_{\rm ext}\) gives \[\begin{align} \label{eq:first-refinement-failure-budget-check} \Lambda\log n &\le c_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} r_2\phi(r_1/3)n_2. \end{align}\tag{8}\] Also, 6 , the identity \(m_1=\lfloor \phi(\rho_\circ/3)n_1/2\rfloor\), and the relation \(\rho_\circ\le r_1\) imply, after increasing \(C_{\rm ext}\) again, that \[\begin{align} \label{eq:first-refinement-approx-concentration-check} \mathsf{s}m_1\min\{r_1^2,1\} &\ge C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} \log\!\left(\frac{e}{r_2\phi(r_1/3)}\right). \end{align}\tag{9}\] The range assumption gives \(r_2\le1\). Thus 8 and 9 verify the conditions ?? and ?? from [prop:simultaneous-refinement-net], respectively, for \[R_{\rm in}=r_1,\qquad R_{\rm out}=r_2,\qquad \gamma=r_1, \qquad n_\star=n_2.\] On \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_2',r_1)}\hypersetup{linkcolor=blue}\), Proposition [prop:simultaneous-refinement-net], applied to \(\mathfrak N\) with \(L_{\rm fail}=\Lambda\log n\), gives for every \(U\in\mathcal{N}\) a set \(U^{(2)}\subseteq V_2'\) with \[|U^{(2)}|\ge \frac{\phi(r_1/3)}{4} n_2\] which is an \((r_2,r_2)\)-cluster with center \(\theta_U\). The event \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_2',r_1)}\hypersetup{linkcolor=blue}\) holds with probability \(1-n_2^{-c\Lambda}\), and the simultaneous refinement failure is at most \(n_1\exp(-\Lambda\log n)\), which is harmless once \(\Lambda\ge\Lambda_0\).

For the second refinement, set \[m_2:=\frac{\phi(r_1/3)}{4} n_2.\] Each \(U^{(2)}\) has size at least \(m_2\), and is an \((r_2,r_2)\)-cluster with center \(\theta_U\). Since \(K_\star/2\ge K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\), the parameter \(K_\star/2\) is admissible in Proposition 31. Proposition 31, in the Local Route case, applied with \(r=r_2\) and \(K=K_\star/2\), gives the common threshold \[\Theta_3:= \widehat{\mathrm{p}}(0)- A_{\text{\tiny{\ref{prop:link-average-separation}}}}r_2\] such that \[\mathfrak N^{(2)} := \{(U^{(2)},\theta_U):U\in\mathcal{N}\}\] is a refinement net with parameters \[\begin{gather} \Theta=\Theta_3,\qquad R_{\rm net}=r_{\rm net},\qquad R_{\rm in}=r_2,\\ R_{\rm out}=r_3,\qquad \gamma=r_2,\qquad M_{\rm net}=n_1. \end{gather}\] The range assumption ?? from 42 ensures the local-window condition \[\tfrac12K_\star r_2=r_3 \le \frac{1}{2}\min\{1,r_\mathrm{p}\},\] which in particular gives \(r_3\le1\), and the condition \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le 3L_\mathrm{p}\rho_\circ\le r_2\] needed to apply that proposition.

7 and the definition of \(m_2\) imply \[\mathsf{s}m_2\min\{r_2^2,1\} \ge C_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} \bigl(\Lambda\log n+\log n\bigr),\] after increasing \(C_{\rm ext}\) and \(\Lambda_0\), since \(r_2=K_\star r_1\) and \(\Lambda\ge\Lambda_0\). This verifies the exact-concentration condition ?? from [prop:simultaneous-refinement-net]. It also verifies the approximate-concentration condition ?? from [prop:simultaneous-refinement-net]: indeed, 7 , \(n_2\le n\), and \(\phi(r_1/3)\le\phi(r_2/3)\) imply, after adjusting constants, that \[r_3\phi(r_2/3)\ge n^{-1} \quad \Rightarrow \quad \log\!\left(\frac{e}{r_3\phi(r_2/3)}\right) \le 2\log n.\] Finally, because \(n_2\le n\), 7 gives \[\frac{\Lambda\log n}{n} \le \frac{\Lambda\log n}{n_2} \le \frac{\mathsf{s}}{C_{\rm ext}}\phi(r_1/3)r_1^2.\] Using \(\phi(r_1/3)\le\phi(r_2/3)\) and \(r_1^2\le r_3\), and increasing \(C_{\rm ext}\) again, we get \[\Lambda\log n \le c_{\text{\tiny{\ref{prop:readable-simultaneous-refinement}}}} r_3\phi(r_2/3)n.\] This verifies the failure-budget condition ?? from [prop:simultaneous-refinement-net]. Hence all three conditions ?? , ?? , and ?? from [prop:simultaneous-refinement-net] hold for \[R_{\rm in}=r_2,\qquad R_{\rm out}=r_3,\qquad \gamma=r_2,\qquad n_\star=n.\] Therefore the exact-inclusion clause of Proposition [prop:simultaneous-refinement-net], applied to \(\mathfrak N^{(2)}\) on \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_3,r_2)}\hypersetup{linkcolor=blue}\) with \(L_{\rm fail}=\Lambda\log n\), gives \[V_3\cap B(\theta_U,r_2) \subseteq U^{(3)} \subseteq V_3\cap B(\theta_U,r_3).\] The occupancy event \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_3,r_2)}\hypersetup{linkcolor=blue}\) holds with probability \(1-n^{-c\Lambda}\), and the remaining failures are absorbed by the same union bound over the seed family. ◻

Proof of Theorem 42. Choose \(r_1,r_2,r_{\rm net},r_3\) as above and \(n_1,n_2\) by 67 . Apply Lemma 13 on \(V_1'\) to obtain \(\widehat{\mathrm{p}}(0)\), the threshold \(\Theta_2\), and a refinement net \(\mathfrak N\). Then apply Lemma 14 to refine every selected seed first into \(V_2'\) and then into \(V_3\).

For each \(v\in V_3\), choose \(U\in\mathcal{N}\) with \[{\rm d}({X_v},{\theta_U})\le r_{\rm net}<r_2,\] and set \(U_v:=U^{(3)}\). The exact inclusion from Lemma 14 gives \(v\in U_v\), while \[U_v\subseteq B(\theta_U,r_3) \subseteq B(X_v,r_{\rm net}+r_3) \subseteq B(X_v,K_\star^2r).\] On \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_3,r_2)}\hypersetup{linkcolor=blue}\), \[|U_v| \ge |V_3\cap B(\theta_U,r_2)| \ge \frac{\phi(r_2/3)}{2}n \ge \frac{\phi(r)}{2}n.\] The estimate for \(\mathrm{p}(0)\) was produced in the first-block step. The number of distinct output sets is bounded by \(|\mathcal{N}|\le n_1\le\binom{n_1}{m_1}\), and the entropy bound in the running-time remark gives the stated estimate. The probability bound follows from the union bound over the first-block event, the two occupancy events, and the two refinement rounds. ◻

Remark 44 (Running time). The constant \(C\) below depends only on the model parameters and may increase from line to line. The first-stage exhaustive search enumerates \[\binom{n_1}{m_1} \le \exp\{n_1H(\phi(\rho_\circ/3)/2)\} \le \exp\left\{ C\, \frac{\log^2\!\left(\frac{3}{r\phi(\rho_\circ/3)}\right)}{\mathsf{s}\rho_\circ^2} \right\}.\] The refinement part uses \[n_2\asymp \frac{\Lambda\log n}{\mathsf{s}\,\phi(r/3)r^2},\] so its cost is absorbed into \[\exp\left\{ C\, \frac{\log^2\!\left(\frac{3}{r\phi(\rho_\circ/3)}\right)}{\mathsf{s}\rho_\circ^2} \right\} n\Lambda\log n.\]

9 Two-round extraction via a coarse fuzzy window oracle↩︎

To prove the main theorem, we run Theorem 42 twice. The first run is a coarse local route extraction at scale \(\sqrt r\). Its only purpose is to build a fuzzy window oracle on a fresh block. The second run is the final oracle route extraction at scale \(r\). The next lemma packages the only new ingredient: coarse exact clusters and a coarse estimate of \(\mathrm{p}(0)\) generate the oracle needed by the second run.

Lemma 15 (Coarse three-block output generates a fuzzy window oracle). Fix constants \(C_0,c_0>0\). There exist constants \(c_{\mathrm{ora}},C_{\mathrm{ora}},\Lambda_{\mathrm{ora}}>0\), depending only on \(C_0,c_0,L_\mathrm{p},\ell_\mathrm{p},M_\mathrm{p},{\rm K}_{\mathrm{sg}}\), such that the following holds whenever \(\Lambda\ge\Lambda_{\mathrm{ora}}\).

Let \(V_0,W\subseteq\mathbf{V}\) be disjoint vertex blocks of size at most \(n\). Suppose that we are given an estimator \(\widehat{\mathrm{p}}(0)\) and, for every \(z\in V_0\), a set \(A_z\subseteq V_0\) such that \[z\in A_z, \qquad A_z\text{ is a }(C_0r,0)\text{-cluster with center }X_z, \quad and \quad |A_z|\ge \frac{1}{2}\phi(c_0r)n.\] Assume also that the centers \(\{X_z:z\in V_0\}\) form a \(C_0r\)-net of \(M\): \[\forall x\in M,\quad \exists z\in V_0 \quad\text{such that}\quad {\rm d}({x},{X_z})\le C_0r,\] and that \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le C_0r.\]

Assume \[r\le c_{\mathrm{ora}}r_\mathrm{p} \qquad\text{and}\qquad \mathsf{s}n\phi(c_0r)r_\mathrm{p}^2\ge C_{\mathrm{ora}}\Lambda\log n.\] For each \(z\in V_0\), define \[W_z := \left\{ v\in W: \hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{A_z}(v)}\hypersetup{linkcolor=blue} \ge \widehat{\mathrm{p}}(0)-\frac{\ell_\mathrm{p}}{32}r_\mathrm{p} \right\}.\] Define an oracle on \(W\) by \[\mathcal{O}_{\mathrm{win}}(v,w)=1\] if there exists \(z\in V_0\) such that \(v,w\in W_z\), and otherwise set \[\mathcal{O}_{\mathrm{win}}(v,w)=0.\] Then, conditionally on the latent points of \(V_0\cup W\) and on the given sets \(\{A_z:z\in V_0\}\), with probability at least \(1-n^{-\Omega(\Lambda)}\), \(\mathcal{O}_{\mathrm{win}}\) is an \((\alpha_{\mathrm{win}},\lambda_{\mathrm{win}})\)-fuzzy window oracle on \(W\), with \[\alpha_{\mathrm{win}}:=\frac{\ell_\mathrm{p}}{128L_\mathrm{p}}, \qquad \lambda_{\mathrm{win}}:=\frac{1}{8}.\]

Proof. See Appendix 11. ◻

Theorem 45 (Two-round extraction via a coarse fuzzy window oracle). There exist constants \[c_{\text{\tiny{\ref{thm:two-round-extraction}}}}, \qquad C_{\text{\tiny{\ref{thm:two-round-extraction}}}}, \qquad C'_{\text{\tiny{\ref{thm:two-round-extraction}}}}, \qquad \Lambda_{\text{\tiny{\ref{thm:two-round-extraction}}}}>0,\] depending only on the model parameters, such that the following holds whenever \(\Lambda\ge\Lambda_{\text{\tiny{\ref{thm:two-round-extraction}}}}\). Split the vertex set into six disjoint blocks of equal size, \[\mathbf{V} = V_1\sqcup V_2\sqcup V_3 \sqcup V_4\sqcup V_5\sqcup V_6, \qquad |V_i|=n.\] Let \(r>0\) be the target scale. Assume \[0<\sqrt r \le c_{\text{\tiny{\ref{thm:two-round-extraction}}}} \min\{1,r_\mu,r_\mathrm{p}\}, \label{eq:two-round-range}\qquad{(12)}\] and \[\mathsf{s}n\, \phi\!\left(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r\right) r^2 \ge C_{\text{\tiny{\ref{thm:two-round-extraction}}}}\Lambda\log n. \label{eq:two-round-scale}\qquad{(13)}\] Then there is a six-block procedure which, with probability \(1-o(1)\), runs in time \[\exp\left( C_{\text{\tiny{\ref{thm:two-round-extraction}}}} \log^2( \tfrac{3}{r\phi(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r)}) \tfrac{1}{\mathsf{s}r^2} \right) n \Lambda\log n ,\] and outputs an estimator \(\widehat{\mathrm{p}}(0)\) and, for every \(v\in V_6\), a set \(U_v\subseteq V_6\) such that \[v\in U_v, \qquad U_v \text{ is a } \left(C'_{\text{\tiny{\ref{thm:two-round-extraction}}}}r,0\right) \text{-cluster with center }X_v,\] and \[|U_v| \ge \frac{1}{2} \phi(r)n.\] Further, the number of sets \(\{U_v:v\in V_6\}\) without counting multiplicity is at most \[\exp\left( C_{\text{\tiny{\ref{thm:two-round-extraction}}}} \log^2( \tfrac{3}{r\phi(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r)}) \tfrac{1}{\mathsf{s}r^2} \right).\] Moreover, \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)| \le r.\]

Proof. Set \[r_0:=\sqrt r.\] Fix once and for all an admissible value \[K_\star\ge\max\{3,2K_{\text{\tiny{\ref{prop:candidate-refinement-net}}}}\}\] depending only on the model parameters, and use it for both applications of Theorem 42. All constants below are allowed to depend on this fixed \(K_\star\), hence only on the model parameters.

Round 1: coarse local-route extraction.  Apply Theorem 42 in the local route to \(V_1,V_2,V_3\) at target scale \(r_0\). The internal-average scale in that run is \[\rho_\circ = K_\star^{-2}r_0^2 = K_\star^{-2}r.\] After decreasing \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\) and increasing \(C_{\text{\tiny{\ref{thm:two-round-extraction}}}}\), the assumptions ?? and ?? imply the range and scale conditions of Theorem 42 for this local run. Thus, with probability \(1-o(1)\), it outputs \(\widehat{\mathrm{p}}_{\rm coarse}(0)\) and sets \(A_z\subseteq V_3\) such that, for every \(z\in V_3\), \[z\in A_z, \qquad A_z\text{ is a }(K_\star^2r_0,0)\text{-cluster with center }X_z, \qquad |A_z|\ge \frac{1}{2}\phi(r_0)n, \qquad |\widehat{\mathrm{p}}_{\rm coarse}(0)-\mathrm{p}(0)| \le r_0.\]

Construct the fuzzy window oracle.  Set \(C_0:=K_\star^2\) and choose any fixed \(c_0\le1\). The event \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(V_3,C_0r_0)}\hypersetup{linkcolor=blue}\) also holds with probability \(1-o(1)\), by Lemma 2. Indeed, after decreasing \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\), the radius \(C_0r_0\) is in range and \(\phi(C_0r_0)\ge \phi(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r)\); then ?? , together with \(\mathsf{s}r^2\le1\), gives the needed occupancy lower bound. Hence the centers \(\{X_z:z\in V_3\}\) form a \(C_0r_0\)-net of \(M\).

The scale requirements in Lemma 15 are checked in the same way. The bound \(r_0\le c_{\mathrm{ora}}r_\mathrm{p}\) follows from ?? . For the concentration requirement, decrease \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\) so that \(\phi(c_0r_0)\ge \phi(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r)\). Since ?? gives \(r_\mathrm{p}^2\gtrsim r_0^2=r\ge r^2\), ?? implies \[\mathsf{s}n\phi(c_0r_0)r_\mathrm{p}^2\ge C_{\mathrm{ora}}\Lambda\log n\] after increasing \(C_{\text{\tiny{\ref{thm:two-round-extraction}}}}\). Applying the lemma with \(V_0=V_3\), \(W=V_4\), and coarse scale \(r_0\), we obtain a fuzzy window oracle \(\mathcal{O}_{\mathrm{win}}\) on \(V_4\), with fixed margins \[\alpha_{\mathrm{win}}:=\frac{\ell_\mathrm{p}}{128L_\mathrm{p}}, \qquad \lambda_{\mathrm{win}}:=\frac{1}{8}.\]

Round 2: fine oracle extraction.  Apply Theorem 42 in the oracle route to \(V_4,V_5,V_6\) at target scale \(r\), using this oracle on \(V_4\). The internal-average scale is \[\rho_\circ=K_\star^{-1}r.\] The range condition of Theorem 42 follows from ?? , since \(r\le r_0\). Its scale condition follows from ?? after decreasing \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\), because then \(\phi(\rho_\circ/3)\ge \phi(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}r)\) and \(\rho_\circ^2=K_\star^{-2}r^2\). Therefore, with probability \(1-o(1)\), the fine run outputs an estimator \(\widehat{\mathrm{p}}(0)\) and, for every \(v\in V_6\), a set \(U_v\subseteq V_6\) satisfying \[v\in U_v, \qquad U_v \text{ is a }(K_\star^2 r,0)\text{-cluster with center }X_v,\] \[|U_v|\ge \frac{1}{2}\phi(r)n, \qquad |\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le r.\] Taking \(C'_{\text{\tiny{\ref{thm:two-round-extraction}}}}\ge K_\star^2\) gives the stated cluster-radius form. The running time and the number of distinct sets are the sum of the two three-block costs and the oracle-construction cost, computed over the distinct coarse clusters. Since the coarse run has internal scale comparable to \(r\), and the fine run has internal scale comparable to \(r\), while \(r\le r_0\le1\), these costs are absorbed into the complexity stated in the theorem.

Probability and independence.  The first coarse extraction, the occupancy event on \(V_3\), the oracle construction using edges from \(V_3\) to \(V_4\), and the final oracle extraction each fail with probability \(o(1)\). The edge sets used in these steps are disjoint: the oracle uses \(V_3\)-to-\(V_4\) edges, while the fine oracle extraction uses internal \(V_4\) edges, then \(V_4\)-to-\(V_5\) edges, and finally \(V_5\)-to-\(V_6\) edges. Thus the conditional concentration arguments from the three-block theorem apply without interference. A union bound gives total success probability \(1-o(1)\). ◻

Remark 46. The first round is used only to construct the fuzzy window oracle on \(V_4\). The final clusters are produced entirely by the second, oracle-assisted run on \(V_4,V_5,V_6\). The choice \(r_0=\sqrt r\) is what makes the local-route coarse condition at scale \(r_0\) match the oracle-route condition at scale \(r\).

A repacking of the two-round procedure gives simultaneous cluster extraction for every vertex in the graph, as follows.

Theorem 47. There exist constants \[c_{\text{\tiny{\ref{theor:main}}}}, \qquad C_{\text{\tiny{\ref{theor:main}}}}, \qquad C'_{\text{\tiny{\ref{theor:main}}}}, \qquad \Lambda_{\text{\tiny{\ref{theor:main}}}}>0,\] depending only on the model parameters, such that the following holds whenever \(\Lambda\ge\Lambda_{\text{\tiny{\ref{theor:main}}}}\). Let \(\mathbf{V}\) be a vertex set of size \(n\), and let \(r>0\). Assume \[0<\sqrt r \le c_{\text{\tiny{\ref{theor:main}}}} \min\{1,r_\mu,r_\mathrm{p}\},\] and \[\mathsf{s}n\,\phi(r)r^2 \ge C_{\text{\tiny{\ref{theor:main}}}}\Lambda\log n.\] Then there is a procedure which, with probability \(1-o(1)\), runs in time \[\exp\left( C_{\text{\tiny{\ref{theor:main}}}} \log^2( \tfrac{3}{r\phi(r)}) \tfrac{1}{\mathsf{s}r^2} \right)n \Lambda\log n ,\] outputs an estimator \(\widehat{\mathrm{p}}(0)\) and, for every \(v\in\mathbf{V}\), a set \(U_v\subseteq\mathbf{V}\) such that \[v\in U_v, \qquad U_v \text{ is a } \left(C'_{\text{\tiny{\ref{theor:main}}}}r,0\right) \text{-cluster with center }X_v,\] and \[|U_v| \ge \frac{1}{13}\phi(r)n.\] Further, the number of sets \(U_v\) without counting multiplicity is at most \[\exp\left( C_{\text{\tiny{\ref{theor:main}}}} \log^2( \tfrac{3}{r\phi(r)}) \tfrac{1}{\mathsf{s}r^2} \right)\,.\] Moreover, \[|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)| \le C'_{\text{\tiny{\ref{theor:main}}}}r.\]

Proof. We may assume \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\le1\), and set \[\bar r:=r/c_{\text{\tiny{\ref{thm:two-round-extraction}}}}.\] Split \(\mathbf{V}\) into six blocks of sizes differing by at most one. Apply Theorem 45 six times, cyclically permuting the blocks so that each block is used once as the output block \(V_6\), and use target scale \(\bar r\) in each run. The assumptions of Theorem 45 follow from the assumptions of this theorem after decreasing \(c_{\text{\tiny{\ref{theor:main}}}}\) and increasing \(C_{\text{\tiny{\ref{theor:main}}}}\): indeed \[c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\bar r=r,\] so the scale condition there is exactly the main scale condition, up to the constant change.

The six success events have total failure probability \(o(1)\). The runtime of each call is bounded by the displayed main-theorem runtime because \(c_{\text{\tiny{\ref{thm:two-round-extraction}}}}\bar r=r\), \(\bar r\ge r\), and the factor \(\bar r^{-2}\) differs from \(r^{-2}\) only by a model-dependent constant. The same comparison applies to the number of distinct output sets.

For a vertex in an output block of size at least \(n/6-1\), Theorem 45 gives a cluster of size at least \[\frac{1}{2}\phi(\bar r)(n/6-1) \ge \frac{1}{13}\phi(r)n\] for all large \(n\), after increasing constants if necessary, since \(\bar r\ge r\) and \(\phi\) is nondecreasing. The cluster radius and estimator error are \(O(\bar r)=O(r)\), so they are absorbed into \(C'_{\text{\tiny{\ref{theor:main}}}}r\). Use the estimator from any one of the six runs. ◻

10 Lower bound↩︎

Proof of Lemma 1. Set \(\alpha=\phi(r_0)\), and consider the three-point metric space \[M=\{a,b,c\},\qquad {\rm d}(x,y)=r_0\quad (x\neq y),\] with \[\mu(a)=\mu(b)=\alpha,\qquad \mu(c)=1-2\alpha.\] For \(0<t\le r_0\), each ball \(B(x,t)\) is a singleton, hence has mass at least \(\alpha=\phi(r_0)\ge \phi(t)\). Thus this space is lower-\(\phi\)-regular up to scale \(r_0\).

We use a simple Bernoulli observation model on this space. Fix a sufficiently small universal constant \(\gamma>0\), keep the sparsity parameter \(\mathsf{s}\), and define a non-increasing link by \[\mathrm{p}(t)=\frac{1}{2}+\frac{\gamma r_0}{2}-\gamma\min\{t,r_0\}.\] Since \(r_0\le1\) and \(\gamma\) is small, \(\mathrm{p}\) takes values in \((0,1)\). It is locally bi-Lipschitz on \([0,r_0]\), and the Bernoulli graph model is obtained from Definition 8 by taking \[\mathbf{F}(t,u)={\boldsymbol{1}}\{u\le \mathrm{p}(t)\}.\] Write \[p_0=\mathsf{s}\mathrm{p}(0),\qquad p_1=\mathsf{s}\mathrm{p}(r_0),\] and \[\mathcal{I} = \max\{\operatorname{kl}(p_0,p_1),\operatorname{kl}(p_1,p_0)\},\] where \(\operatorname{kl}\) is the Bernoulli relative entropy. Since \(\mathrm{p}(0),\mathrm{p}(r_0)\in[1/4,3/4]\) and \(|\mathrm{p}(0)-\mathrm{p}(r_0)|=\gamma r_0\), the elementary Taylor bound for Bernoulli relative entropy gives \[\mathcal{I}\le C\mathsf{s} r_0^2\] for a universal constant \(C\). Let \[I=\{i:X_i\in\{a,b\}\}.\] Then \(|I|\sim{\rm Binomial}(n,2\alpha)\). Put \[m=\lceil 3n\alpha\rceil,\qquad \mathcal{E}=\{2\le |I|\le m\}.\] There is a universal \(C_0\) such that, whenever \(n\alpha\ge C_0\), \[\mathbb{P}(\mathcal{E})\ge \frac{3}{4}.\] This is the standard binomial lower-tail and upper-tail estimate, since \(\mathbb{E}|I|=2n\alpha\).

We now work on \(\mathcal{E}\). Choose \(i_\star\) uniformly from \(I\), and let \(X'\) be obtained from \(X\) by changing only \(X_{i_\star}\), swapping \(a\) and \(b\). For fixed \(X=x\), write \(x'\) for the swapped configuration, and let \(P_x,P_{x'}\) be the corresponding conditional laws of the observed graph.

Only edges from \(i_\star\) to \(I\setminus\{i_\star\}\) can change. There are at most \(m\) such edges, and each affected coordinate changes between Bernoulli\((p_0)\) and Bernoulli\((p_1)\), in one direction or the other. Therefore the product relative entropy is at most \(m\mathcal{I}\), and Pinsker’s inequality gives \[{\rm TV}(P_x,P_{x'}) \le \sqrt{\frac{m\mathcal{I}}{2}}.\]

The two latent distance matrices are separated in sup norm by \(r_0\). Indeed, because \(|I|\ge2\), there is \(j\in I\setminus\{i_\star\}\), and the swap changes the distance between \(i_\star\) and \(j\) from \(0\) to \(r_0\), or from \(r_0\) to \(0\). Hence no estimator output can be within \(r_0/2\) of both distance matrices.

Let \(A_x\) be the event that an estimator is within error \(<r_0/2\) of the distance matrix generated by \(x\), and define \(A_{x'}\) similarly. Then \(A_x\cap A_{x'}=\varnothing\), so \[P_x(A_x)+P_{x'}(A_{x'}) \le 1+{\rm TV}(P_x,P_{x'}) \le 1+\sqrt{\frac{m\mathcal{I}}{2}}.\] The random swap preserves the conditional law of the latent sample on \(\mathcal{E}\), because the two small atoms have the same mass and the swap kernel is reversible. Averaging the last display over \(X\) and \(i_\star\) therefore shows that the conditional success probability of any estimator is at most \[\frac{1}{2}\left(1+\sqrt{\frac{m\mathcal{I}}{2}}\right).\] Removing the conditioning gives \[\mathbb{P}\!\left( \max_{u,v}|\widehat d(u,v)-{\rm d}({X_u},{X_v})|\ge r_0/2 \right) \ge \frac{\mathbb{P}(\mathcal{E})}{2} \left(1-\sqrt{\frac{m\mathcal{I}}{2}}\right)_+ .\]

After increasing \(C_0\) if necessary, \(m\le4n\alpha\). Thus, if \[\mathsf{s}n\alpha r_0^2\le c_0\] for a sufficiently small universal \(c_0\), then \(m\mathcal{I}\) is smaller than a universal constant. The last display is therefore bounded below by a universal constant \(c_1>0\). This proves the probability lower bound; the expectation lower bound follows by multiplying this event by \(r_0/2\). ◻

11 Proof of the coarse fuzzy-window oracle lemma↩︎

Proof of Lemma 15. Set \[m_0:=\frac{1}{2}\phi(c_0r)n, \qquad \varepsilon_{\rm link}:= \sqrt{\frac{\Lambda\log n}{\mathsf{s}m_0}}.\] The assumption \[\mathsf{s}n\phi(c_0r)r_\mathrm{p}^2\ge C_{\mathrm{ora}}\Lambda\log n\] implies, after increasing \(C_{\mathrm{ora}}\), that \[\varepsilon_{\rm link}\le c_{\mathrm{ora}}r_\mathrm{p}.\] By Lemma 3, a union bound over \(|V_0||W|\le n^2\) gives, with conditional probability \(1-n^{-\Omega(\Lambda)}\), \[\left|\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{A_z}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{A_z}(v)}\hypersetup{linkcolor=blue}\right| \le \varepsilon_{\rm link} \qquad (z\in V_0,\;v\in W).\] We work on this event.

We first record the deterministic inclusions \[\left\{ v\in W: {\rm d}({X_v},{X_z})\le \frac{\ell_\mathrm{p}}{64L_\mathrm{p}}r_\mathrm{p} \right\} \subseteq W_z \subseteq \left\{ v\in W: {\rm d}({X_v},{X_z})<\frac{1}{16}r_\mathrm{p} \right\} \qquad (z\in V_0).\] Fix \(z\in V_0\). Since \(A_z\) is a \((C_0r,0)\)-cluster with center \(X_z\), every \(u\in A_z\) satisfies \({\rm d}({X_u},{X_z})<C_0r\). If \[{\rm d}({X_v},{X_z})\le \frac{\ell_\mathrm{p}}{64L_\mathrm{p}}r_\mathrm{p},\] then, after taking \(c_{\mathrm{ora}}\) small enough, all distances \({\rm d}({X_u},{X_v})\) lie in the local bi-Lipschitz window and \[\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{A_z}(v)}\hypersetup{linkcolor=blue} \ge \mathrm{p}(0)-L_\mathrm{p}C_0r-\frac{\ell_\mathrm{p}}{64}r_\mathrm{p}.\] Using \(|\widehat{\mathrm{p}}(0)-\mathrm{p}(0)|\le C_0r\) and the concentration event, \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{A_z}(v)}\hypersetup{linkcolor=blue} \ge \widehat{\mathrm{p}}(0) -(L_\mathrm{p}C_0+C_0)r-\varepsilon_{\rm link} -\frac{\ell_\mathrm{p}}{64}r_\mathrm{p}.\] The choices of \(c_{\mathrm{ora}}\) and \(C_{\mathrm{ora}}\) ensure that the last three error terms are at most \(\ell_\mathrm{p}r_\mathrm{p}/32\), so \(v\in W_z\).

Conversely, if \[{\rm d}({X_v},{X_z})\ge \frac{1}{16}r_\mathrm{p},\] then, again taking \(c_{\mathrm{ora}}\) small enough, \[0< \frac{1}{16}r_\mathrm{p}-C_0r \le r_\mathrm{p}.\] For every \(u\in A_z\), monotonicity and the lower local bi-Lipschitz bound give \[\mathrm{p}({\rm d}({X_u},{X_v})) \le \mathrm{p}(0) - \ell_\mathrm{p}\left(\frac{1}{16}r_\mathrm{p}-C_0r\right).\] Averaging and using the estimator and concentration errors, \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{A_z}(v)}\hypersetup{linkcolor=blue} \le \widehat{\mathrm{p}}(0) -\frac{\ell_\mathrm{p}}{16}r_\mathrm{p} +(\ell_\mathrm{p}C_0+C_0)r+\varepsilon_{\rm link}.\] With \(c_{\mathrm{ora}}\) small and \(C_{\mathrm{ora}}\) large, the final two error terms are at most \(\ell_\mathrm{p}r_\mathrm{p}/64\), and hence \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{A_z}(v)}\hypersetup{linkcolor=blue} < \widehat{\mathrm{p}}(0)-\frac{\ell_\mathrm{p}}{32}r_\mathrm{p}.\] Thus \(v\notin W_z\), proving the displayed inclusions.

We now verify the fuzzy-oracle property. If \(\mathcal{O}_{\mathrm{win}}(v,w)=1\), then there is \(z\in V_0\) with \(v,w\in W_z\). The outer inclusion gives \[{\rm d}({X_v},{X_z})<\frac{1}{16}r_\mathrm{p}, \qquad {\rm d}({X_w},{X_z})<\frac{1}{16}r_\mathrm{p},\] so \({\rm d}({X_v},{X_w})<r_\mathrm{p}/8=\lambda_{\mathrm{win}}r_\mathrm{p}\). Therefore \[{\rm d}({X_v},{X_w})>\lambda_{\mathrm{win}}r_\mathrm{p} \quad\Longrightarrow\quad \mathcal{O}_{\mathrm{win}}(v,w)=0.\]

Conversely, suppose \[{\rm d}({X_v},{X_w})\le \alpha_{\mathrm{win}}r_\mathrm{p} = \frac{\ell_\mathrm{p}}{128L_\mathrm{p}}r_\mathrm{p}.\] Choose \(z\in V_0\) with \({\rm d}({X_v},{X_z})\le C_0r\), using the \(C_0r\)-net assumption. For \(c_{\mathrm{ora}}\) small enough, \[C_0r\le \frac{\ell_\mathrm{p}}{128L_\mathrm{p}}r_\mathrm{p},\] and hence \[{\rm d}({X_v},{X_z})\le \frac{\ell_\mathrm{p}}{64L_\mathrm{p}}r_\mathrm{p}, \qquad {\rm d}({X_w},{X_z})\le \frac{\ell_\mathrm{p}}{64L_\mathrm{p}}r_\mathrm{p}.\] The inner inclusion gives \(v,w\in W_z\), so \(\mathcal{O}_{\mathrm{win}}(v,w)=1\). This proves that \(\mathcal{O}_{\mathrm{win}}\) is an \(\bigl(\ell_\mathrm{p}/(128L_\mathrm{p}),1/8\bigr)\)-fuzzy window oracle. ◻

12 From cluster extraction to distance estimation↩︎

We now explain how the clusters from Theorem 47 yield distance estimates. The arguments in this section are simply variants from what appeared in [11][14]. Here we give a self-contained presentation, with the necessary modifications to fit the current setting.

If \(U_v\) and \(U_w\) are exact clusters around \(X_v\) and \(X_w\), then every pair \(u\in U_v\), \(u'\in U_w\) has \[{\rm d}({X_u},{X_{u'}}) = {\rm d}({X_v},{X_w})+O(r).\] Thus the pair average \(\overline{N}\!\left(U_v,U_w\right)\) estimates \(\mathrm{p}({\rm d}({X_v},{X_w}))\), up to the cluster radius and the fluctuation of the edge average. If \(\mathrm{p}\) is known and invertible on the relevant range, this gives distance estimates.

We use this in conjunction with the uniform pair-average event \[\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(\mathbf{V},\lambda_{\rm pair},m_{\rm pair})}\hypersetup{linkcolor=blue},\] where \[m_{\rm pair}:=\left\lceil \frac{1}{13}\phi(r)n\right\rceil, \qquad \lambda_{\rm pair}:=r.\] Under the scale condition of Theorem 47, Lemma 5 imply that \[\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(\mathbf{V},\lambda_{\rm pair},m_{\rm pair})}\hypersetup{linkcolor=blue}\] holds with probability \(1-o(1)\), after increasing the constant in Theorem 47. Thus, with high probability, both the cluster extraction guarantee and this pair-average event hold simultaneously.

Lemma 16 (Pair averages from exact clusters). Let \(W\subseteq\mathbf{V}\), and suppose that for every \(v\in W\) we are given \(U_v\subseteq W\) satisfying \[|U_v|\ge m, \qquad X_u\in B(X_v,r_{\rm cl}) \quad (u\in U_v).\] Assume the event \[\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}\] holds. Set \[\varepsilon_{\rm pair}:=2L_\mathrm{p}r_{\rm cl}+\lambda.\] Then, for every \(v,w\in W\) such that \[{\rm d}({X_v},{X_w})+2r_{\rm cl}\le r_\mathrm{p},\] we have \[\left| \overline{N}\!\left(U_v,U_w\right) - \mathrm{p}({\rm d}({X_v},{X_w})) \right| \le \varepsilon_{\rm pair}.\] If \(\mathrm{p}\) is globally \(L_\mathrm{p}\)-Lipschitz on \([0,\operatorname{diam}(M)]\), then the same bound holds for every \(v,w\in W\).

Proof. Fix \(v,w\in W\), and set \[d_{vw}:={\rm d}({X_v},{X_w}).\] For every \((u,u')\in\mathcal{D}(U_v,U_w)\), the triangle inequality gives \[\left| {\rm d}({X_u},{X_{u'}})-d_{vw} \right| \le {\rm d}({X_u},{X_v})+{\rm d}({X_{u'}},{X_w}) < 2r_{\rm cl}.\] If \(d_{vw}+2r_{\rm cl}\le r_\mathrm{p}\), then all these distances lie in the local bi-Lipschitz window. Hence the local Lipschitz bound gives \[\left| \mathrm{p}({\rm d}({X_u},{X_{u'}}))-\mathrm{p}(d_{vw}) \right| \le 2L_\mathrm{p}r_{\rm cl}.\] Averaging over \(\mathcal{D}(U_v,U_w)\), we obtain \[\left| \overline{\mathrm{p}}\!\left(U_v,U_w\right) - \mathrm{p}(d_{vw}) \right| \le 2L_\mathrm{p}r_{\rm cl}.\] On \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}\), \[\left| \overline{N}\!\left(U_v,U_w\right)-\overline{\mathrm{p}}\!\left(U_v,U_w\right) \right| \le \lambda.\] Combining the two inequalities proves the claim. In the globally Lipschitz case, the same argument applies without the restriction \(d_{vw}+2r_{\rm cl}\le r_\mathrm{p}\). ◻

Proof. Work on the event where Theorem 47 holds and where \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(\mathbf{V},r,m_{\rm pair})}\hypersetup{linkcolor=blue}\) holds. Then every output cluster satisfies \[U_v\text{ is a }(C_{\text{\tiny{\ref{theor:main}}}}'r,0)\text{-cluster with center }X_v,\] and \[|U_v|\ge m_{\rm pair}.\] Applying Lemma 16 with \[r_{\rm cl}=C_{\text{\tiny{\ref{theor:main}}}}'r, \qquad \lambda=r,\] gives \[\left| \overline{N}\!\left(U_v,U_w\right) - \mathrm{p}({\rm d}({X_v},{X_w})) \right| \le C r \qquad (v,w\in\mathbf{V}).\] Since \(\mathrm{p}\) is known and globally bi-Lipschitz, its inverse is \(1/\ell_\mathrm{p}\)-Lipschitz on its range. Define \(\widehat d(v,w)\) by projecting \(\overline{N}\!\left(U_v,U_w\right)\) onto \(\mathrm{p}([0,\operatorname{diam}(M)])\) and then applying \(\mathrm{p}^{-1}\). Projection can only decrease the error to the range, and inversion gives the stated bound. ◻

Proof. Again work on the intersection of the extraction event and \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(\mathbf{V},r,m_{\rm pair})}\hypersetup{linkcolor=blue}\). Set \[r_{\rm cl}:=C'_{\text{\tiny{\ref{theor:main}}}}r, \qquad \varepsilon_{\rm pair}:=2L_\mathrm{p}r_{\rm cl}+r.\] By decreasing \(c_{\text{\tiny{\ref{theor:main}}}}\), we may assume \[2r_{\rm cl}+\frac{2\varepsilon_{\rm pair}}{\ell_\mathrm{p}} \le \frac{1}{4} r_\mathrm{p}.\] Define \[\mathcal{P}_{\mathrm{loc}} := \left\{ (v,w):v\neq w,\; \overline{N}\!\left(U_v,U_w\right) \ge \mathrm{p}(r_\mathrm{p}/2)-\varepsilon_{\rm pair} \right\}.\] For \((v,w)\in\mathcal{P}_{\mathrm{loc}}\), first note that \[{\rm d}({X_v},{X_w}) \le \frac{1}{2} r_\mathrm{p}+\frac{2\varepsilon_{\rm pair}}{\ell_\mathrm{p}}.\] Indeed, if the distance were larger, then every pair \((u,u')\in\mathcal{D}(U_v,U_w)\) would have distance at least slightly above \(r_\mathrm{p}/2\), and the lower bi-Lipschitz bound together with \(\hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(\mathbf{V},r,m_{\rm pair})}\hypersetup{linkcolor=blue}\) would force \[\overline{N}\!\left(U_v,U_w\right)<\mathrm{p}(r_\mathrm{p}/2)-\varepsilon_{\rm pair},\] a contradiction. Thus \[{\rm d}({X_v},{X_w})+2r_{\rm cl}\le r_\mathrm{p},\] and Lemma 16 applies: \[\left| \overline{N}\!\left(U_v,U_w\right) - \mathrm{p}({\rm d}({X_v},{X_w})) \right| \le \varepsilon_{\rm pair}.\] Project \(\overline{N}\!\left(U_v,U_w\right)\) onto \(\mathrm{p}([0,r_\mathrm{p}])\) and invert \(\mathrm{p}\) on \([0,r_\mathrm{p}]\). Since the inverse is \(1/\ell_\mathrm{p}\)-Lipschitz, the distance error is at most \[\frac{\varepsilon_{\rm pair}}{\ell_\mathrm{p}} \le Cr.\]

Finally, if \[{\rm d}({X_v},{X_w})\le r_\mathrm{p}/2,\] then the same pair-average estimate gives \[\overline{N}\!\left(U_v,U_w\right) \ge \mathrm{p}({\rm d}({X_v},{X_w}))-\varepsilon_{\rm pair} \ge \mathrm{p}(r_\mathrm{p}/2)-\varepsilon_{\rm pair},\] so \((v,w)\in\mathcal{P}_{\mathrm{loc}}\). This proves the claim. ◻

Definition 50 (Chain property at scale \(\rho_0\)). Let \((\mathcal{X},\rho)\) be a metric space, let \[0<\rho_0\le{\rm diam}(\mathcal{X}),\] and let \(\eta\ge0\). We say that \((\mathcal{X},\rho)\) satisfies the \((\rho_0,\eta)\)-chain property if, for every \(x,y\in\mathcal{X}\) with \(\rho(x,y)>\rho_0\), there is a chain \[p_0=x,\;p_1,\ldots,p_k=y\] such that \[\rho(p_i,p_{i+1})\le\rho_0 \qquad (0\le i<k),\] and \[\left| \sum_{i=0}^{k-1}\rho(p_i,p_{i+1})-\rho(x,y) \right| \le\eta.\]

Lemma 17 (Metric extension from local estimates). Let \((\mathcal{X},\rho)\) be a finite metric space with \[0<\rho_0\le{\rm diam}(\mathcal{X}).\] Let \(\eta,\varepsilon\ge0\). Assume that \((\mathcal{X},\rho)\) satisfies the \((\rho_0,\eta)\)-chain property. Let \(\mathcal{P}\subseteq\mathcal{X}\times\mathcal{X}\) be symmetric and suppose that for every \((x,y)\in\mathcal{P}\) we are given an estimate \(\widehat\rho(x,y)\) satisfying \[\left|\widehat\rho(x,y)-\rho(x,y)\right|\le\varepsilon.\] Assume also that \[\rho(x,y)\le\rho_0 \quad\Longrightarrow\quad (x,y)\in\mathcal{P}.\] Define the weighted graph \(G_{\mathcal{P}}\) on \(\mathcal{X}\) by joining \((x,y)\in\mathcal{P}\) with edge weight \[\widehat\rho(x,y)+\varepsilon.\] Let \(\rho_{\rm sp}\) be the shortest-path metric on \(G_{\mathcal{P}}\). Then, for every \(x,y\in\mathcal{X}\), \[\rho(x,y) \le \rho_{\rm sp}(x,y) \le \rho(x,y)+\eta+C\frac{{\rm diam}(\mathcal{X})}{\rho_0}\varepsilon,\] where \(C>0\) is a universal constant.

Proof. This is the standard shortest-path extension from local metric estimates; see [12]. The added \(\varepsilon\) in each edge weight makes every edge length an upper bound on the true distance, while the chain property supplies a path whose accumulated local errors are controlled by \({\rm diam}(\mathcal{X})/\rho_0\). ◻

Corollary 51 (Global distances by chaining local estimates). Assume the conclusions of Corollary 49, and suppose that \((M, {\rm d})\) satisfies the \((r_\mathrm{p}/2,\eta)\)-chain property. Define a weighted graph on \(\mathbf{V}\) with edge set \(\mathcal{P}_{\mathrm{loc}}\) and edge weights \[\widehat d_{\mathrm{loc}}(v,w)+Cr,\] where \(C\) is the constant from Corollary 49. Let \(\widehat d_{\rm sp}\) be the induced shortest-path metric. Then \[{\rm d}({X_v},{X_w}) \le \widehat d_{\rm sp}(v,w) \le {\rm d}({X_v},{X_w}) + \eta + C\frac{\operatorname{diam}(M)}{r_\mathrm{p}}r \qquad (v,w\in\mathbf{V}).\] In particular, if \(M\) is a geodesic space and the sampled points are sufficiently dense at scale \(r\), the chain-property error \(\eta\) is controlled by the sampling resolution, and local distance recovery extends to global distance recovery.

Proof. One thing that is subtle besides directly applying the above lemma together with Corollary 49 is to justify that the sampled points \(X_{\mathbf{V}}\) also have the chain property. This is implied by \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(\mathbf{V},()}\hypersetup{linkcolor=blue}r)\), which holds with high probability under the scale condition of Theorem 47, after adjusting constants. This automatically implies the chain property for the sampled points with \(\eta\) increased by \(C\tfrac{{\rm diam}(M)}{r_\mathrm{p}} r\), because every point along the chain has a sampled point within distance \(r\). ◻

13 Auxiliary probability tools and proofs↩︎

Lemma 18 (Chernoff inequality for \(0\)-\(1\) random variables). Let \(X_1,\dots,X_N\) be independent random variables taking values in \(\{0,1\}\), and set \[S_N:=\sum_{i=1}^N X_i, \qquad \mu:=\mathbb{E} S_N.\] Then for every \(t\ge0\), \[\mathbb{P}(S_N\ge \mu+t) \le \exp\!\left(-\frac{t^2}{2\mu+t}\right),\] and for every \(0\le t\le\mu\), \[\mathbb{P}(S_N\le \mu-t) \le \exp\!\left(-\frac{t^2}{2\mu}\right).\]

This standard form may be found, for instance, in [18].

Remark 52. Suppose we want an upper-tail failure probability at most \(\exp(-L)\), where \(L>0\). A convenient sufficient condition is \[\mathbb{P}\left(S_N\ge(\sqrt{\mu}+\sqrt L)^2\right)\le \exp(-L).\] Indeed, the upper-tail Chernoff bound shows that it is enough to take \[t\ge \frac{L+\sqrt{L^2+8\mu L}}{2},\] and \(t=L+2\sqrt{\mu L}\) is a valid choice.

Lemma 19 (Bernstein’s inequality). Let \(Y_1,\dots,Y_N\) be independent mean-zero random variables such that \(|Y_i|\le M\) almost surely for all \(i\), and let \[\sigma^2:=\sum_{i=1}^N\operatorname{Var}(Y_i).\] Then for every \(t>0\), \[\mathbb{P}\!\left(\left|\sum_{i=1}^N Y_i\right|>t\right) \le 2\exp\!\left(-\frac{t^2}{2(\sigma^2+Mt/3)}\right).\]

This is the standard Bernstein inequality; see, for instance, [18].

Lemma 20 (Net cardinality from lower regularity). Let \((M,{\rm d},\mu)\) satisfy lower \(\phi\)-regularity up to scale \(r_\mu\). If \(\delta\in(0,r_\mu]\) and \(\mathcal{N}\subset M\) is a maximal \(\delta\)-separated set, then \(\mathcal{N}\) is a \(\delta\)-net of \(M\) and \[|\mathcal{N}|\le \frac{1}{\phi(\delta/2)}.\]

Proof. Maximality gives the net property: otherwise a point at distance \(>\delta\) from all points of \(\mathcal{N}\) could be added to \(\mathcal{N}\). If \(\mathcal{N}=\{x_1,\ldots,x_N\}\), then the balls \(B(x_i,\delta/2)\) are pairwise disjoint. Hence \[1=\mu(M) \ge \sum_{i=1}^N \mu(B(x_i,\delta/2)) \ge N\phi(\delta/2),\] which proves the cardinality bound. ◻

Proof of Lemma 2. Let \(\mathcal{N}\) be a maximal \(r/3\)-separated set. By Lemma 20, \[|\mathcal{N}|\le \frac{1}{\phi(r/6)}.\] For \(x_i\in\mathcal{N}\), let \[S_i:=|\{v\in W:X_v\in B(x_i,r/3)\}|.\] Then \(S_i\) is binomial with mean at least \(n_\star\phi(r/3)\). The lower Chernoff bound gives \[\mathbb{P}\left(S_i<\frac{1}{2} n_\star\phi(r/3)\right) \le \exp\left(-\frac{n_\star\phi(r/3)}{8}\right).\] Taking a union bound over \(\mathcal{N}\) and using \(\phi(r/3)\ge\phi(r/6)\ge \Lambda \log n_\star/n_\star\), we get \[\begin{align} \mathbb{P}\left( \exists x_i\in\mathcal{N}:S_i<\frac{1}{2} n_\star\phi(r/3) \right) \le& \frac{1}{\phi(r/6)} \exp\left(-\frac{n_\star\phi(r/3)}{8}\right) \\ \le & \frac{n_\star}{\Lambda\log n_\star} \exp\left(-\frac{\Lambda}{8}\log(n_\star)\right)\\ \le & \exp\left( \log(n_\star) - \tfrac{1}{8}\Lambda \log(n_\star) \right) \le \exp\left(-\frac{\Lambda}{16}\log(n_\star)\right)\,, \end{align}\] provided \(\Lambda\) is large enough than some universal constant.

On the complementary event, fix \(x\in M\). Since \(\mathcal{N}\) is an \(r/3\)-net, choose \(x_i\in\mathcal{N}\) with \({\rm d}({x},{x_i})\le r/3\). Then \[B(x_i,r/3)\subseteq B(x,r),\] because the balls are open. Therefore \[|\{v\in W:X_v\in B(x,r)\}| \ge S_i \ge \frac{1}{2} n_\star\phi(r/3).\] This is exactly \(\hypersetup{linkcolor=black}\hyperref[eq:def-lower-occupancy-event]{\mathcal{E}_{\text{\tiny{\rm pt}}}(W,r)}\hypersetup{linkcolor=blue}\). ◻

Proof of Lemma 3. Fix \(X_U=x_U\) and \(X_v=x_v\), and write \(m:=|U|\), \[a_u:=\mathrm{p}({\rm d}({x_u},{x_v})), \qquad \xi_u:=\widetilde{Z}_{u,v}-a_u, \qquad B_u:=B_{u,v}.\] Then the \(B_u\)’s are independent Bernoulli\((\mathsf{s})\), the \(\xi_u\)’s are independent centered \({\rm K}_{\mathrm{sg}}\)-subgaussian variables, and these two families are independent. Also \(|a_u|\le M_\mathrm{p}\). Since \[Z_{u,v}=B_u(a_u+\xi_u),\] we have \[\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue} = \frac{1}{\mathsf{s}m}\sum_{u\in U}(Z_{u,v}-\mathsf{s}a_u) =T_1+T_2,\] where \[T_1:=\frac{1}{\mathsf{s}m}\sum_{u\in U}(B_u-\mathsf{s})a_u, \qquad T_2:=\frac{1}{\mathsf{s}m}\sum_{u\in U}B_u\xi_u.\]

For \(T_1\), set \(Y_u=(B_u-\mathsf{s})a_u\). Then the \(Y_u\)’s are independent, mean-zero, \(|Y_u|\le M_\mathrm{p}\), and \[\sum_{u\in U}\operatorname{Var}(Y_u)\le \mathsf{s}m M_\mathrm{p}^2.\] Bernstein’s inequality with \(s=t\mathsf{s}m/2\) gives \[\mathbb{P}\left(|T_1|>\frac{t}{2}\mid X_U=x_U,X_v=x_v\right) \le 2\exp\bigl(-c_1\mathsf{s}m\min\{t^2,1\}\bigr).\]

For \(T_2\), let \(N_B:=\sum_{u\in U}B_u\). By Chernoff, \[\mathbb{P}(N_B>2\mathsf{s}m\mid X_U=x_U,X_v=x_v) \le \exp(-c\mathsf{s}m).\] On the event \(N_B\le2\mathsf{s}m\), conditioning on the \(B_u\)’s, the sum \(\sum_u B_u\xi_u\) is subgaussian with parameter at most \({\rm K}_{\mathrm{sg}}\sqrt{2\mathsf{s}m}\). Thus, again with \(s=t\mathsf{s}m/2\), \[\mathbb{P}\left(|T_2|>\frac{t}{2}\mid X_U=x_U,X_v=x_v\right) \le 3\exp\bigl(-c_2\mathsf{s}m\min\{t^2,1\}\bigr).\] Combining the two bounds and adjusting constants proves the claim. ◻

Proof of Lemma 4. For each fixed \(v\in V\), Lemma 3 gives \[\mathbb{P}\left( \left|\hypersetup{linkcolor=black}\hyperref[eq:def-normalized-average]{{\rm N}_{U}(v)}\hypersetup{linkcolor=blue}-\hypersetup{linkcolor=black}\hyperref[def:latent-empirical-link-averages]{{\mathrm{p}}_{U}(v)}\hypersetup{linkcolor=blue}\right|>\hypersetup{linkcolor=black}\hyperref[eq:def-fluctuation-scale]{\varepsilon_{U}(n_\star)}\hypersetup{linkcolor=blue} \;\middle|\;X_U=x_U,\;X_V=x_V \right) \le C_{\rm link} \exp\left( -c_{\rm link}\mathsf{s}|U| \min\{\hypersetup{linkcolor=black}\hyperref[eq:def-fluctuation-scale]{\varepsilon_{U}(n_\star)}\hypersetup{linkcolor=blue}^2,1\} \right).\] Since \(\mathsf{s}|U|\ge \Lambda \log n_\star\), we have \(\hypersetup{linkcolor=black}\hyperref[eq:def-fluctuation-scale]{\varepsilon_{U}(n_\star)}\hypersetup{linkcolor=blue}\le1\), and the exponent is at most \(-c_{\rm link}\Lambda \log n_\star\). A union bound over \(|V|\le n_\star\) gives \[\mathbb{P}\left( \hypersetup{linkcolor=black}\hyperref[eq:def-navigation-event]{\mathcal{E}_{\rm link}(U,V;n_\star)}\hypersetup{linkcolor=blue}^{\,c} \;\middle|\;X_U=x_U,X_V=x_V \right) \le e^{- \tfrac{1}{2}c_{\rm link}\Lambda \log n_\star}\,,\] provided that \(\Lambda\) is larger than some universal constants depending on \(C_{\rm link}\) and \(c_{\rm link}\). ◻

Proof of Lemma 5. Fix a realization \(X_W=x_W\). Let \(U_1,U_2\subseteq W\) have \[|U_1|=a,\qquad |U_2|=b,\qquad a,b\ge m.\] For each unordered pair \(e=\{u,v\}\subseteq W\), define \[w_e:= \mathbf{1}_{\{u\in U_1,\;v\in U_2\}} + \mathbf{1}_{\{v\in U_1,\;u\in U_2\}}.\] Then \(w_e\in\{0,1,2\}\), and \[D:=\sum_{e\subseteq W}w_e =|U_1||U_2|-|U_1\cap U_2| =|\mathcal{D}(U_1,U_2)|.\] For \(e=\{u,v\}\), write \[a_e:=\mathrm{p}({\rm d}({x_u},{x_v})), \qquad \xi_e:=\widetilde{Z}_{u,v}-a_e, \qquad B_e:=B_{u,v}.\] As in the fixed-vertex proof, \[\overline{N}\!\left(U_1,U_2\right)-\overline{\mathrm{p}}\!\left(U_1,U_2\right) = \frac{1}{\mathsf{s}D}\sum_{e\subseteq W}w_e(Z_e-\mathsf{s}a_e) =T_1+T_2,\] where \[T_1:=\frac{1}{\mathsf{s}D}\sum_{e\subseteq W}w_e(B_e-\mathsf{s})a_e, \qquad T_2:=\frac{1}{\mathsf{s}D}\sum_{e\subseteq W}w_eB_e\xi_e.\]

For \(T_1\), set \(Y_e=w_e(B_e-\mathsf{s})a_e\). Then \[|Y_e|\le2M_\mathrm{p}, \qquad \sum_{e\subseteq W}\operatorname{Var}(Y_e) \le \mathsf{s}M_\mathrm{p}^2\sum_{e\subseteq W}w_e^2 \le 2\mathsf{s}D M_\mathrm{p}^2.\] Bernstein’s inequality with \(s=\lambda\mathsf{s}D/2\) gives \[\mathbb{P}\left(|T_1|>\frac{\lambda}{2}\mid X_W=x_W\right) \le 2\exp(-c_1\lambda^2\mathsf{s}D).\]

For \(T_2\), let \(W_B:=\sum_{e\subseteq W}w_eB_e\). Since \(\mathbb{E}[W_B\mid X_W=x_W]=\mathsf{s}D\) and \(0\le w_eB_e\le2\), Bernstein’s inequality gives \[\mathbb{P}(W_B>2\mathsf{s}D\mid X_W=x_W) \le \exp(-c_2\mathsf{s}D).\] On the event \(W_B\le2\mathsf{s}D\), conditioning on the \(B_e\)’s, \[\sum_{e\subseteq W}w_eB_e\xi_e\] is subgaussian with parameter at most \[{\rm K}_{\mathrm{sg}}\left(\sum_{e\subseteq W}w_e^2B_e\right)^{1/2} \le 2{\rm K}_{\mathrm{sg}}\sqrt{\mathsf{s}D}.\] Therefore \[\mathbb{P}\left(|T_2|>\frac{\lambda}{2}\mid X_W=x_W\right) \le 3\exp(-c_3\lambda^2\mathsf{s}D).\] Combining the two bounds, \[\mathbb{P}\left( \bigl|\overline{N}\!\left(U_1,U_2\right)-\overline{\mathrm{p}}\!\left(U_1,U_2\right)\bigr|>\lambda \;\middle|\;X_W=x_W \right) \le 4\exp(-c_4\lambda^2\mathsf{s}D).\]

It remains to union-bound over \(U_1,U_2\). Since \(m\ge2\), \[D=ab-|U_1\cap U_2|\ge ab-\min\{a,b\}\ge\frac{ab}{2}.\] For \(\alpha=a/n_\star\) and \(\beta=b/n_\star\), the number of pairs \((U_1,U_2)\) with these cardinalities is at most \[\binom{n_\star}{a}\binom{n_\star}{b} \le \exp\left((\alpha+\beta)n_\star\log(e/\varphi)\right),\] because \(\alpha,\beta\ge\varphi\). Also \(\alpha+\beta\le2\alpha\beta/\varphi\). Thus, if \[\mathsf{s}n_\star\ge \frac{C\log(e/\varphi)}{\varphi\lambda^2}\] with \(C\) large enough, then each cardinality class contributes at most \[4\exp(-c_5\lambda^2\mathsf{s}\alpha\beta n_\star^2).\] Summing over at most \(n_\star^2\) cardinality pairs and using \(\alpha\beta\ge\varphi^2\), we obtain \[\mathbb{P}\left( \hypersetup{linkcolor=black}\hyperref[eq:def-pair-average-event]{\mathcal{E}_{\text{\tiny{\rm avg}}}(W,\lambda,m)}\hypersetup{linkcolor=blue}^{\,c} \;\middle|\;X_W=x_W \right) \le 4n_\star^2 \exp(-c_5\lambda^2\mathsf{s}\varphi^2n_\star^2).\] The assumed lower bound on \(\mathsf{s}n_\star\), together with \(m\ge2\), implies that the right-hand side is at most \[\exp\{-c\varphi n_\star\log(e/\varphi)\}\] after increasing \(C\) and decreasing \(c\). This proves the lemma. ◻

High-probability consequence of Lemma 5. The failure probability in Lemma 5 is \[\exp\{-c\varphi n_\star\log(e/\varphi)\}.\] If \(\varphi n_\star\log(e/\varphi)\gg\log n_\star\), this is \(n_\star^{-\omega(1)}\). ◻

References↩︎

[1]
P. D. Hoff, A. E. Raftery, and M. S. Handcock, “Latent space approaches to social network analysis,” Journal of the American Statistical Association, vol. 97, no. 460, pp. 1090–1098, 2002, doi: 10.1198/016214502388618906.
[2]
M. Penrose, Random geometric graphs, vol. 5. Oxford University Press, 2003.
[3]
E. A. Valdivia and Y. D. Castro, “Latent distance estimation for random geometric graphs,” in Advances in neural information processing systems 32, 2019, [Online]. Available: https://proceedings.neurips.cc/paper/2019/hash/c4414e538a5475ec0244673b7f2f7dbb-Abstract.html.
[4]
K. Chaudhuri and S. Dasgupta, “Rates of convergence for nearest neighbor classification,” in Advances in neural information processing systems 27, 2014, pp. 3437–3445, [Online]. Available: https://proceedings.neurips.cc/paper/5439-rates-of-convergence-for-nearest-neighbor-classification.
[5]
S. Gadat, T. Klein, and C. Marteau, “Classification with the nearest neighbor rule in general finite dimensional spaces,” The Annals of Statistics, vol. 44, no. 3, pp. 982–1009, 2016, doi: 10.1214/15-AOS1395.
[6]
J. B. Tenenbaum, V. de Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science, vol. 290, no. 5500, pp. 2319–2323, 2000, doi: 10.1126/science.290.5500.2319.
[7]
M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation, vol. 15, no. 6, pp. 1373–1396, 2003, doi: 10.1162/089976603321780317.
[8]
M. Hein, J.-Y. Audibert, and U. von Luxburg, “Graph laplacians and their convergence on random neighborhood graphs,” Journal of Machine Learning Research, vol. 8, pp. 1325–1368, 2007, [Online]. Available: https://www.jmlr.org/papers/v8/hein07a.html.
[9]
J. Heinonen, Lectures on analysis on metric spaces. New York: Springer, 2001.
[10]
Q. Duchemin and Y. de Castro, “Random geometric graph: Some recent developments and perspectives,” arXiv preprint arXiv:2203.15351, 2022, [Online]. Available: https://arxiv.org/abs/2203.15351.
[11]
H. Huang, P. Jiradilok, and E. Mossel, Extended abstract; see arXiv:2402.09591 for full version“Reconstructing the geometry of random geometric graphs,” in Proceedings of thirty seventh conference on learning theory, 2024, vol. 247, pp. 2519–2521, [Online]. Available: https://proceedings.mlr.press/v247/huang24c.html.
[12]
H. Huang, P. Jiradilok, and E. Mossel, “Reconstructing riemannian metrics from random geometric graphs,” arXiv preprint arXiv:2511.05434, 2025, [Online]. Available: https://arxiv.org/abs/2511.05434.
[13]
H. Huang, P. Jiradilok, and E. Mossel, “Denoising distances beyond the volumetric barrier,” arXiv preprint arXiv:2604.00432, 2026, [Online]. Available: https://arxiv.org/abs/2604.00432.
[14]
C. Fefferman, J. Marty, and K. Ren, “Reconstruction of manifold distances from noisy observations,” arXiv preprint arXiv:2511.13025, 2025, [Online]. Available: https://arxiv.org/abs/2511.13025.
[15]
E. Arias-Castro, A. Channarond, B. Pelletier, and N. Verzelen, “On the estimation of latent distances using graph distances,” Electronic Journal of Statistics, vol. 15, no. 1, pp. 722–747, 2021, doi: 10.1214/21-EJS1801.
[16]
P. W. Holland, K. B. Laskey, and S. Leinhardt, “Stochastic blockmodels: First steps,” Social Networks, vol. 5, no. 2, pp. 109–137, 1983, doi: 10.1016/0378-8733(83)90021-7.
[17]
E. Abbe, “Community detection and stochastic block models: Recent developments,” Journal of Machine Learning Research, vol. 18, no. 177, pp. 1–86, 2018, [Online]. Available: https://www.jmlr.org/papers/v18/16-480.html.
[18]
R. Vershynin, High-dimensional probability. An introduction with applications in data science, Second., vol. 47. Cambridge: Cambridge University Press, 2025.