1 Introduction↩︎

Recent advances in self-supervised learning have fueled the development of foundation models in domains ranging from computer vision [1] to geospatial analysis [2], where tasks like semantic similarity search of distant regions or mineral prospectivity mapping require robust representations of spatially distributed data. In particular, masked autoencoders (MAEs) and contrastive methods have emerged as two prominent paradigms. MAEs reconstruct missing content from partially observed inputs, thereby learning localized semantics [1], [3]–[6], while contrastive approaches push away “unrelated” samples to avoid representational collapse [7], [8]. It has recently been shown that plain MAEs can still suffer from partial dimensional collapse in feature space; thus, adding a uniformity or repulsion term can spread out different samples’ embeddings [4].

In geospatial applications, data are often geographically uneven, and when absolute location is provided as an input feature, it may spuriously dominate learned representations if not properly controlled [9]. Our work unifies masked reconstruction and metric learning in a single end-to-end framework on graphs. Here, metric learning refers to learning an embedding geometry in which distances or cosine similarities reflect useful node- and graph-level relationships, using self-supervised objectives rather than supervised pair or triplet labels. To avoid trivial geospatial shortcuts, we exclude absolute/global position signals from the input features, and learn representations from observed features, relative positions, and graph structure alone. Specifically, we encourage each node’s latent feature representation to be accurate for reconstructing its masked features, a positive alignment principle in the reconstruction-relevant component; to remain consistent across matched nodes in augmented views, a cross-view node-correspondence principle in the complementary contextual component; and to remain well spread relative to other sampled nodes in the batch, a node uniformity concept, as well as from entirely different graphs, a graph uniformity concept. We impose a bi-Lipschitz continuity constraint on the feature decoder (acting on a low-dimensional reconstruction component), so that accurate reconstruction encourages proximity in that reconstruction subspace, and we employ Lipschitz pooling operators, notably an adaptive softmax attention pooling mechanism (§2.4), to aggregate node embeddings into robust graph-level representations. These Lipschitz-continuous aggregators ensure that small local perturbations in node embeddings produce proportionally bounded shifts in the resulting graph embedding, even though they do not in general admit a lower-Lipschitz bound or guarantee semantic separation.

Our framework employs normalized/cosine contrastive loss terms inspired by the Spectral Contrastive Loss (SCL) of [10]. Because MABLE uses cosine similarity rather than unconstrained raw inner products, the resulting fixed-cosine surrogate removes scale ambiguity and admits a heuristic second-order Donsker–Varadhan (DV)-style interpretation under boundedness, approximate symmetry, and small-variance assumptions (see Appendix 6.3 for details). In place of a learned discriminator and the InfoNCE log softmax [7], we use a fixed squared-similarity surrogate, and introduce two key advancements: First, we impose spectral constraints on the decoder, bounding its singular values to enforce both upper and lower Lipschitz continuity so that small reconstruction errors encourage proximity in the reconstruction subspace. Second, we embed the surrogate in a masked autoencoder and apply it at both node and graph scales, while also adding an explicit node-level correspondence objective across augmented views in a contextual component of the node embedding, all without explicit hard-negative selection, crucial for geospatial data where “negatives” can still be semantically related.

2 A Unified End-to-End Framework↩︎

Our masked-graph pretraining pipeline addresses three challenges in one pass: (i) encouraging node similarity in a reconstruction-relevant component via masked feature reconstruction and a bi-Lipschitz feature decoder, while also encouraging cross-view consistency of matched nodes in a complementary contextual component; (ii) encouraging graph uniformity by pushing graph embeddings toward a well-spread, approximately isotropic distribution in normalized embedding space; and (iii) suppressing geospatial bias by dispensing with explicit hard-negative selection and absolute/global locations.

Conceptually, the loss is the masked-graph counterpart of SCL, but expressed with cosine or squared-distance penalties instead of any learned discriminator or InfoNCE term. Combined with the spectral constraints, this fixed-surrogate design encourages reconstruction-driven alignment in the reconstruction component, cross-view node consistency in the contextual component, and graph-level alignment/uniformity without a separate substructure-to-graph module, while avoiding the instability of adversarial training. We draw inspiration from mutual-information methods such as Deep InfoMax [11], [12] while remaining fully non-adversarial, and from recent graph self-supervised learning methods including GraphCL, GRACE, and BGRL [13]–[15].

Throughout, we use “graph” to denote a variable-sized collection of node observations with optional explicit edges. The experiments in this paper instantiate the edge-free case: no hand-crafted adjacency matrix is supplied, and interactions are induced by the attention mechanism over node tokens (see §4.1). Explicit edge-aware variants are a natural extension when reliable connectivity is available. The objective itself is not tied to geospatial inputs or to a particular encoder: the mineral-exploration setting below is the experimental demonstration and validation domain.

2.1 Notation and Node Feature Similarity Preservation↩︎

We begin by formalizing the encoder–decoder model and illustrating how masked reconstruction can help preserve node-level similarity in the latent space, provided the decoder is suitably constrained. By hiding portions of each node’s features, we encourage the model to learn robust, cross-modal or cross-attribute relationships, rather than simply memorizing direct inputs.

2.1.0.1 Encoder–Decoder Setup.

Let \(G=(V(G), E(G))\) denote a graph with vertex set \(V(G)\) and edge set \(E(G)\). Each node \(u\in V(G)\) is associated with a feature vector \(x_u \in \mathbb{R}^{d_x}\), where \(d_x\) denotes the feature (attribute) dimension; we write \(X(G):=\{x_u\}_{u\in V(G)}\) for the collection of node features, and by abuse of notation also denote by \(X(G)\in\mathbb{R}^{|V(G)|\times d_x}\) the matrix obtained by stacking \(x_u^\top\) as rows in an arbitrary but fixed node ordering. When \(G\) and \(X\) are clear from context, we write \(X:=X(G)\). Here \(f_\theta\) is a graph-level encoder (e.g.a GNN) that processes the whole graph \(G\) together with an input feature collection \(X\) (which may be masked) and returns a contextualised embedding for every node; for notational simplicity, any additional auxiliary per-node context supplied to the encoder is left implicit. We write \(f(u)\) as shorthand for the more explicit \((f_\theta(G,X))_u\) when \(G\) and \(X\) are clear from context. Our model consists of an encoder that produces node embeddings of dimension \(d\) and a feature decoder that acts on a reconstruction-facing subspace of dimension \(d_e\le d\): \[f_\theta(G,X(G)) \in \mathbb{R}^{|V(G)|\times d} \quad\text{and}\quad g: \mathbb{R}^{d_e} \;\to\; \mathbb{R}^{d_x},\] where \(z_u := f(u) := (f_\theta(G,X(G)))_u \in\mathbb{R}^d\) denotes the node embedding. When a minibatch contains multiple graphs, we distinguish node instances by writing \[\mathcal{B}_{\mathrm{node}} := \{(G,u):\, G\in\mathrm{batch},\;u\in V(G)\},\] and for \((G,u)\in\mathcal{B}_{\mathrm{node}}\) we write \[f(G,u) := (f_\theta(G,X(G)))_u.\] Thus, the shorthand \(f(u)\) is used only when the underlying graph \(G\) (and input view \(X\)) are fixed by context.

To reconstruct node features, we define a low-dimensional feature embedding \(e_u := \pi(z_u)\in\mathbb{R}^{d_e}\) as a coordinate slice \(\pi(z)=z_{1:d_e}\). We denote the complementary contextual slice by \[c_u := \rho(z_u) := z_{d_e+1:d}\in\mathbb{R}^{d-d_e}.\] The feature decoder \(g\) reconstructs node features from \(e_u\), and the autoencoder is \(h(u):=g(e_u)=g(\pi(f(u)))\). In the sequel, masked reconstruction acts on the feature slice \(e_u\), while optional cross-view node-consistency terms may be imposed on the contextual slice \(c_u\). This split embeds a constrained reconstruction pathway inside the broader graph encoder: the feature slice is geometrically tied to reconstructible feature similarity through the decoder, while the contextual slice remains available for view-stable information used by pooling and downstream graph representations. It also decouples the full node-embedding dimension from the reconstruction dimension, allowing high-dimensional contextual node embeddings while keeping the decoder input sized relative to the actual feature dimension \(d_x\).

Finally, let \(P_\phi\) be a permutation-invariant pooling operator mapping multisets of node embeddings to a graph embedding: \[P_\phi:\;(\mathbb{R}^d)^{|V(G)|}\to \mathbb{R}^{d_G},\qquad s_G := P_\phi(\{z_u\}_{u\in V(G)}).\]

In words, \(P_\phi\) pools the unordered multiset of node embeddings into a single fixed-dimensional graph embedding \(s_G\), used for graph-level losses and downstream tasks. Permutation invariance ensures that \(s_G\) depends only on the graph content, not on the arbitrary ordering of its nodes, consistent with the general set-function viewpoint of Deep Sets [16].

2.1.0.2 View operator for masking and augmentation.

We model masking and augmentation as a random view operator \(T\sim\mathcal{A}\) acting on the graph–feature pair: \[(G^{(a)},X^{(a)},\iota^{(a)})=T(G,X).\] Here \(G^{(a)}\) and \(X^{(a)}\) are the augmented graph and feature collection, while \(\iota^{(a)}:V(G^{(a)})\to V(G)\) is an injective map recording the original node identity of each retained node. Feature-only masking is the special case \(G^{(a)}=G\) and \(\iota^{(a)}=\mathrm{id}\); node or subgraph drops remove vertices from \(G^{(a)}\) and restrict \(\iota^{(a)}\) to the retained nodes. Unless otherwise stated, when a single graph \(G\) and input view \(X\) are fixed by context, we continue to write \(f(u):=(f_\theta(G,X))_u\) for the node embedding under the current input \(X\) (masked or unmasked). In minibatch settings involving multiple graphs, we instead write \(f(G,u)\). For two sampled view operators \(T_1,T_2\), we write \((G^{(a)},X^{(a)},\iota^{(a)})=T_a(G,X)\) for \(a\in\{1,2\}\). In our masked setting, \(h\) predicts missing or masked features from partially observed inputs, forcing the learned representation to capture meaningful dependencies within and across features.

2.1.0.3 Matrix norms and singular values.

For any matrix \(W\in\mathbb{R}^{p\times q}\), define \[\sigma_{\max}(W) \;:=\; \max_{\|v\|_2=1}\|Wv\|_2 \;=\; \sqrt{\lambda_{\max}(W^\top W)}, \qquad \sigma_{\min}(W) \;:=\; \min_{\|v\|_2=1}\|Wv\|_2 \;=\; \sqrt{\lambda_{\min}(W^\top W)}.\] When \(p\ge q\) (tall or square) and \(W\) has full column rank, \(\sigma_{\min}(W)>0\) and coincides with the smallest (nonzero) singular value in the usual ordering. When \(p<q\) (wide), \(\sigma_{\min}(W)=0\).

2.1.0.4 Cosine similarity.

We define cosine similarity as \[\cos(z,z') \;:=\; \begin{cases} \dfrac{z^\top z'}{\|z\|_2\,\|z'\|_2}, & \text{if } z\neq 0 \text{ and } z'\neq 0,\\[6pt] 0, & \text{otherwise.} \end{cases}\]

2.1.0.5 Positive Alignment via Masking.

Masked autoencoders, including graph variants [1], [4]–[6], suggest that masking can align nodes in the reconstruction-relevant feature-embedding space when local structures are similar, because reconstructing missing features or, in graph-autoencoding settings, masked edges/structure from context often forces the model to group similar inputs together. If two nodes \(u\) and \(v\) share similar local context, features, or incident structure, then under sufficient masking, \(h(u)\) and \(h(v)\) must correctly fill in similar content or structure. This typically helps their learned representations \(f(u)\) and \(f(v)\) to become proximate. In this paragraph we use \(f(u)\) and \(g(\cdot)\) in the standard autoencoder sense (not MABLE-specific notation).

However, if the decoder \(g\) is unconstrained, the reconstruction loss only requires \(h(u) = g\bigl(f(u)\bigr)\) to match \(u\) in the input–output space. That alone does not strictly guarantee that \(f(u) \approx f(v)\) for two nodes \(u,v\) with similar original features. In other words, \(g\) might learn to “invert” \(f(u)\) in arbitrary ways without necessarily preserving pairwise distances in the latent space. This scenario shares conceptual parallels with issues like posterior collapse in Variational Autoencoders (VAEs), where an overly powerful or flexible decoder can learn to ignore the latent variables, rendering them uninformative, because it can model the data distribution well without relying on structured latent codes [17]. To ensure that the reconstruction-relevant latent space captured by \(e_u=\pi(f(u))\) reflects meaningful similarities and is not trivialized by the decoder’s capacity, we turn to explicit constraints on \(g\).

2.1.0.6 Bi-Lipschitz Feature Decoding for Latent Alignment.

To promote embedding-level alignment, we impose a bi-Lipschitz constraint on \(g\). The next proposition shows that, when \(g\) is bi-Lipschitz and reconstruction is accurate, similarity in the input domain translates into controlled proximity in the reconstruction subspace.

See proof in Appendix 6.1. Thus, node feature similarity is approximately preserved in the feature-embedding space used for reconstruction when the decoder is bi-Lipschitz and reconstruction error is small. The proposition is a pointwise geometric statement, while in training we optimize an empirical/expected mask-dependent reconstruction objective as its practical surrogate. Masking can amplify this effect because the model must predict missing information from partial inputs, implicitly aligning nodes that fill in similar content [4].

Scope of the guarantee. Proposition 1 is a statement about the reconstruction-relevant feature slice \(e_u=\pi(z_u)\) under a bi-Lipschitz decoder and small reconstruction error. It does not by itself imply that the full node embedding \(z_u\) preserves all task-relevant similarities, nor that masking alone guarantees alignment in the full latent space.

Practical note. We approximate the bi-Lipschitz property with a single linear decoder \(g(e)=W_{dec}e+b\) whose singular values are kept in a fixed band \(m\!\le\!\sigma_i(W_{dec})\!\le\!L\). Implementation notes and computational details are discussed in §2.5. For a linear map, achieving a non-trivial lower bound \(m>0\) requires \(\sigma_{\min}(W_{dec})>0\), i.e.\(W_{dec}\) has full column rank (and thus \(d_e \le d_x\) in the Euclidean setting). If \(d_e > d_x\), then \(\sigma_{\min}(W_{dec})=0\) and no nontrivial lower bound exists.

2.2 Fixed-Cosine Surrogates↩︎

InfoMax–style objectives learn representations by contrasting the joint distribution \(p_{\text{pos}}(z,z^{+})\), with \(z=f(x)\) and \(z^{+}=f(x^{+})\) an augmented view, against the product distribution \(p(z)^{2}\) of unpaired embeddings [7], [11], [12], [18]. Positive pairs \((z,z^{+})\) are encouraged to score higher than uncorrelated pairs \((z,z^{-})\) treated as approximately independent negatives (e.g., in-batch), typically by maximising a mutual‑information lower bound such as InfoNCE, a Jensen–Shannon variant, or the binary cross‑entropy (BCE) objective [19]: \[\mathcal{L}_{\text{BCE}} = \mathbb{E}_{(z,z^{+})}\bigl[-\log \sigma\bigl(D(z,z^{+})\bigr)\bigr] + \mathbb{E}_{(z,z^{-})}\bigl[-\log\bigl(1-\sigma\bigl(D(z,z^{-})\bigr)\bigr)\bigr],\] where \(\sigma(u)=1/(1+e^{-u})\) and \(D\) is a learned discriminator. Besides introducing additional parameters, this formulation assumes that in-batch negatives are semantically unrelated, an assumption that often fails for real-world geospatial corpora, where nearby coordinates typically share land-cover, climate, or geology. In practice, when absolute coordinates are available to the model, a powerful discriminator may latch onto absolute location: the simplest way to separate a random negative from its positive is often just coordinate distance, not a meaningful semantic cue. In our setting, we avoid this shortcut at the source by not feeding absolute/global positions as input features. We still prefer fixed-similarity surrogates over learned discriminators to reduce the risk of other trivial separation cues dominating training dynamics in geographically uneven corpora.

To address these limitations and avoid the learned discriminator, we use fixed cosine-similarity objectives. The raw inner-product form of the Spectral Contrastive Loss (SCL) of [10] already expresses the desired pairwise geometry: \[\mathcal{L}_{\text{SCL,raw}} = -2 \, \mathbb{E}_{(z,z^{+})}\bigl[z^\top z^{+}\bigr] + \mathbb{E}_{(z,z^{-})}\bigl[(z^\top z^{-})^{2}\bigr].\] The positive term rewards large positive-pair inner products, while the quadratic negative term penalizes nonzero inner products for unpaired embeddings. The original population objective is written in terms of raw inner products, but the empirical setting in [10] still uses norm control, including projection to a bounded ball with experimentally chosen radius, and the finite-sample analysis assumes norm-controlled function classes. Without normalization, however, this objective mixes angular structure with vector scale: after angular alignment saturates, the positive term can still be improved by increasing norms, and the negative term can be reduced either by orthogonality or by shrinking norms.

In MABLE we remove this scale ambiguity by working directly with cosine similarity, \(\cos(z,z') \in [-1,1]\). For unit-normalized embeddings, \(\cos(z,z')=z^\top z'\), and SCL becomes the normalized/cosine objective \[\mathcal{L}_{\text{SCL}} = -2 \, \mathbb{E}_{(z,z^{+})}\bigl[\cos(z,z^{+})\bigr] + \mathbb{E}_{(z,z^{-})}\bigl[\cos(z,z^{-})^{2}\bigr].\] This cleanly targets angular alignment of positive pairs (\(\cos(z,z^{+}) \to 1\)) and angular decorrelation of unpaired embeddings (\(\cos(z,z^{-}) \to 0\)).

The same geometry can be motivated from a fixed log-based surrogate. For positive pairs, choose an argument whose negative log is minimized as \(\cos(z,z^{+})\to 1\), namely \(\exp(\cos(z,z^{+})-1)\). For unpaired embeddings, choose an argument whose negative log is minimized as \(\cos(z,z^{-})\to 0\), namely \(1-\frac{1}{2}\cos(z,z^{-})^2\). This gives \[\mathcal{L}_{\text{Surrogate}} = -\mathbb{E}_{(z,z^{+})}[\log(\exp(\cos(z,z^{+})-1))] -\mathbb{E}_{(z,z^{-})}[\log(1 - \frac{1}{2}\cos(z,z^{-})^2)].\] The positive term simplifies, up to an additive constant, to \(-\cos(z,z^{+})\). The negative term has the Taylor expansion \[-\log\!\left(1-\frac{1}{2}\cos(z,z^{-})^2\right) = \frac{1}{2}\cos(z,z^{-})^2 + O(\cos(z,z^{-})^4)\] near the desired negative optimum \(\cos(z,z^{-})=0\).

Thus the local quadratic approximation to the log-based surrogate is the scaled normalized SCL objective. Let \(\mathcal{L}_{\text{SSCL}} = \tfrac12 \, \mathcal{L}_{\text{SCL}}\): \[\mathcal{L}_{\text{SSCL}} = -\mathbb{E}_{(z,z^{+})}\bigl[\cos(z,z^{+})\bigr] + \tfrac12 \, \mathbb{E}_{(z,z^{-})}\bigl[\cos(z,z^{-})^{2}\bigr].\] This is the fixed-cosine geometry used throughout MABLE: the first term aligns true pairs, while the quadratic penalty pushes unpaired embeddings toward orthogonality, without requiring a learned discriminator or an InfoNCE softmax.

2.2.0.1 Alignment, Uniformity, and Downstream Separability.

Empirically, achieving strong intra-class “alignment” (similar samples are close in the latent space) and cross-class “uniformity” (dissimilar samples are spread out across the latent space, often on the hypersphere) often correlates with improved linear separability on downstream tasks (see, e.g., [20]; under additional multi-view redundancy assumptions, their Theorem 3 shows that linear functions of suitable embeddings can approximate the Bayes-optimal predictor). Uniformity on the hypersphere is particularly valuable in a fixed-cosine similarity space because random embeddings then have a near-zero expected dot product. This near-zero baseline for random dissimilar pairs provides a natural separation from concentrated positive pairs that exhibit high similarity. Here, we enforce alignment and uniformity via fixed similarity-based objectives at the graph level (SCL-style alignment/uniformity), while at the node level alignment is induced by reconstruction in the feature-embedding subspace, cross-view consistency is encouraged in the complementary contextual slice via matched-node correspondence across augmentations, and uniformity is encouraged by a squared-cosine repulsion term. This design shifts the representational burden primarily onto the encoder, which must learn node embeddings whose reconstruction-relevant slice preserves local feature similarity and whose contextual slice remains stable across views while still spreading unrelated samples apart, all without requiring a learned discriminator or the InfoNCE objective itself. In practice, the resulting representation spaces (in particular the graph embeddings, the feature-embedding component used for reconstruction, and the contextual component used for cross-view consistency) often become easier to separate with simple downstream models, so a linear classifier or even a \(k\)-NN rule can perform competitively [21]. Recent refinements like MIM-Refiner [22] further demonstrate that intermediate-layers’ representations can be re-trained with an instance discrimination head to improve cluster coherence and alignment, thereby making the final embedding layers more suitable for low-shot classification and other downstream tasks.

2.3 From Node Embeddings to Graph Embeddings via Lipschitz Pooling↩︎

After obtaining node embeddings \(\{z_u\}_{u \in V(G)}\), many applications require distilling them into a single graph-level representation. In this subsection, we discuss how to preserve alignment in the embedding space by employing a Lipschitz pooling function.

2.3.0.1 Generic Lipschitz Pooling.

Fix a norm \(\|\cdot\|\) on \(\mathbb{R}^d\) and equip \((\mathbb{R}^d)^n\) with the product metric \(\|(z_i)-(w_i)\|_{\mathrm{sum}} := \sum_{i=1}^n \|z_i-w_i\|\). We say a function \(P:(\mathbb{R}^d)^n\to\mathbb{R}^{d_G}\) is Lipschitz if there exists a constant \(L\ge 0\) such that for all ordered collections \((z_i)_{i=1}^n,(w_i)_{i=1}^n\), \[\|P(z_1,\dots,z_n)-P(w_1,\dots,w_n)\| \;\le\; L\,\|(z_i)-(w_i)\|_{\mathrm{sum}}.\] In the context of pooling, we view a graph as a multiset of node embeddings \(Z=\{z_i\}_{i=1}^n\subset\mathbb{R}^d\), where node order is arbitrary.

Remark. In this section, \(P\) denotes a generic pooling map. Our model uses a (possibly learnable) pooling operator \(P_\phi\) as defined in §2.1, and all results apply to \(P_\phi\) whenever it satisfies the stated assumptions (permutation invariance and Lipschitz continuity).

2.3.0.2 Permutation invariance and matching distance.

In our setting, the pooling operator acts on the (orderless) multiset of node embeddings produced by the encoder, so we assume \(P\) is permutation-invariant: \(P(z_1,\dots,z_n)=P(z_{\xi(1)},\dots,z_{\xi(n)})\) for any permutation \(\xi\in S_n\). Accordingly, when \(P\) is permutation-invariant we write \(P(Z)\) to denote the value of \(P\) on any ordering of the multiset \(Z\). To compare two graphs without a predefined node correspondence, we use the optimal matching distance \[d_{\mathrm{match}}(Z,Z')\;:=\;\min_{\xi\in S_n}\sum_{i=1}^n\|z_i-z'_{\xi(i)}\|, \qquad Z=\{z_i\}_{i=1}^n,\; Z'=\{z'_i\}_{i=1}^n.\] (When graph sizes differ, as happens under node-dropping or subgraph augmentations, one can pad the smaller set with a fixed null embedding or use a partial-matching variant; in the main text we use Proposition 2 as an idealized equal-cardinality stability result rather than as a full theorem for every augmentation regime.)

The proof is given in Appendix 6.4. We emphasize that Proposition 2 is stated for equal-cardinality multisets; when augmentations change graph size, we use it as the natural equal-cardinality stability result, while practical implementations may instead rely on padding or partial-matching variants as noted above.

2.3.0.3 Examples of Lipschitz Pooling: Sum, Mean, and Max.

Many standard pooling operators, such as sum, mean, and coordinate‐wise max, are Lipschitz transformations. Informally, each uses a linear or piecewise‐linear aggregation over the embeddings, ensuring bounded output changes when inputs are perturbed slightly. Moreover, since these operators are permutation-invariant, their Lipschitz bounds on ordered tuples immediately imply stability for unordered multisets under the optimal matching distance \(d_{\mathrm{match}}\) via Proposition 2 (with \(d_{\mathrm{match}}\) defined using the same underlying norm as in the input metric).

The detailed proof is given in Appendix 6.5. The essence is a triangle‐inequality argument. While these operators provide basic Lipschitz aggregation, more expressive mechanisms like Softmax Attention Pooling (detailed in §2.4) can offer enhanced representational power while maintaining controlled Lipschitz properties.

2.4 Adaptive Graph Pooling with Softmax Attention↩︎

While the pooling operators discussed in §2.3 (sum, mean, max) offer simplicity and clear Lipschitz bounds, more adaptive mechanisms can provide enhanced flexibility and empirical performance in generating graph-level embeddings, in the spirit of learned set/graph aggregation methods such as Set Transformer and Graph Multiset Transformer pooling [23], [24]. This is particularly relevant for graphs with heterogeneous node densities or complex relational structures where uniform weighting of nodes may be suboptimal. Motivated by these learned aggregation methods, we use Softmax Attention Pooling as a learnable, context-aware aggregation mechanism: \[P_{\mathrm{attn}}(\{z_i\}_{i=1}^n) = W_{pool} \left( \sum_{i=1}^n \alpha_i(\{z_k\}_{k=1}^n)\, z_i \right),\] where \(\{z_i\}\) are the node embeddings, \(W_{pool} \in \mathbb{R}^{d_G \times d}\) is a trainable projection matrix, and \(\alpha_i \ge 0\) (with \(\sum_i \alpha_i = 1\)) are attention weights. These weights \(\alpha_i\) dynamically depend on the full set of node embeddings \(\{z_k\}_{k=1}^n\), often computed via a softmax over scores derived from small neural networks (e.g., MLPs) applied to node embeddings.

For \(P_{\mathrm{attn}}\) to serve as a robust Lipschitz pooling operator within the MABLE framework, we state the guarantee on bounded subsets of the node-embedding space rather than globally on all of \((\mathbb{R}^d)^n\). In our implementation, the attention-score network is fed unit-normalized inputs \[\bar z_i := \frac{z_i}{\max(\|z_i\|_2,\varepsilon)},\] its linear maps are constrained only from above via spectral-norm clamping, and the pre-softmax logit gaps are clipped in a shift-invariant way: \[\tilde{s}_i := \psi(\bar z_i),\qquad s_i := \mathrm{clip}\!\bigl(\tilde{s}_i - \max_j \tilde{s}_j,\,-c,\,0\bigr), \qquad \alpha_i := \mathrm{softmax}(s)_i.\] Assume additionally that the pooled values lie in a bounded region, \(\|z_i\|_2 \le B\) for all \(i\) in the domain of interest, that the normalization uses a fixed \(\varepsilon>0\), and that the score network \(\psi\) is composed of standard Lipschitz activations together with linear maps whose operator norms are clamped from above. Because the score network acts on normalized inputs, its Lipschitz constant is controlled by the product of the clamped operator norms of its linear layers, while clipping places the logits in a compact set on which softmax is Lipschitz. Under these conditions, the attention aggregation map \[A(\{z_k\}_{k=1}^n)\;:=\;\sum_{i=1}^n \alpha_i(\{z_k\}_{k=1}^n)\, z_i\] admits a Lipschitz constant on the bounded domain of interest, with respect to the chosen product metric on \((\mathbb{R}^d)^n\) (e.g., \(\|\cdot\|_{\mathrm{sum}}\)), depending on the value bound \(B\), the logit-gap bound \(c\), the set size \(n\), and the score network \(\psi\). Consequently, if the final linear projection satisfies \(\sigma_{\max}(W_{pool})\le L_W\), then the composed pooling operator \(P_{\mathrm{attn}}:=W_{pool}\circ A\) is also Lipschitz on that bounded domain. Note that because we do not normalize the pooled values \(z_i\) themselves, this is a bounded-domain (local) Lipschitz statement, not a global one over unbounded node embeddings.

The proof is a direct composition argument and is given in Appendix 6.6.

Beyond this upper-Lipschitz/stability guarantee, one may additionally ask whether the pooling head preserves some pairwise separation. In our implementation, the final pooling projection \(W_{pool}\) is explicitly conditioned by clamping its singular values to a band \([m,L]\) in the regime where the projection is square or full-column-rank, while the score-network matrices used to produce the attention logits are constrained only from above. Accordingly, the corollary below isolates a direct consequence relevant in our setting: once the attention-weighted sum has achieved a non-zero separation for a given pair, the lower singular-value bound on \(W_{pool}\) helps ensure that this separation is not collapsed by the final projection, though this does not by itself yield a general lower bound with respect to a standard set metric.

This does not provide a uniform lower bound in terms of a standard set metric; it states only that any separation realized by the attention-weighted sum is not collapsed by the lower-bounded final projection. In our implementation this lower-bound condition is enforced on \(W_{pool}\), whereas the attention scoring network itself is controlled only through input normalization, upper spectral-norm constraints, and logit clipping (see Appendix 6.7).

2.5 Condition Number Control for Linear Decoders↩︎

While a general Lipschitz or bi-Lipschitz decoder can be implemented via architectural constraints, which often introduce complexity or complicate training [25]–[28], a particularly simple yet powerful approach uses a single linear decoding layer, \(g(e) = W_{dec}e + b\), where \(W_{dec} \in \mathbb{R}^{d_x \times d_e}\). For such a linear map from feature-embedding space (\(\mathbb{R}^{d_e}\)) to feature space (\(\mathbb{R}^{d_x}\)), the maximum singular value \(\sigma_{\max}(W_{dec})=\|W_{dec}\|_2\) dictates its Lipschitz constant (bounding expansion), and the minimum singular value \(\sigma_{\min}(W_{dec})\) determines its co-Lipschitz constant (bounding collapse). Specifically, a non-zero \(\sigma_{\min}(W_{dec}) > 0\) implies injectivity for the linear map (which in particular requires \(d_e \le d_x\)), preventing distinct latent vectors from collapsing to the same output. By ensuring both \(0 < m \le \sigma_{\min}(W_{dec})\) and \(\sigma_{\max}(W_{dec}) \le L < \infty\) (which in particular requires \(d_e \le d_x\)), the linear decoder becomes bi-Lipschitz, guaranteeing that distances are preserved up to bounded factors: \(m\|e_1 - e_2\| \le \|g(e_1) - g(e_2)\| \le L\|e_1 - e_2\|\).

Controlling these singular values in practice adds computational cost. Calculating the full Singular Value Decomposition (SVD) of the \(d_x \times d_e\) matrix \(W_{dec}\) is typically \(O(\min(d_x,d_e) \cdot d_x d_e)\), which can be expensive, especially if performed at every training step for large matrices. However, practical methods exist to approximate or constrain singular values without full SVD. Controlling \(\sigma_{\max}(W_{dec})\), the spectral norm, is relatively efficient; techniques like the Power Method are used in Spectral Normalization [25] to approximate this value iteratively. Controlling \(\sigma_{\min}(W_{dec})\) or the condition number \(\kappa(W_{dec}) = \sigma_{\max}(W_{dec})/\sigma_{\min}(W_{dec})\) is generally more challenging. Nevertheless, for decoder matrices, whose dimensions \(d_x\) and \(d_e\) are often manageable (hundreds or low thousands) compared to other parts of a model, the computational cost may be acceptable or can be mitigated by techniques like singular value clamping or projection-based updates. This approach ensures that the primary representational learning burden resides with the encoder \(f(\cdot)\), while the well-conditioned linear decoder \(W_{dec}\) serves as a predictable mapping that prevents trivial inversions or representational collapse in the output layer.

2.6 Hypothesis: Output Constraints and Encoder Regularity↩︎

Explicitly imposing Lipschitz or bi-Lipschitz constraints layer-by-layer throughout a deep neural network is computationally expensive, often requiring per-layer spectral norm calculations [25], [26]. We treat output-side conditioning as a practical design hypothesis: constraining the feature decoder (§2.5) and using a Lipschitz-controlled graph pooling operation (§2.3, §2.4) may encourage more stable encoder outputs without requiring layer-wise constraints throughout the network.

This is not a formal guarantee on the encoder. Rather, it motivates the architecture: bounded decoder and pooling stages make the final representation-to-loss map less sensitive, while leaving the encoder expressive enough to learn the data-dependent interactions required by the task. Directly measuring how such output-side constraints affect encoder spectra and optimization dynamics is left for future work.

2.7 Discussion and Practical Considerations↩︎

The choice of pooling operator itself presents several options, each with distinct characteristics. Foundational Lipschitz operators such as mean, sum, and max pooling (detailed in §2.3) offer a starting point. Empirically, mean pooling is simple and stable, and it avoids the scaling issues of sum pooling when \(|V(G)|\) varies widely. Sum pooling precisely captures node multiplicities (beneficial if the number of nodes is meaningful), but can become unbounded with large graphs. Max pooling discards duplicates, effectively capturing the “set” of embeddings. Each of these simpler methods has advantages depending on how one wishes to represent the entire graph and whether exact cardinality or distributional cues are prioritized. See [29] for a deeper discussion comparing sum, mean, and max aggregator behaviors in Graph Neural Networks. In MABLE, reconstruction constrains the low-dimensional feature embedding \(e_u=\pi(z_u)\), while pooling operates on the full node embedding \(z_u\); the complementary contextual slice can therefore contribute to graph-level representations while also being regularized by cross-view matched-node correspondence.

In summary, while these simpler Lipschitz operators (mean, sum, or max) provide robust baseline aggregation, more adaptive mechanisms such as Softmax Attention Pooling (§2.4) often yield superior flexibility and empirical performance, especially for graphs with complex or heterogeneous structures. Softmax Attention Pooling, as discussed, can be constructed to maintain overall Lipschitz continuity on bounded embedding domains when the score network uses normalized inputs, upper spectral-norm control, and clipped logits. In our implementation, the final pooling projection is additionally conditioned with both lower and upper singular-value bounds, yielding the partial co-Lipschitz guarantee discussed above.

Furthermore, practical implementations often involve choices about processing the pooled vector before its use in downstream tasks or loss functions. For instance, when angle-based similarity measures are used (such as cosine similarity \(\cos(\cdot,\cdot)\) employed in \(\mathcal{L}_{\mathrm{graph\_align}}\) §3.2, or normalized SCL variants), the final loss signal primarily depends on embedding direction, not magnitude. However, the magnitude of the pooled vectors can still affect numerical stability and optimization dynamics. Applying a bounded activation function like element-wise \(\tanh\) directly after the primary aggregation step (e.g., \(s_G = \tanh\!\bigl(P_\phi(\{z_u\}_{u\in V(G)})\bigr)\in\mathbb{R}^{d_G}\)) can mitigate these issues by constraining the components of the pre-normalized vectors to the range \((-1, 1)\). This ensures inputs to the normalization step are well-scaled, potentially improving stability and removing an unconstrained degree of freedom from the optimization. While introducing saturation (which might compress informative large magnitudes), this approach provides component-wise regularization, maintains the overall Lipschitz nature of the graph embedding function (as \(\tanh\) is 1-Lipschitz), and produces outputs more readily adaptable for efficient similarity search via binarization techniques. As such, it represents a viable practical strategy within the MABLE toolkit, complementing the choice of pooling operator and decoder constraints.

The considerations for condition-number control on a linear feature decoder \(g(e)=W_{dec}e+b\) (§2.5) also involve a conceptual balancing act when setting bounds for its singular values: the minimum \(m = \sigma_{\min}(W_{dec})\) and maximum \(L = \sigma_{\max}(W_{dec})\). The objective is a decoder that is robustly bi-Lipschitz (requiring \(m > 0\) and a finite \(L\)) while remaining sufficiently expressive for accurate feature reconstruction. In practical terms, bounding \(L\) prevents the decoder from hallucinating large feature variances from small latent perturbations, while enforcing a strictly positive \(m\) prevents the decoder from ignoring the latent space entirely (a failure mode akin to posterior collapse). Smaller values for \(L\), particularly approaching \(1.0\), are theoretically appealing as they can enhance training stability by ensuring the decoder is non-expansive or only mildly expansive, fostering more regularized mappings. However, an overly restrictive \(L\) might hypothetically curtail the decoder’s ability to reconstruct features if significant scaling or transformation from the latent space is necessary, suggesting a need for adequate "slack" or expressiveness. Simultaneously, \(m\) must be strictly positive to uphold the bi-Lipschitz property, ensuring distinct latent states can lead to separable reconstructions. The ratio \(L/m\) (the condition number) dictates the decoder’s allowance for anisotropic distortion; a larger condition number implies more "slack" but less uniform geometric preservation, whereas a smaller condition number (larger \(m\) relative to \(L\)) makes the decoder more rigid, strengthening the geometric coupling outlined in Proposition 1. Thus, the selection of \(m\) and \(L\) involves a hypothetical trade-off between decoder flexibility for reconstruction, the stability afforded by tighter singular value control, and the desired strength of the geometric link between the latent and feature spaces, with optimal ranges likely being sensitive to specific dataset characteristics and modeling goals.

3 End-to-End Training Objectives for Node & Graph Embeddings↩︎

We now present the end-to-end objectives for learning both node-level and graph-level representations. By training end-to-end, the model enforces similarity among similar nodes, encourages consistency of matched nodes across augmented views, aggregates those nodes into a graph-level embedding, and encourages graph-level uniformity among unpaired embeddings. Our system uses in-batch sampling for node embeddings and leverages augmented/un-augmented pairs at the graph level.

Drawing inspiration from fixed-cosine surrogate principles (alignment, uniformity, avoiding learned discriminators and the InfoNCE objective itself), our framework employs loss functions instantiated differently across scales. At the node level, we combine a masked reconstruction loss with a bi-Lipschitz feature decoder to promote alignment in the reconstruction-relevant feature slice, an explicit cross-view node-correspondence loss to promote consistency in the complementary contextual slice, and a node-level uniformity penalty. At the graph level, we enforce both alignment of augmented graph views and uniformity among unpaired graph embeddings using penalties analogous to those derived for the fixed-cosine surrogate. This design provides a scalable, geometrically interpretable implementation of contrastive principles, adapted to masked graph data.

3.1 Node Alignment, Node Correspondence, and Node Uniformity↩︎

We start with a masked reconstruction objective that encourages the learned representation to capture meaningful dependencies within and across attributes, and to align similar nodes in the reconstruction-relevant embedding space. This loss penalizes the model’s ability to accurately reconstruct the original or masked features of a node from its latent embedding: \[\mathcal{L}_{\mathrm{node\_align}} = \mathbb{E}_{G\sim\mathcal{D}} \;\mathbb{E}_{T\sim\mathcal{A}} \;\mathbb{E}_{v\sim V(G^{(a)})} \Bigl[\ell_{\mathrm{recon}}\bigl(g(\pi(f_\theta(G^{(a)}, X^{(a)})_v)), x_{\iota^{(a)}(v)}; T\bigr)\Bigr],\]

where \(\mathcal{D}\) is the training distribution over graphs, \(T\sim\mathcal{A}\) is the view operator from §2.1, and \((G^{(a)},X^{(a)},\iota^{(a)})=T(G,X)\). The encoder \(f_\theta\) consumes the augmented input \((G^{(a)},X^{(a)})\) and outputs a contextualized embedding for each retained node \(v\in V(G^{(a)})\), while \(x_{\iota^{(a)}(v)}\) is the original reconstruction target associated with that retained node identity. The slice \(\pi\) selects the feature-embedding subspace used for reconstruction and \(g\) is the (bi-Lipschitz) feature decoder (§2.5). The reconstruction loss \(\ell_{\mathrm{recon}}(\cdot,\cdot;T)\) is a norm-based penalty in feature space and may depend on the view mask (e.g., evaluated only on masked components, only on unmasked components, or on both with feature-type weights).

Masking encourages the encoder to learn feature embeddings that support consistent feature prediction from context for nodes with similar local structure or attributes, thereby promoting grouping in the reconstruction-relevant feature-embedding space. Importantly, due to the bi-Lipschitz nature of \(g\) (see Proposition 1), minimizing reconstruction error acts as a geometric regularizer in the learned feature-embedding space. If two nodes have similar reconstruction targets and both are reconstructed accurately, then their feature-embedding slices are forced to be close up to the reconstruction-error slack in Proposition 1. Without the lower-Lipschitz component of the decoder constraint, accurate reconstruction alone would not rule out arbitrarily arranged latent codes.

3.1.0.1 Cross-View Node Correspondence in the Contextual Slice.

Reconstruction constrains only the feature slice \(e_u=\pi(z_u)\). To encourage the remaining contextual slice \(c_u=\rho(z_u)\) to encode view-stable information rather than augmentation-specific variation, we additionally align matched nodes across two augmented views of the same graph, in the spirit of bootstrap-style cross-view consistency methods such as BGRL [15]. Let \((G^{(a)},X^{(a)},\iota^{(a)})=T_a(G,X)\) for \(a\in\{1,2\}\) denote two masked/augmented views of the same underlying graph \(G\). We define the common retained identity set \[\mathcal{C}(G,T_1,T_2) := \iota^{(1)}(V(G^{(1)}))\cap \iota^{(2)}(V(G^{(2)})) \subseteq V(G).\] For \(u\in\mathcal{C}(G,T_1,T_2)\), write \(v_a(u):=(\iota^{(a)})^{-1}(u)\) for the corresponding node in view \(a\).⁴ We define \[\mathcal{L}_{\mathrm{node\_corr}} \;=\; \mathbb{E}_{G\sim\mathcal{D}} \;\mathbb{E}_{T_1,T_2\sim\mathcal{A}} \;\mathbb{E}_{u\sim \mathcal{C}(G,T_1,T_2)} \Bigl[ \ell_{\mathrm{corr}}\!\bigl( \rho(f_\theta(G^{(1)},X^{(1)})_{v_1(u)}),\, \rho(f_\theta(G^{(2)},X^{(2)})_{v_2(u)}) \bigr) \Bigr],\] where \(\ell_{\mathrm{corr}}\) is a similarity-matching loss on the contextual slice (e.g., squared \(\ell_2\) distance between normalized contextual vectors, or equivalently a cosine-alignment penalty up to constants). This term encourages matched nodes across augmentations to remain close in the contextual subspace while leaving the reconstruction-relevant feature slice to be shaped primarily by the masked autoencoding objective.

3.1.0.2 In-Batch Node Sampling for Node Uniformity.

We then promote uniformity among node embeddings by randomly pairing node instances within the same batch and penalizing the squared cosine similarity between embeddings of unpaired nodes. This term is analogous to the negative term in the Spectral Contrastive Loss (SCL) and the log-based surrogate, which encourages orthogonality among negative pairs: \[\mathcal{L}_{\mathrm{node\_uniform}} \;=\; \mathbb{E}_{\eta,\eta'\sim \mathcal{Z}_{\mathrm{node}},\;\eta\neq \eta'} \Bigl[ \cos(\eta,\eta')^2 \Bigr].\] Here \(\mathcal{Z}_{\mathrm{node}}\) denotes the current minibatch pool of valid retained node embeddings, distinct from the node-instance set \(\mathcal{B}_{\mathrm{node}}\) defined above. In our implementation, \(\mathcal{Z}_{\mathrm{node}}\) contains token embeddings from both augmented views of all graphs in the minibatch, sampled uniformly from this pooled token set. Exact cross-view matches of the same retained node identity are rare under this uniform sampling in large token pools, so we use this term as a lightweight isotropy/decorrelation regularizer rather than as a strict semantic-negative objective. This objective penalizes large absolute dot products between sampled token embeddings, pushing their cosine similarity towards orthogonality (zero).

3.2 Graph-Level Pooling and Positive Pair Augmentation↩︎

Next, we consider the graph-level embedding \(s_G\). For a given view \((G^{(a)},X^{(a)},\iota^{(a)})\), this embedding is typically generated using the Adaptive Graph Pooling with Softmax Attention mechanism (\(P_{\mathrm{attn}}\)), as detailed in §2.4, applied to the set of retained node embeddings: \[s_G^{(a)} = P_{\mathrm{attn}}\bigl(\{f_\theta(G^{(a)},X^{(a)})_v:\, v \in V(G^{(a)})\}\bigr),\] yielding a single vector per graph view. This adaptive mechanism is often favored for its flexibility in handling varied graph structures and for its potential empirical performance benefits. While simpler Lipschitz operators like \(\mathrm{MeanPool}\) (cf. §2.3) can serve as alternatives or for comparative studies, any chosen Lipschitz pooling operator helps control sensitivity to changes in graph size or node embedding scale. However, pooling alone is insufficient to guarantee invariance to structured perturbations such as node masking, feature noise, or irregular subgraph sampling. The role of positive pair augmentation is thus critical: by explicitly training the encoder and pooling to align perturbed views of the same graph, the model learns to ignore incidental variations and focus on stable, semantically meaningful patterns. We create positive graph-view pairs by sampling \(T_1,T_2\sim\mathcal{A}\) from the same underlying \((G,X)\), e.g.:

1. Augmented copy: a view might keep the same node set but apply partial node masking, node dropping, or slight feature corruption.

2. Subgraph sampling: If \(G\) is large, we can sample subgraphs (e.g., via node dropping) to form augmented views, treating the embedding of a subgraph as a positive view of the entire graph, provided they remain semantically related.

Using a penalty based on cosine similarity (for nonzero vectors \(a,b\), writing \(\hat{a} := a/\|a\|_2\) and \(\hat{b} := b/\|b\|_2\), so that the usual identity \(1-\cos(a,b)=\tfrac12\|\hat{a}-\hat{b}\|_2^2\) holds), we define the inter-graph alignment loss: \[\mathcal{L}_{\mathrm{graph\_align}} = \mathbb{E}_{G\sim\mathcal{D}}\;\mathbb{E}_{T_1,T_2\sim\mathcal{A}}\left[ 1 - \cos\!\bigl(s_G^{(1)}, s_G^{(2)}\bigr) \right].\] Here \(s_G^{(a)}\) is computed from the view \((G^{(a)},X^{(a)},\iota^{(a)})=T_a(G,X)\) as above.

Minimizing this loss is equivalent to maximizing the cosine similarity between the two view embeddings. When one view is a subgraph of the other (a possible outcome of our augmentation strategy, as previously described), this alignment objective specifically encourages the larger-view embedding to be predictive of the representation of its own local (subgraph) structure. Such an alignment between global and local views of the same data instance is a core tenet of mutual information maximization strategies like Deep InfoMax [11], [12]. This alignment is particularly important in geospatial and geological datasets, where differences in node count or spatial density often reflect sampling artifacts rather than true semantic divergence. Together, the pooling and augmentation strategy is designed to improve robustness to cardinality mismatches, topological edits, and local feature inconsistencies.

3.3 Graph Uniformity↩︎

Lastly, to promote inter-graph uniformity, for an anchor graph identity \(G\), a view embedding \(s_G^{(a)}\) is pushed away from embeddings of views of other underlying graphs sampled from the current batch. Concretely, for a minibatch \(\mathcal{B}_{\mathrm{graph}}\) of graphs and their augmented views, the anchor identity is the underlying graph \(G\); its own views include all embeddings \(s_G^{(a)}\) derived from that same graph. We take \(\mathrm{NegPool}(G)\) to be the empirical distribution over all graph-level view embeddings in the batch excluding those views of \(G\) itself. In particular, it may comprise full-graph or subgraph view embeddings derived from other graphs \(H\ne G\). Analogous to the SCL negative term, we penalize the squared cosine similarity: \[\mathcal{L}_{\mathrm{graph\_uniform}} = \mathbb{E}_{G,\,T_a,\, Z^- \in \mathrm{NegPool}(G)}\bigl[ \cos\!\bigl(s_G^{(a)}, Z^-\bigr)^2 \bigr].\] Here, for each anchor graph \(G\), \(s_G^{(a)}\) is computed from a sampled anchor view \(T_a(G,X)\), and \(Z^-\) is sampled from the empirical distribution that is uniform over elements of \(\mathrm{NegPool}(G)\). The importance of such explicit uniformity-promoting terms for enhancing feature diversity and preventing issues like representation collapse has also been highlighted in masked autoencoder frameworks such as U-MAE [4]. In practice, these unpaired in-batch graphs are a heuristic source of contrast rather than guaranteed semantic negatives. When \(Z^-\) includes subgraph view embeddings from another graph \(H\), this objective contributes to making views of \(G\) discriminable from local views of other graphs, complementing the positive local-global alignment encouraged by \(\mathcal{L}_{\mathrm{graph\_align}}\).

3.4 Combined Objective↩︎

Putting it all together, we train end-to-end with a single combined objective: \[\mathcal{L} \;=\; \alpha\,\mathcal{L}_{\mathrm{node\_align}} \;+\; \beta\,\mathcal{L}_{\mathrm{node\_corr}} \;+\; \gamma\,\mathcal{L}_{\mathrm{node\_uniform}} \;+\; \lambda\,\mathcal{L}_{\mathrm{graph\_align}} \;+\; \mu\,\mathcal{L}_{\mathrm{graph\_uniform}}.\]

By tuning the above scalars, we balance local (node) alignment via reconstruction, cross-view node correspondence in the contextual slice, node-level uniformity, graph-level augmentation alignment, and graph uniformity across the dataset.

3.5 Holistic Benefits↩︎

This fully integrated pipeline yields robust node and graph embeddings with the following advantages:

• Principled Surrogates: By using fixed-cosine based penalties analogous to normalized SCL, we avoid learned discriminators and their associated stability and bias issues. The normalized/cosine SCL-style surrogate also admits a heuristic second-order connection to a DV-style mutual-information objective under boundedness, approximate symmetry, and small-variance assumptions (6.3).

• Local Quality: Masked node autoencoding plus bi-Lipschitz decoding encourages the feature-embedding slice used for reconstruction to reflect local similarity: similar node attributes lead to nearby feature embeddings up to the reconstruction-error slack in Proposition 1. A complementary node-correspondence loss encourages the remaining contextual slice to stay consistent across matched nodes in augmented views. The node uniformity penalty shapes the full node embeddings by encouraging unpaired in-batch node embeddings to stay near-orthogonal. Together, these terms encourage node representations that are locally faithful, cross-view stable, and well spread.

• Reliable Graph Embeddings: The use of Lipschitz-controlled pooling mechanisms, particularly adaptive Softmax Attention Pooling (§2.4) as well as simpler operators like mean or sum (cf. 3), ensures that small changes in node embeddings lead to proportionally bounded changes in \(s_G\). Furthermore, the graph alignment penalty (\(\mathcal{L}_{\mathrm{graph\_align}}\)) contributes to the robustness of these embeddings against augmentation variations.

• Global Uniformity: By drawing data from the same batch or dataset, both at node and graph level, the uniformity penalties encourage a well-spread, approximately isotropic embedding distribution across the collection of graphs, complementing local alignment and helping prevent representation collapse.

• Implicit Deep InfoMax-Style Learning: By leveraging subgraph augmentations for positive pairs in \(\mathcal{L}_{\mathrm{graph\_align}}\) (aligning two views of the same underlying graph, potentially including full-graph and local-subgraph views) and employing a broad in-batch contrastive strategy for \(\mathcal{L}_{\mathrm{graph\_uniform}}\) (contrasting views of one graph against full-graph or subgraph views of other graphs), MABLE encourages a learning dynamic similar to Deep InfoMax. It learns global representations that are predictive of their own local structures while being distinct from the local structures of other graphs, all achieved without a learned discriminator due to the SCL-based fixed-cosine surrogates.

• End-to-End Efficiency: Because node- and graph-level objectives are co-trained, we do not need separate staging or complex hard-negative selection steps. In-batch sampling, straightforward augmentation, and sampled matched-node correspondence provide the necessary positive and regularizing signals within a single training loop.

• Output-Side Stability: Constraining the bi-Lipschitz decoder and Lipschitz pooling layers provides controlled final-stage mappings from embeddings to reconstruction and graph-level objectives, while leaving encoder-level regularity as a design hypothesis rather than a formal guarantee.

• Conditioned Decoder Control: If \(g\) is a single linear layer with constrained singular values (§2.5), then the final reconstruction map is protected against trivial collapse or unbounded expansion, reducing the risk of poorly conditioned latent-to-feature mappings.

Overall, this holistic design encourages similarity, correspondence, and uniformity at both the node and graph levels in one end-to-end pass. Crucially, by leveraging principled fixed-cosine surrogates, simple augmentations, in-batch sampling, sampled matched-node correspondence, and a well-conditioned decoder, the burden of representation is shifted to the encoder, which must learn meaningful and robust features. This principle is strongly supported by findings like MIM-Refiner [22], which show that lightweight decoders lead to more discriminative intermediate representations, ultimately improving performance in downstream tasks.

4 Experiments↩︎

We evaluate MABLE at two spatial scales (prospect and regional; details deferred to subsequent subsections). Here we first describe the shared backbone and training procedure used across both experiments, including masking, augmentation, and the instantiation of the losses introduced in §3.

4.1 GeoReformer backbone, tokenization, and training procedure↩︎

4.1.0.1 Backbone (GeoReformer).

All experiments use a Reformer-style encoder [30] with LSH self-attention, adapted to geospatial node sets and referred to as the GeoReformer. GeoReformer is the encoder instantiation used in our experiments; the MABLE objectives only require an encoder that maps a graph or node collection to node embeddings, so other GNN, graph-transformer, or set-transformer backbones could be substituted. Each graph is encoded as a padded sequence of node tokens and processed by a stack of LSH attention blocks to support thousands of nodes per sample. We remove sequence/axial positional encodings because node order is arbitrary.

4.1.0.2 Design rationale: edge-free graphs via induced connectivity.

We represent each sample as an edge-free set of node observations and avoid constructing an explicit adjacency matrix. In mineral prospectivity settings, heuristic edges (e.g., \(k\)-NN or radius graphs) can inject strong, dataset-specific priors and become brittle under heterogeneous sampling densities and multi-sensor geometry [2], [31], [32]. Instead, we let the model induce an example-specific sparse connectivity pattern through its attention mechanism: the Reformer-style LSH routing performs a hard partition of tokens into candidate neighborhoods, yielding a block-sparse attention graph that scales to tens of thousands of nodes while remaining agnostic to arbitrary node ordering. This shifts the burden of discovering useful spatial interactions from a hand-crafted graph construction rule to learned attention over data-dependent neighborhoods.

4.1.0.3 Node tokenization and input projections.

Each node token concatenates (i) a feature-type identifier \(\tau_u\in\{1,\dots,K\}\) (stored as a one-hot vector over \(K\) feature IDs, and optionally mapped to a fixed unit-norm code vector \(q(\tau_u)\in\mathbb{R}^{K}\) for training stability), (ii) an optional scalar feature value \(r_u\) (or a sentinel when not applicable), and (iii) relative geometric features (e.g., coordinates expressed relative to the graph center, with dataset-specific scaling). The resulting encoder token augments the reconstructible feature vector with relative geometric information. Concretely, the reconstructible feature vector \(x_u\) comprises the feature-type identifier together with the scalar value when present, while relative geometry is provided to the encoder as auxiliary positional/context information concatenated to this feature vector. The full token is then projected into the model hidden dimension \(d\) by a learned linear embedder, \(t_u = E(\cdot) \in \mathbb{R}^{d}\), providing a minimal and modality-agnostic input interface for heterogeneous geospatial observations.

4.1.0.4 Relative geometry only; no absolute spatial identity.

To avoid trivial location-based shortcuts and to improve transfer across regions, we do not inject absolute or global position embeddings into the sequence. Geometric information enters only through relative positional features and through augmentation, which preserves the local spatial relationships while discouraging memorization of absolute coordinates.

4.1.0.5 LSH self-attention and routing regularization.

GeoReformer uses LSH self-attention to approximate full self-attention with sub-quadratic cost. Concretely, tokens are assigned to buckets by hashing projected representations; these bucket assignments define the support of the attention computation (i.e., which token pairs are eligible to interact) and thus act as a hard, example-specific sparsification of the attention adjacency. During training, we regularize this routing by sampling bucket assignments from a temperature-scaled distribution over the hash projection scores, rather than using a deterministic argmax. This stochastic routing reduces brittleness of the induced block structure and improves robustness. At evaluation time, routing reverts to deterministic argmax assignment, recovering the standard Reformer behavior.

4.1.0.6 Masking with feature-type specific mask tokens.

We use a feature-type aware masking scheme with a dedicated learned mask token (a trainable replacement vector) per feature type. When a node is masked, we keep its relative geometric features unchanged but replace both the feature-ID block and the scalar value (when present) with the corresponding mask token. In this experimental tokenization, the model is trained to reconstruct the original feature-type identifier (represented either as a one-hot vector or its fixed code \(q(\tau_u)\)) together with the scalar value for feature types that admit one; MABLE itself only requires a reconstructible target appropriate to the domain. For feature types without an associated scalar value, the scalar reconstruction term is omitted. Because the mask token is feature-type specific by design, the nontrivial learning signal comes primarily from reconstructing masked values/content from cross-feature context and relative spatial structure, rather than from inferring the feature type itself.

4.1.0.7 Two-view augmentation (mask-only vs drop–rotate–mask).

Training forms positive view pairs from the same underlying graph. The original view applies node masking only. The positive view combines (i) stochastic node dropping, (ii) random Z-axis rotations around a local reference point (e.g., the graph center), and (iii) masking. Dropping is applied both globally and in spatial patches: with some probability we sample a patch (random center and radius) and remove nodes within that region. Patch dropping can be uniform (the entire patch is dropped) or feature-conditional, where each feature type is independently retained or removed with a sampled probability, effectively simulating missing modalities or incomplete acquisition within localized areas. A minimum-retained-node constraint avoids degenerate views under aggressive dropping. Rotation augmentation is used specifically to encourage the learned representations, especially graph-level embeddings, to be insensitive to local orientation, i.e., to favor rotational invariance in the embedding space. This is desirable in the mineral-exploration setting considered here, where local patch orientation is typically not itself the quantity of interest, whereas relative spatial configuration and cross-modal context are.

4.1.0.8 Decoder conditioning and bounded attention scores.

We use a bi-Lipschitz feature decoder for reconstruction by clamping the singular values of the decoder matrix to \([\sigma_{\min}, \sigma_{\max}]\). For graph pooling, we use standard softmax attention: the attention MLP receives unit-normalized node embeddings, its internal linear maps are clamped only in maximum singular value, and the pre-softmax logits are shifted by their within-graph maximum and clipped to \([-c,0]\), thereby bounding relative logit gaps in a shift-invariant way. The final pooling projection \(W_{pool}\) is separately conditioned by clamping its singular values to \([m,L]\). Thus, the score network is controlled for upper-Lipschitz stability, while the output projection additionally provides the partial co-Lipschitz guarantee stated in §2.4.

4.1.0.9 Losses and optimization.

The training objective combines reconstruction with representation-shaping terms: node-level correspondence and uniformity, graph-level alignment between two views of the same underlying graph, and graph-level uniformity within a batch. Throughout, the decoder is trained to predict the full node reconstruction target \(x_u\).⁵ Masking is applied to the encoder input by corrupting a subset of feature dimensions/types with learned mask tokens, so that predicting the corrupted entries is inherently context-dependent and does not admit trivial copy-through shortcuts. In training we use (i) a masked reconstruction loss, evaluated on the masked entries, and (ii) an auxiliary reconstruction loss on the unmasked entries; together these form a weighted reconstruction loss on the full vector \(x_u\), with weights chosen as a task- and modality-dependent implementation choice. We use an \(\ell_2\)-style reconstruction loss (MSE) on all reconstructed components, including the feature-ID representation (one-hot or fixed code) and the scalar value when present. This keeps the reconstruction target in a common Euclidean space consistent with the geometric role of the decoder. In addition, for two augmented views of the same graph we apply a node-correspondence loss on the contextual slice of matched nodes retained in both views; for efficiency and to avoid over-constraining the representation, this term is evaluated on a random subset of valid matched nodes per graph in each minibatch. Uniformity terms use squared cosine similarity surrogates, and node-level uniformity is likewise estimated by sampling valid tokens (optionally from both views) for efficiency. Optimization uses Adam with a cosine learning-rate schedule and linear warmup; some experiment-specific hyperparameters are reported with each experimental setting.

4.1.0.10 Implementation-detail availability.

The experiments were run in a proprietary commercial environment. We report the architectural choices, data splits, embedding dimensions, downstream feature construction, and evaluation protocols needed to interpret the results, but most internal low-level implementation details and training hyperparameters are not fully disclosed here. All reported comparisons use the same internal training configuration unless an ablation explicitly states otherwise, so the experimental claims should be read as controlled comparisons within that fixed implementation rather than as a complete public reproduction recipe.

4.2 Experiment: Regional-scale Arabian Shield dataset↩︎

4.2.0.1 Study area and geological context.

Our second experiment targets the Arabian Shield, a structurally complex Neoproterozoic shield comprising diverse accreted terranes, sutures, and intrusive bodies. This setting provides a large, heterogeneous multi-modal corpus for node and graph-level pretraining at regional scale, where robustness to uneven coverage and strong geological non-stationarity are central.

4.2.0.2 Data modalities.

We assemble a multi-sensor stack over the Arabian Shield using:

• Surface geochemistry: Saudi Geological Survey (SGS) surface geochemistry.

• Optical imagery: Sentinel-2 raw optical channels (Channel 1 removed), with invalid pixels masked using the Sentinel-2 Scene Classification Layer.

• Geophysics: Total Magnetic Intensity (TMI) Reduced-to-Pole (RTP), with partial coverage over the Shield.

• Regional seismic: KSA seismic \(V_S\) data resampled to 600 m horizontal spacing, truncated to 10 km depth, and spatially filtered to the Shield footprint.

• Topography: DEM data (used for dataset collation but not used as an explicit feature).

4.2.0.3 Graph construction (random-patch sampling).

Graphs are constructed by randomly selecting a horizontal center \((y_0,x_0)\) within the dataset footprint and extracting a fixed-radius patch. Each sample is an edge-free graph whose nodes correspond to observations from the available modalities falling inside a 200 m radius disc in the horizontal plane: \[\sqrt{(y-y_0)^2 + (x-x_0)^2} \le 200~\text{m}.\] We retain each observation’s vertical coordinate \(z\) as part of the node token, but do not use \(z\) for inclusion, i.e., the receptive field is a cylinder in \((y,x)\) with full retained vertical structure. Node geometry is encoded in a local relative frame centered at \((y_0,x_0)\), consistent with the relative-geometry-only design of §4.1.

4.2.0.4 Dataset size and structure.

The resulting dataset contains approximately \(145\)k graphs and \(951\)M total nodes. Graphs contain about \(6.6\)k nodes on average (std.\(\approx 1.3\)k), ranging from roughly \(50\) to \(8.0\)k nodes per graph. Across these modalities there are 71 distinct feature types, which are tokenized using the feature-type identifier scheme described in §4.1.

4.2.0.5 Train/validation splits.

We use a 90% / 10% split into training and validation graphs for model selection. Subsequent subsections report embedding diagnostics and downstream analyses under this split.

4.2.1 Study 1: Regional Embedding Structure and Emergent Zonation Patterns↩︎

We evaluate the structure of the 256-dimensional embeddings learned by MABLE over the Arabian Shield using a complementary set of spatial, statistical, spectral, and graph-based analyses. Although the model is trained without access to absolute/global geographic coordinates or geological labels, these diagnostics indicate that the learned representation contains spatially coherent, qualitatively geologically plausible structure across multiple scales.

4.2.1.1 Dimension-Level Spatial Structure and Latent Geometry.

We begin by visualizing individual embedding dimensions geospatially (Fig. 1). Most dimensions exhibit clear spatial coherence, large contiguous patches, gradients, and sharp transitions, rather than unstructured noise. Several dimensions appear to respond to broad terrane-scale domains, while others emphasize finer lithological texture, suggesting meaningful specialization across embedding channels and a genuinely distributed representation.

A complementary geometric sanity check is the cosine-similarity distribution between 5,000 randomly sampled embedding pairs (Fig. 2). Many representation learning pipelines suffer from anisotropy (a “cone effect”), where embeddings concentrate around a dominant direction and cosine similarity becomes less discriminative. Here, the near-zero mean (\(\mu \approx 0.054\)) is consistent with low anisotropy and broad angular coverage, and there is no evident collapse in the learned embedding distribution. The pronounced positive tail, in turn, reflects dense regional domains in latent space: randomly sampled pairs occasionally fall within the same large-scale terrane or lithological regime, yielding high similarity.

4.2.1.2 Manifold Projections Reveal Regional Organization.

Low-dimensional projections provide a compact view of how the embedding space is organized at regional scale (Fig. 3). The UMAP RGB map (cosine metric, \(n_{\mathrm{neighbors}}{=}30\)) exhibits sharply bounded, spatially coherent domains, consistent with terrane-scale segmentation emerging from signal content alone (no absolute/global coordinates were used during training). In contrast, the PCA RGB map captures a broad west–east gradient but appears substantially more diffuse. This combination suggests that the embedding geometry is strongly non-linear: regional structure is present in the representation, but it is not well described by a small number of linear modes. Consistent with this, the first three principal components explain only \(\sim\)​17% of the variance, indicating that information is distributed across many embedding dimensions rather than concentrated into a few dominant axes.

4.2.1.3 Similarity Search Supports Anomalies as Structured Signals.

To test whether outlier scores reflect repeatable structure rather than preprocessing artifacts, we run a cosine-similarity retrieval over the full validation set using representative anomalous and typical query embeddings selected by Isolation Forest (global rarity) and Local Outlier Factor (local density). For each query, the similarity-score distribution is sharply concentrated near zero with a heavy right tail (high kurtosis), indicating that high-similarity neighbors are sparse and highly selective. Importantly, the spatial footprint of the top matches differs by query type: anomaly queries concentrate into a small number of compact, geographically coherent patches, whereas normal queries return broader, higher-coverage regions consistent with membership in a dense manifold region. Because ranking uses only embedding similarity (coordinates are used solely for plotting), the emergence of contiguous match regions supports the interpretation that the detected anomalies correspond to rare but repeatable, spatially structured signatures in the learned representation (Fig. 4).

4.2.1.4 Spatial Correlation and Characteristic Scale.

A global spatial correlogram computed over random embedding pairs shows a clear decay of cosine similarity with physical distance, crossing zero at about 300 km and flattening at approximately 400–600 km (Fig. 5). This provides an estimate of the characteristic correlation length of the learned representation, plausibly associated with regional geological structure. Local radial profiles further differentiate anomalies from normal points: anomalous embeddings exhibit rapid decay to near-orthogonality, while normal points retain weak correlation over longer distances (Fig. 6).

4.2.1.5 Spectral Analysis: Scale, Texture, and Anisotropy.

To quantify the spatial scales encoded by the embeddings, we analyze their frequency content in both 1D and 2D. Along a latitudinal transect (Lat \(\approx 20.30^\circ\)), the mean embedding power spectrum follows an approximate power law with slope close to \(1/f^{0.75}\) over the resolvable band (Fig. 7), consistent with multi-scale geological organization rather than white noise or an overly smoothed field. Because the transect sampling is sparse (grid spacing \(\Delta x \approx 2.05\) km), this result is band-limited by the Nyquist wavelength (\(\lambda_N \approx 2\Delta x \approx 4.1\) km) and should be interpreted as a regional-scale signature. A complementary 2D Fourier analysis over a dense local patch reveals structured texture in the latent RMS field and direction-dependent spectral decay (Fig. 8), indicating that the embeddings preserve both spatial texture and a weak-to-moderate anisotropic fabric aligned with structural trends. Such approximate power-law spectra are commonly observed in natural spatial fields; here we treat it as evidence of multi-scale organization rather than a strict fractal model.

4.2.1.6 Graph Spectral Structure and Distributed Representation.

Finally, we analyze embedding smoothness using Dirichlet energy over a spatial nearest-neighbor graph. Sorting dimensions by energy reveals a near-linear ramp (Fig. 9), with no sharp separation between low- and high-frequency channels. Even the smoothest dimensions exhibit substantial variation, while the highest-energy dimensions encode fine-scale texture. This confirms a distributed, multi-frequency representation rather than a small set of dominant components.

4.2.1.7 Qualitative Comparison via Unsupervised Segmentation.

To qualitatively compare embedding clusters with visible surface domains, we apply K-Means clustering (\(k=3\)) to the embeddings within a dense local patch and compare the result to satellite imagery (Fig. 10). The segmentation aligns closely with visible domains: a dark, rugged western belt consistent with mafic lithologies; a high-albedo eastern domain consistent with felsic or sandy cover; and an intermediate unit concentrated near contacts. Importantly, this spatial coherence emerges from the embedding vectors alone, with coordinates and imagery used only for visualization, suggesting that the embedding space captures surface-domain structure.

4.2.1.8 Summary.

Across independent diagnostics, the Arabian Shield embeddings exhibit consistent, spatially coherent and qualitatively geologically plausible structure without using absolute/global coordinates or labels during training. Individual dimensions form coherent spatial fields (Fig. 1), while random-pair similarities have near-zero mean (\(\mu\approx0.054\)), consistent with no strong embedding collapse and with a heavy right tail reflecting clustered terrane-scale domains (Fig. 2). Nonlinear organization is evident in UMAP, which yields sharply bounded regional zones compared to a diffuse linear PCA baseline (only \(\sim\)​17% variance in the first three PCs; Fig. 3). In a local patch, even a simple K-Means partition (\(k{=}3\)) recovers spatially coherent units visually aligned with visible surface domains (Fig. 10). Outliers selected by IF/LOF are supported by selective, leptokurtic similarity distributions and geographically compact match footprints (Fig. 4). Finally, similarity decays with distance at terrane scales (zero-crossing near \(\sim\)​300 km; Fig. 5) and spectral/graph analyses indicate multi-scale texture with distributed spatial frequencies across embedding channels (Figs. 7–9).

4.2.2 Study 2: Prospect-scale transfer via a dense embedding field over Suwaj↩︎

Study 1 characterized embedding structure at regional scale on the Arabian Shield. Here we test whether the same pretrained encoder transfers across scale when evaluated densely over a focused area of interest (AOI) on a new dense evaluation grid, and whether the resulting embedding field supports reproducible, GIS-consumable unsupervised domain maps and novelty proxies (Figure 11).

Because the model is trained without explicit absolute/global geographic coordinates or geological labels, the key scientific question is whether the learned representation nevertheless induces a spatially coherent latent field at prospect scale. Concretely, we test whether dense evaluation yields: (i) stable, contiguous domains rather than fragmented “salt-and-pepper” structure; (ii) sharp transitions localized in space (consistent with boundaries in the underlying signals); and (iii) controllable granularity under explicit analysis knobs (e.g., varying \(K\) in K-Means) instead of unstable partitioning.

4.2.2.1 Dense embedding grid (100 m).

We evaluate the encoder trained in Study 1 (trained on random-patch samples distributed across the Arabian Shield) on a regular 2D grid over the Suwaj AOI at 100 m spacing. The resulting field contains \(1{,}250{,}414\) embedding vectors of dimension 256, defining a dense latent field \(E(x,y)\in\mathbb{R}^{256}\) over the AOI. The AOI spans approximately 23.26–24.40\(^\circ\)N and 43.55–44.51\(^\circ\)E (CRS: EPSG:4979), with vertical extent \(z \in [-10{,}000,\, 1{,}372]\) m. Importantly, the model and preprocessing are held fixed; only the evaluation sampling density and spatial support change, making this a controlled test of representation stability under a new spatial regime.

4.2.2.2 Derived layers as controlled probes of representation structure.

To probe the organization of the dense embedding field, we derive several families of raster layers (exported as Cloud-Optimized GeoTIFFs for reproducible GIS analysis). (1) Manifold projections (PCA, UMAP) and corresponding RGB composites provide qualitative visualization of neighborhood structure in embedding space (Figure 11a–b), while individual principal components provide interpretable scalar fields (Figure 11e–f). (2) Unsupervised zonation via K-Means across \(K\in\{4,8,16,32,64\}\) supplies an explicit control knob on domain granularity; categorical maps at fixed \(K\) summarize domain partitions (Figure 11c). (3) Novelty / boundary proxies include distance-to-centroid under K-Means (Figure 11d) and multi-scale anomaly scores (Isolation Forest pyramid; Figure 11g) that emphasize signatures that are rare in embedding space and/or concentrated near transitions. Finally, texture proxies such as gray-level co-occurrence matrix (GLCM) statistics computed on selected components provide an additional controlled view of spatial heterogeneity (Figure 11h).

4.2.2.3 Qualitative evidence of coherent domains and multi-scale structure.

The Suwaj rasters indicate that the embedding-derived layers are not spatially random: both nonlinear (UMAP) and linear (PCA) composites form contiguous regions separated by sharp boundaries (Figure 11a–b). Consistent with the strongly non-linear geometry observed at regional scale (Study 1), the UMAP composite tends to emphasize discrete, sharply bounded domains, while PCA captures smoother large-scale variation. K-Means zonation exhibits the expected coarse-to-fine behavior as \(K\) increases: low \(K\) recovers macro-domains, whereas higher \(K\) refines these into more detailed subdomains (example at \(K{=}8\) in Figure 11c). The distance-to-centroid layers provide a complementary continuous view, highlighting mixed/transition zones and locally atypical signatures that are difficult to capture with categorical maps alone (Figure 11d). The anomaly and texture proxies further concentrate on spatially localized deviations and heterogeneous structure (Figure 11g–h), serving as hypothesis-generating layers for downstream validation rather than geological labels.

4.2.2.4 Summary.

This study demonstrates a scale-transfer property of the learned representation: a regional pretrained encoder, trained without absolute/global coordinate input, can be evaluated densely on a 2D grid at 100 m spacing to yield a spatially coherent embedding field over Suwaj (Figure 11). While these layers are not geological labels, their consistency across projection methods, the controllable granularity via \(K\), and the agreement between categorical (zonation) and continuous (distance/anomaly/texture) probes provide evidence that the embedding space captures structured, multi-scale variation suitable for downstream interpretation and targeting.

5 Conclusion↩︎

We presented MABLE, a self-supervised representation learning pipeline for large, heterogeneous geospatial graphs that encourages robust node and graph-level embeddings using a tightly integrated set of objectives and stability constraints. The method couples masked node reconstruction with representation-shaping losses that explicitly encourage useful geometry: (i) node correspondence and node uniformity to promote cross-view consistency while discouraging collapse and improving feature diversity, (ii) graph alignment to enforce invariance across augmented views (including subgraph views), and (iii) graph uniformity to maintain global spread across graphs within a batch. On the architectural side, we combine a modality-agnostic token interface with a Lipschitz-controlled pooling operator and a well-conditioned (bi-Lipschitz) reconstruction head, linking practical stability to the theoretical sensitivity bounds established for the reconstruction slice and pooling components discussed throughout the paper.

Across the mineral-prospectivity experiments, MABLE embeddings are most useful as complementary contextual representations rather than as standalone predictors under strong geographic distribution shift. On the local-scale copper benchmark (redacted Experiment 1), embeddings alone are not sufficient to extrapolate into a geographically distinct holdout region, but compact embedding components improve coordinate-only and engineered geophysical baselines, indicating transferable context not fully summarized by local MAG/ANT statistics. On the regional-scale Arabian Shield corpus, the learned 256-dimensional graph embeddings exhibit strong spatial coherence without using absolute/global coordinates or geological labels during training: manifold projections, similarity search, anomaly detection, and graph/spectral diagnostics consistently indicate a distributed, multi-scale representation that organizes the Shield into stable domains and highlights rare signatures. Finally, a dense 100 m grid evaluation over Suwaj demonstrates scale transfer: a regional pretrained encoder can be evaluated prospect-scale to produce GIS-consumable, embedding-derived layers (zonation, novelty proxies, and texture probes) that are spatially coherent and controllably granular. These regional and prospect-scale products should be interpreted as hypothesis-generating latent layers rather than geological labels.

A core architectural differentiator explored in this work is the feature-embedding slice for reconstruction with a constrained decoder, together with a complementary contextual slice regularized by cross-view node correspondence. Ablations indicate that this architectural choice, when properly tuned, improves training dynamics and reduces overfitting pressure. These results support the practical use of the slice+bi-Lipschitz decoder together with contextual cross-view consistency as stability-oriented design choices that need not materially sacrifice predictive utility when appropriately parameterized.

5.0.0.1 Limitations and future work.

Several directions remain open. First, while embeddings can complement engineered context under spatial distribution shift, stronger downstream models (e.g., residual or hierarchical predictors, or end-to-end fine-tuning) may better exploit the learned context in the strictest extrapolation regimes. Second, the current construction uses edge-free graphs; incorporating lightweight, uncertainty-aware relational structure (e.g., learned sparse edges or physics-informed neighborhoods) is a natural extension when explicit connectivity is meaningful. Third, prospect-scale products derived from embeddings (zonation, anomalies, textures) should be validated more directly against mapped geology and known mineral systems to quantify correspondence beyond qualitative alignment. More broadly, evaluating transfer across regions, sensor stacks, sampling densities and characterizing failure modes under missing modalities or strong coverage gaps will further clarify the operational envelope of MABLE for exploration workflows.

Overall, MABLE provides a principled, end-to-end approach to learning stable geospatial graph representations that (i) remain well-conditioned under augmentation and sampling artifacts, (ii) support both node-level reconstruction fidelity and cross-view node consistency together with global embedding robustness, and (iii) yield coherent, multi-scale latent structure that can be operationalized as GIS-ready layers for interpretation and targeting in the settings studied here.

6 Appendix↩︎

6.1 Proof of Proposition 1 - Node Feature Similarity Preservation.↩︎

6.2 Relationship Between the Log-Based Surrogate and Normalized SCL↩︎

This appendix details the local relationship between the proposed log-based surrogate and the normalized/cosine form of Spectral Contrastive Loss (SCL). Assume embeddings are compared by cosine similarity, equivalently by inner product after unit normalization, so \(\cos(z,z') \in [-1,1]\).

6.2.0.1 Log-Based Surrogate and Approximation by SCL.

We define a log-based surrogate loss designed to explicitly target positive alignment (\(\cos(z,z^{+}) = 1\)) and negative orthogonality (\(\cos(z,z^{-}) = 0\)). The loss takes the form: \[\mathcal{L}_{\text{Surrogate}} = -\mathbb{E}_{(z,z^{+})}[\log (\text{PositiveArg})] - \mathbb{E}_{(z,z^{-})}[\log (\text{NegativeArg})].\] To ensure the arguments are in \((0,1]\) and achieve the value 1 at the desired optimal cosine similarities, we choose: \[\text{PositiveArg} = \exp(\cos(z,z^{+})-1) \quad \text{and} \quad \text{NegativeArg} = 1 - \frac{1}{2}\cos(z,z^{-})^2.\] The log-based surrogate is then: \[\mathcal{L}_{\text{Surrogate}} = -\mathbb{E}_{(z,z^{+})}[\log(\exp(\cos(z,z^{+})-1))] - \mathbb{E}_{(z,z^{-})}[\log(1 - \frac{1}{2}\cos(z,z^{-})^2)].\] Simplifying the positive term: \[\mathcal{L}_{\text{Surrogate}} = -\mathbb{E}_{(z,z^{+})}[\cos(z,z^{+})-1] - \mathbb{E}_{(z,z^{-})}[\log(1 - \frac{1}{2}\cos(z,z^{-})^2)].\] Dropping the constant \(-1\) from the positive term (which does not affect minimization): \[\mathcal{L}_{\text{Surrogate}}' = -\mathbb{E}[\cos(z,z^{+})] - \mathbb{E}[\log(1 - \frac{1}{2}\cos(z,z^{-})^2)].\] Let \(t^{+} = \cos(z,z^{+})\) and \(t^{-} = \cos(z,z^{-})\) and \(\tfrac12 \, \mathcal{L}_{\text{SCL}} = -\mathbb{E}[t^{+}] +\tfrac12\mathbb{E}[(t^{-})^{2}]\). The positive term of \(\mathcal{L}_{\text{Surrogate}}'\) is exactly the positive term of \(\tfrac12 \, \mathcal{L}_{\text{SCL}}\).

We show that the negative term of \(\mathcal{L}_{\text{Surrogate}}'\) is approximated by the negative term of \(\tfrac12 \, \mathcal{L}_{\text{SCL}}\) around the desired negative optimum \(t^{-} = 0\). We perform a Taylor expansion of \(-\log(1 - \frac{1}{2}t^2)\) around \(t=0\): The Taylor expansion of \(-\log(1-x)\) around \(x=0\) is \(x + \frac{x^2}{2} + \frac{x^3}{3} + \dots\) for \(|x|<1\). Substitute \(x = \frac{1}{2}t^2\). For \(t \in [-1, 1]\), \(t^2 \in [0, 1]\), so \(\frac{1}{2}t^2 \in [0, 1/2]\), which means \(|x|<1\) is satisfied. \[-\log(1 - \frac{1}{2}t^2) = (\frac{1}{2}t^2) + \frac{(\frac{1}{2}t^2)^2}{2} + \frac{(\frac{1}{2}t^2)^3}{3} + \dots = \frac{1}{2}t^2 + \frac{1}{8}t^4 + \frac{1}{24}t^6 + \dots\] So, the expectation of the negative term of \(\mathcal{L}_{\text{Surrogate}}'\) is: \[-\mathbb{E}[\log(1 - \frac{1}{2}\cos(z,z^{-})^2)] = \mathbb{E}[\frac{1}{2}\cos(z,z^{-})^2 + \frac{1}{8}\cos(z,z^{-})^4 + \frac{1}{24}\cos(z,z^{-})^6 + \dots].\] Taking only the leading term from this expansion around \(\cos(z,z^{-})=0\), we get \(\mathbb{E}[\frac{1}{2}\cos(z,z^{-})^2]\). Thus, the negative term of \(\mathcal{L}_{\text{Surrogate}}'\) is approximated by the negative term of SCL around \(\cos(z,z^{-})=0\).

Combining the exact positive term and the approximated negative term, we see that \(\mathcal{L}_{\text{Surrogate}}'\) is approximated by scaled normalized SCL around the desired cosine geometry (\(\cos(z,z^{+}) = 1, \cos(z,z^{-}) = 0\)): \[\mathcal{L}_{\text{Surrogate}}' \approx -\mathbb{E}[\cos(z,z^{+})] + \mathbb{E}\!\left[\frac{1}{2}\cos(z,z^{-})^2\right] = \tfrac12 \, \mathcal{L}_{\text{SCL}}.\] This demonstrates that normalized SCL is a computationally simpler form that captures the leading behavior of a principled log-based surrogate near the desired cosine geometry.

6.2.0.2 Conclusion.

Through the construction of a log-based surrogate whose terms explicitly target the desired cosine geometry, we obtain normalized SCL as the leading local quadratic approximation: positive pairs are driven toward angular alignment and unpaired embeddings toward angular decorrelation. This is the fixed-cosine geometry used by MABLE without requiring a learned discriminator.

6.3 A DV-Style Interpretation of the Normalized/Cosine SCL Surrogate↩︎

This appendix concerns the normalized/cosine SCL-style surrogate used in MABLE, not unconstrained raw inner-product SCL except when embeddings are unit-normalized or otherwise tightly norm-controlled. The original spectral contrastive loss of [10] is written using raw inner products, whereas MABLE uses cosine similarity to remove scale ambiguity and keep scores bounded in \([-1,1]\). The DV-style argument below should therefore be read as a heuristic justification for the normalized/cosine surrogate used here.

Mutual information, \(I(Z, Z^+)\), quantifies the statistical dependence between the latent representation \(Z=f(X)\) and its augmented view \(Z^+=f(X^+)\). A standard method to optimize MI is through variational lower bounds like the Donsker–Varadhan (DV) representation. The DV bound states that for any measurable function \(T(z, z')\), a lower bound on MI is given by: \[I(Z,Z^+) \ge \mathbb{E}_{(Z,Z^+)}\bigl[T(Z,Z^+)\bigr] - \log \mathbb{E}_{Z,Z^- \sim p(Z)^2}\Bigl[e^{T(Z,Z^-)}\Bigr].\] Here, \(Z^-\) denotes an independent sample from the marginal \(p(Z)\) (so the second expectation is under the product distribution \(p(Z)p(Z^-)\); we write \(p(Z)^2\) for brevity). To relate this to the normalized/cosine SCL-style surrogate used in MABLE, we choose the test function \(T(z,z') = \cos(z,z')\). Let \(t^+ = \cos(Z,Z^+)\) and \(t^- = \cos(Z,Z^-)\). Assume the latent embeddings \(Z\) are unit-normalized, so \(\cos(z,z') \in [-1,1]\) (and equals the dot product), and that \(p(Z)\) is approximately antipodally symmetric (e.g., close to isotropic on the unit sphere \(\mathbb{S}^{d-1}\)), so that \(t^-=\cos(Z,Z^-)\) is approximately symmetric about \(0\) (hence \(\mathbb{E}[t^-]\approx 0\) and odd moments are negligible). The bound becomes: \[I(Z,Z^+) \ge \mathbb{E}[t^+] - \log \mathbb{E}[e^{t^-}].\]

We analyze the second term, \(-\log \mathbb{E}[e^{t^-}]\), where \(t^-=\cos(Z,Z^-)\) and \(Z^-\sim p(Z)\) is independent of \(Z\). Under this approximate symmetry, the expectation of any odd power of \(t^-\) is approximately zero: \(\mathbb{E}[(t^-)^{2k+1}] \approx 0\) for \(k \ge 0\).

We perform a Taylor expansion of \(e^x\) around \(x=0\): \(e^x = \sum_{n=0}^\infty \frac{x^n}{n!}\). Substituting \(x = t^-\) and taking the expectation: \[\mathbb{E}[e^{t^-}] = \mathbb{E}\left[\sum_{n=0}^\infty \frac{(t^-)^n}{n!}\right] = \sum_{n=0}^\infty \frac{\mathbb{E}[(t^-)^n]}{n!}.\] Using the property that odd moments are approximately zero: \[\mathbb{E}[e^{t^-}] = \frac{\mathbb{E}[(t^-)^0]}{0!} + \frac{\mathbb{E}[(t^-)^1]}{1!} + \frac{\mathbb{E}[(t^-)^2]}{2!} + \frac{\mathbb{E}[(t^-)^3]}{3!} + \dots\] \[\mathbb{E}[e^{t^-}] \approx 1 + 0 + \frac{\mathbb{E}[(t^-)^2]}{2} + 0 + \dots = 1 + \sum_{k=1}^\infty \frac{\mathbb{E}[(t^-)^{2k}]}{(2k)!}.\]

Now, we take the logarithm using the Taylor expansion of \(\log(1+x)\) around \(x=0\), which is \(x - \frac{x^2}{2} + O(x^3)\). Let \(x = \sum_{k=1}^\infty \frac{\mathbb{E}[(t^-)^{2k}]}{(2k)!} = \frac{\mathbb{E}[(t^-)^2]}{2} + \frac{\mathbb{E}[(t^-)^4]}{24} + \dots\).

Assuming \(\mathbb{E}[(t^-)^2]\) is small (typically the case in high dimensions under near-uniform hyperspherical marginals), we can approximate: \[\log \mathbb{E}[e^{t^-}] = \log(1+x) \approx x - \frac{x^2}{2}\] \[\log \mathbb{E}[e^{t^-}] \approx \left(\frac{\mathbb{E}[(t^-)^2]}{2} + \frac{\mathbb{E}[(t^-)^4]}{24} + \dots\right) - \frac{1}{2}\left(\frac{\mathbb{E}[(t^-)^2]}{2} + \dots\right)^2\]

Keeping only the leading terms: \[\log \mathbb{E}[e^{t^-}] \approx \frac{1}{2}\mathbb{E}[(t^-)^2] + \text{higher order terms}.\]

For the raw inner-product choice \(T(z,z')=z^\top z'\), the same DV representation can formally be written, but the second-order reduction to a quadratic negative term requires stronger assumptions: finite exponential moments, approximate zero mean, small variance, negligible higher-order cumulants, and explicit norm control. Without norm control, \(\log \mathbb{E}[\exp(Z^\top Z^-)]\) can be dominated by vector norms rather than angular dependence. This is the scale ambiguity avoided by the cosine surrogate used in MABLE.

Substituting this approximation back into the DV lower bound: \[I(Z,Z^+) \gtrsim \mathbb{E}[t^+] - \frac{1}{2}\mathbb{E}[(t^-)^2].\]

Recall the scaled normalized/cosine SCL-style variant \(\mathcal{L}_{\text{SSCL}}=\tfrac12\mathcal{L}_{\text{SCL}} = -\mathbb{E}[t^+] + \tfrac12 \mathbb{E}[(t^-)^2]\). Minimizing \(\mathcal{L}_{\text{SSCL}}\) is equivalent to maximizing \(-\mathcal{L}_{\text{SSCL}} = \mathbb{E}[t^{+}] -\tfrac12\mathbb{E}[(t^{-})^{2}]\). Thus, we see that: \[I(Z,Z^+) \gtrsim -\mathcal{L}_{\text{SSCL}} \quad\text{(equivalently, } I(Z,Z^+) \gtrsim -\tfrac12 \mathcal{L}_{\text{SCL}}\text{).}\] This shows that, under the stated boundedness, approximate symmetry, and small-variance assumptions, minimizing the normalized/cosine SCL-style surrogate used in MABLE approximately maximizes a second-order approximation to this particular DV-style mutual-information objective. It should not be read as a general claim that unconstrained raw inner-product SCL has the same controlled DV approximation without additional norm-control assumptions.

6.4 Proof of Proposition 2 - Lipschitz Pooling is Stable Under Optimal Matching↩︎

6.5 Proof of Proposition 3 - Sum, Mean, and Max Pooling are Lipschitz↩︎

6.6 Proof of Lemma 1 - Bounded-Domain Lipschitzness of Clipped Softmax Attention↩︎

6.7 Proof of Corollary 1 - Conditioned Projection Preserves Attention-Induced Separation↩︎

6.8 Ablation on feature-slice dimensionality \(d_e\)↩︎

4pt

References↩︎

---

1. Correspondence: yaniv@shulman.info. This work was done while Yaniv Shulman was working at Fleet Space Technologies.↩︎

2. shae.akbarpour@fleet.space↩︎

3. jack.muir@fleet.space↩︎

4. In practice, \(\mathcal{C}(G,T_1,T_2)\) is the set of nodes that survive the two augmentations and can be aligned by identity or inherited indexing; for efficiency we may evaluate the loss on a random subset of \(\mathcal{C}(G,T_1,T_2)\) in each minibatch.↩︎

5. We use \(x_u\) to denote the reconstruction target; the encoder input is formed by masking/corrupting the reconstructible components of \(x_u\) (where selected components are replaced by learned mask tokens) and concatenating the resulting vector with the relative geometric context.↩︎