¹

1 Introduction↩︎

Modern face recognition (FR) systems are trained on large-scale datasets of face crops, typically collected from the internet [1]. To represent a person, the system takes multiple crops of that person’s face and passes each through a deep neural network that produces a 512-dimensional embedding vector, normalised to lie on the unit hypersphere. The training of these models is primarily driven by angular-margin losses [2]–[4], which pull the embeddings of crops belonging to the same identity into a cluster on the hypersphere while pushing apart the clusters of different identities. During training, a classification head maps embeddings to identity logits and this head is discarded after training. When deployed, the system typically uses only the backbone neural network. Enrolled faces are stored as embeddings, and the similarity between two faces is measured by the inner product of their corresponding embeddings.

As angular-margin training directly optimises the compactness of each identity’s cluster, we pose the following question: how much information about training-set membership is retained in the geometry of these embedding clusters? If training identities form systematically tighter or better-separated clusters than unseen identities, then the embedding geometry carries a residual membership signal.

Quantifying this residual signal is related to membership inference attacks (MIA), a line of work investigating whether a given sample or identity was part of a model’s training data [5], [6]. In practice, membership inference serves as an auditing tool for data privacy by allowing individuals or regulators to verify whether a model was trained on data belonging to a specific person in the context of data protection rights such as the right to erasure under the GDPR [7]. MIA literature primarily focuses on closed-set classifiers that expose prediction confidences, loss values, or the model weights directly [5], [6], [8], [9]. In our setting, we consider open-set face recognition where neither logits nor losses nor model weights are available, so the membership signal must be sought entirely in the geometry of the embeddings themselves.

To measure this signal, we compute four different statistics capturing different aspects of cluster geometry on the hypersphere and measure how much they differ between training and unseen identities (Figure 1). We train 180 face recognition models from scratch (Section 4.2) and evaluate each model on nine benchmarks, including a same-domain held-out split that controls for the pose, age, quality, and ethnicity shifts present in the other benchmarks. To our knowledge, this constitutes the largest systematic study of membership information in face embedding geometry. Our contributions are: (1) we quantify the membership information retained in four geometric statistics computed from face embeddings; (2) we identify the primary factors that control how much of this information persists; (3) we show that cross-domain non-member benchmarks (e.g.differing in pose or age) inflate the apparent membership signal by conflating domain shift with training-attributable differences; (4) we demonstrate that fusing all statistics captures membership information beyond the best individual statistic.

2 Related Work↩︎

Membership inference on classifiers. Membership inference literature targets closed-set classifiers, where the attacker can observe prediction confidences, logits, or loss values of the target model. In the open-set face recognition setting, however, none of these signals are available and this has motivated a line of work operating directly on embeddings. Shokri et al. [5] introduce the shadow-model paradigm and train substitute classifiers to learn a binary meta-classifier on output distributions. Salem et al. [8] relax many of these assumptions and show that a single shadow model and only the top-\(k\) predictions often suffice. Carlini et al. [6] recast MIA as a likelihood-ratio attack (LiRA) and derive near-optimal attacks. They show that per-example hardness varies by orders of magnitude. Hu et al. [9] provide a comprehensive survey.

3pt

\(^\ast\)Varies factors individually but not in a fully crossed design.

Membership inference on embedding models. Existing embedding-based approaches differ in the granularity of the membership question: some ask whether a specific image was used for training (sample level), while others ask whether any image of a given person was included (identity level). Li et al. [10] formalise identity-level inference on metric-embedding models and use intra-similarity and inter-dissimilarity of probe embeddings as attack features on person re-identification and face recognition. Chen et al. [11] propose FACE-AUDITOR, a system that queries few-shot face recognition models with designed probing sets and uses the returned similarity scores as auditing features. They report accuracies up to 99%. SLMIA-SR [12] addresses the analogous problem in speaker recognition by training shadow models with controlled membership mixing ratios. All three operate at the identity level. EncoderMI [13], by contrast, operates at the sample level on contrastive-learning encoders including CLIP and exploits the observation that an overfitted encoder produces more similar representations for augmented views of a training sample than for an unseen one.

White-box approaches on face recognition. A parallel line of work allows access to model internals. The motivation is that intermediate representations such as activation maps, batch-normalisation statistics, or gradient signals may carry a stronger membership trace than the final embedding alone, at the cost of requiring more privileged access to the model. These methods also typically operate at the sample level, determining whether a specific image (rather than a person) was included in training. The Membership Inference Test (MINT) of DeAlcala et al. [14] trains an MLP or CNN on intermediate activation maps to classify individual face images as member or non-member and achieves up to 90% accuracy on three ResNet-100 models. A companion study [15] analyses factors that affect MINT performance, including loss function, dropout, and number of training epochs. Mancera et al. [16] extend MINT to object classification and report 70–80% precision depending on layer depth. Huang et al. [17] exploit distances between intermediate features and batch-normalisation parameters for sample-level membership and model-inversion attacks [18]. Even with this level of access, Rezaei and Liu [19] show that the advantage over random guessing can be modest once the model generalises well.

Memorisation and embedding geometry. The question of why a geometric membership signal might exist at all connects to a discussion on memorisation in deep networks. If a model must memorise certain training examples to achieve low error, particularly on the tail of the data distribution, then its internal representations and output embeddings may carry traces of those examples. Feldman [20] formalises this argument, showing theoretically that long-tailed distributions necessitate memorisation for near-optimal generalisation. Angular-margin losses [2]–[4] amplify this effect, potentially making training identities geometrically distinguishable from unseen ones. Unlearning methods [21], [22] and differential privacy [23] aim to erase or bound such signals.

Comparison with this work. Our contribution is a controlled measurement study of how much membership information geometric statistics carry, not a stronger attack feature. Table ¿tbl:tab:related-summary? summarises the literature. Prior embedding-level work proposes specific attack features and demonstrates that membership inference is feasible through evaluating a small number of models trained under a single configuration. Our study differs in three respects. First, the goal: we measure how much membership information is retained in the explicit cluster geometry and identify which training factors control the amount that persists. Second, the experimental scale: 180 models, whereas the largest prior study evaluates nine. Third, domain control: we add a same-domain held-out reference, which aims to separate training-attributable signal from domain-shift effects. Several of the per-identity statistics we evaluate have already been used as features in prior attacks. For instance, the intra-similarity score proposed by Li et al. [10] is equivalent to the mean pairwise cosine similarity that we include as a baseline statistic (defined in Section 3.2).

3 Methodology↩︎

3.1 Access Model and Measurement Procedure↩︎

We assume that an auditor can query an FR model \(f\) with \(k \geq 2\) probe images of a target identity \(y\) and obtain the corresponding \(\ell_2\)-normalised embeddings. From the resulting cluster, the auditor computes four per-identity statistics (Section 3.2) and uses them to assess if the cluster geometry carries information about \(y\)’s membership in the training set. For each model we train, we compute these statistics on member identities (from WebFace4M) and on non-member identities drawn from the benchmarks in Table 1.

3pt

For each statistic, we obtain a scalar score per identity. To quantify how much membership information a statistic carries, we sweep a decision threshold \(\tau\) across the range of observed values and classify an identity as a member if its score exceeds \(\tau\) (or falls below \(\tau\) for statistics where lower values indicate membership). We report the area under the resulting ROC curve (AUC), hereafter threshold-classifier AUC, as a threshold-free summary, which is equal to the probability that a randomly chosen member scores higher than a randomly chosen non-member. This serves as our primary measure of the membership information carried by a given statistic. We use AUC rather than the true positive rate at a fixed false positive rate, because our framing concerns separability across all decision thresholds rather than the operating point most relevant for a deployable attack. Of the four statistics defined in the next section, two require only the embeddings and two additionally require the training-time scale and margin hyperparameters. We refer to these two access levels as black-box (embeddings only) and grey-box (embeddings plus \(s\) and \(m\)). The grey-box setting is realistic when models are released alongside their training configurations.

3.2 Per-Identity Geometric Statistics↩︎

Given an identity \(y\) with \(k \geq 2\) probe images, we extract embeddings \(\{e_1, \ldots, e_k\}\) via the target model and compute four statistics from the resulting cluster. Each statistic captures a different aspect of cluster shape. The underlying intuition is that an angular-margin loss explicitly optimises the representation of training identities, so members are expected to form tighter clusters on the embedding hypersphere than non-members. If that expectation holds, the statistic will carry membership information; the question is how much, and what factors control it.

Pairwise cosine similarity (PairCos). The mean cosine similarity over pairs of embeddings within the identity: \[\bar{s} = \binom{k}{2}^{-1} \sum_{i < j} \cos(e_i, e_j).\] Higher values indicate a more coherent cluster.

vMF concentration \(\kappa\). We model the per-identity embedding distribution as a von Mises-Fisher (vMF) distribution on the unit hypersphere and estimate the concentration parameter \(\kappa\) via maximum likelihood. We use the closed-form approximation of Banerjee et al. [31]: \[\hat{\kappa}_0 = \frac{\bar{R}\,(d - \bar{R}^2)}{1 - \bar{R}^2}, \qquad \bar{R} = \left\| \frac{1}{k}\sum_{i=1}^{k} e_i \right\|,\] where \(d = 512\) is the embedding dimensionality and \(\bar{R}\) is the mean resultant length. Higher \(\hat{\kappa}\) corresponds to greater directional concentration. Note that ordering identities by \(\hat{\kappa}\) is equivalent to ordering them by cluster inertia \(I = k^{-1}\sum_i \|e_i - \mu\|^2\), where \(\mu = k^{-1}\sum_i e_i\) is the cluster mean. For \(\ell_2\)-normalised embeddings, \(I\) reduces to \(1 - \bar{R}^2\), and \(\hat{\kappa}_0\) is strictly decreasing in \(1-\bar{R}^2\), so the two quantities produce identical identity rankings.

Penalised logit (PenLogit). To approximate the margin-penalised logit that the training loss would compute for the cluster, we use a leave-one-out scheme. For each embedding \(e_i\), we compute a prototype \(\hat{w}_{\backslash i} = \bar{e}_{\backslash i}/\|\bar{e}_{\backslash i}\|\) as the \(\ell_2\)-normalised mean of the remaining \(k{-}1\) embeddings, and the margin-penalised logit is \[\ell_i = s \cdot g\left(\cos\angle\left(e_i, \hat{w}_{\backslash i}\right),\; m\right),\] where \(s\) and \(m\) are the scale and margin of the loss head, and \(g\) is the head-specific penalisation function (e.g. \(g(\cos\theta, m) = \cos(\theta + m)\) for ArcFace). The statistic value is the mean \(\bar{\ell} = k^{-1}\sum_i \ell_i\). A higher penalised logit indicates that the embeddings are tightly aligned to their own prototype.

Prototype softmax CE (ProtoCE). We construct a surrogate of the training loss. For each identity \(y\), we compute a class prototype \(\hat{w}_y = \mu / \|\mu\|\) and treat all other identities in the evaluation set as negative classes. The cross-entropy loss with the margin-modified logit for the target class and standard cosine logits for the negatives gives \[\mathcal{L}_y = -\log \frac{\exp\bigl(s\cdot g(\cos\theta_y,\, m)\bigr)}{\exp\bigl(s\cdot g(\cos\theta_y,\, m)\bigr) + \sum_{j \neq y} \exp(s\cdot\cos\theta_j)}\] We report \(-\mathcal{L}_y\) so that higher values indicate membership, consistent with the sign convention of the other statistics. Because the softmax denominator depends on the composition and size of the evaluation set, ProtoCE values are not directly comparable across benchmarks with different identity pools. Throughout, all AUC values are oriented so that \(\text{AUC} > 0.5\) indicates better-than-chance separation.

4 Experimental Setup↩︎

4.1 Training Data↩︎

All models are trained on WebFace4M [1], a large-scale face dataset containing approximately 4 million images of 205,990 identities. WebFace4M is well-suited to this study because its size permits creating training subsets that span several orders of magnitude in identity count while still providing sufficient images per identity for meaningful embedding statistics. Every probe image is aligned to a canonical \(112\!\times\!112\) crop with the standard InsightFace five-landmark pipeline.

4.2 Ablation Design↩︎

We vary four factors in a fully crossed design.

Backbone capacity. We train three IResNet architectures: IResNet-34, IResNet-50, and IResNet-100 [2], [32].

Angular-margin loss head. We pair each backbone with three angular-margin loss heads: ArcFace [2] (\(s\!=\!64\), \(m\!=\!0.5\)), CosFace [3] (\(s\!=\!64\), \(m\!=\!0.4\)), and MagFace [4] (\(s\!=\!64\), per-sample \(m \in [0.45, 0.8]\) depending on the un-normalised feature magnitude).

Training-set size (\(n_{\text{ids}}\)). To study the effect of identity coverage, we create five partitions by randomly sampling \(n_{\text{ids}}\in \{1\text{K}, 10\text{K}, 50\text{K}, 100\text{K}\}\) identities (with a fixed seed), plus one partition that uses all available identities. The identities not selected for training serve as same-domain non-members in the held-out WebFace4M benchmark.

Training duration. We train the backbones for 5, 10, 20, and 40 epochs to track the evolution of cluster-statistic separability. Longer training exposes each identity to more gradient updates and may therefore affect the gap between member and non-member statistic distributions.

The full grid spans \(3~\text{backbones} \times 3~\text{losses} \times 5~n_{\text{ids}}\text{ levels} \times 4~\text{epochs} = 180\) model configurations. All models are trained with SGD (momentum 0.9, weight decay \(5\!\times\!10^{-4}\)), a polynomial-decay learning-rate schedule starting at \(0.1\) with one warmup epoch, batch size 128, mixed-precision (FP16), and random horizontal flips.

4.3 Evaluation Benchmarks↩︎

The nine benchmarks (Table 1) supply standard verification protocols for measuring model quality and provide non-member pools with diverse domain characteristics so that we can study how pose, quality, and demographics affect the apparent cluster-statistic separability.

Eight of the nine benchmarks are drawn from sources disjoint from WebFace4M (we verified that no identity overlap exists by cross-checking subject identifiers); Table 1 lists their sizes and available covariate metadata. Two dataset pairs form natural experiments (matched comparisons where a single attribute varies). CFP-FF vs.CFP-FP [26] contrasts frontal-only against frontal-plus-profile probes for the same 500 identities, so any statistic difference is attributable to pose. LFW [24] vs.XQLFW [28] shares the same identity set and pair protocol but degrades one image per pair to isolate image quality. CPLFW [25] (cross-pose LFW), AgeDB-30 [27] (age gap \(\geq\)​30 yr), RFW [29] (ethnicity-balanced, four groups), and IJB-C [30] complete the cross-domain set.

Held-out WebFace4M split. The ninth benchmark is constructed from WebFace4M itself: for each model trained on \(n_{\text{ids}}\) identities, we sample \(\min(n_{\text{ids}},\, 2{,}500)\) members and 2,500 non-members from the remaining subjects; at \(n_{\text{ids}}{=}\)all no non-members remain. As both groups share the same acquisition pipeline, AUCs are arguably attributable to the cluster-geometry difference between seen and unseen identities rather than to domain shift.

5 Results and Discussion↩︎

This section presents experimental results and analysis for the 180-model grid described in Section 4. We examine verification performance, the effect of training-set size and duration on membership separability, distributional confounds, and statistic fusion. Full per-epoch, per-model results and ROC/DET curves are provided in the supplementary material.

4pt

5.1 Verification Performance↩︎

We first establish that our model grid spans a meaningful range of verification quality. Table ¿tbl:tab:eer? reports the verification equal error rate (EER) at epoch 40 for every combination of backbone, loss head, and \(n_{\text{ids}}\) on the cross-domain benchmarks. The table is dense and provided primarily as a per-cell reference, and the discussion below focuses on patterns and points back to specific cells only as needed. Performance generally improves with \(n_{\text{ids}}\), with diminishing gains beyond \(100\)K identities. Once \(n_{\text{ids}}\) is held fixed, backbone depth provides small reductions in verification EER on domain-shifted benchmarks, and the choice of loss head has an even smaller effect. Notably, the highest cluster-statistic separability occurs at \(n_{\text{ids}}{=}1\)K, where verification performance is worst. As \(n_{\text{ids}}\) grows, verification improves while separability decreases. However, non-trivial separability persists even at \(n_{\text{ids}}\) levels that achieve competitive verification EER, so the patterns reported below are not confined to poorly performing models.

Table ¿tbl:tab:eer? also reports the threshold-classifier AUC for all four cluster statistics. No single statistic achieves the highest AUC across all benchmarks. PenLogit ranks first most often at moderate-to-high \(n_{\text{ids}}\), in particular on the pose- and quality-degraded sets, whereas ProtoCE attains the highest AUC on the frontal and high-quality sets (CFP-FF, LFW, IJB-C), so the best statistic remains dataset-dependent. Because the eight non-WebFace4M benchmarks differ from the training source in pose, quality, and demographics, the observed AUC conflates training-attributable and domain-induced differences, which we discuss in Section 5.3. All four statistics exhibit similar qualitative trends across factors, so we show only PairCos distributions in Figure 2.

3.5pt

5.2 Effect of \(n_{\text{ids}}\) and Training Duration↩︎

Figure 2 presents the main finding of this study. Panel (c) shows that the PairCos distributions of members and non-members converge as \(n_{\text{ids}}\) increases from 1K to all identities. At \(n_{\text{ids}}{=}1\)K the distributions are well separated, whereas at \(n_{\text{ids}}{=}100\)K and beyond the separation narrows, though it remains above chance even on the held-out WebFace4M split (Table 3, discussed in Section 5.4).

Panels (a) and (b) of Figure 2 isolate the effect of training duration at two extremes of identity coverage. At \(n_{\text{ids}}{=}1\)K, the PairCos separation between members and non-members increases with the number of training epochs. At \(n_{\text{ids}}{=}100\)K the same trend is present but attenuated.

Table 2 reports the ANOVA results (\(\hat{\eta}^2_p\), the proportion of variance explained after accounting for all other factors). Training-set size explains the most variance (\(\hat{\eta}^2_p \geq .95\)), followed by training duration (\(\hat{\eta}^2_p \geq .89\)). The strong \(n_{\text{ids}}\!\times\!\text{Epoch}\) interaction (\(\hat{\eta}^2_p \geq .92\)) further shows that the epoch effect is moderated by identity coverage. The \(n_{\text{ids}}\!\times\!\text{Epoch}\) cell means on the same-domain WebFace4M split show that the mean PairCos AUC increase from epoch 5 to epoch 40 ranges from \(+.38\) at \(n_{\text{ids}}{=}10\)K down to \(+.16\) at \(n_{\text{ids}}{=}100\)K, with \(n_{\text{ids}}{=}1\)K reaching a ceiling near 1.0 already at epoch 10. Models with small identity pools thus show the largest separability gains over training, whereas models trained on \(100\)K identities have only a modest increase.

Backbone and loss head. The ANOVA also shows that backbone architecture and loss head explain far less variance in threshold-classifier AUC, the loss-head factor reaching at most \(\hat{\eta}^2_p = .04\) and backbone depth at most \(.09\) across all four statistics — an order of magnitude below the contributions of \(n_{\text{ids}}\) and number of epochs. Beyond the IResNet grid, a ViT-T [33], [34] backbone (CosFace) trained under the identical protocol [35] reproduces the same \(n_{\text{ids}}\) trend in both EER and membership AUC (Table ¿tbl:tab:eer?, bottom block), so the effect is not specific to convolutional backbones.

5.3 Effect of Input-Data Distribution↩︎

The benchmarks used in this study differ in pose, quality, age, and demographics, which affect embedding-cluster geometry independently of membership. The distributional-shift rows of Table 2 quantify these effects, and we discuss each factor below. For matched-pair benchmarks, we also translate each effect into the AUC change it would produce at the same-domain training-attributable baseline at \(n_{\text{ids}}{=}100\)K (where the AUC is \(\approx 0.71\)). We do so by passing each AUC through the inverse standard normal cumulative distribution function (a probit transform), on which equal shifts in the underlying non-member distribution correspond to equal numerical changes regardless of baseline. We take the matched-pair difference on this transformed scale and convert it back to AUC at the chosen baseline. This step is needed because the AUC scale saturates near 1, so a fixed shift in the underlying distribution produces a smaller AUC change near the ceiling than in the middle.

Pose. Profile-view faces produce large shifts in cluster-statistic distributions. The CFP-FP vs.CFP-FF comparison is the cleanest controlled experiment: both sets share the same 500 identities, so the observed Cohen’s \(d'\) (standardised mean difference) is attributable to pose alone (Table 2). CPLFW vs.LFW yields even larger \(|d'|\) for PairCos and PenLogit, but those two sets are not identity-matched, so other factors may contribute. In both comparisons, profile views widen the intra-identity distribution regardless of membership. This pushes statistic values towards the non-member range and inflates separability. At the \(n_{\text{ids}}{=}100\)K baseline, the pose contribution from CFP-FF to CFP-FP corresponds to an AUC change of \(+0.26\), matching or exceeding the training contribution above chance (\(+0.21\)).

Image quality. XQLFW shares LFW’s identity set and pair protocol but degrades one image per pair. For PairCos and PenLogit, the resulting Cohen’s \(d'\) is of the same order as that of cross-pose variation though somewhat smaller; \(\kappa\) shows a smaller quality effect (\(|d'|{=}0.48\) vs. \(1.13\) for pose). Lower image quality increases intra-identity spread in embedding space and shifts cluster-statistic distributions in the same direction as pose variation. At the same baseline, the quality contribution from LFW to XQLFW corresponds to an AUC change of \(+0.22\), comparable to the pose contribution.

Age. Within AgeDB-30, the Spearman rank correlation \(\rho\) (a non-parametric measure of monotonic association) between within-identity age spread and each cluster statistic (Table 2) is weak and varies in sign across statistics. Only \(\kappa\) exhibits the expected negative association (\(\rho = -0.26\)), under which identities spanning wider age ranges form less concentrated clusters, whereas PairCos is essentially uncorrelated (\(\rho = +0.01\)) and both PenLogit and ProtoCE carry small positive correlations (\(\rho = +0.17\) and \(\rho = +0.33\)). Age-range variation therefore exerts the weakest and least consistent influence on cluster geometry among the confounds examined here.

Ethnicity. A Kruskal–Wallis test (non-parametric one-way ANOVA) across the four RFW demographic groups (African, Asian, Caucasian, Indian) yields \(\epsilon^2 \approx 0.02\) (small) for PairCos, \(\kappa\), and PenLogit, and \(\epsilon^2 = 0.12\) (medium) for ProtoCE. Cluster-statistic values therefore vary across demographic groups within the non-member pool, with ProtoCE again being the most sensitive statistic. Although these effect sizes are modest relative to the \(n_{\text{ids}}\) factor, they indicate that the demographic composition of a non-member benchmark affects the baseline against which membership is measured.

Pose and quality produce the largest shifts in cluster-statistic distributions, with demographic composition and age contributing smaller and, for age, less consistent effects. The matched-pair pose and quality contributions show that the domain-induced component can match or exceed the training contribution itself at higher \(n_{\text{ids}}\), so the threshold-classifier AUC on any non-WebFace4M benchmark cannot be interpreted as membership signal alone.

5.4 Statistic Fusion↩︎

To test whether combining statistics recovers more membership information, we train a two-hidden-layer MLP (\(64{\to}32\) units, ReLU, BatchNorm, dropout 0.3; Adam with \(\eta=10^{-3}\)) on all four cluster statistics. For each epoch-40 configuration we perform five-fold stratified cross-validation with an 80/20 identity split on same-domain WebFace4M identities, reserving 15% of each training fold as a validation set for early stopping.

3pt

Table 3 reports the best single-statistic AUC alongside the fold-averaged MLP AUC. The MLP consistently matches or outperforms the best individual statistic, with the largest gains at \(n_{\text{ids}}{=}50\text{K}\) and \(100\text{K}\) where individual statistics are weakest (one-sided Wilcoxon signed-rank: \(W{=}666\), \(p<10^{-10}\)).

5.5 Limitations↩︎

Generalisation across training data. All experiments use a single training source (WebFace4M). Whether the trends generalise to other training datasets is an open question, as comparably large datasets are not publicly available or have been retracted.

Experimental controls. We do not evaluate defences such as DP-SGD [23] or knowledge distillation [36]. The number of probe images \(k\) per identity varies across benchmarks and is not controlled for, which may affect estimates of the cluster statistics non-uniformly. The held-out split is constructed by random partitioning; in practice, membership boundaries may correlate with identity-level attributes.

Statistical assumptions and scope. Because our four statistics capture only cluster geometry, the measured separability is a lower bound on the membership information available from embeddings; a richer model operating on raw embedding vectors could extract more. The probit-scale translation in Section 5.3 assumes approximately Gaussian member and non-member statistic distributions and stable variances across matched pairs.

6 Conclusion and Future Work↩︎

We evaluated how much membership information is retained in the embedding geometry of 180 open-set face recognition models across nine benchmarks. The results lead to three practical conclusions. First, training-set size has the largest effect on member/non-member separability, while backbone architecture and loss head contribute far less. Training on more identities is therefore the only design choice we tested that substantially reduces the geometric membership signal. Second, cross-domain non-member benchmarks inflate the apparent signal by conflating domain shift with training-attributable differences. Privacy audits should therefore use same-domain held-out references, since cross-domain non-members overestimate the apparent membership signal. Third, even at high \(n_{\text{ids}}\) where individual statistics are weak, fusing all statistics with a learned classifier recovers additional membership information (Table 3). As our statistics capture only cluster geometry, the measured separability is a lower bound on the membership information available from embeddings. Future work includes extending these benchmarks to other backbones and evaluating whether defences such as [23] can reduce the geometric signal without degrading performance.

Acknowledgments↩︎

This work has received funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement No. 101189650 (CERTAIN: Certification for Ethical and Regulatory Transparency in Artificial Intelligence), and the Swiss State Secretariat for Education, Research and Innovation (SERI).

References↩︎