1 Introduction↩︎

In operational contexts, 1:N face identification systems contend with two persistent challenges. The first is image quality. Probes often come from surveillance footage, with poor resolution, blur, occlusion, and other quality problems. Quality has a drastic effect on identification performance: state-of-the-art systems with near-perfect accuracy on high-quality images [1] exhibit FPIRs upwards of 20% and substantial demographic disparities when probe and/or gallery quality is poor [2]–[4].

The second challenge is uncertainty about gallery enrollment. An investigator searching surveillance footage cannot know in advance whether a subject of interest has ever been enrolled. This open-set identification problem—determining whether a probe’s true identity is present in the gallery—is fundamental to investigative workflows. If a probe is mate-absent (MA), any returned candidate is by definition a false positive, which can lead to wasted investigative effort and, in the worst case, a wrongful arrest. Conversely, if a mate-present (MP) probe is misclassified as MA, valid investigative leads are missed.

Existing approaches to classifying gallery membership rely predominantly on similarity scores, either through fixed or adaptive thresholds [5], [6] or through probabilistic modeling of score tail distributions [7]–[9]. However, score distributions shift substantially across operational variables like image quality and gallery structure, and vary across demographic groups  [2]–[4]. A fixed threshold cannot accommodate changing operational variables, and would likely be unfair across demographics. An adaptive threshold would require recalibration for every combination of probe quality, gallery composition, and demographic makeup—infeasible in deployment, since the combination a probe will present is not known in advance.

We therefore introduced 1-consistency, the first and only method based on rank-1 identity consensus across multiple independently trained matchers, rather than the scores of a single matcher [10]. The underlying intuition is: a mate-present probe should produce a stable rank-1 prediction, with its true gallery mate consistently rising to the top across matchers. A mate-absent probe should be less stable, with each matcher’s unique embedding space surfacing a different gallery impostor. (See example in Fig. 2.) The \(k\)-consistency rule, then, labels a probe MP if \(M\) matchers return \(\leq k\) unique identities; 1-consistency is the \(k=1\) case. When compared against two fixed-threshold methods across configurations varying image quality and gallery composition, 1-consistency consistently had the highest MP recall and overall accuracy and the lowest demographic disparities.

This work extends that analysis. The remainder of the paper is organized as follows. Sec. 2 reviews prior MP/MA classification methods. Sec. 3 describes the expanded set of matching configurations. Sec. 4 characterizes \(k\)-consistency performance as consensus variables \(M\) and \(k\) vary (previously fixed at \(M=10\) and \(k=1\)). Sec. 5 establishes the two threshold-based methods, against which 1-consistency is compared in Sec. 6.

1.1 Key Contributions↩︎

• Evaluation of Varied Matching Configurations. We conduct the performance comparison and consensus characterization experiments across 36 (gallery, probe quality) scenarios. Two gallery families isolate the effects of image depth (1–4 images per identity) and identity depth (2,500–12,500 enrolled identities). Four probe quality settings—original, moderate blur (\(\sigma\)=4) or resolution reduction (18×18), high resolution reduction (14\(\times\)​14)—isolate the effects of degradation type versus severity.

• Characterization of Rank Consensus Variables. We characterize the cross-matcher consensus signal across both the number of matchers \(M\) (from 2–10) and the consensus value \(k\) (from 1–10). For degraded probes, increasing \(M\) increases MA recall but decreases MP recall, while increasing \(k\) has the opposite effect. We also observe that for 1-consistent probes (\(k=1\)), the agreed-upon gallery identity is nearly always the correct mate: the rank-1 TPIR is \(\sim100\%\) for original-quality or medium degradation, and only drops to 97% for maximum degradation.

• Comparison to Realistic and Best-Case Thresholds. We compare 1-consistency to two threshold-based methods that optimize Equal Error Rate (EER). Fixed Score-Thresholding (FST), calibrated once in baseline conditions, is included to represent a realistic deployment. FST performance degrades drastically as conditions diverge from the calibration point. Oracle Score-Thresholding (OST), re-calibrated for each (gallery, probe quality) scenario, represents the best-case thresholding deployment—but is purely theoretical, as it would be impossible in practice. For degraded probes, 1-consistency offers the highest accuracy, beating FST by 22.1 pp on average and matching OST within a fraction of a point, with no tuning required.

2 Related Work↩︎

Open-set recognition (OSR) assumes incomplete knowledge of the world at training time: unknown classes may appear during testing [8]. Scheirer et al. posed OSR as a constrained risk-minimization problem and introduced score-calibration methods grounded in extreme value theory (EVT) [7]–[9]. Bendale and Boult extended these ideas to incremental open-world learning [11] and deep networks [12]. General OSR literature, developed largely for object recognition, is now extensive; see  [13] for an overview.

Open-set face identification, by contrast, has received little dedicated attention—even though transduction-based [14] and visitor-interface [15] approaches predate the general formalizations above—and so has no standard terminology or protocol. NIST evaluations frame it as mated versus non-mated search and report two error types: false positives, when a probe with no gallery mate is matched to an enrollee, and misses, when an enrolled probe’s mate is not returned; these are measured by the False Positive and False Negative Identification Rates (FPIR and FNIR) [1], [5]. Other work refers to known unknowns versus unknown unknowns [6], and our own recent work used in-gallery and out-of-gallery [10], [16], which we now term MP and MA, respectively. In the absence of a canonical taxonomy, we group prior work by its general methodological approach.

The most direct approach thresholds a similarity score: a probe whose top score falls below a fixed value is labeled MA [1], [5]. This is simple but brittle, because the ideal threshold shifts with the matcher and operational/demographic variables and so must be re-derived for each deployment. A large body of work therefore replaces the raw score with a calibrated quantity. EVT models the tail of the score distribution to estimate the probability that a top match comes from a genuine rather than an impostor identity: meta-recognition [7], the 1-vs-set machine [8], and the CAP and W-SVM models [9] all produce such calibrated decision boundaries, and the Extreme Value Machine [17] and later operational refinements [18] continue this line. Günther et al. show that thresholding EVM probabilities outperforms thresholding verification-like scores on an LFW-based open-set protocol [6]. Even these methods, however, still depend on a decision boundary placed on a score-derived quantity, and so can be sensitive to the same operational drift in quality and gallery composition; their calibration rests on distributional assumptions that may not hold across deployment conditions.

Deep open-set methods instead build rejection into the network. OpenMax replaces the softmax layer with an EVT-calibrated estimate of the unknown-class probability [12], and maximal-entropy and objectosphere losses train the network to respond with low confidence to unknowns [19]. These improve robustness but still calibrate a score-derived decision, and they require training with curated known-unknown examples. Such data is difficult to assemble representatively by construction, since the unknowns a system meets in deployment are precisely the ones it did not see during training [6], [12].

A related line improves identification robustness by aggregating the multiple enrolled images of an identity rather than by rejecting unknowns. These range from simple averaging of embeddings to learned fusion: heterogeneous feature fusion [20], neural aggregation [21], cluster-and-aggregate for large probe sets [22], probabilistic embeddings [23], and sparse-expert fusion [24]. They are typically evaluated on unconstrained benchmarks such as IJB-A [25], IJB-B [26], and IJB-S [27], and reliably raise accuracy under pose and quality variation. But they address the accuracy of the match, not the enrollment decision, and the fused score is still subject to the limits of thresholding.

Our recent works leverage rank information. In [16], we classify a probe as MP or MA using the ranks of the other enrolled images of the rank-1 identity, training a classifier on these rank-feature vectors. This approach avoids distributional assumptions and is agnostic to the MP/MA mix, but it needs more than one enrolled image per identity and still trains a dedicated classifier. Our other prior work introduced rank-1 identity consistency across independently trained matchers [10], which the present paper extends.

We know of no work other than our own [10] that uses matcher consensus to classify a 1:N matching result as MP or MA. Additionally, while several of the works above evaluate on unconstrained imagery, only ours [10], [16] characterize performance as a controlled function of probe characteristics. For in-the-wild datasets, quality factors co-occur and cannot be varied independently, so a change in performance cannot be attributed to any single cause. Our evaluation instead applies controlled degradations to a fixed set of images, following the approach used by NIST [28]. This isolates the effect of each degradation type and severity while holding gallery size and composition fixed.

3 Matching Configurations↩︎

The matching configurations described in this section are used across characterization and comparison experiments. Each is defined by two variables: the gallery being searched against and the quality of the probe image.

Probe and gallery images are from MORPH v5  [29], [30]: mugshot-quality images captured under controlled conditions (frontal pose, neutral expression, consistent lighting) and annotated with ground-truth demographic information.

We curate two non-overlapping subsets of the Black Male (BLM) cohort, totaling 12,926 unique identities. The “probe set” contains 2,123 identities with 5+ images; each identity’s most recent image is its probe and its gallery mate(s) are selected from among older images. The “gallery set” contains 10,803 identities with 4+ images; these populate the gallery as needed.

3.1 Gallery Families↩︎

We construct two gallery families that isolate axes of composition variation. The first axis is image depth \(x\)—number of enrolled images per identity. The second axis is identity depth \(y\)—number of identities enrolled in the gallery.

In the image depth galleries (denoted \(E_x\)), we fix identity depth (\(y=12{,}926\)) and systematically increase \(x\) from 1 to 4. Each gallery \(E_x\) is a strict superset of the previous one (see Fig. 3), yielding the following total image counts: 12,926 (\(E_1\)), 25,852 (\(E_2\)), 38,778 (\(E_3\)), and 51,704 (\(E_4\)).

In the identity depth galleries (denoted \(I_y\)), we fix image depth (\(x=2\), the average image depth used by NIST in [1]) and systematically increase \(y\) from 2,500 to 12,500 in steps of 2,500. The two images used for each identity correspond to those in the \(E_2\) experiment.

3.2 Probe Quality Levels↩︎

Starting with the original good-quality MORPH v5 images (abbreviated ORG), we apply two kinds of degradations to represent low-quality operational probes.

Gaussian blur simulates degradation due to defocus. The \(\sigma\) parameter controls blur severity, with higher values indicating more severe degradation. NIST quality testing uses a range of \(\sigma = 1\)–\(7\) [28]; we use the middle value \(\sigma = 4\) as the moderate severity blur (MS-B) condition.

Downsampling–upsampling simulates native low-resolution capture. We use \(18 \times 18\) for a moderate severity resolution degradation (MS-R), as prior work [4] showed its impact on recognition to be roughly equivalent to the selected \(\sigma = 4\) blur. We use \(14 \times 14\) for a high severity resolution degradation (HS-R). The corresponding interpupillary distances (pixels between the eyes) are approximately 6 px for MS-R and 5 px for HS-R.

Comparing MS-B and MS-R isolates the effect of degradation type; comparing MS-R and HS-R isolates the effect of severity. Fig. 4 shows the relative quality of the original (a) and degraded (b–d) images.

The nine galleries and four quality levels yield 36 unique (gallery, probe quality) scenarios. To keep results concise and readable, we adopt two presentation conventions. First, because gallery composition affects performance far less than probe quality (as we show throughout), many tables and figures report only the “bookend” galleries: \(E_1\) and \(E_4\) for the image depth family, \(I_{2500}\) and \(I_{12500}\) for the identity depth gallery. Second, because degradation type matters far less than severity (likewise shown throughout), we often omit MS-B and report ORG, MS-R, and HS-R to capture the severity range.

4 Characterizing Rank Consensus↩︎

In this section, we isolate each consensus variable in turn: number of matchers \(M\) and consensus value \(k\)—the maximum number of unique rank-1 identities for an MP probe. Our previous work fixed \(M=10\) and \(k=1\). Here, we first fix \(k=1\) and measure how 1-consistency performance varies as \(M\) increases from 2 to 10. Then, fixing \(M=10\), we vary \(k\) from 1 to 10 to examine its effect on classification.

4.1 Varying the Number of Matchers \(M\)↩︎

The matchers used here (and in [10]) are 10 independently trained instances of AdaFace [31] from [32]; AdaFace is chosen because it is particularly suited to low-quality images. The instances share a backbone (ResNet100 [33]) and training set (WebFace4M [34]) and differ only in their starting weights. Using multiple matchers raises a natural question: how does the number of matchers affect performance? Would \(M<10\) yield the same results, and would \(M>10\) improve them?

We evaluate 1-consistency across size-\(M\) subsets of the 10 matchers, for \(M \in \{2, \dots, 10\}\). At each \(M\) we draw 20 random subsets without replacement and report the mean and standard deviation of MP recall (\(R(\text{MP})\)), MA recall (\(R(\text{MA})\)), and accuracy (which is simply the average of the two). Fig. 5 shows overall results, with performance at three \(M\) values highlighted in Tab. 1.

As \(M\) increases:

• R(MP) remains consistently near 100% for ORG probes, but decreases for degraded probes by 5–11 pp for MS-R and 22–24 pp for HS-R.

• R(MA) increases at a similar rate across quality levels (by roughly 29–32 pp for ORG, 25–29 pp for MS-R, 22–26 pp for HS-R), though its starting value at \(M=2\) is higher for more degraded probes (60–65% for ORG up to 71–76% for HS-R).

• Accuracy continually increases for ORG probes (by 14–16 pp, still rising at \(M=10\)), plateaus for MS-R probes (around \(M=8\)), and for HS-R probes peaks early (\(M=3-4\)) before declining slightly (1–2 pp) through \(M=10\).

Overall, curve shapes couple tightly with probe quality and are relatively consistent across galleries. However, the starting points for degraded probes (at \(M=2\)) reveal a consistent gallery ordering. R(MP) orders the galleries \(I_{2500} > E_4 > I_{12500} > E_1\) at both degraded quality levels. R(MA) shows the inverse: \(E_4, I_{12500},\) and \(E_1\) cluster higher while \(I_{2500}\) sits visibly lower. Accuracy reflects the balance of the two—it follows the R(MP) ordering at HS-R, but at MS-R the low R(MA) of \(I_{2500}\) pulls it down to third. This inversion reflects gallery size: with the fewest enrolled identities, \(I_{2500}\) offers the least opportunity for a lookalike to outrank a true mate (raising R(MP)) but also the fewest chances for matchers to disagree on an impostor (lowering R(MA)).

No single \(M\) is optimal everywhere, so the choice is a trade-off. A deployment can tune \(M\) for better R(MA), reducing wasted investigative effort, or R(MP), reducing missed leads—higher \(M\) favors the former, lower \(M\) the latter.

4.2 Varying the Consensus Value \(k\)↩︎

Recall we have used \(k=1\); one might ask what happens at higher values. Tab. 2 shows the mean number of unique rank-1 identities for MP and MA probes across configurations. For both probe types the mean rises as quality degrades, but the two move differently. MA counts rise gradually and modestly (about +0.6 then +0.4 across the two degradation steps). MP counts barely move from ORG to MS-R (about +0.3) but jump sharply at HS-R (about +1.8), as severe degradation finally breaks the cross-matcher agreement that holds for milder degradation. The MP and MA means stay well separated at each quality level, though the gap narrows mainly at HS-R.

Within any single (gallery, probe quality) configuration, the MP and MA means are cleanly separated—a \(k\) value can be chosen that splits them. But an operational deployment must handle images of varying quality, and no single \(k\) separates them all: a very low-quality MP probe can produce as many unique identities as a good-quality MA probe, so their counts overlap.

To see how this plays out across the full range of \(k\) values, we sweep \(k\) from 1 to 10 (Fig. 6). As probe quality decreases (left to right), the \(k\) that maximizes accuracy and the \(k\) that equalizes MP and MA recall (the Equal Error Rate, EER) diverge. For ORG probes the two coincide at \(k=1\); for HS-R probes the EER-optimal \(k\) is consistently one step above the accuracy-optimal \(k\). As with \(M\), \(k\) trades the two recalls against each other: a higher \(k\) admits more probes as MP, favoring R(MP), while a lower \(k\) favors R(MA).

5 Threshold Methods↩︎

We now describe the score-thresholding methods we will evaluate 1-consistency against. We are not aware of a standardized operational protocol for MP/MA classification; NIST’s evaluations [1], for example, characterize 1:N error rates (FPIR/FNIR) as a function of operating threshold rather than prescribing a fixed decision rule. We therefore evaluate against two score-thresholding regimes that bracket what a deployed system could achieve, one realistic and one best-case but theoretical.

Fixed Score-Thresholding (FST) sets a single threshold once and applies it unchanged to every scenario. This reflects the constraint any deployed score-thresholding system operates under: the threshold must be fixed before any probe is seen, as per-probe recalibration is not feasible operationally. It is also the regime of our prior work [10].

Oracle Score-Thresholding (OST) is re-tuned to the optimal threshold for each (gallery, probe quality) combination. It is not deployable—it presumes per-scenario knowledge of the MP and MA score distributions, the very labels the system exists to predict—but it bounds the best performance any score threshold could attain, serving as a ceiling against which both FST and 1C can be compared.

The same tension between accuracy-maximizing and error-balancing operating points observed for \(k\) arises in setting the actual score threshold for either method. Equal Error Rate (EER) places the threshold where the MP and MA error rates are equal; Maximum Accuracy (MaxAcc) places it wherever overall accuracy is highest. We resolve the choice with respect to OST, where the threshold is optimal for each scenario by construction, so the comparison between criteria is not confounded by calibration error.

Tab. 3 reports OST performance for EER and MaxAcc criteria. For ORG probes the two are effectively identical: the accuracy-maximizing threshold already balances the error rates, with an accuracy difference below 0.03 pp. The criteria diverge only under degradation. MaxAcc’s accuracy advantage is small—at most 2.8 pp under HS-R, and roughly 0.9 pp averaged across all 36 scenarios. What it gives up for that advantage is not: to recover its last fraction of accuracy, MaxAcc drives the threshold until MP recall collapses—the R(MP)/R(MA) gap under HS-R runs 25 to 43 pp, with R(MP) as low as 46% while R(MA) sits in the mid-to-high 80s—whereas EER holds the two recalls within 1.3 pp. The mechanism is direct: under severe degradation the MP score distribution shifts down into the MA range, so the only way to raise accuracy further is to reject nearly everything near the overlap, most of which is MP.

Because the accuracy gained is marginal and the recall imbalance severe, we adopt EER for both OST and FST throughout. This also keeps FST consistent with the fixed-EER threshold of our prior work [10]. FST is calibrated in the baseline (\(E_2\), ORG) scenario; the single threshold is 0.495. The tuned thresholds for OST are given in Tab. 4.

6 Comparing Classification Performance↩︎

Finally, we compare the three methods across all scenarios. Tab. 5 reports results with signed 1C–OST difference (\(\Delta\) block); Fig. 7 shows the same comparison averaged within each gallery family. MS-Blur tracked MS-Res within 2–3 pp on every metric, so the figures follow the ORG \(\rightarrow\) MS-R \(\rightarrow\) HS-R severity ladder.

6.1 1-Consistency versus the Oracle Score Threshold↩︎

The accuracy comparison reverses with probe quality, and the reversal is the central result. For ORG probes OST leads 1C by a few points. The reason is in the component recalls: 1C’s MP recall is saturated near 100%, so accuracy is governed by MA recall, and 1C’s MA recall at ORG is only about 92–94%. The strict unanimity rule (i.e., \(k=1\)) occasionally rejects a clean MA probe whose ten matchers happen to disagree, and with nothing moving on the MP side, that lost MA recall accounts for the accuracy gap.

As soon as quality degrades the ordering inverts: 1C holds the best or tied-best accuracy in 25 of 27 degraded-probe scenarios. Under degradation, OST’s MA recall falls steeply—to roughly 62–72% at HS-R—as degraded MP scores intrude into the range OST must accept, dragging MA probes with them. 1C’s MA recall, by contrast, stays near 97–98% throughout, because the matchers are rarely unanimous on an MA probe’s rank-1 identity regardless of where its scores fall.

The 1C–OST MA margin scales with severity: as small as 1.2 pp under the mildest degradation and as large as 35.8 pp at the most severe. The ordering holds across every gallery in both families—at fixed quality the spread across galleries is a few points, while across quality within a fixed gallery it is tens of points.

In accuracy, the two methods are nearly indistinguishable under degradation: \(\Delta\)Acc stays within 1.3 pp of zero across all degraded scenarios, as 1C’s MP loss and MA gain relative to OST very nearly cancel. The Accuracy panel of Fig. 7 shows this directly—1C and OST track together across the ladder. What separates them is not how often they err but which error they make: the \(\Delta\) block shows \(\Delta\)R() negative and \(\Delta\)R(MA) positive under degradation, of similar magnitude, with \(\Delta\)Acc near zero between them.

6.2 Fixed Score-Thresholding↩︎

FST applies its single threshold of \(s=0.495\), calibrated in the (\(E_2\), ORG) scenario, unchanged everywhere. For probes matching the calibration point it performs comparably—ORG accuracy within about a point of 1C and OST. As quality moves away from the calibration point the fixed threshold sits far above the degraded MP distribution, and the consequence is severe: at HS-R, FST’s MP recall falls to between 0.6% and 1.7%—it rejects very nearly every genuine mate—and its accuracy drops by up to 49.4 pp relative to its own ORG performance. In Fig. 7, the FST MP line falls off the bottom of the axis.

6.3 Identity Correctness of MP Decisions↩︎

The comparison so far asks whether a probe with a gallery mate is labeled MP. A separate question remains: when a method does label a probe MP, is the identity it returns the correct mate? We measure this as the true positive identification rate (TPIR) conditional on an MP label—of the MP-labeled probes, the fraction whose returned identity is the true mate. 1-consistency is an ensemble decision (one label per probe across all M matchers), so its returned identity is the single consensus identity. The threshold methods are single-matcher decisions; consistent with the recalls reported above, we evaluate them per probe and per matcher, asking for each run whether a score above the threshold was attached to the correct mate. Fig. 8 pairs R(MP) with this conditional TPIR for the three methods.

When 1-consistency labels a probe MP, the consensus identity is almost always the true mate: TPIR is 100% for ORG probes and stays at 97–99% even for HS-R. Unanimous agreement across ten matchers is rarely agreement on a wrong identity. OST is likewise near-perfect at ORG (100%) and MS-R (96–99%), but at HS-R its conditional TPIR falls to 66–84%. Under severe degradation, a substantial share of the probes it accepts are matched to an impostor rather than the true mate. FST’s conditional TPIR is 100% throughout, but this must be read against its R(MP): at HS-R it labels only 1–2% of MP probes as MP, and the few probes that clear its high fixed threshold are precisely the unambiguous ones whose top match is correct. Its perfect identity correctness is therefore a consequence of accepting almost nothing, rather than a strength.

Read together, the two rows separate the methods by the character of their MP decisions. 1-consistency accepts a substantial fraction of degraded MP probes and is almost always right about who they are. OST accepts a comparable or larger fraction but, under severe degradation, is markedly less reliable about identity. FST accepts so few that the question of identity correctness is moot.

7 Discussion↩︎

When methods tie in accuracy, they differ in which error they make. Any gallery-membership method trades MP recall against MA recall: pushing one up generally pulls the other down. Under degradation, 1-consistency holds the best or tied-best overall accuracy in 25 of 27 scenarios (the two exceptions trail the oracle by less than 0.3 pp). Relative to OST, it loses on MP recall but gains nearly the same on MA recall, so the two methods’ accuracies never differ by more than 1.3 pp. 1-consistency errs toward missing a MP probe, while a score threshold errs increasingly toward accepting an MA one. Which error is more tolerable is deployment-specific, and is a choice the operator must make rather than one the method should impose.

No fixed threshold can generalize across quality. A real deployment cannot recalibrate per probe. It would need to commit to threshold before any probe is seen, which is the constraint FST captures. When probe quality matches the calibration point, FST performs comparably to the other methods; when it does not, FST does not degrade gracefully but collapses, with MP recall falling to 0.6–1.7% under HS-R and accuracy dropping up to 49.4 pp. The oracle explains why this is unavoidable rather than incidental. OST is the best any threshold could do given the “optimal value” for every scenario. Yet that threshold value ranges from 0.296–0.519 across our scenarios, varying almost entirely with probe quality (ORG thresholds occupy 0.46–0.52, HS-R 0.30–0.33) and only slightly across galleries. The threshold a deployment would need is thus a function of the operational conditions it encounters, which are not known in advance—they are unknown unknowns.

When 1-consistency accepts a probe, it is more reliable about identity than the oracle score threshold. Accuracy and recall measure whether the membership decision is right, but not whether the returned identity is. Conditional on labeling a probe mate-present, 1-consistency surfaces the true mate in 97–100% of cases across all qualities, because ten matchers rarely agree unanimously on a wrong identity. The oracle, despite its per-scenario tuning, falls to 66–84% under HS-R: a substantial share of the probes it accepts are high-scoring impostors rather than true mates. This is the one axis on which 1-consistency does not merely match the oracle but exceeds it—the oracle is a ceiling on accuracy, not on identity reliability. (FST’s perfect conditional identity rate is not a counterexample: it accepts so few degraded probes, 1–2% at HS-R, that the measure is moot.)

Probe quality outweighs the gallery variation we test. Across every analysis, the dominant axis of variation is probe quality. For fixed quality, metrics vary by a few points across the nine galleries spanning both structural axes. For fixed gallery, the same metrics vary by tens of points across the four quality levels. This dominance holds over the ranges we vary, but should not be read as gallery structure being unimportant in general, because those ranges are narrower than operational reality on every axis. Our largest gallery (\(E_4\) with \(\sim\)​51k images of \(\sim\)​13k identities) is roughly three orders of magnitude smaller than that used by NIST in  [1] (\(\sim\)​26M images of \(\sim\)​12M identities), and up to six orders of magnitude smaller than some commercial galleries—Clearview AI, for instance, reports over 70 billion images in its law enforcement database [35] (though not the number of identities this corresponds to). We also enroll every identity at a uniform image depth, whereas operational galleries enroll unevenly—some identities with many images, others with few—and this imbalance itself affects recognition [3]. Both axes we vary are thus academic samples of operational conditions: we can neither experiment at deployed gallery scales nor reproduce the mixed, co-occurring degradations of real probes. Our controlled single-degradation sweep isolates causes precisely because it does not attempt to mimic that uncontrolled mixture. Whether quality continues to dominate once these axes reach operational scale—where a larger impostor population gives MA probes more chances to draw spurious agreement on a single look-alike—remains open.

7.1 Limitations↩︎

• These results use synthetic degradations on good-quality imagery. Real surveillance probes additionally exhibit pose, occlusion, expression, and illumination variation, typically in combination.

• This study evaluates a single demographic cohort (BLM) and therefore does not test the cross-demographic fairness behavior we previously reported [10].

• Our ten matcher instances are homogeneous by construction: they share a backbone, training set, and loss, differing only in initialization. Consensus among more diverse matchers—varying the architecture, the training data (e.g., independently sampled subsets), or the loss—may behave differently, and a pool could also be assembled by selecting the most diverse matchers from a larger set. Whether such diversity strengthens or weakens the consensus signal is untested.

8 Conclusion↩︎

This paper extends the 1-consistency method for MP/MA classification in operational 1:N face identification [10]. By separating gallery structure into image depth and identity depth, we show that probe quality, not gallery composition, drives classification performance across the configurations tested. Against both a fixed threshold set once (FST) and an oracle threshold tuned per scenario (OST), 1-consistency holds the best or near-best accuracy in nearly all degraded scenarios, with no calibration required. And when it accepts a probe, it identifies the correct mate more reliably than even the oracle (97–100% versus 66–84% under severe degradation). The advantage over a threshold is not higher accuracy so much as the removal of a commitment a deployment cannot honor: a single threshold cannot track the probe-quality regime it will encounter, while rank-1 consensus requires none over a wider range of probe quality.

This last point extends to operational reality. We are aware of no standard for the MP/MA decision—in the literature or in deployed practice—and operational systems may return rank-1 candidates with no explicit enrollment check at all. That absence is itself the risk this work addresses: an unscreened MA search yields a confident false match, with no signal that the returned identity is spurious. A method needing no threshold and no per-condition tuning lowers the barrier to making this decision at all, which may matter as much as which method is used.

Several directions remain open. The limitations above each point to a next step—evaluation on real surveillance probes rather than synthetic degradations, a consensus pool diversified in architecture, training data, or loss (or selected for diversity from a larger set) rather than instances of a single model, and larger-scale galleries on both the identity and image-depth axes. Beyond these, one direction is internal to the method: 1-consistency exposes two consensus controls, the number of matchers \(M\) and the consensus value \(k\), each trading MP against MA recall. We characterized each in isolation, but how they interact—and whether tuning them jointly widens the operating range a deployment can reach—is untested. Of the gallery-scale questions, the sharpest is whether probe quality continues to dominate as the impostor population grows: a larger gallery gives MA probes more opportunities to draw coincidental cross-matcher agreement on a single look-alike, the one regime in which 1-consistency’s MA advantage could erode.

References↩︎