MGFace: Mask-Gated Face Matching via Conditional Similarity Routing
July 14, 2026
Abstract
Face identification has achieved remarkable performance under normal conditions. Yet, its accuracy often degrades significantly when query faces are partially occluded, especially by facial masks. Existing re-ranking approaches improve robustness by exploiting patch-level similarities. Still, they often rely on costly, fine-grained matching mechanisms, which limit their efficiency in large-scale retrieval scenarios. In this paper, we propose MGFace, a mask-gated face identification pipeline that predicts the mask status of a query face and conditionally routes the similarity computation accordingly. Specifically, MGFace distinguishes between masked and unmasked queries, applies global embedding matching to unmasked queries, and activates mask-aware patch-level re-ranking only for masked queries. This design focuses on reliable upper-face regions while avoiding unnecessary fine-grained computation. Experiments on the extended LFW-Mask dataset show that MGFace achieves over 80% identification accuracy with the FaceNet backbone and over 90% with the ArcFace backbone. Compared with a previous EMD-based re-ranking method, MGFace achieves better identification performance while reducing query time by approximately 20\(\times\). These results demonstrate the effectiveness of MGFace in improving masked-face identification accuracy with low computational overhead. The source code is available at https://github.com/chequanghuy/MGFace.
MGFace, FaceID, Masked Face, Computer Vision, Re-ranking
1 Introduction↩︎
Face identification [1]–[5] is a fundamental computer vision task that aims to identify a person from facial images. It has been widely deployed in real-world applications such as access control, surveillance, and intelligent attendance systems [6]. As multimedia databases continue to grow in scale, practical face identification systems must satisfy two requirements simultaneously: they must remain accurate under challenging visual conditions and efficient enough for large-scale retrieval.
Recent deep face recognition models achieve strong performance under common variations such as pose, illumination, and age [7], [8]. However, their accuracy can degrade significantly when discriminative facial regions are occluded by masks or sunglasses [9], [10]. masked-face identification is especially challenging because masks hide the lower face and remove important identity cues; previous work [9] reports that identification accuracy can drop from 98.41% to 39.79% when the query face is masked. To improve robustness, existing two-stage methods [9], [10] first retrieve candidates using global embeddings and then refine the ranking with patch-level similarity. Although effective, these methods often incur costly local matching for all queries, even for unmasked faces where global embeddings are already reliable. Moreover, patch-level matching may include occluded lower-face regions, which can introduce unreliable similarity signals. Thus, existing re-ranking methods [9], [10] improve masked-face retrieval but do not explicitly address when fine-grained matching should be used or where it should focus.
In this paper, we propose MGFace, a mask-gated face identification pipeline for efficient masked-face retrieval. MGFace predicts the query mask status using a lightweight head integrated into a pretrained face recognition backbone, and then routes the query to an appropriate matching branch. Unmasked queries are matched directly with global cosine similarity, while masked queries are processed by a two-stage mask-aware branch that retrieves top-\(k\) candidates and re-ranks them using patch-level similarity over reliable upper-face regions. This design applies fine-grained matching only when necessary and avoids unreliable occluded patches. Figure 1 illustrates the effectiveness of the proposed method, achieving higher accuracy with significantly lower query times.
Our main contributions are summarized as follows: (1) We propose MGFace, a mask-gated face identification pipeline that augments pretrained face recognition models with a lightweight mask classification head to predict whether a query
face is masked or unmasked. (2) We introduce a conditional similarity routing strategy that adaptively selects the matching process according to the predicted mask status: unmasked queries are matched using global
cosine similarity, while masked queries are processed by mask-aware patch-level re-ranking over reliable upper-face regions. (3) We evaluate MGFace on the extended LFW-Mask dataset with different pretrained backbones, demonstrating that it
improves face identification accuracy under masked-face conditions while reducing computational cost compared with existing re-ranking-based methods.
2 Related work↩︎
Face identification methods commonly rely on CNN-based [1]–[3], Transformer-based [5], or hybrid architectures [10] as backbone networks to extract deep facial representations. These models are typically trained using classification-based learning [1], [2] or metric learning objectives [3], [4], which encourage images from the same identity to be close in the embedding space while pushing images from different identities farther apart. Although such methods achieve strong performance on in-distribution benchmarks [7], [8], their accuracy often degrades under out-of-distribution conditions, such as masked faces and sunglasses. Facial masks are a common and challenging form of occlusion, particularly in real-world identification systems where masked queries became widespread during the COVID-19 pandemic. Consequently, face-related tasks involving masks [11]–[13], especially masked-face identification, have attracted increasing research attention. To address masked-face identification, prior works retrain models on synthetic masked images [14], [15], reconstruct occluded regions [16], [17], or aggregate multiple features [18], [19]. However, these methods often require extra training, may introduce unreliable information, or increase model complexity. In addition, feature aggregation methods [18], [19] have been explored to enrich visual representations, while recent re-ranking strategies aim to improve identification under OOD scenarios [9], [10], [20]. For example, DeepFace-EMD [9] exploits Earth Mover’s Distance to compare patch-level features, thereby improving robustness but substantially increasing computational cost during re-ranking. Hybrid-ViT [10] uses Transformer features and requires additional training on masked-face data. These methods also apply fine-grained matching to all queries, causing unnecessary computation for unoccluded faces. A straightforward solution is to use an addition classifier [12], [21] to select matching strategies based on query conditions. However, employing an additional standalone classifier introduces extra inference overhead.
In contrast, our work proposes a mask-gated face identification pipeline that integrates a lightweight mask classification head into a pretrained face recognition model. This enables efficient conditional routing between global matching and mask-aware local re-ranking, improving robustness under masked-face scenarios while reducing unnecessary computation.
3 Method↩︎
In this section, we introduce MGFace, a mask-gated face identification pipeline that adaptively selects the matching strategy according to the mask status of the query face. Instead of applying a uniform matching procedure to all queries, MGFace first estimates whether the query face is masked and then routes it to an appropriate identification branch. For unmasked queries, the full facial region is considered reliable, and identification is performed using the global matching branch. For masked queries, where the lower facial region may be occluded, MGFace employs the two-stage mask-aware branch. It first retrieves the top-\(k\) gallery candidates using global image-level similarity and then re-ranks these candidates by computing patch-level similarity over reliable facial regions after mask-aware patch filtering. Figure 2 provides an overview of the proposed mask-gated face identification pipeline.
3.1 Extending pretrained face recognition models↩︎
To separately process masked and unmasked face images for identity recognition, a straightforward approach is to employ an additional mask classification model alongside the face recognition model. However, such a design substantially increases
inference latency, as the system must sequentially execute two separate models: one for mask-status classification and another for facial feature extraction. To address this limitation, we propose an integrated architecture built upon a pretrained face
recognition model. Specifically, the pretrained face recognition backbone is frozen, and an additional classification head is attached to the backbone’s convolutional layer to predict two classes: masked and unmasked. The
classification branch applies global average pooling followed by fully connected layers to estimate the probability of whether a face is masked. This design enables simultaneous facial feature extraction and mask-status classification within a single
forward pass, thereby reducing computational overhead compared with using two independent models. Figure 3 illustrates the proposed architecture. Given an input image, the proposed model produces three outputs:
\[(f, P, p)=\mathcal{F}(\cdot)\]
where \(f \in \mathbb{R}^{d_f}\) is the global feature of the image, \(P \in \mathbb{R}^{d_p \times N \times N}\) is the patch-level features, and \(p \in \{\texttt{masked}, \texttt{unmasked}\}\) is the predicted mask status.
3.2 Mask-Gated Face Identification↩︎
Given a query face image \(q\), MGFace represents it as \((f_q, P_q, p_q)\), where \(f_q\) is the global feature, \(P_q\) is the patch-level feature map, and \(p_q\) is the predicted mask status. Similarly, each gallery image \(g_j\) in the gallery set \(\mathcal{G}=\{g_j\}_{j=1}^{n}\) is represented as \((f_j, P_j)\). Based on \(p_q\), MGFace routes the query to one of two matching branches:
\[\mathcal{R}(q,\mathcal{G})= \begin{cases} \mathcal{R}_{\text{global}}(q,\mathcal{G}), & p_q=\texttt{unmasked},\\ \mathcal{R}_{\text{patch}}(q,\mathcal{G}), & p_q=\texttt{masked}. \end{cases}\]
Here, \(\mathcal{R}_{\text{global}}(q,\mathcal{G})\) denotes the ranked list produced by global feature matching, where gallery images are sorted according to their cosine similarity with the query global feature. In contrast, \(\mathcal{R}_{\text{patch}}(q,\mathcal{G})\) denotes the ranked list produced by the two-stage mask-aware matching branch. This branch first retrieves the top-\(k\) candidates using global similarity and then re-ranks them using patch-level similarity computed only over reliable facial regions.
3.2.1 Global Face Matching for Unmasked Queries↩︎
When the query image is predicted as unmasked, the entire face is considered reliable for identity matching. Therefore, MGFace directly computes the cosine similarity between the query global feature \(f_q\)
and each gallery global feature \(f_j\):
\[\mathbf{S}_{\text{global}}(q,\mathcal{G}) = \left\{ \cos(f_q,f_j) \right\}_{j=1}^{n}, \quad \cos(f_q,f_j) = \frac{f_q^\top f_j}{\|f_q\|\|f_j\|}.\]
The final ranked list is obtained by sorting all gallery images according to their global similarities:
\[\begin{align} \mathcal{R}_{\text{global}}(q,\mathcal{G}) &= \mathrm{Sort} \left[ \mathbf{S}_{\text{global}}(q,\mathcal{G}) \right] \\ &= \{g_{r_1},g_{r_2},\dots,g_{r_n}\}, \\ \text{s.t.}\quad \cos(f_q,f_{r_1}) &\geq \cos(f_q,f_{r_2}) \geq \dots \geq \cos(f_q,f_{r_n}). \end{align}\]
This branch avoids unnecessary patch-level computation for normal face images, thereby improving efficiency.
3.2.2 Two-Stage Mask-Aware Matching for Masked Queries↩︎
When the query image is predicted as masked, the lower facial region may contain unreliable information due to occlusion. To improve robustness, MGFace performs matching in two stages: global candidate retrieval followed by mask-aware
patch-level re-ranking.
3.2.2.1 Stage 1 - Global candidate retrieval
MGFace first computes the global cosine similarity between the query and all gallery images. The top-\(k\) most similar gallery images are then selected as candidate matches:
\[\begin{align} \mathcal{G}_k(q,\mathcal{G}) &= \mathrm{TopK} \left[ \mathbf{S}_{\text{global}}(q,\mathcal{G}) \right] \\ &= \mathcal{R}_{\text{global}}(q,\mathcal{G})_{[1:k]}, \quad k<n \\ &= \{g_{r_1},g_{r_2},\dots,g_{r_k}\}. \end{align}\]
This stage provides an efficient coarse retrieval step and reduces the search space for subsequent patch-level re-ranking.
3.2.2.2 Stage 2 - Mask-aware patch-level re-ranking
MGFace then re-ranks the top-\(k\) candidates using patch-level features. Since face images are aligned using facial landmarks before feature extraction, their spatial layouts are relatively consistent. This allows MGFace to approximately localize occlusion-prone regions in the patch-level feature map.
For masked faces, the lower facial region is typically occluded, while the upper region, including the eyes and forehead, remains visible and informative for identity matching. Therefore, MGFace applies a mask-aware patch filter that retains only the upper spatial rows of the patch-level feature map. Given a patch-level feature map \(P \in \mathbb{R}^{d_p \times N \times N}\), the filtered feature map is defined as:
\[\tilde{P} = P_{[:,\,1:M,\,1:N]} \in \mathbb{R}^{d_p \times M \times N}, \quad M=\left\lfloor\frac{N+1}{2}\right\rfloor.\]
This process is illustrated in Figure 4. The cropped feature map is then flattened along the spatial dimensions into a sequence of \(L\) patch vectors:
\[\tilde{P} = \{p^i \in \mathbb{R}^{d_p}\}_{i=1}^{L}, \quad L=M\times N.\]
For each candidate \(g_{r_j} \in \mathcal{G}_k(q,\mathcal{G})\), MGFace computes the average cosine similarity between corresponding valid patch pairs after mask-aware filtering:
\[S_{\text{patch}}(q,g_{r_j}) = \frac{1}{L} \sum_{i=1}^{L} \cos(p_q^i,p_{r_j}^i).\]
The patch-level similarities for the top-\(k\) candidates are denoted as \(\mathbf{S}_{\text{patch}}(q,\mathcal{G}_k) = \{S_{\text{patch}}(q,g_{r_j})\}_{j=1}^{k}\). Finally, the top-\(k\) candidates are re-ranked according to their patch-level similarities, while the remaining gallery images keep their original global-similarity order:
\[\mathcal{R}_{\text{patch}}(q,\mathcal{G}) = \mathrm{Sort} [ \mathbf{S}_{\text{patch}}(q,\mathcal{G}_k) ] \oplus \mathcal{R}_{\text{global}}(q,\mathcal{G})_{[k+1:n]},\]
where \(\oplus\) denotes list concatenation.
4 Results↩︎
4.1 Settings and Datasets↩︎
4.1.1 Datasets↩︎
We evaluate our method on the LFW-Mask dataset [9], which contains unmasked gallery images and synthetic masked query images generated from the original faces. To better reflect real-world scenarios, we extend the query set by combining both masked and unmasked images, while keeping the gallery set unchanged with only unmasked faces. During evaluation, duplicate images and pairs of synthetic masked images with their original counterparts are removed from the retrieval results. After retaining only identities with multiple images, the final extended dataset contains 1,659 identities. To train the mask classification head, we use the Mask Classifier dataset1, which includes 2,757 training images and 1,276 validation images. The pretrained face recognition backbone is frozen, and only the mask classification head is optimized during training.
4.1.2 Settings↩︎
To evaluate the effectiveness of the proposed method, we conduct experiments with different pretrained face recognition backbones, including FaceNet [3] and ArcFace [1]. Following the input requirements of each backbone, face images are resized to \(128 \times 128\) for ArcFace and \(160 \times 160\) for FaceNet. Unless otherwise stated, the number of candidates for the re-ranking stage is set to \(k\)=100. The default patch grid size is \(3 \times 3\) for FaceNet and \(4 \times 4\) for ArcFace. We adopt common face identification evaluation metrics, including Precision@1 (P@1), R-Precision (RP), and MAP@R (M@R), following prior works [9], [10], to assess identification accuracy. In addition, we report the average query time to evaluate the computational efficiency of the proposed method.
4.2 Face mask classification↩︎
| Model | Parameters | Accuracy | |
|---|---|---|---|
| 2-3 | Pretrained backbone | Mask head | |
| FaceNet [3] | 27,910,327 | 115,010 | 95.44% |
| ArcFace [1] | 25,627,081 | 33,090 | 97.91% |
Table 1 reports the parameter cost and accuracy of the proposed mask classification head when integrated with pretrained face recognition backbones. Although the mask head is trained on the Mask Classifier dataset, it is
evaluated on the masked and unmasked images from the extended LFW-Mask dataset to assess its generalization ability in the target evaluation setting. The results show that the mask head achieves high classification accuracy for
both backbone settings, with 95.44% for FaceNet and 97.91% for ArcFace. Importantly, the number of additional parameters introduced by the mask head is small compared to those of the pretrained backbone. These results indicate that the proposed mask
classification branch can provide reliable masked/unmasked predictions for mask-gated routing while adding only lightweight overhead to the original face recognition model.
4.3 Face identification results↩︎
| Method | FaceNet | ArcFace | ||||
|---|---|---|---|---|---|---|
| 2-4(lr)5-7 | P@1 | RP | M@R | P@1 | RP | M@R |
| Cosine Similarity | 77.54 | 59.12 | 55.84 | 87.22 | 66.55 | 64.11 |
| DeepFaceEMD\(_{\text{APC}}\) | 81.83 | 59.63 | 55.17 | 89.71 | 65.45 | 63.15 |
| DeepFaceEMD\(_{\text{Uniform}}\) | 76.99 | 57.09 | 53.73 | 89.15 | 65.81 | 63.52 |
| DeepFaceEMD\(_{\text{SC}}\) | 74.71 | 55.92 | 52.39 | 88.51 | 65.33 | 62.95 |
| MGFace (ours) | 82.92 | 60.91 | 58.05 | 91.91 | 68.57 | 66.81 |
Table 2 reports the main face identification results using FaceNet and ArcFace as pretrained backbones. Overall, MGFace consistently achieves the best performance across all evaluation metrics and both backbones. Compared with conventional cosine similarity, MGFace improves P@1 by approximately 5% on both FaceNet and ArcFace, while also achieving consistent gains in RP and M@R. Compared with DeepFaceEMD-based re-ranking methods, MGFace also obtains stronger results in most cases. On FaceNet and ArcFace, MGFace further improves all metrics over DeepFaceEMD\(_{\text{APC}, \text{Uniform}, \text{SC}}\). These results suggest that the proposed mask-gated routing and mask-aware patch similarity are more effective for masked-face identification than applying general patch-wise re-ranking strategies. The routing strategy allows MGFace to choose the most suitable matching scheme based on the query conditions. It activates fine-grained patch-level matching only when masks are detected, concentrating on reliable facial regions. This approach enhances identification accuracy while minimizing unnecessary matching complexity for queries without masks.
To evaluate the computational efficiency of MGFace, we compare our conditional re-ranking strategy with DeepFaceEMD\(_{\text{APC}}\) using the FaceNet backbone. As shown in Table 3, our proposed method reduces the total query time from 182.59s to 8.21s, corresponding to a speedup of over 20\(\times\). In addition, MGFace reduces peak VRAM usage from 2.77 GB to 1.79 GB. These results demonstrate that MGFace provides a more efficient alternative to EMD-based re-ranking by avoiding unnecessary patch-level matching and using a lightweight mask-aware re-ranking strategy only when required.
| Method | Query time | VRAM allocated |
|---|---|---|
| DeepFaceEMD [9] | 182.59s | 2.77 GB |
| MGFace (ours) | 8.21s | 1.79 GB |
4.4 Qualitative results↩︎
Figure 5 presents representative top-5 retrieval results of cosine similarity, DeepFaceEMD\(_{\text{APC}}\), and MGFace. For unmasked queries, cosine similarity remains
highly effective, motivating the mask-gated routing design of MGFace to avoid unnecessary re-ranking. For masked queries, re-ranking methods generally improve retrieval quality. At the same time, MGFace achieves more reliable results by
concentrating patch-level matching on visible face regions. The results show that MGFace effectively adapts to both masked and unmasked queries, improving masked-face retrieval while preserving strong performance on unmasked faces
through mask-gated routing.
4.5 Ablation study↩︎
4.5.1 Different query sets↩︎
We evaluate all methods with the FaceNet backbone under three query settings: unmasked, masked, and unmasked \(\oplus\) masked. As shown in Table 4, cosine similarity performs best on unmasked queries, while DeepFaceEMD\(_{\text{APC}}\) slightly degrades performance, suggesting that patch-level re-ranking is
unnecessary for fully visible faces. The proposed method maintains nearly the same performance as cosine similarity through mask-gated routing. For masked queries, MGFace achieves the best results by applying mask-aware patch-level matching on
reliable facial regions. Overall, MGFace obtains the best performance on the combined query set, showing its effectiveness across different query conditions.
| Query set | Metric | Cosine | DeepFaceEMD\(_{\text{APC}}\) | MGFace |
|---|---|---|---|---|
| unmasked \(\oplus\) masked | P@1 | 77.54 | 81.83 | 82.92 |
| RP | 59.12 | 59.63 | 60.91 | |
| M@R | 55.84 | 55.17 | 58.05 | |
| unmasked | P@1 | 95.62 | 94.76 | 95.61 |
| RP | 74.73 | 72.86 | 74.71 | |
| M@R | 73.61 | 71.45 | 73.58 | |
| masked | P@1 | 59.47 | 68.89 | 70.24 |
| RP | 43.50 | 46.41 | 47.11 | |
| M@R | 38.08 | 41.49 | 42.51 |
4.5.2 Effect of different k↩︎
We study the effect of the number of candidates \(k\) used in the re-ranking stage. As shown in Figure 6, increasing \(k\) generally improves identification performance. Still, it reduces query speed due to the larger number of candidates involved in patch-level re-ranking. However, the performance gain becomes marginal when \(k \geq 100\). In addition, P@1 increases only from 82.92 to 82.97 when \(k\) increases from 100 to 500. At the same time, the query speed decreases from approximately 2224 to 2113 queries per second. Therefore, we set \(k=100\) as the default value, which provides a good trade-off between accuracy and efficiency.
4.5.3 Grid size selection for patch-level features↩︎
We study the effect of different patch grid sizes. As shown in Table 5, increasing the grid size from \(4\times4\) to \(8\times8\) slightly improves all metrics, suggesting that finer patch representations provide more informative local cues for mask-aware re-ranking. However, performance drops substantially at \(16\times16\), likely because these features are extracted from shallower layers with weaker semantic representations. Therefore, we use the \(4\times4\) setting as the default, which provides a good balance between accuracy and computational efficiency.
| Grid size | P@1 | RP | M@R |
|---|---|---|---|
| 4\(\times\)4 | 91.91 | 68.57 | 66.81 |
| 8\(\times\)8 | 92.30 | 68.75 | 67.04 |
| 16\(\times\)16 | 81.84 | 58.08 | 55.05 |
5 Discussion and conclusion↩︎
5.1 Limitations↩︎
Although MGFace improves the efficiency and robustness of masked-face identification, it has two main limitations. First, its conditional routing depends on the accuracy of the mask classification head; incorrect predictions may send a query to a suboptimal matching branch. Second, the current mask-aware filtering uses a fixed upper-face crop based on the assumption that masks occlude the lower facial region. This may be less effective for inaccurate face alignment or other irregular occlusions such as sunglasses. Future work will explore adaptive occlusion localization and learnable region selection to handle more diverse occlusion patterns.
5.2 Conclusion↩︎
We proposed MGFace, a mask-gated face identification pipeline that conditionally routes query images according to their predicted mask status. By applying global matching to unmasked queries and mask-aware patch-level re-ranking only to masked queries, MGFace improves masked-face identification while avoiding unnecessary computation. Experiments on the extended LFW-Mask dataset demonstrate that MGFace achieves better performance and lower computational cost than existing re-ranking-based methods. Future work will explore more adaptive occlusion localization strategies to handle diverse occlusion types beyond facial masks.
6 ACKNOWLEDGMENT↩︎
This research is funded by University of Information Technology-Vietnam National University of Ho Chi Minh city under grant number CS4-2026-80038.