1 Introduction↩︎

Face recognition [1] is a highly suitable biometric modality for automated border control (ABC) applications, owing to its non-contact acquisition capability and seamless integration with established workflows [2]. Nevertheless, achieving robust performance at land border crossings remains a significant challenge. Under operational conditions, subjects are frequently captured with non-frontal head pose, motion blur, variable ambient illumination, and partial facial occlusion. These difficulties are further exacerbated when image acquisition is performed through automotive windows, which introduces additional optical degradations that materially degrade recognition accuracy.

A particularly underexplored yet operationally significant scenario is in-vehicle border screening, in which a traveler is observed while seated inside a vehicle through a windshield or side window. In addition to the typical challenges inherent to unconstrained face recognition, automotive windows introduce complex optical phenomena. Modern windshields frequently incorporate infrared-reflective coatings engineered to attenuate solar heat gain; such coatings can reflect in excess of 50% of near-infrared (NIR) radiation, substantially reducing the signal available to NIR-based imaging systems. Variability in window tint across vehicle makes and models further increases variability  [4], while specular reflections arising from ambient lighting introduce additional noise and artifacts.

VIS-NIR face recognition further suffers from the cross-spectral domain gap [5]: enrollment references are typically acquired in the visible (VIS) spectrum (, passport or mobile photographs), whereas checkpoint probes are captured in NIR, yielding substantial differences in facial texture and reflectance that challenge models trained on VIS data alone. Although benchmarks such as CASIA NIR-VIS 2.0 [6] have advanced heterogeneous face recognition, they are collected in controlled settings and do not reproduce the combined effects of through-glass imaging, subject motion, and unconstrained outdoor environments. Datasets acquired in operational border settings remain largely proprietary [7].

To address these limitations, we introduce DriveFace , a NIR dataset that captures the end-to-end biometric workflow at vehicular border crossings and enables systematic study of cross-spectral matching, through-glass degradation, and adverse illumination within a single operationally representative benchmark collected over two months. A dedicated presentation attack subset further supports anti-spoofing research under realistic deployment conditions. Figure 1 illustrates the capture settings.

The main contributions of this work are listed below:

• DriveFace , a publicly available dataset for face recognition at vehicular border crossings, pairing VIS pre-enrollment images with in-vehicle NIR probes acquired through automotive windows, and a presentation attack subset (DriveFace-PAD ).

• Rich per-acquisition metadata: tint level, illumination, head pose, and vehicle speed, enabling fine-grained analysis of individual degradation sources.

• Standardized face recognition and PAD protocols with baseline results establishing reference performance on DriveFace . The dataset is available at: ^{1 2}.

2 Related Work↩︎

While there are a large number of face recognition datasets publicly available [8]–[12], most are collected in controlled environments and do not reflect the challenges of vehicular border control. Below, we review datasets relevant to this setting. 0.5em -1em In-Vehicle Face Datasets In-vehicle face datasets are typically introduced for driver monitoring rather than identity recognition. Representative examples include Drive&Act [13], DMD [14], and DriverMVT [15], which capture in-cabin subjects under varying pose, occlusion, illumination, and seating position for tasks such as distraction, drowsiness, gaze, and head-pose estimation. Datasets more oriented toward in-vehicle authentication include AVICAR [16], VFPAD [17], and iCarB [18]. While valuable for automotive facial analysis, none target heterogeneous cross-session identity matching from outside the vehicle through tinted or clear windows with profile-pose variation, leaving a clear gap for external-view, cross-modal recognition. 0.5em -1em NIR Face Datasets Near-infrared heterogeneous datasets support face recognition beyond the visible spectrum, particularly under low light and for RGB-to-NIR matching. Representative datasets include CASIA NIR-VIS 2.0 [6], PolyU-NIRFD [19], Oulu-CASIA NIR&VIS [20], and BUAA-VisNir [21], alongside broader multimodal datasets such as the Tufts Face Database [22] and LAMP-HQ [23]. Although these have advanced heterogeneous face recognition, they are collected indoors under controlled conditions and do not capture through-glass acquisition, in-vehicle deployment, motion, or outdoor settings. 0.5em -1em Face Acquisition Through Glass Through-glass face acquisition remains relatively underexplored. The most related works are the through-windshield driver recognition study of Cornett  [7], the Face Image Reflection Removal (FIRR) dataset [24], and PP4AV [25], which cover subjects viewed through windshields, real outdoor driving, and reflection-degraded imagery, respectively. However, [24] and [25] target different tasks and do not model heterogeneous mobile-RGB to external-NIR matching, while [7] captures multiple occupants through tinted windows. Public benchmarks for through-glass face recognition therefore remain limited. 0.5em -1em Presentation Attack Detection Datasets PAD datasets assess robustness to print, replay, and mask attacks. Established benchmarks such as Replay-Attack [26], CASIA-FASD [27], MSU-MFSD [28], OULU-NPU [29], and SiW [30] rely on RGB capture in mobile or desktop settings, while more recent datasets such as CASIA-SURF [31], CASIA-SURF CeFA [32], CelebA-Spoof [33], WMCA [34], and HiFiMask [35] extend toward multi-modal sensing, cross-ethnicity evaluation, and 3D masks. However, none of them jointly address PAD in NIR, through-glass acquisition, and external-view capture under tint, reflections, and motion. The in-vehicle datasets VFPAD [17] and iCarB [18] are closer: VFPAD targets in-vehicle NIR PAD and iCarB supports in-car PAD-oriented evaluation, but both focus on in-cabin capture rather than external probes acquired through automotive glass.

3 The DriveFace Dataset↩︎

The proposed dataset was designed to support biometric research in operational border-control settings. It consists of three components: (i) pre-enrollment face captures, (ii) in-vehicle face captures, and (iii) presentation attacks. Together, these components reflect a realistic workflow in which a traveler first provides reference biometric data prior to arrival and is subsequently observed under challenging checkpoint conditions. The dataset was collected from 70 consenting subjects over two recording sessions, with corresponding reference samples and probes. Figure 2 shows examples of the pre-enrollment face captures for reference and probe images. For most participants, the two sessions were collected approximately two months apart in order to capture realistic short-term variation in appearance, including changes in facial hair, hairstyle, grooming, makeup, and other mild changes in facial appearance.

4pt

3.1 Demographics↩︎

The dataset was collected from 70 consenting volunteers. The same participant pool was used across the data collection campaign, which allows direct association between pre-enrollment reference samples and in-vehicle operational captures. This design supports both verification and identification experiments in which clean enrollment data are matched against challenging probes. This dataset effort contains a demographically diverse group of participants. In terms of gender, the dataset is relatively balanced, with 45.7% female and 54.3% male subjects. The age distribution spans a broad range from 18 to 85 years old. The dataset also includes variation in apparent skin tone, which we summarize into three broad groups: light, medium, and dark, containing 51, 11, and 6 subjects, respectively, based on the available annotations. Figure 4 illustrates the age and skin-tone distributions of the subjects in the dataset.

3.2 Pre-enrollment Face Captures↩︎

The pre-enrollment subset was designed to simulate a trusted-traveler or pre-registration procedure performed before arrival at a checkpoint. For each subject and each session, close-range facial videos were acquired using the front-facing cameras of two consumer smartphones: an iPhone 12 and a Samsung Galaxy S9. This subset provides high-quality reference data intended to serve as identity ground truth for subsequent comparisons with more challenging operational captures. During acquisition, subjects were instructed to perform controlled head movements in order to introduce natural pose variation rather than a single static frontal view. This subset is therefore suitable as an enrollment/reference set for verification and identification experiments under cross-device and cross-condition settings.

3.3 In-vehicle Infrared Face Captures↩︎

The in-vehicle subset targets face acquisition and recognition for subjects seated inside vehicles and observed through automotive glass. This scenario represents a particularly challenging condition for border biometrics, due to the combined effects of reflections, window tint, low transmission, viewpoint changes, motion blur, and reduced illumination. The in-vehicle data were acquired primarily in the near-infrared (NIR) spectrum under two complementary protocols using a VIDAR PAX infrared sensor [3] with an external illuminator.
Real-vehicle outdoor captures: In the first protocol, subjects were recorded inside a real vehicle from an acquisition distance of approximately 1–2 m from the glass. Recordings were captured from multiple viewpoints, including approximately \(0^\circ\), \(45^\circ\), and \(90^\circ\), and under both stationary and moving-vehicle conditions. This protocol was designed to reproduce realistic checkpoint approach conditions, including motion-related degradation and perspective distortion.
Controlled indoor simulated-car captures: In the second protocol, a controlled indoor setup was used to isolate the effects of glass transmission and scene illumination. Recordings were acquired under five window tint levels corresponding to 5, 15, 20, 30, and 45 Visible Light Transmission (VLT) levels, and under three illumination levels of 5%, 20%, and 30%. Instead of using real cars, we mounted automotive glass panels with different VLT levels on wooden frames simulating these variations. This protocol provides a structured benchmark for studying the limits of face recognition systems under through-glass conditions.
Real-vehicle indoor captures: For the third protocol, a smaller set of real-vehicle indoor captures was also collected to study face acquisition in a more controlled environment while preserving the in-car setting. In this protocol, subjects were recorded inside a real vehicle at approximately \(90^\circ\) relative to the vehicle direction through the clear window. Two facial pose conditions were considered, namely CAM and FWD, and recordings were acquired at 5%, 20%, and 30% illuminator power. This part of the dataset contains 418 NIR video clips.
Overall, the in-vehicle subset provides a challenging and structured benchmark for evaluating robustness to through-glass degradation, viewpoint variation, and adverse lighting in border-control scenarios.

3.4 Presentation Attacks↩︎

The dataset also includes a presentation attack subset designed to evaluate the robustness of face anti-spoofing systems in the proposed vehicular border-control setting. Attacks are presented in outdoor and indoor conditions from inside the vehicle and captured from the outside using the infrared sensor positioned at approximately \(45^\circ\) relative to the vehicle direction.

The PAD subset includes three attack categories: print, replay, and mask. Print and replay attacks are generated from subjects for whom bona fide samples are available in other parts of the dataset, while mask attacks are collected from external subjects. All attack types are recorded under the same factors: four infrared illumination levels (5%, 10%, 20%, 30%) and two attack poses, namely CAM and FWD. The subset also includes variation within each attack category. For print attacks, the dataset contains different combinations of printer type and paper material, including inkjet and laser printers, and matte, normal, and glossy paper. For replay attacks, it contains two laptop display types. For mask attacks, the dataset includes paper, resin, silicone, transparent plastic, transparent plastic with makeup, thin colored plastic, rubber, silicone over mannequin, and mannequin. Figure 3 shows example attacks, while Table 2 summarizes the main factors covered by the subset.

3.5 Annotations and Metadata↩︎

Automated detection and tracking were used to isolate consenting participants and remove bystanders, followed by a final human-in-the-loop check to ensure that no non-consenting identities remained visible. The dataset provides annotations and metadata at the subject, session, and video levels, with most acquisition settings encoded in the filename. Identity labels include the subject ID and, for in-vehicle captures, the identities and seat positions of all occupants. Session labels are defined for both reference and probe data: reference filenames encode subject ID, date, and session number, while the parent folder indicates the mobile device; in the probe subset, session 1 denotes outdoor captures and session 2 indoor captures. Additional subject-level metadata include age, gender, and skin tone. Viewpoint labels specify camera direction relative to the vehicle (\(0^\circ\), \(45^\circ\), \(90^\circ\)) and face pose (CAM, FWD). Tint and illumination metadata include illuminator setting, scenario, speed, weather, and window tint level. For the PAD subset, attack labels also encode the attack category, instrument, and material.

4 Experiments↩︎

This section describes the benchmarking performed on the DriveFace dataset, including both FR and PAD experiments.

4.1 Face Recognition Experiments↩︎

0.5em -1em Protocols In the operational setting, RGB videos are used for enrollment. These enrollment videos capture subjects performing controlled head movements (left–right and up–down). In the probe scenario, subjects are recorded using an NIR camera while seated inside vehicles under different operational conditions. For enrollment, a frontal face image is selected for each subject using a pose estimator. For the probe set, we use face crops produced by a face detector and manually annotated with identity labels corresponding to the operational scenarios, selecting up to 10 samples per subject from each video.

For training and evaluation, the dataset is partitioned into disjoint train and test sets based on identity. Specifically, 60% of identities are used for training and the remaining 40% for testing. Each subject has two capture sessions, where samples from different sessions are used for enrollment and probe sets to ensure cross-session evaluation.

The dataset includes three operational conditions: (1) outdoor in-vehicle scenarios, (2) indoor in-vehicle scenarios, and (3) indoor simulation scenarios with varying automotive glass tint levels. These correspond to the protocols outdoor, indoor_car, and simulation. Additional metadata, such as vehicle speed, tint level, weather, and illumination conditions are provided to support more detailed analysis. The dataset statistics are shown in Table 3.

0.5em -1em Metrics To evaluate face recognition performance, we report several standard metrics commonly used in the literature, including Area Under the Curve (AUC), Equal Error Rate (EER), Rank-1 identification rate, and Verification Rates (VR a.k.a 1-FNMR) at specific False Acceptance Rate (FARs a.k.a, FMRs) of 1%. 0.5em -1em Baselines To benchmark performance on the dataset, we consider several publicly available open-source face recognition models. Although there exists a modality gap between enrollment (RGB) and probe (NIR), this gap is relatively small compared to other heterogeneous settings [36]. We evaluate a set of baseline models that are originally trained for standard face recognition tasks, and additionally include models that are explicitly trained on the training split of DriveFace . AdaFace [37] introduced an adaptive margin-based loss for face recognition that uses the feature norm as a proxy for image quality. In our experiments, we used the publicly available AdaFace model with a ResNet-100 backbone trained on the WebFace12M dataset. LVFace [38] introduced a Vision Transformer (ViT) based FR model designed to better exploit large vision models for face recognition. In our experiments, we used the publicly available LVFace model based on a ViT-L backbone trained on the Glint360K dataset. EdgeFace [39] is a lightweight face recognition model built on a hybrid CNN–Transformer backbone. It is trained on RGB face images from the WebFace dataset [12]. In our experiments, we used the publicly available EdgeFace-Base variant. xEdgeFace [40] is a contrastive self-distillation framework designed to adapt a pretrained face recognition backbone to heterogeneous face recognition. The method retains the original EdgeFace [39] architecture and introduces cross-modal alignment along with teacher–student supervision to enable effective adaptation while preserving performance on the source (RGB) task. As a result, the model gains robust heterogeneous recognition capability without catastrophic forgetting, while remaining lightweight.

0.5em -1em Experiment details For AdaFace, EdgeFace, and LVFace, we use publicly available pretrained models and perform baseline evaluations by comparing each probe sample against all gallery templates using cosine distance. All images are first preprocessed using face detection and alignment, and the resulting \(112 \times 112\) crops are fed into the pretrained networks for feature extraction and evaluation. For xEdgeFace, we adopt the EdgeFace backbone and further tune it using the training split of DriveFace following the method proposed in [40], specifically optimizing it for the cross-spectral setting. The performance reported corresponds to evaluation on the test set.

0.5em -1em Experimental results Tab. 4 summarizes the face recognition performance across the three evaluation protocols.

Outdoor protocol. The outdoor protocol achieves the strongest overall performance across all models, indicating that despite the RGB-NIR modality gap, robust FR models perform reasonably well. AdaFace achieves the best performance with an AUC of 99.45% and the lowest EER of 2.69%, along with the highest Rank-1 accuracy (97.42%). This suggests that its adaptive margin mechanism generalizes well even under mild modality shifts. xEdgeFace improves over its backbone (EdgeFace), reducing the EER from 5.37% to 4.18% and increasing verification rates, demonstrating the effectiveness of cross-spectral adaptation.

Simulation protocol. The simulation protocol is the most challenging scenario, with all models exhibiting a noticeable drop in performance. This can be attributed to the presence of controlled variations such as different glass tint levels and potentially more severe spectral distortions. AdaFace and xEdgeFace achieve comparable performance, with AUC values around 96% and EERs near 8%, significantly higher than in other protocols. Importantly, xEdgeFace slightly outperforms AdaFace in VR@FAR=1%, highlighting the advantage of explicit cross-spectral adaptation under challenging conditions. EdgeFace and LVFace show the largest degradation, indicating that models trained purely on RGB data struggle to generalize in this scenario.

Indoor car protocol. In the indoor in-vehicle setting, performance remains high but shows a slight degradation compared to outdoor conditions, likely due to more constrained lighting and reflections from vehicle interiors. AdaFace achieves the best overall results in this protocol, with the lowest EER (3.11%) and highest Rank-1 accuracy (97.11%), indicating strong robustness to controlled in-car conditions. Notably, xEdgeFace again improves over EdgeFace, confirming that domain adaptation consistently benefits cross-spectral matching. However, the gains are less pronounced than in the simulation scenario, suggesting that the domain gap is smaller in this setting.

Across all protocols, the results highlight two key aspects: (i) the impact of cross-spectral and environmental variations on recognition performance, and (ii) the benefit of adapting models to the target domain. Pretrained RGB models (AdaFace, EdgeFace, and LVFace) demonstrate strong baseline performance, indicating that the RGB–NIR gap in this dataset is small, although it still introduces noticeable degradation under more challenging conditions. Among these, AdaFace consistently achieves the most robust performance, likely due to its quality-aware training strategy. At the same time, the consistent improvements of xEdgeFace over its backbone EdgeFace across all protocols emphasize the importance of domain adaptation for heterogeneous face recognition, with the gains being particularly evident in the more challenging simulation scenario. It should be noted that, EdgeFace and xEdgeFace are significantly more lightweight compared to AdaFace and LVFace, yet they remain competitive, highlighting an important trade-off between efficiency and performance. Overall, while strong pretrained models generalize reasonably well, explicit cross-spectral adaptation is crucial for achieving robust performance under varying operational conditions, especially when illumination changes and sensor differences introduce larger domain shifts.

Figure 5 shows representative failure cases in the simulation protocol. From the images, it is evident that most failures occur under conditions such as extreme tint, low contrast, extreme profile views, and the presence of occlusions.

Effect of Tint Level We performed a set of experiments on the controlled subset, reporting AUC separately for each VLT level and for clear glass (Tab. 5). Across the evaluated models, performance generally improves as light transmission increases, with clear glass giving the best results. This directly quantifies the effect of tint.

4.2 Presentation Attack Detection↩︎

As noted in the study in [41], replay attacks do not constitute a significant challenge in the near-infrared (NIR) domain, as they are largely invisible in NIR. Therefore, this work focuses on more challenging and operationally relevant attack types, namely print and mask-based presentation attacks [42], which exhibit greater attack potential.

0.5em -1em Protocols We define three evaluation protocols to assess model robustness under different attack conditions: grandtest, where all attack types appear across training, development, and evaluation splits, representing a known-attack setting with disjoint subjects and attack instruments; unseen_print, where print attacks are excluded from training and development and appear only at evaluation, simulating an unseen print-attack scenario; and unseen_mask, which follows the same setup for mask attacks to measure generalization to unseen mask-based attacks. A detailed summary of the dataset statistics for each protocol is provided in Table 6.

0.5em -1em Metrics To evaluate the performance of presentation attack detection (PAD) systems, we adopt the standardized metrics defined in ISO/IEC 30107-3 [43]. Specifically, we report the Attack Presentation Classification Error Rate (APCER) and the Bona Fide Presentation Classification Error Rate (BPCER), where BPCER corresponds to the rate at which bona fide samples are incorrectly classified as attacks. Additionally, we compute the Average Classification Error Rate (ACER), defined as the mean of APCER and BPCER. The decision threshold is determined on the development set of each protocol using the Equal Error Rate (EER) criterion.

0.5em -1em Baselines We evaluate several baseline models across the protocols. DeepPixBiS [44] employs a DenseNet-based architecture with pixel-wise binary supervision to enhance PAD performance. In our implementation, we utilize only the pixel-wise binary loss during training, and final predictions are obtained by averaging the resulting pixel-wise score maps. For CLIP [45], we utilize the vision encoder of the CLIP model, specifically the ViT-B/32 variant. Two training configurations are considered: (i) CLIP (fc only), where only the final fully connected layer is trained while keeping the backbone frozen, and (ii) CLIP (full), where all model parameters are fine-tuned. The DinoV2 [46] baseline utilizes the DinoV2 backbone, specifically the ViT-B/14 variant. Only the final fully connected classification layer is fine-tuned, while the backbone remains frozen. ConvNeXtV2-Tiny and EfficientNet-B0 are lightweight convolutional architectures based on ConvNeXt V2 [47] and EfficientNet [48], respectively. Both models are pretrained on ImageNet and subsequently adapted for the binary PAD classification task.

0.5em -1em Experimental Settings All images were first preprocessed with face detection using the SCRFD [49] model and alignment, after which they were cropped to a fixed spatial resolution of \(224 \times 224\) pixels. All models were trained for 100 epochs using a learning rate of \(1 \times 10^{-4}\) and a weight decay of \(1 \times 10^{-6}\), with a batch size of 64. The training process was implemented in PyTorch and used NVIDIA RTX 3090 GPUs. To improve generalization, data augmentation was applied during training, including random horizontal flipping, RandAugment, random rotations up to \(30^\circ\), and color jitter with brightness, contrast, and saturation variations of 0.15. To mitigate class imbalance, the training data was sampled in a balanced manner.

0.5em -1em Experimental Results on PAD

Table 7 summarizes the PAD performance across all three protocols. In the grandtest protocol all models achieve low error rates, with EfficientNet-B0 performing best (ACER = 0.50%), followed by ConvNeXtV2-Tiny (0.70%). This indicates that the evaluated models perform well in the known-attack setting. CLIP-based models show comparatively higher errors, particularly in BPCER.

In contrast, performance drops sharply in the unseen attack scenarios. For unseen_mask, all models exhibit very high APCER, with DinoV2 (fc only) performing best (ACER = 26.20%), suggesting better generalization from self-supervised (SSL) features. However, overall results remain poor, highlighting the difficulty of mask attack detection under domain shift. For unseen_print, the degradation is less severe. DeepPixBiS achieves the best result (ACER = 18.40%), followed by ConvNeXtV2-Tiny (21.50%).

These results motivate further research on domain adaptation and self-supervised representations for generalization to unseen attacks.

5 Discussion↩︎

The experiments on DriveFace demonstrate that both FR and PAD remain challenging in realistic vehicular border-control conditions, even when strong pretrained models are used. On the FR side, the results show that the RGB-NIR modality gap is manageable but still non-negligible, especially under more difficult conditions such as the simulation protocol, where glass tint and controlled illumination changes introduce a larger domain shift. AdaFace consistently provides the strongest overall baseline, while xEdgeFace shows clear gains over EdgeFace across all protocols, confirming the value of explicit cross-spectral adaptation. Importantly, EdgeFace and xEdgeFace are significantly smaller and more lightweight than AdaFace and LVFace, yet they remain competitive, making them particularly attractive for deployment in practical resource-constrained systems. On the PAD evaluations, the grandtest protocol results indicate that known attacks can be detected reliably, but performance degrades substantially in unseen attack settings, especially for mask attacks, revealing a clear generalization gap. These findings suggest that DriveFace is a challenging and realistic benchmark that exposes limitations not only in cross-spectral face recognition, but also in robust spoof detection under operational conditions, highlighting the need for models that jointly address efficiency, domain shift, and generalization to unseen scenarios.

6 Conclusion↩︎

In this work, we introduced the new DriveFace and DriveFace-PAD datasets for cross-spectral through-glass face analysis in on-the-move vehicular border-control scenarios, together with benchmark protocols for both face recognition and presentation attack detection. The datasets capture realistic operational challenges, including RGB-to-NIR matching, through-glass acquisition, unconstrained viewpoints, varying illumination, and presentation attacks. Experimental results show that while modern pretrained face recognition models already provide strong baselines, their performance still drops under more challenging environmental and spectral conditions, and explicit cross-spectral adaptation further improves robustness. Similarly, PAD results reveal that detecting known attacks is relatively easy, whereas generalization to unseen attacks remains difficult. Overall, DriveFace and DriveFace-PAD provide a valuable benchmark for advancing research on robust and efficient biometric systems for real-world border-control applications. To support reproducibility and further research, the dataset, evaluation protocols, and code are publicly available.

7 Ethics Statement↩︎

All participants in this data collection volunteered and provided informed consent, agreeing to the collection and use of their data for the specified research purposes. The project under which the data was collected was approved by the institution’s Data Research Ethics Committee (DREC).

8 Acknowledgments↩︎

The project leading to this work has received funding from Frontex under the Frontex Research Grants Programme. Call for Proposals 2024/CFP/INNOVATE/01 Grant Agreement No. 2025/280. This work reflects only the authors’ view. Neither the European Union nor Frontex are responsible for any use that may be made of the information it contains. This research was also partly funded by the European Union project CarMen (Grant Agreement No. 101168325).

References↩︎