1 Introduction↩︎

With the widespread adoption of face recognition in applications like identity verification and online payments, security concerns have become increasingly prominent. Presentation Attack Detection (PAD) is essential for detecting presentation attacks (PAs) using printed images [1], video replays [2], and 3D masks [3]. While existing PAD methods, ranging from hand-crafted features [4]–[6] to deep-learning-based approaches [7], [8], have shown promising results in controlled settings, they struggle to generalize across different domains due to significant domain shifts. Mitigating domain shifts between seen source samples and unseen target samples remains a considerable challenge for PAD models.

Traditional PAD methods [9]–[11] seek to bridge the distribution gap between source and target domains using adversarial adaptation, meta-learning, or feature alignment, etc. However, their performance is constrained by the uni-modal nature of these approaches. With the emergence of pretrained vision-language models (VLMs), the generalization of downstream tasks [12] improves through their rich multi-modal knowledge. Recent studies [7], [8] further show that CLIP-like frameworks significantly enhance PAD performance. These approaches outperform traditional PAD methods by providing additional context. However, human-level prompts struggle to convey the critical visual clues for PAD classification. For instance, clues like photo edges and moiré patterns can be described using natural language, while subtle geometric distortions and illumination-reflection discrepancies are difficult to express. As a result, constructing and learning prompts relying on human-level understanding limit the performance of existing CLIP-like PAD methods.

Surprisingly, we find that leveraging eXplainable AI (XAI) methods [13]–[15] enables us to transfer general knowledge embedded in pretrained VLMs into PAD process from a model-level perspective and feature space. This finding motivates us to construct and learn model-oriented prompts informed by these explanatory model-level signals. Specifically, with the aid of concept-based XAI techniques, we successfully discover visual concepts within the model’s feature space and enhance domain data using corresponding feature heatmaps. These concept-associated heatmaps enable PAD models to learn multiple model-oriented prompts that are aligned with the discovered visual concepts, which guide PAD model to capture critical clues for classification beyond human-level understanding. A comparison with existing CLIP-like PAD methods is illustrated in 1.

Based on the above, we propose Concept-informed Prompts Guided Presentation Attack Detection (CPG-PAD), which exploits and incorporates visual concepts from pretrained VLMs into PAD process. Specifically, we introduce Visual Concept-driven Enhancement (VCE), a mechanism that leverages general pretrained VLMs to discover PAD-relevant visual concepts, while enhancing the domain data by generating fine-grained feature heatmaps corresponding to each discovered concept. The discovered concepts are closely linked to the objectives of attack detection, from a model-level perspective and within its feature space. With the aid of these concept-associated heatmaps, we introduce Prompt-based Concept Injection (PCI) to learn multiple concept-informed prompts by aligning textual prompts with the discovered visual concepts. In particular, learnable prompts interact with visual features through the Visual-Prompt Decoder (VPD) and are supervised by concept-associated heatmaps to inject concept information into the prompts. In this way, CPG-PAD accurately captures generalizable visual features, suppressing interference from domain-specific information when encountering unseen target domains, thereby enhancing the generalizability of PAD task.

Our contributions can be summarized as follows:

• We propose Concept-informed Prompts Guided PAD (CPG-PAD), a novel framework which enhances domain generalization performance by incorporating visual concepts discovered by employing XAI techniques on pretrained VLMs.

• CPG-PAD introduces Visual Concept-driven Enhancement (VCE), and Prompt-based Concept Injection (PCI). VCE automatically discovers PAD-relevant visual concepts, and enhances domain data by generating corresponding feature heatmaps. With the concept-associated heatmaps, PCI learns multiple concept-informed prompts through Visual-Prompt Decoder (VPD).

• Extensive experiments and analysis demonstrate the superiority of CPG-PAD over state-of-the-art competitors on widely-used benchmark datasets.

2 Related Works↩︎

2.1 Presentation Attack Detection↩︎

The earliest attempts at Presentation Attack Detection (PAD) focused on handcrafted feature engineering. Techniques such as Local Binary Patterns (LBP) [16], [17], Histogram of Oriented Gradients (HOG) [18], and Scale-Invariant Feature Transform (SIFT) [19] were employed to capture texture irregularities caused by presentation attacks. Although these methods provided a foundation for early PAD, they were easily affected by changes in environmental conditions, including lighting variations, scaling factors, and head pose, which hindered robustness in practice.

Deep learning advanced PAD research by enabling robust and discriminative feature extraction. Convolution Neural Networks (CNNs) quickly became the mainstream choice [20]–[22]. Yet, the inherently local nature of convolutions limited their capacity to jointly capture fine-grained details and long-range dependencies, leading to vulnerabilities in challenging attack scenarios [23]. To overcome these shortcomings, transformer-based models [24], [25] have been explored. By leveraging self-attention mechanisms, they model global relationships across image patches and extend beyond CNNs’ receptive field constraints. While these architectures have achieved strong intra-domain evaluation, they often fail to generalize well in unseen environments.

To address the domain shift problem, researchers have explored Domain Adaptation (DA) and Domain Generalization (DG) strategies. DA [26]–[28] aims to bridge distribution gaps between source and target domains using unlabeled target data. Existing methods employ adversarial adaptation [26], meta-learning [27], or extract generalizable features from large-scale unlabeled data [28]. However, obtaining target domain data during training remains a significant challenge in real-world scenarios. In contrast, DG [9], [10], [29]–[31] tackles this issue by enhancing model robustness across multiple source domains without relying on target data. Traditional DG approaches learn domain-invariant features through feature alignment [9] or hierarchical relationships [32] to improve generalizability in PAD. Despite these improvements, single-modality DG-PAD still faces difficulties when domain discrepancies are large or training data limited.

Unlike DA and DG, which focus on improving feature generalization, auxiliary-supervised approaches provide additional supervisory signals to guide feature learning. Some works augment the classification task with physical or physiological auxiliary signals such as depth maps or remote photoplethysmography (rPPG). For example, Liu et al. [20] jointly estimate pixel-wise facial depth and rPPG rhythm along with the binary real/fake decision, showing that these auxiliary losses help guide the network to focus on fundamental attack-discriminative cues rather than overfit to texture patterns. Subsequent works [22] refine this supervision by introducing spatial gradient features and contrastive depth losses to sharpen the depth supervision signal. Other methods [33]–[35] extend depth supervision across multiple frames or design adaptive fusion with rPPG, e.g. exploiting temporal and depth information using multiframe depth estimation. Related efforts also introduce explanation-based or attention-based guidance as auxiliary supervision, using saliency maps to regularize model attention and improve interpretability [36].

Recent progress in large-scale pretrained vision and vision-language models, particularly CLIP [12], has opened new avenues for PAD. A first line of research [37], [38] investigates how foundation models can be adapted to PAD through simple transfer strategies, such as feature extraction, lightweight adaptation, or zero-shot inference. These studies suggest that pretrained models provide strong general visual priors, but directly transferring such knowledge to PAD remains challenging without task-specific guidance. Building on this, CLIP-based PAD methods further exploit cross-modal supervision by aligning facial images with textual prompts. FLIP [7] fine-tunes pretrained vision-language models by associating facial images with textual prompts such as “a photo of a real face” or “a photo of a fake face”, achieving substantial improvements in DG-PAD. Building on this, subsequent works [8], [39]–[43] propose frameworks that more effectively leverage multi-modal knowledge by disentangling features or constructing fine-grained prompts. Although existing CLIP-like methods achieve better performance than traditional PAD methods, their generalizability is still constrained by domain-specific biases.

2.2 Visual Concept Discovery in XAI↩︎

Explainable AI (XAI) aims to improve the interpretability and transparency of deep learning models by providing explanations for their decisions. In computer vision, gradient-based methods [13], [44]–[46] leverage activation and gradient information to attribute decisions to specific image regions. Perturbation-based methods [47] perturb the input image and observe the resulting changes in the model’s output to infer decision attribution. However, these methods focus on attributing individual decisions locally and lack the ability to explain the model from a global perspective. In contrast, concept-based approaches aim to generate multiple global explanations by analyzing groups of data that share common characteristics. Kim et al. [48] first introduced a method that goes beyond attribution-based approaches by measuring the influence of pre-selected concepts on a model’s outputs. ACE [14] and CRAFT [15] further advance this idea by automatically discovering concepts through clustering and factorization of intermediate neural activations. Unfortunately, existing concept-based XAI methods are primarily designed for general classification tasks and are not well suited to specific downstream applications. Building on these ideas, we seek to adapt concept-based XAI techniques for PAD, using visual concepts as a novel source of supervision.

2.3 Motivation↩︎

In the history of PAD research, auxiliary supervision from signals such as rPPG, depth maps, and NIR images has been shown to improve domain generalization by providing complementary cues beyond RGB data. At the same time, explainability techniques enable us to uncover machine-level concepts and heatmaps from powerful pretrained models, offering a new form of supervision that does not rely on additional sensors. This perspective differs from prior attention-guided PAD methods [36] that provide sample-level saliency supervision, as well as recent CLIP-based methods such as TF-FAS [41] and CCPE [40] that mainly transfer pretrained knowledge through richer semantic guidance or prompt engineering. Motivated by this, we propose a Visual Concept-driven Enhancement (VCE) process to distill PAD-relevant concepts and heatmaps from the CLIP visual encoder. These concepts and heatmaps are incorporated into a novel learning framework as auxiliary supervision signals, which are injected into learnable prompts to strengthen the generalization ability of PAD models.

3 Method↩︎

The proposed Concept-informed Prompts Guided PAD (CPG-PAD) can be decomposed into two steps. We first enhance each domain through Visual Concept-driven Enhancement (VCE). Then we learn multiple concept-informed prompts via Prompt-based Concept Injection (PCI) process. The total framework of CPG-PAD is shown in 2.

3.1 Preliminaries↩︎

We adopt a general domain generalization PAD training setting. \(\mathcal{D}\)=\(\{D_i, i = 1, 2, \cdots, I\}\) denotes the domain data where the \(i-th\) domain \(\mathcal{D}_i = \{(\boldsymbol{x}_j, y_j), j=1, 2, \cdots, N_i\}\) contains \(N_i\) image and label pairs. \(\mathcal{C}(\cdot)\) denotes CLIP model with pretrained parameters. CLIP model \(\mathcal{C}(\cdot)\) contains two parts, the visual encoder \(VisEnc(\cdot)\) extracts CLS token \(V_{cls} \in \mathbb{R}^{d_v}\) and image tokens \(V_{img} \in \mathbb{R}^{H \times W \times d_v}\) and the text encoder \(TextEnc(\cdot)\) generates text feature \(F \in \mathbb{R}^{d_t}\) for each category: \[\begin{align} V_{cls}, V_{img} = VisEnc(\mathcal{X}); \;\;\;F = TextEnc(\mathcal{T}) \nonumber \end{align}\] where \(\mathcal{X}\) and \(\mathcal{T}\) indicate input image and prompts respectively. Finally, visual tokens \(V_{cls}, V_{img}\) and text feature \(F\) are projected to the same dimension \(C\) to calculate similarity scores. Notably, \(d_t=C=512\), \(d_v=768\) in ViT-B/16.

3.2 Visual Concept-driven Enhancement↩︎

In this section, we introduce Visual Concept-driven Enhancement (VCE), which operates in two stages: concept discovery and concept-associated heatmap generation. Notably, VCE utilizes a pretrained CLIP model without requiring any parameter updates. The detailed process is illustrated in 3. For clear illustration, we use the symbol \(N\) to represent the number of domain inputs for all domains.

3.2.1 Concept Discovery↩︎

The process of concept discovery is illustrated in 3 (a) with dark yellow box. We generally consider a single domain \(\mathcal{D}_i\), the data can be simply represented as input images \((\boldsymbol{x}_1, \cdots, \boldsymbol{x}_{N}) \in \mathcal{X}^{N}\) where \(\boldsymbol{x_j} \in \mathbb{R}^{D}\) and their associated labels \((y_1, \cdots, y_{N}) \in \mathcal{Y}^{N}\). Firstly, we select subsets of images \(\mathcal{X}^{fake/real}_{sub} \in \mathbb{R}^{N_s \times D}\) from the original dataset \(\mathcal{X}^{N}\) for both real and fake class to discover concepts. Here we randomly select \(r\) frames from each video to form the subset where \(r\) is a hyperparameter (e.g. select a single frame from each live video to form \(\mathcal{X}^{real}_{sub}\)). We define \(\pi(\cdot)\) as a straightforward filter function to create candidate concepts. Specifically, \(\pi(\cdot)\) crops the image into \(64 \times 64\) patches at regular intervals in both vertical and horizontal directions and resize each patch to dimension \(D\). Feed \(\mathcal{X}_{sub}\) into \(\pi(\cdot)\) to obtain an auxiliary dataset \(\mathcal{X}^{fake/real}_{patch} \in \mathbb{R}^{N_a \times D}\) which contains all candidate concepts. The following part will focus on the auxiliary dataset of a single class \(\mathcal{X}_{patch} \in \mathbb{R}^{N_a \times D}\), as the same operations apply to both the fake and real classes.

To automatically discover visual concepts from domain auxiliary dataset \(\mathcal{X}_{patch}\), we feed it to the pretrained visual encoder to obtain activation \(\mathbf{A} = VisEnc (\mathcal{X}_{patch}) \in \mathbb{R}^{N_a \times H \times W \times C}\) where \(HW\) indicates the shape of activation map and \(C\) indicates the number of channels. We apply Semi-NMF (Semi Non-negative Matrix Factorization) [49] to factorize activation maps, as it can naturally handle mixed-sign activations in CLIP features while encouraging interpretable, parts-based representations, which are difficult to achieve with other factorization methods. For instance, standard NMF is not applicable due to its non-negativity constraint on inputs, while PCA tends to produce less semantically interpretable components [50]. Semi-NMF decomposes the average pooled activations \(\mathbf{\bar{A}} = AvgPool(\mathbf{A}) \in \mathbb{R}^{N_a \times C}\) into a product of concept coefficients \(\mathbf{U} \in \mathbb{R}^{N_a \times K}\) and concept basis \(\mathbf{W} \in \mathbb{R}^{C \times K}\) by solving: \[\begin{align} (\mathbf{U}, \mathbf{W}) = \mathop{\arg\min}\limits_{\mathbf{U} \geq 0, \mathbf{W}} \Vert \mathbf{\bar{A}} - \mathbf{U} \mathbf{W}^\intercal \Vert^2_F, \label{eq:seminmf} \end{align}\tag{1}\] where \(K\) indicates the number of concepts one wishes to discover and \(\Vert \cdot \Vert^2_F\) denotes the Frobenius norm.

According to Ding et al. [49], the objective can be solved by iteratively updating \(\mathbf{U}\) and \(\mathbf{W}\) as follows: \[\begin{align} & \mathbf{W} = \boldsymbol{\bar{A}}^\intercal \mathbf{U} (\mathbf{U}^\intercal \mathbf{U})^{-1}, \\ & \mathbf{U} = \mathbf{U} \cdot \sqrt{\frac{ (\boldsymbol{\bar{A}} \mathbf{W})^{+} + \mathbf{U} (\mathbf{W}^\intercal \mathbf{W})^{-} }{ (\boldsymbol{\bar{A}} \mathbf{W})^{-} + \mathbf{U} (\mathbf{W}^\intercal \mathbf{W})^{+} } }, \end{align}\] where the separated parts of matrix \(\mathbf{M}\) are: \[\begin{align} \mathbf{M}^{+} = (\vert \mathbf{M} \vert + \mathbf{M}) / 2, \;\;\;\mathbf{M}^{-} = (\vert \mathbf{M} \vert - \mathbf{M}) / 2 \end{align}\]

\(\mathbf{W}\) is the discovered concepts where each column \(\mathbf{W}_k \in \mathbb{R}^{C}\) corresponds to a single concept basis. These concepts will be utilized in the subsequent heatmap generation process.

3.2.2 Concept-Associated Heatmap Generation↩︎

The process of concept-associated heatmap generation is illustrated in Figure 3 (b) with green box. After concept discovery, we get \(K\) concept basis \(\mathbf{W} \in \mathbb{R}^{C \times K}\) from the selected subset \(\mathcal{X}_{sub}\) with \(N_s\) images, which serves as an approximation of the concepts present in the original domain containing \(N\) images. Given an image \(\boldsymbol{x}\) (from the original domain), we factorize the activations \(\mathbf{A} = VisEnc(\boldsymbol{x}) \in \mathbb{R}^{H \times W \times C}\) in each position into several concept coefficients \(\mathbf{U} \in \mathbb{R}^{H \times W \times K}\) corresponding to \(\mathbf{W}\) by solving similar objective in Equation 1 : \[\begin{align} (\mathbf{U}(s, t), \mathbf{W}) = \mathop{\arg\min}\limits_{\mathbf{U}(s, t) \geq 0, \mathbf{W}} \Vert \mathbf{A}(s, t) - \mathbf{U}(s, t) \mathbf{W}^\intercal \Vert^2_F, \label{eq:seminmf2} \end{align}\tag{2}\] where \(\mathbf{W}\) is fixed based on the values obtained during the concept discovery process, and \(\mathbf{A}(s, t) \in \mathbb{R}^{C}\) represents the activation vector at position \((s, t)\) within the spatial dimensions \((H, W)\). The result \(\mathbf{U}(s, t) \in \mathbb{R}^{K}\) indicates the concept coefficients in position \((s, t)\).

Specifically, we can regard \(\mathbf{U}(s, t, k)\) as the influence of concept \(k\) at \((s, t)\) position of the activation map and the \(\mathbf{U}(\cdot, \cdot, k)\) can be seen as an attention map of concept \(k\). In this way, we factorize a single input image into several concepts and their corresponding attention maps. With the help of \(\mathbf{U}\), we can get the fine-grained feature heatmaps by standardizing them to have zero mean and unit variance: \[\begin{align} \mathcal{H}^{gt}(k) = \frac{\mathbf{U}(\cdot, \cdot, k)- mean(\mathbf{U}(\cdot, \cdot, k))}{std(\mathbf{U}(\cdot, \cdot, k))}, \end{align}\] where \(mean\) and \(std\) are operators to calculate the mean and variance of activation map \(\mathbf{U}(\cdot, \cdot, k)\).

3.3 Prompt-based Concept Injection↩︎

In this section, we propose Prompt-based Concept Injection (PCI), which learns concept-informed prompts by integrating visual concept information into textual prompts. PCI constructs multiple learnable prompts \(\mathcal{S}\) for both fake and real classes. For concept injection, \(\mathcal{S}\) is decoded into heatmaps \(\mathcal{H}^{dec}\) via a proposed Visual-Prompt Decoder (VPD) and supervised by concept-associated heatmaps \(\mathcal{H}^{gt}\) generated by the former VCE to align with discovered visual concepts. For PAD classification, the well-learned concept-informed prompts \(\mathcal{S}\) guide CPG-PAD to identify generalizable features and project to the final classification logits \(l\).

3.3.1 Learnable Prompts↩︎

The learnable prompts \(\mathcal{S}\) consist of two components: fixed class-wise text descriptions \(\mathcal{P}^{text}\) and multiple learnable prompt embeddings \(\{ \mathcal{T}^{prompt}_j \}^{J}_{j=1}\), where \(J\) is a hyperparameter. Following the setup in [7], we construct the fixed class-wise text description \(\mathcal{P}^{text}\) as {A photo of a fake face} and {A photo of a real face}. These texts are tokenized and embedded using CLIP’s tokenizer and embedding layer, resulting in fixed class-wise embeddings \(\mathcal{T}^{text} \in \mathbb{R}^{n_t \times d_t}\). The learnable prompt embeddings \(\{ \mathcal{T}^{prompt}_j \}^{J}_{j=1}\) where \(\mathcal{T}^{prompt}_j \in \mathbb{R}^{n_p \times d_t}\) are concatenated with the fixed class-wise embeddings \(\mathcal{T}^{text}\) and the necessary \([SOS]\) and \([EOS]\) embeddings to form complete learnable prompts \(\mathcal{S}\). The process can be formulated as follows: \[\begin{align} & [SOS], \mathcal{T}^{text}, [EOS] = \mathcal{C}_{Embed}( \mathcal{C}_{tokenize}( \mathcal{P}^{text} ) ), \nonumber \\ & \;\;\;\;\;\;\;where \;\; \mathcal{T}^{text} \in \mathbb{R}^{n_t \times d_t},\;\;[EOS], [SOS] \in \mathbb{R}^{d_t},\nonumber \\ & \mathcal{S}_j = concat([SOS], \mathcal{T}^{prompt}_j, \mathcal{T}^{text}, [EOS]) \in \mathbb{R}^{77 \times d_t}, \nonumber \\ & \mathcal{S} = \{ \mathcal{S}_j \}^{J}_{j = 1} \in \mathbb{R}^{J \times 77\times d_t}, \nonumber \end{align}\] where \(\mathcal{C}\) indicates CLIP model, \(d_t\) indicates embedding dimension of the text encoder in CLIP. The parameters \(n_t\) and \(n_p\) correspond to the embedding lengths of the fixed class-wise embeddings and the learnable prompt embeddings, respectively, ensuring that their sum satisfies \(n_t + n_p = 75\). The complete learnable prompts \(\mathcal{S}\) are subsequently aligned with visual concept information through concept injection. Notably, the same learnable prompts \(\mathcal{S}^{r/f} =\mathcal{S}\) are constructed for both fake and real classes.

3.3.2 Learning Prompts via Concept Injection↩︎

The prompt learning process via concept injection is illustrated in Figure 2 with blue box. With the constructed learnable prompts \(\mathcal{S}^{r/f}\), we propose a Visual-Prompt Decoder (VPD) to better learn multiple concept-informed prompts for both fake and real classes. VPD can effectively decode multiple feature heatmaps with the interaction between multiple prompts \(\mathcal{S}^{r/f}\) and image tokens \(V_{img}\), which is later aligned with discovered concepts. For clear description, we use \(\mathcal{S}\) to represent \(\mathcal{S}^f\) or \(\mathcal{S}^r\).

Feed the prompts \(\mathcal{S}\) into the text encoder to obtain the prompt features \(F = TextEnc(\mathcal{S}) \in \mathbb{R}^{J \times C}\). Similarly, process the input image through the image encoder to derive a CLS token and image tokens \(V_{cls}, V_{img} = VisEnc(X)\) where \(V_{cls} \in \mathbb{R}^{C}\) and \(V_{img} \in \mathbb{R}^{H \times W \times C}\). VPD takes the prompt features \(F\) and image tokens \(V_{img}\) as inputs and decodes into heatmaps \(\mathcal{H}^{dec} = VPD(F, V_{img})\) through their interactions. The decoded heatmaps are later supervised by concept-associated heatmaps \(\mathcal{H}^{gt}\).

Specifically, VPD is a module with several layers of multi-head self-attention mechanism \(MSA(\cdot)\) (identical to the traditional Transformer [51]) and a newly proposed multi-head dual-attention module \(MDA(\cdot)\). Multi-head dual-attention calculates a single attention map for the two inputs and updates both through matrix multiplication to enable effective interaction, illustrated in Figure 4. For each layer in VPD, it first applies multi-head self-attention to both the image tokens \(V_{img}\) and the prompt features \(F\), producing \(\hat{V}_{img}\) and \(\hat{F}\), respectively. Then, the processed image tokens \(\hat{V}_{img}\) and prompt features \(\hat{F}\) are projected into query \(Q_*\) and key \(K_*\) representations using separate linear layers with layer normalization, resulting in \(Q_V, K_V\) for image tokens and \(Q_F, K_F\) for prompt features where \(Q_V, K_V \in \mathbb{R}^{H \times W \times C}\) and \(Q_F, K_F \in \mathbb{R}^{J \times C}\). Next, the attention matrix \(Attn\) is computed using a scaled dot-product between \(Q_V\) and \(Q_F\), followed by a softmax operation forming the shape \(\mathbb{R}^{H \times W \times J}\). The final representations are obtained by applying two separate Multi-Layer Perceptrons (MLPs) to the attended features. The image tokens are updated using the attended prompt keys \(K_F\), while the prompt features are refined through the attended image keys \(K_V\). Notably, the MLPs consist of two linear layers with GELU activation.

After passing through the Depth layer of the VPD, the decoded heatmap \(\mathcal{H}^{dec} \in \mathbb{R}^{H \times W \times J}\) is obtained via \(\mathrm{Conv2d} \left(F @ {V_{img}}^\intercal\right)\). With the generated heatmaps \(\mathcal{H}^{gt}\), we inject visual concept information into learnable prompts through concept mapping loss \(\mathcal{L}_M\) which supervises the class-specific heatmaps conditioned on the ground-truth label by minimizing the optimal Mean Square Error (MSE) through Hungarian Matching [52]: \[\begin{gather} \mathcal{H}^{dec} = (1-\delta)\mathcal{H}^{dec}_{fake} + \delta\mathcal{H}^{dec}_{real}, \text{where } \delta = \mathbb{I}(\text{label=real}), \nonumber \\ \mathcal{L}_M = \frac{1}{J} \min_{\pi \in S_J} \sum_{j=1}^{J} MSE(\mathcal{H}^{dec} (j), \mathcal{H}^{gt}(\pi(j))). \end{gather}\] where \(\mathbb{I}(\cdot)\) is the indicator function, \(S_J\) is the set of all permutations of \(\{1, \dots, J\}\), \(\pi\) is the optimal permutation found via the Hungarian algorithm.

Thus, we learn the concept-informed prompts \(\mathcal{S}^{r/f}\) through visual concept injection with the help of the VPD.

3.3.3 PAD Classification↩︎

With the learned concept-informed prompts, CPG-PAD can identify significant features in PAD classification process, illustrated in 2 with pink dotted box. Specifically, prompt features \(F^{r/f}\) are used to calculate the similarity score \(s = concat(F^r @ V_{cls}, F^f @ V_{cls})\) where \(s \in \mathbb{R}^{J*2}\). A Linear layer is used to project the similarity score to logits along with a softmax operation \(l=Softmax(Linear(s))\). We supervise the logits using traditional cross entropy loss \(\mathcal{L}_{cls} = CE(l, label)\).

The total loss \(\mathcal{L}\) can be formulated in two parts, classification loss \(\mathcal{L}_{cls}\) and concept mapping loss \(\mathcal{L}_M\): \[\begin{align} &\mathcal{L} = \mathcal{L}_{cls} + \alpha \mathcal{L}_M. \end{align}\]

4 Visual Concept Discovery Analysis↩︎

In this section, we analyze the automatically discovered visual concepts and the generated heatmaps through Visual Concept-driven Enhancement (VCE) process. We choose to analyze CelebA-Spoof [53] dataset which is a large-scale PAD dataset containing rich attributes on face, illumination, environment and attack types.

4.1 Concept Discovery Stability↩︎

To evaluate the stability of our concept discovery algorithm, we conduct experiments with three different settings (\(r=1,2,3\)) on each dataset-class combination. For each setting, we repeat the concept discovery process with three different random seeds. For each pair of runs, we compute the cosine similarity between the extracted concept basis vectors and apply maximum bipartite matching (Hungarian algorithm) to find the optimal one-to-one correspondence. Note that Semi-NMF uses a deterministic K-means initialization, and thus different random seeds do not introduce additional variation.

1 reports the average cosine similarity of the Top-\(K\) matched concepts across all pairwise comparisons. The Top-5 concepts achieve 97.1% average similarity, Top-10 concepts achieve 93.7%, and all 15 concepts achieve 84.4%, all significantly higher than the random baseline (5.5%). We also observe that the most discriminative concepts (e.g., paper edges, cut holes) consistently appear within the Top-10 concepts. These results indicate that the concept discovery process is highly stable across different \(r\) values.

4.2 Visual Concept Visualization↩︎

We employ ActMax [54], a technique used in the XAI domain for concept visualization, to project the concepts, originally represented as features, into the RGB space. Specifically, ActMax selects the top-k most activated patches for each concept and treats them as representatives.

The discovered concepts in CelebA-Spoof dataset are presented in 5. In 5 (a), VCE identifies semantic attack concepts, such as the ‘cut holes’ made for the eyes or mouth and the ‘fold lines’ created by bending the paper, whereas others may be more subtle, such as the ambiguity introduced by print attacks. Notably, although some concept visualization may appear similar for real and fake classes, they are distinguishable in the feature space. The results demonstrate that our method can uncover meaningful visual concepts.

4.3 Concept-Associated Heatmap Visualization↩︎

With the discovered visual concepts, VCE generates concept-associated heatmaps for each sample. After normalization, the heatmaps are resized to 224 × 224 for clearer visualization. The visualizations on the CelebA-Spoof dataset are presented in 6, where red denotes stronger attention and blue denotes weaker attention. As shown, the generated heatmaps align well with the discovered visual concepts. For instance, in 5, the heatmaps of concept 1 and concept 4 highlight the eyes and mouth, while those of concept 2 and concept 3 capture clear attack cues such as ‘fold lines’ and ‘cut holes’.

5 Experiments↩︎

5.1 Experimental Setups↩︎

5.1.1 Datasets↩︎

We employ nine datasets to evaluate the performance of our method, including MSU-MFSD (M) [66], CASIA-FASD (C) [67], Idiap Replay-Attack (I) [68], OULU-NPU (O) [69], CASIA-SURF (S) [70], CASIA-CeFA (C) [71], WMCA (W) [72], CelebA-Spoof [53] and SiW-Mv2 [73]. Idiap Replay-Attack, CASIA-FASD, MSU-MFSD and OULU-NPU datasets (ICMO) differ in material, lighting, background, and resolution. CASIA-SURF, CASIA-CeFA, and WMCA datasets (CSW) encompass a broader range of topics, various types of attacks, and diverse sampling environments. In all experimental protocols, each dataset is treated as an individual domain, and we use the notation \(XY \rightarrow Z\) to indicate that datasets X and Y serve as the source domain for training, while dataset Z is used as an unseen target domain for evaluation.

5.1.2 Protocols↩︎

Following previous works [7], [8], we adopt 4 evaluation protocols to assess the generalizability of our method: multi-source evaluation (P1), limited-source evaluation (P2 and P3), single-source evaluation (P4), unknown PAI (Presentation Attack Instruments) evaluation (P5). For each protocol, we train the model on the training sets of the source domains and evaluate it on the test set of the target domain. Following previous works, evaluation is conducted at the video level, where two frames are sampled from each video and their scores are averaged to obtain the final video-level prediction.

5.1.3 Evaluation Metrics↩︎

Following standard evaluation principles, we employed three key metrics, HTER, AUC, and TPR, to assess the model’s performance. (1) HTER (Half Total Error Rate) quantifies the trade-off between false rejection and false acceptance errors, computed as the average of the False Rejection Rate (FRR) and the False Acceptance Rate (FAR). (2) AUC (Area Under the Curve) measures the overall classification performance by representing the area under the Receiver Operating Characteristic (ROC) curve. (3) TPR (True Positive Rate) evaluates the model’s ability to correctly identify attack samples, with the classification threshold adjustable based on specific application requirements to optimize performance. Besides, following standard ISO/IEC 30107-3 [65] for biometric PAD, we report (1) the BPCERs (Bona Fide Presentation Classification Error Rate) observed at APCER (Attack Presentation Classification Error Rate) values or security thresholds of 1% (BPCER100), 5% (BPCER20), and 10% (BPCER10) and draw DET curves for P1 protocol.

Thresholding protocol and limitation: Following common cross-database protocols, HTER is computed using the EER threshold estimated from the target test scores for direct comparison with prior works. We acknowledge that such a posteriori thresholding may introduce optimistic bias, and therefore additionally report results under a stricter setting where the threshold is determined on the validation set and then evaluated on the test set (P1). We also provide BPCER/APCER metrics and DET curves to complement HTER.

5.1.4 Implementation Details↩︎

For the PAD task, we use a pretrained CLIP [12] model from OpenAI⁶ with ViT-B/16 as the image encoder, the same as previous methods. We keep the parameters in the text encoder unchanged, and finetune the image encoder with a simple Convpass adapter. We resize the image to 224×224 with a batch size of 24, and train all models for 50 epochs. We use the Adam optimizer and set the learning rate to 1e-4 for training. In all experiments we set \(r=2\), \(K=15\) and set the number of learnable concept-informed prompts \(J=K=15\). We set the depth of VPD \(Depth=4\), the coefficient of concept mapping loss \(\alpha=0.01\).

5.2 PAD Performance↩︎

5.2.1 Multi-source evaluation (P1)↩︎

Multi-source evaluation (P1). In P1 protocol, we construct a domain generalization benchmark using four datasets, ICMO, resulting in 4 scenarios where three datasets are seen as source domains and the remaining one serves as the target domain. We include traditional PAD methods [9], [10], [27]–[29], [31], [55]–[59] and current proposed CLIP-based methods [8], [12], [39], [40], [60], [61] as baseline. As shown in 2, we can draw the following conclusions: (1) CLIP-based methods outperform the traditional DG methods, even the finetuned CLIP model has a lower average HTER (5.43%) compared to the most competitive traditional DG method GAC-FAS (6.52%), supporting the motivation that general knowledge from CLIP model can benefit PAD task. (2) Without supplementary training data, the proposed CPG-PAD outperforms SOTA methods with an average HTER of 2.08%, surpassing S-CPTL [39] by 1.25% and even surpassing S-CPTL* which is trained with additional CelebA-Spoof dataset. On all 8 settings, CPG-PAD achieves top-two performance on 7 of the settings. (3) With supplementary training data (CelebA-Spoof), CPG-PAD also surpasses the recently proposed methods with an average HTER of 1.19%. The above results and conclusions prove that CPG-PAD can better leverage the general knowledge from pretrained CLIP model and effectively enhance PAD generalizability in multi-source scenarios.

Recent MLLM-based FAS methods [41], [64], [84] introduce large language models or multimodal reasoning into PAD, but usually require substantially larger model capacity than CLIP-based frameworks. As we can see in 3, our method CPG-PAD surpasses TF-FAS [41] and I-FAS [64] on average HTER metric with a small margin. However, achieving a performance level that is nearly on par is already sufficient to highlight the effectiveness of CPG-PAD since CPG-PAD is a vision-only model that does not rely on any extra annotations or LLMs.

Following the standard ISO/IEC 30107-3 [65], we further compare CPG-PAD with the reproducible FLIP baseline using extra metrics as reported in 4, together with the DET curves in 7. Extra metrics include BPCER10, BPCER20, BPCER100 and BPCER/APCER/HTER results using EER threshold on validation set. Although FLIP achieves better performance in one protocol, CPG-PAD obtains better results in most cases and shows stronger overall operating-point performance. The DET curves further demonstrate that CPG-PAD yields a more favorable APCER-BPCER trade-off.

5.2.2 Limited-source evaluation (P2 and P3)↩︎

In this evaluation experiment, we test on two protocols (P2 and P3) with two datasets as source domains and one dataset as target domain. For P2, we establish a domain generalization benchmark focused on multimodal sensors, incorporating the CSW datasets, leading to 3 sub-experiments where two datasets are seen as source domain and the remaining one serves as target domain. As 5 shows, without additional training data, the proposed CPG-PAD outperforms the previous SOTA method S-CPTL [39] with an average HTER of 9.91% and achieves top-two performance across eight of nine metrics. With CelebA-Spoof as additional training data, the proposed CPG-PAD also achieves SOTA performance with an average HTER of 6.59%. Notably, as the results show, the SW \(\rightarrow\) C scenario is difficult for all previous methods since the domain gap is larger than the other two scenarios. However, CPG-PAD* achieves nearly half the HTER of the second-best CFPL-FAS*. For P3, we follow [83] and construct 2 scenarios with ICMO datasets which set M and I as source domains and test on C and O separately. In all settings, the proposed CPG-PAD surpasses previous methods, demonstrating its strong generalizability in limited-source PAD scenarios.

5.2.3 Single-source evaluation (P4)↩︎

For P4, we strictly follow the setup in [7] to construct 12 scenarios using a single-source-to-single-target approach, leveraging the ICMO datasets. As shown in 6, the proposed CPG-PAD surpasses suboptimal method CDFTN-L [78] on nine of twelve settings with an average HTER of 5.33%, lower than half of CDFTN-L. Since it’s not fair to leverage pretrained models, we trained CPG-PAD with an additional CelebA-Spoof dataset to compare with FLIP-MCL* (both leverage pretrained models). The results indicate that CPG-PAD can outperform FLIP-MCL* with an average HTER of 3.36% demonstrating its strong generalizability on single source scenarios.

5.2.4 Unknown PAI evaluation (P5)↩︎

We evaluate the generalizability of the CPG-PAD for the challenging scenario of unknown PAI, including 3D masks (i.e. silicone masks, transparent masks and mannequin head) and make-up (obfuscation, impersonation and cosmetic). For this purpose, we follow the leave-one-out protocol in [37] using SiW-Mv2 [73] database: thirteen PAI species are used for training and the remaining PAI species is tested. 7 reports in compliance with ISO/IEC 30107-3 and benchmarks against the SiW-Mv2 baseline in terms of HTER and BPCER.

Compared with our CLIP baseline, CPG-PAD achieves clear improvements under the unknown-PAI protocol on SiW-Mv2, reducing the average HTER from 3.03% to 1.02% and the average BPCER100 from 11.08% to 6.65%. The gains are particularly evident on attack instruments with stronger appearance variation or localized attack cues, such as obfuscation make-up, cosmetic make-up, Funny Eye covering, transparent mask, and print attacks. For example, the HTER on obfuscation decreases from 14.79% to 0.77%, while the BPCER100 on cosmetic make-up drops substantially from 61.69% to 22.61%. These results indicate that concept guidance is especially beneficial for unseen PAIs involving facial region modification, occlusion boundaries, and material inconsistencies.

5.3 Ablation Studies↩︎

We conduct ablation studies on the same P1 benchmark and report results across all scenarios to demonstrate the effectiveness of each component of CPG-PAD as well as the influence of key hyperparameters.

5.3.1 Effect of CPG-PAD components↩︎

To investigate the effect of each component in CPG-PAD, we gradually add them to the baseline CLIP and report the results of P1 in 9. The differences of each component are reported in parentheses, with green denoting improvements and red denoting degradations. The CLIP baseline in 9 refers to a CLIP-based PAD classification baseline with fixed class prompts, a frozen text encoder, and visual-side adaptation through a lightweight Convpass adapter, rather than a fully finetuned CLIP trained with contrastive loss. On the base of CLIP, we first add Multiple Learnable Prompts (MLP) to encourage the model to freely discover different sub-direction features that limitedly improve the performance. Adding MLP improves average HTER from 5.43% to 3.49% since it provides the model with greater flexibility. Then we add the Concept Mapping Loss (CML) with a simple 2D convolution layer after the visual encoder. This makes a further step to improve the performance since the auxiliary supervision of concept-associated heatmaps helps model to learn more generalizable features. However, it can not fully leverage the visual concept information since the decoder is only a simple 2D convolution layer. Finally, we change the simple decoder with the proposed Visual-Prompt Decoder (VPD) to make full use of the concept-associated heatmaps, which improve the average HTER to 2.08%. This allows the model to effectively align the textual prompts with visual concepts, thereby enhancing the generalizability of PAD task. While adding different components of CPG-PAD to baseline CLIP, HTER and AUC is improved on almost all scenarios (only one except), indicating the effectiveness of each component proposed in CPG-PAD.

5.3.2 Effect of Learnable Prompt Number↩︎

We report the result of P1 with different \(J\) in 10. Since the concept number \(K\) is set to 15 empirically, there are only 15 heatmaps for each sample. Thus, the number of learnable prompt \(J\) can’t exceed \(K\). We uniformly select values in the range of 3 to 15 to analyze the impact. As \(J\) increases, the model is able to learn and exploit a greater number of concepts concurrently. In 10, we come up with the conclusion that the optimal value of \(J\) is 15 which utilizes all visual concepts. Moreover, larger values of \(J\) correspond to better performance, indicating that the more concepts discovered by VCE are utilized, the better the results. This demonstrates the effectiveness of VCE.

5.3.3 Effect of Visual-Prompt Decoder Depth↩︎

Visual-Prompt Decoder (VPD) plays a crucial role in the Prompt-based Concept Injection (PCI) process (shown in 9). To investigate its impact on cross-domain performance, we experimented with different \(Depth\) of VPD layers, where 0 layers indicate replacing VPD with a simple 2D convolution layer. We observed that increasing the number of layers improves performance when \(Depth\) is below 4; however, when \(Depth\) is set to 8, the performance drops. Our analysis suggests that an overly complex network may make the decoder too strong, preventing useful information in the heatmaps from being effectively back-propagated to the learnable prompts, thereby leading to degraded performance. Moreover, an 8-layer network also introduces a significant increase in parameters and computational costs. Therefore, setting \(Depth\) to 4 offers the best trade-off, achieving optimal performance without excessive computational burden.

5.3.4 Effect of Concept Mapping Loss Coefficient↩︎

The coefficient of Concept Mapping Loss (CML) is also an important hyper-parameter of CPG-PAD method. The result of different coefficient \(\alpha\) is reported in 12. A small \(\alpha\) (0.005 and 0.001) dilutes the effect of CML with the basic cross-entropy loss, leading to performance degradation. On the other hand, a large \(\alpha\) (0.1) also results in degraded performance. This is because the concept-associated heatmaps are extracted from the visual feature space of the pretrained CLIP, whereas in the PAD task, the optimal visual space may not be perfectly aligned with that of pretrained CLIP. A large \(\alpha\) forces the model to over-align with the pretrained CLIP, which ultimately harms performance. Therefore, an intermediate value of \(\alpha=0.01\) proves to be the optimal choice.

5.3.5 Effect of Backbone↩︎

To study the effect of different backbone version, we use different ViT version as backbone including ViT-B/16, ViT-B/32 and ViT-L/14. The results are shown in 13. We can draw two conclusions from the results: (1) The best performance is achieved when using ViT-L/14 as the backbone, which is consistent with expectations since ViT-L/14 (\(\sim\)​304M parameters) has significantly more capacity than ViT-B/16 (\(\sim\)​86M). (2) Using ViT-B/32 leads to a noticeable performance degradation. This is mainly because, given a 224×224 input, it produces only 7×7 patches, which is much coarser compared to the 14×14 patches in ViT-B/16, resulting in a loss of fine-grained information.

5.3.6 Effect of VCE Frame Number↩︎

In the concept discovery stage of VCE, we sample \(r\) frames per video to discover visual concepts. To investigate the impact of \(r\), we conduct experiments with \(r=1,2,3\), and report the results in 14. The results show that the best value is \(r=2\). Using fewer frames may lead to information loss, while incorporating more frames can introduce redundant information and noise interference. Across different values of \(r\), CPG-PAD demonstrates stable performance with only a marginal variation.

5.4 Visualization and Analysis↩︎

We compare CPG-PAD against the degraded model (CLIP+MLP) through T-SNE [85] visualization on OCM \(\rightarrow\) I scenarios in P1. As shown in 8, the CLIP+MLP baseline produces feature distributions where red (real) and blue (fake) samples are intermixed, and the dark-colored target samples are scattered widely within each class, reflecting large intra-class variation and unclear decision boundaries. By contrast, CPG-PAD forms two visibly distinct clusters of real and fake samples, and the target samples (dark points) are tightly grouped around their source counterparts (light points), showing improved compactness and cross-domain alignment.

5.5 Computational Overhead↩︎

Table 15 shows the computational time for training and inference on a single RTX 3090 GPU, the results show that CPG-PAD is not a heavy burden for both training and inference. The comparison between CPG-PAD and CLIP demonstrates the applicability of CPG-PAD in real-world scenarios.

6 Conclusion↩︎

In this paper, we introduce Concept-informed Prompts Guided PAD (CPG-PAD), a novel framework that enhances the generalizability of PAD models by incorporating visual concepts from pretrained VLMs into PAD process. Concretely, we propose Visual Concept-driven Enhancement (VCE) which can discover PAD-relevant visual concepts and enhance domain data with fine-grained feature heatmaps corresponding to each discovered concept. With the help of these concept-associated heatmaps, Prompt-based Concept Injection (PCI) encourages the model to learn multiple concept-informed prompts via the cooperation of Visual-Prompt Decoder (VPD) and concept mapping loss, allowing CPG-PAD to precisely identify generalizable features when encountering unseen target domains. Extensive cross-domain quantitative experiments and in-depth analyses demonstrate the effectiveness of the proposed CPG-PAD method.

References↩︎

---

1. Haoyuan Zhang is with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China; the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China (e-mail: zhanghaoyuan2023@ia.ac.cn).↩︎

2. Xiangyu Zhu, Ajian Liu and Siran Peng are with the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China (e-mails: xiangyu.zhu@ia.ac.cn, ajian.liu@ia.ac.cn, pengsiran2023@ia.ac.cn).↩︎

3. Li Gao is with the China Mobile Financial Technology Co., Ltd., Beijing 100032, China (e-mail: gaolids@chinamobile.com).↩︎

4. Zhen Lei is with the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China; the Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences, Hong Kong, China; the School of Computer Science and Engineering, the Faculty of Innovation Engineering, Macau University of Science and Technology, Macau, China (e-mail: zhen.lei@ia.ac.cn).↩︎

5. Corresponding author: Zhen Lei.↩︎

6. https://github.com/OpenAI/CLIP↩︎