1 Introduction↩︎

The proliferation of deepfake (DF) content presents an escalating challenge to information integrity across various domains, from journalism and politics to legal proceedings [1], [2]. As generative models continue to advance, realism has surpassed previously unimaginable thresholds, necessitating the development of robust detection methodologies. The majority of existing approaches focus on binary classification [3], [4]. While effective, this approach suffers from a critical limitation: a lack of transparency and justification. In contexts demanding accountability a simple real or fake label is insufficient. Instead, understanding why a piece of content is flagged as manipulated is paramount.

This need for explainability has spurred recent research into Explainable Deepfake Detection (XDFD). Existing approaches broadly fall into two categories: spatio-temporal localization [5]–[7] and textual explanation methods [8].

Spatio-temporal localization methods aim to identify manipulated regions by highlighting areas of the image or segments of the video that are presumed to contain forged content. The motivation is to move beyond binary prediction and provide visual evidence supporting the decision. However, in practice, these methods often produce coarse localization, frequently defaulting to highlighting the entire face rather than isolating specific manipulation artifacts. As a result, they provide limited insight into what precise cues led to the prediction.

Textual explanation methods instead generate natural language justifications describing why an image was classified as fake. While these explanations improve human interpretability, they often lack accurate spatial grounding and may not faithfully reflect the underlying evidence used by the detector. Consequently, despite their explanatory intent, existing XDFD methods primarily function as prediction tools with attached justification mechanisms, rather than systems that deliver precise and causally grounded explanations.

In this work, we explore XDFD from a different perspective by analyzing deepfake detectors through the lens of post-hoc Explainable Artificial Intelligence (XAI). Rather than modifying detectors to explicitly produce explanations, we aim to uncover and interpret the intrinsic features learned during standard training.

This perspective provides several advantages. First, explanations are derived directly from the representations that drive the detector’s prediction, ensuring tight alignment between explanation and decision function and thereby improving faithfulness. Second, because our approach is purely post-hoc, it introduces no architectural modifications or auxiliary training objectives, avoiding potential trade-offs between interpretability and predictive performance. Third, the framework can be applied to existing pretrained detectors without retraining or artifact-level supervision, making it deployable across models and datasets.

Our technical approach centers on Encoding-Decoding Direction Pairs (EDDP) [9], a recent technique that can unveil a) the concepts that a deep vision network uses to make predictions (its semantic vocabulary), and b) the network’s mechanism for encoding and decoding these concepts in internal representations, under the linear representation hypothesis [10]. We argue that EDDP concepts are a natural fit for XDFD because access to the network’s encoding-decoding mechanism for real and fake concepts enables: (a) global model understanding, by identifying which concepts drive predictions toward real or fake; (b) spatially aware, concept-based local explanations; and (c) counterfactual what-if analysis.

To the best of our knowledge, this is the first application of post-hoc, concept-based XAI within the deepfake detection domain, opening a new avenue for improving transparency and trust in automated content verification systems.

2 Related Work↩︎

2.1 DeepFake Generation↩︎

Deepfake synthesis methods can be broadly categorized into face-swapping, face-reenactment, and audio-driven lip-syncing.

Face-swapping replaces identity while preserving pose and expression. Early works relied on encoder–decoder frameworks and required person-specific training [11], [12]. Later many-to-many models such as SimSwap [13] enabled identity transfer across unseen subjects. Subsequent works improved temporal coherence and realism using transformers and motion–appearance decoupling strategies [14]–[16]. More recently, diffusion-based approaches have emerged [17]–[20]. They leverage pretrained generative priors for high-fidelity synthesis.

Face-reenactment transfers motion and expression from a driving source to a target identity. Face2Face [21] pioneered 3D model-based reenactment, followed by learning-based approaches that emphasize generalization and one-shot transfer [22], [23]. Large-scale diffusion video models further improve realism and temporal stability [24].

Lip-syncing methods generate mouth motion from speech signals. Wav2Lip [25] introduced adversarial synchronization learning, while SadTalker [26] models 3D motion coefficients for natural head dynamics. Diffusion-based systems such as [27] extend this paradigm toward holistic audio-driven human video generation.

2.2 Deepfake Detection and Datasets↩︎

Deepfake detection has evolved alongside advances in generation techniques. Early methods relied on hand-crafted cues, such as abnormal eye blinking or head-pose inconsistencies [28], [29]. CNN-based approaches like MesoNet [3] and XceptionNet [4] captured subtle spatial artifacts, while frequency-aware methods such as F3-Net [30] exploited frequency-domain inconsistencies. Transformer-based architectures later integrated spatial and temporal frequency representations for improved robustness [31], and recent CLIP-based approaches leverage vision-language representations with parameter-efficient fine-tuning to enhance cross-dataset generalization [32], [33].

Datasets have grown in parallel to match these advances, from early benchmarks like UADFV [29] and Deepfake-TIMIT to FaceForensics++ (FF++) [4] and Celeb-DF [34], which improved realism and temporal consistency. Larger-scale collections such as DFDC [35], DeeperForensics-1.0 [36], ForgeryNet [37], and recent benchmarks DF40 [38] and DDL [39] provide greater diversity and coverage of modern generation techniques. These datasets form the foundation for evaluating robust and generalizable deepfake detectors.

2.3 Explainability in DeepFake Detection↩︎

Explainability for deepfake detection remains comparatively underexplored, with most existing works relying on post-hoc saliency maps like Grad-CAM [40] that provide limited insight into model reasoning. Recent research has introduced several distinct strategies to address this. ProtoExplorer [41] utilizes prototype-based reasoning within a human-in-the-loop framework, allowing predictions to be grounded in learned visual archetypes. In a different approach, the Locally-Explainable Self-Blended (LESB) detector [42] focuses on Local Feature Discovery to decompose global predictions into discrete, region-level contributions. This is achieved with part specific self-blending augmentations that focus on key areas like eyes,nose and mouth.

Multimodal transparency is explored in ExDDV [8], which pairs spatial annotations with textual justifications tailored for vision-language models. Furthermore, network dissection is utilized in [43] to quantify transparency by aligning individual internal CNN neurons with semantic facial concepts.

While these individual methods advance the interpretability of detection models, they often introduce significant operational overhead. For instance, the network dissection approach in [43] requires fine-tuning standard architectures, such as VGG-16, Inception V3, or ResNet-50, on forensic datasets to map internal activations to meaningful concepts. Similarly, methods like ProtoExplorer [41] impose specific architectural constraints through the requirement of specialized prototype layers. These dependencies suggest that high interpretability currently relies on additional supervision or task-specific retraining, leaving open the challenge of developing truly scalable, model-intrinsic explanation mechanisms that can be applied to diverse, unlabelled datasets without architectural modification.

3 Unveiling DeepFake Concepts with Encoding-Decoding Direction Pairs (EDDP)↩︎

To understand how deepfake detectors represent and utilize information about real and fake content, we leverage a technique called Encoding-Decoding Direction Pairs (EDDP). A core assumption underlying EDDP is the linear representation hypothesis [10]. This hypothesis posits that concepts, in this case, features indicative of real or fake images, are encoded within image representations as linear combinations of concept embeddings, i.e., vectors in the latent space, each associated with a concept.

3.1 Method↩︎

Given a feature representation \({\boldsymbol{X}}\in \mathbb{R}^{D \times H \times W}\) and its constituent patch embeddings \({\boldsymbol{x}}_{{\boldsymbol{p}}}\in \mathbb{R}^D\), EDDP jointly learns the matrix of decoding directions \({\boldsymbol{W}}\in \mathbb{R}^{D \times I}\), the encoding directions \({\boldsymbol{S}}\in \mathbb{R}^{D \times I}\), and a vector of thresholds \({\boldsymbol{b}}\in \mathbb{R}^I\) for a predefined number of concepts \(I \in \mathbb{N}^+\).

The learned decoding directions \({\boldsymbol{W}}\), coupled with thresholds \({\boldsymbol{b}}\), serve as deepfake related concept detectors. For any patch \({\boldsymbol{x}}_{{\boldsymbol{p}}}\), the presence of concept \(i\) is determined by the condition: \[{\boldsymbol{w}}_i^T {\boldsymbol{x}}_{{\boldsymbol{p}}}- b_i > 0\] The encoding directions \({\boldsymbol{S}}\) represent the concept embeddings, which denote the specific directions in the latent space along which each concept is represented.

3.2 Concept Detection Maps and Concept Contribution Maps↩︎

We localize each concept within an image using concept detectors. To do this, we generate a Concept Detection Map or Concepts Presence Map (CPMs): first, the detector is applied to each spatial element of the image representation \({\boldsymbol{X}}\), then a hard thresholding step creates a binary map, and finally, the map is up-sampled to match the original image resolution. While Concept Detection Maps identify the presence of a concept, they do not quantify that concept’s actual influence on the model’s final prediction.

To quantify this influence, we compute Concept Contribution Maps (CCMs). CCMs are based on a sample’s concept contributions and baseline concept contributions to the explanation logit, which is the difference between the network’s class logit for an input image and a class logit corresponding to a baseline artificial representation from the uncertainty region, where concept detectors are at their decision threshold, as defined by [9]. CCMs provide a spatially aware breakdown and quantify the contributions of each concept across patches.

Visualization examples for CPMs and CCMs are shown in Figure 2.

4 Concept Analysis and Global Understanding↩︎

4.1 Experimental Setup↩︎

For our experimental framework, we employ an Xception [44] architecture pre-trained on the FaceForensics++ (FF++) [4] dataset. We apply the EDDP method to the activations of the model’s 12th residual block, using the FF++ training set as our concept extraction source.

The selection of the 12th residual block is motivated by a trade-off between semantic depth and the dimensionality of the feature space. While deeper layers represent higher-level semantic information, they often undergo a "rank collapse" as the network converges toward a single classification decision. Our empirical analysis using Principal Component Analysis (PCA) indicates that the 12th block maintains a sufficiently high-rank feature space compared to subsequent layers. PCA reveals that this layer maintains a high-rank feature space (rank \(\approx 25\)), capturing over 90% of the variance. We empirically set the number of concepts to 16 concepts after evaluating the trade-off between concept-transfer accuracy, misclassified samples correction success, and semantic simplicity across settings of 12 to 24 concepts.

4.2 Concept Identification↩︎

The objective of our concept identification process is to isolate human-interpretable concepts that govern the model’s final decision-making process. To characterize the learned concepts, establish their semantic groundedness, and analyze their distribution across the dataset, we employ three validation procedures: Relative Concept Activation Vector (RCAV) sensitivity analysis, semantic mapping, and distribution analysis.

It is important to note that the concepts identified through this process are intrinsically coupled with both the architectural priors of the Xception backbone and the specific data distribution of FF++. Because EDDP decomposes the model’s internal latent space, the resulting encoding and decoding directions represent the specific features and artifacts that the detector has learned to prioritize for this particular classification task. Consequently, these concepts reflect the intersection of the model’s representational capacity and the visual cues present within the FF++ dataset

We evaluate the functional influence of each identified concept on the model’s decision-making process using RCAV [45] sensitivity scores. As detailed in Table 1, these scores quantify the model’s sensitivity to specific concepts \(c_i\) with respect to the "Real" and "Fake" class predictions. Our analysis reveals that a subset of concepts (\(c_1, c_3, c_9, c_{10}, c_{12}, c_{14}\)) consistently biases the model’s output toward the "Fake" class, serving as primary indicators of manipulation. Conversely, concepts such as \(c_0, c_5, c_7, c_8,\) and \(c_{11}\) strongly correlate with the "Real" class prediction. Notably, certain concepts (\(c_2, c_4, c_6, c_{15}\)) exhibit negligible impact on the final logit, suggesting they represent features that, while present, are not utilized as discriminative evidence by the detector.

Following this, we quantitatively map the concepts to specific facial regions using Interesection over Union (IoU) scores. These are calculated between concept presence maps and a ground-truth proxy derived from facial segmentation predictions. As detailed in Table 2, this mapping allows us to associate abstract concept directions with interpretable semantic classes (e.g. \(c_5\) showing high IoU with the "nose" region).

Furthermore, we use the IoU metric, the RCAV sensitivity scores as well as qualitative inspection of the top-5 most strongly activated samples (as shown in Figure 1) and assign descriptive names to the identified concepts. A summary of this naming is provided in Table 3.

4.3 Concept Distribution and Manipulation Statistics↩︎

To understand how these concepts manifest across the dataset, we calculate the dataset-wide concept presence as shown in Table 4. This reveals the prevalence of specific features in real versus manipulated imagery. Further, we analyze the overlap between identified concepts and specific manipulation methods present in the FF++ dataset (FS,NT,DF,FF) in Table 5.

Notably, there are certain concepts that exhibit high specificity to certain deepfake types. For instance, \(c_3\) and \(c_9\) show significantly higher presence in DeepFakes (DF) and Face2Face (FF) compared to real images, suggesting these concepts capture artifacts inherent to those specific generative processes. This distribution remains consistent across both training and testing splits, indicating the robustness of the identified concepts.

5 Explanation Faithfulness Assessment↩︎

To evaluate the faithfulness of the learned concept explanations, we introduce a concept cloning intervention that directly manipulates the classifier’s internal representation and measures whether the resulting predictions change in a controlled and predictable manner. If the learned concepts are genuinely integrated into the model’s decision process, then intervening on the concept coefficients should systematically affect the model output.

We begin by decomposing the intermediate representation at a selected layer. Specifically, we express the representation as a base component \({\boldsymbol{X}}_{bc}\), which lies in the uncertainty region of all concepts, and a coefficient vector \({\boldsymbol{u}}\) that quantifies the deviation from this concept-neutral point along each learned signal direction.

To assess faithfulness, we sample pairs of inputs \(({\boldsymbol{X}}_s, {\boldsymbol{X}}_t)\), referred to as source and target examples, and compute their respective decompositions: \[{\boldsymbol{X}}_s = {\boldsymbol{X}}_{bc,s} + {\boldsymbol{S}}^\top {\boldsymbol{u}}_s, \label{eq:source95decomp}\tag{1}\] \[{\boldsymbol{X}}_t = {\boldsymbol{X}}_{bc,t} + {\boldsymbol{S}}^\top {\boldsymbol{u}}_t. \label{eq:target95decomp}\tag{2}\]

We then construct a synthetic representation by combining the source base component with the target concept coefficients: \[{\boldsymbol{X}}_{\mathrm{syn}} = {\boldsymbol{X}}_{bc,s} + {\boldsymbol{S}}^\top {\boldsymbol{u}}_t. \label{eq:synthetic95rep}\tag{3}\]

This operation preserves the structural component of the source representation while transplanting the concept-specific deviations of the target. The synthetic representation is injected into the classifier, and faithfulness is quantified by measuring the agreement between the predictions of the synthetic example and those of the target.

In a complementary evaluation, we assess whether concept-level interventions can correct misclassified samples. For each misclassified instance, we apply targeted interventions to the concept coefficients by adding or removing specific concepts to support the flipping operation. The goal is to flip the prediction to the correct label and report the resulting correction accuracy.

The results of both evaluations are presented in Table 6. The concept-transfer experiment yielded 87.34% accuracy while the intervention on misclassified samples achieved a 99.8% success rate. These results suggest that the concept coefficients successfully capture the specific features the model uses for classification. Specifically, cloning these coefficients onto a different base representation reliably results in the target prediction. Furthermore, by performing targeted interventions on these coefficients, one can successfully steer the model’s internal logic toward correcting misclassified samples. These two observations provide strong empirical evidence that the identified concepts are the primary drivers of the final output.

6 Local Concept-Based Explanations↩︎

To understand how individual concepts influence a specific classification, we decompose the model’s logit with the help of CCMs. We focus on a test sample with a ground truth label of "Fake", which the model correctly classifies.

We employ CCMs to visualize the spatial influence of each concept on the final decision. In Figure 2 , we depict the concept presence maps (CPMs) (left) with their corresponding CCMs (right). We observe that the heatmaps are highly localized, for instance, the activations of \(c_{12}\) and \(c_3\) are highly activated for regions around the eyes and mouth respectively. This confirms that the model is reacting to specific artifacts within these facial components rather than some global noise.

Furthermore in Figure 3 we provide the Grad-CAM [40] heatmap, the CPMs union, and the quantitative decomposition of the prediction where we decompose the total logit into individual concepts contributions and a residual term that is equivalent to the proportion of logits not explained by any concept.

7 What-if-Analysis: Counter-Factual Explanations↩︎

To evaluate the causal influence of our identified concepts on the model’s final decision, we perform a series of counterfactual interventions. By systematically adding or removing concepts, we can observe whether the model’s prediction "flips".

7.1 Intervention methodology↩︎

We define a set of candidate concepts for intervention based on their RCAV sensitivity scores. Specifically, any concept \(c_i\) whose absolute sensitivity score satisfies Eq. 4 is considered a primary driver of the model’s prediction and thus a candidate for manipulation. Formally, \[\label{eq:counterfactual-rcav-sens} |S(c_i)| > \tau,\tag{4}\] where \(\tau = 0.9\) and \(S(c_i) \in \mathbb{R}\) denotes the scalar RCAV sensitivity score associated with concept \(c_i\).

Prior to intervention, we compute concept-specific activation statistics. For each concept, we estimate representative signal values using distributional statistics and compute the corresponding top and bottom quantiles, which are later used during intervention, similar to [9].

Interventions are performed by modifying the latent representation along the direction of the selected candidate concepts. We consider three types of interventions:

(i) Concept addition, where we randomly sample a mask from samples in which the concept is active and substitute the concept-related information using the top-quantile statistics to strengthen its presence; (ii) Concept removal, where we suppress a concept associated with the opposing class by substituting the representation with the bottom-quantile statistics; and (iii) No intervention, where the latent representation remains unchanged if a high-sensitivity concept is already present (or absent) in a manner consistent with the target class.

7.2 Single-Sample Intervention Trace↩︎

Figure 4 illustrates a successful counterfactual trajectory for a sample initially classified as “Fake”. We apply a sequence of interventions, beginning with the removal of fake-associated concepts and followed by the addition of real-associated concepts.

As shown in the logit plot, the initial state exhibits a high “Fake” logit and a low “Real” logit. By removing concepts such as \(c_3\) (fake-mouth) and \(c_9\) (fake-skin), and adding \(c_{11}\) (real-eyes), each intervention progressively shifts the logits toward the target direction, ultimately flipping the prediction. This step-wise transition demonstrates that the model’s decision is strongly tied to the identified concepts.

7.3 Statistical Robustness of Interventions↩︎

To verify that the observed counterfactual behavior is not limited to a single example, we repeat the analysis on a balanced set of 20 samples: 5 real (ground-truth real, predicted real), 5 real (ground-truth real, predicted fake), 5 fake (ground-truth fake, predicted fake), and 5 fake (ground-truth fake, predicted real). Figure 5 reports the mean and range of logit changes across all samples, grouped by their initial prediction.

For samples initially classified as “Real”, interventions consistently decrease the mean “Real” logit while increasing the “Fake” logit. Conversely, samples initially classified as “Fake” exhibit the opposite trend, being steered toward “Real”. The consistency of these trends, reflected in the shaded ranges, suggests that the identified concepts are robust across samples and that the model systematically relies on them to distinguish between real and manipulated faces.

8 Discussion and Conclusion↩︎

In this study, we applied the EDDP method to provide post-hoc interpretability for deepfake detectors. We successfully identified a semantic vocabulary of internal concepts, effectively mapping abstract latent activations to interpretable human-understandable semantics.

Our faithfulness assessments confirmed that the learned concept directions unveil the internals of the model’s decision-making process. The ability to reliably reproduce target predictions by changing their coefficients suggests they capture the essential characteristics used for classification. This is further supported by the CCMs, where concepts and their logit contributions are grounded to facial areas. Finally, the counterfactual "what-if" analysis provides causal evidence for the role of these concepts. By systematically manipulating concept presence and observing the resulting shifts in prediction logits, we demonstrated that the model’s internal logic can be effectively steered. This series of experiments add a level of transparency that is crucial for deepfake detectors.

Despite these strengths, several limitations define the current scope of this work. First, while EDDP does not require retraining the underlying deepfake detector, learning of the EDDP is still required on the representation space of the model to unveil the encoding and decoding directions of concepts. Second, the number of concepts remains a manually defined hyperparameter which may lead to redundant or insufficient signal directions if not selected correctly. Lastly, the identified concepts are inherently optimized for the specific model architecture and dataset used during training. These semantic directions are likely not directly transferable across different models or datasets, meaning a separate training process may be required for each model-dataset combination.

This work demonstrates that the "black-box" nature of deepfake detection can be unraveled through concept-based interpretability. By grounding classification logic in identifiable facial regions and providing a mechanism to steer model predictions we enable more transparent and trustworthy detection.

9 Acknowledgments↩︎

This research has been supported by the European Commission funded program DETECTOR, under Horizon Europe Grant Agreement 101225942.

References↩︎

---

1. Equal contribution↩︎