1 Introduction↩︎

In recent multimedia forensics research, a significant effort has been dedicated to the design of effective synthetic speech detectors [1], [2], generating a wide range of processing schemes and model architectures, ranging from statistical and sound properties [3] to attention schemes [4]. Even though such solutions have shown increasing accuracy over the recent years, there are still many issues that remain unresolved [5].

One of the main issues is adaptability: the performance of existing methods can be very good on controlled and stable scenarios, but rapidly degrades as datasets and acquisition conditions change. To mitigate this, continual learning [6] and domain adaptation strategies [7] have been proposed, together with Mixture-of-Experts [4] and feature fusion [8]. Indeed, this problem is mainly due to the adoption of large classification networks which tend to adapt and overfit to specific dataset conditions. Moreover, model sizes make their adoption in embedded systems more difficult due to complexity constraints.

A second issue concerns the explainability of detector outcomes. Most of the proposed solutions resort to learned features that permit achieving top accuracy metrics but fail to provide adequate justifications. For such reasons, recent works have investigated eXplainable AI (XAI) solutions for audio deepfake detection [9], [10], which aim at highlighting those parts of the audio sequence that mostly contribute to the final decision. Despite this, the involved detectors remain very complex, thus making the justification and description of the whole decision chain very hard.

The approach presented in this paper targets both justifiability and complexity following an explainability-by-design architecture [11], where decisions can be directly related to hand-crafted and interpretable features by back-propagating outcomes through a low-complexity network.

Based on the hypothesis that real and synthetic speech may exhibit different predictability characteristics (due to windowing or inconsistencies in propagations), the input audio sequence is predicted using a Wiener-Hopf estimation strategy: then, filter taps are stacked into a 2D matrix processed by a small Convolutional Neural Network (CNN). Final decisions are supported by Grad-CAM [12] heatmaps revealing the most distinctive traces and evaluating the reliability of the outcome. Experimental results show that the proposed solution achieves competitive performances with respect to state-of-the-art works on heterogeneous datasets, while requiring a significantly lower computational effort.

The main contributions and novelties of the work can be summarized as follows.

• The paper presents a new lightweight explainable-by-design scheme for synthetic audio detection, where Wiener-Hopf filter coefficients are processed by means of a Convolutional Neural Network (CNN) and connected to a saliency map generated by Grad-CAM. Indeed, this is one of the first approaches that combines physically-relevant audio features with explainability algorithms at a limited computational cost while achieving the same performance as more complex schemes.

• The detector’s outcomes are justified by a Grad-CAM heatmap revealing hard-to-predict audio segments that are characteristic of real speech.

• The approach provides competitive performance across heterogeneous datasets (ASVspoof 2019, Fake-or-Real, DiffSSD), thus showing its ability to adapt to different settings.

• Experimental tests were re-run under different real-world adversarial conditions to test the robustness of the approach; results show that, under proper training, no significant performance degradation is observed.

• The required computational complexity is \(20\) times smaller than other state-of-the-art works with similar detection performance.

The paper is organized as follows. Section 2 reviews some of the recent literature works on synthetic speech detection, focusing on interpretability and explainability of features. Section 3 describes the proposed method, while Section 4 presents the adopted experimental setting. Section 5 reports the detection accuracy and highlights the revealing elements in the detection, including the interpretability study based on Grad-CAM and the robustness evaluation under post-processing operations. Finally, Section 6 summarizes the main findings and outlines future research directions.

2 Related works↩︎

Synthetic speech detection has been widely investigated in recent years, with significant progress in feature extraction strategies, classification architectures, and benchmark datasets [2], [13]. Many recent approaches rely on end-to-end deep learning pipelines that process raw waveforms or spectrograms. More recently, large Self-Supervised Learning (SSL) models, such as Wav2Vec2 [14], have been adopted as deep feature extractors, replacing traditional hand-crafted acoustic descriptors [13], [15], [16]. Multimodal approaches are also worth mentioning whenever audio traces are supported by visual information; such solutions rely on discrepancies between video and sound or unnatural behaviors [17], [18]. Although they often display a high accuracy, their performance seems to be related to an aggregation of revealing traces rather than to explanatory characteristics in real speech. Finally, recent research efforts have also focused on ensuring a high accuracy under different acquisition conditions [19] or datasets [6], [20]. For this reason, the latest papers are concerned with continual learning and domain adaptation techniques that improve generalization across different datasets and acquisition conditions.

These approaches achieve strong detection performance, but they often rely on black-box representations whose internal structure is difficult to interpret in forensic scenarios, and their large computational complexity may limit the deployment in resource-constrained settings.

To address these limitations, a parallel line of research has explored physically grounded acoustic representations, to improve generalization and interpretability. Several works employ formant-based features to reveal artifacts introduced by neural vocoders and generative models. Cuccovillo et al. [21] proposed a spectrogram transformer based on formant magnitude and phase information, showing that phonetic features enhance both detection performance and interpretability. The work in [22] extended phoneme-level analysis to person-of-interest scenarios, showing that articulatory cues provide discriminative information for speaker-specific deepfake detection. Salvi et al. [23] analyzed the spectral regions that most influence detector decisions by applying XAI techniques including LIME [24] and Grad-CAM [12], showing that specific frequency bands carry highly discriminative cues. Mari et al. [3] showed that first-digit statistical properties of silence regions provide effective low-complexity cues for synthetic audio detection, suggesting that non-speech passages carry forensically relevant information. Following this direction, this work adopts Wiener-Hopf filter coefficients as interpretable feature within an explainability-by-design architecture [11]. This enables decisions to be traced back to the physical properties of the signal, while maintaining a low computational complexity.

3 Method↩︎

An explainable approach to audio classification implies adopting a feature representation that is deeply connected to signal peculiarities for synthetic speech. Previous works have shown silence regions are very difficult to synthesize [3], while stationarity and reverberation are closely related to the natural propagation of sounds within recording environments [25]. Detecting physical discontinuities in such features (which can hardly be compensated by learned approaches processing single segments of signal) proves to be both effective and interpretable for human analysis. To this purpose, we resort to representing the analyzed audio tracks by means of Wiener-Hopf filter coefficients whose values exhibit oscillatory patterns when reverberant components cannot be properly estimated or when signal properties change abruptly due to windowing effects introduced by synthesis algorithms.

A Wiener-Hopf linear predictor [26], [27] of order M is applied to consecutive overlapping windows of 1024 samples, with a stride of 512 samples (50% overlap). The predictor coefficients \(\mathbf{h} \in \mathbb{R}^M\) are obtained by solving the Wiener-Hopf equation: \[\mathbf{R} \cdot \mathbf{h} = \mathbf{p},\] where \(\mathbf{R} \in \mathbb{R}^{M \times M}\) is the Toeplitz autocorrelation matrix of the input signal \(x\), and \(\mathbf{p} = \bigl[r(1),r(2),\dots,r(M)\bigl]^T\) is the vector of autocorrelation values, with \(r(k) = \mathbb{E}\bigl[x[n] \cdot x[n-k]\bigl]\) the autocorrelation of the signal \(x\) at lag \(k\). The coefficients are computed using the Cholesky decomposition, exploiting the positive-definite structure of \(\mathbf{R}\), and finding the least-squares solution in degenerate cases. The predictor estimates each sample \(\hat{x}[n]\) as a linear combination of the \(M\) preceding samples, as follows: \[\hat{x}[n] = \sum_{k=1}^M \mathbf{h}[k] \cdot x[n-k].\]

The coefficients \(\mathbf{h}[k]\) of the Wiener-Hopf predictor extracted from each window are stacked across frames to form a 2D matrix of shape \((F \times M)\), where \(F\) is the number of frames and \(M\) is the predictor order. As utterance length varies across audio samples, the matrices are padded to a fixed number of frames using symmetric padding, which avoids discontinuities at the boundaries. The resulting matrix, treated as a single-channel 32-bit floating-point (float32) image, is passed to a simple 2D CNN classifier. The network consists of four convolutional layers with batch normalization and ReLU activations, grouped in two blocks, with max pooling after the first block. A global adaptive max pooling layer collapses the spatial dimensions, and the representation is passed to a fully connected classifier with dropout, which outputs the prediction logits for the classes present in the dataset. The network architecture is shown in Figure 1 and has a total of 422K trainable parameters.

4 Experimental setup↩︎

This section describes the experimental protocol adopted to evaluate the proposed approach. We first introduce the considered datasets, followed by the training configuration and hyperparameter selection strategy.

4.1 Datasets↩︎

The experimental setup of our detection approach was defined considering several state-of-the-art audio deepfake datasets.

• ASVspoof 2019 (ASV19) [28]. Introduced to assess countermeasures against synthetic and voice-converted English speech, the dataset covers a wide range of generation systems, and is one of the most widely adopted benchmarks in the audio deepfake detection community. For our purposes, we focused on the Logical Access (LA) scenario.

• FakeOrReal (FoR) [29]. The dataset contains English bona fide speech, sourced from publicly available corpora, and synthetic speech generated by a variety of open-source and commercial systems.

• Diffusion-Based Synthetic Speech Dataset (DiffSSD) [30]. This corpus comprises approximately 200 hours of labeled English speech, with synthetic samples generated by 8 open-source and 2 commercial diffusion-based Text-To-Speech (TTS) systems. The real speech data was sourced from LJSpeech [31] and LibriSpeech [32].

4.2 Hyperparameter tuning↩︎

The Wiener-Hopf predictor order was set to \(M = 30\), offering a good trade-off between prediction accuracy and computational complexity. For the CNN classifier, the maximum number of frames was set to \(F = 300\) using symmetric padding. This value covers the majority of utterances in the datasets without excessive padding. To evaluate the impact of this design choice on detection performance, we also tested with standard zero-padding of the input. The model was trained using cross-entropy loss with the AdamW optimizer with a learning rate of \(3 \times 10^{-4}\) and weight decay of \(10^{-4}\). The learning rate was reduced by a factor of \(0.5\) when the validation loss did not improve for \(3\) consecutive epochs. The training was performed for a maximum of \(50\) epochs with early stopping using a patience of \(10\), a batch size of \(32\) and a dropout rate of \(0.5\). Hyperparameters were determined empirically by evaluating multiple configurations on the development set.

5 Results and discussion↩︎

This section presents the classification performance of the model and the analysis of the most discriminative elements in the detection process. Moreover, we evaluate the robustness of the model to data perturbations. The evaluation is performed on all considered datasets using both the Equal Error Rate (EER) and the Area Under the Curve (AUC) for the Receiver Operating Characteristic (ROC) curve.

5.1 Deepfake detection performance↩︎

We tested the proposed model on the three evaluation benchmarks, using the same training, validation, and test splits defined in the original protocols. The detection performance is presented in Table 1.

The 2D CNN employing symmetric padding achieves competitive results across all datasets, yielding an average EER of 3.16% and an AUC of 0.987. Generally speaking, the mentioned approach achieves the highest average accuracy with respect to other solutions presented during the last years [20], [33], [34].

On ASVspoof 2019, the proposed system obtains an EER of 5.97%, which outperforms the official challenge baselines [28] and remains competitive with state-of-the-art approaches, many of which rely on complex ensemble strategies. On DiffSSD, our model achieves 2.95% EER on the test set, performing better than all the baselines reported in [30], including large semi-supervised transformer-based models, such as Wav2Vec2 (3.00%) and MFCC-ResNet (11.00%). On the FoR dataset, the proposed model reaches an EER of 0.56%, matching the performance of other high-complexity systems reported in [38].

Across all three datasets, the symmetric padding variant consistently outperforms the zero-padding counterpart, confirming that preserving boundary continuity is critical for robust feature extraction. Overall, the proposed explainable-by-design architecture provides a physically grounded and interpretable alternative to black-box detectors, while maintaining competitive detection performance across diverse evaluation benchmarks.

5.2 Interpretability analysis↩︎

To better understand the discriminative behavior of the proposed approach, we apply Gradient-weighted Class Activation Mapping (Grad-CAM) [12] to the last convolutional layer of the classifier. Grad-CAM produces a saliency map over the input matrix, highlighting which Wiener-Hopf coefficients and temporal frames are the most informative for the classification decision. Some examples of the generated images are presented in Figure 2.

The analysis reveals that the activations concentrate on low-order predictor coefficients, which primarily encode the coarse spectral envelope of the signal, characterizing the primary formant resonances of the vocal tract and approximating its gross geometry under the acoustic tube interpretation of linear prediction [39], [40].

To characterize the temporal distribution of the activations, we classify each frame as voiced or unvoiced based on the short-time energy (STE) of the signal. In particular, a frame is labeled as voiced if its energy exceeds \(5\%\) of the maximum RMS energy of the utterance, and unvoiced otherwise. We observe that high activations of the network are frequently present in silence regions and on transitions between voiced and unvoiced passages. Our observations are consistent with previous studies [41], [42], which report that models analyzing synthetic speech rely predominantly on unvoiced regions, suggesting that the most salient artifacts are concentrated in these segments. In contrast, [10] reports that Grad-CAM applied to a Wav2Vec2-AASIST model on ASVspoof 2019 highlights speech regions as the most discriminative. This discrepancy may be related to the different feature space: while the referenced model operates on raw waveforms, our approach focuses on Wiener-Hopf prediction coefficients, which are sensitive to the statistical structure of silence passages. We hypothesize that this behavior reflects the ability of the Wiener-Hopf predictor to capture features related to the reverberation effects of the recording environment. In fact, while natural speech recordings typically exhibit reverberation tails in silent passages, deepfake generators tend to generate clean speech that lacks this acoustic pattern. The linear predictor may therefore expose the absence of natural reverberation as a discriminative cue, providing a physically-based interpretation of the detection performance of the model.

5.3 Ablation study↩︎

To further investigate the contribution of different temporal regions to the detection performance, we conduct an ablation study by selectively removing voiced and silence frames (thus, generating no-voice and no-silence configurations, respectively). For each configuration, the Wiener-Hopf method is applied to the entire original audio excerpt to avoid discontinuities, discarding the coefficient vectors corresponding to the removed frames; the resulting matrices are then used to retrain the CNN classifier from scratch.

The results of the ablation study are presented in Table 2. As expected, removing silence frames leads to a significant degradation in performance (increasing the EER by 14.42 % on average), confirming the importance of non-speech segments in the detection process. On the other hand, the exclusion of voiced frames increases the EER by 2.89 % on average, preserving good detection performance. These findings are consistent with the Grad-CAM analysis presented in Section 5.2, and further support the importance of voice-to-silence and silence-to-voice passages in the detection (where Wiener-Hopf coefficients highlight inconsistencies at transition boundaries).

4pt

5.4 Robustness to post-processing↩︎

To evaluate the robustness of the proposed method to real-world post-processing conditions, we apply a set of common signal degradations and measure the resulting detection performance. We considered additive noise, introduced using white, pink, and brown noise at four Signal-to-Noise Ratio (SNR) levels: 20, 15, 10, and 5 dB. Secondly, MP3 compression is applied at different bitrates (32, 64, 128 kbps), to simulate lossy compression artifacts commonly introduced during audio sharing and storage. Moreover, bandpass filtering is applied using two frequency-range configurations (300-3400 Hz, 500-3000 Hz) in order to simulate the bandwidth limitations of telephone and VoIP transmissions.

Table 3 reports the EER (%) of the proposed model under the considered perturbations, evaluated in two conditions: frozen model (F), where the classifier trained on clean data is tested on the perturbed test set, and full network fine-tuning (N), where the model is retrained on the perturbed training data. The fine-tuning of the model is performed considering the original learning rate reduced by a factor of \(0.1\), to limit the performance degradation on the original data. As expected, the frozen model suffers a significant performance degradation under most perturbations, in particular under white noise and telephone filtering, with an EER exceeding 50 % at low SNR levels. In contrast, the addition of pink and brown noise causes a more moderate degradation. Fine-tuning consistently recovers most of the lost performance across all perturbation types and datasets, with EER values close to the clean baseline in most conditions. Remarkably, on FakeOrReal the fine-tuned model achieves very low EER under MP3 compression and telephone filtering, suggesting that the discriminative cues captured by the Wiener-Hopf predictor are partially preserved even after a considerable bandwidth reduction.

4pt

6 Conclusions↩︎

The paper presents an explainable-by-design framework for audio deepfake detection based on Wiener-Hopf linear prediction. Unlike conventional black-box architectures, the proposed approach employs physically grounded and interpretable features processed by a lightweight CNN classifier, enabling a direct and transparent link between detection decisions and the acoustic properties of the signal. The proposed method achieves competitive detection performance across heterogeneous benchmarks while maintaining a significantly lower computational complexity compared to state-of-the-art methods.

Interpretability analysis using Grad-CAM reveals that the classifier focuses on low-order linear prediction coefficients, as well as on silence and transition regions of the audio signal. We hypothesize that this behavior reflects the ability of the Wiener-Hopf predictor to capture reverberation characteristics and subtle statistical structures that synthetic generation systems struggle to reproduce accurately.

Furthermore, the robustness analysis shows that fine-tuning effectively recovers detection performance under a wide range of post-processing conditions, including additive noise, MP3 compression, and telephone filtering. Future work will investigate domain generalization strategies and the development of other acoustic features within the explainability-by-design framework.

References↩︎