PC-Mix: Partial-Component Audio Spoofing Detection under Mixed Speech and Environmental Sound Conditions
July 11, 2026
Abstract
Recent studies on partial audio spoofing mainly focus on studio-recorded speech with temporal localization of spoofed segments. However, these studies often overlook realistic conditions where spoofed and bonafide segments simultaneously coexist across speech and environmental sound components. In this paper, we present PC-Mix, the first dataset for partial-component spoofing detection, where either or both audio components may be partially spoofed. In PC-Mix, bonafide and partially spoofed environmental-sound components are first constructed and mixed with speech signals from an existing partial-spoof dataset, producing audio in which either or both components may be locally manipulated. This design addresses two major gaps in existing partial spoofing benchmarks: the lack of realistic environmental sounds in speech partial spoofing scenarios and the absence of partial spoofing detection for environmental sound components. We further establish standardized evaluation protocols and design a joint learning framework to optimize spoofing detection across speech, environmental sound, and mixed audio. Experiments highlight the increased difficulty introduced by mixed conditions. The results demonstrate that training under matched target conditions is more effective than directly transferring models trained on speech or environmental sound components.
Audio Anti-Spoofing, Partial Spoofing Detection, Component-Level Audio Anti-Spoofing, Joint Learning
1 Introduction↩︎
Audio anti-spoofing has become an important task in trustworthy speech and audio processing, as increasingly realistic synthetic and manipulated audio continues to pose serious security risks to real-world applications [1]–[3].
Spoofed audio can be generated or manipulated using increasingly powerful techniques, including voice conversion methods [4], [5], zero-shot speech synthesis methods [6], [7], and audio editing methods [8]–[10], posing significant threats to authentication systems and downstream applications.
Most research [11]–[14] only focuses on utterance-level spoof, ignoring localized manipulation within an utterance and treating the entire signal as either fully real or fully spoofed. More recently, attention has shifted toward partial spoofing, where only localized temporal regions of an utterance are manipulated while the rest remains bonafide [15], [16]. This scenario is more realistic, as modern audio editing tools enable fine-grained manipulation of selected segments without altering the entire recording. Existing benchmarks [15], [17]–[21] for partial-spoofing and temporal deepfake localization have shown that localized attacks are particularly challenging to detect.
However, these studies predominantly focus on the speech component alone and typically assume clean or studio-recorded conditions, overlooking the role of environmental background sounds [11], [12], [22], [23]. In real-world scenarios, audio recordings are inherently composed of multiple acoustic components, including foreground speech and environmental sounds. Recent work has begun to explore component-level spoofing, where spoofing may target either the speech component, the environmental component, or both [24], [25]. This formulation reflects practical scenarios where adversaries may manipulate only selected audio components such as speech or environmental sound or possibly both. Such mixed conditions introduce additional complexity, as detection systems must reason about multiple interacting sources rather than a single homogeneous signal.
Nevertheless, existing component spoofing studies mainly consider component-level manipulation at the utterance level and do not explicitly address temporally localized spoofing within each component. To address this gap, we establish a partial component spoof evaluation protocol. We propose PC-Mix, a new dataset for partial-component spoofing under mixed speech and environmental sound conditions. In PC-Mix, we construct bonafide and partially spoofed environmental-sound components and mix them with speech signals from an existing partial-spoof dataset. Each resulting sample may contain localized spoofing in the environmental sound component, the speech component, or both. Original recordings are also included to distinguish naturally recorded audio from constructed mixtures. A joint learning framework is further proposed for frame-level partial-component spoofing detection. The framework adopts a two-stage training strategy, first learning component-specific cues for speech spoofing, environmental-sound spoofing, and original-mix discrimination, and then jointly optimizing them under a unified partial-component spoofing objective. This design combines component-level localization with mix-level scene discrimination.
Our main contributions are summarized as follows:
We introduce partial-component spoofing detection as a new evaluation task for partially edited audio, where localized spoofing may occur in speech, environmental sound, or both within a mixed acoustic scene.
We propose PC-Mix, the first dataset for partial-component spoofing detection and the first benchmark dataset that includes partially spoofed environmental sounds. PC-Mix addresses two major gaps in existing partial spoofing benchmarks: the lack of realistic environmental sounds in speech partial spoofing scenarios and the absence of partial spoofing detection for environmental sound components.
We propose a jointly trained baseline framework for partial-component spoofing detection. The framework performs speech-component spoofing detection, environmental-sound spoofing detection, and original-mix discrimination within a unified model.
The source code and dataset are publicly released1 to facilitate reproducible research and future studies.
2 PC-Mix Dataset↩︎
PC-Mix is designed to study partial-component spoofing under mixed speech and environmental sound conditions. It contains two types of data: original samples and constructed mixed samples. The original samples are real-world recordings that naturally contain speech and environmental sounds. The constructed mixed samples are generated by mixing partially spoofed speech utterances with partially spoofed environmental sound components.
2.1 Audio Source↩︎
2.1.1 Original recorded audio↩︎
Original recordings are included as a reference domain for naturally recorded speech–environmental sound scenes. These samples are not created through artificial mixing, representing real acoustic conditions where speech and environmental sounds coexist. In PC-Mix, original recordings are collected from VGGSound [26] and SBCASE [27]. They are used to help distinguish naturally recorded audio from constructed speech–environmental sound mixtures.
2.1.2 Partially spoofed speech↩︎
For the speech component, audio is from PartialSpoof [15].
2.1.3 Partially spoofed environmental sound↩︎
| Split | #All | #Bona | #Spoof | BG | Event Source | Fusion | Shift |
|---|---|---|---|---|---|---|---|
| Train | 25,380 | 12,715 | 12,665 | SONYC | ALDM2/US8K | Duck. | Train |
| E0 | 17,812 | 8,931 | 8,881 | SONYC | ALDM2/FSD50K | Duck. | ID |
| E1 | 14,248 | 7,181 | 7,067 | SONYC | AudioGen | Duck. | Gen. OOD |
| E2 | 14,248 | 7,045 | 7,203 | SONYC | ALDM2 | Cross. | Fus. OOD |
| E3 | 17,809 | 8,757 | 9,052 | DEMAND | ALDM2 | Duck. | BG OOD |
| E4 | 7,125 | 3,562 | 3,563 | WHAM! | ALDM2 | Duck. | Noise OOD |
1.6pt
This work introduces the first environmental-sound partial spoofing dataset designed for component-level spoofing analysis. For the environmental sound component, partially spoofed samples are constructed by inserting real or generated event sounds into real environmental background recordings.
In the training set, SONYC [28] is used as the background source. Spoofed event sounds are generated by AudioLDM2 [29] and real events are curated from UrbanSound8K [30]. The event sounds are inserted into background recordings using a controlled ducking-overlay strategy [31], where the background is locally attenuated during the inserted event. This preserves acoustic overlap while keeping the event perceptually salient.
| Level | ID | Mixed | Speech | Env. | Description |
|---|---|---|---|---|---|
| Utt. | 0 | No | – | – | Original outdoor recording |
| 1 | Yes | bonafide | bonafide | bonafide speech with bonafide env. | |
| 2 | Yes | Spoofed | bonafide | spoofed speech with bonafide env. | |
| 3 | Yes | bonafide | Spoofed | bonafide speech with spoofed env. | |
| 4 | Yes | Spoofed | Spoofed | spoofed speech with spoofed env. | |
| Frame | 0 | Yes | bonafide | bonafide | bonafide speech and env. frame |
| 1 | Yes | Spoofed | bonafide | spoofed speech frame | |
| 2 | Yes | bonafide | Spoofed | spoofed env. frame | |
| 3 | Yes | Spoofed | Spoofed | spoofed speech and env. frame |
2.2pt
In the evaluation set, five evaluation subsets are constructed, denoted as E0–E4. E0 serves as the in-domain evaluation set. It uses SONYC backgrounds, together with a new event pool composed of non-overlapping events generated by AudioLDM2 and real events from FSD50K [32]. E1 evaluates generator shift by replacing AudioLDM 2 with AudioGen [33]. E2 evaluates fusion shift by replacing the default ducking-overlay strategy with hard replacement followed by crossfade [34], while keeping the source domains unchanged. E3 evaluates background-domain shift by replacing SONYC with DEMAND [35]. E4 evaluates noise-domain shift by using WHAM! noise as the background source [36], resulting in more noise-dominated acoustic conditions. The statistics of the training and evaluation splits are summarized in Table 1.
| Split | Orig | BB | BS | SB | SS | #Total | Duration. |
|---|---|---|---|---|---|---|---|
| Train | 12,155 | 1,287 | 1,293 | 11,427 | 11,373 | 37,535 | 98.95h |
| Eval | 17,809 | 3,703 | 3,666 | 31,773 | 32,100 | 89,051 | 41.71h |
| Total | 29,964 | 4,990 | 4,959 | 43,200 | 43,473 | 126,586 | 140.65 h |
3.2pt
2.2 Mix Process↩︎
Each partially spoofed speech utterance is paired with a partially spoofed environmental-sound recording and inserted at a random position on its timeline, with the output duration kept equal to the environmental-sound component by zero-padding shorter speech or truncating longer speech. All audio is sampled at 16 kHz. Training and evaluation mixtures are generated with nominal speech-to-environmental-sound SNRs of 8 dB and 10 dB, respectively, using fixed speech and environmental-sound gains of 1.0 and 1.4 with clipping protection; the effective SNR may vary slightly because environmental-sound scaling is computed for each pair.
PC-Mix provides utterance-level and frame-level labels for both speech and environmental-sound components. Speech labels are inherited from PartialSpoof, while environmental-sound labels are derived from the constructed environmental-sound component. Segment labels are provided at 40, 80, 160, 320, and 640 ms resolutions. In addition, original-mix labels are also provided to distinguish naturally recorded audio from constructed mixtures. Based on these annotations, PC-Mix defines five utterance-level classes and four frame-level classes, as summarized in Table 2; frame-level classes are defined only for constructed mixtures.
Overall, PC-Mix contains 126,586 clips and 140.65 hours of audio. The utterance-level class distribution of the training and evaluation splits is reported in Table 3.
3 Methodology↩︎
3.1 Baseline System↩︎
We conduct two types of PC-Mix baseline evaluations, as illustrated in Fig. 1. Original real-world recordings in PC-Mix are excluded from these component-level baseline experiments because they do not provide separate speech- or environmental-sound spoofing annotations; they are later reintroduced in the joint learning experiments.
3.1.1 Cross-Condition Transfer to PC-Mix↩︎
We trained single-component speech and environmental-sound detectors on PartialSpoof and the constructed environmental-sound data, respectively. Then we transferred the single-component detectors to PC-Mix evaluation. During evaluation, two input settings are considered. In the mixed-audio input setting, both detectors are directly evaluated on the PC-Mix mixtures using their corresponding component labels. In the separated-stream input setting, each PC-Mix waveform is first separated into speech and environmental-sound streams using the MVSEP separator [37], after which each detector is evaluated on its corresponding separated stream.
3.1.2 Matched PC-Mix Baselines↩︎
We train matched PC-Mix detectors under two input settings. In the mixed-audio input setting, speech-focused and environmental-sound-focused detectors are trained on PC-Mix training mixtures and evaluated on PC-Mix evaluation mixtures using the corresponding component labels. In the separated-stream input setting, both the training and evaluation mixtures are first separated into speech and environmental-sound streams using MVSEP [37]; each detector is then trained and evaluated on its corresponding separated stream. These matched baselines provide references for assessing the effect of matched mixed-condition training and for comparing with the proposed joint learning framework.
3.2 Three-head Joint Learning Framework↩︎
3.2.1 Framework Architecture↩︎
A three-head joint learning framework is established for partial-component spoofing detection, as illustrated in Fig. 2. The framework consists of three MultiResoModel [18] branches: a speech branch, an environmental sound branch, and a mix branch. The speech and environmental sound branches are tailored to detect partial spoofing in the corresponding audio components, while the mix branch is designed to distinguish constructed speech–environmental sound mixtures from original real-world audio recordings. Each branch follows the same multi-resolution architecture, where SSL-based acoustic representations are processed by temporal modeling layers and multiple detection heads. During joint training, the three branches are optimized in a unified loss, allowing component-level spoofing cues and mix-level detection cues to be learned simultaneously.
Given an input waveform \(\mathbf{x}\), the three branches produce: \[\mathbf{z}^{s} = f_s(\mathbf{x}), \quad \mathbf{z}^{e} = f_e(\mathbf{x}), \quad \mathbf{z}^{m} = f_m(\mathbf{x}),\] where \(f_s(\cdot)\), \(f_e(\cdot)\), and \(f_m(\cdot)\) denote the speech, environmental sound, and mix branches, respectively. Their outputs \(\mathbf{z}^{s}\), \(\mathbf{z}^{e}\), and \(\mathbf{z}^{m}\) contain multi-resolution frame-level prediction scores and an utterance-level prediction score. The speech and environmental sound branch predict partial spoofing in the corresponding components, while the mix branch predicts whether the input audio is an original recording or a constructed mix.
3.2.2 Two-Stage Training Strategy↩︎
The framework is trained in two stages. In the first stage, the three branches are trained independently using their corresponding labels. The speech branch is trained with speech-component partial spoofing labels, the environmental sound branch is trained with environmental-sound partial spoofing labels, and the mix branch is trained with original-mix labels. In the second stage, the three pretrained branches are combined into a single joint framework and optimized together. For the mix branch, only the utterance-level original-mix loss is used during second-stage joint training, while its frame-level outputs are excluded from loss supervision and not used for reported metrics. This design allows each branch to first learn component-specific spoofing cues and then adapt jointly under the partial-component spoofing setting.
3.2.3 Joint Training↩︎
Let \(\mathcal{K}\) denote the set of frame-level temporal resolutions. For \(i\)-th sample, the speech branch output consists of frame-level scores \(\{\mathbf{z}^{s}_{i,k}\}_{k\in\mathcal{K}}\) and an utterance-level score \(z^{s}_{i,u}\). Similarly, the environmental-sound branch produces \(\{\mathbf{z}^{e}_{i,k}\}_{k\in\mathcal{K}}\) and \(z^{e}_{i,u}\), and the mix branch produces \(\{\mathbf{z}^{m}_{i,k}\}_{k\in\mathcal{K}}\) and \(z^{m}_{i,u}\).
The corresponding speech and environmental-sound labels are denoted as \(\{\mathbf{y}^{s}_{i,k}\}_{k\in\mathcal{K}}\), \(y^{s}_{i,u}\), \(\{\mathbf{y}^{e}_{i,k}\}_{k\in\mathcal{K}}\), and \(y^{e}_{i,u}\). For the mix branch, \(y^{m}_{i,u}\) denotes the utterance-level original-mix label, where \(y^{m}_{i,u}=1\) indicates a constructed mix and \(y^{m}_{i,u}=0\) indicates an original sample. During first-stage independent training of the mix branch, \(\mathbf{y}^{m}_{i,k}\) denotes the corresponding frame-level original-mix label at resolution \(k\). The P2SGrad loss[38] is used for all binary detection branches.
For the speech branch, the multi-resolution loss is defined as \[\begin{align} \mathcal{L}^{s}_{i} = \sum_{k \in \mathcal{K}} \ell(\mathbf{z}^{s}_{i,k}, \mathbf{y}^{s}_{i,k}) + \ell(z^{s}_{i,u}, y^{s}_{i,u}). \end{align}\] The environmental sound branch follows the same form: \[\begin{align} \mathcal{L}^{e}_{i} = \sum_{k \in \mathcal{K}} \ell(\mathbf{z}^{e}_{i,k}, \mathbf{y}^{e}_{i,k}) + \ell(z^{e}_{i,u}, y^{e}_{i,u}). \end{align}\]
For the mix branch, first-stage independent training uses both frame-level and utterance-level supervision: \[\begin{align} \mathcal{L}^{m,\mathrm{P1}}_{i} = \sum_{k \in \mathcal{K}} \ell(\mathbf{z}^{m}_{i,k}, \mathbf{y}^{m}_{i,k}) + \ell(z^{m}_{i,u}, y^{m}_{i,u}). \end{align}\] During second-stage joint training, only the utterance-level original-mix loss is used: \[\mathcal{L}^{m,\mathrm{P2}}_{i} = \ell(z^{m}_{i,u}, y^{m}_{i,u}).\]
Since original samples do not contain separated speech and environmental-sound spoofing annotations, component-level losses are applied only to constructed mixtures. This is implemented as \[v_i = \begin{cases} 1, & y^{m}_{i,u}=1, \\ 0, & y^{m}_{i,u}=0. \end{cases}\]
The final joint objective over a mini-batch \(\mathcal{B}\) is \[\mathcal{L}_{\mathrm{joint}} = \frac{1}{|\mathcal{B}|} \sum_{i \in \mathcal{B}} \left( \lambda_s v_i \mathcal{L}^{s}_{i} + \lambda_e v_i \mathcal{L}^{e}_{i} + \lambda_m \mathcal{L}^{m,\mathrm{P2}}_{i} \right).\] where \(\lambda_s\), \(\lambda_e\), and \(\lambda_m\) are the loss weights for the speech, environmental sound, and mix branches. Thus, mix-level supervision is applied to all samples, while speech and environmental sound losses are masked out for original recordings without component-level annotations.
4 Experiments↩︎
4.1 Experimental Settings↩︎
4.1.1 Data Processing↩︎
Audio samples are resampled to 16 kHz and processed as 9.6-second segments with a trailing padding duration of 0.005 s. All models use 160 ms frame-level supervision and produce frame-level outputs at the same resolution, together with an utterance-level output. Random seeking and mask-based training are enabled.
4.1.2 Training Configuration↩︎
All detectors are implemented with MultiResoModel [18] as the backbone. All models are optimized using Adam with a learning rate of \(1\times10^{-5}\). A StepLR scheduler is applied with a step size of 10 epochs and a decay factor of 0.5.
Equal branch weights are used in all joint-training experiments, i.e., \(\lambda_s=\lambda_e=\lambda_m=1\). During joint training, the mix branch is trained on all samples, while the speech and environmental-sound branches are optimized only when valid component-level annotations are available.
4.1.3 Evaluation Metrics↩︎
Performance is evaluated using both binary and multi-class metrics. For the baseline evaluations, only equal error rate (EER) is reported at both the utterance and frame levels. For each binary branch in the joint framework, utterance-level metrics include EER, F1 score, and accuracy. For the speech and environmental sound branches, the same binary metrics are also computed at the frame level. Branch predictions are further mapped to the class definitions in Table 2. At the utterance level, five-class classification is evaluated with a mix-gated decision rule. Accuracy, macro F1, and weighted F1 are reported for this evaluation. At the frame level, four-class classification is evaluated on mixed utterances based on the authenticity states of speech and environmental sounds, using the same multi-class metrics. Since original-mix supervision is defined at the recording level, the mix-branch frame-level outputs are not used in joint evaluation; therefore, frame-level five-class classification is not reported.
4.2 Experimental results↩︎
4.2.1 Single-Component Partial Spoofing Detection Results↩︎
| Train data | Eval data | Epoch | Utt. EER (%) | Frame EER (%) |
|---|---|---|---|---|
| Speech train | Speech eval | 25 | 2.16 | 11.36 |
| Env. sound train | Env. sound eval | 25 | 3.55 | 5.89 |
3.5pt
| Train data | Eval data | Epoch | Utt. EER (%) | Frame EER (%) |
|---|---|---|---|---|
| Env. sound train | Env. sound eval E0 | 25 | 0.67 | 2.47 |
| Env. sound train | Env. sound eval E1 | 25 | 1.88 | 7.98 |
| Env. sound train | Env. sound eval E2 | 25 | 0.53 | 2.57 |
| Env. sound train | Env. sound eval E3 | 25 | 9.15 | 13.04 |
| Env. sound train | Env. sound eval E4 | 25 | 3.69 | 3.93 |
3.5pt
We train MultiResoModel-based speech and environmental-sound spoofing detectors on PartialSpoof and the constructed partial-spoof environmental-sound training data, respectively, and evaluate them on the corresponding single-component test sets. Table 4 shows that both speech and environmental-sound spoofing detectors perform well under matched conditions, indicating that each detection head is effective on its corresponding component.
Table 5 presents the evaluation results of the environmental-sound detector on the proposed E0–E4 subsets. The results show that these subsets expose different challenges for partial environmental-sound spoofing detection. The detector performs well on the in-domain E0 set and remains stable under fusion shift in E2, suggesting that inserted spoofed events can be localized when the background domain is consistent. However, performance degrades most under background-domain shift in E3, indicating that environmental context mismatch is a key challenge. These results demonstrate that the proposed subsets provide a useful benchmark for evaluating localized spoofing detection under different domain shifts.
4.2.2 Baseline result↩︎
| Setting | Target | Input | Epoch | Utt. EER (%) | Frame EER (%) |
|---|---|---|---|---|---|
| Mixed | Speech | Mixed audio | 25 | 29.35 | 32.88 |
| Mixed | Env. sound | Mixed audio | 25 | 44.80 | 33.44 |
| Separated | Speech | Sep. speech | 25 | 51.38 | 37.51 |
| Separated | Env. sound | Sep. env. | 25 | 22.89 | 17.65 |
2.2pt
Note: Env. sound denotes environmental sound, and Utt. denotes utterance-level.
This experiment corresponds to the cross-condition transfer setting, where single-component detectors are evaluated directly under PC-Mix conditions. As shown in Table 6, both input settings suffer substantial degradation compared to the single-component results in Table 4. This indicates that detectors trained only on speech or environmental sounds cannot be directly transferred to PC-Mix conditions. Although separated-stream evaluation improves environmental-sound detection over mixed-audio evaluation, it remains far from the single-component setting and further degrades speech detection.
| Train & eval setting | Target | Epoch | Utt. EER (%) | Frame EER (%) |
|---|---|---|---|---|
| Mixed-audio input | Speech | 30 | 8.53 | 22.52 |
| Mixed-audio input | Env. sound | 30 | 3.57 | 5.68 |
| Separated-stream input | Speech | 30 | 7.83 | 22.61 |
| Separated-stream input | Env. sound | 30 | 7.43 | 8.21 |
3.5pt
Note: Env. sound denotes environmental sound, and Utt. denotes utterance-level.
Table 7 reports the matched PC-Mix baseline results, where component-specific detectors are trained and evaluated under the same PC-Mix input setting. Compared with cross-condition transfer in Table 6, matched PC-Mix training greatly improves performance. In the mixed-audio setting, speech utterance-level EER decreases from 29.35% to 8.53%, and environmental-sound utterance-level EER decreases from 44.80% to 3.57%. This confirms that the cross-condition degradation mainly comes from the mismatch between single-component training and mixed acoustic conditions.
| (a) Binary head performance | ||||
|---|---|---|---|---|
| Level | Head | EER \(\downarrow\) | F1 \(\uparrow\) | ACC \(\uparrow\) |
| Utterance | Speech | 8.72 \(\rightarrow\) 7.86 (-0.86) | 94.56 \(\rightarrow\) 94.58 (+0.02) | 89.69 \(\rightarrow\) 89.72 (+0.03) |
| Env. | 3.59 \(\rightarrow\) 3.12 (-0.47) | 76.41 \(\rightarrow\) 92.67 (+16.26) | 69.36 \(\rightarrow\) 92.17 (+22.81) | |
| Mix | 0.96 \(\rightarrow\) 0.42 (-0.54) | 99.43 \(\rightarrow\) 99.22 (-0.21) | 99.10 \(\rightarrow\) 98.77 (-0.33) | |
| Frame | Speech | 22.54 \(\rightarrow\) 21.84 (-0.70) | 71.70 \(\rightarrow\) 72.55 (+0.85) | 77.27 \(\rightarrow\) 78.07 (+0.80) |
| Env. | 5.18 \(\rightarrow\) 3.87 (-1.31) | 89.93 \(\rightarrow\) 90.80 (+0.87) | 96.94 \(\rightarrow\) 96.96 (+0.02) | |
| (b) Multi-class decision performance | ||||
| Task | ACC \(\uparrow\) | Macro F1 \(\uparrow\) | Weighted F1 \(\uparrow\) | |
| Utterance 5-class | 69.40 \(\rightarrow\) 85.12 (+15.72) | 44.80 \(\rightarrow\) 54.31 (+9.51) | 64.78 \(\rightarrow\) 81.52 (+16.74) | |
| Frame 4-class | 74.92 \(\rightarrow\) 75.72 (+0.80) | 71.58 \(\rightarrow\) 72.58 (+1.00) | 75.16 \(\rightarrow\) 76.04 (+0.88) | |
5pt
The separated-stream setting also improves over cross-condition transfer, especially for environmental-sound detection, where utterance-level EER decreases from 22.89% to 7.43%. However, it does not consistently outperform mixed-audio training: speech results are close across the two settings, while environmental-sound detection is clearly worse after separation. This suggests that source separation may introduce artifacts or remove environmental-sound spoofing cues. Therefore, the proposed joint learning framework adopts the mixed-audio input setting rather than relying on explicit source separation.
4.2.3 Ablation Study↩︎
Table 8 presents an ablation study of the second-stage joint optimization by comparing the jointly trained model with an assembled model composed of the three independently trained branches from stage 1. In this baseline, the independently trained speech and environmental-sound branches from the previous mixed-audio input experiment, together with the independently trained mix branch, are directly assembled and evaluated without second-stage joint optimization. Each entry is reported as “Pre-joint \(\rightarrow\) Joint-E8”, where Pre-joint denotes the assembled stage-one baseline and Joint-E8 denotes the model after eight epochs of joint training.
The results show that joint training is more effective than directly assembling independently trained heads. The improvement is most pronounced for the environmental-sound branch, whose utterance-level F1 and accuracy increase from 76.41% and 69.36% to 92.67% and 92.17%, respectively. This suggests that environmental-sound spoofing detection is particularly sensitive to the final mixed-audio training condition, where spoofing cues are entangled with speech and other acoustic components. In contrast, the speech branch shows smaller but consistent gains, indicating that it is already relatively stable after stage-one training and mainly benefits from further refinement during joint optimization. The mix branch also maintains strong original-mix discrimination, with a slight decrease in EER, although its F1 and accuracy slightly decrease, suggesting that its operating point may require further calibration.
The multi-class results further confirm the benefit of joint training. At the utterance level, 5-class accuracy increases from 69.40% to 85.12%, and weighted F1 also improves substantially, indicating better consistency among the mix, speech, and environmental-sound branches. Macro F1 also increases, although it remains lower than weighted F1 due to the imbalanced class distribution, especially for the smaller mixed classes. This suggests that joint training improves overall decision consistency while partially mitigating the effect of class imbalance. At the frame level, the gains are smaller but still positive, with accuracy, macro F1, and weighted F1 all increasing after joint training. This indicates that frame-level localization remains more difficult than utterance-level classification, since the model must identify short spoofed regions under overlapping speech and environmental-sound components. Overall, joint training mainly strengthens utterance-level component reasoning, while fine-grained temporal localization remains challenging under mixed-audio conditions.
5 Conclusion↩︎
In this paper, we introduce partial-component spoofing detection as a new evaluation task for partially edited audio. In this setting, spoofing may occur in speech, environmental sounds, or both within a mixed acoustic scene. We propose PC-Mix, a benchmark dataset for this task. It is, to the best of our knowledge, the first dataset that includes partially spoofed environmental sounds. It also addresses two key gaps in prior benchmarks: the lack of realistic environmental sounds in speech-based partial spoofing and the absence of evaluation for environmental sound spoofing. We further propose a jointly trained baseline framework for this task. It performs speech-component spoofing detection, environmental-sound spoofing detection, and original-mix discrimination within a unified model.
Acknowledgment↩︎
The authors used OpenAI ChatGPT only for language editing, grammar checking, and clarity improvement across the manuscript. The tool was not used to generate research ideas, experimental designs, datasets, experimental results, figures, tables, or conclusions. The authors reviewed and verified all content, claims, and results, and remain fully responsible for the manuscript.