1 Introduction↩︎

Anomalous Sound Detection (ASD) involves analyzing the operating sounds of machinery to identify potential malfunctions, providing non-contact, cost-effective, and efficient real-time monitoring [1]. The fundamental principle of ASD is to model the distribution of normal audio, such that any deviation from this learned distribution is flagged as an anomaly [2], [3].

Existing ASD methods can be broadly categorized into three main types. First, traditional methods rely on hand-crafted features extracted from time-domain [4] (e.g., amplitude, energy) or frequency-domain [5] (e.g., Fourier transform) representations of the audio signal [6]. While some approaches incorporate neural network-based audio features [7], these methods generally depend heavily on domain-specific feature engineering, which increases annotation costs and limits their generalization capabilities. Second, self-supervised learning methods [8]–[10] leverage auxiliary tasks, often guided by meta-labels (e.g., machine ID, operational conditions), to learn a discriminative representation of normal audio. The goal is to create a feature space where anomalous sounds are clear outliers [2], [3], [11], [12]. However, their performance can be sensitive to ambient noise, and the reliance on auxiliary labels restricts their applicability in scenarios where such metadata is unavailable.

The third category, generative methods, learns to reconstruct normal audio with high fidelity. When presented with an anomalous audio, these models produce a high reconstruction error, which serves as the anomaly score. Common approaches include Autoencoders (AEs) [13], [14] and, more recently, denoising diffusion models [15]–[17]. While these methods reduce the dependency on annotations, they have significant drawbacks. AE-based models often suffer from poor generalization and may degenerate into learning identity functions [18]. More critically, existing generative approaches, both AE- and diffusion-based, typically require training a separate model for each machine type. This one-model-per-machine paradigm leads to prohibitive storage and training costs in real-world, multi-machine environments. When multiple machines speak together, these methods require training multiple models to monitor for anomalies.

To address these limitations, we propose UD-ASD, a Unified Diffusion model for Anomalous Sound Detection, which employs a unified conditional diffusion model to reconstruct spectrograms and detect anomalies with Gaussian Mixture Model (GMM) for all machines. Our framework consists of three main components: a Condition Projector (CP), a conditional diffusion model, and a Gaussian Mixture Model (GMM) for anomaly scoring. The CP first encodes machine ID information into a condition embedding. This embedding is concatenated with the input log-Mel spectrogram, providing the necessary context to the diffusion model. The model is trained exclusively on normal data to perform a denoising-based reconstruction task. During inference, the model attempts to reconstruct any given input as a normal sound. Consequently, anomalous inputs result in a high reconstruction error. To account for temporal context, we apply temporal pooling to the frame-wise reconstruction errors. Finally, a GMM, also trained only on the reconstruction errors of normal data, models this error distribution. The final anomaly score is derived from the negative log-likelihood of a given sample’s error, effectively measuring its deviation from the norm. Our contributions are: 1) We propose a unified diffusion framework that reduces training overhead by eliminating the one-model-per-machine constraint. 2) We introduce Condition Projector, a lightweight module provides condition information guidance for the framework. 3) We achieve sota performance on the DCASE2022 Task 2 dataset [19], demonstrating both superior accuracy and efficiency. The code will be open-source after the review process.

2 Proposed method↩︎

The architecture of our proposed UD-ASD model is illustrated in Figure 1. The framework operates in three main stages. First, input audio is transformed into a log-Mel spectrogram via the Short-Time Fourier Transform (STFT). Concurrently, a CP module encodes the machine’s ID into a condition embedding, which is concatenated with the corresponding spectrogram. Second, a conditional Diffusion Model, trained exclusively on normal data, takes the conditional data as input to perform a reconstruction task. During inference, this process yields a high reconstruction error for anomalous inputs. Finally, an Anomaly Scoring system, based on a GMM, models the distribution of reconstruction errors from normal data and calculates the final anomaly score for a given test sample.

2.1 Condition projector↩︎

Our primary innovation directly addresses that one-model-per-machine limitation inherent in most existing generative ASD methods, simple yet significantly reducing deployment overhead. We introduce the Condition Projector, a lightweight module shown in Figure 1, to map discrete machine identity labels \(c\) (e.g., machine type or ID) into dense condition embeddings.

In UD-ASD, different machine IDs \(c\) are encoded as condition embeddings with CP. The embeddings are combined with the spectrograms and passed through each layer of UD-ASD to ensure full integration of condition information. Diffusion models adds noise to the original clear spectrograms \(\mathbf{x}^{org}_0\) with \(t\)-th step, resulting in the corrupted data \(\mathbf{x}^{org}_t\). \[\begin{align} \label{eq:32conditional95diffusion} \mathbf{x}_t = {\rm concat} (\mathbf{x}^{org}_t, {\rm CP}(c)) \;, \end{align}\tag{1}\] where the \({\mathbf{x}}_t\) is the \(t\)-th input with condition and \(\mathbf{x}^{org}_t\) is the \(t\)-th original data without condition (\(t \in [0, 1, ..., T]\)). For convenience, subsequent instances of \({\mathbf{x}}_t\) denote conditional data. The \({\rm concat} (\cdot)\) means channel concatenation. \({\rm CP}(\cdot)\) denotes CP module, whose structure is shown in the lower left of Figure 1. CP maps discrete labels to embeddings, allowing UD-ASD to generate machine-specific data with additional information.

2.2 Diffusion model↩︎

Our reconstruction backbone is a conditional diffusion model, trained using the Denoising Diffusion Probabilistic Model (DDPM) framework [20] (the forward diffusion process \(q(\mathbf{x}_t \, | \, \mathbf{x}_{t-1})\) and the reverse denoising process \(p_{\theta}(\mathbf{x}_{t-1} \, | \, \mathbf{x}_t)\)) and accelerated during inference using the Denoising Diffusion Implicit Model (DDIM) sampler [21].

In the forward diffusion process, the corrupted data \(\mathbf{x}_t\) in the \(t\)-th step is generated with a Markov chain by adding Gaussian noise to the original input data \(\mathbf{x}_0\) step by step: \[\begin{align} \label{eq:32forward95process95neibor} q(\mathbf{x}_t \, | \, \mathbf{x}_{t-1}) = \mathcal{N}(\mathbf{x}_t \, | \, \sqrt{{\alpha}_t}\mathbf{x}_{t-1}, \;(1 - {\alpha}_t)\mathbf{I}) \;, \end{align}\tag{2}\] and \(\alpha_t\) represents the noise variance schedule used to regulate the noise added at the \(t\)-th step. As the noise schedule we employ in the forward diffusion is a linear Gaussian process, the closed-form solution can be directly computed from \(\mathbf{x}_0\): \[\begin{align} \label{eq:32forward95process} q(\mathbf{x}_t \, | \, \mathbf{x}_{0}) = \mathcal{N}(\mathbf{x}_t \, | \, \sqrt{\bar{\alpha}_t}\mathbf{x}_{0}, \;(1 - \bar{\alpha}_t)\mathbf{I}) \;, \end{align}\tag{3}\] where \(\bar{\alpha}_t = \displaystyle \prod_{i=1}^{t} \alpha_i\). So the corrupted data \(\mathbf{x}_t\) could be computed based on the original input data \(\mathbf{x}_0\) and the \(t\)-th step: \[\begin{align} \label{eq:32forward95proces95x} \begin{aligned} \mathbf{x}_t = \sqrt{\bar{\alpha}_t} \;\mathbf{x}_0 + \sqrt{1 - \bar{\alpha}_t} \;\epsilon_t \;, \end{aligned} \end{align}\tag{4}\]

In the actual model, Eq. (4 ) is used to generate the pure Gaussian noise from the original conditional log-Mel spectrograms \(\mathbf{x}_0\) with the noise schedule and the random noise \(\epsilon_t\) generated at step \(t\).

Reverse process and training. In the reverse denoising process, the corrupted data \(\mathbf{x}_t\) are progressively denoised to noise-free data \(\mathbf{\tilde{x}}_0\) by predicting the real Gaussian noise \(\epsilon_t\). We employ a UNet-like attention-based network [22] \(\mu_{\theta}(\cdot, \cdot)\) with parameters \(\theta\) to predict \(\epsilon_t\). \(\mu_{\theta}(\cdot, \cdot)\) takes diffused spectrograms \(\mathbf{x}_t\) and time step \(t\) as inputs, and outputs the predicted noise. By minimizing Eq. (5 ), the denoising UNet can predict the corresponding noise based on the specific \(t\) and \(c\). \[\begin{align} \label{eq:32loss} \mathcal{L} = \mathbb{E}_{t, \mathbf{x}_t, \epsilon_t}[(\epsilon_t - \mu_{\theta}(\mathbf{x}_t,t))] \;. \end{align}\tag{5}\]

It enables gradual denoising from \(\mathbf{x}_t\) through the reverse denoising process to reconstruct the spectrograms. \[\begin{align} \label{eq:32denoising95process} p_{\theta}(\mathbf{x}_{t-1} \, | \, \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-1} \, | \, \mu_{\theta}(\mathbf{x}_t,t), \;(1 - \alpha_t)\mathbf{I} ) \;, \end{align}\tag{6}\] And the reconstructed \(\mathbf{\tilde{x}}_{t-1}\) is calculated with the diversity coefficient \(\beta_t\) and the reconstructed \(\mathbf{\tilde{x}}_{0}\) is obtained iteratively: \[\begin{align} \label{eq:32stepwise95denoising} \mathbf{\tilde{x}}_{t-1} = \frac{1}{\sqrt{\alpha_t}} ( \mathbf{x}_{t} - \frac{1-\alpha_t}{\sqrt{1-\bar \alpha_t}} \epsilon_{\theta}^{(t)}(\mathbf{x}_t) ) + \beta_t \epsilon_t \;, \end{align}\tag{7}\]

Inference with DDIM. While DDPM training is efficient, a major drawback of DDPM is its slow sampling speed due to the Markov chain. DDIM addresses it with a non-Markovian sampling process, which we employ to speed up inference. DDIM redefines the joint distribution \(q(\mathbf{x}_{1:T} \, | \, \mathbf{x}_0)\) to enable faster and more efficient generation: \[\begin{align} q_{\sigma}(\mathbf{x}_{1:T} \, | \, \mathbf{x}_0) = q_{\sigma}(\mathbf{x}_{T} \, | \, \mathbf{x}_0) \prod_{t = 2}^{T}q_{\sigma}(\mathbf{x}_{t-1} \, | \, \mathbf{x}_{t}, \mathbf{x}_0) \;, \end{align}\] where \(q_{\sigma}(\mathbf{x}_{T} \, | \, \mathbf{x}_0)\) is same as Eq. 3 . Based on this, the reconstructed noise-free data \(\mathbf{\tilde{x}}_0\) could be predicted as follows: \[\begin{align} \label{eq:32onestep95denoising} \mathbf{\tilde{x}}_0 = \frac{1}{\sqrt{\bar{\alpha}_t}} (\mathbf{x}_t - \sqrt{1 - \bar{\alpha}_t} \epsilon_\theta^{(t)}(\mathbf{x}_t, t)) \;. \end{align}\tag{8}\] Compared to stepwise denoising in Eq. (7 ), Eq. (8 ) enables skipping-step denoising or even one-step denoising.

2.3 Anomaly score↩︎

The core principle for detection is that since UD-ASD is trained only on normal data, it will attempt to reconstruct any input as a normal audio. Naturally, computing the reconstruction error (\(\mathbf{x}_0 - \mathbf{\tilde{x}}_0\)): normal audio yields low reconstruction error, while anomalous audio, which the model tries to reconstruct as normal, resulting in high reconstruction error.

While simple norms like L1 or L2 can quantify this error, they often fail to capture the complex distribution of reconstruction errors and ignore crucial temporal-spectral features. The process of denoising Gaussian noise in diffusion models makes GMM well-suited for modeling the distribution of residual noise in reconstruction error.

Getting the difference (\(\mathbf{x}_0 - \mathbf{\tilde{x}}_0\)), we apply temporal pooling across the time axis to capture temporal continuity, yielding a 256-dim vector. Next, a GMM with 2 mixture components (which is motivated by the bimodal distribution of machine sounds and noise) and full covariance matrices is trained on the set of error vectors derived from the normal data. During inference, the same feature extraction process is applied to a test sample. The final anomaly score is the negative log-likelihood of the sample’s error vector under the trained GMM. A higher score indicates that the sample’s reconstruction error pattern deviates significantly from the distribution of normal errors, and is thus classified as anomalous. This approach provides a more robust and fine-grained measure of anomaly by modeling the complete statistical distribution of normal reconstruction errors.

3 Experiment↩︎

3.1 Implementation↩︎

Dataset. We evaluate UD-ASD on the DCASE2022 Challenge Task 2 development dataset [19]. This dataset comprises audio from seven distinct machine types, each with three operational sections. The training set consists of 1,000 clips of normal operational sounds per machine section, while the test set contains a balanced mix of 50 normal samples and 50 anomalous samples. Each audio is 10 s.

Implementation details. Audio is first converted to log-Mel spectrograms with a 1024-point FFT and 256 Mel filters, then rescaled to \([-1, 1]\) in a 256\(\times\)​256 resolution. The architecture of UD-ASD is based on [22]. Key hyperparameters are detailed in Table 1. We employ the DDPM for training and the DDIM sampler for inference acceleration with 100 steps. Random seed is 42.

Evaluation metrics. Following the official DCASE challenge protocol, we report the Area Under the ROC Curve (AUC) and the partial AUC (pAUC) over a low false positive rate (p=0.1). We also report the harmonic mean (Hmean) of the average AUC and pAUC scores across all machine types as the primary performance indicator.

4pt

3.2 Performance comparison↩︎

Table 2 compares our method with other models. We report two versions of our model: UD-ASD-S, which trains a separate model for each machine, and UD-ASD-U, our proposed unified model trained on all machine types simultaneously. Our UD-ASD-U achieves the highest overall Hmean, noting superior performance across machines. It secures the best results on four out of seven machine types. Notably, on the Fan dataset, where anomalies often manifest as continuous frequency shifts, UD-ASD-U outperforms the official-AE baseline by +36.95% AUC. This highlights the diffusion model’s superior capability in capturing and reconstructing fine-grained spectral details compared to standard AEs. While some specialized models like HMIC-AGC [25] excel on specific machines by incorporating extra hierarchical information, our unified model proves more robust and generalizable across the board.

The enhanced performance of UD-ASD-U suggests that training on diverse machines acts as data augmentation, improving the model’s ability to learn a more generalized representation of "normalcy". CP then effectively guides the model to reconstruct machine-specific spectrograms, combining the benefits of broad generalization with specialized generation.

AE implicitly learns compressed representations of data through information bottlenecks, which are manifold distributions spread across lower dimensions. This compression causes AE to easily overlook subtle variations, resulting in the model learning only a coarse approximation of the manifold distribution. Diffusion, in contrast, performs explicit, fine-grained repairs. This forced “normalization” repair often becomes overly sensitive to subtle anomalous variations, making it difficult to perfectly reconstruct any data deviating from the normal distribution and thus causing greater reconstruction errors. Furthermore, the one-model-per-machine paradigm is prone to overfitting within specific domains (i.e., corresponding machine types). Cross-domain knowledge transfer, however, compels a unified model to learn more fundamental decoupled feature spaces. This acts as a powerful regularization, enhancing the model’s generalization capabilities and interpretability.

In Figure 2, the distinction between the blue and the red is pronounced, while the green and the yellow tend to resemble. This occurs because UD-ASD attempts to reconstruct abnormal data into normal ones, enabling the identification of anomalies through reconstruction error. In Figure 2, the data for each color is divided into two parts, so we use two mixture components in GMM. Regarding efficiency, while diffusion models are known for slow inference, the DDIM sampler makes our approach practical. On an NVIDIA A40 GPU, UD-ASD (24.4M parameters) achieves an average inference time of 0.32 s per clip, which is well within acceptable limits for many industrial monitoring applications. UD-ASD also faces limitations: 1) It relies on the availability of accurate machine labels, its efficacy in scenarios with missing or error labels requires further investigation. 2) UD-ASD is not designed for zero-shot detection on entirely new machine types not seen during training.

3.3 Ablation study↩︎

Effectiveness of unified model and CP. To validate the efficacy of our unified approach, we compare UD-ASD-U against UD-ASD-S in Table 2. While UD-ASD-S already shows competitive results, UD-ASD-U surpasses it in Hmean and on five machines. This demonstrates that CP successfully provides the necessary conditioning for the model to differentiate between machine features. More importantly, it confirms our hypothesis that a unified model can leverage cross-machine data to enhance its overall representation learning, leading to better performance than isolated training.

Impact of anomaly scoring. We investigate the choice of anomaly scoring by comparing GMM against standard L1 and L2 norms. As shown in Table 3, GMM consistently yields the best results. To visualize this, Figure 3 plots the distributions for normal (green) and abnormal (gray) samples from the challenging Fan machine. For L1 and L2 norms, the distributions exhibit significant overlap, making it difficult to set an effective decision threshold. In contrast, GMM shows a clearer separation between the two classes. This is because the GMM models the complex, potentially multi-modal distribution of normal reconstruction errors, providing a more powerful and robust statistical measure of deviation than simple pixel-wise norms.

4 Conclusion↩︎

We propose the unified diffusion-based model, UD-ASD, for ASD, which is only trained on normal audio, resulting in high reconstruction errors for abnormal audio. The multistep diffusion and denoising process gives UD-ASD strong generative capabilities. Condition Projector enables a single unified model to handle multiple machine types, significantly reducing computational costs. Experiments on the DCASE2022 Task 2 dataset confirm the effectiveness of UD-ASD.

5 Generative AI Use Disclosure↩︎

We use generative AI solely to help polish our paper. And we do not use generative AI to complete any part of the work on its own.

References↩︎