1 Introduction↩︎

Any production pipeline may inevitably produce some defective products, either through human error or machine failures. Therefore, detecting anomalies has always been a key task in increasing the efficiency and robustness of any industrial manufacturing line. However, anomalies can be very rare and are often impossible to predict beforehand, while there is a vast availability of defect-free data. This has led to the birth of semi-supervised anomaly detection (AD), where models learn from normal data only.

With the rise of deep learning, several popular AD benchmarks, along with well-performing methods, have been proposed [1], [2]. The bulk of AD research focuses on single-view images, where objects are usually rigidly aligned in the exact same spot in front of the camera. However, as objects get more complex, a single camera may no longer suffice to detect all possible anomalies. More sophisticated production lines and the ubiquitous usage of imaging systems have caused multi-view anomaly detection to gain more traction [3], [4]. This lets the multi-view AD models also detect anomalies, that may not be covered by just a single camera. The new setting of multi-view AD has not only spawned a variety of new data sets [3]–[5] but also many new AD methods [6]–[8]. An example of such a detection task is visualized in 1, where anomalies are only present in some views.

Currently, a large variety of well-performing AD models use Normalizing Flows for estimating the likelihood of an image [10]–[14]. However, these Normalizing Flow models, which are almost always chosen to be models based on RealNVP realnvp?, may struggle to handle the high dimensionality of (multi-view) images. Furthermore, their coupling block architecture limits them to a few transformation steps of expressivity. A new generative modelling technique termed Flow Matching [15] has begun to gain interest very quickly. Contrary to RealNVP, it can use more general network architectures and more easily represent even very high-dimensional data, while still functioning as both a generative model and a method for density estimation. This is especially interesting for the task of multi-view AD. Here, image data is usually very high dimensional, especially when different view points of an object need to be considered. In AD, very small deviations in the data may already constitute an anomaly. Thus, the decision boundary between normal and anomalous data may be a manifold of much smaller size, embedded within the high-dimensional data. This makes flow matching better suited for multi-view anomaly detection. A toy example, which visualizes the effect of learning a high-dimensional distribution, is shown in 2. Clearly, Flow Matching outperforms the inflexible RealNVP model, as it captures the underlying distribution even when it is embedded in higher dimensions.

Therefore, we propose Multi-view Anomaly Detection using Flow Matching, abbreviated to MATCH. To the best of our knowledge, it is the first Flow Matching-based solution to multi-view AD. We leverage the capabilities in expressiveness to process feature maps of different spatial resolutions. Adding onto that, a few key simplifications are proposed to simplify the likelihood estimation and achieve new state-of-the-art results for both detecting and segmenting anomalies in popular multi-view AD benchmarks. All the while, our model still remains fast to use, reaching \(18.77\) FPS on consumer-level GPUs.

We summarize our contributions as follows:

• We propose a Flow Matching-based multi-view anomaly detection model, outperforming previous methods on both Real-IAD and MANTA-Tiny.

• We propose to omit the divergence term during likelihood estimation, which increases model throughput and thereby real-world usability, while leaving the detection performance unharmed and the score meaningful.

• We are the first to thoroughly benchmark the MANTA-Tiny data set.

• We release all code upon acceptance.

2 Related Work↩︎

We provide an overview of (multi-view) anomaly detection, as well as recent developments in Normalizing Flows, including Flow Matching.

2.1 Anomaly Detection↩︎

2.1.0.1 Overview.

In recent years, research on anomaly detection (AD) has seen lots of success, with most of the popular benchmark data sets, such as MVTec AD [1] solved with almost perfect scores [16]–[18]. In almost all cases, AD is learned in a semi-supervised fashion, where only anomaly-free data is available during training, and one model is trained per object class. Some newer research started incorporating data modalities other than RGB, like depth information or normal maps [19]–[21]. Besides focusing on structural properties of single objects, some benchmarks also include logical anomalies, multiple objects, occlusions, and vast varieties of normal patterns [22], [23].

Besides data sets, there exist lots of categories of AD methods. Firstly, Reconstruction-based methods aim to reconstruct the image features through some bottleneck mechanism [24]–[27]. Closely related are student-teacher networks, which distil information from a (potentially frozen) teacher network into a student network [9], [18], [28]–[30]. Some works also incorporate anomalous information, either real or synthetically generated, to cast the task into a supervised problem [17], [31]–[33]. As large vision-language-models (VLM) become more potent, zero-shot AD focuses on detecting anomalies without any prior knowledge [34]–[37]. Using one model for all classes within a data set, a task termed multi-class anomaly detection, has also seen a rise in popularity [27], [38], [39].

Many methods, including ours, may be grouped into the field of density estimation. By learning the distribution of the normal training data, the likelihood of a sample is approximated and used as the anomaly score. Methods based on k-nearest Neighbors (kNN) or memory banks form one strand of research [16], [40]–[42], but rely heavily on the choice of features [43]. A more principled way of estimating the density is offered by Normalizing Flows (NF), which have seen wide application in AD [6], [10]–[14]. Here, the network explicitly learns the likelihood of a given sample using maximum likelihood estimation. A natural next step from NFs is Flow Matching [15], which is the core of our method as will be explained in [sec:related_flows,sec:method].

2.1.0.2 Multi-view Anomaly Detection.

Mirroring real industrial scenarios with complex objects, a single camera may not suffice to capture all anomalies. Therefore, research on multi-view anomaly detection has started gaining traction, with the Real-IAD data set [3] serving as a new important benchmark. It shows each object from five different view points, with anomalies having to be detected both in an image-wise, as well as an aggregated object/sample-wise fashion. First methods for this exact setting have been proposed already [6], with some of them focused on multi-class AD [44].

Fan   present MANTA, a data set which features a camera setup very similar to Real-IAD [5]. MANTA covers a much higher variety of samples and categories, while also being magnitudes larger than anything before. Therefore, the authors also released MANTA-Tiny, a subset with largely the same test split, which is more easily usable for research purposes and solely used in this paper next to Real-IAD. To the best of our knowledge, we are the first to report comprehensive benchmark results on MANTA-Tiny with our experiments.

Another strand of research on multi-view AD uses more complex objects with dozens or up to hundreds of views per object [4], [45], [46]. Here, approaches based on novel-view synthesis methods have seen the most success [4], [7], [46], [47]. This remains, however, limited to one inflexible 3D model of normal samples, which fails when there is a larger variability of objects. This is something naturally addressed by most other anomaly detection methods, as they do not explicitly save one normal prototype to compare to. Concretely, density estimation techniques like Flow Matching allow for learning more complex distributions of normal samples.

There exist other variants of AD that intersect with multi-view/3D methods, such as finding one abnormal object out of many [48], leveraging 3D ground truth information [49], or dealing with complex lighting scenarios [46], [50]. All of these multi-view settings have significantly increased the difficulty in anomaly detection benchmarks and thereby inspired the research community.

2.2 Normalizing Flows↩︎

2.2.0.1 Preliminaries.

The concept of Normalizing Flows [51] (NF) is based in the change of variables formula, which forms a relationship between a source distribution \(p\) (usually a standard Gaussian) and the target or data distribution \(q\), which are both distributions over \(\mathbb{R}^d\). The NF model \(\phi: \mathbb{R}^d \rightarrow \mathbb{R}^d\) learns the mapping between data samples \(x \sim q\) and \(p\) via \[\label{eq:change95of95variables} q\left(x\right) = p\left(\phi\left(x\right)\right) \left|\det \left[ \frac{\partial \phi}{\partial x}\left(x\right) \right] \right|,\tag{1}\] with \(p\left(\cdot\right)\) and \(q\left(\cdot\right)\) denoting the respective probability density functions (PDFs). Most notably, the flow \(\phi\), usually parametrized by a neural network, needs to be both invertible and have a Jacobian whose determinant is easy to compute. This has lead to the creation of specifically designed networks, that retain these properties [52]–[55], with RealNVP being used the most in the anomaly detection literature for its ease of use [6], [10], realnvp?.

Regular NFs usually chain together several invertible transformations \(\phi = \phi_n \circ \ldots \circ \phi_1\), limiting their expressivity by the number of chained flows \(n\). This is alleviated with Continuous Normalizing Flows (CNF), which aim to learn the change in probability mass at every step using an ODE formulation [56]. The flow model now mimics the velocity field \(u:\mathbb{R}^d \times \left[0, 1\right] \rightarrow \mathbb{R}^d\), learning the transformation needed to move a sample from \(p_0\) at time \(t=0\) to \(p_1 = q\) at time \(t = 1\). In practice, however, CNFs are notoriously costly to train, requiring solving the entire ODE for a single training step [56], [57].

2.2.0.2 Flow Matching.

With Flow Matching (FM) [15] training of CNFs has been simplified greatly by constructing conditional probability paths guiding from noise \(p\) to data \(q\). Formally, an ODE is defined by its time-dependent vector field \(u_t\), which is solved by its respective flow \(\phi_t(x_0)\) as \[\frac{d}{dt} \phi_t\left(x_0\right) = u_t\left(\phi_t\left(x_0\right)\right),\] where the flow solution aims to satisfy \[\phi_t\left(x_0\right) \sim p_t \;\text{with initial condition}\;\, \phi_0\left(x_0\right)\sim p_0.\] Thus, we aim to learn the vector field \(u_t\), such that its flow \(\phi_t\) generates the probability path from \(p_0\) to \(p_1\), where each \(p_t\) is again a distribution over \(\mathbb{R}^d\). As shown by Lipman  [15], the marginal vector field \(u_t\left(x_0\right)\) turns out to be equivalent to the expectation of the conditional vector field over the data distribution as \[\label{eq:vector95field} u_t\left(x_0\right) = \mathbb{E}_{x_1 \sim q} \left[ u_t\left(x_0 | x_1\right)\right].\tag{2}\] Thereby, sampling data \(x_1 \sim q\) and regressing against the conditional vector field \(u_t\left(x_0 | x_1\right)\) suffices for learning a valid flow. This conditional vector field can easily be constructed given the available data, using Gaussian probability paths. This also results in the loss becoming a simple regression against the constructed conditional vector field. Subsequent work has explored different probability paths and other formulations, resulting in models with strong generative capabilities [58], [59].

3 Method↩︎

3.0.0.1 Problem Statement.

During training, anomaly-free multi-view images of a certain object category are given. At inference, both anomalous and defect-free images appear, with the model having to detect the occurring anomalies. An object is considered anomalous if there is an anomaly visible in any of its views.

3.0.0.2 Objective.

As motivated with 2, the distribution of normal images is learned using FM, specifically the optimal transport conditional Flow Matching (OT-CFM) formulation [58]. The loss for optimizing the parameters \(\theta\) of the flow network \(u_t^\theta\) is \[\label{eq:ot95cfm95loss} \mathcal{L}_{\text{CFM}}^{\text{OT}} = \mathbb{E}_{\begin{subarray}{c} x_0 \sim p_0, x_1 \sim q \\ t \sim \mathcal{U}[0,1] \\ \end{subarray}}\; || u_t^\theta \left(x_t\right) - \left(x_1 - x_0\right) ||^2,\tag{3}\] where \(p_0\) is standard normal Gaussian noise, \(q\) the data distribution and \(x_t = (1-t) x_0 + t x_1\) an intermediate point along the flow.

Using the trained \(u_t^\theta\) and any off-the-shelf ODE solver, we can use our algorithm in two ways. Running the ODE from \(t=0\) to \(t=1\) transforms noise \(x_0 \sim p\) into data samples \(x_1 \sim q\) (generative direction). Reversing it, and integrating the ODE from \(t=1\) to \(t=0\) lets us move real data samples to the Gaussian space, where we are able to measure likelihoods (density estimation direction). The likelihood of a sample \(x\) will serve as a proxy for the anomaly score and may be calculated as \[\label{eq:log95likelihood95equation} \log p_1 \left( x_1 \right) = \log p_0 \left(x_0\right) - \int_0^1 \text{div}\left(u^\theta_t\right) \left(x_t\right) dt,\tag{4}\] where \(p_0 \left(x_0\right)\) is the density of the resulting sample in Gaussian space and \(\text{div}\left(u_t^\theta\right)\) the divergence of the vector field at the intermediate solutions \(x_t \sim p_t\) along which we integrate. In practice, this divergence term is estimated using methods such as Hutchinson’s trace estimator [60] or randomized quasi monte-carlo (RQMC) [61]. Even then, estimating the divergence term would result in (at the very least) one backpropagation per ODE step, which would vastly increase the computational requirements. This would leave the model unsuitable for deployment in real-time industrial production scenarios. An ablation conducted in 4.4.1 shows that omitting the divergence term greatly saves computation time and memory while still performing well for AD.

Thus, our anomaly score just becomes \(S\left(x\right) = -\log p_0 \left(x_0\right),\) which in practice, calculated along the feature dimension, becomes \[\label{eq:anomaly95score} S(x_1) = -\log p_0\left(x_0\right) = \frac{||x_0||^2}{2} + \frac{d}{2}\log\left(2\pi\right),\tag{5}\] where \(x_0\) is the sample in Gaussian space, and \(d\) the feature dimensionality. This score may be interpreted, up to the additive constant, as the squared Mahalanobis distance of the sample \(x_0\) to the standard Gaussian \(\mathcal{N}\left(0, I\right)\). This interpretation is used in many related papers on AD and normalizing flows [10], [11], [13].

3.1 Model Architecture↩︎

Normalizing flows trained on raw images have been shown to fail in detecting anomalies [62]. Further, Flow Matching, just as diffusion models, performs well on latent feature distributions [59]. Therefore, the flow is instead learned on the latent distribution of a WideResNet [63] (pre-trained on ImageNet [64]) which serves as a frozen feature extractor.

We take inspiration by the ReverseDistillation (RD) architecture [9], with a high-level schematic of our adapted architecture shown in 3. A longer and completely in-depth graph of all layers and blocks can be found in the supplement in 6. Firstly, the data samples are extracted from the first three layers \(E_i\) of the WideResNet as \(\{x_1^i \sim q^i\}_{i=1}^3\), where \(i\) denotes the current layer of the extractor. To train, a time step \(t \in \mathcal{U}\left(0,1\right)\) and noise of the same size as the latent features \(\{x_0^i \sim p^i\}_{i=1}^3\) are sampled. Then, the intermediate features \(\{x_t^i = (1-t) x_0^i + tx_1^i\}_{i=1}^3\) are constructed as a linear combination and fed through a bottleneck. After that, several ResNet-style layers \(D_i\) in reversed order are used to reconstruct features to the correct spatial dimension, similar to the architecture for RD4AD [9].

Time and View Embedding. To make the model usable for Flow Matching, we embed the time step information \(t\) using a \(1024\)-dimensional sinusoidal positional encoding [65] and a simple two-layer MLP with SiLU activation [66]. Similarly, the index of the current view \(v\) is fed into a learned embedding layer as well as an MLP with the same dimensions. Both view and time information are concatenated, passed through another fully connected (FC) layer with SiLU activation and added to the output of the bottleneck. They also serve as input to the reconstruction layers \(D_i\), with each layer having their own FC and SiLU layer for fine-tuning features to their needs.

Bottleneck Network. The input features \(\{x_t^i \in \mathbb{R}^{C_i \times H_i \times W_i}\}_{i=1}^3\) are fed into the bottleneck network, consisting of an alignment network and bottleneck blocks. For the alignment, the map at \(i=3\) has the lowest feature resolution. We thereby run the other maps through a down sampling using \(3 \times 3\) convolutional layers with stride \(2\), using one layer for \(i=2\), and two for \(i =1\). The spatially aligned maps are then concatenated to produce one feature vector. Each of the mentioned convolutional layer consist of its respective convolution followed by group normalization with \(32\) groups, and ReLU activation. Afterwards, three ResNet-style bottleneck blocks [9], [67] (again with group normalization) further distil the feature representation, forcing the network to focus on the normal structures found in training only. Residual connections are either directly implemented, or using a downsampling with another convolution.

Decoder Blocks. The decoder consists of three layers \(D_i\) each consisting of \(d_i\) decoder blocks, that progressively upsample the bottleneck feature representation back to the original spatial dimensions, which mostly follows RD [9], [68]. These decoder blocks use \(1 \times 1\) convolutions followed by \(3 \times 3\) convolutions with a stride of \(2\) for upsampling [68], group normalization, ReLU activation and a final \(1 \times 1\) convolution. Here, the time-view embedding is again added to the latent features. Again, the residual connection may be done either using an identity map or a single transposed convolutional layer for upsampling.

Anomaly Scores. We produce three anomaly score maps \(M_i \in \mathbb{R}^{H_i\times W_i}\) by calculating the anomaly score along the feature dimension of our decoded features as in 5 . All of these maps are resized to the full spatial dimension of our images using bilinear interpolation. For segmentation, we simply use the combination of all maps \(M_\text{segment} = \sum_i M_i\) as our final segmentation map. This includes both good segmentation of high-level defects, as well as the tiny low-level and hard-to-spot anomalies.

For detection, it has been shown that higher level representations are better suited to produce a single per-image anomaly score [62]. Thereby, we use \(M_{\text{detect}} = M_2 + M_3\) as the sum of only the last two anomaly maps, which mostly focuses on higher-level concepts while not being influenced too much by lower level deviations. The final image-wise anomaly score is then produced by taking the maximum value found in this detection map \(M_\text{detect}\). To produce a sample-wise anomaly score, we again take a maximum across all anomaly scores produced for one multi-view instance.

4 Experiments↩︎

We evaluate our method on data sets Real-IAD [3] and MANTA-Tiny [5], while also reporting numbers for many competing state-of-the-art approaches.

4.1 Data Set and Metrics↩︎

We use both Real-IAD [3] and MANTA-Tiny [5] for evaluating our method. Real-IAD [3] is a large multi-view anomaly detection data set, made up of 30 different object classes taken from industrial production contexts, totalling roughly 100k normal and 50k anomalous images. Defects such as scratches, missing parts or deformations are some of the most common categories of anomalies.

MANTA [5] contains multi-view images of tiny objects, with the different object classes spanning across the categories of agriculture, medicine, electronics, mechanics, and groceries. With MANTA being about five times the size of Real-IAD, a smaller benchmark called MANTA-Tiny has been proposed, where images are resized and the amount of training instances is reduced to roughly match previous data sets and increase its popularity. The test sets of both splits remain largely the same. Depending on the category, defects may appear as misprints, stains, scratches, mold, or mildew, amongst many other possible faults.

For quantitative results, we report detection performance using the Area Under the Receiver Operating Curve (AUROC), which is independent of a specific threshold. The AUROC will be reported both image-wise (each image is treated independently, named I-AUROC) and sample-wise, where all images belonging to the same object are grouped together to form one single anomaly score (named S-AUROC). Lastly, for segmenting the anomalies, we also report the AUROC in a pixel-wise fashion (P-AUROC), as well as P-AP (average precision), and the more balanced P-AUPRO metric [1].

4.2 Implementation Details↩︎

All experiments are run on a Linux machine with a RTX 3090 consumer GPU. All metrics are calculated efficiently using the GPU-optimized ADEval library [71]. We resize all images to \(256 \times 256\) pixels, resulting in extracted feature maps of sizes \((256 \times 64 \times 64)\), \((512\times 32 \times 32)\), and \((1024 \times 16 \times 16)\) using the WideResNet50.

For optimization we use the AdamW optimizer [72] with a learning rate of \(1e{-4}\), momentum parameters \(\beta_{1,2} = \left(0.9, 0.95\right)\), and weight decay set to \(0.01\). Contrary to most NF models, we do not need any clipping or scaling for the gradients [73]. The decoder layers \(D_i\) consists of \(3\), \(4\) and \(6\) decoder blocks respectively. All convolutional layers work with a base width of \(768\) dimensions. We train on both data sets for \(150\) epochs with a batch size of \(8\). For inference using the ODE, we use the simple Euler method with step sizes of \(0.2\), resulting in \(5\) forward passes per inference batch. We will later make this choice more founded in an ablation in 4.4.2.

4.2.0.1 Baselines.

A variety of state-of-the-art AD methods are reproduced using the Anomalib [74] and ADer [71] benchmarking libraries. We chose them to be executed with the same consumer-level GPUs and setup as our method. The baselines include CFA [42], DSR [69], EfficientAD [18], FastFlow [12], Multi-Flow [6], PaDiM [40], ReverseDistillation (RD4AD) [9], SimpleNet [17], and STFPM [70].

4.3 Anomaly Detection Results↩︎

We run MATCH on both Real-IAD and MANTA-Tiny. For Real-IAD, all metrics are reported in 1. Here, we outperform all competitors in most metrics by convincing margins, reaching an I-AUROC of \(91.17\) for detection, and an P-AUPRO of \(94.76\) for segmentation, all while being very stable with a standard deviation of \(.08\) and \(.03\) respectively. Most notably, the direct flow-based competitor Multi-Flow, based on RealNVP, underperforms in detection by roughly one point, while MATCH is far more precise at segmentation, as visible in the P-AUPRO. This strengthens us in our claim that FM is better able to handle the high dimensionality of the feature maps, as it is still able to pinpoint the locations of the occurring anomalies.

As for the sample-wise detection (with S-AUROC), we equip all baselines with a simple max-aggregation, just as used by MATCH, to calculate their scores. We achieve an S-AUROC of \(95.63\) on Real-IAD. This score only lacks slightly behind the most direct competitor Multi-Flow, which is also optimized for multi-view AD, by \(.2\) points. On all other metrics and tasks, we outperform the baselines, especially in segmentation tasks.

Segmentation results for the newly proposed and until now not thoroughly evaluated MANTA-Tiny data set are presented in 2 3. The most important results on segmentation are presented with the P-AUPRO in 2. Here, we achieve a P-AUPRO of \(89.66\) across all categories, which outperforms all baselines. We further note that classes containing a larger variance in their normal samples, such as the naturally produced agriculture and groceries categories, are far harder to solve than categories focusing on classical industrial contexts. Also, the ReverseDistillation (RD4AD) model almost always comes in second, highlighting the effectiveness of the chosen architecture. Still, the Flow Matching formulation managed to gain very strong improvements, with a P-AUPRO more than four points higher.

Only in the detection metric I-AUROC does MATCH fall behind, as SimpleNet is able to get a score of \(93.19\) while MATCH reaches a competitive \(90.66\). We posit that this may be due to the much higher variance in defects found in MANTA-Tiny, as some tiny defects may be harder to translate into a high score for the entire image. Handling this issue may be interesting for future work. Lastly, we also evaluated our model (trained on MANTA-Tiny) on the full MANTA test split, which is roughly 70% identical. We observed no noticeable performance drops, as shown in the supplementary. This shows MANTA-Tiny to be a good real-world benchmark, that is worth exploring.

For both of the mentioned benchmarks, we provide a variety of qualitative anomaly maps in 4. Here, the high segmentation quality of MATCH becomes apparent, which is a clear advantage to preceding Normalizing Flow-based methods [6], [11]. We also provide additional qualitative segmentations for every category of MANTA-Tiny, as well as an analysis of some failure cases in the supplement in 11, together with the comprehensive results for all metrics on both data sets in 12.

4.4 Ablation Studies↩︎

We conduct ablations to motivate design choices and provide further insight.  5 shows the impact of different base widths of our network. The AD performance being best around a width of \(768\) confirms that strong and expressive (FM-based) models are most suited. Also, matching the parameter count of RD4AD to MATCH, RD4AD reaches an I-AUROC of \(86.75\), confirming that gains stem from the FM formulation, and not just model capacity. We do not consider higher widths, as we want MATCH to remain usable on affordable consumer-level hardware.

4.4.1 Impact of the Divergence Term.↩︎

As described in 3, we omit the divergence term in 4 , which is equal to the trace of the Jacobian matrix \(\text{tr}[\partial_x u_t\left(x\right)]\), to increase the computational efficiency of our model. There are several possible methods to estimate this divergence term. Using Hutchinson’s trace estimator, we may sample \(M\) vectors \(z \sim \mathcal{Z} = \mathcal{N}\left(0,I\right)\) and calculate \[\text{div}\left(u_t\right)\left(x\right) = \text{tr}\left[ \partial_x u_t\left(x\right) \right] = \mathbb{E}_{z \sim \mathcal{Z}} \; \text{tr}\left[z^T \partial_x u_t\left(x\right) z \right]\] as an unbiased estimator of the divergence [51], which is added to the anomaly score to form the likelihood. By increasing the number of random vectors \(M > 0\), we get a more accurate estimation.

While Hutchinson’s estimator draws its \(z\) vectors independent of each other, we may try to decrease the variance in our prediction by sampling them from a dependent distribution. This is the basis for randomized quasi monte-carlo (RQMC) [61]. For the vectors for RQMC we use Latin hypercube sampling [75], due to the high dimensionality of our feature vectors. The results for all estimation methods are shown in ¿tbl:tab:results95ablation95div95term? for different values for \(M\).

Including the divergence does not lead to any noticeable gains for the detection AUROC. The same behavior applies to all metrics, as completely shown in the supplementary materials. Most interestingly, with \(M = 1\), the divergence only makes up a total of \(.57\%\) of the final score magnitude on Real-IAD, since \(\frac{\left|\text{div}\left(u_t^{\theta}\right)\right|}{\left|\log\left( x_0\right)\right|} = 0.0057\). Moreover, leaving it out gets a boost of at least three times more model executions per second. Also, increasing \(M\) linearly increases both execution time and memory requirements, while also not gaining any precision in detection. We hypothesize that our model learns a very smooth vector field flowing from \(p_0\) to \(p_1\), which has almost no noticeable local divergence. Furthermore, this strengthens us in our decision to leave RealNVP-style models behind, which rely on architectures specifically designed to make this divergence tractable. Lastly, the flexibility gained with Flow Matching (as was visualized in 2) also provides an advantage on important real-world benchmarks, when comparing it to its more constrained RealNVP competitors.

4.4.2 Impact of ODE Solver.↩︎

We test a variety of popular methods for solving ODEs on the Real-IAD data set. For this we choose methods with fixed step sizes, namely the Euler method, the Midpoint method, and the fourth order variant of Runge-Kutta (RK4) [76]. We vary their step sizes from \(0.2\) to \(0.05\) and record their impact on AD performance and model throughput (as FPS) in [tbl:tab:ablation95ode]. We also compare to the adaptive step version of Runge-Kutta (Dopri5). Additional adaptive solvers, such as Euler-Heun, Fehlberg2, or Bosh3 achieved similar scores, but are reported in the supplementary for brevity.

Going from Euler to Midpoint to RK4, the amount of function evaluations per batch doubles. This is also visible in the throughput, which gives the Euler method a clear edge for fast in-production use, since it is the only method capable of processing more than \(10\) images per second. The adaptive Dopri5-solver is slower than all competing methods while not bringing any more precision to the detection. However, none of the methods show an all-too-clear advantage in the measured AUROC except for the Euler method. Here, a small improvement can be measured only for the higher step sizes. All other methods achieve an AUROC of roughly \(91.06\), irrespective of step size.

Clear hypotheses can, however, not be made, as the AUROC improvement with Euler lies roughly within the standard deviation reported in 1. Nevertheless, the simplicity still makes the Euler method the fastest and best usable ODE solver for our AD use case, as there is no trade-off between speed and accuracy. It is also consistent with previous research, which showed no substantial numerical errors, when using simpler ODE solvers [15].

4.4.3 Supplementary Experiments.↩︎

We conduct three additional experiments in our supplementary materials in [sec:appendix_mvtecad] [sec:appendix_multiclass] [sec:appendix_dinov3], to further validate our model’s strength. Firstly, we report metrics for the classic MVTec AD benchmark in the appendix, where MATCH is still competitive. Furthermore, we also test the popular multi-class AD setting. Even though MATCH is not explicitly focused on multi-class AD, it achieves a very competitive I-AUROC of \(87.5\) on Real-IAD, only slightly beaten by Dinomaly [27]. Lastly, we highlight the robustness to other backbones, by swapping MATCH and PaDiM & RD4AD to a DINOv3 [77] backbone. Even then, the AD performance stays the same or deteriorates slightly. Furthermore, with all methods fixed to a DINOv3 backbone, MATCH still outperforms the other methods. Still, we prefer the classic WideResNet due to its lower computational costs.

5 Conclusion↩︎

We have proposed MATCH, an effective framework for detecting anomalies in the multi-view setting. Using the simple Flow Matching formulation and a multi-layer feature extraction pipeline, we are able to establish state-of-the-art results on Real-IAD, as well as new comprehensive benchmarking results on MANTA-Tiny. Future work will focus on including more prior knowledge, such as explicit 3D constraints or multi-view correspondences, into the Flow Matching pipeline. More recent improvements in Flow Matching and generative modelling, such as energy matching [78], may also suit themselves to future endeavours in anomaly detection.

Acknowledgements↩︎

This work was supported by the MWK of Lower Saxony within Hybrint (VWZN4219) and LCIS (VWZN4704), the Deutsche Forschungsgemeinschaft (DFG) under Germany’s Excellence Strategy within the Cluster of Excellence PhoenixD (EXC2122) and Quantum Frontiers (EXC2123), the European Union under grant agreement no. 101136006 – XTREME.

6 Architecture Details↩︎

We describe the architecture in greater detail, noting down all layers and their configurations. The respective input dimension, output dimension, and stride (s1 for a stride of \(1\)) are shown below the convolutional blocks. "Conv" describes regular, while "ConvT" are transposed convolutions. The Group Normalization ("GN") blocks also show the number of groups (always 32), as well as the feature dimension. Occasionally, tensor dimensions are displayed below the data nodes.

6.0.0.1 Embeddings.

The short MLPs for embedding time and view information are shown in 6.

6.0.0.2 Bottleneck.

The full bottleneck, including feature alignment and bottleneck blocks, is shown in 7. Three feature vectors \(x^i\) are aligned by convolutional layers and then concatenated. After adding the already concatenated embedding of view and time information ("emb"), they are passed to three bottleneck blocks, with input dimensions \(3072\), \(2048\), and \(2048\) respectively. This results in the first block needing to use downsampling for its residual connection, while the others use identity residual connections.

6.0.0.3 Decoder.

A decoder block is shown in 8. Again, whenever input and output dimension do not match (which is the case for every first block), the residual connection is done with a convolutional upsampling. Each first block (with the downsampling residual connection), also uses ConvT2x2 instead of Conv3x3. The output of one block serves as input for the next.

Each decoder block is determined by a \((\text{input}, \text{hidden}, \text{output})\)-tuple of dimensions. Additionally, the embedding is also added onto the resulting feature. The mentioned blocks \(D_i\) of 3 consist of three, four, and six blocks respectively. Thus \(D_1\) has a \((2048, 3072, 1024)\) block, followed by two \((1024,3072,1024)\) blocks. \(D_2\) contains a \((1024,1536,512)\) block and then three \((512,1536,512)\) blocks. Lastly, \(D_3\) consists of a \((512,768,256)\) block and five \((256,768,256)\) blocks.

Changing the width_per_group parameter, currently set to \(768\) and ablated in 5 in the main paper, controls the width of these layers. Each decoder \(D_i\) produces an output \(\hat{x}^i\) of the same shape as the respective input feature \(x^i\), with which the Flow Matching loss is calculated.

7 Additional Multi-Class Anomaly Detection↩︎

Additional evaluation is done with the popular multi-class AD setting, despite MATCH not being explicitly designed for this task. Here, all training splits of the classes of Real-IAD are combined. After training for \(100\) epochs, all classes are evaluated independently.

The average performance across all classes is shown in 4. Baseline numbers are taken from multi-class AD method Dinomaly [27], which takes some numbers from the ADer benchmark paper [71]. We outperform all regular single-class methods by large margins, while being competitive with the multi-class baselines. Concretely, we achieve an I-AUROC of \(87.5\), only lacking two points behind Dinomaly.

8 Additional Results on MVTec AD↩︎

We report results on the popular MVTec AD data set [1] in 5. Since we did not optimize our method specifically for this data set, we do not reach peak state-of-the-art performance, but are still comfortably competitive, with an I-AUROC of \(97.46\) and a P-AUPRO of \(93.88\). Some qualitative examples are found in 9.

9 Additional Results for DINOv3 Backbone↩︎

We also test the new DINOv3 [77] model as a backbone for MATCHas well as baselines RD4AD [9] and PaDiM [40]. We chose these baselines, since they most naturally support swapping to a different custom backbone. Since we still restrict ourselves to consumer-level hardware, we use the DINOv3 ViT-S/16 model, which is pretrained on the enormous LVD-1689M data set. The results are presented in 6. As MATCH still performs strongest, we confirm its robustness to different and newer backbones. However, we still prefer to use our WideResNet, since it achieves similar or stronger scores, while being less computationally expensive. Expanding to different backbones or even learning on raw pixel scores could therefore be interesting avenues for future research. Further, introducing huge pretraining data sets may not necessarily lead to an increase in performance.

10 Two Moons Toy Experiment↩︎

We describe the setting for the toy experiment with the "Two Moons" data set shown in 2 (main paper). While the data itself is two-dimensional, we linearly embed it in \(d\)-dimensional space by sampling a random matrix in \(\mathbb{R}^{d\times d}\), orthogonalizing it with QR decomposition and using the first \(n = 2\) dimensions for embedding our data. We use our flow models to map the moons, lifted to \(d\) dimensions, onto a \(d\)-dimensional multivariate standard Gaussian, from which we will later sample new instances. For visualization, we use the inverse of our embedding to go back to two dimensions. For our experiments, we use the values \(d \in \{2, 8, 32, 128,256,512\}\). Both models use the Adam [82] optimizer with a learning rate of \(0.001\) and \(\beta_{1,2} = \left(0.9, 0.999\right)\). We train for \(50\)k epochs with a batch size of \(512\).

For Flow Matching, we use the same loss as in 3. The model is a series of six simple four-layer MLPs, with a hidden dimension of \(32\) and SELU activation, which has them ranging between 20k and 39k parameters, depending on \(d\).

The RealNVP realnvp? model consists of \(8\) coupling blocks trained with a maximum likelihood loss. These blocks have a fixed random permutation, which simulates random masking needed to have all features interact with each other. Each coupling block has a network for scaling \(s\) and translation \(t\). These are implemented as four-layer MLPs with LeakyReLU activation, layer normalization and the same hidden dimension of \(32\). Clamping the scale factors \(s\) to the interval \([-3,3]\), as suggested by Ardizzone et al. [73], further increases training stability. Since RealNVP needs to preserve the dimension of the input after every coupling block, the models lie in the range of 38k to roughly 170k parameters, which is much larger than their respective Flow Matching counterparts.

10.1 Quantitative Experiment↩︎

We quantitatively measure the quality of our density estimation using bits per dimension (BPD) [55], which is the normalized average negative log likelihood.

For the Two Moons experiment, the FM model achieves consistently sensible BPD values (in the range of \(1.3\) to \(2.0\)) for all dimensions. RealNVP, on the other hand, produces degenerate (negative) BPD values in all dimensions \(> 2\).

Additionally, we want to evaluate the quality of the density estimation on the actual feature space used by MATCH. For this, we construct a RealNVP-based normalizing flow baseline, which is based on the strong teacher network of [28]. We use one teacher network for each of the feature-map scales, with four coupling blocks and a hidden dimension of \(256\) each. We follow all settings of Rudolph  [28], aggregating features across scales using the maximum operator. Likelihoods for MATCH are calculated using Hutchinson’s trace estimator (with \(M=1\) sample).

7  shows the result for both AD and density estimation. The NF baseline achieves non-trivial I-AUROC (even surpassing some other baselines), while its BPD of \(160.5\) is vastly higher than MATCH, which lies at \(1.83\). Additionally, Multi-Flow, which is trained on a different feature-space, also reaches a higher BPD of \(2.10\). This confirms the FM formulation as better suited for not only the AD task, but also more capable for density estimation in these feature spaces.

11 Additional Qualitative Results↩︎

We provide a larger qualitative overview of well-segmented anomalies in 11.

11.0.0.1 Failure Cases.

We present a variety of common failure cases of MATCH in 10, discuss them here and point towards possible solutions for future models.

Both 10 (a) 10 (b) show very large ground truth anomalies, which are not fully segmented. The "Shriveling" anomaly is caused by the raw paddy rice being less wet and underdeveloped. However, the overall structure remains intact, while only a very slight deviance in colour needs to be detected. Due to the very large and diverse training set, detecting this anomaly is magnitudes harder than most of the industrial anomalies, which have less ambiguity in colour.

As for the crack in the soybean in 10 (b), the crack itself is properly segmented. However, with large chunks missing, the entire soybean is now considered an anomaly, even when locally it may appear intact.

10 (c) presents a false positive, which consists of a black spot (where the soybean was once connected to the plant). This black spot is also visible in some of the training images and is naturally not considered an anomaly. However, since this spot is rather small, most of the training data does not show it, making it rarer and therefore naturally more prone to be close to the decision boundary and a false positive. Here, data augmentation or focussing more on sub-optimally covered areas in the training set may help in eliminating this error.

10 (d) shows a capsule with two blue caps, whereas one cap should be white. Again, the entire object is considered an anomaly. Each of the blue caps could, by themselves, be considered locally plausible. Only in their global combination do they appear as an anomaly. However, the segmentation is currently more focused towards local defects, while detection is reasoning more globally. Informing the segmentation with the more global anomaly score could improve the model here.

[fig:failure_cases_subfig5,fig:failure_cases_subfig6] both present false positives caused by the ambiguity of the data. The last view of 10 (e) has a small foreign object in the corner, that is not part of the anomaly mask. This failure may be alleviated by segmenting the objects before processing them. In 10 (f), the object is rotated in a way such that it reflects the light from source into the camera. This lighting is never found in the training data and thereby it is treated as an anomaly by the model, even when the 3D structure is plausible. Additional 3D scans/depth estimation could help circumventing the issue of lighting irregularities.

12 Comprehensive Experimental Results↩︎

12.0.0.1 Comprehensive Ablations.

The impact of removing the divergence on every metric (as described in 4.4.1 in the main paper) is shown in 8. We executed this experiment with a batch size of \(2\), and record the average GPU memory usage across the entire test set.

All metrics for the ablation of the ODE solvers from 4.4.2 are shown in 9.

12.0.0.2 Comprehensive experiments on Real-IAD and MANTA-Tiny.

The results for the test splits of MANTA versus MANTA-Tiny are shown in 10.

We also provide the comprehensive results of every model tested on every object category/class for the metrics I-AUROC, S-AUROC, P-AUPRO, P-AUROC, and P-AP on Real-IAD and MANTA-Tiny. Metrics for Real-IAD are reported in 11 12 13 14 15, while 16 17 18 19 20 show results on MANTA-Tiny.

References↩︎