1 Introduction↩︎

The escalation of cyber threats has made anomaly detection central in computer security. Organizations face increasingly sophisticated attacks, from targeted intrusions and APTs to advanced fraud schemes [1]–[3]. Malicious actions are rare and often hidden within massive volumes of legitimate activity, making minority-class detection one of the key challenges. Automated systems must identify new threats without excessive false positives, while ensuring interpretability for compliance [4]–[6]. Large-scale analyses [7] highlight that most cyber incidents are not headline-grabbing, but sector risks differ greatly. Fraud types keep evolving, including affiliate marketing abuse [8], and user response is shaped by liability and reimbursement differences across countries [9]. Detecting frauds in financial settings [10], [11], especially in systems based on rule induction [12] where it is rare, remains a central challenge. Similar issues arise in domains like deepfake detection [13]–[15]. New biases such as ‘impostor bias’ [16] further complicate anomaly detection. Traditional machine learning models such as Logistic Regression [17], Random Forest [18], and XGBoost [19] are widely used, but their effectiveness drops with heavy imbalance, often requiring sophisticated sampling or cost-sensitive learning. Among oversampling strategies, two main families dominate:

• SMOTE-based methods [20]: Synthesize new minority samples via interpolation, balancing the training set. They are effective but can produce redundant or overly smooth data.

• Generative models (VAEs [21], GANs [22], DMs [23], [24]): These generate diverse samples from learned distributions, but often require significant tuning and are usually trained only on minority data, which can limit diversity and generalization.

Deep classifiers are effective for fraud detection [25], but are often black-boxes, making interpretability difficult in sensitive domains. Generative methods typically focus on augmenting the minority class without shaping the latent space for better decision-making. To address these limitations, we propose the Causal Prototype Attention Classifier (CPAC), a lightweight and interpretable architecture that uses prototype-based reasoning and feature attention for robust classification under imbalance. Coupling CPAC (or similar classifiers) with generative model encoders such as VAE-GAN enables latent space shaping that maximizes class separability and interpretability, outperforming SOTA oversampling methods in both clustering and detection metrics. We validate our approach on the Kaggle Credit Card Fraud Detection [26] dataset, benchmarking CPAC-augmented models against traditional classifiers, SMOTE, and generative oversamplers, as well as MLP-based latent classifiers. The main contributions of this work are as follows:

• We present the CPAC, an interpretable classifier that combines prototypes and attention for reliable fraud detection under extreme class imbalance.

• We introduce a classifier-guided latent shaping approach by attaching a classifier to the encoder of a VAE-GAN, enforcing class-aware clustering and improving downstream classification performance.

• We included the CPAC to the encoder of the VAE-GAN to improve the results of the generic classifiers by exploiting the inner qualities of the CPAC.

• We prove and discuss how training generative models only on fraud data proves ineffective despite the high performances that these might lead the classifiers, causing overconfidence and poor representation of the actual data.

This work will be structured as such: in Section 2 we present the current SOTA of the literature and all the works that pushed and inspired ours. In Section 3 we list and explain all the techniques and models that this work uses and presents. Later, in Section 4 we explain and analyze all the results we obtained. Section 5 explores the importance of each component in the proposed architecture by removing them and explaining why they are critical. In Section 6 we analyze and motivate why current SOTA might represent a liability for fraud detection, despite the good metrics and overall performances. Ultimately, Section 7 concludes the paper with some hints at some plausible future works.

2 Related Work↩︎

Research in fraud detection and anomaly identification has evolved along several key dimensions: data-level oversampling, deep generative modeling, and the development of interpretable or explainable classifiers [27]. Below, we review foundational and recent advances in these areas, with particular focus on techniques most relevant to the design of robust and interpretable fraud detection systems.

2.1 Data-Level Oversampling and Generative Models↩︎

Handling extreme class imbalance in fraud detection has long been addressed at the data‑level. Early work introduced SMOTE [28], with refinements and surveys over the years [29]. More recently, deep generative models, including VAEs, GANs, and hybrid approaches have been applied to rebalance fraud datasets. For instance, Tang et al. [30] proposed combining GANs and VAEs to simulate realistic transaction flows for anomaly detection, showing improved detection of rare fraud behaviors. Complementary surveys summarized the landscape of GAN‑based augmentation techniques for credit card fraud detection, highlighting a diversity of architectures and strategies [31]. Despite these advances, most generative methods focus on augmenting the minority class alone, which may lead to overconfident or overfit classifiers and limited cluster separation in latent space.

2.2 Prototypes, Attention, and Explainability↩︎

Beyond data augmentation, prototype-based networks provide intrinsic interpretability by comparing inputs to learned class exemplars. ProtoPNet [32] pioneered this for image classification, while subsequent work further quantified the visual or semantic attributes that drive similarity [33]. ProSeNet extended prototype reasoning to sequential data [34], but all such methods must guard against mismatches between prototypes and actual data features [35]. Attention mechanisms have likewise been adopted for per-feature weighting and have been proposed as explanation tools, though their reliability as explanations remains debated [36], [37]. Model-agnostic post-hoc methods such as LIME [38] and SHAP [39] also remain popular for explaining classifier decisions.

2.3 Recent Detection and Generative Oversampling Approaches↩︎

Recent years have witnessed a proliferation of generative oversampling strategies and detection methods for credit card fraud detection task, with most approaches focused on synthesizing minority class (fraud) data to address the severe class imbalance typical of this domain. Below, we summarize the methodological contributions and experimental setups of recent and representative works in this area. Rakhshaninejad et al. (2021) [40] propose an ensemble method that uses a weighted voting system algorithm to enforce and build more reliable classifiers for detecting frauds. Wang et al. (2022) [41] proposed the use of Unrolled Generative Adversarial Networks (Unrolled GAN) for the oversampling of fraudulent transactions. Their method, designed to overcome issues such as mode collapse in classical GANs, generates synthetic fraud samples to augment the minority class. The Unrolled GAN is explicitly trained only on the fraudulent samples, and the generated data is added to the original dataset before training downstream classifiers. Their experiments demonstrate that Unrolled GAN-based oversampling improves classification results over classical methods like SMOTE, highlighting the capacity of deep generative models to capture minority class distributions. Ding et al. (2023) [42] present a hybrid model that combines VAE with adversarial training (VAE-GAN) to generate synthetic fraud transactions. Their model learns the distribution of the minority class (fraud) and is trained solely on fraudulent examples, which are then used to augment the training set for downstream classifiers. Shi et al. (2025) [43] propose a class-imbalance-aware VAE with a transformer-based attention mechanism (Bal-VAE-Attention). Their model employs a loss function with class-aware weights to better learn from minority samples, and generates synthetic frauds for augmentation after training. Unlike earlier works, their results show that employing architectural or loss-based corrections can produce more robust synthetic samples and improved downstream detection rates. Ahmed et al. (2025) [44] propose a hybrid data sampling approach for credit card fraud detection by combining SMOTE with Edited Nearest Neighbors (ENN) [45]. Their method, evaluated on the Kaggle credit card dataset, demonstrates that this hybrid technique can significantly enhance the performance of ensemble models (including RF, KNN, and AdaBoost) and a voting ensemble. By first oversampling the minority class and then using ENN to remove noisy samples, they achieve high scores in accuracy, precision, recall, F1, and AUC, outperforming many traditional oversamplers and showing that careful data balancing is crucial for robust fraud detection. Overall, the prevailing trend in recent literature is to leverage generative models, typically trained only on minority class data, as advanced oversamplers.

2.4 Gaps: Classifier-Guided Latent Shaping↩︎

While the integration of auxiliary classifiers within generative frameworks has been explored to improve training stability and class-conditional sample quality [46], [47], existing approaches primarily treat the classifier as a black-box supervisory signal. In auxiliary-classifier GANs, the classifier loss is typically used to encourage the generator to produce samples that are easily recognizable as belonging to a target class, without explicitly constraining the structure of the latent space or the geometry of the generated data manifold. As a result, these methods focus on class-conditional realism rather than on shaping representations that are discriminative with respect to downstream decision boundaries. Moreover, prior GAN-based oversampling strategies are generally trained exclusively on minority-class data or generate samples conditioned only on the class label, which risks reinforcing local density without accounting for the global relationship between minority and majority classes. This can lead to synthetic samples that appear plausible in isolation but do not meaningfully improve class separation when evaluated against the full data distribution. In contrast, our approach integrates an interpretable classification head, the Causal Prototype Attention Classifier (CPAC), directly into the latent space of a VAE-GAN. Unlike standard auxiliary classifiers, CPAC does not merely provide a label-based loss to the generator. Instead, it actively structures the latent manifold through two explicit mechanisms: Prototype Anchoring, which aligns class-specific prototypes with the centroids of latent clusters, and a Causal Attention mechanism that assigns differential importance to latent dimensions based on their contribution to discrimination. This design induces a bidirectional interaction between representation learning and sample generation: the generator is guided not only to produce class-consistent samples, but to position them in regions of the latent space that clarify the separation between fraud and non-fraud classes. By embedding discriminative structure directly into the generative process, the proposed VAE-GAN+CPAC framework goes beyond auxiliary-classifier supervision and yields synthetic samples that are both realistic and decision-aware, addressing a key limitation of prior classifier-guided GAN-based oversampling methods.

3 Methodologies↩︎

This section introduces the proposed CPAC model and evaluates its performance against standard classifiers. We also describe the proposed classifier-guided latent shaping strategy and analyze its effect on latent space structure and class separability. Then we will introduce how we used a classifier to influence and help the VAE-GAN latent space, shape a better representation of the two classes. Ultimately, it will be shown how the CPAC classifier might be a better fit for a classification head due to its nature and its structure and its results will be compared to other classification heads.

3.1 Dataset and Preprocessing↩︎

We conduct our experiments on the publicly available Credit Card Fraud Detection dataset, hosted on Kaggle [26]. This dataset, released by Worldline and the Machine Learning Group of ULB, contains 284,807 anonymized transactions made by European cardholders in September 2013. Only 492 of these are labeled as fraudulent, resulting in an extreme class imbalance (approximately 0.17% fraud rate). Each transaction includes 30 features, where 28 have been transformed via principal component analysis (PCA) to protect confidentiality. The remaining two features are the Time and Amount. The target variable Class is binary, with 1 indicating fraud and 0 otherwise. To ensure each feature contributes uniformly to model training, we apply robust normalization to all input features. Specifically, for each feature \(x\), we compute its median \(\tilde{x}\) and interquartile range \(\text{IQR}(x) = Q_3(x) - Q_1(x)\), and normalize using Equation 1 : \[\label{eq:iqr} \begin{align} x_{\text{norm}} = \frac{x - \tilde{x}}{\text{IQR}(x)} \end{align}\tag{1}\] This transformation centers features around zero and scales them while remaining robust to outliers, a crucial property in fraud detection where anomalies naturally exhibit extreme values. For features where \(\text{IQR}(x) = 0\), we default to a unit divisor to prevent numerical instability. We then split the dataset into 80% training, 10% validation, and 10% test sets as shown in Table 1. Stratified sampling is used to preserve the class distribution across splits. Importantly, the splits are strictly disjoint: each transaction is assigned to exactly one subset, and no samples or derived information are shared across training, validation, and test sets, preventing any form of data leakage between partitions.

3.2 Oversampling Strategies↩︎

To address the extreme class imbalance in credit card fraud detection, we implemented and compared two oversampling strategies: SMOTE and a custom VAE–GAN pipeline. Both techniques were used to synthetically augment the minority (fraudulent) class, and the generated samples were added only to the training set, leaving the evaluation and test sets only with pure transactional data, emulating a real-life deploy scenario. We chose, 50, 75, 100 samples to generate for two main reasons: the first one is that using a higher number of samples could lead classifiers to overfit, especially if the number of generated frauds is higher than the original number in the dataset and the synthetic data is not of high quality. The second reason is that most of times, despite the increasing number of frauds in the training set, the models inevitably plateau as we will see in the next section.

3.2.1 SMOTE-Based Oversampling↩︎

The Synthetic Minority Over-sampling Technique (SMOTE) is a widely-used baseline for addressing class imbalance. Rather than learning the data distribution, SMOTE interpolates directly between existing minority-class samples. For two fraud instances \(\mathbf{x}_i\) and \(\mathbf{x}_j\), it generates a synthetic sample \(\tilde{\mathbf{x}}\) along the line connecting them (Equation 2 ): \[\label{eq:smote} \tilde{\mathbf{x}} = \mathbf{x}_i + \alpha \cdot (\mathbf{x}_j - \mathbf{x}_i), \qquad \alpha \sim \mathcal{U}(0, 1),\tag{2}\] where \(\alpha\) is uniformly sampled. This process, repeated with each sample’s \(k\) nearest neighbors, creates new minority points distributed across the feature space. In our experiments, we generated 50, 75, and 100 synthetic frauds using SMOTE and merged them into the training set. While SMOTE is simple and effective, it can produce overly smooth or redundant samples, especially when the minority class has complex or non-linear structure. As shown in Figure 1, SMOTE’s interpolated samples often “connect the dots” between real fraud clusters, potentially resulting in synthetic points that are too similar to the originals. Despite remaining a strong baseline, SMOTE can be outperformed by generative models that better capture the underlying data distribution in highly imbalanced settings.

3.2.2 VAE–GAN Oversampling↩︎

The Variational Autoencoder-Generative Adversarial Network (VAE-GAN) has emerged as a powerful approach for generating synthetic data in imbalanced classification problems (its structure is visible in Figure 2). In accordance with prevailing practice in fraud detection, we employ the VAE-GAN exclusively as a minority-class oversampler: it is trained using only genuine fraud transactions, then used to synthesize new fraud-like samples that supplement the training set. The VAE-GAN consists of three neural modules: an encoder \(E_\phi\), a decoder \(D_\theta\), and a discriminator \(C_\psi\). The encoder maps an input \(\mathbf{x} \in \mathbb{R}^d\) through multiple hidden layers to the parameters of a multivariate Gaussian: mean vector \(\boldsymbol{\mu}\) and log-variance \(\log\boldsymbol{\sigma}^2\). Using the reparameterization trick, the latent code (Equation 3 ) is sampled as \[\label{eq:latent} \mathbf{z} = \boldsymbol{\mu} + \boldsymbol{\sigma} \odot \boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(0, I).\tag{3}\] The decoder \(D_\theta\) reconstructs the input from \(\mathbf{z}\), while the discriminator \(C_\psi\) distinguishes real from reconstructed samples. Training proceeds by minimizing a weighted sum of three losses:

• The VAE loss (Equation 4 ), which includes both a reconstruction error and a Kullback–Leibler divergence term: \[\begin{align} \label{eq:vae95loss} \mathcal{L}_{\mathrm{VAE}} = \mathbb{E}_{q_\phi(\mathbf{z}|\mathbf{x})}\!\left[\|\mathbf{x} - D_\theta(\mathbf{z})\|_2^2\right] + \beta \cdot \mathrm{KL}\!\left(q_\phi(\mathbf{z}|\mathbf{x})\,\|\,p(\mathbf{z})\right), \end{align}\tag{4}\] where \(p(\mathbf{z})\) is the standard normal prior and \(\beta\) controls the KL penalty.

• The GAN loss (Equation 5 ) for the discriminator, encouraging \(C_\psi\) to distinguish real fraud samples from reconstructions: \[\label{eq:gan95loss} \mathcal{L}_{\mathrm{GAN}} = -\mathbb{E}_{\mathbf{x} \sim p_{\mathrm{data}}}\!\left[\log C_\psi(\mathbf{x})\right] - \mathbb{E}_{\hat{\mathbf{x}} \sim D_\theta(\mathbf{z})}\!\left[\log(1 - C_\psi(\hat{\mathbf{x}}))\right].\tag{5}\]

• The generator adversarial loss (Equation 6 ), which pushes the decoder to generate samples that the discriminator cannot distinguish from real frauds: \[\label{eq:adv95loss} \mathcal{L}_{\mathrm{Adv}} = -\mathbb{E}_{\hat{\mathbf{x}} \sim D_\theta(\mathbf{z})}\!\left[\log C_\psi(\hat{\mathbf{x}})\right].\tag{6}\]

During training, the encoder and decoder are optimized together to minimize both reconstruction and adversarial losses, while the discriminator is trained to distinguish real from generated samples. Early stopping and validation are used to ensure generalization. After training, the decoder generates new fraud samples by sampling from the learned latent distribution, augmenting the training set for subsequent classification. This VAE-GAN-based oversampling is widely used for its ability to produce more realistic and varied fraud examples than simpler interpolation techniques like SMOTE. In our experiments, we generated and merged 50, 75 and 100 synthetic fraud samples into the training data. However, as illustrated in Figure 3, such oversampling often concentrates the synthetic data in a narrow region of latent space, which can make downstream classifiers either overconfident or poorly calibrated, and limits cluster separation. This limitation provides the motivation for our classifier-guided methods.

3.2.3 SMOTE vs VAE-GAN Oversampling↩︎

The fundamental difference between SMOTE and VAE–GAN oversampling lies in how synthetic minority-class examples are generated and distributed. SMOTE creates new samples by interpolating between real frauds and their nearest neighbors, filling gaps and uniformly spreading synthetic points within known clusters. While easy to use and computationally light, SMOTE may also generate borderline samples that overlap with the majority class. In contrast, VAE–GAN learns a latent representation and generates new frauds by sampling from a global latent prior, which often leads to denser clustering around the main fraud mode but can miss peripheral fraud patterns. VAE–GAN models require more careful design and tuning, but can create more realistic, nonlinear samples that better reflect complex feature relationships. Therefore, SMOTE is suited for quick and broad coverage of known minority regions, whereas VAE–GAN offers more expressive synthetic data at the cost of increased complexity and training effort. The choice ultimately depends on the application’s requirements and available resources.

3.3 Baseline Classifiers↩︎

To benchmark the performance of our generative and prototype-based approaches, we evaluated three standard classifiers widely used in fraud detection: Logistic Regression, Random Forest, KNN and XGBoost. These models provide strong baselines due to their interpretability, ensemble nature, and ability to handle imbalanced data with appropriate modifications. To assess the performance of all classifiers in this work, we report four standard metrics: precision, recall, F1-score, and AUC-ROC. Each metric captures a different aspect of performance for highly imbalanced classification problems such as fraud detection.

3.4 Causal Prototype Attention Classifier (CPAC)↩︎

The Causal Prototype Attention Classifier (CPAC) embeds interpretable, class‐aware structure directly into a lightweight neural module. Given an input \(\mathbf{x}\in\mathbb{R}^d\), CPAC learns two prototype vectors (Equation 7 )

\[\label{eq:protss} \begin{align} \mathbf{p}_0,\;\mathbf{p}_1\;\in\;\mathbb{R}^d \end{align}\tag{7}\]

representing the centroids of the non‐fraud and fraud classes. An attention network (Equation 8 )

\[\label{eq:att} \begin{align} \mathbf{w} \;=\; \mathrm{Att}(\mathbf{x}) \;=\;\sigma\bigl(W_2\,\mathrm{ReLU}(W_1\mathbf{x}+b_1)+b_2\bigr) \;\;\in(0,1)^d \end{align}\tag{8}\]

where \(\mathbf{x} \in \mathbb{R}^d\) is the latent input, \(W_1\) and \(W_2\) are learnable weight matrices, \(b_1\) and \(b_2\) are biases, \(\mathrm{ReLU}\) is the rectified linear unit, and \(\sigma\) is the sigmoid function. The output \(\mathbf{w} \in (0,1)^d\) is a per-feature attention mask. It produces a per‐feature mask highlighting dimensions most predictive of fraud. A learnable scale \(\alpha>0\) adjusts sensitivity, and we compute weighted squared distances (Equation 9 ).

\[\label{eq:weight95dis} \begin{align} d_c(\mathbf{x}) \;=\; \alpha\,\sum_{i=1}^d w_i\,\bigl(x_i - p_{c,i}\bigr)^2 \quad,\;c\in\{0,1\}. \end{align}\tag{9}\]

where \(w_i\) is the \(i\)-th element of the attention vector \(\mathbf{w}\), \(x_i\) is the \(i\)-th feature of the latent vector \(\mathbf{x}\), \(p_{c,i}\) is the \(i\)-th coordinate of the prototype for class \(c\) (\(c=0\) for non-fraud, \(c=1\) for fraud), \(\alpha\) is a learnable scaling factor, and \(d\) is the latent dimensionality.

Interpreting negative distances as logits (Equation 10 ),

\[\label{eq:neg95dis} \begin{align} \boldsymbol{\ell}(\mathbf{x}) =\begin{bmatrix}-d_0(\mathbf{x})\\-d_1(\mathbf{x})\end{bmatrix} \quad\Longrightarrow\quad \hat{y} \;=\;\mathrm{softmax}\bigl(\boldsymbol{\ell}(\mathbf{x})\bigr)_1, \end{align}\tag{10}\]

where \(d_0(\mathbf{x})\) and \(d_1(\mathbf{x})\) are the weighted distances to the non-fraud and fraud prototypes, respectively, \(\boldsymbol{\ell}(\mathbf{x})\) is the vector of negative distances used as logits, and \(\mathrm{softmax}(\cdot)_1\) denotes the softmax probability for class 1 (fraud). This yields the predicted fraud probability \(\hat{y}\in(0,1)\).

The term “causal” here is used loosely to suggest that the attention weights may help highlight which latent features have a higher impact on the outcome. Its structure can be visualised in Figure 4.

3.4.1 Training Loss↩︎

To counter the extreme imbalance, we adopt the Focal Loss [48], which adds two hyperparameters (Equation 11 ), \(\alpha_{\mathrm{FL}}\) and \(\gamma\) to the standard binary cross‐entropy:

\[\begin{align} \mathcal{L}_{\mathrm{FL}}(y,\hat{y}) =\; & -\,\alpha_{\mathrm{FL}}\,(1-\hat{y})^{\gamma}\,y\,\log\hat{y} \notag \\ & -\; (1-\alpha_{\mathrm{FL}})\,\hat{y}^{\gamma}\,(1-y)\,\log(1-\hat{y}) \label{eq:focal} \end{align}\tag{11}\]

Where:

• \(\alpha_{\mathrm{FL}}\in[0,1]\) balances the importance of the two classes. We set \(\alpha_{\mathrm{FL}}=0.95\) to give more weight to the minority (fraud) class.

• \(\gamma\ge0\) is the focusing parameter. When \(\gamma=0\), \(\mathcal{L}_{\mathrm{FL}}\) reduces to ordinary cross‐entropy. As \(\gamma\) grows, well-classified examples (where \(\hat{y}\) is close to the true label) incur much smaller loss, forcing the model to concentrate on harder, often minority‐class cases. We performed a grid search for the most optimal values for \(\alpha_{\mathrm{FL}}\) and \(\gamma\) and we found out that the best results are obtained with 0.95 and 2.0 respectively.

By tuning \(\alpha_{\mathrm{FL}}\) and \(\gamma\), Focal Loss both re‐weights the underrepresented class and focuses learning on its most difficult examples (Alg. 5), which is crucial in fraud detection. Model selection uses a composite score (Equation 12 ):

\[\label{eq:comp95score} \begin{align} S \;=\; 0.50\,\mathrm{Precision}\;+\;0.50\,\mathrm{Recall} \end{align}\tag{12}\]

on the validation set. We checkpoint the CPAC weights whenever \(S\) improves, and invoke early‐stopping after \(p\) epochs without gain.

3.5 Adaptive Threshold Selection via Differentiable Agent↩︎

To improve classification under class imbalance, we introduce a differentiable agent that adaptively learns the optimal classification threshold \(\tau^*\) for maximizing the F1 score. Instead of using the default \(\tau = 0.5\), we parameterize the threshold as a scalar \(\theta\), mapping it to \(\tau = \sigma(\theta)\) (where \(\sigma\) is the sigmoid function) so that \(\tau\) lies in (0,1). For each sample \(i\), with predicted probability \(p_i\) and ground truth \(y_i \in \{0,1\}\), the agent approximates a hard threshold with a smooth sigmoid function (Equation 13 ):

\[\label{eq:bin95dec} \begin{align} \hat{y}_i = \sigma\bigl(\beta(p_i - \tau)\bigr), \end{align}\tag{13}\]

where \(\hat{y}_i\) is the soft binary prediction for sample \(i\), \(p_i\) is the predicted probability, \(\tau\) is the learned threshold, and \(\beta\) controls the sharpness of the sigmoid (making the function step-like). The loss for optimizing the threshold is given by Equation 14 :

\[\label{eq:dummy95loss} \begin{align} \mathcal{L} = \frac{1}{N} \sum_{i=1}^N \left(\hat{y}_i - y_i\right)^2, \end{align}\tag{14}\]

where \(N\) is the batch size, \(\hat{y}_i\) is the soft prediction, and \(y_i\) is the true binary label. By minimizing this loss via gradient descent and monitoring validation F1, the agent effectively learns an optimal threshold suited to the classifier’s output probabilities. This method consistently yielded improved F1 scores, often with a learned threshold above 0.7, especially when paired with classifiers such as CPAC or those using VAE–GAN encoders.

3.6 VAE-GAN with Classification Heads↩︎

Most state-of-the-art techniques [41]–[43], train the generative model on only the fraud (minority) class. This classical approach has a crucial limitation: it fails to expose the model to the characteristics of non-fraudulent (majority) data, resulting in a generator that can merely interpolate among known frauds, often producing synthetic samples that are near-duplicates, lacking true discriminative power. The latent space learned in such a setup is inevitably narrow and uninformative about what actually distinguishes fraud from normality. In contrast, our approach is fundamentally different. We jointly train the VAE-GAN with a classification head on the full dataset, even though the generative (VAE-GAN) component is optimized only on the minority class. The critical distinction is that the encoder, which is shared between the generator and the classifier head, receives supervised feedback from both classes via the classification objective. This means that the latent space, as learned by the encoder, encodes information about the entire data distribution,both fraud and non-fraud. This approach fundamentally differs from conventional pipelines in which the generative model is trained exclusively on the minority class. The VAE-GAN and classifier pipeline is fully aware of both classes, since the encoder’s parameters are influenced by the generative loss applied to fraud samples and the classification loss computed over the entire dataset. Focusing the generative loss on the minority class is a deliberate choice that enables the decoder to specialize in high-fidelity, targeted synthesis of frauds. At the same time, the classifier head continuously regularizes the encoder, ensuring that it organizes the latent space to distinguish between both fraud and non-fraud classes. As a result, the latent space does not lose information about the normal class; rather, it is shaped to maximize class discrimination. The encoder integrates the objectives of both components, while the decoder utilizes this enriched representation to generate synthetic frauds that are not only realistic but also truly distinct within the broader data context. Thus, although the generative focus is on the minority class, the overall supervision ensures that the synthetic samples are robust, generalizable, and valuable for distinguishing between classes.

3.6.1 Classifier Heads (MLP Variants)↩︎

We consider three multilayer perceptron (MLP) heads, as shown in Figure 6, of increasing complexity, each implementing a function \(h_\theta: \mathbb{R}^d \to [0,1]\), with \(d=2\) in our experiments:

1. MLPHead1: One hidden layer with 32 units (ReLU), output through a sigmoid (Equation 15 ): \[\label{eq:mlp1} \begin{align} h_1(z) = \sigma \left( W_2\, \mathrm{ReLU}(W_1 z + b_1) + b_2 \right) \end{align}\tag{15}\] where \(W_1 \in \mathbb{R}^{32 \times 2}\), \(b_1 \in \mathbb{R}^{32}\), \(W_2 \in \mathbb{R}^{1 \times 32}\), \(b_2 \in \mathbb{R}\).

2. MLPHead2: One hidden layer with 64 units, batch normalization, dropout (\(p=0.2\)), ReLU and sigmoid (Equation 16 ): \[\label{eq:mlp2} \begin{align} h_2(z) = \sigma \left( W_2\, \mathrm{Dropout}\big(\mathrm{ReLU}(\mathrm{BN}(W_1 z + b_1))\big) + b_2 \right) \end{align}\tag{16}\] with \(W_1 \in \mathbb{R}^{64 \times 2}\), \(b_1 \in \mathbb{R}^{64}\), \(W_2 \in \mathbb{R}^{1 \times 64}\), \(b_2 \in \mathbb{R}\).

3. MLPHead3: Two hidden layers with 128 and 64 units (ReLU), output through a sigmoid (Equation 17 ): \[\label{eq:mlp3} \begin{align} h_3(z) = \sigma \left( W_3\, \mathrm{ReLU}(W_2\, \mathrm{ReLU}(W_1 z + b_1) + b_2) + b_3 \right) \end{align}\tag{17}\] where \(W_1 \in \mathbb{R}^{128 \times 2}\), \(b_1 \in \mathbb{R}^{128}\), \(W_2 \in \mathbb{R}^{64 \times 128}\), \(b_2 \in \mathbb{R}^{64}\), \(W_3 \in \mathbb{R}^{1 \times 64}\), \(b_3 \in \mathbb{R}\).

These structures will all be tested and evaluated as potential heads for our encoder.

3.6.2 Joint Training Procedure↩︎

The model is trained in two coordinated phases at each epoch:

1. VAE-GAN update: For each mini-batch (from all classes), the encoder maps \(x\) to \(z, \mu, \log\sigma^2\), and the decoder reconstructs \(x_{\text{rec}}\). The discriminator receives both \(x\) and \(x_{\text{rec}}\) and tries to distinguish real from generated data. The VAE-GAN is trained with the following loss (Equation 18 ): \[\label{eq:vaegan95mlp} \begin{align} \mathcal{L}_\mathrm{VAE-GAN} = \mathcal{L}_\mathrm{recon} + \mathcal{L}_\mathrm{KL} + \mathcal{L}_\mathrm{GAN} \end{align}\tag{18}\] where \(\mathcal{L}_\mathrm{recon}\) is MSE between \(x\) and \(x_{\text{rec}}\), \(\mathcal{L}_\mathrm{KL}\) is the KL divergence, and \(\mathcal{L}_\mathrm{GAN}\) is the adversarial loss for the generator decoder.

2. Classifier head update: Using the same mini-batch, the encoder’s mean \(\mu\) is passed to the classifier head \(h_\theta\) to predict the label \(y\). The head is trained with binary cross-entropy (Equation 19 ): \[\label{eq:bce} \begin{align} \mathcal{L}_\mathrm{clf}(y, \hat{y}) = -y \log \hat{y} - (1-y)\log(1-\hat{y}), \quad \hat{y} = h_\theta(\mu) \end{align}\tag{19}\] The gradient of \(\mathcal{L}_\mathrm{clf}\) flows back not only to the classifier head parameters \(\theta\), but also to the encoder parameters. As a result, the encoder is explicitly encouraged to organize the latent means \(\mu\) so that different classes become more separable for the downstream classifier. This joint training guides the encoder to learn a latent representation where fraud and non-fraud samples are more easily discriminated.

The pseudo-code of the algorithm can be observed in Alg. 8. Despite all three experiments with all the MLPs (as shown in Figures 7 (a), 7 (b), 7 (c)) clearly suggest at a slight cluster separation, the overlap between the two classes is still significant. The first and third MLPs produces similar results despite the different structures while the second MLP is the only one leaning into a polynomial decision boundary that could help separating the two clusters. These results prove that we need a more powerful network capable of adapting into the encoder logic and working on a non-linear and more complex space. Due to the substantial overlap between the two classes, using this latent representation to generate additional fraud samples degrades the performance of downstream classifiers.

3.7 VAE-GAN with CPAC Head↩︎

Similarly to how we did with the MLPs, we tested how the CPAC behaves as a classification head paired with the encoder. With the MLPs, there was a slight tendency to inner cluster separation in the encoder but the overlap was still significant. CPAC’s structure, is very akin with the encoder, because its prototypes might offer an anchor point where the centroid of each cluster might find its position pushing for a better separation. The training is similar to what has already been done with the MLPs.

3.7.1 Joint Training Procedure↩︎

At each training epoch, we alternate between:

1. VAE-GAN training: For each batch, the encoder \(E_\phi\) encodes \(x\) to \((z, \mu, \log\sigma^2)\), the decoder \(G_\psi\) reconstructs \(x_\text{rec} = G_\psi(z)\), and the discriminator \(D_\omega\) distinguishes \(x\) from \(x_\text{rec}\). The standard VAE-GAN loss (Equation 20 ) is: \[\label{eq:vaegan95cpac} \begin{align} \mathcal{L}_\text{VAE-GAN} = \mathcal{L}_\text{recon}(x, x_\text{rec}) + \mathcal{L}_\text{KL}(\mu, \log\sigma^2) + \mathcal{L}_\text{GAN}(D_\omega(x_\text{rec}), 1) \end{align}\tag{20}\] where \(\mathcal{L}_\text{recon}\) is mean squared error (MSE), \(\mathcal{L}_\text{KL}\) is the KL divergence, and \(\mathcal{L}_\text{GAN}\) is the adversarial loss for the generator.

2. CPAC update: In the same batch, the encoder’s mean \(\mu\) is passed to the CPAC head, which computes distances (Equation 21 ) to two learnable prototypes (\(\mathbf{p}_0\), \(\mathbf{p}_1\)) via feature-wise attention weights \(\mathbf{w}\): \[\label{eq:dist} \begin{align} d_c = \alpha \sum_{j=1}^d w_j \left(\mu_j - (\mathbf{p}_c)_j\right)^2, \qquad c \in \{0, 1\} \end{align}\tag{21}\] where \(w_j \in (0,1)\) are attention weights (from a neural branch), and \(\alpha\) is a learnable scaling parameter. Fraud probability is given by Equation 22 : \[\label{eq:fr95probs} \begin{align} \hat{y} = \mathrm{Softmax}(-d_0, -d_1) \end{align}\tag{22}\] The CPAC is trained to minimize the BCE loss. To further regularize learning, two penalties are added:

   • Scale penalty: encourages the attention scaling parameter to stay bounded (Equation 23 ): \[\label{eq:scale} \begin{align} \mathcal{L}_\text{scale} = \lambda_\text{scale} \cdot \|\alpha\|^2 \end{align}\tag{23}\]

   • Prototype anchoring: aligns each prototype to the centroid of its class in latent space, encouraging the encoder to cluster samples around the correct prototype (Equation 24 ): \[\label{eq:proto} \begin{align} \mathcal{L}_\text{anchor} = \lambda_\text{anchor} \left( \|\mathbf{p}_0 - \bar{\mu}_0\|^2 + \|\mathbf{p}_1 - \bar{\mu}_1\|^2 \right) \end{align}\tag{24}\] where \(\bar{\mu}_0\), \(\bar{\mu}_1\) are the means of latent vectors in the current batch with \(y=0\) and \(y=1\) respectively.

The total loss (Equation 25 ) for CPAC becomes: \[\label{eq:tot95loss} \begin{align} \mathcal{L}_\text{CPAC-total} = \mathcal{L}_\text{clf} + \mathcal{L}_\text{scale} + \mathcal{L}_\text{anchor} \end{align}\tag{25}\] and the overall optimization alternates VAE-GAN and CPAC updates. Just like we did for the Focal Loss we performed a grid-search and found the best \(\lambda_\text{scale}\) and \(\lambda_\text{anchor}\) to be respectively 0.001 and 0.01. When using CPAC as a classification head within our VAE-GAN architecture, we switch to BCE loss. Just like the MLPs, the gradient of the CPAC loss gets backpropagated to both the CPAC parameters and also to the encoder parameter. After each epoch, we evaluate the CPAC on a held-out validation set. Early stopping is applied based on recall, conditional on maintaining a minimum precision, as in other head experiments. The full joint optimization procedure of the VAE-GAN with CPAC head is summarized in Algorithm 9.

4 Experimental Results↩︎

Given the proposed methodologies, we delve into all the experiments with the models and methods listed in Section 3. First we test the CPAC against the other four classifiers and the results obtained by other works. In all experiments in this section, we use \(k=5\) neighbors for KNN, as this configuration consistently yielded the best performance across all tests. We then analyze and compare the results obtained with the VAE-GAN equipped with a classification head against other oversampling techniques. All reported metrics are computed on the held-out test set.

4.1 Preliminary Results↩︎

Unlike MLP heads, the CPAC’s architecture and losses (especially prototype anchoring) push the encoder to organize latent means \(\mu\) into two tight, well-separated clusters, each surrounding its prototype. The attention weights \(\mathbf{w}\) add interpretability, highlighting which latent features are most influential for distinguishing frauds. This process results in a latent space that is both discriminative and interpretable. As we can see on Figure 10 (a) the CPAC was able to improve separation creating two distinct clusters with each prototype being anchored to its cluster centroid, essentially acting like one. A limited number of misclassifications is observed, which is expected in highly imbalanced scenarios, together with a small residual overlap between the two classes (visible in Figure 10 (b)) which is caused by borderline transactions. This is the first step to a more aware encoder that knows what actually constitutes a fraud.

4.2 Oversampling results and experiments↩︎

Now, we will cover the results obtained with the two state-of-the-art oversampling strategies [20], [21] applied to Logistic Regression, Random Forest, XGBoost and CPAC. First they will be tested without oversampling, to asses how oversampling can aid overall performances; then they will be tested with SMOTE oversampling and VAE-GAN.

4.2.1 No Oversampling↩︎

All models are first evaluated on the original, non-augmented dataset to establish a baseline, with results reported in Table 2. Despite the strong class imbalance, all methods achieve solid performance, indicating that the dataset already provides informative signals for fraud detection. Logistic Regression shows balanced Precision and Recall but is slightly outperformed by more expressive models, reflecting its limited ability to capture complex decision boundaries. Random Forest attains the highest Precision at the expense of Recall, suggesting a conservative classification behavior. XGBoost achieves a strong balance between Precision and Recall, resulting in high F1-score and AUC-ROC values, and confirming its robustness on the original data distribution. KNN also performs competitively, though with a lower AUC-ROC, indicating weaker global ranking performance. The CPAC classifier demonstrates well-balanced results, achieving high Recall and the highest AUC-ROC among all models. This suggests that its prototype-based structure effectively captures meaningful latent representations even without oversampling. While CPAC performs comparably to tree-based ensembles in this setting, it is likely to further benefit from enriched training data, where additional minority samples could help stabilize prototypes and refine decision boundaries.

4.2.2 Single-Sample Inference Time↩︎

Inference efficiency is a critical requirement in real-world credit card fraud detection systems, where models are often deployed in latency-sensitive environments. Table 3 reports the average single-sample inference time measured on the test set for the different classifiers considered in this study, all trained without oversampling. As expected, linear and tree-based models such as Logistic Regression and XGBoost exhibit extremely low inference latency, making them well suited for high-throughput real-time scoring. More instance-based methods such as KNN incur higher inference costs due to their reliance on neighborhood search at prediction time. The proposed CPAC model introduces a moderate computational overhead compared to traditional classifiers, while remaining significantly faster than KNN and well within practical latency constraints. Importantly, CPAC is not intended to replace lightweight production scorers in isolation. Its primary role is to act as a classifier-guided regularizer during training, shaping the latent space of the generative model through prototype anchoring and attention-weighted feature interactions. This latent structuring improves class separability, robustness, and downstream detection performance under extreme class imbalance. In practical deployment scenarios, CPAC can therefore be used either offline or within hybrid pipelines, while final real-time decisions may still rely on simpler classifiers operating on the improved representations. Overall, these results indicate that the proposed approach achieves a favorable trade-off between representational benefits and computational cost, supporting its applicability in realistic fraud detection settings.

4.2.3 SMOTE Oversampling Performances↩︎

Applying SMOTE oversampling with 50, 75, and 100 synthetic fraud samples leads to consistent performance changes across all classifiers, as reported in Table 4. While Random Forest achieves very strong Precision and F1-score, particularly at higher oversampling levels, its performance varies noticeably across different augmentation sizes. This behavior suggests that the model is highly sensitive to the specific structure induced by the synthetic samples during training, resulting in decision boundaries that do not generalize consistently to the real, unseen test data. In contrast, XGBoost demonstrates the most reliable behavior under SMOTE augmentation, maintaining a stable balance between Precision and Recall and achieving consistently high F1-score and AUC-ROC across all oversampling levels. This stability highlights XGBoost’s robustness to class imbalance and its ability to exploit synthetic augmentation without excessively distorting the learned decision function. Logistic Regression benefits modestly from oversampling, with gradual improvements in Recall and F1-score, but remains constrained by its linear decision boundaries. CPAC shows competitive performance at lower oversampling levels, particularly in terms of AUC-ROC, but its metrics degrade as the number of synthetic samples increases, indicating sensitivity to changes in the training data distribution. Overall, these results indicate that while SMOTE oversampling can enhance performance, especially for ensemble methods.

4.2.4 VAE–GAN Oversampling Performances↩︎

VAE-GAN-based oversampling with 50, 75, and 100 synthetic fraud samples leads to consistent performance improvements across most classifiers, as reported in Table 5. Compared to SMOTE, the VAE-GAN generates higher-quality synthetic samples that more effectively support discriminative learning, particularly for ensemble models. Random Forest achieves very strong Precision and F1-score across all oversampling levels; however, its performance varies with the number of synthetic samples, indicating a higher sensitivity to training-time distribution changes. In contrast, XGBoost exhibits the most reliable behavior, maintaining a stable balance between Precision and Recall and achieving consistently high F1-score and AUC-ROC across all configurations. This stability confirms XGBoost’s ability to exploit generative oversampling while preserving generalization to real test data. KNN shows largely unchanged performance across different oversampling levels, mirroring the behavior observed with SMOTE. As an instance-based method, KNN remains primarily influenced by local neighborhood structure, and the additional synthetic samples, while plausible,do not substantially alter the nearest-neighbor geometry governing its predictions. CPAC demonstrates competitive performance, particularly in terms of AUC-ROC, but its Recall and F1-score degrade as the number of synthetic samples increases, suggesting sensitivity to shifts in the latent data distribution. Logistic Regression continues to benefit only marginally from oversampling due to its limited representational capacity. Overall, these results indicate that VAE-GAN oversampling provides more informative synthetic data than SMOTE, but its benefits are best realized by models with strong regularization and margin-aware learning.

4.2.5 SMOTE vs VAE-GAN Results Observations↩︎

Comparative analysis shows that VAE–GAN-generated synthetic frauds provide greater improvements in recall and F1-score than SMOTE, particularly for XGBoost and CPAC. SMOTE easily fills gaps within clusters but can generate less realistic points at class boundaries. VAE–GAN, while more complex, generates realistic samples that support better model generalization. When paired with VAE–GAN, CPAC shows pronounced gains in recall and AUC-ROC, especially with more synthetic samples. However, excessive oversampling can lead to plateauing or overfitting, highlighting the need to find an optimal level of augmentation.

4.3 VAE-GAN with CPAC Head as Oversampler↩︎

Table 6 shows the results obtained using the VAE-GAN+CPAC to generate synthetic fraud samples. Compared to previous oversampling strategies, performance gains are more moderate and quickly plateau as the number of synthetic samples increases, suggesting that the generated data closely matches the real fraud distribution and avoids aggressive overfitting. Logistic Regression benefits only marginally from this oversampling strategy, while Random Forest achieves strong Precision but exhibits limited performance variation across oversampling levels. XGBoost demonstrates the most reliable behavior, maintaining a stable balance between Precision and Recall and achieving consistently high F1-score and AUC-ROC across all configurations. KNN remains largely unaffected due to its instance-based nature. Overall, the VAE-GAN+CPAC oversampler favors stable and generalizable learning, with XGBoost emerging as the most suitable downstream classifier.

4.3.1 Applying Oversampling before VAE-GAN+CPAC Training↩︎

To further improve latent space separation in the VAE-GAN+CPAC framework, we explored the effect of applying a slight oversampling to the fraud class before pipeline training. Specifically, we employed SMOTE, as it produces a broader spread of synthetic fraud samples across the cluster, offering a better foundation for the encoder to learn generalizable boundaries. Although this choice may appear counter-intuitive in light of our previous discussion on minority-only oversampling, in this case we need a slight "accentuation" of the frauds samples defined in the overlap in order to allow the model to detect them better and further reduce the overlap. Pre-training augmentation with VAE-GAN tended to generate samples in a narrower region of the minority class, providing less diversity and thus limited improvement in latent separation. Guided by previous experiments, we selected 75 SMOTE-generated samples as the optimal amount, since further augmentation did not yield additional gains (see Figure 12). This approach visibly reduced cluster overlap in the latent space (Figures 11 (a), 11 (b)) and improved overall model performance. Thus, SMOTE-based pre-training oversampling proved more effective than VAE-GAN augmentation in this context, with 75 synthetic samples offering the best balance between diversity and separation.

4.3.2 Results Obtained with Pre-Training Oversampled VAE-GAN+CPAC↩︎

Table 7 reports the results obtained when the VAE-GAN+CPAC pipeline is trained on a pre-oversampled dataset. Compared to the standard VAE-GAN+CPAC configuration, this strategy yields clearer and more consistent performance gains, particularly for XGBoost, which achieves the highest F1-score and AUC-ROC across all oversampling levels. Random Forest continues to obtain very strong Precision and competitive F1-scores, but its performance varies with the number of synthetic samples. In contrast, XGBoost demonstrates superior stability, maintaining a well-balanced Precision-Recall trade-off and consistently high AUC-ROC, indicating improved generalization to real test data. Logistic Regression benefits only marginally, while KNN remains largely unaffected. Overall, pre-training oversampling further enhances the effectiveness of the VAE-GAN+CPAC pipeline by producing cleaner latent representations that are best exploited by robust, margin-aware classifiers, with XGBoost emerging as the most reliable downstream model.

4.4 Comparison with State of the Art Approaches↩︎

To evaluate the effectiveness of our approach, we benchmarked the best model obtained in this work (XGBoost) trained with pre-training oversampled VAE-GAN+CPAC data against recent state-of-the-art methods [42]–[44]. We conducted these experiments on these works instead of others listed in Section 2 because they are more recent. Since the code for the selected works is not available we reproduced their classification settings at the best of our abilities and accordingly to the information given; Ding et al. [42] uses an XGBoost as a baseline classifier just like us, so the reported results of our models reflects theirs with our generated dataset proving that our method greatly improves the performances. Shi et al. [43] uses a multi-head attention classifier in their generative pipeline to directly classify the samples reporting perfect metrics, so, we took just the classifier to test with our generated data. Ahmed et al. [44] uses a voting ensemble classification method that comprises a Random Forest Classifier, AdaBoost Classifier and KNN, similary reporting perfect metrics. The Table 8 reports the evaluation results of each method compared to ours with their relative setup: Shi et al. uses a 75/25 split for the dataset and balances the training set so that the minority class reaches the same number of the majority class. Ahmed et al. uses a 80/20 split and balances the training set as well. We ran both tests and compared the results with each method and our XGBoost outperforms in every metric. Table 9 reports the results obtained with our setup (80/10/10 split and only 100 generated samples in the training set) and even in this instance our model outperforms the other proposed methodologies. The main difference between our proposed method of training and the related works’s is that we do not balance the training set because we believe that the models can only benefit from learning an extreme imbalanced distribution during training, adapting it to real world applications even appropriately.

4.5 Evaluation on an Independent Synthetic Credit Card Dataset↩︎

To further assess the generalization capability of the proposed VAE-GAN+CPAC oversampler, we evaluate our approach on an additional credit card fraud detection dataset publicly available on Kaggle and published by Kartik Shenoy [49]. The use of an independent dataset is particularly important in fraud detection, where the availability of publicly accessible and well-documented datasets is extremely limited due to privacy and regulatory constraints. The dataset is fully synthetic and was generated through a controlled simulation process, providing a viable alternative for cross-dataset validation. It consists of simulated credit card transactions spanning the period from January 1, 2019 to December 31, 2020, modeling the activity of 1,000 customers interacting with a pool of 800 merchants. Both legitimate and fraudulent transactions are included. The data were generated using the Sparkov Data Generation tool [50], an open-source simulator designed to emulate realistic credit card usage patterns over time. The individual simulation outputs were subsequently aggregated and converted into a standardized format. Importantly, the dataset is distributed with a predefined and fixed train-test split, which we preserve throughout all experiments. This ensures a fair and reproducible evaluation, prevents any form of data leakage, and allows us to isolate the effect of oversampling techniques, as synthetic data generation and model training are strictly confined to the training partition, while all reported results are computed exclusively on the untouched test set.

4.5.1 Subsequent Results↩︎

To evaluate the proposed VAE-GAN+CPAC framework on the newly introduced dataset, we first trained the model without any pre-training oversampling to assess its behavior on clean, untouched data. Even under this setting, the encoder learns a well-structured latent representation, as illustrated in Figure 13, where fraud and non-fraud samples form clearly separated clusters with minimal overlap, a characteristic expected in highly imbalanced scenarios. The learned prototypes are positioned far apart in the latent space, further confirming the model’s ability to capture discriminative structure without relying on synthetic augmentation. Building on this baseline, we progressively augmented the training set with 50, 75, and 100 synthetic fraud samples generated by the VAE-GAN+CPAC oversampler. The downstream classifiers were then trained on the augmented data and evaluated exclusively on the predefined test set, which remains untouched by any synthetic samples. We first report the results of the classifiers without any oversampling in Table 10 as a reference. The quantitative results are reported in Table 11. Ensemble methods consistently outperform simpler classifiers across all oversampling levels. Random Forest achieves strong Precision and competitive F1-scores, but its performance fluctuates with the number of synthetic samples. In contrast, XGBoost exhibits the most reliable behavior, achieving the highest F1-score and AUC-ROC at higher augmentation levels while maintaining a stable Precision-Recall balance across all configurations. Logistic Regression and KNN show limited sensitivity to the oversampling process, reflecting their constrained representational capacity and instance-based nature, respectively. Overall, these results confirm that the VAE-GAN+CPAC oversampler generalizes effectively to a distinct synthetic dataset and that its benefits are most consistently exploited by robust, margin-aware classifiers such as XGBoost.

5 Ablation Study↩︎

In order to establish the relevance of each component in our method, we systematically analyze the contribution of each key component in our VAE-GAN+CPAC architecture. By selectively removing or disabling certain elements, we demonstrate that every part of the model is essential for achieving discriminative and robust latent representations, particularly in the context of extreme class imbalance.

5.1 Effect of Removing the CPAC Head↩︎

To evaluate the impact of the CPAC head, we trained the VAE-GAN with the same settings as our main pipeline, but without the CPAC head. In this configuration, the encoder is updated only via generative objectives, with no explicit supervision guiding the latent space. Our results show that (as shown in Figure 14), without the classifier, the latent representations of fraud and normal transactions remain heavily overlapped, making downstream classification significantly less accurate. This experiment confirms that the CPAC supervision is crucial for inducing clear separation between classes and for shaping the latent space in a way that supports reliable detection.

5.2 Effect of Removing the Attention Mechanism from CPAC↩︎

The ablation in Figure 15 illustrates the effect of disabling the attention mechanism in the CPAC, reducing it to a pure prototype-based classifier. Without feature-wise attention, each latent dimension is weighted equally when computing distances to the class prototypes. The resulting latent space is visibly less expressive: data points for both normal and fraud classes collapse along a near-linear manifold, with limited separation between the classes and their prototypes. This collapse indicates that the model is unable to adaptively emphasize the most discriminative latent features, leading to suboptimal cluster separation and reduced interpretability. Both class prototypes tend to be positioned close to the respective cluster means, but the lack of per-dimension weighting prevents effective partitioning of ambiguous or borderline samples. As a result, the discriminative power of the model is noticeably diminished compared to the full CPAC, where the attention mechanism enables more nuanced, non-linear separation of minority-class samples.

5.3 Effect of Removing the Prototypes from CPAC↩︎

Figure 16 shows the latent space organization when the CPAC architecture is ablated to remove its prototype mechanism, leaving only the attention branch. In this setting, class discrimination relies exclusively on feature-wise weighting, with no explicit anchoring to learned class prototypes. The resulting latent representations display a pronounced collapse along a narrow, near-linear manifold, with both normal and fraud samples largely overlapping. The absence of prototypes deprives the classifier of distinct, class-specific anchors in the latent space, severely restricting its ability to drive separation between classes. Although the attention branch allows the model to adaptively weight latent features, this alone proves insufficient for robust clustering, especially in the presence of extreme class imbalance. Consequently, the discriminative structure of the latent space deteriorates, making class boundaries ambiguous and reducing interpretability.

5.4 Effect of Removing the Anchor and Scale Penalties↩︎

Figure 17 visualizes the latent space structure when the CPAC classifier is trained without the scale and anchor regularization terms. In this setting, both the attention mechanism and learnable class prototypes remain active, but the model no longer receives explicit constraints on the spread and positioning of prototypes with respect to the cluster means. The resulting latent representations maintain a moderate level of separation between normal and fraud clusters, with prototypes positioned near the centers of their respective class distributions. However, the boundaries between clusters are less crisp than in the fully regularized setting, and the spread of both clusters along the main latent direction increases. The absence of scale and anchor penalties allows prototypes to drift from the empirical cluster centers and can result in more diffuse class boundaries, reducing both interpretability and the sharpness of latent cluster assignments.

5.5 Effect of Using Focal Loss for the CPAC↩︎

To further explore these ablations, we trained the VAE-GAN+CPAC pipeline using Focal Loss in place of standard binary cross-entropy as a loss function for the CPAC. As shown in Figure 18 (a), this modification results in a more distinct and well-separated clustering of fraud (minority) and normal (majority) transactions in the latent space, with prototypes more cleanly anchoring their respective clusters. The 3D latent visualization (Figure 18 (b)) further confirms improved class-wise separation and greater dispersion of the minority class, compared to previous experiments using BCE (see Section 4). Despite this evident structural improvement, Table 12 reveals a slight decrease in downstream classification metrics across all evaluated classifiers and augmentation regimes. For instance, with 100 synthetic fraud samples, F1-scores for XGBoost and Random Forest remain above 92%, but are marginally lower than those achieved with BCE-based training. This modest drop in quantitative performance can be attributed to the nature of Focal Loss: by prioritizing hard-to-classify minority samples, it encourages the model to push fraud examples away from the decision boundary, at the expense of global average accuracy and sometimes increased variability in the majority class predictions.

6 Discussion: Rethinking Oversampling in Fraud Detection↩︎

Despite significant advances in generative oversampling, most notably with SMOTE and VAE-GAN variants, most current state-of-the-art approaches in credit card fraud detection adopt a common strategy: training oversamplers exclusively on the minority (fraud) class. This “minority-only” paradigm is widespread, underpinned by the rationale that a focused model can better capture rare fraudulent patterns and help rebalance the dataset. However, our results suggest that this strategy may have important limitations when applied to real-world fraud detection. Training an oversampler solely on fraud data risks generating synthetic samples that closely mimic observed frauds, rather than learning the nuanced differences between normal and fraudulent activity. As a result, such generated data may reflect interpolations within the minority class, lacking the true discriminative boundaries that are critical for effective classification. Deep and more complex classifiers rather than standard ones trained on these oversampled datasets often display overly optimistic metrics, sometimes failing to generalize robustly to new, unseen data or even overfitting even with few generated samples; simpler and classic classification approaches like the ones we used in this work struggle to improve their metrics especially in term of recall given the similarity between the synthetic frauds and the real ones. Both SMOTE and vanilla VAE–GAN oversamplers succeed in enriching the minority class and boosting downstream classifier scores. However, SMOTE’s impressive precision and recall are largely artifacts of its linear interpolation, which generates highly similar fraud examples and trains classifiers to be overconfident within a narrow region of feature space. The unsupervised VAE–GAN, while producing more realistic fraud instances, still suffers from overconfidence and mode collapse, generating synthetic cases that closely mimic the original frauds, limiting generalization. When applying our CPAC classifier to these augmented datasets, we confirm a shared limitation: traditional oversamplers simply replicate, rather than expand, the fraud distribution. In contrast, our proposed VAE-GAN+CPAC pipeline is designed to address these limitations. By training on the entire dataset, including both fraud and normal transactions, and integrating a CPAC head to provide explicit, supervised class information, our approach encourages the encoder to structure its latent space to distinguish fraud from non-fraud. This design is not simply an auxiliary feature, but a central objective of the model: the classifier head directly shapes the latent representations so that generated synthetic frauds meaningfully reflect the learned class boundaries. During inference, only the minority samples are generated, but the key insight is that the model’s holistic training allows it to create more realistic and discriminative synthetic data. This approach yields several practical advantages, as reflected in our benchmarks:

• Reduced Overfitting and Overconfidence: Models trained with our VAE-GAN+CPAC-generated frauds exhibit more stable and realistic performance, avoiding the inflated precision or recall sometimes observed with traditional oversamplers.

• Improved Generalization: The modest drop in certain metrics compared to some SOTA methods is, in fact, evidence of better generalization. Our synthetic frauds help downstream classifiers learn the true structure of the data, rather than simply memorizing training examples.

• No Plateau Effect: While standard oversamplers quickly reach a performance plateau or even degrade as more synthetic samples are added, our approach enables incremental improvements until larger sample sizes begin to induce overfitting, as expected.

• Cluster Separation and Interpretability: The explicit supervision provided by CPAC results in clearer separation between fraud and non-fraud clusters in latent space, facilitating interpretability and model transparency.

A key insight is that fraud can only be understood in the context of normal transactions, its definition is inherently relational. Oversamplers that ignore this context may generate synthetic data that resides within the convex hull of observed frauds, without sufficiently capturing the critical distinctions needed for robust classification. This can result in models that are prone to memorization, rather than effective discrimination.

6.1 Operational Considerations and Business Impact↩︎

In real-world fraud detection systems, performance must be interpreted not only in terms of aggregate metrics, but also with respect to operational costs and customer experience. In particular, false positives may lead to transaction friction, customer dissatisfaction, and increased manual review effort. The proposed CPAC-guided approach is designed to favor a recall-oriented behavior, prioritizing the identification of fraudulent transactions while maintaining a controlled increase in false positives. This design choice reflects common industrial practice, where the cost of missed fraud typically outweighs the cost of additional verification steps. The use of an explicit decision threshold allows practitioners to tune the precision-recall trade-off according to domain-specific risk tolerance and business constraints. Compared to traditional oversampling strategies such as SMOTE, CPAC provides a more structured and stable decision boundary, resulting in improved recall without relying on aggressive oversampling that may amplify noise. From a deployment perspective, CPAC primarily serves as a training-time mechanism to shape latent representations via prototype anchoring and attention, improving robustness under extreme class imbalance. In practical systems, this enables hybrid pipelines in which the improved representations can be paired with lightweight classifiers for real-time scoring, while preserving the interpretability and robustness benefits introduced during training.

6.2 Our Contribution↩︎

By training the VAE-GAN+CPAC on the full dataset and using the CPAC head to guide the latent representation, we bridge the gap between oversampling and supervised discriminative learning. The synthetic frauds produced by our approach are not naive copies, but instead reflect meaningful, learned differences between classes. This allows downstream classifiers, particularly XGBoost in our experiments, to achieve strong, generalizable performance, even under extreme class imbalance. In summary, our findings support the view that the field would benefit from moving beyond the traditional minority-only oversampling paradigm toward more context-aware, discriminative approaches. Our VAE-GAN+CPAC pipeline offers a principled step in this direction.

7 Conclusions and Future Works↩︎

This work has demonstrated the limitations of minority-class-only oversampling for fraud detection and highlighted the benefits of classifier-guided, class-aware latent shaping using the CPAC within a VAE-GAN framework. Our results show that relying solely on synthetic augmentation of the minority class can lead to overconfident, poorly generalized models, whereas supervised feedback and prototype-driven clustering provide more meaningful separation and robust detection performance, as reflected in improved F1-score, recall, and AUC. The CPAC approach offers unique interpretability and flexibility, allowing both effective visualization and strong performance under extreme class imbalance. These findings advocate for a shift away from traditional oversampling techniques towards architectures that explicitly leverage class structure, supervised objectives, and interpretability. Looking ahead, there are several promising directions for advancing this line of research. Deeper or more complex neural classifiers, either as standalone detectors or as encoder heads, could potentially capture more nuanced fraud patterns. The integration of denoising strategies, such as Denoising Autoencoders, may further enhance confidence and reduce latent overlap. Broader validation across other imbalanced noise detection tasks and domains, as well as the development of richer, more transparent explanation methods, represent important next steps. Finally, exploring alternative strategies for latent space shaping, such as contrastive or manifold-regularized objectives, may yield further gains, especially in low-label or highly imbalanced settings. Overall, our approach not only advances performance metrics but also addresses the critical needs of interpretability, reliability, and resilience. While focused on credit card fraud, the proposed methods and insights generalize to a wide range of anomaly detection challenges, paving the way for more robust and trustworthy machine learning systems.

Acknowledgments↩︎

This study has been partially supported by SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union - NextGenerationEU.

References↩︎