=Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning =Ruixin Li, Jin Liu, Yuling Shi, Stefano Lodi

1 Introduction↩︎

Self-supervised learning (SSL) has become a central paradigm for visual representation learning, enabling large-scale pretraining without manual annotation. Existing methods broadly fall into two categories: discriminative frameworks, which align representations across augmented views through contrastive or self-distillation objectives [1]–[5], and reconstruction-based frameworks, which learn local semantics by recovering masked content or features [6]–[8]. These paradigms have produced highly transferable representations across a wide range of domains, including recent medical foundation models [9]. However, standard SSL pipelines generally do not treat reflection as meaningful structure: discriminative methods often suppress it through invariance, while reconstruction-based methods usually leave it implicit.

This limitation becomes particularly relevant in bilaterally organized data, where mirrored regions are structurally related but not strictly identical. In medical imaging, contralateral anatomy often shares broad spatial organization while differing in diagnostically important local detail; unilateral abnormalities in chest radiographs or asymmetric patterns in brain MRIs are therefore informative rather than nuisance variation [10]. A similar pattern arises in facial images, where mirrored regions remain correlated but can differ in expression, illumination, or localized attributes [11]. In such settings, treating reflection as a generic label-preserving transformation risks weakening precisely the asymmetric cues that downstream tasks rely on. A natural alternative is to move from strict invariance toward reflection-aware or equivariant representation learning, so that features respond predictably rather than being forced to suppress geometric structure. However, existing approaches often either impose symmetry through rigid architectural constraints or encourage it only through the training objective. For reflection-structured but imperfectly symmetric data, this leaves open the need for a lightweight representation-level mechanism that can model mirror correspondence while retaining informative asymmetry.

We address this gap with Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a simple framework for injecting a soft reflection prior into standard Vision Transformers. MFASSL forms mirror-paired views during pretraining and introduces a lightweight Mirror-Fusion Attention (MFA) block that exchanges information between corresponding mirror tokens through adaptive gating. Rather than enforcing strict left–right equivalence, the model is trained to combine two complementary objectives: encouraging reflection-consistent structure where correspondence is reliable, and retaining local discrepancies where asymmetry is informative. This design keeps the backbone unchanged in spirit, while enabling reflection-aware reasoning at an intermediate representation level.

2 Related Work↩︎

2.0.1 Equivariance and Symmetry Priors.↩︎

Equivariance offers an alternative inductive bias to invariance by encouraging representations to transform predictably under geometric operations. Classical group-equivariant CNNs [12], [13] and steerable architectures [14] encode symmetry groups directly into the model, while SE(3)-equivariant networks and related transformer variants extend these ideas to richer geometric settings [15]–[19]. More recent work has introduced softer symmetry priors through self-supervised objectives or local consistency constraints. E-SSL [20], [21] promotes transformation-aware features through equivariant learning objectives; EquiMod [22] adds an equivariance module for visual instance discrimination; transformation-learning objectives directly train equivariant representations from self-supervised transformations [23]; and OcticViT [24] incorporates discrete symmetry groups into ViT-based SSL. Related pixel-level frameworks also encourage local geometric consistency through correspondence or uncertainty-guided alignment [25]. These studies demonstrate the usefulness of geometric priors for representation learning. However, their symmetry assumptions are usually introduced at the level of model design or optimization, rather than through an explicit feature-level interaction between mirror-paired regions. As a result, they may be less suitable for data with approximate bilateral regularity.

2.0.2 Self-Supervised Learning in Medical Imaging.↩︎

Self-supervised learning has become increasingly important in medical imaging [26]–[28], where large volumes of unlabeled data are available but expert annotation is expensive. Early reconstruction-based approaches such as Models Genesis [29] and TransVW [30] demonstrated the value of surrogate-task pretraining, and later contrastive, distillation-based, and masked-reconstruction methods further improved transfer across radiology and MRI applications [31]–[33]. More recently, medical foundation models have extended SSL to broader clinical imaging settings. Despite these advances, most medical SSL methods do not explicitly model contralateral or mirrored regions as structured correspondences. Some supervised or task-specific methods have incorporated bilateral or reflection priors for segmentation, adaptation, and brain-imaging analysis [34]–[36], but they typically rely on dense supervision, modality-specific assumptions, or downstream-task-specific formulations. As a result, reflection-aware SSL for general-purpose medical pretraining remains relatively underexplored.

2.0.3 Symmetry and Reflection in Natural Vision.↩︎

Beyond medical imaging, symmetry is also an important cue in natural vision and has been exploited in tasks such as facial analysis, human pose estimation, and fine-grained recognition. In facial image modeling, symmetry-aware methods use left–right consistency to improve completion or restoration of facial structure and appearance [37]. In pose estimation, mirror-based geometric constraints help reduce ambiguity in 3D human reconstruction [38]. In fine-grained recognition and canonical representation learning, symmetry can support semantically aligned templates and more structured feature spaces [39], [40]. Related two-branch designs in segmentation further show the benefit of structured feature interaction between global context and spatial detail [41]. However, most of these methods use symmetry in task-specific or fixed-fusion ways rather than as a general prior for visual pretraining. More flexible handling of approximate reflection structure at the token level remains relatively underexplored.

3 Method↩︎

We propose Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a plug-and-play framework for ViTs that injects a soft reflection prior into standard SSL. MFASSL combines mirror-paired view generation, a lightweight Mirror-Fusion Attention (MFA) block for token-level cross-mirror interaction, and a symmetry-aware objective for global and token-level reflection consistency. An overview of the MFASSL pretraining pipeline is shown in Fig. 1 (a).

A key design choice is that MFA is used as a pretraining-only adapter rather than as a replacement for a transformer block. Ordinary SSL views follow the standard ViT path, while mirror-paired views are processed to layer \(\ell\), where pre-fusion tokens provide the inputs for the reflection-aware losses and MFA. The fused mirror-view representations then continue through the remaining transformer blocks and contribute to the same base SSL objective used by the underlying method. At inference time, MFA and mirror-paired inputs are removed, and the learned ViT is deployed with the standard single-image forward pass.

Pretraining data flow is as follows:

1. Given an image \(x\), a base SSL pipeline \(B\), and a target layer \(\ell\), sample ordinary SSL views \(V=B(x)\) and encode them with the standard ViT path.

2. Sample mirror crops \((x_L,x_R)\) around the jittered vertical axis; flip only \(x_R\) for alignment.

3. Run both crops to layer \(\ell\) and store pre-fusion patch tokens \(\phi_\ell(x_L),\phi_\ell(x_R)\).

4. Compute \(\mathcal{L}_{\text{eq}}\) and \(\mathcal{L}_{\text{mid}}\) from these pre-fusion tokens.

5. If \(t<t_{\text{mfa}}\), continue with the unfused tokens; otherwise apply MFA and continue through blocks \(\ell{+}1,\ldots,L\).

6. Compute the base SSL objective using the ordinary SSL views together with the post-fusion mirror representations, then optimize Eq. 8 .

3.1 Mirror-Fusion Attention (MFA)↩︎

Given mirror-aligned token sequences \(X_L, X_R \in \mathbb{R}^{N \times D}\) produced by a ViT encoder, MFA performs cross-view attention with learnable gating. Fusion is computed in both directions \(L \leftarrow R\) and \(R \leftarrow L\); for brevity, we describe the \(L \leftarrow R\) branch only. MFA is a lightweight cross-attention block specialized for mirror-paired tokens and equipped with an additional discrepancy-preserving channel. The architecture is illustrated in Fig. 1 (b).

We use standard scaled dot-product cross-attention to model interactions between the original and mirrored features, with queries from \(X_L\) and keys and values from \(X_R\): \[A_{L\leftarrow R} = \mathrm{softmax}\!\left( \frac{Q_L K_R^\top}{\sqrt{D_h}} \right)V_R \in \mathbb{R}^{N \times D}.\] where \(D_h\) denotes the attention feature dimension used for scaling.

3.1.1 Learnable per-token distance-gating.↩︎

To modulate fusion strength based on token-level spatial correspondence, we introduce a per-token gate. For the \(i\)-th token pair (\(1 \leq i \leq N\)): \[g_i = \sigma\!\Big( a - b \, \textstyle\sum_{j=1}^{D}\sqrt{(X_{L,i,j} - X_{R,i,j})^2 + \epsilon^2} \Big), \label{eq:gate}\tag{1}\] where \(X_{L,i,j}\) and \(X_{R,i,j}\) denote the \(j\)-th feature of the \(i\)-th token from \(X_L\) and \(X_R\) respectively, \(a, b\) are learnable scalars, \(\sigma\) is a sigmoid, and \(\epsilon = 10^{-6}\).

This computes a smooth \(\ell_1\) distance over the feature dimension and yields a gate vector \(g \in \mathbb{R}^{N \times 1}\), which is broadcast during fusion. The smoothed form is used instead of an exact \(\ell_1\) norm to keep the gate differentiable at zero. The gate therefore allows MFA to suppress fusion for locally mismatched tokens while retaining stronger cross-view interaction at spatial positions with high mirror correspondence.

We enforce \(b \geq 0\) via \(b = \mathrm{softplus}(\tilde{b})\) and initialize \((b, a) = (1.0, 0.5)\), which biases the gate toward conservative fusion early in training. We note that the effective threshold depends on the feature magnitude at layer \(\ell\); rather than a fixed geometric interpretation, this initialization is intended to keep the gate in a conservative, near-neutral regime for commonly observed mid-layer feature scales and to let it adapt during training.

3.1.2 Fusion update with an asymmetry-preserving channel.↩︎

The final fused representation combines identity, gated cross-view attention, and a discrepancy-preserving residual: \[Z_L = X_L + \alpha \big( g \odot A_{L \leftarrow R} \big) + \gamma (X_L - X_R), \label{eq:update}\tag{2}\] and symmetrically for \(Z_R\), where \(\alpha\) and \(\gamma\) are learnable scalars (initialized conservatively to 0.1) and \(\odot\) denotes element-wise multiplication.

Structurally, the gated attention branch shares information across corresponding mirror locations, while the discrepancy term preserves non-symmetric evidence. When mirror correspondence is strong, the gated attention branch dominates cross-view interaction; when local asymmetry emerges, the discrepancy term helps retain informative differences. This design enables selective bilateral information exchange without enforcing strict left–right equivalence, while the small initialization improves training stability and mitigates early-stage distribution shift.

MFA is used only during pretraining. At inference time, the pretrained ViT encoder is applied normally to a single input image without mirror fusion. This choice is empirical rather than a formal invariance guarantee: the staged schedule and small residual initialization are used to keep the train–test feature shift small. To quantify this shift, we measure \(\|Z_L-X_L\|/\|X_L\|\) at the fusion layer on the CheXpert validation set after training; the mean relative perturbation is 3.8% (\(\pm\)​1.2%).

3.2 Mirror-Paired View Generation↩︎

Mirror-paired view construction is required in MFASSL because both the MFA gate and the token-level consistency loss assume spatial correspondence across the two views. A standard horizontal flip alone does not provide this: the two views remain in opposite orientations and therefore cannot be compared token-by-token without explicit alignment. To establish correspondence, we construct mirror-paired crops \((x_L, x_R)\) by slicing around a jittered vertical midline and horizontally flipping the right crop so that both views share the same spatial orientation, as illustrated in Fig. 1 (a). Throughout the paper, \(x_R\) denotes this already aligned right crop, and no additional flip is applied in the losses. In our experiments, images are first resized and oriented using the dataset preprocessing protocol, and the symmetry axis is taken as the crop-center vertical line with a random horizontal jitter of up to 3% of image width. Preliminary validation showed that 1% jitter produced overly concentrated gates near the estimated midline, whereas 5% jitter slightly degraded BraTS segmentation; we therefore keep 3% fixed across all reported experiments. For domains with explicit landmarks or registration, the same construction can instead use the estimated anatomical or facial midline.

Mirroring is not applied to every augmentation. The standard SSL crops keep the original MoCo-v3, DINO, or MAE augmentation recipes, while the mirror pair is added as a separate paired branch. For multi-crop SSL methods such as MoCo-v3 and DINO, the mirror pair is appended as the last two crops in the multi-crop set, allowing reuse of the original training pipeline. For MAE, which does not use multi-crop training, \((x_L, x_R)\) is processed in a dedicated paired forward pass. Each crop is masked independently, MFA operates on the visible tokens at layer \(\ell\), and the symmetry-aware losses \(\mathcal{L}_{\text{eq}}\) and \(\mathcal{L}_{\text{mid}}\) are computed only on token positions visible in both crops, while the reconstruction objective supplies the corresponding base SSL supervision after the paired branch continues through the decoder path.

3.3 Symmetry-Aware Objective↩︎

Let \(\mathcal{L}_{\text{base}}\) denote the underlying SSL objective (contrastive, distillation-based, or reconstruction-based). MFASSL augments it with two reflection-aware terms: \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{base}} + \lambda_{\text{eq}} \mathcal{L}_{\text{eq}} + \lambda_{\text{mid}} \mathcal{L}_{\text{mid}}. \label{eq:total-loss}\tag{3}\] Here, \(\mathcal{L}_{\text{eq}}\) encourages global reflection-consistent alignment, while \(\mathcal{L}_{\text{mid}}\) enforces token-level correspondence at the same pre-fusion layer where MFA operates. These two losses regularize the inputs to MFA. The base term is kept objective-compatible with the selected SSL framework: standard augmented views use the original MoCo-v3, DINO, or MAE objective, and the post-fusion mirror representations are passed through an identically formed branch of the same objective. Therefore, MFA parameters receive gradients through \(\mathcal{L}_{\text{base}}\) without introducing an additional contrastive, distillation, or reconstruction loss family.

3.3.1 Reflection-consistency loss.↩︎

To enforce reflection-aware global consistency jointly with mid-layer consistency, we compute \(\mathcal{L}_{\text{eq}}\) at the same designated layer as \(\mathcal{L}_{\text{mid}}\), before fusion. We use a negative cosine similarity loss [42]: \[\mathcal{L}_{\text{eq}} = 1 - \cos\big( s_\ell(x_L), s_\ell(x_R) \big), \label{eq:leq}\tag{4}\] where \(s_\ell(\cdot)\) denotes mean-pooled, \(\ell_2\)-normalized patch-token embeddings from layer \(\ell\). Since the mirror crops are already spatially aligned (Sec. 3.2), this loss encourages global bilateral feature similarity between corresponding left–right regions, promoting reflection-consistent representations at the designated layer. Cosine distance is used here because \(s_\ell\) is a global representation and scale should not dominate the alignment signal.

3.3.2 Mid-layer consistency loss.↩︎

Head-level alignment alone does not adequately constrain token correspondences at the fusion layer. A token-wise constraint before MFA provides a stronger prior [43] and stabilizes MFA once activated. Thus, we impose: \[\mathcal{L}_{\text{mid}} = \frac{1}{N} \sum_{i=1}^{N} \Vert \hat{\phi}_\ell(x_L)_i - \hat{\phi}_\ell(x_R)_i \Vert_2^2, \label{eq:lmid}\tag{5}\] where \(\hat{\phi}_\ell(\cdot)_i \in \mathbb{R}^D\) denotes the \(\ell_2\)-normalized token representation at spatial position \(i\) from transformer block \(\ell\): \[\hat{\phi}_{\ell,i} = \frac{\phi_{\ell,i}}{\|\phi_{\ell,i}\|_2}, \label{eq:phinorm}\tag{6}\] where \(\phi_\ell \in \mathbb{R}^{N \times D}\) is the raw patch-token output of block \(\ell\) and each token \(\phi_{\ell,i} \in \mathbb{R}^D\) is independently normalized to unit \(\ell_2\) norm. Compared to prior pixel-level or region-level consistency losses [25], [31], \(\mathcal{L}_{\text{mid}}\) is defined directly on mid-layer ViT tokens and is tightly aligned with the mirror-paired view construction in Sec. 3.2. The squared \(\ell_2\) form is applied after per-token normalization, so it acts as a token-wise cosine alignment up to a constant factor while preserving the FitNet-style token matching interpretation.

3.4 Training and Implementation↩︎

We follow the base SSL optimizers and schedules commonly used for ViTs. To avoid noisy cross-branch coupling before mirror correspondence has formed, MFASSL uses a staged training strategy.

3.4.1 (i) Symmetry-loss ramp↩︎

For the first \(T_{\text{sym}}\) epochs, we apply the symmetry-aware losses (\(\mathcal{L}_{\text{eq}}, \mathcal{L}_{\text{mid}}\)) with a linear ramp \(w(t)\), allowing the backbone to learn stable, aligned representations from the symmetry prior before cross-mirror fusion: \[w(t) = \mathrm{clip}\!\left( \frac{t}{T_{\text{sym}}}, 0, 1 \right), \quad T_{\text{sym}} = 10 \;\text{epochs}. \label{eq:sym-ramp}\tag{7}\] The total optimization objective at epoch \(t\) is therefore \[\mathcal{L}(t) = \mathcal{L}_{\text{base}} + w(t)\big[ \lambda_{\text{eq}} \mathcal{L}_{\text{eq}} + \lambda_{\text{mid}} \mathcal{L}_{\text{mid}} \big]. \label{eq:loss-time}\tag{8}\]

3.4.2 (ii) MFA activation and gate ramp↩︎

We disable MFA during the early stage of training and activate it only after the representation has become sufficiently aligned. Specifically, MFA is inserted at epoch \(t_{\text{mfa}} = 12\), after which its gate is progressively released using \[r_t = \mathrm{clip}\!\left(\frac{t - t_{\text{mfa}}}{T_{\text{gate}}}, 0, 1\right), \qquad g_t = r_t \, g, \label{eq:gate-ramp}\tag{9}\] where \(g \in \mathbb{R}^{N \times 1}\) denotes the gate vector defined by Eq. 1 for the current token sequence. With \(T_{\text{gate}} = 10\) epochs, this schedule ensures that the perturbation introduced by gated cross-view fusion starts near zero and increases gradually. The discrepancy residual is not ramped directly. Instead, its early influence remains limited through the conservative initialization of \(\gamma\), which keeps this branch small before stable left–right correspondence has formed.

For efficiency, standard crops are encoded first, and the mirror pair is processed once through a dedicated paired forward pass that returns both pre-fusion and post-fusion tokens. This keeps the additional computational cost modest relative to other equivariance-aware SSL formulations.

4 Experiments↩︎

4.1 Datasets↩︎

We use five datasets in total: CheXpert [10] and BraTS [44] as medical downstream benchmarks, OASIS-3 [45] as additional unlabeled MRI pretraining data, and CelebA-HQ [46] and WFLW [11] as natural-image downstream benchmarks. CheXpert contains 224,316 chest radiographs with 14 labels; for our multi-label classification experiments, we use frontal-view images. BraTS 2023 provides multi-modal brain MRI scans with annotations for three tumor subregions: enhancing tumor (ET), tumor core (TC), and whole tumor (WT). OASIS-3 provides additional unlabeled T1-weighted MRIs from an aging neuroimaging cohort. CelebA-HQ contains 30,000 high-resolution face images with 40 annotated attributes, and WFLW provides 98 facial landmarks per image. All medical splits are patient-wise.

4.2 Pretraining Configuration↩︎

All experiments use a ViT-B/16 backbone with 12 transformer blocks and a patch size of \(16\times16\). All images are resized to \(224\times224\). We evaluate three SSL paradigms: MoCo-v3, DINO, and MAE, and pretrain all models for 300 epochs using AdamW with cosine learning-rate decay. MFASSL inserts the MFA block at block 8 and applies both the reflection-consistency loss \(\mathcal{L}_{\text{eq}}\) and the mid-layer consistency loss \(\mathcal{L}_{\text{mid}}\) at the same pre-fusion layer. After fusion, the mirror-pair branch continues through the remaining transformer blocks and is optimized with the original SSL objective, without changing the form of the base MoCo-v3, DINO, or MAE loss. We set \(\lambda_{\text{eq}}=0.5\) and \(\lambda_{\text{mid}}=1.0\), selected on the CheXpert validation set and then fixed for all other datasets and backbones. The symmetry-aware losses are linearly ramped up during the first 10 epochs, MFA is activated at epoch 12, and the gate is further released with a 10-epoch ramp. This gives a simple transfer heuristic: use a middle-late layer (about two-thirds depth), keep the two symmetry-loss weights fixed, and only reduce the weights if the base SSL loss becomes unstable. Appendix summarizes the consolidated hyperparameter settings.

For medical classification, pretraining is performed on CheXpert. For MRI-based segmentation and robustness experiments, we jointly pretrain on unlabeled BraTS and OASIS-3 using 50/50 mixed mini-batches. For natural-image experiments, pretraining is performed on CelebA-HQ. At inference time, MFA is removed and the finetuned model remains a standard ViT encoder.

4.2.1 Controlled baselines.↩︎

To isolate the effect of the proposed reflection-aware components, all baselines are trained from random initialization without ImageNet pretraining, dataset-specific augmentation tuning, or test-time augmentation. This standardized setting enables a cleaner assessment of method-level differences under matched training conditions, but it also limits direct comparison with ImageNet-initialized pipelines and flip test-time ensembling. We therefore interpret MFASSL as improving the single-forward-pass encoder under matched pretraining rather than as a replacement for all inference-time augmentation strategies.

4.2.2 Competitor methods.↩︎

We compare against two recent equivariant SSL approaches under the same ViT-B/16 backbone and training budget: (i) E-SSL [20], which adds an equivariant prediction loss to the base SSL objective, and (ii) OcticViT [24], which embeds discrete symmetry groups into the ViT architecture. We evaluate two OcticViT variants, OcticViT-H\(_8\) and OcticViT-I\(_8\), using the hyperoctahedral and icosahedral groups of order 8, respectively. OcticViT is evaluated only under the DINO backbone because its group-equivariant architectural modifications, specifically the group-structured patch token projections, are incompatible with MoCo-v3’s momentum contrast objective and MAE’s masked-token reconstruction pipeline. E-SSL is applied to DINO, MoCo-v3 and MAE. All competitors are re-implemented and trained under identical settings.

4.3 Evaluation Protocols↩︎

CheXpert is evaluated with both linear probing and full fine-tuning. BraTS segmentation is evaluated slice-wise in 2D, reporting Dice and Hausdorff distance at the 95th percentile (HD95) for the three standard subregions (ET, TC, WT) and their means, as commonly done in nnU-Net [47], together with voxel-level calibration metrics: expected calibration error (ECE [48]) and negative log-likelihood (NLL [49]). In the main text, we report mean Dice and mean HD95 for compact comparison, while the full ET/TC/WT breakdown is provided in the Appendix. CelebA-HQ is evaluated using classification accuracy, NLL, ECE, and Flip-Consistency. WFLW is evaluated using normalized mean error (NME), AUC@0.1, Failure@0.1, and Flip-Consistency. For BraTS we report mean \(\pm\) standard deviation over three runs. For the other benchmarks, the main tables report point estimates averaged over repeated runs.

Flip-Consistency measures prediction agreement between an image and its horizontal reflection. For CheXpert, it is the fraction of test images for which the predicted multi-label set is unchanged after reflection. For CelebA-HQ, it is computed over all 40 attribute predictions. For WFLW, flipped predictions are first remapped to the corresponding landmark indices before consistency is measured.

4.4 Main Results↩︎

Across four representative downstream domains, CheXpert, BraTS, CelebA-HQ, and WFLW, MFASSL generally improves task performance and reflection-related consistency under identical ViT-B/16 backbones and training budgets (Tables 1–3). The gains are strongest in fine-tuning and remain visible across contrastive, distillation-based, and reconstruction-based SSL backbones. Linear-probe gains are smaller and mixed, so we do not claim that MFASSL universally improves frozen representations; rather, the evidence supports better fine-tunable initialization and reflection-aware robustness.

4.4.1 Medical benchmarks: CheXpert and BraTS.↩︎

On CheXpert (Table 1), MFASSL delivers the strongest overall improvements across all three SSL baselines. Under fine-tuning, it raises AUROC from 84.72 to 85.96 on DINO, from 83.94 to 84.50 on MoCo-v3, and from 81.62 to 82.31 on MAE, while also improving AUPRC, F1, and Flip-Consistency in each case. Competing symmetry-aware baselines yield smaller gains: for example, on DINO, E-SSL improves AUROC by only +0.16 pp and OcticViT by +0.23 pp, compared with +1.24 pp for MFASSL. Calibration is also generally improved, especially in fine-tuning, with DINO ECE decreasing from 0.039 to 0.029 and NLL from 0.372 to 0.360. The linear-probe block is more modest: DINO and MAE improve slightly, while MoCo-v3 F1 decreases from 54.28 to 54.06 despite AUROC and AUPRC gains. We therefore treat the linear-probe result as evidence that the frozen representation is largely preserved, not as the main source of the method’s benefit. On BraTS (Table 2), MFASSL similarly improves segmentation quality across all three backbones, achieving the best mean Dice on DINO (0.836), MoCo-v3 (0.825), and MAE (0.851), together with lower HD95 in each setting. The largest gain appears on DINO, where mean Dice increases from 0.827 to 0.836 and HD95 decreases from 8.6 mm to 8.1 mm, whereas OcticViT variants provide only marginal Dice gains under the same protocol. MAE + MFASSL shows a small ECE increase (0.041 \(\rightarrow\) 0.043), but still attains the best overall Dice and NLL (0.190). Overall, the medical results in Tables 1 and 2 show that MFASSL improves predictive quality and reflection consistency in the evaluated settings, with calibration gains that are strongest for DINO and MoCo-v3.

4.4.2 Natural-image benchmarks: CelebA-HQ and WFLW.↩︎

The same trend extends beyond medical data. On CelebA-HQ and WFLW (Table 3), MFASSL improves performance across all three SSL backbones. For CelebA-HQ, accuracy increases from 90.3 to 91.2 on DINO, from 89.4 to 90.6 on MoCo-v3, and from 90.7 to 91.1 on MAE; Flip-Consistency also rises consistently, reaching 93.6, 92.8, and 94.0, respectively. For WFLW, MFASSL reduces NME from 4.61 to 4.46 on DINO, from 4.74 to 4.58 on MoCo-v3, and from 4.55 to 4.39 on MAE, while also improving Fail@0.1 and Flip consistency. The best overall WFLW result is obtained by MAE + MFASSL, which reaches 4.39 NME and 2.9 Fail@0.1 (Table 3).

4.5 Ablation and Extended Analysis↩︎

We conduct controlled ablations on CheXpert and BraTS using the DINO backbone to isolate the effects of mirrored inputs, symmetry-aware losses, and MFA, and then examine the impact of insertion depth and backbone scale.

4.5.1 Component ablation.↩︎

Table 4 shows that naive mirrored-input duplication has negligible effect, changing CheXpert AUROC from 84.72 to 84.73 and leaving BraTS mean Dice unchanged at 0.827. Adding the symmetry-aware losses alone already improves performance on both datasets, raising AUROC to 84.99 and Dice to 0.831. By contrast, MFA without \(\mathcal{L}_{\text{eq}}\) or \(\mathcal{L}_{\text{mid}}\) yields smaller gains than the full formulation, indicating that its benefit is realized most clearly when paired with symmetry-aware supervision. The full model performs best, reaching 85.96 AUROC on CheXpert and 0.836 mean Dice with 8.1 mm HD95 on BraTS. Together, these results show that the main gains come from combining feature-level mirror interaction with explicit symmetry-aware supervision, rather than from mirrored views or added module capacity alone.

4.5.2 Layer placement.↩︎

Table 5 evaluates representative insertion depths for MFA and the symmetry-aware losses in the 12-layer ViT-B/16 backbone. Layers 4 and 6 diverge during training, suggesting that early features are too unstable for reliable mirror correspondence. In these failed runs, instability appears shortly after MFA activation: large pre-fusion token discrepancies lead to low-entropy gate values and large paired-branch gradients, causing the token-alignment term to dominate early patch features. Among the convergent settings, layer 8 gives the best performance, while layers 10 and 12 are slightly weaker. This pattern indicates that mid-level representations provide the best balance between spatial structure and semantic maturity for reflection-aware fusion. In practice, we monitor the MFA gate distribution and the gradient norm of the paired branch; if either becomes unstable, delaying \(t_{\mathrm{mfa}}\) or reducing \(\lambda_{\mathrm{mid}}\) is safer than moving MFA to an earlier layer.

4.5.3 Multi-Architecture Evaluation↩︎

Table 6 shows that the gains are not specific to ViT-B/16. On ViT-S/16, MFASSL consistently improves CheXpert AUROC by 0.85–0.88 pp across DINO, MoCo-v3, and MAE, and improves BraTS mean Dice by 0.7–0.8 pp. The ViT-B/16 results follow the same trend, with AUROC gains of 0.56–1.24 pp and Dice gains of 0.8–0.9 pp. Overall, MFASSL remains effective across both backbone scales and all three SSL paradigms.

5 Discussion↩︎

5.0.1 Limitations.↩︎

MFASSL has several limitations that define its current scope. First, the method assumes an approximately known vertical reflection axis, so its effectiveness may decrease when this axis is poorly aligned, ambiguous, or semantically weak. Our current jitter handles small horizontal shifts but does not fully evaluate angular midline errors such as \(\pm5^\circ\) or \(\pm10^\circ\) rotations. Second, MFASSL is intended for bilaterally structured domains; on scenes, cluttered objects, or other weakly symmetric images, the mirror prior may become uninformative or harmful, and such settings require explicit validation rather than direct extrapolation. Third, the current formulation is restricted to planar bilateral symmetry and does not yet cover richer structural priors such as rotational symmetries, multiple axes, or learned symmetry groups. Fourth, the most effective MFA insertion depth may vary across backbone families and scales, and early-layer insertion remains unstable in our current implementation. Finally, our study uses matched pretraining from scratch and single-forward-pass inference; ImageNet-initialized pretraining and flip test-time averaging are complementary baselines that should be compared in larger-scale follow-up studies. The residual inference protocol should be regarded as an empirical property of the present design rather than a general guarantee for stronger fusion variants.

5.0.2 Conclusion.↩︎

We presented Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a reflection-aware training framework that augments standard SSL with mirror-paired supervision and selective mid-layer interaction, without requiring a redesigned backbone. Across the evaluated settings, the results indicate that soft symmetry guidance can complement invariance-driven self-supervision and improve representation quality when bilateral structure is informative. These findings suggest that lightweight, geometry-aware training priors are a promising direction for future self-supervised representation learning in vision.

Appendix↩︎

6 Relation of MFA to Reflection Equivariance↩︎

We briefly relate MFA to equivariance theory in order to clarify its design objective. Let \(\mathcal{R}\) denote horizontal reflection, and let \(f_\theta : \mathcal{X} \rightarrow \mathcal{Z}\) be an encoder. Standard invariance-based self-supervised learning encourages \[f_\theta(\mathcal{R}x) = f_\theta(x), \label{eq:app95inv}\tag{10}\] which removes reflection-related variation by mapping an image and its reflected version to the same representation. By contrast, strict reflection equivariance requires \[f_\theta(\mathcal{R}x) = \rho(\mathcal{R}) f_\theta(x), \label{eq:app95eq}\tag{11}\] where \(\rho(\mathcal{R})\) denotes the induced action in feature space. This preserves reflection structure by requiring the representation to transform predictably under reflection.

However, real bilaterally structured data are usually only approximately symmetric. In chest radiographs, brain MR images, and faces, global bilateral regularity often coexists with meaningful local asymmetry. As a result, strict invariance may suppress useful left–right differences, whereas exact equivariance everywhere may be unnecessarily rigid. MFASSL is designed for this intermediate regime: rather than enforcing a hard group-equivariant constraint, it introduces a soft reflection-aware bias that encourages reflection-consistent processing where bilateral correspondence is reliable while preserving informative asymmetric evidence.

This behavior can be interpreted through a local symmetric–discrepant decomposition. For a mirror-aligned token pair at spatial position \(i\), define \[S_i = \frac{1}{2}(X_{L,i} + X_{R,i}), \qquad \Delta_i = \frac{1}{2}(X_{L,i} - X_{R,i}), \label{eq:app95decomp}\tag{12}\] so that \[X_{L,i} = S_i + \Delta_i, \qquad X_{R,i} = S_i - \Delta_i. \label{eq:app95decomp2}\tag{13}\] Substituting these expressions into the MFA update \[Z_{L,i} = X_{L,i} + g_i \alpha A_{L\leftarrow R,i} + \gamma(X_{L,i} - X_{R,i}), \label{eq:app95mfa}\tag{14}\] yields \[Z_{L,i} = S_i + (1 + 2\gamma)\Delta_i + g_i \alpha A_{L\leftarrow R,i}. \label{eq:app95local}\tag{15}\] Here, \(A_{L\leftarrow R,i}\) denotes the cross-mirror attention output at token \(i\), while \(\alpha\) and \(\gamma\) are learnable scalars controlling the strength of gated fusion and discrepancy preservation, respectively.

Equation 15 clarifies the role of each term. The base term \(S_i\) captures locally shared bilateral structure. The discrepancy term \((1+2\gamma)\Delta_i\) preserves left–right differences, and the gated cross-attention term adds mirror context only when correspondence is supported by the data. In this sense, MFA does not collapse the representation into pure reflection invariance: it retains a pathway through which local asymmetry can remain visible after fusion.

The gate \(g_i\) makes this mechanism spatially adaptive. Because \(g_i\) is explicitly parameterized as a monotone function of token discrepancy, it increases when mirror-aligned tokens are similar and decreases as local mismatch grows. The learnable scalars \(a\) and \(b\) control the effective threshold and sensitivity of this transition during training. As a result, MFA exchanges information more strongly across regions with reliable bilateral correspondence, while suppressing cross-mirror fusion when local asymmetry is pronounced; in the latter case, the discrepancy-preserving residual helps retain informative differences rather than averaging them away.

This interpretation is reinforced by the training objective. The mirror-paired construction aligns bilateral regions into a shared token coordinate system, and the losses \(\mathcal{L}_{\mathrm{eq}}\) and \(\mathcal{L}_{\mathrm{mid}}\) encourage global and token-level agreement before fusion at the same layer where MFA operates. Under good bilateral alignment, these terms reduce the mismatch between corresponding mirror tokens, making the cross-mirror interaction in MFA more reliable. When asymmetry is meaningful, the residual branch allows controlled deviation from exact agreement rather than forcing all bilateral differences to vanish.

Accordingly, MFASSL should not be interpreted as instantiating a fixed feature-space action \(\rho(\mathcal{R})\) in the strict group-equivariant sense. Instead, it provides a data-dependent and locally adaptive relaxation of reflection-equivariant behavior for approximately bilateral data: reflection-consistent correspondence is encouraged where the data support it, while informative asymmetry is preserved where exact symmetry does not hold.

7 Additional Experimental Results↩︎

7.1 Full BraTS Subregion Breakdown↩︎

For readability, we split the full BraTS results into two tables. Table 7 reports Dice scores, and Table 8 reports HD95 and calibration metrics. As shown in these tables, MFASSL improves mean Dice for DINO, MoCo-v3, and MAE, and reduces mean HD95 across all three SSL backbones. The calibration results show lower NLL for all three backbones, while ECE is improved for DINO and MoCo-v3 and remains comparable for MAE.

7.2 Full Multi-Architecture Results↩︎

For completeness, we report the full fine-tuning metrics on CheXpert and the full segmentation metrics on BraTS for both ViT-S/16 and ViT-B/16. Table 9 gives the CheXpert fine-tuning results, Table 10 gives the BraTS Dice results, and Table 11 gives the corresponding BraTS HD95 and calibration results. Across both ViT scales, MFASSL consistently improves CheXpert AUROC, AUPRC, F1, and flip consistency, and it also improves BraTS mean Dice while reducing mean HD95.

8 Reproducibility Notes↩︎

8.1 Consolidated Hyperparameter Settings↩︎

Table 12 summarizes the consolidated hyperparameter settings used in the reported experiments. Settings marked “per official recipe” follow the unmodified MoCo-v3, DINO, or MAE configuration for the corresponding backbone and are not changed by MFASSL. The MFASSL-specific settings are selected on the CheXpert validation set and then fixed across datasets, SSL backbones, and ViT scales unless otherwise stated.

8.2 Full Component-Ablation Metrics↩︎

For completeness, Table ¿tbl:tab:app95combined95ablation95full? reports the full metric version of the component-ablation study summarized in the main paper.

References↩︎