1 Introduction↩︎

Disorders such as autism spectrum disorder (ASD) are driven by complex interactions between genetic risk factors and brain morphology [1]. Understanding these relationships is crucial for biomarker identification; however, acquiring both neuroimaging and genetics data from the same individuals is costly and often impractical at scale. As a result, the largest public resources in precision psychiatry are non-overlapping and modality specific. For example, the Autism Brain Imaging Data Exchange (ABIDE) [2] offers comprehensive neuroimaging data, while the Simons Simplex Collection (SSC) [3] provides extensive genomic data for ASD for a different set of participants. Current multimodal analyses require paired samples and cannot directly be applied to these large datasets.

Traditional imaging-genetics is based on linear models between a restricted set of neuroimaging and genetics features [4]. This paradigm struggles to capture the complex dependencies associated with neuropsychiatric disorders [5]. Deep learning offers a promising alternative for imaging-genetics. For instance, latent fusion methods combine low-dimensional encodings from each modality [6], [7]; however, the information compression obscures the link between brain regions and genomic features, thus limiting biological interpretability. Early fusion approaches using cross-attention can explicitly model such interactions [5], [8], [9], but they cannot enforce consistency in learned associations across subjects, and they often require models with a large number of parameters [8], [9]. Graph-based methods have been used to incorporate a priori biological knowledge [10] but operate at the gene level and are prone to overfitting on small samples. Critically, these approaches require paired data from the same individuals, whereas public neuroimaging and genetic repositories are largely disjoint.

Recent works have addressed the problem of unpaired multimodal learning, but at the cost of interpretability [11]. For example, spectral embedding methods such as SUE [12] align the modalities through a canonical correlation analysis and distribution matching, but the learned representations lack clear biological meaning. Additive fusion strategies require both modalities to share identical dimensionality, similarly obscuring interpretable structure [6], [13]. In contrast, optimal transport (OT) methods compute a minimum-cost coupling between two distributions [14], [15] and is the most natural when distributions describe comparable entities under a shared geometry. However, brain morphometry and genetic pathways are fundamentally different entities, and no biologically grounded metric exists to quantify distances between them, making the transport cost dependent on modeling assumptions rather than biological structure.

To address these limitations, we propose Class-conditional Alignment with Linear Maps (CALM), a framework that discovers interpretable associations between completely unpaired neuroimaging and genetics datasets. CALM pretrains modality-specific encoders that preserve diagnostic groupings while ensuring a direct link between latent dimensions and neurobiological features. Cross-modal alignment is implemented as low-dimensional linear mappings in the latent space that optimize both a class-conditional alignment loss to match the latent imaging and genetics distributions within each diagnostic group, and a contrastive loss to pull same-class samples together while pushing apart different-class samples, regardless of modality. We validate CALM by training on ABIDE (neuroimaging) and SSC (genetics) and testing on the independent and paired ACE dataset. We show that associations recovered from unpaired data closely agree with those obtained from paired data and remain stable across cross-validation folds. CALM lays a crucial foundation to use independent unimodal datasets for cross-modal biomarker discovery, thus capitalizing on existing data resources.

2 Interpretable Linear Alignment for Unpaired Data↩︎

CALM tackles the problem of discovering biomarker associations between neuroimaging and genetics data, when the modalities are collected from separate and non-overlapping groups of subjects. Our framework is outlined in Fig. 1.

The neuroimaging input to CALM consists of region-wise morphological features from structural MRI. Formally, let \(\mathbf{S}^{(i)} \in \mathbb{R}^{n_I \times n_F}\) denote the neuroimaging input for subject \(i\), where \(n_I\) is the number of brain regions, and \(n_F\) is the number of morphological features. The genetics input to CALM is obtained by aggregating genetic risk scores across established biological pathways [16]. Due to the heterogeneity of neuropsychiatric disorders, we include pathway-level features for several phenotypic traits. Details of this process are provided in Section 3. Formally, let \(\mathbf{P}^{(j)} \in \mathbb{R}^{n_G \times n_T}\) denote the genetics input for subject \(j\), where \(n_G\) and \(n_T\) are the number of genetic pathways and phenotypic traits, respectively.

2.1 Data Encoding↩︎

CALM uses pretrained autoencoders and classifiers to construct the latent representations for each modality. The loss combines reconstruction quality, classification accuracy, and a contrastive term that encourages latent representations from the same cohort to cluster together. This cohort-level grouping enables CALM to use linear alignment maps to discover stable correspondences between the two modalities. Crucially, the encoders consist of small per-entity layers that preserve the region-level information from the neuroimaging input and pathway-level information from the genetics input. This strategy preserves a one-to-one correspondence between rows and neurobiological entities, thus maintaining the interpretability of CALM. Let \(\mathbf{X}_I^{(i)} \in \mathbb{R}^{n_I \times d}\) be the latent neuroimaging representation for subject \(i\) and \(\mathbf{X}_G^{(j)} \in \mathbb{R}^{n_G \times d}\) be the latent genetics representation for subject \(j\). The latent dimension \(d\) is shared between the modalities.

2.2 Linear Projection↩︎

Given the latent encodings \(\mathbf{X}_I^{(i)}\) and \(\mathbf{X}_G^{(j)}\), we map the modalities into a shared
latent space using learnable linear projections \(\mathbf{W}_I \in \mathbb{R}^{d \times n_I}\) and \(\mathbf{W}_G \in \mathbb{R}^{d \times n_G}\): \[\label{eq:encoding} \mathbf{Z}_I^{(i)} = \mathbf{W}_I \mathbf{X}_I^{(i)}, \quad \mathbf{Z}_G^{(j)} = \mathbf{W}_G \mathbf{X}_G^{(j)}\tag{1}\]

For convenience, we flatten the matrices in Eq. 1 into the vectors: \(\mathbf{z}_I^{(i)} = \text{vec}(\mathbf{Z}_I^{(i)})\) and \(\mathbf{z}_G^{(j)} = \text{vec}(\mathbf{Z}_G^{(j)})\) when defining the distributional loss in the next section. The association matrix \(\mathbf{A} = \mathbf{W}_I^\top \mathbf{W}_G \in \mathbb{R}^{n_I \times n_G}\)quantifies the similarity between cross-modal projection vectors, i.e., \(\mathbf{A}_{r,p}=\mathbf{w}_{I,r}^\top\mathbf{w}_{G,p}\). As seen, \(\mathbf{A}_{r,p}\) is large when ROI \(r\) and pathway \(p\) load along the same latent directions, indicating they co-vary in the aligned space. This formulation parallels that of canonical correlation analysis (CCA) but does not require paired samples.

2.3 Class-Conditional Distribution Alignment↩︎

CALM leverages a compound objective function to align the imaging and genetic distributions without requiring paired multi-modal samples: \[\mathcal{L}_{total} = \lambda_{cmmd} \mathcal{L}_{cmmd} + \lambda_{con} \mathcal{L}_{con} + \lambda_{orth} \mathcal{L}_{orth} \label{eq:total95loss}\tag{2}\] The loss terms in Eq. 2 address three challenges of unpaired learning: stabilizing geometric alignment, preventing negative transfer between diagnostic groups, and enforcing semantic discriminability in the shared latent space. The hyperparameters \(\lambda_{cmmd}, \lambda_{con}, \lambda_{orth}\) control the contribution of each term.

Distributional Loss (\(\mathcal{L}_{cmmd}\)): We use Class-Conditional Maximum Mean Discrepancy (CMMD) to align the distributions of latent neuroimaging and genetics encodings within each diagnostic group. Unlike the standard MMD [17], which is label agnostic, \(\mathcal{L}_{cmmd}\) calculates the discrepancy separately for each class \(c\): \[\mathcal{L}_{cmmd} = \sum_{c \in \mathcal{C}} \left\| \frac{1}{N_c} \sum_{i \in c} \phi(\mathbf{z}_I^{(i)}) - \frac{1}{M_c} \sum_{j \in c} \phi(\mathbf{z}_G^{(j)}) \right\|_{\mathcal{H}}^2, \label{eq:cmmd}\tag{3}\] where \(\phi(\cdot)\) maps features to a reproducing kernel Hilbert space (RKHS), and \(N_c, M_c\) denote the number of imaging and genetic samples in class \(c\), respectively.

In our experiments, the classes \(c \in \mathcal{C}\) are ASD patients and neurotypical controls (NC), but the CMMD loss can be generalized to other scenarios.

Contrastive Loss (\(\mathcal{L}_{con}\)): While \(\mathcal{L}_{cmmd}\) aligns the within-class distributions, it does not enforce a separation between classes in the shared latent space. Thus, we employ a supervised contrastive loss [18] that treats all same-class samples as positives regardless of modality. Let \(\mathcal{B}\) denote the set of latent representations from both modalities included within a mini-batch. For an anchor \(\mathbf{z}^{(i)}\) with label \(y_i \in \mathcal{C}\), the positive set contains all other samples sharing the same label, \(\mathcal{P}(i) = \{j \neq i \in \mathcal{B} : y_j = y_i\}\). The contrastive loss is computed \[\mathcal{L}_{con} = - \sum_{i \in \mathcal{B}} \frac{1}{|\mathcal{P}(i)|} \sum_{p \in \mathcal{P}(i)} \log \frac{\exp(\mathrm{sim}(\mathbf{z}^{(i)}, \mathbf{z}^{(p)}) / \tau)}{\sum_{j \in \mathcal{B} \setminus \{i\}} \exp(\mathrm{sim}(\mathbf{z}^{(i)}, \mathbf{z}^{(j)}) / \tau)}, \label{eq:con}\tag{4}\] where \(\tau\) is a temperature parameter, and \(|\mathcal{P}(i)|\) is the total number of positive samples for anchor \(\mathbf{z}^{(i)}\). Eq. 4 simultaneously pulls samples with the same class together across modalities and pushes apart samples in different classes. This procedure encourages a shared discriminative structure in the latent space.

Regularization (\(\mathcal{L}_{orth}\)): To prevent degenerate solutions for the projection matrices, we use regularization to encourage orthonormality in the rows: \[\mathcal{L}_{orth} = \left\| \mathbf{W}_{I} \mathbf{W}_{I}^\top - \mathbf{I} \right\|_F^2 + \left\| \mathbf{W}_{G} \mathbf{W}_{G}^\top - \mathbf{I} \right\|_F^2 \label{eq:orth}\tag{5}\] where \(\|\cdot\|_F\) denotes the Frobenius norm, and \(\mathbf{I}\) is the identity matrix. Each row of \(\mathbf{W}_{I}\) and \(\mathbf{W}_{G}\) defines the projection of a single latent dimension across input features. Encouraging their outer products to be identity ensures that the projections remain orthogonal with unit-norm. This prevents rank collapse and encourages each latent dimension to capture non-redundant information.

2.4 Implementation Details↩︎

CALM ¹ follows a two-stage training procedure: modality-specific encoders and classifiers are pretrained for 50 epochs, the classifiers are frozen while the linear projection matrices are trained for cross-modal alignment over 30 epochs. The inputs \(\mathbf{S^{(i)}}\) and \(\mathbf{P^{(j)}}\) are z-score normalized using per-feature statistics computed solely from the training set. The hyperparameters in Eq. 2 were set to \(\lambda_{cmmd} = 5\), \(\lambda_{con} = 5\), and \(\lambda_{orth} = 0.01\), with a contrastive temperature of \(\tau = 0.07\). The projector learning rate was set to \(1 \times 10^{-3}\), with a reduced rate of \(1 \times 10^{-5}\) for fine-tuning the encoders. We used layer normalization, a batch size of 64, and applied Gaussian feature noise (\(\sigma = 0.15\)) to the imaging modality for regularization. The shared latent dimension was set to \(d = 6\).

Class Prediction on Paired Test Data: The distributional and contrastive losses refine both encoders during the second stage of training, so the updated representations now capture cross-modal structure. As a quantitative validation, we use the updated encoders and previously frozen stage one classifiers to predict the diagnosis of unseen test subjects. As described in Section 3, our test cohort contains paired data. Thus, the final prediction is obtained by averaging the output probabilities for both the neuroimaging and genetics classifiers.

Baseline Comparisons: We evaluate the quantitative performance of CALM in predicting diagnostic labels against several state-of-the-art multimodal baselines. G-MIND [6] and UNSEEN [13] both project modality-specific latent encodings into a shared latent space and merge them through summation. Unlike CALM, these methods blur information across input features, thus limiting our ability to extract feature-wise associations between the modalities. SUE [12] aligns independently learned spectral embeddings across modalities without paired samples, but also lacks interpretable cross-modal associations. For consistency, all baselines use the same pathway inputs and imaging feature representations as CALM. We also compare CALM against an optimal transport ablation that replaces the \(\mathcal{L}_{cmmd}\) loss with a global Sinkhorn divergence [14]. Finally, we include two other ablations of CALM, omitting the distributional loss \(\mathcal{L}_{cmmd}\) and omitting the contrastive loss \(\mathcal{L}_{con}\), to assess their contributions.

3 Data and Preprocessing↩︎

We evaluate CALM using four publicly available datasets for ASD. For training, we use neuroimaging data from the ABIDE I and II datasets [2] (\(N=1,898\); \(889\) ASD) and genetics data from SSC [3] (\(N=2,452\)). For SSC simplex families, we use proband genotypes as ASD samples and construct pseudo-controls from untransmitted parental alleles [19]. For testing, we use the independent ACE² dataset (\(N=198\); \(105\) ASD), which contains paired MRI and genetic data. Neuroimaging: T1-weighted MRI from ABIDE and ACE were preprocessed using FreeSurfer [20] and parcellated according to the Brainnetome atlas [21] (\(n_{I} = 246\) ROIs). For each ROI, we extract \(n_{F}=4\) structural features: volume size, surface area, average cortical thickness, and standard deviation of the cortical thickness. We use ComBat [22] to remove ABIDE site effects. The input features \(\mathbf{S}^{(i)}\) were normalized based on statistics from the training set.

5pt

Genetics: SSC and ACE participants were genotyped on different Illumina chips. Quality control was performed separately per batch and cohort using PLINK [23]. Analyses were restricted to a European subpopulation identified via projection onto the first three principal components of the 1,000 Genomes reference panel [24]. We used the genetics preprocessing steps outlined in [25], which after LD clumping, yielded 44,102 SNPs. The SNPs were grouped into 3,907 genes defined by the \(n_P = 177\) non-cancer pathways in the KEGG database [16].

Following [8], each entry \(\mathbf{P}^{(n)}_{gt}\) in the input \(\mathbf{P}^{(n)} \in \mathbb{R}^{n_G \times n_T}\) corresponds to the genetic risk for a single pathway \(g\) and phenotypic trait \(t\). This value is obtained by first selecting the SNPs associated with the pathway genes, weighting them by the effect sizes obtained from a published GWAS [26], and computing a weighted sum across SNPs. We capture genetic risk across multiple phenotypes by leveraging the effect sizes from \(m=6\) independent GWAS spanning disorder-related traits: ASD [27], [28], ADHD [29], educational attainment (EA) [30], intelligence [31], major depression disorder (MDD) [32], and epilepsy [33].

4 Experimental Results↩︎

Quantitative Disease Prediction: Table 1 reports the ASD vs. NC classification performance on the ACE dataset when the models are trained on ABIDE and SSC within a 5-fold CV setup. As seen, CALM achieves the best performance across most metrics, with statistically significant improvement over G-MIND, UNSEEN, and SUE. While below the significance threshold, we note the consistent improvement of CALM over its model ablations. This improvement can be attributed to the complementary roles of the two loss terms.

Association Validation: To validate that CALM learns pathway–ROI associations consistent with those obtained from true paired data, we replace the class-conditional distribution matching loss (\(\mathcal{L}_{\text{CMMD}}\)) with a paired contrastive loss (InfoNCE [34]) that directly maps modalities from the same subject to similar latent representations. Fig. 2 (a) correlates the entries of the mean association matrix \(\bar{\mathbf{A}}\) between the unpaired CALM and the paired baseline. The strong Pearson correlation (\(r = 0.87\)) indicates that CALM recovers consistent pathway–ROI associations without requiring paired data. We also assess cross-fold stability within the unpaired setting by computing the signal-to-noise ratio (\(\text{SNR} = \mu / \sigma\)) of each association across five folds. Fig. 2 (b) shows that 72% of associations achieve \(|\text{SNR}| > 1\) (stable) and 47% achieve \(|\text{SNR}| > 2\) (reproducible), confirming that most associations are robust across data splits.

Learned Imaging-Genetics Associations: To determine the most salient pathway–brain ROI influences, we first rank pathways by their \(\ell_2\) column norm of the association matrix \(\mathbf{A}\) and select the top four pathways with the strongest brain-wide effects. We then merge bilateral ROIs by summing the association values of their left and right counterparts. For visualization, we display the top four brain ROIs with the highest association strength for each pathway in Fig. 3.

The IgA Immune Network, Wnt Signaling, and Sulfur Metabolism pathways converge on occipital, temporal, and parietal regions, linked to atypical visual and multisensory processing in ASD [35]. These pathways are functionally related through homeostatic regulation of neurodevelopment mechanisms that are well-established in ASD pathology [36]. In contrast, NOD-like Receptor Signaling maps onto different ventral temporal and limbic areas, aligning with socioemotional and face processing deficits in ASD [37].

Channel Attribution Analysis: To quantify the contribution of each phenotypic trait to the shared latent space, we compute gradient-based input attribution [38] through the genetics encoder \(\mathcal{E}_G\) and the learned projection matrix \(\mathbf{W}_G\), averaged across the top four pathways and five cross-validation folds. Fig. 3 (b) shows that ASD, ADHD, EA, and IQ contribute most strongly, while MDD and epilepsy contribute moderately. This is consistent with the well-documented genetic and phenotypic overlap among these neurodevelopmental traits [39].

5 Conclusion↩︎

We have proposed CALM, a framework that learns interpretable cross-modal associations between brain ROI morphology and genetic pathways from entirely unpaired and disjoint populations. The backbone of CALM is a class-conditional linear transport model that aligns the group-wise distributions across modalities in the a shared latent space. We validate the learned associations by comparing them against a paired baseline trained on matched subjects, and we demonstrate the stability of the association matrix. CALM generalizes to a completely unseen external dataset and surpasses state-of-the-art baselines. The discovered pathway-ROI associations are supported by established ASD literature.

References↩︎