1 Introduction↩︎

Metabolic dysfunction-associated steatotic liver disease (MASLD), previously known as non-alcoholic fatty liver disease (NAFLD), is the most common chronic liver disease worldwide, affecting nearly one-third of the global population [1], [2]. Its progressive subtype, metabolic dysfunction-associated steatohepatitis (MASH), formerly referred to as non-alcoholic steatohepatitis (NASH), can lead to fibrosis, cirrhosis, hepatocellular carcinoma, and liver failure if left untreated [3]. Early detection is therefore essential for enabling timely therapeutic intervention and preventing irreversible liver damage [4]. For consistency with the clinical labels used in this study and prior literature, we subsequently refer to these conditions as NAFLD and NASH.

Although liver biopsy remains the clinical reference standard for diagnosing and staging NASH, its invasive nature, sampling variability, procedural risks, and cost limit its use for large-scale screening and longitudinal monitoring [5]. Consequently, ultrasound has emerged as one of the most widely used imaging modalities for liver disease assessment due to its low cost, safety, portability, and widespread availability [4], [5]. However, accurate ultrasound-based classification of NAFLD and NASH remains challenging. Conventional B-mode ultrasound images are affected by speckle noise, low contrast, acoustic artifacts, operator dependence, and variability across imaging systems and acquisition protocols [4]–[9]. Moreover, early-stage disease is often characterized by subtle parenchymal changes that may not be readily visible on standard B-mode imaging, limiting diagnostic sensitivity and contributing to variability in clinical interpretation [4].

To address the limitations of conventional ultrasound imaging, numerous artificial intelligence (AI) approaches have been proposed for automated liver disease assessment. Deep learning methods, including convolutional neural networks (CNNs), transformers, radiomics-based frameworks, and self-supervised learning models, have demonstrated promising performance for steatosis detection, fibrosis staging, and liver disease classification [6]–[12]. Prior studies have explored CNN-based approaches for fatty liver disease detection and NAFLD classification from B-mode ultrasound images [6]–[12], while others have incorporated enhanced ultrasound representations to improve classification performance using CNNs [9]. Despite these advances, differentiating NASH from NAFLD remains particularly challenging because disease-related imaging differences are often subtle and inadequately captured by conventional B-mode ultrasound. Moreover, existing studies [6]–[12] have predominantly employed supervised CNN-based architectures focused on steatosis grading or broad liver disease classification, with no prior work investigating ultrasound-based NASH versus NAFLD differentiation as a dedicated diagnostic task.

In this work, we investigate the use of complementary ultrasound representations for differentiating NASH from NAFLD. Specifically, we integrate conventional B-mode imaging with physics-guided and local phase-based image representations and evaluate their effectiveness using two learning paradigms that have not previously been explored for this task: self-supervised masked autoencoders (MAEs) [13] and graph convolutional networks (GCNs) [14]. The proposed framework is evaluated on a multi-site Mayo Clinic cohort comprising 2,547 liver ultrasound scans from 125 patients. Experimental results demonstrate consistent performance improvements over conventional B-mode ultrasound alone across demographic subgroups and acquisition sites. Our contributions include: 1- We present the first study investigating complementary ultrasound representations for AI-based differentiation of NASH and NAFLD. 2- We evaluate, for the first time, MAEs and GCNs for ultrasound-based NASH versus NAFLD classification. 3- We perform a comprehensive multi-site evaluation, including subgroup analyses across age, sex, race, ethnicity, and acquisition site, demonstrating consistent improvements over conventional B-mode ultrasound imaging.

2 Patients and Methods↩︎

2.1 Data Sources and Study Population↩︎

De-identified DICOM ultrasound examinations were organized at the patient level and linked to corresponding clinical and pathological records. The reference standard diagnosis was established using liver biopsy results. The dataset, provided by the Mayo Clinic, consists of 2,547 liver ultrasound scans acquired from 125 patients, including 1,274 NAFLD scans and 1,273 NASH scans collected from multiple Mayo Clinic sites. Some patients were referred to Mayo Clinic after undergoing ultrasound examinations at external healthcare institutions.

2.2 Complementary Ultrasound Representations (CUR)↩︎

To provide complementary information for downstream learning, each ultrasound examination was represented using three image modalities: conventional B-mode ultrasound (BUS), a quality-improved ultrasound representation (QiUS), and an enhanced ultrasound representation (EnUS) (Fig. 1).

-12 -0.5em Quality-Improved Ultrasound (QiUS). QiUS images are generated using a physics-guided attenuation characterization framework that models ultrasound signal propagation within tissue. The method estimates spatially varying transmission and attenuation characteristics and applies contextual regularization to compensate for signal degradation caused by acoustic attenuation. The resulting representation enhances attenuation-dependent tissue information while reducing acquisition-related artifacts and image inconsistencies commonly encountered in ultrasound imaging (Fig. 1). The concept was previously utilized for breast, bone shadow and needle shaft/tip enhancement from ultrasound data [15]–[17] and is based on the patented work of [18]. The regularization parameter was set to 2, consistent with previous studies [15], [16], [18]. The tissue echogenicity constant was defined as 80% of the maximum intensity value of the corresponding B-mode ultrasound (BUS) image, following the methodology described in [15], [16], [18].

-12 -0.5em Enhanced Ultrasound (EnUS). EnUS images are generated through local phase-based feature extraction in the frequency domain through band-pass quadrature filtering. Specifically, alpha-scale space derivative (ASSD) filtering is employed [9], [15], [16], [18], to BUS image, to derive local phase representations that emphasize structural and morphological information while remaining relatively invariant to image intensity, scanner settings, and acoustic gain [9], [15], [16], [18]. Consequently, the EnUS representation enhances tissue interfaces, parenchymal texture, and anatomical boundaries that may not be readily visible in conventional B-mode images (Fig. 1). The ASSD filter is defined as \[\mathrm{ASSDF}(\omega)= \begin{cases} n_{c}\,\omega^{a}\exp\!\bigl(-(\sigma\omega)^{2\alpha}\bigr) & \omega>0,\\[2pt] 0 & \text{otherwise.} \end{cases} \label{eq:assd}\tag{1}\] where \(n_{c}\) is a unit normalization constant calculated from the filter \(\alpha\) value. During this work following parameters were used: filter alpha-scale parameter \(\sigma = 25\) pixels, \(a = 1.83\) and constant derivative parameter \(\alpha = 0.2\). During \(n_{c}\) calculation we have used number of scales \(s = 2\) and number of orientations \(= 6\) (\(30^{\circ}\) increments starting from \(0^{\circ}\)).

Complete methodological details and validation of the QiUS and EnUS generation processes can be found in previous publications [9], [15], [16], [18].

The QiUS and EnUS representations provide complementary views of the same ultrasound acquisition. Whereas QiUS emphasizes acoustic propagation and attenuation-related characteristics, EnUS captures structural and morphological information derived from local phase analysis (Fig. 1). Together with the original BUS image, these representations form a set of complementary ultrasound representations (CUR) that provide distinct yet synergistic descriptions of tissue appearance, enabling downstream learning models to leverage both acoustic and structural information for liver tissue characterization.

2.3 Learning with Complementary Ultrasound Representations↩︎

To evaluate the proposed CUR approach, we investigate two learning paradigms. First, a self-supervised masked autoencoder (CUR-MAE) is employed to learn robust representations from BUS, QiUS, and EnUS inputs. Second, a graph convolutional network (CUR-GCN) is utilized to model relationships among ultrasound examinations through graph-based learning. Both models leverage the complementary acoustic and structural information encoded within the CUR representations for NASH versus NAFLD classification.

2.3.1 CUR-Masked Autoencoder (CUR-MAE) Network↩︎

As illustrated in Figure 2, the three ultrasound representations derived from the same acquisition are first combined through a CUR Mixing module inspired by channel-mixing operations [19]. The channels from BUS, EnUS, and QiUS images are concatenated, randomly shuffled within each image patch, and partially sampled to generate mixed CUR-feature inputs (Fig. 2). This process encourages the network to learn feature interactions across representations while reducing dependence on any single image type. The resulting mixed images are partitioned into \(16\times16\) patches and embedded into a latent feature space using a Vision Transformer (ViT) encoder. Positional embeddings are added to preserve spatial information, and the encoder processes the patch sequence through 12 transformer blocks comprising multi-head self-attention, multilayer perceptrons, normalization layers, and residual connections (Fig. 2). The self-attention mechanism enables the model to capture long-range dependencies and relationships among anatomical structures across the entire image. During self-supervised pre-training, randomly masked image regions are reconstructed, forcing the encoder to learn informative structural, textural, and acoustic characteristics shared across the BUS, EnUS, and QiUS representations. After pre-training, the decoder is discarded and only the learned encoder is retained for downstream diagnostic classification. Final classification is performed using global average pooling followed by a fully connected classification layer.

To improve representation learning and mitigate the limited availability of labeled liver ultrasound data, the MAE was first pretrained on more than 230,000 publicly available de-identified ultrasound images and videos collected from multiple open-source datasets [20]–[29]. For CUR-MAE, each ultrasound acquisition was represented using three complementary image modalities (BUS, QiUS, and EnUS), effectively tripling the number of training representations available during pretraining and enabling the model to learn richer acoustic and structural features.

2.3.2 CUR-Graph Convolutional Network (CUR-GCN) Architecture↩︎

Unlike conventional image-based classifiers that process each examination independently, CUR-GCN explicitly models relationships among examinations by representing them as nodes in a similarity graph.

For each ultrasound representation, a dedicated CNN encoder extracts a 512-dimensional feature vector. The resulting BUS, EnUS, and QiUS features are concatenated to form a 1,536-dimensional multi-feature representation, which is subsequently compressed through a multilayer perceptron (MLP) into a unified 256-dimensional embedding. This fused representation serves as the node descriptor for graph construction (Fig. 2).

A \(k\)-nearest-neighbor (\(k\)-NN) graph is then generated using cosine similarity between fused feature vectors, with \(k = 7\). This value was selected through preliminary validation experiments to balance local neighborhood consistency and global graph connectivity, enabling effective information propagation across clinically similar examinations. By incorporating information from all three ultrasound representations during graph construction, the resulting graph exhibits improved homophily compared with graphs generated from a single image representation, thereby reducing fragmentation caused by image variability and noise.

CUR-GCN classification network consists of a graph convolution layer followed by layer normalization, ReLU activation, dropout, and a fully connected classification layer. During training, edge dropout is applied to improve robustness and reduce overfitting [14]. Through graph-based message passing, CUR-GCN aggregates information from neighboring examinations, enabling more reliable classification than single-feature graph learning approaches.

3 Results↩︎

3.1 Implementation Details↩︎

Model performance was evaluated using patient-level three-fold cross-validation. To prevent data leakage, all ultrasound examinations from a given patient were assigned exclusively to a single fold. Stratified sampling was employed to preserve the NASH and NAFLD class distributions across the training, validation, and test subsets. Within each fold, patients were split into training (70%), validation (10%), and testing (20%) sets, and identical data partitions were used for all models to ensure fair comparison. Performance metrics were reported as the mean and standard deviation across three-fold cross-validation folds. To assess robustness across patient populations and acquisition environments, subgroup analyses were performed based on age, sex, race, ethnicity, and acquisition site; accuracy for each subgroup was computed by pooling patient-level predictions from the held-out test sets across all folds. The CUR-GCN model was trained for 125 epochs using cross-entropy loss and the AdamW optimizer, with the model achieving the highest validation accuracy selected for final evaluation. The CUR-MAE model was initialized using pretrained weights and fine-tuned for 10 epochs. A limited number of fine-tuning epochs was selected to mitigate overfitting, as longer training resulted in increased validation loss. The initial learning rate was set to \(2\times10^{-4}\) and reduced by a factor of 10 using a step-based schedule. Additional regularization included weight decay (0.05) and layer-wise learning rate decay (0.75). Model parameters were updated after each iteration without gradient accumulation. All experiments were implemented in Python using PyTorch (v2.7.0). Training was performed on NVIDIA A100 GPUs (80 GB) and TPUs (128 GB) with CUDA 12.6 support. Automatic Mixed Precision (AMP) was employed to improve computational efficiency and reduce memory consumption. As a baseline comparison, we additionally implemented a CNN-EBV architecture as a baseline [9], [30], which utilizes the same complementary ultrasound representations (CUR) and an Equiangular Basis Vector (EBV) classifier [30].

3.2 Quantitative Results↩︎

Table 1 summarizes the classification performance of the investigated models. Across all architectures, incorporating CUR consistently improved classification performance compared with conventional B-mode ultrasound. The largest gains were observed for CUR-GCN, which improved accuracy, F1-score, and AUC by 32.4%, 91.2%, and 11.8%, respectively. CUR-MAE also demonstrated substantial improvements, achieving gains of 27.1% in accuracy and 33.8% in F1-score. Analysis of sensitivity and specificity demonstrated that CUR improved disease discrimination across architectures. CUR-GCN achieved the largest sensitivity gain (+111.9%), while CUR-MAE improved both sensitivity (+36.0%) and specificity (+14.7%), suggesting improved identification of both NASH and NAFLD cases. These findings indicate that complementary ultrasound representations provide discriminative information beyond conventional B-mode imaging and improve the differentiation of NASH and NAFLD. Unlike the MAE and GCN frameworks, the CNN-EBV baseline primarily exhibited a tradeoff between sensitivity and specificity following incorporation of CUR, resulting in unchanged F1-score despite improved accuracy. This observation suggests that the benefits of complementary ultrasound representations depend on the downstream learning architecture, with representation-learning and graph-based models appearing better suited to exploit the additional acoustic and structural information encoded within CUR.

3.3 Metadata Analysis↩︎

Figure 3 summarizes classification accuracy across demographic subgroups and acquisition sites. Overall, incorporating complementary ultrasound representations (CUR) consistently improved performance relative to conventional B-mode ultrasound across nearly all age, ethnicity, race, location, and sex categories. The largest gains were observed for the GCN-based framework, particularly among patients aged 40–60 years (0.67\(\rightarrow\)​0.92), patients scanned at the Florida site (0.44\(\rightarrow\)​0.89), and female patients (0.62\(\rightarrow\)​0.81). Similarly, CUR-MAE demonstrated consistent improvements across all evaluated subgroups, achieving notable gains in Hispanic or Latino patients (0.71\(\rightarrow\)​1.00), male patients (0.80\(\rightarrow\)​0.93), and examinations acquired at the Florida site (0.44\(\rightarrow\)​0.67). Importantly, no demographic subgroup exhibited a substantial performance degradation following incorporation of CUR, suggesting that the complementary representations provide robust and generalizable information across diverse patient populations and acquisition environments. These findings indicate that the benefits of CUR extend beyond overall classification performance and remain consistent across demographic and site-specific strata.

4 Discussion and Future Work↩︎

This study investigated the use of complementary ultrasound representations (CUR) for differentiating NASH from NAFLD using self-supervised MAEs and GCNs. Across all evaluated architectures, CUR consistently improved classification performance relative to conventional B-mode ultrasound, with the largest gains observed for CUR-GCN. Specifically, CUR-GCN achieved improvements of 32.4% in accuracy, 91.2% in F1-score, and 11.8% in AUC, while CUR-MAE improved accuracy and F1-score by 27.1% and 33.8%, respectively. Analysis of operating characteristics further revealed that CUR-GCN achieved the largest sensitivity improvement (+111.9%), whereas CUR-MAE improved both sensitivity (+36.0%) and specificity (+14.7%), indicating more balanced discrimination between NASH and NAFLD. Furthermore, metadata analysis demonstrated consistent performance improvements across age, sex, race, ethnicity, and acquisition-site subgroups, suggesting that the benefits of CUR are not restricted to a particular patient population or imaging environment.

Several limitations should be acknowledged. First, although the dataset comprised 2,547 ultrasound scans, these data were acquired from only 125 patients. Consequently, some demographic subgroups contained relatively small numbers of patients, and subgroup-specific results should therefore be interpreted with caution. While certain subgroups achieved near-perfect performance, these observations require validation in larger and more balanced cohorts. Second, several evaluation metrics exhibited relatively large standard deviations across cross-validation folds, particularly for AUC. This variability likely reflects the limited cohort size and the inherent heterogeneity of liver ultrasound imaging. Although consistent improvements were observed across architectures and demographic groups, larger cohorts are needed to obtain more stable estimates of model performance. Third, the study was conducted retrospectively using data from a single healthcare system. Although the cohort included examinations acquired across multiple Mayo Clinic locations and a subset of patients referred from external institutions, additional external validation is required to assess generalizability across different scanners, imaging protocols, and patient populations.

Future work will focus on expanding the study through larger multi-institutional cohorts and external validation datasets as well as investigating multimodal learning frameworks that combine complementary ultrasound representations with language-based features extracted from radiology reports to further improve non-invasive NASH versus NAFLD classification. In addition, future studies will explore the utility of the proposed framework for fibrosis staging, longitudinal disease monitoring, and risk stratification. These efforts will help establish the clinical utility of complementary ultrasound representations for non-invasive liver disease assessment and support the development of robust AI tools for differentiating NASH from NAFLD.

-12 -0.5em Acknowledgement. Research reported in this work was supported by the National Institute on Minority Health and Health Disparities of the National Institutes of Health under award number 1R43MD020006-01A1. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We would like to thank the Mayo Clinic and the Mayo Accelerate Program for their valuable support and collaboration throughout this project. Their guidance, expertise, and provision of data were instrumental in the development and evaluation of our complementary ultrasound representation learning framework.

-12 -0.5em Ethics Statement. This study involves analysis of de-identified data via the Mayo Clinic Discover Platform. In accordance with the Code of Federal Regulations, 45 CFR 46.102, the noted activity does not require IRB review. Data shown and reported in this manuscript have been extracted from this environment using an established protocol for data extraction, aimed at preserving patient privacy. The data has been de-identified pursuant to an expert determination in accordance with the HIPAA Privacy Rule. Any data beyond what is reported in the manuscript, including but not limited to the raw electronic health record data, cannot be shared or released due to the parameters of the expert determination to maintain the data de-identification.

References↩︎