3pt

1 Introduction↩︎

CT-based body composition analysis is widely used for assessing musculoskeletal health, metabolic disorders, and disease prognosis [8], [9]. In particular, quantitative assessment of lower extremity tissue volume and spatial distribution, including bone, skeletal muscle, and adipose tissue, can provide valuable imaging biomarkers for diseases such as sarcopenia, osteoporosis, obesity, and peripheral vascular disease [10]–[12]. However, reliable and efficient extraction of these biomarkers requires accurate tissue segmentation.

Manual segmentation and annotation of these tissues are labor-intensive and require domain expertise, which limits the scalability of body composition studies. To reduce this annotation burden, prior studies have explored a range of strategies for maintaining model robustness under limited annotation, including increasing the number of annotated volumes under a fixed annotation budget [13], leveraging domain adaptation across different imaging modalities [14], and using self-supervised or semi-supervised learning with unlabeled data [15], [16].

Beyond methods development, public datasets and released model weights have also played an important role in enabling reproducible evaluation, external validation, and broader adoption of segmentation tools [3], [17], [18]. However, existing public tools remain limited for comprehensive lower extremity CT tissue segmentation.

For example, TotalSegmentator [3] provides related masks such as bone and thigh-based muscle, but these tasks do not constitute a full lower extremity tissue model. Existing public CT body composition models primarily focus on the chest, abdomen, and pelvis rather than the lower extremities [9]. Prior lower extremity CT studies have explored lymphedema tissue segmentation [6], thigh muscle and bone assessment [4], and thigh muscle segmentation [7]. However, these models are often disease-specific, anatomically restricted, or not released for publicly available use. These limitations highlight the demand for a publicly available model for comprehensive lower extremity CT tissue segmentation.

To address this issue, we present LegSegNet, a deep learning framework specifically designed for multi-compartment segmentation of lower extremity CT scans, with public model weights for unified segmentation of bone, skeletal muscle, subcutaneous adipose tissue, and inter/intramuscular adipose tissue. We also develop a publicly available application for quantitative lower extremity CT analysis. Our main contributions are summarized below:

• We provide a multi-class lower extremity CT segmentation model for bone, muscle, and adipose tissue.

• We develop a comprehensive benchmark including multiple segmentation architectures, including CNN-based and transformer-based methods, and demonstrate that LegSegNet achieves the best average performance.

• We introduce LegSegNet, which, to our knowledge, is the first publicly available end-to-end system for lower extremity CT tissue quantification and analysis, and we have made the code and model weights publicly available.

2 Related Work↩︎

2.0.0.1 Computer vision for medical image segmentation.

Computer vision methods have been widely used for medical image segmentation, enabling automated extraction of anatomical structures and quantitative biomarkers from large imaging cohorts. Convolutional neural networks (CNNs), such as UNet [19] and its variants [20]–[22], are still commonly used for biomedical segmentation. More recently, transformer-based models and foundation segmentation models, such as SAM [23] and MedSAM [24], have also been explored and developed for medical imaging tasks. Among these methods, nnUNet remains one of the state-of-the-art (SOTA) baselines [25], [26], as it automatically adapts preprocessing, training pipeline, and post-processing to the target dataset. Therefore, in this work, we use nnUNet as the basis for LegSegNet development and benchmark representative CNN-based and transformer-based methods.

2.0.0.2 CT body composition analysis.

CT-derived measurements of skeletal muscle, adipose tissue, and bone can provide clinically useful biomarkers for nutritional assessment, sarcopenia, osteoporosis, obesity, and disease prognosis [8], [10]–[12]. Traditional body composition analysis often relies on manual or semi-automated delineation of selected axial slices or anatomical regions, which limits efficiency and introduces observer variability. Thus, deep learning has become an important approach for scalable tissue quantification. Recent CT models have automated muscle and fat segmentation across the chest, abdomen, and pelvis [9], [27]. However, these systems are not designed to provide a comprehensive lower extremity tissue quantification covering bone, skeletal muscle and adipose tissues. This motivates the development of an automated lower extremity CT tissue segmentation and quantification system.

2.0.0.3 Public segmentation models for CT lower extremity segmentation.

Public systems specifically designed for lower extremity CT tissue segmentation and quantification remain limited. TotalSegmentator provides a broad, general-purpose CT segmentation system for many anatomical structures and includes part of the lower extremity labels such as bones and selected muscle groups [3]. However, it mainly focuses on thigh-based muscle and is not designed as an end-to-end lower extremity body composition system for joint quantification of bone, skeletal muscle and adipose tissue. In addition, other lower extremity CT studies have focused on narrower segmentation settings, including lymphedema tissue segmentation [6], hip-to-knee bone and muscle assessment [4], and thigh muscle segmentation [7]. These studies show the feasibility of deep learning for lower extremity CT analysis, but existing models are commonly disease-specific, anatomically restricted, or unavailable for public use (As shown in Table 1). Moreover, most existing models do not provide a unified quantification of clinically important inter/intramuscular adipose tissue [28], [29]. Thus, LegSegNet aims to close this gap by focusing on comprehensive lower extremity CT tissue segmentation and quantification, with a public release of the system and model weights.

3 Methods↩︎

3.1 Cohort Selection↩︎

We selected 107 patients for model development and an additional 30 patients for independent model evaluation who took CT examination at the Duke Health System between 2013 and 2020. The development cohort mainly consists of two CT protocols: CT chest abdomen pelvis with contrast and CT angiography with runoff. All imaging data were deidentified before export.

For both protocols, we only annotate axial slices below the pelvis, as shown in Figure 1. Based on [13], instead of annotating the whole volume, we randomly annotate between 1 and 50 axial slices (median 3) for each volume in the development cohort. This results in 1,302 annotated 2D slices for model development, which are then split by patient into 1,041 training slices and 261 validation slices. For the independent test set, we select 30 axial slices evenly below the pelvis from each of the 30 held-out patients, resulting in 900 testing slices.

We further test the model’s generalizability on external test set. We randomly select 30 whole-body CT scans from SAROS [30], a publicly available multi-center dataset, and evaluate LegSegNet on the lower extremity range. SAROS provides annotations for subcutaneous adipose tissue and skeletal muscle.

3.2 Label Description↩︎

Each axial CT slice has been annotated into four foreground classes. Subcutaneous adipose tissue (SAT) is defined as the soft tissue between the skin and the deep muscular fascia, and therefore includes the subcutaneous fat layer as well as related soft-tissue changes within this region. Skeletal muscle (SM) includes the visible muscle of the thigh and calf. Inter/intramuscular adipose tissue (IAT) refers to fat located within the muscle, either between muscle groups or within muscle tissue. Bone includes the visible bone structures, including femur, tibia, fibula, and patella. Small structures that are not part of the target label set, such as small vessels and edema, are assigned to the surrounding label to maintain a consistent compartment-level annotation strategy, similar as [27]. Therefore, these labels provide a practical representation of major lower extremity tissues for CT-based body composition analysis.

3.3 Annotation Pipeline↩︎

We adopt an interactive model-in-the-loop annotation pipeline to improve annotation efficiency. To do this, we first randomly select 10 volumes to annotate from scratch. A 2D nnUNet is then trained using the available annotations and used to generate pseudo masks for the next batch of volumes. The annotators load these pseudo masks into 3D Slicer [31] and manually correct the sampled slices. The corrected masks are then added back to the training set, and the model is retrained for the next annotation round.

3.4 Segmentation Methods↩︎

LegSegNet is built upon nnUNet [25], a self-configuring deep learning framework that has become one of the strongest and most widely used approaches for biomedical image segmentation [26], [32]. Unlike manually designed pipelines, nnUNet automatically configures key components of the segmentation workflow, including preprocessing, patch size, network architecture, training pipelines, data augmentation, and postprocessing, based on an extracted dataset-specific fingerprint. In this study, we evaluate the default 2D nnUNet configuration and two residual-encoder variants, nnUNet 2D ResEnc-L and nnUNet 2D ResEnc-XL (See Table 3 for details).

3.5 Automated Quantification System Design↩︎

In addition to mask prediction, LegSegNet is designed as an automated lower extremity CT tissue quantification system. As shown in Figure 2, we implement the system with Gradio interface, which allows users to upload CT volumes or slices, run the trained segmentation model, visualize the predicted masks, and export quantitative measurements. The system supports both CPU and GPU inference, allowing it to run on standard local computers while also taking advantage of GPU acceleration when available. The system applies the trained model at slice-level, reconstructs the predicted labels in the original image space, and computes tissue measurements from the resulting masks.

For each tissue class \(c \in \{\boldsymbol{SAT}, \boldsymbol{SM}, \boldsymbol{IAT}, \boldsymbol{Bone}\}\) and axial slice \(k\), the cross-sectional area can be computed as following:

\[A_{c,k} = s_x \times s_y \times N_{c,k}\]

where \(N_{c,k}\) is the number of pixels assigned to class \(c\) and \(s_x, s_y\) are the pixel spacings. Then, tissue volume can be computed by summing slice areas across the analyzed lower extremity region:

\[V_c = \sum_k A_{c,k} \cdot \Delta z_k\]

where \(\Delta z_k\) is the slice thickness. The system also computes mean CT attenuation for each class as below:

\[\mu_c = \frac{1}{N_c}\sum_{i \in M_c} I_i\]

where \(M_c\) is the predicted mask for class \(c\), \(N_c\) is the number of voxels in the mask, and \(I_i\) is the CT intensity in Hounsfield units.

Based on the available labels, LegSegNet can report a set of directly computable lower extremity body composition metrics. Total adipose tissue volume is defined as the sum of subcutaneous and inter/intramuscular adipose tissue as:

\[V_{\mathrm{Fat}} = V_{\mathbf{SAT}} + V_{\mathbf{IAT}}.\]

The IAT-to-SAT ratio computes relative fat infiltration within the muscle compared with subcutaneous fat as:

\[R_{\mathbf{IAT}/\mathbf{SAT}} = \frac{V_{\mathbf{IAT}}}{V_{\mathbf{SAT}} + \epsilon}\]

where \(\epsilon=10^{-9}\) is a small positive constant added only to the denominator to avoid division by zero when the denominator tissue volume is zero or extremely small. The muscle-to-fat ratio is computed as:

\[R_{\mathbf{SM}/\mathrm{Fat}} = \frac{V_{\mathbf{SM}}}{V_{\mathbf{SAT}} + V_{\mathbf{IAT}} + \epsilon}\]

In addition, the system presents the skeletal muscle attenuation and adipose tissue attenuation as below:

\[\mu_{\mathbf{SM}} = \frac{1}{N_{\mathbf{SM}}}\sum_{i \in M_{\mathbf{SM}}} I_i\] \[\mu_{\mathrm{Fat}} = \frac{1}{N_{\mathbf{SAT}} + N_{\mathbf{IAT}}}\left(\sum_{i \in M_{\mathbf{SAT}}} I_i + \sum_{i \in M_{\mathbf{IAT}}} I_i\right)\]

Moreover, LegSegNet can summarize area-based measurements at each axial position to produce longitudinal tissue profiles \(\{A_{c,k}\}_{k=1}^{K}\) along the lower extremity, which can be useful for identifying regional muscle loss or abnormal fat distribution.

4 Experiments↩︎

4.1 Baselines↩︎

We compare LegSegNet with a broad set of 2D segmentation baselines, including convolution-based and transformer-based architectures. Specifically, we train five models from scratch: UNet [19], Attention UNet [20], SegResNet [21], FlexibleUNet with an EfficientNet-B0 encoder [22], and SwinUNETR [33]. In addition, we evaluate finetuned ViT-B versions of SAM [23] and MedSAM [24] for automated multi-class segmentation. Following the adapter-based finetuning strategies, only the inserted adapter layers are trainable, while the original foundation model parameters are frozen.

4.2 Evaluation Metrics↩︎

We evaluate segmentation performance using Dice similarity coefficient (DSC) and average symmetric surface distance (ASSD). DSC measures the overlap between the predicted mask \(P\) and the ground-truth mask \(G\):

\[\mathrm{DSC}(P, G) = \frac{2|P \cap G|}{|P| + |G|}\]

A higher DSC indicates better model performance. ASSD measures the average boundary distance between the predicted surface \(S_P\) and the ground-truth surface \(S_G\):

\[\mathrm{ASSD}(S_P, S_G) = \frac{\sum_{p \in S_P} d(p, S_G) + \sum_{g \in S_G} d(g, S_P)}{|S_P| + |S_G|}\]

A lower ASSD indicates better boundary accuracy. Both metrics are computed separately for bone, skeletal muscle, subcutaneous adipose tissue, and inter/intramuscular adipose tissue at volume level.

4.3 Implementation Details↩︎

LegSegNet is implemented using nnUNet v2 [25]. We evaluate three 2D nnUNet configurations: the default plain UNet, ResEnc-L, and ResEnc-XL. All three models use the default nnUNet training pipeline without modification, including automatic preprocessing, patch size selection, data augmentation, deep supervision, SGD optimization, polynomial learning-rate decay, and training for 1000 epochs. The automatically selected patch size is 192 \(\times\) 256.

For comparison, we train additional 2D baselines from MONAI, including UNet, Attention UNet, SegResNet, FlexibleUNet with an EfficientNet-B0 encoder, and SwinUNETR. For preprocessing, CT intensities (HU) are clipped to \([-200, 200]\) and normalized to \([0, 255]\). These models are trained from scratch for 1000 epochs with input resolution of 256 \(\times\) 256, AdamW optimization, and standard spatial and intensity augmentations. We also finetune SAM and MedSAM in an automated multi-class setting for up to 300 epochs using adapters following [17], while keeping the original foundation model weights frozen. All experiments are run using a single NVIDIA A6000 GPU.

4.4 Results↩︎

4.4.1 Segmentation↩︎

Table 2 presents the quantitative segmentation performance across all methods. The default nnUNet (2D) achieves the highest average DSC of 89.31% and the lowest average ASSD of 0.515. The residual encoder variants (ResEnc-L and ResEnc-XL) achieve comparable overall performance, with ResEnc-L obtaining the best SAT DSC (93.27%).

Among CNN-based methods, SegResNet performs best with an average DSC of 88.56%, followed by UNet and Attention UNet. For Transformer-based methods, SwinUNETR achieves 86.96% average DSC. Finetuned SAM and MedSAM obtain average DSC values of 86.40% and 79.17%, respectively.

Across all tissues, bone segmentation achieves the highest performance, with Dice scores exceeding 95% for all methods except MedSAM. IAT segmentation achieves the lowest performance, with the best DSC of 73.98% obtained by the default nnUNet. The qualitative results in Figure 3 show that LegSegNet produces anatomically consistent masks across representative axial slices, and the 3D reconstruction in Figure 4 further illustrates the volumetric continuity of the predicted lower extremity tissue segmentation.

In addition, LegSegNet achieves a comparable SAT performance on the external test (92.13% DSC). However, the DSC of SM seems to decrease to around 87% due to different annotation protocols, as shown in Figure 7.

4.4.2 Body Composition Correlation↩︎

To evaluate whether segmentation performance translates into reliable body composition measurements, we also compare LegSegNet-derived tissue volumes with reference volumes computed from the ground truth masks. This analysis is performed on stacked 3D test volumes. As shown in Figure 5, the Pearson correlation coefficients are 0.997 for SAT, 0.998 for SM, 0.968 for IAT, and 0.999 for Bone.

4.5 LegSegNet System↩︎

As shown in Figure 2 and Figure 6, LegSegNet provides an end-to-end workflow for lower extremity CT analysis. Given CT scans, the user can specify the axial range for analysis, after which the system automatically starts segmentation within the selected region. The predicted masks are then used to generate visual outputs including slice-level overlays, as well as quantitative body composition measurements such as tissue volume, attenuation, and ratios between different tissue volume.

4.6 Discussion↩︎

4.6.0.1 Performance analysis.

Our benchmark demonstrates that LegSegNet, based on nnUNet, consistently outperforms CNN-based, transformer-based, and finetuned foundation-model baselines. The default 2D nnUNet achieves the best overall performance with an average DSC of 89.31 and ASSD of 0.515. For class-wise results, ResEnc-L obtains the highest SAT DSC (93.27), while the default 2D nnUNet achieves the best performance on SM, IAT, and Bone. These results suggest that nnUNet’s self-configuring pipeline effectively adapts to lower extremity CT characteristics without manual tuning. The residual encoder variants do not improve overall performance over the default 2D nnUNet, indicating that increasing model capacity alone may not add further benefits to this task.

The performance drop for SM on SAROS is mainly due to differences in annotation protocols. As shown in Figure 7, IAT is included within SM in the SAROS masks, whereas our protocol labels IAT as a separate class. Moreover, LegSegNet maintains stable performance for SAT on SAROS, demonstrating good generalizability of our model.

4.6.0.2 Class-wise performance.

Bone segmentation is the most reliable target, likely because cortical bone has strong CT contrast and well-defined boundaries. In contrast, IAT remains the most challenging class because it is relatively sparse and can have similar intensity to adjacent adipose tissue compartments (e.g., SAT). This suggests that future improvements should focus on more consistent IAT annotation, and including more representative cases with diverse fat distribution.

4.6.0.3 Body composition analysis.

The strong tissue-volume correlations in Figure 5 indicate that high segmentation accuracy also translates into reliable quantitative measurements. The correlations are especially high for SAT, SM, and Bone, while IAT remains slightly lower, consistent with its lower segmentation DSC. These results support the use of LegSegNet not only for mask generation but also for automated lower extremity CT body composition analysis.

5 Conclusion↩︎

In this work, we present LegSegNet, an end-to-end deep learning system for lower extremity CT tissue segmentation and body composition quantification. LegSegNet can segment bone, skeletal muscle, SAT, and IAT, and convert the predicted masks into quantitative measurements including tissue volume, attenuation, and tissue volume ratios. Our comprehensive benchmark demonstrates that LegSegNet achieves the best overall segmentation performance compared with CNN-based, transformer-based, and finetuned foundation model-based baselines. In addition, the strong correlation between predicted and ground truth tissue volumes shows that LegSegNet preserves clinically relevant quantitative measurements for downstream body composition analysis. By releasing the system and model weights, we believe LegSegNet can serve as a public system for evaluating future computer vision models on clinically meaningful CT tissue segmentation and quantification tasks.

References↩︎