1 Introduction↩︎

Deep learning has transformed medical image segmentation, with U-Net architectures [1] and Vision Transformers [2] achieving remarkable performance across modalities [3]. However, pixel-based methods trained with cross-entropy or Dice losses [4] lack explicit anatomical knowledge, producing segmentations with topological inconsistencies, holes, and irregular boundaries under challenging conditions [5], [6]. Graph-based segmentation addresses these limitations by representing anatomical boundaries as connected graphs with guaranteed topology [7]. Recent advances in implicit anatomical correspondence learning [8] enable training on standard pixel-wise masks using Chamfer distance, where the \(i\)-th predicted landmark emerges to consistently represent the same anatomical location across patients, providing competitive accuracy with superior robustness and enabling automatic atlas generation.

Despite anatomical consistency across patients, these models process frames independently, ignoring temporal coherence. For cardiac imaging, this produces frame-to-frame discontinuities that degrade ejection fraction estimation, prevent accurate wall motion tracking required for clinical assessment, introduce artificial jitter incompatible with physiological cardiac dynamics, and limit translation to clinical workflows. Existing temporal consistency approaches rely on post-processing with optical flow [9], Kalman filtering, or autoencoder refinement [10]. While effective, these operate independently of the segmentation model and cannot leverage learned anatomical structure.

In this work, we introduce self-supervised temporal regularization as a post-training refinement stage for landmark-based cardiac segmentation. Starting from a model with established implicit correspondences, we fine-tune on ultrasound sequences by penalizing velocity and acceleration discontinuities between consecutive frames. This exploits the fundamental principle that anatomically-corresponding landmarks should move smoothly due to physiological constraints. Critically, this requires no per-frame annotations: the temporal behavior itself provides the supervisory signal. We further leverage learned correspondences to automatically map landmarks to the AHA 17-segment standard [11], enabling clinically-interpretable regional analysis. This combination of temporal consistency and automatic regional mapping allows detection of pathological wall motion patterns such as regional dyssynchrony and hypokinesis, making the approach directly applicable to clinical workflows.

1.1 Related Work↩︎

Landmark-Based Cardiac Segmentation. Point Distribution Models [12] and Active Shape Models pioneered landmark-based anatomical modeling through statistical shape variation. HybridGNet [13], [14] combined CNNs with spectral graph convolutions for variational cardiac segmentation, demonstrating superior domain shift robustness but requiring manually annotated landmarks with anatomical correspondences. Recent work [8] eliminates this requirement by training on standard segmentation masks, where the \(i\)-th landmark emerges to consistently represent the same anatomical location through optimization with Chamfer distance and edge regularization. This enables topologically-guaranteed segmentation with automatic correspondence discovery. However, temporal consistency remains unaddressed as models process frames independently.
Temporal Consistency in biomedical image sequences. Classical approaches employ registration or optical flow based techniques [9] as well as Kalman filtering for exploiting temporal coherence. Ledesma-Carbayo et al. [9] introduced a B-spline based spatio-temporal registration framework to exploit temporal coherence for the estimation of cardiac motion on ultrasound sequences. Sundar et al. [15] proposed simultaneous 4D registration where all cardiac phases are jointly optimized for temporal smoothness. Recent deep learning methods include recurrent architectures and test-time refinement. Painchaud et al. [10] use a constrained autoencoder for post-hoc correction of temporal inconsistencies in the attributes driven latent representation of echocardiography segmentations, operating on outputs from any segmentation method. Our work differs by integrating temporal regularization as a fine-tuning stage for landmark-based segmentation that leverages learned anatomical correspondences, enabling the model to refine its temporal representations while preserving spatial accuracy and structure correspondence.

2 Methods↩︎

2.1 Background: Implicit Correspondence Learning↩︎

Our framework builds on the implicit correspondence learning approach [8] where a graph-based VAE model learns to map input images \(\mathbf{I}\) to fixed-size landmark graphs \(\mathbf{G} = \langle V, \mathbf{A}, \mathbf{X} \rangle\). Here \(V\) represents a set of \(M\) nodes corresponding to anatomical landmarks, \(\mathbf{A}\) is an adjacency matrix encoding connectivity between organ contours, and \(\mathbf{X} \in \mathbb{R}^{M \times 2}\) contains the landmark coordinates. The encoder maps images to a latent distribution \(\mathbf{z} \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\sigma}^2)\), which the graph-convolutional decoder transforms into landmark coordinates. Completing the Mask-HybridGNet Dual architecture, an auxiliary convolutional decoder generates dense pixel-level masks and routes its multi-scale features directly to the graph decoder via image-to-graph skip connections.

The graph adjacency matrix is constructed automatically from an atlas derived from CAMUS ground-truth segmentation masks to represent the left ventricular endocardium, epicardium, and left atrium as a unified graph. This structure enforces anatomical constraints through shared boundaries: the left ventricular endocardium shares nodes with the inner myocardial wall, and the basal endocardium shares nodes with the left atrium at the mitral valve plane.

We train the base model with a compound loss functions which uses Chamfer distance (\(\mathcal{L}_{CD}\)) to match variable-length ground truth contours \(\mathbf{P}\) with the fixed-size predictions \(\mathbf{G}\), combined with edge regularization for uniform spacing between landmarks (\(\mathcal{L}_{uniform}\)), elasticity regularization to encourage compact contours (\(\mathcal{L}_{elastic}\)), and curvature regularization (\(\mathcal{L}_{smooth}\)) to promote local smoothness by penalizing abrupt direction changes between consecutive edges, along with a standard VAE Kullback–Leibler divergence on the variational latents (\(\mathcal{L}_{KL}\)). We also include a Dice pixel-loss (\(\mathcal{L}_{pixel}\)) computed by comparing the ground-truth masks with both, the rasterized graph-contour predictions and the dense output of the auxiliary CNN decoder. This process discovers anatomical correspondences where landmark \(i\) represents consistent anatomy across all subjects. The standard frame-independent training objective is: \[\mathcal{L}_{base} = \mathcal{L}_{CD} + \lambda_{KL}\mathcal{L}_{KL} + \lambda_e\mathcal{L}_{elastic} + \lambda_u\mathcal{L}_{uniform} + \lambda_s\mathcal{L}_{smooth} + \lambda_{p}\mathcal{L}_{pixel} \label{eq:loss95base}\tag{1}\]

For a more detailed description of these loss terms and the Mask-HybridGNet Dual architecture we refer the reader to [8]. After training with this objective, the model produces anatomically consistent segmentations with implicit correspondences. However, the combination of fixed-size predictions and Chamfer distance matching can cause temporal instability: landmarks may shift by one or two positions along the circular contour between consecutive frames, resulting in correspondence oscillations and abrupt temporal transitions.

2.2 Self-Supervised Temporal Regularization↩︎

To address the temporal instability of the predicted landmarks while preserving learned correspondences, we introduce temporal regularization as a post-training refinement stage. Starting from a converged model with established anatomical correspondences, we fine-tune on video sequences by adding temporal constraints that exploit the principle that anatomically-corresponding landmarks should exhibit smooth motion profiles characteristic of physiological cardiac dynamics.

Let \(\{\mathbf{I}^{(t)}\}_{t=1}^T\) denote a temporal sequence of \(T\) consecutive frames from the same cardiac cycle. For each frame, the encoder produces latent code \(\mathbf{z}_t \sim \mathcal{N}(\boldsymbol{\mu}_t, \boldsymbol{\sigma}_t^2)\) and the decoder generates a set of landmarks \(\mathbf{X}(t)=\big\{x_i(t)\big\}_{i=1}^M\in \mathbb{R}^{M \times 2}\). We introduce two complementary temporal regularization terms that penalize motion discontinuities while maintaining the spatial accuracy learned during initial training.
Velocity regularization encourages smooth trajectories for anatomically corresponding landmarks by penalizing large frame-to-frame displacements:

\[\mathcal{L}_{vel} = \frac{1}{M}\sum_{i=1}^{M} \Bigg(\frac{1}{T-1}\sum_{t=1}^{T-1} \big\Vert\,\mathbf{x}_i(t+1) - \mathbf{x}_i(t)\big\Vert^2\Bigg) \label{eq:loss95vel}\tag{2}\]

Acceleration regularization penalizes abrupt changes in velocity by minimizing the difference between consecutive velocity vectors, enforcing physiologically plausible acceleration and deceleration patterns:

\[\mathcal{L}_{accel} = \frac{1}{M}\sum_{i=1}^{M} \Bigg(\frac{1}{T-2}\sum_{t=1}^{T-2} \big\Vert\mathbf{x}_i(t+2) - 2\,\mathbf{x}_i(t+1) + \mathbf{x}_i(t)\big\Vert^2\Bigg) \label{eq:loss95accel}\tag{3}\]

The refinement objective combines the original spatial terms with temporal regularization: \[\mathcal{L}_{refine} = \mathcal{L}_{base} + \lambda_v\mathcal{L}_{vel} + \lambda_a\mathcal{L}_{accel} \label{eq:loss95refine}\tag{4}\] where \(\lambda_v, \lambda_a\) control the strength of temporal regularization.
Training Procedure. We implement an alternating optimization strategy to balance spatial accuracy with temporal consistency. Each training iteration processes a batch consisting of: (1) \(N\) individual annotated frames optimized using \(\mathcal{L}_{base}\), followed by (2) one temporal sequence of \(T\) consecutive frames optimized using \(\mathcal{L}_{vel}\) and \(\mathcal{L}_{accel}\). This alternating scheme is essential to prevent the model from converging to trivial solutions that minimize temporal losses by predicting static contours with minimal motion. The sequence length \(T\) is constrained by GPU memory requirements, as processing full sequences requires maintaining activations for all \(T\) frames during backpropagation.

2.3 Automatic AHA Mapping↩︎

We leverage the learned implicit correspondences to automatically map landmarks to the standardized AHA 17-segment model [11], translating established clinical echocardiography workflows [16] into our automated pipeline. Because the landmark \(i\) consistently represents the same anatomical location across the dataset, this mapping is computed just once on a population-averaged reference atlas, constructed by averaging the spatial coordinates of all subjects at the end-diastolic frame, and applied universally. As illustrated for the 4-chamber view in Figure 1, the procedure automatically identifies the mitral valve center from shared left ventricular and atrial nodes, establishing the primary long-axis vector toward the endocardial apex. The left ventricle is then divided along this long axis into basal (up to 33%), mid-cavity (33% to 66%), and apical (66% to 98%) levels, reserving the final 2% for the apex cap. These levels are perpendicularly bisected to separate the opposing walls (anterolateral and inferoseptal in the 4-chamber view; anterior and inferior in the 2-chamber view). Finally, epicardial nodes are assigned to the segment of their nearest endocardial neighbor.

3 Experiments, results & discussion↩︎

Datasets and Evaluation. The CAMUS dataset [17] provides 500 patients with 2-chamber and 4-chamber apical echocardiography sequences of approximately 20 frames each. Expert annotations mark LV endocardium, epicardium, and left atrium at end-diastole and end-systole, with metadata including ejection fraction and image quality scores. We use the 400-50-50 train/val/test split provided.

For spatial accuracy, we compute the Dice coefficient by rasterizing the predicted landmarks into binary masks, and the 95th percentile Hausdorff distance (HD95) to quantify boundary error while reducing sensitivity to outlier points. Clinical metrics were selected to reflect standard measures used in routine cardiac practice, and performance was evaluated by computing the mean absolute error (MAE) between the reference and predicted values for ejection fraction (EF), end-systolic volume (ESV), and end-diastolic volume (EDV), with ventricular volumes computed from masks using the biplane Simpson’s method.

To quantify the temporal regularity of the trajectories while excluding the expected displacements induced by cardiac motion, we use a jitter metric that captures high-frequency perturbations. Specifically, jitter is defined as the root-mean-square (RMS) deviation between each aligned trajectory and a low-pass filtered version of the same trajectory over time:

\[\text{Jitter} = \frac{1}{M} \sum_{i=1}^{M} \Bigg(\sqrt{\frac{1}{T}\sum_{t=1}^{T}\;\big\Vert \; \mathbf{x}_i(t) - \bar{\mathbf{x}}_i(t) \,\big\Vert ^2}\Bigg)\] where \(\bar{\mathbf{x}}_i(t)\) denotes the smoothed trajectory obtained by applying a Savitzky–Golay filter independently to each aligned trajectory. As complementary measures of trajectory quality and smoothness, we also report frame-to-frame displacement (FTD), which quantifies the RMS excursion distance for each node \[\text{FTD} = \frac{1}{M} \sum_{i=1}^{M} \Bigg(\sqrt{\frac{1}{T-1} \sum_{t=1}^{T-1} \big\Vert\, \mathbf{x}_i(t+1) - \mathbf{x}_i(t)\big\Vert^2}\Bigg)\] and RMS jerk, which measures the temporal variation of acceleration and penalizes abrupt motion changes using the 3rd-order finite difference

\[\text{Jerk} = \frac{1}{M} \sum_{i=1}^{M} \Bigg(\sqrt{\frac{1}{T-3} \sum_{t=1}^{T-3} \big\Vert\, \Delta^3\mathbf{x}_i(t) \big\Vert^2}\Bigg)\]

Results. We first trained Mask-HybridGNet Dual on CAMUS without temporal regularization, using the loss in Eq. 1 and ED and ES frames with ground-truth masks from both 2-chamber and 4-chamber views. We selected the dual branch architecture because it showed the best performance in the experiments reported in [8]. Figure 2 summarizes all spatial, clinical, and trajectory metrics. This baseline achieves strong segmentation performance (Dice and HD95), but relatively poor trajectory quality and smoothness metrics, consistent with the absence of temporal regularization during training.

For comparison, we also trained nnUNet [18] on the same CAMUS images and masks. Since nnUNet is a pixel-wise segmentation model and does not provide landmarks or temporal correspondences, trajectory-based temporal consistency metrics cannot be computed directly from its output. To enable this analysis, we trained a Mask-HybridGNet model that takes multiclass segmentation masks as input (instead of images) and predicts spatially consistent nodes using a unified multiclass contour. This enables the post-processing of nnUNet predictions over the cardiac cycle, converting them into trajectories and quantifying their temporal regularity. Although this pipeline enables trajectory analysis, its temporal coherence metrics show statistically significant differences compared to those of the proposed temporally regularized approach. (see Figure 2).

Starting from the supervised Mask-HybridGNet Dual baseline, we then performed post-training temporal regularization following Section 2.2, using the loss in Eq. 4 , which includes velocity (Eq. 2 ) and acceleration (Eq. 3 ) terms with weights selected by grid search on the validation set. We evaluated two variants, with and without rasterization in the \(\mathcal{L}_{pixel}\) loss-term (Mask-HybridGNet Dual + Temp. Reg. and Mask-HybridGNet Dual + Temp. Reg. w/o Raster).

Among the evaluated models, Mask-HybridGNet Dual + Temp. Reg. w/o Raster achieves the best trajectory quality metrics, with smoother trajectories than both the non-regularized Mask-HybridGNet baseline and the nnUNet + post-processing pipeline. To perform a qualitative regional analysis, predicted landmarks were assigned to AHA segments using the self-supervised procedure described in Section 2.3. Figure 3 shows AHA segment identification and landmark trajectories from ED to ES in two test cases, together with longitudinal displacements of the segment centroids. The temporally regularized model exhibits smoother segment-centroid trajectories than the post-processed nnUNet, consistent with the improvements observed in the temporal coherence metrics. As illustrated in Figure 3, graph-based models with implicit correspondence learning show strong potential for downstream clinical tasks, including segment-wise AHA longitudinal strain estimation. The Supplementary Video shows the dynamic representation of the comparative performance for 6 test cases.

Discussion & Conclusions. In this work, we introduced a self-supervised post-training temporal regularization strategy for graph-based cardiac ultrasound segmentation that improves temporal coherence while preserving anatomical correspondences, allowing automatic AHA-based regional motion analysis. Our results show that high pixel-level segmentation accuracy does not necessarily ensure temporally consistent trajectories. Although nnUNet performs strongly on conventional spatial and clinical metrics, it lacks explicit landmark correspondences and temporal regularization, limiting direct trajectory-based analysis. Converting nnUNet masks into trajectories partially addresses this limitation, but yields inferior temporal quality compared to models enforcing consistency. The proposed regularized Mask-HybridGNet improves trajectory smoothness while maintaining spatial and clinical performance comparable to nnUNet, and naturally supports standardized regional motion analysis.
Acknowledgments. EF was supported by the Google Award for Inclusion Research, and a Googler Initiated Grant. MJL acknowledges the support of the Spanish Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, under grant PID2022-141493OB-I00 (10.13039/501100011033/MCIN/AEI/ERDF, UE), cofinanced by European Regional Development Fund (ERDF), ’A way of making Europe, and partial funding from the MAGERIT-CM project (TEC-2024/COM-44), supported by the Research and Development Activities Program by the Comunidad de Madrid. DMG was supported by an FPU PhD fellowship funded by the Spanish Ministerio de Ciencia, Innovación y Universidades.

References↩︎