1 Introduction↩︎

Radio detection and ranging (radar) system was originally designed for military detection of large aircraft by emitting electromagnetic waves and evaluating the reflections. The follow-up research has investigated the civilian use of radar systems for contactless sensing in various scenarios, such as autonomous driving [1] and human monitoring [2]. Over the past decade, radar-based sensing has been empowered by deep neural networks to process non-stationary reflected signals or high-dimensional data, enabling versatile applications to replace contact- or visual-based measurement for convenience or privacy concerns (e.g., vital sign monitoring [3], gesture recognition [4], person identification [5]).

Radar-based vital sign monitoring, as a popular branch of radar-based sensing, has been explored for decades to measure heart rate or respiration rate in a contactless manner [2], and some further studies leverage the deep neural network to realize domain transformation from cardiac mechanical activities (i.e., heartbeat) to electrical activities (i.e., electrocardiogram (ECG)), providing a fine-grained cardiac measurement for wellness monitoring or clinical diagnosis [6]–[12]. In the literature, radar-based ECG recovery is only realized by deep-learning-based methods, because the domain transformation is extremely complex to be modeled mathematically while such transformation can be learned by a deep learning model due to the great nonlinear mapping ability [9].

Similar to other research fields involved with deep learning, radar-based ECG recovery also asks for numerous radar signals to train the deep learning model with synchronous ECG ground truths [8], [11], [13]. According to previous research, the performance of the deep learning model degrades heavily after reducing \(30\%\) of the training data even after applying proper data augmentation techniques [10], causing difficulties for the deployment in new scenarios due to the demand for hours of ECG collection [7]. However, the method for reducing dependence on data quantity is rarely investigated for radar-based ECG recovery, and all the deep-learning-based ECG recovery models are trained in a supervised manner with large dataset containing \(3-32\) hours of synchronous radar-ECG pairs [9], [11], [14].

In contrast, most studies are dedicated to inventing advanced signal processing algorithms to enhance the signal quality, because the deep learning model for ECG recovery is vulnerable to the inputs contaminated by noise and requires high signal-to-noise ratio (SNR) radar signals as inputs [3], [9]. The methods for capturing high-SNR radar signals can be categorized into two groups:

• The first type of method focuses on designing advanced radar front-end with multiple transmitters (Tx) and receivers (Rx) [15], [16] or calibrating baseband radar signals from in-phase and quadrature (IQ) channels to a circular shape [17]–[19].

• The second type of method assumes that the rough localization of human body provides accurate chest region with the majority of range bins containing useful cardiac features, and high-SNR signal can be obtained by selecting useful channels [11], [20], applying clustering algorithms [9] or accumulating the signals from various dimensions (e.g., chirps, frames, antennas) [3].

The first type of method is not suitable for some commonly used frequency-modulated continuous-wave (FMCW) radar platforms (e.g., TI AWR-x radar) due to the on-board digital front-end module filtering the frequency-modulated feature of baseband signal (i.e., circular IQ plot) [21], preventing the broad applications of this approach in commercial radar. The second type of method relies on accurate localization of the chest region, while existing methods only provide a rough location of the human body, causing a deviation of several decimeters due to different postures of the subject [22]. Therefore, the methods based on signal accumulation may fail because only a minority of range bins contain cardiac features, hence not subjecting to the law of large numbers [3]. Although some aforementioned studies have proposed methods for selecting or clustering the useful range bins with cardiac features [9], [11], the computational cost for traversing a large objective space can be huge without an accurate cardiac location.

Based on the above discussion, it is still a challenge to: (a) precisely locate and track the cardiac location during data collection to efficiently extract high-SNR radar signal; (b) develop a deep learning framework for radar-based ECG recovery with less demand for ECG collection to realize an efficient model training especially for new scenarios with limited data. To overcome these two challenges, the contributions of this study can be listed as:

• A cardio-focusing and -tracking (CFT) algorithm is proposed based on derivative-free optimization (DFO) to find the cardio-focused (CF) point by iteratively evaluating the potential points in a discontinuous objective space, with a universal signal template designed to adaptively assess the signal SNR as costs.

• A transfer learning framework RFcardi is proposed following a self-supervised learning (SSL) paradigm to effectively learn the latent representations from radar signals by leveraging an appropriate pre-text task. Accordingly, this work further designates the sparse signal recovery (SSR) as the pre-text task, assisting the RFcardi to learn essential representations for the later ECG recovery.

• The proposed CFT algorithm has been validated on sitting subjects in various scenarios and could provide radar measurements with better SNR compared with existing methods. In addition, the pre-trained RFcardi framework can be easily adapted to realize radar-based ECG recovery with a small number of synchronous radar-ECG measurements for fine-tuning.

The rest of the paper is organized as follows. Section 2 provides the background information for radar-based ECG recovery and SSL. Section 3 elaborates the proposed CFT algorithm and RFcardi framework, with the experimental settings and results shown in Section 4 and 5. The final conclusion is shown in Section 6.

2 Theoretical Background and Challenges↩︎

2.1 FMCW Radar Foundations↩︎

FMCW radar has been widely used in nowadays millimeter-wave (mmWave) sensing to measure the range, velocity and angle of arrival (AoA) of the objects appearing in the field of view [23], with three critical concepts that configure the transmitted waveform:

• Chirp is the minimum component in the FMCW signal with microsecond-level duration and is often called fast time. The waveform of a single chirp is a sinusoidal signal with frequency that changes linearly over time, with the key characteristics designated by start frequency, bandwidth and chirp duration to get range information of the object.

• Frame is a collection of multiple chirps that forms a complete observation window to get the velocity information based on the range bins extracted from chirps and is often referred to as slow time.

• Virtual antenna array (channel) is a commonly used concept in multiple-input and multiple-output (MIMO) radar systems and is able to realize complex modulations or beamforming [16]. However, this study mainly leverages the phase difference across antenna channels to estimate the AoA of the objects.

The popular commercial radar platforms have provided a convenient interface for radar configuration, signal modulation and demodulation [24]. Therefore, detailed equations of FMCW radar signal processing might be redundant in this paper, while the theoretical explanation can be found in previous papers [23], [25].

2.2 Cardiac Signal Extraction from FMCW Radar↩︎

ECG recovery relies on the high-SNR radar inputs that describe the mechanical cardiac activities, and the reflected signal from a given point \(E=(x,y,z)\) in 3D space can be expressed as: \[\label{equ:raw95sig} R(E, t)=\sum_{v=1}^V \sum_{c=1}^C \sum_{n=1}^N s_{v, c, n}(t)\cdot e^{j 2 \pi \frac{2k\cdot d(E, v)}{\text{light speed}} n} \underbrace{e^{j 2 \pi \frac{2\cdot d(E, v)}{\lambda}}}_{\text{phase term}\;\phi}\tag{1}\] where \((x,y,z)\) represents the (horizontal, radial, vertical) axis, \(V\) is the number of virtual antenna channels, \(C\) means the number of chirps within one frame, \(N\) is the total sample points within one chirp, \(s_{v, c, n}(t)\) denotes the original received signal, \(k\) is the slope of frequency raising, \(\lambda\) means wavelength and \(d(E, v)\) represents the distance between point \(E\) and virtual antenna \(v\) [9]. In FMCW processing for vital sign monitoring, the time sample \(t\) corresponds to one frame instead of the sample point \(n\), and the signal from different chirps \(c\) and antenna channels \(v\) will be accumulated to improve SNR [3].

The interested term in (1 ) is the variation of distance \(d(E,v)\), because it represents the displacements caused by respiration and heartbeat (without considering any other noise). Therefore, the chest region displacement \(h(E, t)\) can be unwrapped from phase variation \(\Delta \phi\) as \[\label{equ:phase} h(E, t) = \frac{\lambda \Delta \phi}{4\pi}\tag{2}\]

At last, some common noises, such as respiration and thermal noise, can be easily removed using a band-pass filter and differentiator to make sure that the final \(h(E, t)\) mostly contains cardiac-related features from point \(E\) as shown in Figure , with prominent vibration corresponding to the R peaks in the ECG signal.

Challenge: It is natural to think the high-SNR radar signal can be searched in a constrained space by optimization, while there is no appropriate method to assess the signal SNR in terms of cardiac features contained. In addition, the objective space is highly discontinuous, and the signals extracted from adjacent points reveal totally different SNR, as indicated by the poor radar collection with much noise that buries the cardiac features (peaks) in Figure , restricting the application of common gradient-based optimization algorithms.

2.3 Transfer Learning↩︎

In addition to high-SNR radar inputs, accurate ECG recovery also relies on the scale of datasets to train the deep learning model, with the previous research normally adopting the dataset with the length of \(3-32\) hours [9], [11], [14]. Unfortunately, the initial experimental results show that the well-trained deep learning model cannot be directly used for the signal collected from new scenarios even using a similar radar configuration as shown in Figure  and  , because different indoor scenarios and radar configurations may have unknown interference on radar signals [2].

Inspired by other signal-based research [20], [26], transfer learning is a promising paradigm to learn the latent representation from unlabeled radar signal to capture basic cardiac features in an SSL manner, reducing the need for cumbersome ECG collection. Then, only a small number of synchronous radar-ECG pairs is required to fine-tune the pre-trained model to realize the ECG recovery for new scenarios.

Challenge: The efficient SSL requires an appropriate design of the pre-text task to help the deep learning model capture essential features that assist ECG recovery, while no existing work has investigated only learning from radar signals without the aid of ECG ground truth.

3 Methodology↩︎

3.1 Overview of CFT-RFcardi Framework↩︎

The pipeline of the proposed CFT-RFcardi framework is shown in Figure 2 with three steps:

• The received radar signal will be converted into a standard format in terms of chirp, frame and virtual antenna channel to obtain the general location of the subject, as shown in Figure 2 (a).

• The rough location acts as the initial state for the CFT algorithm, and the points within a constrained space will be evaluated to find the red CF point with the best SNR as shown in Figure 2 (b).

• Signals extracted from the ten best points will be converted into spectrograms to pre-train the backbone with SSR as the pre-text task. Then, the same backbone will be used for the fine-tuning stage so that the latent representations learned in the pre-trained model can be seamlessly transferred for the ECG recovery task, as shown in Figure 2 (c).

In addition, TI-AWR 1843 radar operated at \(77\)GHz with \(2\) Tx and \(4\) Rx will be used for data collection, enabling \(8\) virtual antenna channels for high-quality signal extraction. The detailed scenario descriptions and radar configurations will be provided in Section 4.1.

3.2 Rough Localization↩︎

The received radar signal is first formatted as a standard data matrix in terms of different chirps, frames and virtual channels to provide measurements of range, velocity and AoA, respectively. For the current research level of radar-based vital sign monitoring, the subjects are all quasi-static without velocity, and only the range-angle (RA) map will be calculated using fast Fourier transform (FFT) as shown in Figure 2 (a), with a detailed illustration of signal waveform and processing shown in Figure 3.

3.2.1 Range FFT↩︎

According to (1 ), the signal propagation after transmitting introduces a constant phase shift \(\phi_s\) in the received signal and is expressed as \[\label{equ:range} \phi_s = \frac{4\pi d_0}{\lambda}\tag{3}\] with \(d_0\) representing the distance between radar and human body. Therefore, the distance \(d_0\) can be extracted from each received signal along fast time using FFT as shown in Figure 3 (a), and the updated data matrix now reveals the range information, i.e., a static object denoted as blue along slow time axis.

3.2.2 Angle FFT↩︎

The ability of AoA detection relies on the MIMO system using time division multiplexing (TDM-MIMO), with multiple Tx alternately transmitting chirp signals and the corresponding reflections can be distinguished during receiving as shown in Figure 3 (b). Due to the physical distance varies for different Tx/Rx combinations (i.e., Tx\(2\)Rx\(4\) creates \(8\) virtual channels), an extra propagation \(\Delta \phi_v\) delay will be introduced as: \[\label{equ:angle} \begin{align} \Delta \phi_v &= \frac{4\pi d_v}{\lambda} \\ d_v &= l\;sin(\theta) \end{align}\tag{4}\] where \(d_v\) represents the extra propagation distance, \(l\) means the distance between adjacent antenna channels and \(\theta\) is the incident angle. Similar to range FFT, the phase differences across different channels can be used to extract AoA information for each range bin by performing FFT along the channel axis, as shown in red squares in Figure 3 (b).

After combining the FFT results for all chirps and channels, the final RA map for the current time sample (frame) can be obtained as shown in Figure 3. The same procedure can be repeated along the slow time axis to get the rough human body location for all the time samples, but this study only requires the location obtained from the very first frame as the initial point \(E_0\) for CFT algorithm.

3.3 Cardio-focusing and -tracking (CFT) Algorithm↩︎

The radar signal for any point can be extracted following (1 ) and (2 ), and the search progress from \(E_0\) to the best point \(E_{b}\) (i.e., CF point with high SNR) requires: (a) an appropriate metric to assess whether the radar signal contains wanted cardiac features; (b) an optimization method that is applicable to the discontinuous objective space based on the assessed SNR values as costs.

3.3.1 Template Design for Assessing SNR↩︎

An explicit SNR can be calculated with the known “clean” signal, while the “clean” signal for vital signs normally reveals two prominent vibrations corresponding to the ventricular contraction and relaxation [7], as shown in Figure . However, considering the vibrations may have subtle differences due to different scenarios or radar configurations (e.g., noise figure and sampling frequency), a universal template \(h_m\) is designed in this study to fit the envelope of the radar signal as: \[h_m (t) = a_1 \exp(-\frac{(t-b_1)^2}{2c_1^2}) + a_2 \exp(-\frac{(t-b_2)^2}{2c_2^2})\] with \(a_1\), \(a_2\) controlling the amplitudes of the peaks, \(b_1\), \(b_2\) determining the centers of the peaks and \(c_1\), \(c_2\) adjusting the width of the peaks. In practice, \(a_1\) and \(b_1\) will be fixed based on the amplitude and position of the dominant peaks detected as the red points in Figure , and other parameters are left to be determined as a simple curve fitting problem as an optimization problem. Finally, the mean square error (MSE) between the radar signal envelope and the synthetic template is reckoned to be an assessment of signal SNR as shown in Figure , because fewer components could fit the designed template for low-SNR radar signal without obvious cardiac features, as shown in Figure  and .

3.3.2 Derivative-free Optimization (DFO)↩︎

The MSE values obtained from template matching for the radar pieces extracted from point \(E\) will be used as costs \(\mathcal{F}(E)\) in searching CF point, but the traditional gradient-based optimization method is not applicable because there is no explicit cost function. Therefore, the CFT algorithm is developed in a derivative-free manner based on coordinate search (CS) algorithm [27], to asymptotically approach the CF point.

The definition of the DFO problem is formulated as: \[E_b = \underset{E \in \mathbb{R}^n}{\arg \min }\left\{\mathcal{F}(E) : E\in \Omega \right\}\] with \(\Omega\) representing a user-defined constrained \(n\)-dimensional search space near initial point \(E_0\) as shown in Figure 2 (b), and the cost of points out of the constraint will be set as \(\mathcal{F}(E\not\in \Omega)=\infty\). During each iteration \(k\), many trial points \(E_k\) within the constraint will be evaluated to find the incumbent points \(E_i\) as the temporary best point for the next iteration.

To perform a derivative-free search, the traditional CS algorithm starts from the initialization of grids \(G_k\): \[\label{equ:grid} G_k := \{E_k + \gamma_kD\} \subset \mathbb{R}^n\tag{5}\] where \(\gamma_k>0\) is the grid size parameter and \(D\) contains several vectors \(p\) for possible searching directions, as shown in Figure . The local convergence of CS is ensured by dense search directions \(D\) and a refined grid size \(\gamma_k\) to find better \(E_i\) compared with current \(E_k\) [27]. As a basis for all CS-based DFO methods, \(D\) can be formed by generating the orthogonal unit vectors based on the seed vector selected from the Halton sequence to ensure an evenly distributed search direction around the \(E_k\). However, the highly discontinuous objective space for radar-monitored vital signs may have numerous local minima that distract the optimization algorithm, i.e., the signal SNR of the adjacent points might be very different, as shown in Figure  and .

To jump out of the potential local minimum, CFT algorithm is proposed by introducing search region \(S_k\) to restrict the possible search directions \(p\), alleviating the difficulty of searching in numerous dense grid points and allowing the adjustment of search region and grid size iteratively to break the local minimum. The detailed procedures of CFT are shown in Algorithm 6, with an illustration of \(\mathbb{R}^2\) space shown in Figure 5.

In CFT, the grids \(G_k\) is still expressed as in (5 ) and the newly introduced search region \(S_k\) is expressed as: \[S_k := \{E\in G_k : \| E-E_k \|_{\infty} \leq \Gamma_k \}\] with \(\Gamma_k\) as the size parameter for the search region. An intuitive interpretation of \(S_k\) is the point set that contains grid points inside and on the boundary of the bold line controlled by \(\Gamma_k\), as shown in Figure .

Based on the well-constructed grids \(G_k\) and search region \(S_k\), the remaining CFT algorithm is performed with searching and resizing stages:

Searching: The searching stage simply asks for the evaluation of \(\mathcal{F}(E)\) on a subset of grids \(G_k\) based on any sampling algorithm (e.g., Latin hypercube sampling [27]), as indicated in line [line:s1] in Algorithm 6.

Resizing: The resizing stage depends on the result of searching stage:

• If a new incumbent point \(E_i\) is found with better SNR, the search region will be doubled as \(\Gamma_{k+1}=2\Gamma_k\) (line [line:r1]), and the grid size will be empirically set as \(\gamma_{k+1}=\min(\Gamma_k, {\Gamma_k}^2)\) (line [line:r4]), enabling the search in a broader space in the next iteration.

• If there is no better point than the current \(E_k\) on the current grids \(G_k\), another searching stage will be performed only within the search region \(S_k\) (line [line:s2]). Then, If a better point \(E_i\) is found, \(\Gamma_{k+1}\) and \(\gamma_{k+1}\) are obtained as above (line [line:r2]), otherwise, the search region will be halved as \(\Gamma_{k+1}=\Gamma_k/2\) (line [line:r3]) for a finer search with \(\gamma_{k+1}=\min(\Gamma_k, {\Gamma_k}^2)\) (line [line:r4]).

The searching step enables the finding of better points \(E_i\) in a broad space, and the resizing step either refines the grid if the current \(\gamma_{k}\) is not enough or enlarges the search space when stalling at the local minimum, as shown in Figure . Finally, the CFT algorithm will be terminated after achieving a desired SNR\(_d\) or iteration limit \(k_{max}\).

The visualization of the CFT algorithm in Figure 2 (b) shows that the initial few iterations search in a large space, and the algorithm could jump out of the green local minima to find the red CF point within the fine blue grid points. In addition, the tracking of the CF points along time can be naturally realized by repeating Algorithm 6 with previous \(E_b\) as the new \(E_0\), and the SNR evaluated on the previous point might have already achieved SNR\(_d\) due to the quasi-static human body, saving a huge amount of time for calculating useless channel information for filtering or clustering [3], [9], [11].

3.4 Transfer Learning for ECG Recovery↩︎

3.4.1 Deep Learning Model Design↩︎

The radar signal extracted from the \(10\) best points from CFT algorithm will be converted to spectrograms as the input for deep learning model according to our previous work [8], providing extra frequency-domain information to assist model training. The deep learning model adopts the popular backbone-decoder structure as designed in [8]:

• The backbone leverages ResNet [28] framework with deformable 2D convolution layer [29] to efficiently extract cardiac features from image-like spectrogram inputs.

• The decoder is based on 1D convolutional neural network (CNN) to generate corresponding signals either for the pre-text task or ECG signal recovery.

3.4.2 Pre-text Task Training and Fine-tuning↩︎

The pre-text task used for SSL should reveal certain inherent features in the radar-monitored vital sign and can be transferred to help the ECG recovery. According to our previous work [8], two major features of a good ECG recovery are accurate morphological features and accurate peak locations. The learning of morphological features fully relies on the non-linear mapping ability of the DL model by training with massive ECG data and is impossible to be pre-trained using radar signal only. However, the peaks in radar (heartbeats) and ECG signals (R peaks) are synchronized and have been extracted when assessing the radar signal SNR, as shown in Figure .

It is natural to leverage the task for heartbeat detection as the pre-text task for SSL. Two categories of methods used for traditional heart rate estimation are based on periodicity and sparsity [2]. In this work, the duration of each segment is \(4\) sec and may not reveal strong periodicity. Therefore, SSR will be used as the pre-text task in RFcardi and is defined as: \[h=\Phi x+n\] where \(h\) is the high-SNR radar signal, \(\Phi\) means the observation matrix, \(x\) is the sparse representation for heartbeats and \(n\) represents the residual noise. The traditional SSR task can be seamlessly converted to a system identification problem by viewing \(\Phi\) as a multi-channel adaptive filter, and the estimation of the filter coefficient is the same as training a CNN-based neural network (i.e., training CNN kernels) [30].

In this case, the SSR task is realized by using the aforementioned CNN-based backbone-decoder structure with the loss function: \[\label{equ:sparse} \mathcal{L} = \|x - \tilde{x} \|_2 + \underbrace{\lambda_s\frac{\|x\|_1 / \| x\|_2-1}{\sqrt{m}-1}}_{\text{sparse penalty}}\tag{6}\] where \(m\) is the length of the signal, \(x\) is the output from deep learning model, \(\tilde{x}\) is the sparse ground truth with values for the radar peaks maintained (1st vibration in Figure ) while other values are set to \(0\). The sparse penalty has a range of \([0,\lambda_s)\) with a smaller value indicating better sparsity [31].

After pre-training based on SSR, the parameters of backbone will be retained with a new decoder connected (same structure as for pre-text task training), and a few radar-ECG pairs are used for fine-tuning the pre-trained RFcardi model using MSE as the loss function.

4 Details of Experiment and Dataset↩︎

4.1 Experimental Details↩︎

4.1.1 Dataset Collection and Preparation↩︎

The dataset contains a total of \(80\)-minute synchronous radar-ECG pairs collected for \(5\) healthy subjects (\(3\) men, \(2\) women) in \(2\) indoor scenarios as shown in Figure 7. The subjects are asked to sit casually and are allowed to change postures during data collection, and each data trial lasts for \(1\) minute. The upper body travel distances vary from \(1\) to \(9\)m during \(1-\)minute trials based on the statistics analyzed after the experiment. The distance between radar and human body varies from \(0.5\) to \(1.2\)m, and a longer distance causes a decrease in signal SNR, with a smaller portion of the space points containing useful cardiac features.

TI-AWR 1843 radar with \(2\) Tx and \(4\) Rx is used for data collection with \(8\) virtual antenna channels created [24], and the radar configurations are listed in Table 1 with the names provided in TI mmWave-Studio interface. Meanwhile, the ECG ground truth is collected using TI ADS1292, with AC coupling and an integrated right-leg drive (RLD) amplifier to remove potential baseline drift or power-line noise. The related ECG processing (e.g., smoothing and peak finding) is realized by NeuroKit2 Python package [32]. All the signals are sampled at \(200\)Hz, and only a band-pass filter from \(0.5\) to \(50\)Hz and a differentiator are used for removing respiration noise because the radar signal extracted from CF points already has high SNR. Lastly, the CFT algorithm is run on the Apple M2 Pro CPU with \(32\)GB RAM.

4.2 Implementation Details↩︎

4.2.1 Parameters for CFT Algorithm↩︎

The constraint \(\Omega\) for the CF point search is centered at the initial state \(E_0\) with a range of \(0.4\times 0.2 \times 0.4\)m as illustrated in Figure 2 (b). In addition, the initial grid and search region size should be adjusted to fit the real-life physical unit as \(\Gamma_k= \gamma_k = 0.1\)m, and the size will be limited as \(\Gamma_k\geq\gamma_k\geq 0.001\)m to prevent an exhaustive search within a meaningless small space. Then, SNR\(_d\) is empirically set to \(0.01\) for the desired MSE between the normalized synthetic template and the signal envelope, ensuring that the extracted signal contains enough cardiac features for ECG recovery. At last, \(k_{max}\) is empirically set to \(100\) to balance the quality and cost of the space search.

4.2.2 Deep Learning Model Training↩︎

The deep learning model adopts the same backbone, ECG decoder and hyperparameters as in our previous open-sourced work [8] coded in PyTorch and trained on NVIDIA RTX 4090 (24GB), and the total training epoch is set to \(100\) with batch size \(8\). All the experiments are repeated 5 times, and the mean values are taken to ensure the results are statistically reliable, but the confidence interval is not provided because all the improvements show statistical significance.

The dataset is split based on a 5-fold cross-validation training strategy, with the trials from 1 fixed subject for testing and the other 4 subjects alternately selected for training or validation, ensuring to make use of all the possible trials while not involving the testing data in the training phase. During each training iteration, the 4-sec input radar segment is split from each 1-minute trial with a step length of 1 sec, generating a total of 4560 samples for training, evaluation and testing.

4.3 Methods for Comparison↩︎

The comparison is performed with the representative methods based on accumulation and clustering to extract high-SNR radar signals:

• De-ViMo [3] is proposed for real-time heart rate (HR) monitoring and is based on the accumulation of signals from various dimensions (i.e., \(2\) chirps, \(8\) virtual antennas, \(10\) spatial points) to enhance cardiac features while mitigating noise. However, original De-ViMo only generates a sinusoidal wave, which cannot be used for ECG recovery. Therefore, an extra signal extraction stage is needed, but the real-time monitoring will be unachievable.

• MMECG [9] requires the calculation of numerous points in 3D space and applies clustering algorithm to improve SNR. Then, a pattern-matching process is performed to learn the common pattern from the clustered result and select the best radar signal(s). In this study, the number of 3-D points used for clustering is \(6\times 6\times 6 = 216\).

In addition, to the best of our knowledge, this is the first paper that leverages SSL with an appropriate pre-text task for pre-training of radar-based ECG recovery. Therefore, two widely applicable SSL methods for biological signals are used for comparison:

• MaeFE [33] is designed based on the famous masked autoencoder (MAE), with masked reconstruction of signal patches as the pre-text task. To fit our task, each 4-sec input radar signal will be masked using faint Gaussian noises (\(3\) dB) with a length of \(1\) sec.

• The method proposed in [34] is based on contrastive learning (CL) by constructing positive and negative pairs and maximizing similarity between the latent vectors generated for positive pairs while minimizing similarity for negatives. To fit our task, the positive pairs are built by the records taken from two different channels in the same 4‑sec window, while the negative pairs are formed by pairing a window with another window more than 20 seconds apart in time, ensuring a different physiological state driven by changing activity.

The core features of all the methods used in this paper have been summarized in Table 2 for clarity.

4.4 Evaluation Metrics↩︎

The performance of the proposed CFT-RFcardi framework will be separately evaluated in terms of the quality (SNR) of the collected radar signal and the recovered ECG signal using the following metrics:

• For both radar collection and recovered ECG signals, the peak accuracy is crucial because it links to many important cardiac features (e.g., heart rate variability) for clinical diagnosis. Thus, absolute peak error between ECG R peaks and the dominant peaks for the first vibrations (i.e., heartbeats) in the radar or recovered ECG signal is used to measure the peak accuracy.

• Another metric related to peaks is missed detection rate (MDR) to count the cardiac cycles with no peak detected or with the absolute peak error larger than \(150\)ms [9], providing a quantified evaluation of the corrupted radar signal collection or ECG recoveries due to the noise.

• For ECG recovery, the morphological accuracy is measured using MSE and Pearson-correlation coefficient (PCC) as calculated in (7 ), with MSE sensitive to the peak deviation and PCC focusing on the similarity between the ECG patterns.

  \[\label{equ:mse95pcc} \begin{align} \mathrm{MSE}&=\frac{1}{N} \sum_{i=1}^N\left(x_i-x_i^{\prime}\right)^2 \\ \operatorname{PCC}&=\frac{\sum_{i=1}^N\left(x_i-\bar{x}\right)\left(x_i^{\prime}-\bar{x}^{\prime}\right)}{\sqrt{\sum_{i=1}^N\left(x_i-\bar{x}\right)^2} \sqrt{\sum_{i=1}^N\left(x_i^{\prime}-\bar{x}^{\prime}\right)^2}} \end{align}\tag{7}\] where the predicted and ground truth ECG pieces are \(x\), \(x^{\prime}\), with length \(N\) and mean values \(\bar{x}\), \(\bar{x}^{\prime}\).

5 Experimental Results and Evaluations↩︎

5.1 Performance of CFT Algorithm↩︎

5.1.1 Effectiveness of CFT Algorithm↩︎

The examples of the extracted radar signal for different methods are shown in Figure 8, illustrating that precise cardiac localization has a huge effect on the signal quality. For example, if the rough body location is around CF point, all three methods can obtain high-SNR signals with clear first and second vibrations using either space search (CFT, Figure ), clustering (MMECG, Figure ) or accumulation (De-ViMo, Figure ).

In contrast, only a few range bins will contain useful cardiac features if the rough body location is far from CF points, especially when increasing the monitoring range. Therefore, the signal accumulation may enhance the noises as shown in Figure  while the signal clustering may also encounter a failure due to the lack of homogeneous cardiac signals as shown in Figure . However, The proposed CFT can precisely locate the CF point with good SNR subject to the designed signal template and DFO searching strategy, and the extracted radar signal still shows clear peaks as shown in Figure .

During the data collection of this study, the subjects are allowed to change postures to alleviate discomfort, with a resultant CF point deviation of several decimeters, while the rough location provided by FMCW signal processing is still unchanged. Therefore, the proposed CFT algorithm is essential because the posture change is inevitable, and a thorough evaluation in terms of different monitoring ranges will be performed in the next part.

5.1.2 Impact of Monitoring Range↩︎

To evaluate the performance of different methods when increasing the monitoring range, the quality of extracted radar signals for all trials is evaluated in terms of peak error and MDR. Figure  and  illustrate the peak error and MDR for all \(80\) trials, with fitting lines indicating the mean peak error or MDR.

All the methods show similar performance in the short range and experience certain degradation with respect to the increasing range, because the SNR naturally decreases as the monitoring range increases. Following the link budget analysis for a similar radar platform used in [7], the degradation in SNR is around \(14\) dB when the monitoring range increasing from \(0.5\)m to \(1.1\)m, causing a similar trend of degradation as shown in Figure  and .

In particular, MMECG shows larger degradation and variance for longer-range cases because the rough localization based on FMCW signal processing cannot provide accurate cardiac location, and the resultant evaluated points may not capture useful information for clustering. In contrast, De-ViMo could provide a better cardiac location, and the accumulated results show better accuracy than MMECG, while the long-range monitoring still affects the quality because the accumulation is not robust to non-Gaussian noise. At last, the proposed CFT could precisely focus on the CF point regardless of the monitoring range, providing the best results with small variance for both peak error and MDR.

5.1.3 Impact of Posture Change↩︎

As one of the core innovations, CFT could locate and keep tracking the CF point to ensure a high-quality signal acquisition after the posture change. To provide a quantified evaluation, the posture change is counted as the travel distance of the CF points during each \(1\)-minute trial to measure the extent of upper body movement, and the results are shown in Figure  and .

Similar to the impact of range, the peak error and MDR increase with moving distance because the signal collected during the posture change is heavily ruined, also known as random body movement distortion [9]. MMECG and De-ViMo will be affected if the CF points deviate too much from the initial body location and cannot guarantee a high SNR, because they do not have enough features for clustering or accumulation. In contrast, CFT could effectively find the new CF points after rounds of spatial search, and the extracted signal quality is better than the other two methods, as indicated by the fitting lines in Figure  and .

5.1.4 Signal Qualities for All Trials↩︎

The cumulative distribution function (CDF) plots for all trials are shown in Figure  and . The proposed CFT algorithm achieves the best peak error with a median value of \(0.022\) sec, while DE-ViMo and MMECG have worse performances with larger median values of \(0.028\) sec and \(0.033\) sec, respectively. Similarly, the precise localization and tracking of CF point also reduces the MDR for CFT results with a median value of \(14\%\), while DE-ViMo and MMECG may be affected by the accumulated noise or inaccurate cardiac localization with the median MDR of \(17\%\) and \(23\%\), respectively.

5.1.5 Evaluations of Computational Cost↩︎

The outstanding performance of CFT owes to the space search mechanism that triggers new rounds of search if the current points no longer satisfy the desired SNR (SNR\(_d\)). In comparison, MMECG and De-ViMo perform a fixed number of evaluations for every iteration, wasting a huge amount of time to calculate redundant points or channels. For all three methods, the most time-consuming step is to extract signals from points as calculated in (1 ) and (2 ), while the time consumed for clustering, accumulation, and space search is negligible. Therefore, the quantitative computational cost can be estimated by counting the evaluation times for each 4-sec signal segment:

• MMECG takes \(216\) spatial points for clustering, and the total number of evaluations for a \(1\)-minute trial is \(3240\).

• De-ViMo needs to consider the signals extracted from \(2\) chirps, \(8\) virtual antennas, \(10\) spatial points, and the number of evaluations is \(2400\).

• CFT only evaluates the point during space search to get the cost values, and the number of evaluations recorded for each trial is shown in Figure 10.

According to the experiment, each evaluation takes about \(65\)ms, and the overall time consumed for evaluating a \(1\)-minute trial is also shown in Figure 10. Apparently, the proposed CFT saves a huge amount of time compared with other methods, and the time used for space search increases if the subjects change their posture more frequently (i.e., with longer moving distance), as shown in Figure . In addition, there is no obvious relationship shown between evaluation numbers and monitoring ranges as shown in Figure , indicating that the CFT algorithm can keep extracting high-quality radar signals regardless of monitoring range and could save huge computational resources if the subject is static.

5.1.6 Impact of Signal Quality on ECG Recovery↩︎

The signals extracted using different methods are used for supervised training to verify the impact of different input qualities on the ECG recovery task. Table 3 shows the performance of the deep learning model trained with datasets yielded by different methods. The training based on CFT dataset achieves the best results on both morphological accuracy (MSE\(=0.0082\) and PCC\(=85.47\%\)) and R peak recovery (Peak Error\(=7.61\)ms and MDR\(=6.85\%\)), because the high-SNR inputs provide accurate peak locations with minor noise that affects the ECG pattern generation, as shown in Figure  and .

In contrast, MMECG and De-ViMo cannot preserve the signal quality especially for long-distance cases, and the noisy inputs will prevent the deep learning model from identifying the accurate position of ECG pieces, causing large peak error and MDR, as shown in Figure  and . It is worth noticing that poor signal SNR causes more degradation in peak error than morphological accuracy, because the ECG patterns share a similar shape and can be learned from other cardiac cycles, while the peak recovery (detection) fully relies on the current radar input and can be ruined by noises.

5.2 Performance of ECG Recovery using Transfer Learning↩︎

5.2.1 Evaluations and Ablation Studies of SSL in RFcardi↩︎

The SSR task is crucial in the proposed transfer learning framework to provide latent representations that assist further ECG pattern recovery, reducing the demand for radar-ECG pairs in the fine-tuning stage. The results of SSL are shown in Table [tab:ssr] in terms of MSE and sparsity to illustrate the former (MSE term) and latter part (sparse penalty \(\lambda_s\)) in the loss function (6 ) for SSL training.

The experiment is repeated for different dataset scales with the ablation study on the use of sparse penalty, and the results indicate that both MSE and sparsity decrease with the reduction in training data as shown in Table [tab:ssr]. Training with \(100\%\) or \(80\%\) dataset could achieve convergence and realize a successful SSR with similar MSE (\(0.0091, 0.0082, 0.0081\)) or (\(0.0096, 0.0085, 0.0088\)). However, the performance of SSR degrades heavily when further decreasing the training data without the constraint of sparse penalty (\(\lambda_s=0\)), because the SSR results might fluctuate, as shown in Figure . In contrast, introducing the sparse penalty could suppress the fluctuation and force the deep learning model to focus only on the dominant peaks of the input radar signals, as shown in Figure . Therefore, the training with sparse penalty loss still achieves good results with an MSE of \(0.0092\) and \(0.0098\) by using \(60\%\) and \(40\%\) of the dataset, while the model cannot be well-trained without sparse penalty by using \(40\%\) dataset as shown in Table [tab:ssr] and Figure .

It is also revealed in Table [tab:ssr] about the consequence of enlarging the sparse penalty \(\lambda_s\). Considering that even the ground truth shown in Figure 12 has a sparsity of \(0.05\), using a large coefficient (e.g., \(\lambda_s=0.05\)) will overwhelm the MSE term, causing the DL model to generate the result with only \(1\) non-zero value to give a sparsity of \(0\). In Table [tab:ssr], \(\lambda_s=0.02\) gives a higher MSE with lower sparsity compared with \(\lambda_s=0.01\), especially when using \(60\%\) and \(40\%\) of the dataset (MSE\(=0.0118\) and \(0.0124\), sparsity\(=0.024\) and \(0.026\)). In other words, using a large sparse penalty cannot help the convergence when using a limited dataset, but may disrupt the training process due to the dominance of the sparse penalty term, causing suboptimal performance in capturing cardiac-related features (i.e., heartbeats).

5.2.2 Evaluations and Ablation Studies of Fine-tuning Results↩︎

To verify the impact of SSL quality on the fine-tuning result, the same amount of labeled data is used for fine-tuning with different sparse penalty \(\lambda_s\), as shown in Table [tab:ssr]. The overall improvements are also provided in the last column by calculating the percentage of improvement across all four metrics to provide straightforward evaluations for different methods, as shown in Table [tab:ssr] and Figure .

The results coincide with the expectation that more labeled data generates better ECG recovery performance, and the discrepancy between using different \(\lambda_s\) is not obvious if the training data is enough (e.g., for \(100\%\) and \(80\%\)), as indicated in Figure . However, if the labeled data is not enough, the best pre-training model (\(\lambda_s=0.01\)) gives the best ECG recovery for all cases with different labeled data scales, because SSL with \(\lambda_s=0.02\) or \(0\) does not properly train the backbone with the ability to capture the peak-related information. In addition, the gap between using different \(\lambda_s\) is larger than the case with enough data as shown in Figure , revealing the advantage of using SSL for pre-training. According to our experiment, \(\lambda_s=0.01\) guarantees a good performance for different scales of input datasets and will be adopted for the comparison with other SSL methods later.

5.2.3 Comparison with Other SSL Methods↩︎

The comparison is based on the model pre-trained on a \(100\%\) dataset using different SSL methods, and the experiment is repeated for different percentages of labeled data (i.e., radar signal with ECG ground truth). In addition, the same deep-learning model will be trained in a supervised manner by using the same amount of labeled data as the benchmark for transfer-learning ECG recovery. The overall performances are also listed in the last column and plotted in Figure .

The fine-tuning with \(100\%\) labeled data provides very similar performance in morphological accuracy, with the MSE and PCC around \(0.0080\) and \(85.47\%\). It is worth noticing that the peak error and MDR are slightly improved, because the pre-text task SSR for pre-training is equivalent to identifying the peak position of the radar signal, and the learned representations can be seamlessly transferred to improve the accuracy of the recovered ECG R peaks, contributing to the overall improvement for RFcardi (\(3.66\%\) and \(1.75\%\)). Similarly, MaeFE and CL also achieve slight improvement, although they are not pre-trained by the cardiac-feature-related pre-text tasks, enough data still guarantees an effective pre-training and fine-tuning.

Reducing \(20\%\) of labeled data causes a \(10\%\) overall degradation for supervised learning as shown in Table 4, and the decline of peak error and MDR is more than MSE and PCC. The reason is that ECG morphological patterns for different cardiac cycles are similar and can be well-learned from \(80\%\) labeled data with good MSE and PCC (\(0.0084\) and \(84.60\%\)), while the location of each ECG piece is random and can be distorted by noise, requiring more training data for convergence. In addition, \(80\%\) dataset is still enough for MaeFE and CL to achieve a similar performance with RFcardi, with the performance of \(-7.12\%\) and \(-5.19\%\) compared with \(-4.02\%\) for RFcardi with sparse penalty.

The supervised training with \(60\%\) labeled data cannot ensure a good morphological and peak accuracy, and the overall degradation is \(23.37\%\), with the PCC drop below \(80\%\) as shown in Table 4. In contrast, the pre-trained model still provided good results with mild degradations of \(8.71\%\) and \(12.68\%\). It is noticed that the effectiveness of the sparse penalty in the SSL stage also affects the fine-tuning stage, because both peak error and MDR for transfer learning with sparse penalty are better than those without sparse penalty, causing a large gap in the overall improvement compared with the previous training with \(100\%\) and \(80\%\) datasets. In this stage, CL shows a larger gap compared with other SSL methods as shown in Figure , with the overall performance of \(-21.65\%\), close to the result achieved by supervised learning (\(23.27\%\)). The reason is that CL relies on the distinction between positive and negative samples, while the construction designed in [34] is not suitable for cardiac signals, because the cardiac activities may share the same pattern even after applying more than \(20\) seconds of delay.

Lastly, the deep learning model can barely learn from \(40\%\) labeled dataset and yield a bad morphological and peak accuracy for supervised learning. In addition, CL, MaeFE, and RFcardi, without a sparse penalty, all struggle to learn both morphological and peak features from limited data if the pre-text task is not well-trained. CL and MaeFE do not leverage specially designed pre-text to highlight the peak-related features and cannot achieve an effective SSL with limited data, as evaluated in Section 5.2.2. Similarly, RFcardi without a sparse penalty cannot achieve convergence during SSL, with the examples of the recovered ECG signals shown in Figure 14. It is obvious that the well-pretrained model generates the ECG signal with good peak and morphological accuracy after fine-tuning as shown in Figure , while the large peak deviation and bad ECG morphology are caused by the pre-trained models that cannot capture essential peak-related features, as shown in Figure .

5.2.4 Summary of Transfer-learning-based ECG Recovery↩︎

Previous evaluations in terms of SSL and fine-tuning stages have illustrated the ability of the proposed RFcardi to learn from unlabeled data and transfer the knowledge to the ECG recovery task using limited radar-ECG pairs. Here are some key conclusions drawn from Figure 13:

• The performance of supervised learning drops heavily and cannot ensure high-quality ECG recovery after reducing \(40\%\) labeled dataset.

• Transfer learning could enhance the performance of ECG recovery for cases with ample labeled training data, and the quality of the pre-trained model has a minor effect on the final result because the deep learning model could learn from numerous radar-ECG pairs.

• The proposed RFcardi shows outstanding performance in learning latent representations from unlabeled data through a specially designed pre-text task, SSR. In comparison, MaeFE and CL are not designed for this task and cannot effectively capture the peak-related features when the input data is limited.

• For the cases with limited labeled data (\(40\%\), \(60\%\)), the quality of the pre-trained model does matter to alleviate the burden of the deep learning model to learn both morphological ECG patterns and peak locations, as indicated by the increasing gap in both Figure  and .

5.3 Limitations and Future Work↩︎

5.3.1 Real-time Processing↩︎

Figure 10 indicates that the time consumed for each \(1\)-minute trial is around \(30\) sec, but the proposed CFT algorithm still needs improvement to achieve real-time processing, because the searching time can reach \(6.5\) sec (\(k_{max}=100\)) for a \(4\)-sec segment if the collected raw radar signal is heavily distorted or the CF point is far from the current \(E_k\). In future work, real-time processing ability should be considered as an essential ability, and the high-SNR signal collection algorithm should be tested on the edge devices to enable real-life applications.

5.3.2 Energy Consumption↩︎

In this study, the CFT algorithm requires multiple function evaluations, and the RFcardi model involves matrix multiplications and convolutional operations that are computationally intensive during inference. Considering that the mobile platforms typically face strict power budgets, the searching strategy of CFT can be optimized to reduce point evaluations by using adaptive grid adjustment and search directions [35]. For deep learning, model pruning and knowledge distillation can decrease the computational load, reducing both runtime and energy demand during inference [36].

5.3.3 Multi-target Vital Sign Monitoring↩︎

In real-world monitoring, a multi-target scenario will introduce extra interference and also require the separation of the signals reflected from different subjects [2]. The future work should focus on high-resolution and adaptive beamforming to better isolate the signal reflected from individual targets, and the CFT algorithm should be advanced to support multi-target vital signal extraction.

6 Conclusions↩︎

This paper investigates the efficient collection of high-SNR radar signals with ample cardiac features for transfer-learning-based ECG recovery. Previous methods adopted signal accumulation or clustering to suppress the noise, while the rough localization based on FMCW radar cannot accurately reveal the chest region, requiring a time-consuming traverse through a 3D space for compensation. In this paper, a novel CFT algorithm is proposed to dynamically find the points with the best SNR and could track the cardiac location over time if the subjects change posture. In addition, a transfer learning framework RFcardi is designed with SSR as a pre-text task for pre-training to reduce the dependency on cumbersome ECG ground truth collection. The experiments performed in different scenarios prove the feasibility of the CFT-RFcardi framework in radar signal extraction and ECG recovery with limited labeled data, enabling a convenient deployment in new scenarios with limited data for remote healthcare monitoring.

References↩︎