1 Introduction↩︎

Decoding continuous hand joint kinematics from surface electromyography (EMG) is a longstanding challenge with direct implications across a range of wearable human–machine interfaces – from consumer input devices and virtual and augmented reality to clinical applications such as prosthetic control and motor rehabilitation. The clinical setting is especially demanding: despite decades of progress in myoelectric prosthetics, upper-limb prosthesis abandonment rates have remained at approximately 44% over the past two decades, with inadequate device control cited as a primary driver of rejection [1], [2]. A key bottleneck is the gap between the discrete, sequential control strategies deployed in commercial devices [3] and the continuous, simultaneous, proportional control required for natural hand use.

The dominant paradigm remains discrete gesture classification [4]–[7], in which EMG windows are assigned to one of a finite set of predefined postures – an approach that cannot represent the continuous, graded nature of hand movement, limiting user utility.

Contrastive representation learning is a powerful framework for learning transferable, structured embeddings within and across modalities [8]–[10]. CEBRA [11] uses auxiliary variables to guide contrastive sampling for single-encoder neural embeddings. CPEP [12] and EMBridge [13] use cross-modal contrastive pre-training for discrete gesture classification from EMG. No prior work has applied cross-modal contrastive alignment to continuous kinematic regression from EMG.

We present KinEMbed, the first cross-modal contrastive framework for continuous EMG-to-kinematics regression. KinEMbed jointly trains dual encoders – one for windowed EMG features and one for kinematic targets – to align the two modalities on a shared unit hypersphere via a cross-modal InfoNCE loss [10]. The resulting EMG embedding is shaped by the geometric structure of the kinematic space, providing an inductive bias for downstream regression, without requiring kinematic signals at inference time. We evaluate on the NinaPro DB8 dataset [14] of \(N{=}11\) subjects, including two limb-difference subjects under a strict cross-session protocol in which the test session is independently recorded and never used during model selection.

Our main contributions are:

1. We introduce KinEMbed, the first cross-modal contrastive learning framework for continuous hand kinematics regression from EMG, embedding EMG and joint angle vectors as paired modalities in a shared embedding space.

2. We evaluate KinEMbed on a cohort of able-bodied and limb-difference subjects and benchmark it against five representation-learning baselines (PCA, PLS, AE-Recon, AE-Super, CEBRA) spanning the supervised/unsupervised and linear/nonlinear axes, plus ARIMA as a temporal baseline.

2 Related Work↩︎

2.0.0.1 Continuous joint angle regression from EMG.

While discrete gesture classification has dominated the EMG decoding literature, a substantial body of work has pursued continuous joint angle regression for proportional prosthetic control. Classical approaches apply linear regression, kernel ridge regression, Gaussian process regression, or support vector regression to hand-crafted time-domain and frequency-domain feature windows [15]–[17].  [18] evaluated regression-based decoding of finger kinematics on the NinaPro DB8 dataset – the same dataset used in this work – and demonstrated accurate per-subject prediction of five degrees of actuation using EMG features and a regularized Wiener filter. More recent deep learning approaches apply CNNs, LSTMs, and temporal convolutional networks (TCNs) directly to raw or lightly processed EMG signals [19].

2.0.0.2 Contrastive and self-supervised learning for biosignals.

SimCLR [8] and InfoNCE [10] established that contrastive objectives learn representations that transfer well across tasks, with the SimCLR projection head - discarded at inference - decoupling the contrastive geometry from the downstream task. VICReg [20] extends this to non-contrastive variance-invariance-covariance objectives. CEBRA [11] brings contrastive learning to neural recordings, typically raw and high-dimensional rather than hand-crafted features. It can be used either self-supervised (discovery-driven, label-free), sampling positive pairs by temporal proximity, or supervised (hypothesis-driven), using an auxiliary variable such as behaviour as a sampling oracle to define positives; both yield consistent, decodable latent spaces.

Closely related work, CPEP [12] and EMBridge [13], use cross-modal contrastive pre-training for discrete gesture classification, aligning EMG with pose representations, including from large-scale datasets such as emg2pose [21] and kinematics data. In contrast, KinEMbed targets continuous joint angle regression, operating directly on fine-grained kinematics in low-data per-subject clinical settings (including limb difference), rather than predicting gesture labels.

3 Method↩︎

3.0.0.1 Problem Formulation.

Given a window of \(C{=}16\) surface EMG channels, we extract a feature vector \(\mathbf{x} \in \mathbb{R}^{256}\) (detailed below) and wish to predict five continuous degrees of actuation (DoA): thumb rotation, thumb flexion, index flexion, middle flexion, and ring/little flexion, collectively \(\mathbf{y} \in \mathbb{R}^5\). Degrees of actuation are aligned with those of the IH2 Azzurra robotic hand [18], [22], and are extracted from a hand motion capture glove using a transformation matrix, as described in Appendix 6.1.

3.0.0.2 EMG Feature Extraction

EMG signals are windowed at 128 ms (256 samples at 2000 Hz) with a 52 ms stride. Per channel, we compute time-domain features (RMS, Waveform Length, Log Variance, Zero Crossings, Slope Sign Changes, MAV, Willison Amplitude, Hjorth Mobility and Complexity), spectral features (Mean Power Frequency, Median Frequency), and STFT magnitude across five bands. Concatenating over 16 channels yields a 256-dimensional feature vector. This feature group was selected over three alternative variants by measuring baseline regression performance on average across all methods under cross-validated session splits. All features are standardised using per-training-set \(z\)-score normalisation.

3.1 KinEMbed Architecture↩︎

3.1.0.1 Dual encoders.

We train two separate MLPs jointly as shown in Figure 1. The EMG encoder \(f_\text{emg}\) has hidden dimensions \([256, 128]\) with batch normalisation, ReLU activations, and dropout (\(p{=}0.1\)). The kinematic encoder \(f_\text{kin}\) has hidden dimensions \([64, 32]\) with the same activations. Both encoders project to an \(E\)-dimensional space (\(E{=}16\)) followed by \(\ell_2\)-normalisation onto a unit hypersphere:

\[\begin{align} \mathbf{h}_\text{emg} &= \ell_2\!\left(f_\text{emg}(\mathbf{x})\right), \quad \mathbf{h}_\text{emg} \in \mathbb{R}^{16}, \\ \mathbf{z}_\text{kin} &= \ell_2\!\left(f_\text{kin}(\mathbf{y})\right), \quad \mathbf{z}_\text{kin} \in \mathbb{R}^{16}. \end{align}\]

3.1.0.2 Projection head.

Following SimCLR [8], a two-layer nonlinear projection head \(g_{proj}:\mathbb{R}^{16}\!\to\!\mathbb{R}^{16}\) (hidden width 32, batch normalisation and ReLU) is appended to the EMG trunk during contrastive training. The EMG embedding seen by the contrastive loss is therefore the \(\ell_2\)-normalised projection-head output \[\mathbf{z}_\text{emg} = \ell_2\!\left(g_{proj}(\mathbf{h}_\text{emg})\right) \in \mathbb{R}^{16},\] whereas the kinematic branch has no projection head, so \(\mathbf{z}_\text{kin}\) enters the loss directly. At inference the projection head is discarded and the downstream decoder consumes the encoder representation \(h_{emg}\). This decouples the geometry directly optimised by the contrastive loss (over \(\mathbf{z}_\text{emg}\)) from the geometry seen by the decoder (over \(\mathbf{h}_\text{emg}\)).

3.1.0.3 Contrastive objective.

Given a batch of \(N\) synchronised (EMG, kinematic) pairs, we use a cross-modal InfoNCE loss:

\[\mathcal{L}_\text{contr} = -\frac{1}{2N}\sum_{i=1}^{N} \left[ \log \frac{e^{\mathbf{z}_i^\text{emg} \cdot \mathbf{z}_i^\text{kin}/\tau}}{\sum_{j} e^{\mathbf{z}_i^\text{emg} \cdot \mathbf{z}_j^\text{kin}/\tau}} + \log \frac{e^{\mathbf{z}_i^\text{kin} \cdot \mathbf{z}_i^\text{emg}/\tau}}{\sum_{j} e^{\mathbf{z}_i^\text{kin} \cdot \mathbf{z}_j^\text{emg}/\tau}} \right], \label{eq:ntxent}\tag{1}\]

where \(\tau{=}0.2\) is the temperature and positive pairs are temporally aligned (EMG window \(i\), DoA measurement \(i\)). We also evaluated a Soft InfoNCE variant that replaces the one-hot contrastive target with a Gaussian-kernel soft target over DoA distances (addressing the false-negative problem for regression), and VICReg [20]. The standard InfoNCE loss was selected via grid search, although performance differences were limited (Section 4; per-loss results in Appendix 7).

3.1.0.4 Downstream TCN decoder.

After contrastive pre-training, the EMG encoder is frozen and a Temporal Convolutional Network (TCN) [23] decoder is trained to map sequences of EMG embeddings to 5-dimensional DoA predictions. The TCN (\(g\)) uses dilated causal convolutions (64 channels, kernel size 3, dilations \([1,2,4]\)) with weight normalisation and residual connections. The decoder is trained for 100 epochs with Adam [24] (\(\eta{=}10^{-3}\), MSE loss). In order to maximise like-for-like comparison, all representation-learning baselines share this decoder. An MLP, GRU, and TCN were evaluated as potential decoders. TCN was chosen as the best performing on average across all methods, based on the \(R^2\) across a subsection of tested subjects via 2-fold CV. Full training details are available in Appendix 7.

4 Experiments↩︎

4.0.0.1 Dataset and Evaluation Protocol.

We evaluate on the NinaPro DB8 dataset [14], which comprises synchronised EMG and instrumented glove recordings from 12 subjects performing continuous hand grasps and a wide range of finger movements. Subject 4 was excluded due to low-fidelity data, yielding \(N{=}11\) participants: 9 able-bodied (AB) and 2 LD subjects. Although the LD cohort is small, we deliberately retain it, as evaluating on limb-difference data is essential for advancing continuous, proportional control for prosthetic applications, a central long-term goal of this work.

Each subject completed three sessions (d1, d2, d3). Sessions d1 and d2 are used for training; d3 is the strictly held-out test set, never touched during model selection. We use 2-fold session CV during hyperparameter tuning. We report the mean coefficient of determination (\(R^2\)) across the 5 DoA channels on the held-out session d3. Each subject’s \(R^2\) is averaged over three seeds, and Table 1 reports the mean \(\pm\) standard deviation across subjects.

4.0.0.2 Baseline selection.

We initially evaluated seven candidate DR methods in preliminary experiments: PCA, UMAP, PLS, a reconstruction autoencoder (AE-Recon), a supervised autoencoder (AE-Super), a contrastive autoencoder, a conditional VAE (CVAE) and CEBRA. The five baselines retained for the final comparison – PCA, PLS, AE-Recon, AE-Super and CEBRA – were selected by ranking all candidates on mean \(R^2\) under 2-fold session CV and keeping the top performers. The set spans the key methodological axes: unsupervised linear (PCA), supervised linear (PLS), unsupervised nonlinear (AE-Recon), and supervised nonlinear (AE-Super). To the best of our knowledge, this is also the first instance of CEBRA applied to EMG decoding. We also included a non–representation-learning baseline, Autoregressive Integrated Moving Average (ARIMA), to benchmark against a classical statistical time-series model.

4.0.0.3 Baseline configurations.

Because the aim of this work is to introduce KinEMbed rather than to produce an exhaustive benchmark, baseline architectures follow standard designs with minimal tuning. AE-Recon uses a symmetric MLP encoder-decoder (\(256 \to 128 \to 64 \to d\)) trained with MSE reconstruction loss (600 epochs, Adam, \(\eta{=}10^{-4}\)). AE-Super shares the same encoder but adds an auxiliary DoA regression head from the bottleneck; the reconstruction/regression loss weighting \(\alpha\) is selected from \(\{0.1, 0.5, 1.0\}\) via 2-fold session CV. PCA and PLS are fit on the training split using standard sklearn implementations. CEBRA was tuned as described in Appendix 7.3. To ensure a fair comparison, each baseline’s embedding dimension is independently optimised via grid search over \(d \in \{2,3,4,8,16,32\}\) using 2-fold session CV. The canonical dimension is the mode of per-subject best values (PCA: 32, PLS: 32, AE-Recon: 32, AE-Super: 16, CEBRA: 8).

4.1 Results↩︎

Table 1 reports overall mean \(R^2\) across the 5 DoA for all subjects. KinEMbed achieves the highest mean \(R^2\) overall, with a mean of 0.732 (all subjects), 0.769 (AB), and 0.568 (LD). Figure 4 illustrates the \(R^2\) distribution across AB subjects. Within the top-performing methods, KinEMbed shows a tighter inter-quartile distribution. ARIMA, which exploits only the temporal autocorrelation of the DoA signal without any EMG input, achieves negative \(R^2\) across all groups, confirming that EMG data carries essential non-redundant kinematic information. On the LD cohort, KinEMbed and CEBRA attain comparable performance, mean \(R^2\) (0.568 vs., respectively). With only two LD subjects and high inter-subject variance, no firm conclusions on performance can be drawn.

Table 2 breaks results down by DoA for AB subjects. KinEMbed achieves the best mean \(R^2\) and leads on four of five DoA. The largest gains are on thumb flexion (+2.3 pp over the next-best method, CEBRA) and thumb rotation (+1.4 pp), which are the most variable and clinically relevant degrees. The ring/little finger DoA is already well-predicted by all methods and differences are negligible, as shown in Figure 2.

5 Discussion and Conclusion↩︎

We present KinEMbed as an initial demonstration that cross-modal contrastive alignment is a viable and competitive paradigm for continuous decoding of hand kinematics from EMG, rather than as an exhaustively optimised system. Throughout this work, we prioritised controlled, like-for-like comparison over absolute performance: every representation method is given the same hand-crafted feature input and the same downstream decoder. The decoder exemplifies this trade-off – we adopt a TCN because it was the best performer on average across all methods, a choice that limits confounders but is, by design, not tailored to any individual method, KinEMbed included. Such conservative choices make the reported advantage more naturally read as a lower bound than a ceiling: a feature pipeline, encoder, or decoder optimised specifically for each KinEMbed could plausibly widen the margin. A more exhaustive exploration of these components is a clear and promising next step.

While the cohort is limited in size, the evaluation is deliberately stringent: every method is assessed under a strict cross-session protocol, training and testing on independently recorded sessions. This is the appropriate standard for both commercial and clinically motivated applications due to distribution shift caused by a variety of factors including fatigue, sensor placement, and changes in the signal-to-noise ratio. This protocol is considerably more demanding than the within-session splits commonly seen in the literature.

5.0.0.1 Limitations.

This work introduces KinEMbed and presents an initial evaluation against relevant baselines; however several limitations temper its conclusions. (i) Improvements over the strongest baselines are small in absolute terms and, with only \(N{=}11\) subjects, the study is underpowered to establish statistical significance; we therefore treat the method rankings as suggestive and defer significance testing on larger cohorts to future work. (ii) The limb-difference cohort comprises only two subjects, too few to draw firm conclusions for prosthetic applications. (iii) We do not characterise when the false-negative problem matters: the Soft InfoNCE variant designed to address it did not outperform the standard InfoNCE in our limited grid search, leaving open whether the effect is genuinely mild or simply undetected at this scale.

5.0.0.2 Future directions.

The most natural next step is stronger evaluation across larger and more diverse cohorts – in particular more participants with limb difference. Together with paired significance tests and bootstrap confidence intervals, this would allow for more conclusive analysis on the performance of this methodology. A complementary direction is mechanistic analysis of the learned representation and a more systematic study of the contrastive objective –including the conditions under which the Soft-InfoNCE variant meaningfully mitigates the false-negative problem – which would resolve the question left open above. Several methodological extensions also follow naturally. Similarly, temporal contrastive objectives (e.g., CPC-style) could exploit the sequential structure of EMG more explicitly.

Cross-subject and cross-session adaptation, via contrastive pre-training on pooled data followed by subject-specific fine-tuning, offers a direct path to clinical deployment with reduced per-user calibration. The dual-encoder design is also well suited to foundation-model pre-training, in which a kinematic encoder pre-trained on large motion-capture corpora supplies a rich target embedding space without requiring synchronised EMG.

We hope KinEMbed serves as a useful baseline for EMG regression tasks and a conceptual stepping stone for representation learning on structured wearable biosignals.

Acknowledgments↩︎

S.G. thanks Dr Sadra Sadeh for his early mentorship on a closely related project, which sparked her interest in this area of research.

Impact Statement↩︎

This paper presents work whose goal it is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

6 Dataset↩︎

We evaluate KinEMbed and baselines on the NinaPro DB8 dataset. The EMG data in DB8 was recorded using 16 active double-differential wireless sensors from a Delsys Trigno IM Wireless EMG system [25] as shown in Figure 5. Muscle activity was recorded from the participants’ right forearm (i.e., the remnant limb for subjects with limb difference). Motion capture data was recorded with a Cyberglove II - a motion capture glove that contains 18 joint-angle measurement sensors, distributed as shown in Figure 6 (b).

6.1 Cyberglove Data↩︎

Although the Cyberglove II has 18 joint measurement sensors, the objective of this research is the control of an external device such as a dexterous robotic hand, or a simulated hand. Hence, a mapping strategy is employed to translate the 18 channels of joint angle data to DoA of a robotic hand. The mapping of Cyberglove joint states to hand position reconstruction has been explored for visualisation [28] and robot manipulation [29] and has further been refined for the actuation of a robotic hand for prostheses using a linear mapping from the 18 joint states to the 5 DoA of the IH2 Azzurra robotic hand [18], [22]. In this transformation, the 18 calibrated measurements of the dataglove (\(\boldsymbol{x} \in \mathbb{R}^{18}\)) are mapped to the DoA of the robotic hand (\(\boldsymbol{y} \in \mathbb{R}^{5}\)) via the transformation matrix in Eq.(3 ). The DoA correspond to the movement of the five fingers: \(y_1\) , thumb rotation; \(y_2\), thumb flexion; \(y_3\), index flexion; \(y_4\), middle flexion; \(y_5\), ring/little finger flexion. The ring and little finger are controlled together due to a mechanical coupling in the Azzurra robotic hand [18]. Using this model allows for a more explainable decoding performance and renders the model more directly transferable for the control of an external device.

\[\boldsymbol{y = Ax} \label{eq:linear-mapping-equation}\tag{2}\]

\[\label{eq:matrix95AT} A^{T} = \begin{bmatrix} 0.639 & 0 & 0 & 0 & 0 \\ 0.383 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ -0.639 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.4 & 0 & 0 \\ 0 & 0 & 0.6 & 0 & 0 \\ 0 & 0 & 0 & 0.4 & 0 \\ 0 & 0 & 0 & 0.6 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.1667 \\ 0 & 0 & 0 & 0 & 0.3333 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.1667 \\ 0 & 0 & 0 & 0 & 0.3333 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ -0.19 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ \end{bmatrix}\tag{3}\]

7 Training and Hyperparameter Optimisation Details↩︎

7.1 KinEMbed↩︎

Contrastive pre-training uses AdamW [30] with weight decay \(10^{-4}\), learning rate \(3{\times}10^{-3}\), cosine annealing (\(T_\text{max}{=}200\)), batch size 256, and gradient clipping (norm 1.0). Training runs for up to 200 epochs with early stopping (patience 20) on a fixed 10% validation split.

Hyperparameters were selected via a principled 4-stage sequential grid search, designed to explore each axis of variation independently while fixing decisions from prior stages. This staged strategy reduces the effective search space from a prohibitive full factorial (tens of thousands of combinations) to targeted combinations evaluated on 5 representative subjects (1, 3, 6, 8, 11; all able-bodied except subject 11) via 2-fold session CV, using mean CV \(R^2\) as the selection criterion.

Stage 1: loss \(\in\) , temperature \(\tau \in \{0.05, 0.07, 0.1, 0.2, 0.5\}\), embedding dim \(d \in \{4,8,16,32,64\}\). Winner: InfoNCE, \(\tau{=}0.2\), \(d{=}16\).

The full per-loss comparison (best configuration per loss; mean 2-fold CV \(R^2\) across the five tuning subjects) is given in Table 3. Standard InfoNCE marginally outperformed the Soft-InfoNCE variant, with VICReg lowest. Although the Soft-InfoNCE target was motivated by the false-negative problem for continuous regression (Appendix 10), it did not improve over standard InfoNCE in this setting, suggesting the false-negative effect is empirically mild in our batch-local, per-subject regime; a larger-scale study of this variant is warranted.

Stage 2: architecture variant \(\in\) . Winner: projection head.

Table 4 reports the architecture-variant comparison. The projection head gave only a marginal improvement over the plain (standard) encoder, indicating that most of KinEMbed’s benefit derives from the cross-modal contrastive objective rather than the projection-head decoupling itself; the momentum-EMA variant was markedly unstable.

Stage 3: EMG hidden dims \(\in \{[128,64],[256,128],[512,256,128], [512,256,128,64]\}\), kinematic hidden dims \(\in \{[32],[64,32],[128,64]\}\). Winner: EMG \([256,128]\), kin \([64,32]\).

Stage 4: learning rate \(\in \{10^{-4}, 5{\times}10^{-4}, 10^{-3}, 3{\times}10^{-3}\}\), batch size \(\in \{128,256,512\}\), dropout \(\in \{0.0, 0.1, 0.2\}\). Winner: \(\eta{=}3{\times}10^{-3}\), batch 256, dropout 0.1.

7.2 ARIMA↩︎

As a time-series baseline, we evaluated an autoregressive integrated moving average (ARIMA) model applied independently to each degree of actuation (DoA). Unlike the EMG-feature-based methods, ARIMA operates solely on the DoA time series, exploiting temporal autocorrelation in the kinematic signal rather than instantaneous EMG observations.

Model orders \((p, d, q)\) were selected per-DoA and per-subject using the auto_arima procedure from the pmdarima library [31], with the Bayesian Information Criterion (BIC) as the selection objective. BIC was preferred over AIC as its stronger complexity penalty yields parsimonious models that generalise well from limited data. A stepwise search [32] was used in place of an exhaustive grid to reduce computation. Seasonal components were not modelled, as the DoA signal has no fixed periodic structure.

Order selection was performed on the final 20% of the second training session (dataset 2), a single contiguous recording that is temporally adjacent to the held-out test session (dataset 3). This segment provides a representative characterisation of the signal’s local autocorrelation structure; BIC-based order selection is known to stabilise with a few hundred observations [33].

Evaluation followed the same protocol as all other methods: models were applied to dataset 3.

7.3 CEBRA↩︎

CEBRA hyperparameters were tuned via Optuna Tree-structured Parzen Estimator (TPE) search [34], run independently for each subject using 2-fold session cross-validation with mean \(R^2\) as the objective. Up to 50 trials per subject were evaluated over a categorical search space spanning training iterations \(\in \{1000, 3000, 5000, 10000\}\), batch size \(\in \{256, 512, 1024\}\), learning rate \(\in \{10^{-4}, 5{\times}10^{-4}, 10^{-3}\}\), temperature \(\in \{0.5, 1.0, 2.0\}\), and time offset \(\in \{5, 10, 20\}\) windows. Canonical hyperparameters were set to the mode across subjects, yielding: batch size 512, learning rate \(10^{-3}\), 1000 iterations, temperature 2.0, time offset 10, and embedding dimension 8 (selected in a separate dimensionality study); these values were fixed for all final evaluations.

8 Additional Results↩︎

9 Relationship to CEBRA↩︎

CEBRA [11] is a contrastive representation learning framework designed to produce structured latent embeddings of neural or physiological recordings by leveraging a continuous auxiliary behavioural variable (or operating in a self-supervised fashion using temporal proximity). It has shown strong performance for neural data, and is thus a relevant baseline for our work. Although KinEMbed shares the broad goal of learning an auxiliary-modality-structured embedding of a primary physiological signal, the two methods differ in architecture, supervisory mechanism, and the geometric properties of the resulting embedding space.

9.0.0.1 Architecture.

CEBRA employs a single-encoder design: only the primary signal is passed through a parametric network, a temporal convolutional network (TCN) with learnable temperature [11]. Kinematics serve as a sampling oracle that defines which pairs of EMG windows should be close in the embedding space. KinEMbed instead uses a dual-encoder design: an MLP encoder processes EMG windows and a separate, shallower MLP encoder processes the corresponding DoA vectors. Both encoders are trained jointly, and their outputs are aligned in a shared embedding space.

9.0.0.2 Positive pair construction.

CEBRA constructs positive pairs via a global kinematic nearest-neighbour lookup: for each anchor EMG window, it searches the entire dataset for the recording whose simultaneous kinematics most closely matches a sampled kinematic target state [11]. KinEMbed uses a simpler synchronous scheme: the EMG window at time \(t\) is paired directly with the simultaneously recorded DoA vector \(y_t\), with all other pairs in the mini-batch serving as negatives.

10 Relationship to CPEP↩︎

A concurrent line of work, CPEP [12], applies cross-modal contrastive learning between surface EMG and hand pose in a broadly similar spirit to KinEMbed. Both methods train a dual-encoder architecture using synchronised EMG–kinematic pairs and discard the kinematic encoder at inference. Despite this surface similarity, the two methods differ fundamentally in task formulation, architectural design, loss construction, and intended clinical context. We detail these differences below.

10.0.0.1 Task formulation.

The most consequential difference is the prediction target. KinEMbed is designed for continuous regression: the decoder produces real-valued estimates of five joint angles simultaneously, evaluated by coefficient of determination (\(R^2\)) on a held-out recording session. CPEP targets discrete gesture classification. This distinction is not superficial. Continuous kinematic regression enables direct, proportional control, which is essential for clinical settings as well as fluid interaction in virtual and augmented reality; discrete gesture recognition is a coarse proxy. Every subsequent design choice in each method follows from this task difference.

10.0.0.2 The false-negative problem in regression.

Because KinEMbed targets a continuous output space, the standard InfoNCE objective introduces a structural problem that does not arise in classification: two windows recorded at different times but with nearly identical joint angles are treated as hard negatives and actively pushed apart in the embedding space, despite representing the same kinematic state. Although not used for final evaluation, we propose to address this with a Soft-InfoNCE variant that replaces one-hot positive targets with a Gaussian kernel over pairwise DoA distances, weighting each negative by its kinematic proximity to the anchor and using the median pairwise distance as a parameter-free bandwidth. Future work should explore the value of this loss further. CPEP does not address this problem, nor does it need to: in a discrete classification setting, two windows from different gesture classes are genuine negatives by construction.

10.0.0.3 Architecture and feature representation.

KinEMbed uses lightweight MLP encoders operating on 256-dimensional EMG feature vectors (time-domain and spectral statistics per channel), producing 16-dimensional embeddings. A SimCLR-style projection head is appended during contrastive training and discarded at inference, decoupling the contrastive geometry from the downstream regression space. The downstream decoder is a Temporal Convolutional Network (TCN) trained on frozen embeddings to regress the five joint angles. CPEP uses Transformer encoders (four layers, \(d{=}256\)) operating on raw 2-second EMG waveforms pre-trained via Masked Autoencoder (MAE) [35], producing 256-dimensional embeddings. The downstream task being classification, no regression decoder is trained.

10.0.0.4 Dataset scale and subject population.

KinEMbed is evaluated on NinaPro DB8 [14], comprising 11 subjects (9 able-bodied, 2 with limb difference) across three recording sessions each. The inclusion of limb-different participants introduces substantial variability in neuromuscular structure and EMG signal characteristics, providing a challenging testbed for representation robustness under distribution shift. CPEP is evaluated on the emg2pose dataset [21], comprising 193 participants and approximately 370 hours of recording, and focuses entirely on able-bodied users. The scale difference reflects different scientific goals: CPEP demonstrates large-scale pre-training and zero-shot transfer across users and unseen gestures; KinEMbed demonstrates strong potential for contrastive kinematic alignment in the low-data, per-subject regime characteristic of EMG-based control.

10.0.0.5 Evaluation protocol.

KinEMbed uses a strict cross-session evaluation protocol: models are trained on sessions 1 and 2 and evaluated on the independently recorded session 3, with no overlap. This directly tests the robustness of the embedding to electrode shift and inter-session variability – the primary failure mode for clinical EMG decoders. CPEP evaluates cross-user generalisation and zero-shot transfer to held-out gesture classes, which is the relevant generalisation axis for a large-scale consumer application.

10.0.0.6 Summary.

KinEMbed and CPEP share the cross-modal contrastive pre-training paradigm but diverge in every dimension that matters for their respective applications. KinEMbed is a continuous regression framework, operating in the low-data regime with limb difference subjects and evaluated by joint angle prediction accuracy. CPEP is a discrete classification method for large-scale consumer gesture recognition, operating with orders of magnitude more data and evaluated by gesture identification accuracy. The surface resemblance in training objective reflects the generality of the cross-modal contrastive framework; the differences in task, loss, architecture, data, and evaluation reflect the distinct requirements of the two applications.

References↩︎