1 Introduction↩︎

Surface electromyography (sEMG) captures the electrical activity of muscle fibers and motor neurons through non-invasive sensors, providing rich insight into muscle activation during movement [1]. In particular, sEMG-based hand gesture recognition seeks to decode these signals to identify discrete hand movements, enabling intuitive control of robotic systems, prosthetic devices, and assistive technologies—especially for individuals with motor impairments. Owing to its non-invasive nature and high temporal resolution, sEMG has been widely adopted in applications such as human–computer interaction, virtual and augmented reality, rehabilitation, and wearable robotics [2]–[4]. Despite these advances, achieving reliable performance across different users and recording conditions remains a fundamental challenge. In practice, models must learn representations that generalize beyond subject-specific characteristics and session-dependent variations, which often limits their robustness in real-world deployments.

sEMG signals are inherently complex and highly variable [5], exhibiting strong sensitivity to both inter-subject differences [6] and inter-session variability [7]. These variations arise from intrinsic factors, such as differences in muscle anatomy and physiology across individuals, as well as extrinsic factors, including electrode displacement, variations in skin–electrode contact, and changes in muscle condition across recording sessions. As a result, sEMG feature representations can vary significantly even when different subjects perform the same gesture under nominally identical conditions. This pronounced subject-dependent variability poses a major obstacle to cross-subject generalization, as models trained on a limited set of users often capture subject-specific patterns rather than task-relevant muscle activation dynamics. Notably, the individuality encoded in sEMG signals is so strong that it has been successfully leveraged for biometric identification, further underscoring the challenge of separating gesture-related information from subject-specific signatures [8]–[11].

Several strategies have been explored to address inter-subject variability in sEMG-based gesture recognition, with increasing emphasis on learning subject-invariant representations. Early approaches relied on carefully designed handcrafted features to improve robustness across users and recording conditions [9], [12]. However, the inherently strong subject-specific nature of sEMG signals limits their effectiveness. Subsequently, transfer learning and domain adaptation methods were introduced to mitigate cross-subject performance degradation by adapting models trained on source subjects to a target user [13], [14]. While these approaches can improve performance, they typically require access to target-subject data and may suffer from error accumulation when pseudo-labels become unreliable under large inter-subject distribution shifts. More recent work has explored adversarial learning as a means of encouraging subject-invariant representations without explicit target-domain supervision. By introducing a subject discriminator and training the encoder to confuse it, these methods aim to suppress subject-identifying information in the latent space. Representative examples include multi-source adversarial alignment and feature disentanglement approaches that explicitly promote invariance across subjects [11], [15]. However, adversarial objectives alone do not explicitly enforce separation between gesture classes and may lead to over-alignment or class overlap in the latent space. Recent benchmark studies, such as EMGBench [10], evaluate a broad range of models and training strategies under cross-subject and cross-session settings, and show that performance remains limited under strong distribution shifts. These findings highlight the difficulty of learning representations that generalize across subjects and suggest that invariance alone is insufficient without strong task-discriminative constraints. Recent work has explored Lipschitz-based constraints to improve robustness of sEMG classifiers to input perturbations [16]; however, such approaches primarily focus on stability and do not explicitly address cross-subject variability. Moreover, Lipschitz continuity has been studied as a principled framework for controlling gradient behavior and improving training stability in deep networks [17], [18]. In this context, it provides a useful interpretation of how the scale of a loss function relates to the magnitude of its gradients during optimization. These insights motivate the use of adaptive mechanisms to balance the influence of adversarial and discriminative objectives in multi-objective training.

Alongside domain adaptation and adversarial strategies, deep learning models have been widely adopted to exploit the rich temporal and spectral characteristics of sEMG signals. Convolutional neural networks (CNNs) have been used to automatically extract discriminative features from both time and frequency domains [19], [20]. In addition, hybrid architectures that combine CNNs with recurrent neural networks (RNNs) have been proposed to more effectively capture the spatiotemporal dependencies in muscle activation patterns [21]–[23]. Although these models significantly enhance representation capacity and recognition accuracy, increased expressive power alone does not guarantee robustness to subject variability. In practice, deep models often learn latent representations in which gesture-related information is entangled with subject-specific characteristics, limiting their ability to generalize across users [24]. To address this limitation, recent work has explored explicit disentanglement strategies aimed at separating task-relevant and subject-dependent factors in the latent space. For instance, Fan et al. [11] proposed a multi-encoder architecture with a shared decoder to decompose sEMG features into gesture-related and subject-related components, demonstrating improved robustness in cross-subject and cross-session settings. Similarly, Zhang et al. [25] introduced an extended variational autoencoder (VAE) framework that partitions the latent space into gesture-specific, subject-specific, and residual components to enhance generalization. Despite these advances, existing disentanglement-based approaches typically rely on architectural separation or reconstruction objectives and do not explicitly enforce a discriminative structure within the gesture-related latent representation. As a result, the learned features may still lack sufficient intra-class compactness or inter-class separability. Furthermore, many of these approaches depend on fixed weighting of multiple objectives, which can lead to unstable optimization when the relative scales of adversarial and discriminative losses differ significantly. This limitation highlights the need for adaptive mechanisms that balance competing objectives during training.

sEMG signals exhibit strong subject-dependent and session-dependent variability, posing a major challenge for reliable cross-subject gesture recognition. To address this issue, we propose a conservative subject-invariant learning framework that jointly optimizes multiple complementary objectives acting directly on the encoder representation. Here, conservative denotes a training strategy that prevents any single objective from dominating the optimization process, thereby promoting stable and balanced multi-objective learning. Specifically, the framework integrates three components. First, an adversarial subject loss enforces subject invariance by encouraging the encoder, via a gradient-reversal mechanism, to remove subject-identifying information and align latent representations across users. Second, a gesture classification loss promotes intra-class compactness, ensuring that samples corresponding to the same gesture form cohesive clusters in the latent space despite variability in muscle activation, electrode displacement, or noise. Third, a triplet loss explicitly enforces inter-class separability by increasing the margin between embeddings of different gestures, thereby reducing overlap between gesture manifolds under subject or session shifts. To ensure stable optimization of these competing objectives, we further introduce a Lipschitz-inspired adaptive weighting mechanism that dynamically balances their contributions based on their relative scales. By combining structured representation learning with conservative, adaptive optimization, the proposed approach learns a latent representation that is simultaneously subject-invariant, discriminative, and well structured, providing a robust foundation for cross-subject sEMG-based gesture recognition. Unlike existing approaches, it explicitly addresses both representation structure and optimization stability within a unified framework. The main contributions of this paper are summarized as follows:

• We propose a conservative subject-invariant representation learning framework for sEMG-based gesture recognition that addresses inter-subject variability without requiring target-subject adaptation.

• We develop a multi-objective learning strategy that jointly integrates adversarial subject confusion, gesture classification, and triplet-based metric learning to enforce invariance, intra-class compactness, and inter-class separability.

• We introduce a Lipschitz-inspired adaptive weighting mechanism that dynamically balances competing objectives during training, improving optimization stability and reducing sensitivity to manual hyperparameter tuning.

• We demonstrate through extensive cross-subject evaluations that the proposed framework achieves more robust and consistent performance than existing adversarial and discriminative approaches.

The remainder of this paper is organized as follows. Section 2 provides the proposed conservative subject-invariant network (CSIN) methodology. Section 3 presents the experimental setup and results. Section 4 discusses future work. Finally, Section 5 concludes the paper. A list of notations used throughout the paper is provided in Table 1.

2 Proposed method↩︎

In this section, we present the proposed conservative subject-invariant network (CSIN) for gesture recognition from sEMG signals. The overall architecture is shown in Fig. 1. CSIN is built on a multi-head neural architecture with a shared feature encoder, designed to learn a latent representation invariant to subject-specific variations while preserving gesture-related factors. The method aims to obtain a representation that is simultaneously intra-class compact, inter-class separable, and subject invariant. Intra-class compactness is encouraged by the gesture classification objective, which drives samples of the same gesture toward cohesive clusters in the latent space. Inter-class separability is enforced using a triplet loss, which increases the relative distance between embeddings of distinct gestures. To reduce subject-specific effects, we use an adversarial strategy: a subject classifier attempts to identify the subject, while the encoder, through gradient reversal, is trained to remove this information from the representation. To ensure stable and balanced optimization of these competing objectives, we further introduce a Lipschitz-inspired adaptive weighting strategy that dynamically adjusts the contributions of the adversarial and triplet losses with respect to the primary gesture-classification objective. By jointly optimizing these objectives with adaptive loss balancing, CSIN learns a representation that remains discriminative while being more robust to subject variability, leading to improved cross-subject generalization.

Let \(\mathbf{x} \in \mathbb{R}^{d}\) denote the extracted feature vector from a window of sEMG signals, and let \(y_g \in \{1,\dots, C\}\) and \(y_s \in \{1,\dots, S\}\) denote the corresponding gesture and subject labels, respectively. A shared encoder is used to transform the input feature vector into a latent embedding: \[\mathbf{z} = f_{\theta}(\mathbf{x}) \in \mathbb{R}^{p},\] where \(\theta\) denotes the encoder parameters and \(p\) is the embedding dimension. The encoder maps the input to a latent embedding \(\mathbf{z}\) that preserves gesture-discriminative information while suppressing subject-specific variations, forming the basis for learning representations that are both discriminative and invariant across subjects.

2.1 Multi-head output structure↩︎

The proposed architecture employs two primary prediction heads operating on the shared latent representation \(\mathbf{z}\): a gesture classification head and a subject classification head.

The gesture head is defined as: \[\hat{\mathbf{p}}_g = h_{\phi}(\mathbf{z}) = \mathrm{softmax}(\mathbf{W}_g \mathbf{z} + \mathbf{b}_g),\] where \(\phi\) denotes the parameters of the gesture classifier. This head estimates the class probabilities over the \(C\) gesture classes.

The subject head is defined as: \[\hat{\mathbf{p}}_s = h_{\psi}(\mathbf{z}) = \mathrm{softmax}(\mathbf{W}_s \mathbf{z} + \mathbf{b}_s),\] where \(\psi\) denotes the parameters of the subject classifier. This head predicts the subject identity and is used exclusively for adversarial training.

During optimization, the gesture head encourages the encoder to learn gesture-discriminative representations. In contrast, the subject head is coupled with a gradient reversal layer, which inverts the gradients propagated to the encoder during backpropagation. As a result, the encoder is trained to suppress subject-specific information while retaining task-relevant features.

2.2 Loss functions↩︎

The proposed framework is formulated as a multi-objective optimization problem on the shared latent representation. Specifically, it jointly optimizes three objectives: (i) gesture classification, (ii) adversarial subject classification for invariance, and (iii) triplet-based metric regularization.

1) Gesture Classification Loss↩︎

The gesture head is trained using the standard cross-entropy loss as follows: \[\mathcal{L}_{\mathrm{gest}} = - \sum_{c=1}^{C} \mathbb{I}(y_g = c)\, \log \hat{p}_g(c),\] which encourages the encoder to learn gesture-discriminative representations. This term serves as the primary task objective and acts as the reference scale for balancing auxiliary losses during training.

2) Adversarial subject loss↩︎

To suppress subject-dependent information in the latent representation, a subject classifier is trained jointly with the encoder. The subject head minimizes the cross-entropy loss as follows: \[\mathcal{L}_{\mathrm{sub}} = - \sum_{s=1}^{S} \mathbb{I}(y_s = s)\, \log \hat{p}_s(s).\]

To enforce subject invariance, a gradient reversal mechanism is applied between the subject head and the encoder. During backpropagation, the gradients from the subject loss are multiplied by \(-1\) before reaching the encoder. As a result, the subject head is trained to correctly predict subject identity, while the encoder is trained adversarially to remove subject-discriminative information. The adversarial contribution to the encoder objective is therefore given by \(\mathcal{L}_{\mathrm{adv}} = - \mathcal{L}_{\mathrm{sub}}\). This adversarial interaction encourages the latent representation to be invariant to subject identity while retaining task-relevant gesture information.

3) Triplet loss↩︎

To enhance the discriminative structure of the latent space, we employ a triplet loss that enforces both intra-class compactness and inter-class separation. Each training sample is organized into triplets of the form \((\mathbf{z}^a_i,\mathbf{z}^p_i,\mathbf{z}^n_i)\), where the anchor and positive embeddings correspond to the same gesture class (typically from different subjects). In contrast, the negative embedding belongs to a different gesture class.

Using the squared Euclidean distance as follows: \[d(\mathbf{u},\mathbf{v}) = \|\mathbf{u}-\mathbf{v}\|_2^2,\] the triplet loss is defined as: \[\mathcal{L}_{\mathrm{trip}} = \frac{1}{N}\sum_{i=1}^{N} \left[ d(\mathbf{z}^a_i,\mathbf{z}^p_i)-d(\mathbf{z}^a_i,\mathbf{z}^n_i)+\alpha\right]_+,\] where \(\alpha > 0\) is a margin parameter and \([\cdot]_+ = \max(0,\cdot)\). This objective brings embeddings of the same gesture closer together while pushing those of different gestures further apart, improving class separability and reducing overlap in the latent space.

2.3 Lipschitz-adaptive multi-objective optimization↩︎

A key challenge in joint optimization is the relative scaling of the adversarial and triplet objectives with respect to the main gesture-classification loss. Fixed coefficients often require extensive manual tuning and may lead to unstable training when one auxiliary objective dominates the others. To address this issue, we introduce a Lipschitz-inspired adaptive weighting strategy that dynamically adjusts the weights of the adversarial and triplet losses during optimization.

The total loss used is written as \[\mathcal{L}_{\mathrm{total}}=\lambda_{\mathrm{gest}} \mathcal{L}_{\mathrm{gest}}+\lambda_{\mathrm{adv}} \mathcal{L}_{\mathrm{sub}}+\lambda_{\mathrm{trip}} \mathcal{L}_{\mathrm{trip}}+\lambda_{\mathrm{wd}} \|\Theta\|_2^2,\] where \(\Theta\) collects all trainable weights and \(\lambda_{\mathrm{wd}}\) denotes the weight-decay coefficient. Since the encoder is trained adversarially with respect to subject identity, the loss used to compute gradients for the shared encoder becomes \[\label{eq:encoder95loss95placeholder} \mathcal{L}_{\mathrm{enc}}=\lambda_{\mathrm{gest}} \mathcal{L}_{\mathrm{gest}}-\lambda_{\mathrm{adv}} \mathcal{L}_{\mathrm{sub}}+\lambda_{\mathrm{trip}} \mathcal{L}_{\mathrm{trip}}+\lambda_{\mathrm{wd}} \|\Theta\|_2^2.\tag{1}\]

In contrast, the subject-classification head itself is trained in the standard direction to minimize \(\mathcal{L}_{\mathrm{sub}}\). This results in the usual adversarial interaction between the encoder and the subject classifier.

To avoid manually fixing \(\lambda_{\mathrm{adv}}\) and \(\lambda_{\mathrm{trip}}\), we estimate their relative magnitudes with respect to the gesture loss. To simplify the notation, we treat the subject classification loss \(\mathcal{L}_{\mathrm{sub}}\) as a proxy for the adversarial loss magnitude, as both operate on a similar scale. Let \[r_{\mathrm{adv}} = \frac{\mathcal{L}_{\mathrm{sub}}}{\mathcal{L}_{\mathrm{gest}} + \epsilon}, \qquad r_{\mathrm{trip}} = \frac{\mathcal{L}_{\mathrm{trip}}}{\mathcal{L}_{\mathrm{gest}} + \epsilon},\] computed at predefined intervals during training, whenever \(\mathcal{L}_{\mathrm{gest}}\) is sufficiently large. These ratios act as practical proxies for the relative Lipschitz scales of the objectives, since the magnitude of a loss is related to the norm of its gradients during stochastic optimization. We then maintain exponentially smoothed estimates as follows: \[\hat{L}_{\mathrm{adv}}^{(t)}=\beta \hat{L}_{\mathrm{adv}}^{(t-1)}+(1-\beta)\, r_{\mathrm{adv}}^{(t)},\]

\[\hat{L}_{\mathrm{trip}}^{(t)}=\beta \hat{L}_{\mathrm{trip}}^{(t-1)}+(1-\beta)\, r_{\mathrm{trip}}^{(t)},\] where \(\beta \in [0,1)\) is a smoothing coefficient.

The adaptive weights are then updated inversely proportional to these estimated scales, \[\lambda_{\mathrm{adv}}^{(t)} = \frac{\lambda_{\mathrm{gest}}}{\hat{L}_{\mathrm{adv}}^{(t)}}, \qquad \lambda_{\mathrm{trip}}^{(t)} = \frac{\lambda_{\mathrm{gest}}}{\hat{L}_{\mathrm{trip}}^{(t)}}.\] This update rule ensures that auxiliary objectives remain on a scale comparable to the main classification objective, thereby improving optimization stability and reducing the need for exhaustive hyperparameter tuning.

To improve stability, the adaptive updates are applied only after an initial warm-up phase. The resulting weights are also clipped to predefined ranges, \[\lambda_{\mathrm{adv}} \in [\lambda_{\mathrm{adv}}^{\min}, \lambda_{\mathrm{adv}}^{\max}], \qquad \lambda_{\mathrm{trip}} \in [\lambda_{\mathrm{trip}}^{\min}, \lambda_{\mathrm{trip}}^{\max}],\] to avoid extreme values caused by noisy mini-batch estimates.

This Lipschitz-adaptive weighting strategy provides two main advantages. First, it prevents the adversarial and metric-learning terms from dominating the gesture objective. Second, it reduces the sensitivity of the training process to dataset-specific tuning, which is particularly important in cross-subject EMG learning, where the balance between invariance and discrimination can vary across datasets.

2.3.0.1 Why Lipschitz-Adaptive Balancing Improves Training Stability

The proposed adaptive weighting strategy can be interpreted through the lens of Lipschitz continuity and gradient scaling. In multi-objective optimization, each loss term contributes gradients of potentially different magnitudes with respect to the shared encoder parameters. When fixed coefficients are used, the optimization process becomes highly sensitive to the relative scales of the gradients, often leading to one objective dominating the others or to unstable oscillatory behavior.

By estimating the relative magnitude of each auxiliary loss with respect to the primary gesture-classification loss, the proposed method effectively balances their gradient contributions. Since the norm of a loss gradient is related to its Lipschitz constant, the ratio \(\frac{\mathcal{L}_{\mathrm{sub}}}{\mathcal{L}_{\mathrm{gest}}}\) and \(\frac{\mathcal{L}_{\mathrm{trip}}}{\mathcal{L}_{\mathrm{gest}}}\) serves as a practical indicator of the relative scale of each objective. Adjusting the weights inversely to these ratios therefore acts as a form of gradient normalization, helping prevent any single objective from dominating the shared representation.

Moreover, exponential smoothing helps stabilize these estimates across mini-batches, reducing the impact of stochastic fluctuations and avoiding abrupt changes during training. Together, inverse scaling and temporal smoothing lead to a more stable and adaptive optimization process, making the model less sensitive to initialization and dataset-specific variations. From a representation learning perspective, this results in a better balance between discriminability (gesture classification), invariance (adversarial subject confusion), and structural organization (triplet loss), producing embeddings that generalize more effectively to unseen subjects. This balance is especially important in cross-subject EMG recognition, where the relative difficulty of these objectives can vary across individuals.

2.4 Optimization details↩︎

The network is trained using the Adam optimizer. For each mini-batch, the gesture, subject, triplet, and reconstruction losses are computed from the shared embedding. The encoder is then updated using the adversarially signed objective in 1 , while the subject head is updated in the normal direction using subject classification loss.

Gradient clipping is used to avoid exploding updates, and \(L_2\) regularization is applied to the trainable weights. During inference, only the encoder and gesture-classification head are retained; the subject-classification and reconstruction heads are used only during training.

The complete training procedure of the proposed framework is summarized in Algorithm 2. For each mini-batch, the encoder produces a latent representation that is simultaneously optimized for gesture discrimination, subject invariance, and metric structure. The gesture loss acts as the primary task objective, while the adversarial subject loss and triplet loss are adaptively reweighted during training using the proposed Lipschitz-inspired balancing mechanism. After an initial warm-up stage, the relative magnitudes of the subject and triplet losses are periodically estimated relative to the gesture loss, and their coefficients are updated via exponentially smoothed inverse scaling. This allows the network to maintain stable multi-objective optimization without relying on manually fixed loss weights. During inference, only the encoder and gesture head are used.

3 Results↩︎

3.1 Datasets↩︎

3.1.1 UCI EMG dataset↩︎

The UCI dataset³ comprises raw EMG recordings captured using a MYO Thalmic bracelet positioned on the forearm of 36 healthy subjects. During data collection, participants executed a series of static hand gestures. At the same time, eight evenly distributed sensors around the forearm recorded myographic signals at a sampling rate, yielding approximately 40,000-50,000 time-stamped measurements per recording session. Each participant completed two separate trials consisting of six to seven basic hand movements (here we use 6 gestures), with each gesture maintained for 3 seconds, followed by a 3-second rest period. The recorded gestures included resting hand position, fist clenching, wrist flexion and extension, radial and ulnar deviations, and an extended palm configuration (though not all subjects performed this). Windows of length 250ms with a 50ms step length are used for feature extraction. Four features of mean absolute value (MAV), root mean square (RMS), wave length (WL), zero crossing (ZC) are extracted.

3.1.2 NinaPro DB5 dataset↩︎

The Ninapro DB5 dataset⁴ includes surface EMG recordings from 10 intact subjects performing hand and wrist movements [26]. Data were acquired using two Thalmic Myo Armbands, each equipped with 8 dry electrodes equally spaced around the forearm, providing a total of 16 EMG channels sampled at 200 Hz. The first armband was positioned at the height of the radio-humeral joint, while the second was placed with a 22.5-degree clockwise rotation. For this study, we utilize only the EMG channels and focus on a subset of 10 movement classes: Rest, Finger Abduction, Fist, Finger Adduction, Middle Axis Supination, Middle Axis Pronation, Wrist Flexion, Wrist Extension, Radial Deviation, and Ulnar Deviation. These movements span basic finger articulations and fundamental wrist motions, representing functionally relevant gestures for prosthetic control and human-computer interaction applications. Each movement repetition lasted approximately 5 seconds, followed by a 3-second rest, for a total of 6 repetitions per gesture class. The movements were organized into three exercises: basic finger movements; isometric and isotonic hand configurations with wrist movements; and functional grasping movements. The dataset provides synchronized recordings, including raw EMG signals and movement labels with refined temporal annotations (restimulus) that more accurately reflect the actual movement execution. The same features as in the UCI dataset are extracted from standard windows of length 250ms with a 50ms step size for suitable real-time use [27].

3.2 Network architecture↩︎

The proposed CSIN network consists of a shared encoder followed by two specialized prediction heads for gesture classification and subject classification. The complete architecture is detailed in Table 2. The shared encoder processes z-score normalized 32/64-dimensional input features (8/16 electrodes \(\times\) 4 features: MAV, RMS, WL, ZC) through two fully connected blocks of 64 units each, with layer normalization, ReLU activation, and 30% dropout. The encoder outputs a 16-dimensional embedding that serves as the bottleneck representation for both downstream heads. The gesture classification head consists of three 16-unit fully connected layers with ReLU activations, layer normalization, and dropout, followed by a 6/10-way softmax output for the UCI/NinaPro dataset. The subject classification head mirrors this structure but uses 36-unit layers and outputs 35/10 classes (corresponding to training subjects in leave-one-subject-out cross-validation). During training, the subject head minimizes cross-entropy loss with normal gradients, while the shared encoder receives reversed gradients from this head to remove subject-identifying information. Training employs the Adam optimizer (\(\beta_1=0.9\), \(\beta_2=0.999\), \(\epsilon=10^{-8}\)) with a fixed learning rate of 0.001, \(L2\) regularization (\(10^{-4}\)), batch size of 512, and 300 epochs. The total loss combines gesture classification (\(\lambda_{gest}\)), adversarial subject loss (\(\lambda_adv\)), and triplet loss (\(\lambda_{trip}\), margin \(\alpha=0.5\)). Triplets are constructed with anchor and positive samples sharing the same gesture from different subjects, while negative samples represent different gestures, enforcing both inter-class separability and cross-subject consistency.

3.3 Ablation study on training schedules↩︎

In addition to the main Lipschitz-adaptive training strategy, we investigated whether the timing of adversarial interaction between the encoder and the subject-classification head affects final performance. In all cases, the network architecture, feature extraction procedure, and loss definitions remained unchanged; only the update schedule of the encoder and subject head differed.

We considered four training schedules:

3.3.0.1 Scenario 1 (fully coupled training)

All modules are trained jointly from the first epoch onward. The encoder is optimized with the full objective, while the subject head is trained continuously with a subject-classification loss.

3.3.0.2 Scenario 2 (warm-up without subject-head training)

During the initial stage, the subject head is frozen, and the encoder is optimized only with gesture and triplet losses. After the warm-up period, adversarial subject learning is activated, and full joint training begins.

3.3.0.3 Scenario 3 (delayed adversarial interaction)

The subject head is trained from the start, but the encoder does not receive adversarial subject gradients during the warm-up stage. After initialization, the full objective is used for encoder training.

3.3.0.4 Scenario 4 (early subject supervision with later freezing)

The subject head is trained during the warm-up stage to establish a discriminative subject signal. After initialization, the subject head is frozen, while the encoder continues to receive adversarial gradients from the fixed subject classifier.

4pt width=

Table ¿tbl:tab:scenario95summary? summarizes these schedules. The purpose of this ablation is to determine whether the final performance gains arise primarily from the proposed Lipschitz-adaptive balancing itself or from a particular adversarial training schedule. As shown in the experimental results, the differences among scenarios are relatively small, indicating that the proposed method is robust to the exact scheduling strategy. This observation suggests that the main benefit derives from the multi-objective, subject-invariant formulation together with adaptive loss balancing, rather than from a highly specialized training schedule.

width=

width=

width=

width=

3.4 Performance comparison↩︎

Tables ¿tbl:tab1? and ¿tbl:tab2? show results against published baselines on UCI EMG and NinaPro DB5, respectively, under leave-one-subject-out cross-validation (LOSOCV). On UCI EMG, the proposed full CSIN reaches 84.48% mean accuracy, clearing the EMGBench reference of 78.2% by a meaningful margin. The naïve CSIN (the proposed CSIN with constant Lipschitz) already lands at 80.88%, which itself surpasses all published comparators. The cost is a slightly wider spread (std 15.74% vs.%), driven by a few low-performing subjects where subject-invariant training does not fully close the gap. On NinaPro DB5, the picture is different. Both the naïve CSIN (60.77%) and full CSIN (61.44%) substantially outperform EMGBench (41.30%), and the key gain from the proposed method is not accuracy but variance: std drops from 6.06% to 1.23%, and the worst-case subject accuracy rises from 46.00% to 59.11%. For a system intended to generalize to unseen users, a nearly five-point improvement in the floor matters more than a fractional gain in the mean.

Tables ¿tbl:tab11? and ¿tbl:tab22? expand this picture across fixed-weight configurations and four training schedules. On UCI EMG (Table ¿tbl:tab11?), the baseline (gesture classification only, no adversarial or triplet objectives) sits at 81.75% (std 14.33%). Adding triplet loss alone—at any of the three tested weights—barely moves the needle, with means clustering within 0.4% of baseline regardless of scenario. Adversarial loss alone (with no triplet term) similarly provides negligible lift. The modest gains emerge when both auxiliary objectives are active together. The best fixed-weight result in Table ¿tbl:tab11? is 82.26% (std 13.99%, min 37.76%) at \(\lambda_{\mathrm{adv}}=0.01\) and \(\lambda_{\mathrm{trip}}=1\) under Scenario 3—a setup where the subject head is frozen after the warm-up. This pairing sits squarely in the middle of the search grid; notably, raising the triplet weight to 10 rarely helps and occasionally degrades the minimum accuracy, suggesting that aggressive metric pushing conflicts with the classification objective at these embedding scales.

Across all four scheduling scenarios the results are strikingly stable. For any fixed pair of \((\lambda_{\mathrm{adv}}, \lambda_{\mathrm{trip}})\), swapping between S0 (fully coupled from epoch one) and S3 (delayed adversarial with later head freezing) typically shifts the mean by under 0.3%. The practical takeaway is that the benefit does not hinge on a particular initialization trick; the multi-objective formulation is the operative ingredient, not the schedule.

On NinaPro DB5 (Table ¿tbl:tab22?), the baseline (gesture classification only, no adversarial or triplet objectives) is 60.85% with std 5.60%. Triplet loss alone leaves performance essentially unchanged. Once adversarial loss is introduced at \(\lambda_{\mathrm{adv}}=0.1\), mean accuracy rises to around 61.34–61.47% across several configurations, and the standard deviation consistently falls to roughly 5.1–5.3%, compared to 6.14% without adversarial training. The best individual result, 61.47% with std 5.12%, comes at \(\lambda_{\mathrm{adv}}=0.1\) and \(\lambda_{\mathrm{trip}}=0.01\) under Scenario 0. A lighter adversarial weight of 0.01 \((\lambda_{\mathrm{adv}}=0.01)\) narrows the std gain slightly but still consistently outperforms triplet-only configurations. One structural difference from UCI stands out: on DB5, Scenario 0 (fully coupled training) tends to match or beat the warm-up variants, whereas on UCI the differences were negligible in either direction. With only 10 subjects available for training, the subject head appears to benefit from updating throughout rather than being frozen mid-training—consistent with the intuition that a stronger, continuously updated adversary provides a more persistent invariance signal when number of subjects is less.

Taken together, the ablation results point to two clear conclusions. First, neither adversarial training nor triplet loss alone is sufficient; their combination is what drives consistent improvement. Second, the adaptive loss weighting proposed in CSIN removes the need to hand-tune this balance: the Lipschitz-inspired mechanism keeps both auxiliary objectives scaled relative to the classification loss throughout training, which matters especially for datasets where the relative difficulty of invariance and gesture discrimination differs across subjects.

3.5 Feature extraction progress↩︎

In this section, we visualized the extracted features under the regular training and the proposed method by t-distributed stochastic neighbor embedding (t-SNE). We trained the two variants with the same architecture on data from 30 subjects and tested them on 6 held-out subjects of the UCI EMG dataset. Fig. 3 shows the visualization result under different loss weights and scenarios. The original feature exhibits a highly scattered, unstructured distribution in the 2D latent space. Except for three gesture classes—wrist extension, radial deviation, and hand at rest—which exhibit relatively well-clustered structures in the original feature space, most other classes are highly scattered and poorly organized, indicating substantial overlap and limited discriminability. Among these, ulnar deviation is the most challenging class, exhibiting the greatest dispersion and the most confusion with other gestures. By comparing regular and adversarial training across all figures, it is evident that the proposed CSIN with fixed weights achieves greater class compactness and stronger inter-class discrimination, particularly for the most challenging gesture, ulnar deviation. Scenario 2 with \(\lambda_{\mathrm{trip}}=0.1\) achieves the clearest class separation compared to all other cases and Scenario 4 with \(\lambda_{\mathrm{trip}}=10\) provides superior separation among different gestures compared with the other scenarios cases.

4 Discussion↩︎

While our approach deliberately removes subject-specific information to improve cross-subject generalization, there are scenarios where retaining some subject characteristics could actually improve performance. In clinical applications involving patients with neuromuscular disorders or limb differences, subject-specific features may encode crucial information about residual muscle function, compensatory movement patterns, or atypical muscle activation sequences, all of which are essential for accurate gesture recognition. For instance, a stroke patient might consistently activate muscles in an unconventional order to perform a specific gesture, and removing this subject-specific pattern could degrade rather than improve recognition accuracy for that individual. Similarly, in prosthetic control applications where the system is intended for long-term use by a single individual, preserving subject-specific features during an initial calibration phase can lead to more intuitive and responsive control than a completely subject-invariant system. The trade-off becomes particularly relevant when we consider the difference between a universal model that works reasonably well for everyone versus a personalized model that works exceptionally well for one person. In other words, the proposed method could provide a universal baseline in which no after-tuning is applied, and personalization is used.

The fundamental challenge we address—removing subject-dependent variations while preserving task-relevant information—extends naturally to other neural recording modalities, though each presents unique considerations. EEG signals exhibit similar cross-subject variability, including differences in skull thickness, brain anatomy, and electrode placement relative to cortical regions, making our adversarial disentanglement approach potentially valuable for motor imagery classification or cognitive state decoding. ECoG and intracortical recordings offer higher signal quality but introduce subject variability due to electrode placement variations, differing tissue responses to implantation, and progressive changes in recording quality over time as the tissue-electrode interface evolves. The similar rules are applied to the other modalities such as KMG [28].

One finding that deserves more attention than the summary numbers suggest is the behavior of the minimum accuracy across subjects. On UCI EMG, even the best configurations leave certain subjects at 35–38% accuracy — roughly twice chance level for a 6-class problem, but far enough below the 81% group average to suggest these individuals are maybe outliers rather than just hard cases at the tail of a normal distribution. This is not a failure of the adversarial mechanism per se; it is a sign that, for some individuals, the four time-domain features used here (MAV, RMS, WL, ZC) may not capture sufficient of the relevant muscle-activation structure to support generalization in either direction. The adversarial loss cannot manufacture a discriminative signal that was never in the features to begin with. This points toward a practical ceiling on what representation-level disentanglement can achieve with a fixed, compact feature set. Learning the features jointly with the encoder, rather than treating them as a fixed preprocessing step, might expose the network to richer signals currently discarded by windowed statistics. The broader point is that invariance learning and feature engineering are not independent problems; in other words, asking the encoder to remove subject-specific information becomes significantly harder when that information is already baked into the input features themselves.

5 Conclusion↩︎

Cross-subject generalization in sEMG-based gesture recognition is hard precisely because the signals that identify a person and the signals that identify a gesture are not cleanly separable. This paper addresses that entanglement through a conservative multi-objective framework—CSIN—that jointly optimizes gesture classification, adversarial subject confusion, and triplet-based metric learning on a shared encoder, with a Lipschitz-inspired adaptive mechanism that keeps the auxiliary objectives scaled relative to the primary task throughout training.

The experimental results on UCI EMG and NinaPro DB5 support several concrete observations. On UCI EMG, the proposed method reaches 84.48% mean accuracy under LOSOCV, compared to 78.2% from the best published comparator. The ablation in Table ¿tbl:tab11? shows that triplet loss or adversarial loss in isolation provides negligible gain over the classification-only baseline; the improvement requires both objectives active together, with moderate weights. On NinaPro DB5, where only 10 subjects are available and the task involves 10 gestures, the more meaningful gain is in consistency rather than mean accuracy: the standard deviation across subjects drops from 6.06% to 1.23%, and the worst-case subject accuracy rises by over 13 points relative to published baselines. Table ¿tbl:tab22? further shows that the method is robust across training schedules—fully coupled training performs comparably to warm-up variants on UCI, while continuous adversarial updating is slightly preferable on DB5 where number of subjects is limited.

Despite these gains, persistent inter-subject variance remains, particularly on UCI EMG where standard deviations stay above 13% even in the best configurations. Some subjects are simply harder to generalize to, and the current adversarial mechanism does not fully resolve that. Whether this reflects irreducible entanglement between subject identity and gesture-specific activation patterns—or a limitation of the feature set and architecture—is an open question worth pursuing. Extending the framework to richer temporal representations, or incorporating explicit domain randomization during training, could be productive next steps. Applying similar disentanglement strategies to EEG or intracortical recordings, where analogous cross-subject variability challenges exist, is another natural direction for future work.

Code availability↩︎

All results and codes will be available at www.github.com/Hamed-Rafiei/CSIN.

Data availability↩︎

All datasets used in this paper are publicly available.

References↩︎