1 Introduction↩︎

EEG is noninvasive and has high temporal resolution, which enables the capture of affect related neural dynamics and is therefore regarded as an ideal signal for emotion decoding [1], [2]. Neuropsychological studies indicate that emotion processing exhibits regional selectivity across the cortex, with frontal systems showing particular sensitivity [3]. In practice, some electrodes contribute little to emotional representations and are more susceptible to ocular and myogenic artifacts [4], [5], which leads to pronounced spatial nonuniformity in full channel EEG. Using all channels without discrimination dilutes discriminative information and reduces recognition accuracy, and it also increases dimensional redundancy and computational cost. Identifying and emphasizing electrodes that are more informative for emotion decoding, while suppressing redundant and noisy sources, is therefore a key path to improving the quality and deployability of EEG-based emotional representations.

Prior work has explored emotion recognition with a small set of channels and found that using only a limited number of emotion-relevant electrodes as input does not markedly reduce accuracy [6], [7]. Other studies employ attention mechanisms [6], [8] or graph convolutions [9], [10] to assign dynamic weights across channels. However, many existing approaches either do not adequately account for differences in cortical responses across distinct emotion elicitation paradigms, or they rely on a single weighting configuration, which limits adaptability across tasks, paradigms, and settings. Given heterogeneous elicitation conditions and application constraints, supporting multiple weighting configurations that update in a data adaptive manner is both practically meaningful and methodologically valuable.

Beyond channel redundancy, EEG exhibits pronounced cross-subject heterogeneity, that is, substantial innate differences among individuals in anatomy, physiological state, and psychological responses. As a result, the EEG distributions produced by different individuals under the same elicitation conditions can differ markedly, and even the same subject may drift over time [11]. These distributional discrepancies make the shift between source and target subjects one of the primary causes of degraded cross subject recognition performance. Techniques such as feature alignment [12], subdomain adaptation [13], [14], and adversarial graph contrastive learning [15] have made progress in mitigating this issue. However, they often require many labels or highly accurate pseudo labels, and they seldom model intra EEG structure explicitly, for example, channel level differences, which leaves training sensitive to noise and to pseudo-label drift. To cope with label scarcity, these methods are often paired with semi-supervised [11], [15] and unsupervised learning strategies [13], [16], [17]. However, they typically rely on additional auxiliary components such as graph neural networks or attention mechanisms, or they lack tight integration with standard backbones, which complicates practical use and limits plug-and-play deployment.

Building on the discussion above, we can summarize that EEG-based emotion recognition faces two main challenges:

• Which EEG channels are most informative under different emotion elicitation conditions, and how can a model elevate electrodes that contribute to specific emotions while suppressing interference from redundant channels?

• How can cross-subject heterogeneity be mitigated, particularly in target settings with scarce labels, so that the learned representations remain reliable and generalizable?

To this end, we think that it is necessary to prioritize channel importance, and there is a pressing need for an end-to-end framework that, under label scarcity, simultaneously strengthens model generalization and performs domain alignment. Inspired by advances in mixture-of-experts (MoE) [18] and semi-supervised domain adaptation [19], we adopt multiple lightweight expert sub-networks that operate in parallel and select a subset of experts conditioned on the input and task, thereby instantiating multiple weighting configurations that naturally fit EEG channel prioritization. In addition, semi-supervised domain adaptation integrates supervised learning, unsupervised consistency regularization, and domain-alignment constraints, which directly addresses the cross-subject setting with limited labels.

Accordingly, we propose PRISM (PRioritized channel Importance with Semi-supervised doMain adaptation), a framework that, across diverse EEG emotion recognition tasks, assigns data-dependent soft weights to each channel and performs cross-subject, semi-supervised domain adaptation under limited labels. Specifically, PRISM first encodes spatiotemporal EEG features with a backbone network, then augments it with a lightweight expert ensemble that learns differentiable, adaptive per-channel weights to amplify reliable electrodes while suppressing distractors. In parallel, confidence-filtered pseudo labels on unlabeled target data support consistency regularization and domain alignment, which mitigates heterogeneity and improves generalization. The framework is model agnostic and compatible with mainstream time-series architectures, readily accommodating emotion recognition across different label densities.

The main contributions of this paper can be summarized as follows:

• We propose PRISM, which realizes channel prioritization via a lightweight expert ensemble, yielding learnable multi-weight configurations that adapt to diverse emotion-elicitation paradigms and task settings.

• Under label-scarce circumstances, we develop and validate a semi-supervised domain adaptation strategy tailored to EEG, significantly improving cross-subject robustness and label efficiency.

• On public benchmarks including DEAP, DREAMER, and SEED, PRISM consistently outperforms state-of-the-art methods under limited annotations, and it can be integrated in a plug-and-play manner into existing models to further enhance performance.

The remaining sections of this article are organized as follows. Section 2 reviews the related work on channel selection, mixture of experts and semi-supervised learning. Section 3 presents the pipeline of PRISM. Section 4 details the procedure of the conducted experiments and experimental results. A more in-depth discussion is provided in Section 5. Finally, the study is concluded in Section 6.

2 Related Work↩︎

2.1 Channel Selection↩︎

The brain engages distinct regions across cognitive activities [20]. Converging evidence indicates that the frontal and temporal lobes are associated with emotion [6], [8], [10], [21], with particularly strong effects in frontal regions [20], [22]. Negative and neutral emotions show greater activation in the prefrontal cortex, whereas positive emotions are more active in the left hemisphere [5]. Tao et al. [8] introduced an attention mechanism to adaptively allocate weights and observed higher weights for electrodes over the frontal, temporal, and parietal areas. Lin et al. [9] regulated the proportion of selected channels by leveraging attention distributions on a graph structure. Similarly, Yang et al. [10] employed a channel weighting network to estimate channel importance parameters. Selecting channels that contribute more to emotion recognition does not reduce accuracy and can improve model interpretability [6].

2.2 Mixture of Experts↩︎

MoE [18] instantiates multiple submodels and uses a gating network or router to dynamically select a small subset of experts for each input. It has been widely adopted in natural language processing, computer vision, and time series prediction. For example, Switch Transformers [23] and GShard [24] maintain massive parameter counts while controlling compute, thereby improving efficiency. V-MoE [25] routes capacity preferentially to target regions and downweights background. MMVAE [26] combines MoE to fuse latent representations from different modalities. Methods such as Pathformer [27], Time-MoE [28], InterpGN [29], and SoftShape [30] assign different experts to different scales, which improves model stability and interpretability.

2.3 Semi-supervised Learning↩︎

Semi-supervised learning requires only a small number of labels while achieving strong target-domain generalization. Early work MixMatch [31] combines label guessing, entropy minimization, consistency regularization, and MixUp [32] to form an efficient semi-supervised framework. FixMatch [33] uses high-confidence pseudo labels together with a constraint that enforces consistency between weak and strong augmentations, leading to strong performance. AdaMatch [19] provides a unified training framework that covers semi-supervised learning, unsupervised domain adaptation, and semi-supervised domain adaptation. FlexMatch [34] and FreeMatch [35] adopt more flexible threshold selection strategies to adapt across classes. SoftMatch [36] replaces hard thresholds with Gaussian weighting. AllMatch [37] fully exploits unlabeled data through class-adaptive thresholds and class-consistency constraints. Similarly, FullMatch [38] integrates FixMatch and FlexMatch and can also maximize the use of all unlabeled data.

3 Method↩︎

In this section, we will introduce PRISM, which is composed of two modules: (i) a prioritized channel-importance module, and (ii) a semi-supervised domain-adaptation module. As illustrated in Fig. 1, the prioritized channel-importance module is implemented in three stages, namely Seasonality Mining (SM), Channelwise State Space (CSS), and Expert Router (ER). Fig. 2 depicts the semi-supervised domain-adaptation module tailored for cross-subject EEG emotion recognition, which integrates weak and strong augmentations, confidence-thresholded pseudo labeling, consistency regularization, entropy minimization, and a feature distribution alignment term for domain adaptation.

3.1 Prioritized Channel Importance↩︎

3.1.1 Seasonality Mining↩︎

Seasonal or scale-specific temporal cues are informative for sequence modeling [39], [40]. As shown in Fig. 1, we extract multi-scale temporal representations from an EEG segment \(x\in\mathbb{R}^{L\times D}\) (length \(L\), channels \(D\)) in three steps: frequency-guided scale selection, blockwise multi-scale perception, and weighted fusion.

3.1.1.1 Frequency-guided scale selection.

Let \(\mathcal{F}\) denote the fast Fourier transform and \(\mathcal{A}\) the amplitude operator. We compute the spectrum \(A=\mathcal{A}(\mathcal{F}(x))\), select the top-\(K\) prominent frequencies \(\{f_i\}_{i=1}^{K}=\mathrm{TopK}(A)\), and convert them to periods \(p_i = \Big\lfloor \frac{L}{f_i}\Big\rfloor.\) Nonnegative scale weights are obtained by a softmax over spectral amplitudes: \[w_i \;=\; \frac{\exp(\mathcal{A}(f_i))}{\sum_{j=1}^{K}\exp(\mathcal{A}(f_j))}, \quad i=1,\dots,K . \label{eq:seasonality-weights}\tag{1}\]

3.1.1.2 Blockwise multi-scale perception (MSP)

For each period \(p_i\), we pad \(x\) to a length divisible by \(p_i\), then reshape the sequence into 2-D blocks through a period-wise rearrangement operator \(\mathcal{R}_{p_i}\): \[X^{(i)}_{2\mathrm{D}} \;=\; \mathcal{R}_{p_i}\!\big(\mathrm{Pad}_{p_i}(x)\big) \;\in\; \mathbb{R}^{p_i \times q_i \times D},\] where \(q_i\) is the number of blocks after padding. (Note: the subscripts “1D/2D” indicate the number of temporal axes only, and the channel axis \(D\) is always present in the tensor shape but omitted in the subscript for brevity.) On these blocks, we apply a multi-scale perception (MSP) operator with kernel set \(\{K_m\}_{m=1}^{M}\), \[\widetilde{X}^{(i)}_{2\mathrm{D}} \;=\; \sum_{m=1}^{M} \mathrm{Conv}_{K_m}\!\big(X^{(i)}_{2\mathrm{D}}\big),\] and fold the result back to a one-dimensional time–channel layout: \[x^{(i)}_{1\mathrm{D}} \;=\; \mathcal{R}^{-1}_{p_i}\!\big(\widetilde{X}^{(i)}_{2\mathrm{D}}\big) \;\in\; \mathbb{R}^{L\times D}.\]

3.1.1.3 Multi-scale fusion.

Finally, we fuse the per-scale representations using the weights in 1 : \[x_{\mathrm{ms}} \;=\; \sum_{i=1}^{K} w_i\, x^{(i)}_{1\mathrm{D}} \;\in\; \mathbb{R}^{L\times D}. \label{eq:xms}\tag{2}\] The tensor \(x_{\mathrm{ms}}\) serves as the input to the subsequent Channelwise State Space stage.

3.1.2 Channelwise State Space↩︎

EEG channels recorded at the same time step may correspond to different neural events. Some channels can be at a peak while others are at a trough. Mapping signals from different channels at the same time into a single token risks mixing heterogeneous events [7]. Moreover, a single time step rarely captures a complete event [41]. Motivated by these considerations, we adopt an inverted embedding scheme: instead of forming tokens by concatenating channels at the same time step (the conventional choice), we form tokens by concatenating the temporal trajectory of a single channel. This preserves channel structure and strengthens long-range temporal modeling. Formally, let \(x_{\mathrm{ms}}\in\mathbb{R}^{L\times D}\) be the output of Seasonality Mining. We exchange the time and channel axes using a permutation operator \(\mathrm{SwapAxes}_{L, D}\) (it swaps axis \(L\) with axis \(D\)): \[\widehat{x} \;=\; \mathrm{SwapAxes}_{L,D}\!\big(x_{\mathrm{ms}}\big) \;\in\; \mathbb{R}^{D\times L}.\]

A Mamba [42] block \(m_{\theta}(\cdot)\) is then applied in this channel-token space to capture spatiotemporal interactions, and the result is mapped back to the time–channel layout: \[\tilde{h} \;=\; \mathrm{SwapAxes}_{D,L}\!\big(m_{\theta}(\widehat{x})\big) \;\in\; \mathbb{R}^{L\times D}.\] We treat \(m_{\theta}\) as an encoder here. Mamba views a one dimensional sequence as a process driven by a continuous time dynamical system. Compared with the quadratic complexity of attention, Mamba performs training and inference with nearly linear complexity, which makes it suitable for EEG signals that span multiple temporal scales. Concretely, an input \(x(t)\in\mathbb{R}\) evolves through a hidden state \(h(t)\in\mathbb{R}^{d}\) and produces an output \(y(t)\in\mathbb{R}\). The evolution is controlled by three parameter matrices \(A\in\mathbb{R}^{d\times d}\), \(B\in\mathbb{R}^{d\times 1}\), and \(C\in\mathbb{R}^{1\times d}\), namely \[\frac{d}{dt}h(t)=A\,h(t)+B\,x(t), \quad y(t)=C\,h(t).\]

Real world time series are discrete. Mamba therefore adopts a zero-order hold discretization via time scale parameter \(\Delta\) and obtains the discrete parameters and the new recursion: \[\overline{A}=\exp(\Delta A),\quad \overline{B}=(\Delta A)^{-1}\big(\exp(\Delta A)-I\big)\,\Delta B,\] \[h_{t}=\overline{A}\,h_{t-1}+\overline{B}\,x_{t}, \quad y_{t}=C\,h_{t}.\]

To enable parallelization, the entire mapping can be written as a single structured convolution. The convolution kernel and the output are: \[\widehat{K} = \big( C\,\overline{B}, \;C\,\overline{A}\,\overline{B}, \;\ldots, \;C\,\overline{A}^{\,L-1}\overline{B} \big), \quad y=x * \widehat{K},\] where \(\widehat{K}\) is a structured convolution kernel generated from \(A\), \(B\), and \(C\). This formulation enables efficient parallel convolution for long sequences while preserving the capacity to capture long range temporal dependencies.

3.1.3 Expert Router↩︎

After Seasonality Mining and Channelwise State Space, we obtain an EEG representation that captures long-range temporal dependencies and fine-grained spatiotemporal interactions. We then introduce an expert router to prioritize channel importance. As shown in the dashed box (middle-top) of Fig. 1, the \(i\)-th expert consists of a channel-weight vector \(c_i\!\in\!\mathbb{R}^{D}\) and a channel mapping network \(\phi_i:\mathbb{R}^{D}\!\to\!\mathbb{R}^{D}\) implemented by a two-layer MLP. For any time index \(t\), \[\label{eq:8} u_i(t) = \tilde{h}(t)\odot c_i, \qquad E_i(t) = \phi_i\!\big(u_i(t)\big).\tag{3}\] Stacking over time yields \(E_i(\tilde{h})\in\mathbb{R}^{L\times D}\), where each expert learns a specific channel-weighting composition. In parallel, we summarize the temporal dimension by a mean operator to obtain a time-averaged descriptor \(\mu=\tfrac{1}{L}\sum_{t=1}^{L}\tilde{h}(t)\in\mathbb{R}^{D}\) and compute noise-free expert logits \(\ell=W_{\text{gate}}\mu\in\mathbb{R}^{E}\). During training, Gaussian noise is injected to stabilize routing and to prevent the model from collapsing onto a single expert: \[\sigma=\mathrm{softplus}\!\big(W_{\text{noise}}\mu\big)+\varepsilon_{0}, \tilde{\ell}=\ell+\epsilon, \epsilon\sim\mathcal{N}\!\big(0,\mathrm{diag}(\sigma^{2})\big),\]

where \(\varepsilon_{0}\) is a constant and \(\epsilon\) is Gaussian noise. At inference time we use the noise-free logits \(\ell\) for stable predictions. We then select the top-\(k\) experts \(S=\mathrm{TopK}(\tilde{\ell},k)\) and normalize on the selected indices: \[s_{S}=\mathrm{softmax}(\tilde{\ell}_{S}), \quad s_{j}=0 \;\; (j\notin S).\]

The final routed representation is a weighted mixture of expert outputs: \[y \;=\; \sum_{i=1}^{E} s_i\, E_i(\tilde{h}) \;\in\; \mathbb{R}^{L\times D}.\] A downstream classification head takes \(y\) to produce predictions. Using multiple experts enables a diverse set of channel-weight combinations rather than a single fixed pattern. \(\{c_i\}\) realize channel-wise soft prioritization, while \(s\) provides sample-adaptive expert mixing. The router is fully data-driven, and the expert parameters and channel weights are learned end-to-end jointly with the rest of the network.

3.2 Semi-supervised domain adaptation for EEG↩︎

In this subsection, we propose the semi-supervised domain adaptation used for EEG emotion recognition. The overall pipeline is shown in Fig. 2. To enhance the effective capacity of the model while remaining label-efficient, we generate two views of each target sample with weak and strong augmentations that are tailored to EEG. Let \(a_w\) and \(a_s\) denote the weak and strong augmentations, respectively. They are defined as: \[\begin{align} a_w(x) &= x + \epsilon, \epsilon \sim \mathcal{N}(0, \sigma_w^2), \\ a_s(x) &= x + \epsilon' + \delta_{\mathrm{drop}} + \delta_{\mathrm{jitter}} . \end{align}\] \(\epsilon\) and \(\epsilon'\) are both Gaussian noise. \(\delta_{\mathrm{drop}}\) is a channel-wise random zero mask and \(\delta_{\mathrm{jitter}}\) is a perturbation along the temporal axis. The weak view preserves the main structure of the original signal, whereas the strong view combines multiple perturbations to improve robustness. For labeled samples \((x^\ell, y^\ell)\), we minimize \(\mathcal{L}_{\mathrm{sup}} = \mathrm{CE}\!\big(z(x^\ell),\, y^\ell\big)\), where \(z(\cdot)\) denotes the network logits. For an unlabeled target sample \(x^u\), we compute the weak-view logits and probabilities as follows: \[z_w = z\!\big(a_w(x^u)\big), \quad p_w = \mathrm{softmax}(z_w).\] Obtaining the hard pseudo label \(\hat{y}=\arg\max p_w\), and build a confidence mask \(m = \mathbf{1}\{\max p_w \ge \tau\}\). Only high-confidence samples contribute to the consistency objective. With the strong-view logits \(z_s = z\!\big(a_s(x^u)\big)\), the loss is: \[\mathcal{L}_{\mathrm{cons}} = \frac{1}{\|m\|_{1}} \sum m \cdot \mathrm{CE}\!\big(z_s,\, \hat{y}\big).\]

To encourage confident predictions on the weak view, we minimize: \[\mathcal{L}_{\mathrm{ent}} = \frac{1}{C}\sum_{c=1}^{C} \big[-\, p_w^{(c)} \log p_w^{(c)} \big].\]

Due to the source and target batches not being identically distributed in the cross-subject setting, which degrades generalization [11]. We align the mean predictive distributions of the two domains: \[\bar{p}_s = \mathrm{mean}\!\big(\mathrm{softmax}(z(x^s))\big), \bar{p}_t = \mathrm{mean}\!\big(\mathrm{softmax}(z(a_w(x^u)))\big),\] \[\mathcal{L}_{\mathrm{dom}} = \big\| \bar{p}_s - \bar{p}_t \big\|_2^{2}.\] The mean operator is taken over the minibatch. Finally, the total loss combines all terms with nonnegative weights \(\lambda_{\mathrm{cons}}\), \(\lambda_{\mathrm{ent}}\) and \(\lambda_{\mathrm{dom}}\) as follows: \[\mathcal{L} = \mathcal{L}_{\mathrm{sup}} + \lambda_{\mathrm{cons}} \mathcal{L}_{\mathrm{cons}} + \lambda_{\mathrm{ent}} \mathcal{L}_{\mathrm{ent}} + \lambda_{\mathrm{dom}} \mathcal{L}_{\mathrm{dom}} .\]

4 Experiment and Results Analysis↩︎

4.1 Datasets and preprocessing↩︎

We systematically evaluate PRISM on three public EEG emotion datasets, DEAP [43], DREAMER [44], and SEED [45]. DEAP contains 32 participants who watched music videos to induce emotions, and 32-channel EEG was recorded for each participant. DREAMER provides 14-channel EEG from 23 participants. Both DEAP and DREAMER include emotion annotations along the valence and arousal dimensions. SEED contains recordings from 15 participants collected in three sessions with 62 channels, and it provides three discrete emotion categories (positive, neutral, and negative). For preprocessing, DEAP and DREAMER were downsampled to 128 Hz and filtered with a 4–45 Hz bandpass. SEED was downsampled to 200 Hz and filtered to 0–75 Hz. All datasets were segmented into 1 s nonoverlapping windows, and z-score standardization was applied per channel.

4.2 Baselines and evaluation↩︎

We compare against six advanced time series models that are widely used as baselines from Time-Series-Library ⁴: iTransformer [41], DLinear [46], TimesNet [39], NTransformer [47], Informer [48], and TCN [49]. To assess performance and generalization, we adopt three protocols, inter-subject, cross-subject and subject-dependent. In the inter-subject setting, we pool data from all subjects, shuffle, and split it into training and test sets with a \(3{:}1\) ratio. In the cross-subject setting for semi-supervised adaptation, we construct a disjoint target-domain subset comprising \(30\%\), \(20\%\), and \(10\%\) of participants on DEAP, DREAMER, and SEED, respectively. For each target subject, only 30% samples are annotated. For subject-dependent case, model outputs prediction individually and 25% of samples for each subject are used for testing. SEED contains three sessions per participant. We therefore report inter-session results and also evaluate each session independently. Classification accuracy is used as the primary metric since our sample distribution for each class is balanced.

4.3 Implementation details↩︎

We implement PRISM in PyTorch and run all experiments on two NVIDIA RTX 4090 GPUs. For DEAP and DREAMER, the length of a single sample is \(128\), and for SEED, it is \(200\). We use Adam optimizer with an initial learning rate of \(1\times 10^{-4}\), a batch size of \(32\), and train for \(10\) epochs. In seasonality mining, we retain the two scales with the highest spectral amplitudes \((K=2)\). The expert router instantiates up to eight experts and selects the top four for each sample \((E=8,\;k=4)\). Loss weights are set to \(\lambda_{\mathrm{cons}}=1\), \(\lambda_{\mathrm{ent}}=0.1\), and \(\lambda_{\mathrm{dom}}=0.1\). The confidence threshold is \(\tau=0.95\).

7pt

4.4 Experimental results↩︎

4.4.1 Inter-subject results↩︎

As shown in Table ¿tbl:tab:inter95subject?, PRISM achieves the best and most stable performance across all datasets and settings. On DEAP, PRISM surpasses TCN by 3.79% on valence and TimesNet by 3.69% on arousal. On DREAMER, the margins over the second best are 8.57% for valence and 6.50% for arousal. The gains are largest on SEED. Under the inter setting PRISM exceeds TCN by 22.97%, and across sessions S0, S1, and S2 the margins are 5.47%, 5.20%, and 11.82%, respectively. Compared with DEAP and DREAMER, SEED has more channels and multiple recording sessions, which yields stronger channel redundancy and cross-session variation. PRISM benefits most in this regime because it highlights stable electrodes while suppressing noisy or redundant ones. Although TCN is stronger than most baselines on SEED, it still struggles with the large channel count and session variability. DLinear is relatively strong on DREAMER, indicating that trend and seasonal components can fit a reasonable decision boundary. PRISM nevertheless improves on this baseline through multi-expert channel weighting and multi-scale temporal modeling, providing additional discriminative power.

7pt

4.4.2 Cross-subject results↩︎

Table ¿tbl:tab:cross95subject? reports the cross-subject results. Since individual variability and domain shift, all baselines drop notably compared with the inter-subject setting, whereas PRISM remains clearly ahead on every dataset and evaluation dimension. On DEAP, PRISM reaches 87.28% on valence and 87.45% on arousal, exceeding the second best by 10.02% and 8.33%, respectively. On DREAMER, PRISM attains 84.69% on valence and 92.62% on arousal, which are higher than DLinear at 69.79% and TimesNet at 86.77% by 14.90% and 5.85%. On SEED, the margins over the second best exceed 20% in all sessions (Inter, S0, S1 and S2). We attribute the consistent advantage to three complementary factors. First, channel prioritization suppresses weak or noisy electrodes and highlights stable, emotion-relevant spatial signals. Second, inverted embedding combined with a state-space backbone captures longer-range, multi-scale spatiotemporal structure, which stabilizes representations under large across-subject variation. Third, the semi-supervised adaptation module uses a confidence threshold of 95% for pseudo labels, entropy minimization on weak views, and source–target alignment, thereby reducing pseudo-label noise and mitigating domain shift. Although TCN remains the strongest baseline on many settings, indicating the value of local temporal inductive bias, PRISM consistently surpasses it, especially on DEAP valence and across all SEED protocols. TimesNet leads among baselines on DREAMER arousal, suggesting stronger periodic or multi-scale components in this dimension, yet PRISM still achieves the best overall results.

7pt

4.4.3 Subject-dependent results↩︎

We also perform experiments under the subject-dependent setting, where a separate model is trained and evaluated for each participant. The results are shown in Table ¿tbl:tab:intra95subject?. First, on DEAP and DREAMER the overall performance is already near a ceiling, with most methods in the range of 96% - 98%. The performance gaps are therefore compressed, which suggests that within a single subject the emotion related temporal patterns are relatively consistent and the task behaves like a standard sequence classification problem, where complex cross domain alignment is not the key factor. Models that rely on attention or multi-scale convolutions, such as Informer and DLinear, tend to be slightly ahead, while PRISM is comparable but not dominant on these datasets, which is consistent with the fact that PRISM is not designed specifically for the subject-dependent scenarios. Second, PRISM shows the clearest advantage on SEED. Whether we use the inter split or the three independent sessions, PRISM achieves the highest accuracy. This aligns with the characteristics of SEED, which has many channels and larger variation across sessions. The results indicate that even within a subject, PRISM brings stable gains across sessions by reducing redundancy and noise. Finally, DLinear remains strong on DREAMER, which implies that trend and seasonal components can model within subject emotion signals well. Overall, the subject-dependent setting emphasizes the precise modeling of a single subject’s stable patterns, while PRISM provides the most value when channel dimensionality is high and session variability is large.

4.4.4 Comparison with state-of-the-art methods↩︎

Table 1 presents a comparative analysis of PRISM against state-of-the-art methods, categorized into channel selection and domain adaptation (DA) techniques. The top section compares methods based on channel selection, distinguishing between static selection (manually pre-determined channels) and dynamic selection (weights automatically assigned by model). The bottom section benchmarks various domain adaptation paradigms, including semi-supervised, unsupervised, graph-based and other approaches. The results clearly indicate that methods employing dynamic channel selection generally outperform static selection approaches, underscoring the necessity of data-dependent channel prioritization to mitigate redundancy. Crucially, PRISM achieves the best overall performance on the DEAP and SEED datasets. Although it is slightly outperformed by ARCNN on DREAMER, the margin is negligible (only 1.03 points). In the challenging subject-independent setting, PRISM’s combination of prioritized channel importance and semi-supervised domain adaptation consistently yields both superior accuracy compared to unsupervised or graph-based DA methods. Furthermore, even in the subject-dependent setting, PRISM maintains performance that is either comparable to or better than the existing optimal methods. These results highlight the robust and synergistic efficacy of PRISM’s two core modules in addressing the fundamental challenges of inter-subject variability and channel redundancy in EEG emotion recognition.

4.5 Ablation studies↩︎

4.5.1 Inter-subject↩︎

7pt

As shown in Table ¿tbl:tab:ablation95inter?, we report the impact of removing each module under the inter-subject setting. The three modules play distinct roles and also reinforce one another, and CSS is the most critical component. Removing CSS drops SEED-inter from 97.62% to 63.89%, and all three sessions also decline markedly. This indicates that without explicit modeling of spatiotemporal structure, redundancy and noise are amplified. ER delivers steady gains. When ER is removed, the PRISM will degenerate into miMamba [7] using static channel selection. Without ER, SEED-inter remains at 91.96% but is clearly lower than the full model, and S1 and S2 decrease to 95.09% and 91.08%. This shows that soft routing is an effective unified mechanism across datasets for suppressing channel redundancy. SM improves generalization overall, especially on SEED where the number of channels is large and cross-session variation is strong. Removing SM reduces SEED inter from 97.62% to 88.88%. There is a small reversal on DEAP valence, where 90.48% slightly exceeds 90.35%. This likely occurs when samples are short, channels are fewer, or the periodic structure is weak, in which case explicit multi-scale seasonal modeling brings limited benefit and may overlap with other submodules. More importantly, removing two modules at the same time leads to structural collapse. Removing CSS and SM yields 66.88% on SEED inter, which suggests that the model is left without multi-scale temporal cues and without channel-state constraints, and thus relies almost only on a lightweight expert ensemble and cannot resist cross-subject shift. The degradation is most severe when ER and CSS are both removed, which confirms that the combination of soft channel selection and channelwise temporal modeling is the core defense against channel redundancy.

4.5.2 Cross-subject↩︎

7pt

Table ¿tbl:tab:ablation95cross? reports the ablation studies under the cross-subject setting. The full model remains the top performance on DREAMER and SEED. Compared with the inter-subject results in Table ¿tbl:tab:ablation95inter?, removing CSS causes a huge drop, indicating that CSS plays the key role in spatiotemporal feature extraction. Removing ER produces a consistent but moderate degradation, and the effect is more visible on SEED where the channel count and variability are higher. The effect of removing SM is data dependent. On DEAP it can match or slightly exceed the full model, whereas on DREAMER and SEED it generally degrades performance. Eliminating two modules leads to substantial deterioration, especially combinations that exclude CSS. This pattern mirrors Table ¿tbl:tab:ablation95inter? but is amplified in the cross-subject regime, highlighting the complementarity of the three modules and the indispensable role of CSS. Overall, PRISM reaches its best performance through the synergy of the three modules, and weakening any two breaks this complementarity and causes a pronounced drop in accuracy.

7pt

4.5.3 Subject-dependent↩︎

Table ¿tbl:tab:ablation95intra? reports the ablation results under the subject-dependent setting. On SEED, the full PRISM consistently achieves the best performance, and removing any submodule leads to clear degradation. The drop is largest when both ER and CSS are removed, indicating that the combination of channelwise state modeling and soft routing is critical in regimes with many channels and strong cross-session variation. When removing CSS or ER alone, the changes on DEAP and DREAMER are small, whereas SEED still degrades, which suggests that explicit spatiotemporal modeling and channel routing are less beneficial for easier within-subject cases but become indispensable when channel redundancy and cross-session variability are stronger. SM mainly contributes to stability and refinement. Removing SM alone causes milder declines than removing CSS, but removing both SM and CSS produces a huge drop, showing that multi-scale temporal cues and channelwise state modeling are complementary. Overall, on datasets with many channels or large cross-session differences, the synergy among the three modules is irreplaceable.

6pt

4.5.4 Effect of top-k channel filtering↩︎

PRISM supports two implementations of the channel selection strategy. The first applies the weighting in Eq. 3 to all channels and aggregates them by a weighted sum. The second selects the top-\(k\) channels by the coefficients \(c_i\) in Eq. 3 implementation (Results in Table ¿tbl:tab:inter95subject?, ¿tbl:tab:cross95subject?, ¿tbl:tab:intra95subject?, ¿tbl:tab:ablation95inter?, ¿tbl:tab:ablation95cross? and ¿tbl:tab:ablation95intra? are implemented by this way). To make the comparison explicit, Table ¿tbl:tab:topk95channels? reports results under a fixed \(k=4\) for the two settings with and without top-\(k\) channel filtering. On DEAP and DREAMER the change after enabling top-\(k\) is small and slightly negative. On SEED the gains are pronounced, most notably on the inter setting and consistently across the three sessions. These results indicate that top-\(k\) channel filtering is more effective in regimes with many channels and stronger cross-session variation, where it suppresses redundant or session-specific noise and emphasizes stable electrodes. In datasets with fewer channels, hard filtering may discard weak yet useful signals. Consequently, the advantage of the channel selection strategy becomes more salient as the channel dimensionality increases.

6pt

4.5.5 Sensitivity to the number of experts and top-\(k\) channels.↩︎

Table ¿tbl:tab:experts95topk? reports how the number of experts and the choice of top-\(k\) channels affect accuracy. When the number of experts is fixed to 8, choosing a very small \(k\) weakens the selectivity of the router, whereas a very large \(k\) introduces redundancy. Aggregating results on DEAP, DREAMER, and SEED, \(k=4\) is the most stable choice. The benefit is most evident on SEED, where the channel count is high and the across session variation is strong. When \(k\) is fixed to 4, using too few experts limits the expressive power of routing, while too many experts can lead to unstable training. A smaller \(k\) paired with a medium-sized expert set strikes a better balance between computation and accuracy. Overall, setting the default to \(k=4\) and using 6 to 8 experts yields robust and efficient performance.

2pt

4.5.6 Effect of test rate and label ratio↩︎

To assess how the test rate and the amount of target labels affect PRISM under the cross-subject setting, we evaluate a grid of configurations without enabling top-\(k\) channel filtering. As show in Table ¿tbl:tab:label95ratio?, the results reveal three patterns. First, increasing the label ratio almost always yields gains. With a fixed test rate, raising the labeled ratio from 0.1 to 0.3 leads to improvements on both dimensions of DEAP and DREAMER, and SEED shows concurrent gains for the inter split and all three sessions when the test rate is 0.1 or 0.2. This indicates that more target supervision amplifies the benefits of PRISM. Second, enlarging the test rate is generally unfavorable, most evidently on SEED. At a fixed label ratio, moving the test rate from 0.1 to 0.3 causes systematic drops on S0 and S2, and the inter split also falls when labeled ratio is 0.3, whereas S1 degrades less and can even rise at higher label ratios. This suggests heterogeneous sensitivity of sessions to changes in sample size. Third, robustness differs across datasets. DREAMER on the arousal dimension maintains high performance across settings and increases steadily with more labels. DEAP shows no obvious degradation when the test rate grows and continues to improve with a higher label ratio. Overall, a smaller test rate combined with a larger label ratio is the most reliable regime. When the test rate is large, especially SEED-S0 and SEED-S2, it becomes more sensitive to the specific allocation of data and labels.

5 Discussion↩︎

5.1 Channel importance analysis↩︎

The core strength of PRISM’s Prioritized Channel Importance Module lies in its capacity to dynamically identify and weight the most relevant EEG channels in a data-driven manner, tailored to different datasets and emotional dimensions. To validate this, we visualized the channel importance learned by the model across five experimental settings (DEAP-V/A, DREAMER-V/A, SEED), as presented in Fig. 3.

We observed that channels deemed important for valence classification primarily concentrated in the frontal and prefrontal cortices, exhibiting slight hemispheric lateralization. Specifically, DEAP-V was dominated by the left frontal region (Fp1/F3/F7), whereas DREAMER-V showed greater reliance on the bilateral prefrontal area with a slight rightward tendency (AF4/F4). This pattern is consistent with the classical findings linking frontal asymmetry to emotional valence [3].

In contrast, the channel importance for arousal conditions significantly shifted toward the posterior and right hemispheric regions. High weights in DEAP-A and DREAMER-A concentrated in the occipitotemporal, posterior parietal, and central regions (e.g., O2, T8, CP6, Cz). This spatial distribution aligns with the cognitive control resources accompanying high arousal states [55].

By comparison, the learned weights for the SEED dataset were more distributed, primarily engaging the temporal and parietal regions (e.g., CPz/CP6/P4/F4/P2). We attribute this shift to several factors: the SEED three-classification task (Positive, Negative, Neutral) may dilute the clear frontal asymmetry cues present in pure binary opposition. Furthermore, given SEED’s use of longer video stimuli, enhanced visual processing and emotion-vision interaction are prominent. The known involvement of the right posterior temporal lobe in processing complex visual and emotional cues[56] collectively drives the importance away from the prefrontal cortex toward the parietal and posterior-temporal regions.

5.2 Analysis of frequency-guided scale selection↩︎

Our multi–scale seasonality mining block selects the top–\(k\) frequencies with the highest amplitudes and converts their periods into different scales. A natural concern is that if the selected frequencies concentrate within a narrow band, the resulting different scales may become similar and undermine the goal of multi–scale analysis. To this end, we address this concern from qualitative and quantitative aspects.

5.2.0.1 Qualitative analysis

EEG signals typically exhibit activity across multiple frequency bands, including delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–100 Hz). In emotion-related or other cognitively demanding tasks, activity usually appears in more than one band. For example, during an awake state, alpha may dominate under relaxation, whereas theta, beta, and gamma often become pronounced when the cognitive or emotional load increases. This multi–band behavior makes it likely that the top–\(k\) frequencies fall into different bands and thus yield diverse scales. As an illustration, at a sampling rate of 128 Hz, selecting frequencies between 8 Hz and 32 Hz produces scales that range approximately from 4 to 16, which is consistent with the intended multi–scale design.

5.2.0.2 Quantitative evidence on DEAP

We further quantify the likelihood of frequency concentration using the DEAP dataset. The dataset contains \(N=2{,}457{,}600\) windowed samples. For each sample we identify the top–\(k\) frequencies by amplitude in the frequency domain and set \(k=2\). Let \(f_{1i}\) and \(f_{2i}\) denote the two dominant frequencies for the \(i\)-th sample. We compute two statistics: \[D \;=\; \frac{1}{N}\sum_{i=1}^{N} \lvert f_{1i}-f_{2i}\rvert ,\] which measures the average absolute distance between the two dominant frequencies, and \[R \;=\; \frac{1}{N}\sum_{i=1}^{N} \mathbb{I}\!\left( \lvert f_{1i}-f_{2i}\rvert \le 1 \right),\] which is the proportion of samples whose two dominant frequencies are within 1 Hz, indicating potentially similar scales. On DEAP the empirical results are \(D=17.70\) Hz and \(R=6.05\%\). The average distance indicates a substantial spread between the two dominant frequencies, and the small proportion \(R\) shows that only a small fraction of samples present near identical frequencies. These findings suggest that, in practice, frequency concentration within a narrow band is uncommon for EEG signals and that the risk of degenerate scales is small.

5.2.0.3 Additional safeguard through MSP

Even in rare cases where the selected top–\(k\) frequencies are close and thus yield similar scales, MSP module preserves multi–scale feature extraction. MSP applies a bank of convolutions with different kernel sizes to the feature maps produced by the temporal block. This design captures local as well as global patterns through diverse receptive fields, maintaining the multi–scale characterization regardless of the exact patch sizes.

Although the concentration of top–\(k\) frequencies in a narrow band is a theoretical possibility, qualitative properties of EEG and quantitative evidence on DEAP indicate that this scenario is uncommon. Moreover, the MSP module offers an additional safeguard by enforcing multi–scale receptive fields at the convolutional stage. Overall, these observations support the robustness of our scale selection strategy and the effectiveness of the overall multi–scale design.

6 Conclusion↩︎

In this work, we presents a novel framework called PRISM that integrates channel prioritization with semi-supervised domain adaptation. On the modeling side, PRISM emphasizes stable and emotion-relevant electrodes through three coordinated stages, Seasonality Mining, Channelwise State Space, and Expert Router, which capture multi-scale temporal structure, channel dependencies, and channel importance. Under label-scarce target domains, PRISM mitigates cross-subject shift using high-confidence pseudo labels, consistency regularization, and distribution alignment. These components address the dual bottlenecks of channel redundancy and cross-subject distribution shift in EEG emotion recognition. Experiments on DEAP, DREAMER, and SEED across diverse settings demonstrate superior performance. Overall, PRISM shows that jointly modeling channel importance and domain shift is an effective route to improved generalization in EEG emotion recognition, and it offers a plug-and-play solution for label-limited cross-subject applications.

References↩︎

---

1. Xin Zhou, Xiang Zhang and Lijun Yin are with the School of Computing, T. J Watson College of Engineering and Applied Science, Binghamton University - State University of New York, Binghamton, NY 13902 USA (e-mail: xzhou11@binghamton.edu; zxiang4@binghamton.edu; lyin@binghamton.edu).↩︎

2. Hao Deng is with the Massachusetts General Hospital, Harvard University, Boston, MA 02114 USA (e-mail: hdeng1@mgh.harvard.edu).↩︎

3. \(^*\)Corresponding author.↩︎

4. Baseline implementations are taken from the public repository at https://github.com/thuml/Time-Series-Library.↩︎