Ahmad et al.

1 Introduction↩︎

Hyperspectral images (HSIs) capture detailed spectral signatures across hundreds of contiguous and narrow wavelength bands, offering rich information that enables accurate classification and identification of diverse materials and land cover types [1]. This fine spectral resolution has established HSIs as a vital tool across numerous domains, including remote sensing [2], environmental monitoring [3], food quality and safety inspection [4]–[6], forensic analysis [7], and biomedical imaging [8], [9]. However, despite these widespread applications, achieving robust HSI classification remains a formidable challenge. The high dimensionality of spectral data, the limited availability of labeled samples, and significant spectral-spatial variability due to varying acquisition conditions collectively hinder generalization performance [10], [11]. These issues become particularly pronounced in cross-domain classification tasks, where domain shifts between datasets severely limit model transferability.

In recent years, transformer-based architectures have garnered increasing interest for HSI classification [12]–[14], owing to their powerful self-attention mechanisms that capture long-range spectral and spatial dependencies. This capability leads to richer and more discriminative feature representations [15], [16]. Spatial-Spectral Transformers (SSTs), in particular, have emerged as a promising alternative to traditional convolutional neural networks (CNNs). SSTs can model intricate spectral-spatial interactions by operating on image patches, and inherently scale well to high-resolution HSI data [17], [18]. Unlike CNNs, SSTs eliminate the need for handcrafted feature extractors or complex pooling strategies by learning hierarchical representations directly from raw pixels [19]. Furthermore, their interpretability via attention maps enhances trust in model decisions by highlighting salient regions that drive classification outcomes [20], [21].

Building on this foundation, several SST-based models have been proposed better to harness the spectral and spatial properties of HSIs. For instance, Shi et al. [22] introduced a dual-branch transformer architecture that separately captures spectral and spatial dependencies, allowing for more effective multi-scale contextual learning. Nevertheless, deploying SSTs for large-scale HSI tasks presents several challenges. First, the self-attention mechanism incurs quadratic computational complexity with respect to sequence length, posing scalability concerns for longer inputs [23]. Second, unlike CNNs, SSTs lack built-in translation invariance, which may affect the consistency of spatial feature extraction [24]. Third, tokenizing HSIs into fixed-size patches can result in the loss of fine-grained information, particularly in small or irregular structures. Moreover, SSTs typically rely on large amounts of labeled data to achieve strong generalization, making them vulnerable to overfitting under low-label regimes [25]. Given the high cost and effort involved in acquiring labeled HSI samples [26], [27], this reliance on fully supervised learning is often impractical. Consequently, strategies that leverage both labeled and unlabeled data, such as semi-supervised and active learning (AL), have become increasingly important for reducing annotation demands while maintaining classification accuracy [28], [29].

AL, in particular, has gained significant traction in the HSI community as a means of selectively labeling the most informative samples [30], [31]. A number of effective approaches have been introduced to address this challenge [32]. Liu et al. [33] proposed MDL4OW, a multitask framework that combines classification and reconstruction to detect unknown classes via reconstruction errors. Liao et al. [34] introduced CGE-AL, a graph-embedding-based AL method that optimizes sample selection using graph CNNs and uncertainty estimation. Zhao et al. [35] designed MAT-ASSAL, which integrates a multi-attention transformer with adaptive superpixel segmentation to preserve spatial structures during query selection. Similarly, Wang et al. [36] presented a co-auxiliary learning strategy, coupling pseudo-labeling with diverse sample selection to ensure both uncertainty and representativeness. Liu et al. [37] further advanced the field by introducing an adversarial domain alignment framework that incorporates contrastive learning, enhancing robustness and cross-domain generalization under few-shot conditions.

Despite these promising directions, deep AL methods still face key limitations when applied to HSIs. The inherent scarcity of labeled data constrains the diversity of queried samples, often resulting in suboptimal generalization [38]. Additionally, complex spectral-spatial correlations present in HSIs are not always effectively captured by conventional AL techniques [39]. To address these gaps, the research community has started to explore active transfer learning (ATL) frameworks that combine the strengths of AL and transfer learning. In such frameworks, knowledge learned from well-labeled source domains can be transferred to improve sample selection and feature extraction in sparsely labeled target domains. This approach is especially valuable in dynamic remote sensing scenarios, where spectral characteristics can vary substantially over time and location. By enabling domain adaptation with minimal retraining, ATL offers a viable pathway to enhance HSI classification with limited annotation overhead.

Motivated by these insights, this work introduces an SST-based ATL framework (SST-ATL) specifically designed for cross-domain HSI classification. The proposed SST-ATL framework directly tackles critical issues such as limited label availability, spectral-spatial heterogeneity, and domain shifts, providing a comprehensive and adaptive solution. The key contributions of this work are summarized below:

We propose SST-ATL, a unified ATL framework for HSI classification that integrates an SST with a multistage fine-tuning strategy and a dynamic layer freezing mechanism. This design allows for efficient cross-domain adaptation under annotation scarcity by selectively updating model layers based on domain shift characteristics, thereby enhancing generalization while reducing computational overhead. To further improve robustness against spectral variability, a novel self-calibrated attention refinement module is introduced, which dynamically adjusts spatial and spectral attention weights using uncertainty-guided feedback. In addition, we develop a hybrid query strategy that jointly optimizes informativeness, diversity, and spectral coverage through a combined uncertainty-diversity sampling criterion and a spectral diversity-enforcing selection scheme. Extensive evaluations across six standard HSI benchmarks demonstrate that SST-ATL consistently outperforms CNN-, Transformer-, and state-space model-based baselines by +2.5% to +3.8% OA under constrained annotation budgets. The complete implementation is publicly available at https://github.com/mahmad000/ATL-SST.

2 Related Works↩︎

Recent advancements in HSI classification have explored several paradigms, including transformer-based models, AL, and their integration into ATL. We categorize and review the literature under these themes. Table 1 summarizes each method’s key contributions, limitations, and the improvements offered by our proposed SST-ATL framework.

2.0.0.1 Transformer-Based HSI Classification

Transformer-based models, particularly SSTs, have shown exceptional capacity in modeling long-range spectral-spatial dependencies. S2FTNet [40] introduces a dual-branch SST using separate spectral and spatial attention modules, achieving 3–5% gains over CNNs. Zhang et al. [41] propose ELS2T, a resource-efficient SST variant with reduced embedding dimensions. Zhang et al. [42] further design a multi-range SST that attends to spectral-spatial intervals at multiple scales. Ahmad et al. [43] tackle spectral redundancy using a wavelet-based compression (WaveFormer), which reduces FLOPs while preserving spectral detail. Despite these innovations, most transformer-based methods require extensive labeled data and struggle with domain shifts. These limitations motivate the integration of adaptive learning strategies such as AL and ATL to improve generalization under low-label regimes.

2.0.0.2 Active Learning Strategies for HSI

AL has been leveraged to reduce annotation costs by prioritizing informative sample selection. Zhao et al. [35] use adaptive superpixel segmentation with a multi-attention transformer for sample querying. Liao et al. [34] propose class-wise graph-based uncertainty sampling via graph embeddings, though with high computational costs. Wang et al. [36] combine pseudo-labeling with co-auxiliary learning to improve sample representativeness. Di et al. [44] propose ALSN, an adversarially trained Siamese network employing dual uncertainty criteria (AUAL, ICUAL), showing strong label efficiency but with increased model complexity. Cao et al. [49] integrate Markov Random Field smoothing with CNN-based AL to improve spatial coherence, while Ahmad et al. [47] adopt multiple sample selection criteria, including fuzziness, mutual information, and breaking ties in a 3D CNN setting. These methods, while effective, often rely on heuristic sampling or are computationally intensive.

2.0.0.3 Active Transfer Learning for HSI

The fusion of AL and transfer learning forms ATL frameworks, aimed at learning transferable representations under limited labels. Deng et al. [45] were among the first to propose ATL for HSI using stacked sparse autoencoders. Lin et al. [46] extend this by introducing an AL-driven transfer learning pipeline that does not require prior knowledge of the target domain. Yang et al. [48] provide a conceptual overview of ATL for HSI, but lack architectural detail. He et al. [50] focus on heterogeneous ATL from RGB to HSI using pretrained CNNs, but struggle with spectral misalignment. Our previous work [47] shows that ATL with 3D CNNs can achieve strong results, but at the cost of computational complexity and limited domain adaptation flexibility.

In contrast, our SST-ATL framework offers a principled fusion of SSTs, AL, and ATL by incorporating dynamic layer freezing, uncertainty-calibrated attention, and hybrid uncertainty-diversity querying, providing a robust and computationally efficient solution to cross-domain HSI classification.

3 Proposed Methodology↩︎

Given an HSI cube \(\mathbf{X} \in \mathbb{R}^{M \times N \times k}\), where \(M\) and \(N\) are the spatial dimensions and \(k\) is the number of spectral bands, we first extract overlapping spatial windows of size \(W \times W\) from \(\mathbf{X}\). These windows provide localized spatial-spectral context for feature encoding via a transformer-based model. Each window is further divided into smaller subpatches used for embedding.

3.1 Spectral-Spectral Transformer (SST)↩︎

The SST backbone comprises four stages: patch embedding, positional encoding, self-attention-based encoder blocks, and the final classification head.

3.1.0.1 Patch Embedding

Each spatial window of size \(W \times W \times k\) is processed by a 3D convolutional layer to extract \(p \times p\) subpatches. Let \(\mathbf{W}_e\) be the learnable convolutional kernel bank. The \(i\)-th embedded subpatch \(\mathbf{P}_i\) is computed as:

\[\mathbf{P}_i(u,v,c) = \sum_{m=0}^{p-1} \sum_{n=0}^{p-1} \sum_{q=0}^{k-1} \mathbf{X}(s_u+m, s_v+n, q) \cdot \mathbf{W}_e(m,n,q,c)\] where \((s_u, s_v)\) are the coordinates of the \(i\)-th subpatch, \(p\) is the subpatch size, \(c\) indexes output channels, and \(N_p = \left(\frac{W}{p}\right)^2\) is the number of subpatches in a window. These subpatches are projected into a \(d\)-dimensional feature space.

3.1.0.2 Positional Encoding

To inject spatial information into the embeddings, a sinusoidal positional encoding \(\mathbf{E}_{\text{pos}}\) is added:

\[\begin{align} \mathbf{E}_{\text{pos}}(i, 2j) &= \sin\left(\frac{i}{10000^{2j/d}}\right) \\ \mathbf{E}_{\text{pos}}(i, 2j+1) &= \cos\left(\frac{i}{10000^{2j/d}}\right) \end{align}\]

The encoded patch sequence is \(\mathbf{Z} = \mathbf{P} + \mathbf{E}_{\text{pos}}\).

3.1.0.3 Transformer Encoder Block

Each block applies multi-head self-attention followed by feed-forward processing:

\[\begin{align} \text{MultiHead}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) &= \text{Concat}(\text{head}_1, \dots, \text{head}_h)\mathbf{W}_o \\ \text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) &= \text{Softmax}\left(\frac{\mathbf{QK}^T}{\sqrt{d_k}}\right)\mathbf{V} \end{align}\] where \(\mathbf{Q}_i = \mathbf{Z}_i \mathbf{W}_Q\), \(\mathbf{K}_i = \mathbf{Z}_i \mathbf{W}_K\), \(\mathbf{V}_i = \mathbf{Z}_i \mathbf{W}_V\), and \(d_k\) is the attention key dimension. The output of one encoder block is:

\[\begin{align} \mathbf{Z}' &= \text{LayerNorm}(\mathbf{Z} + \text{Dropout}(\text{Attention}(\mathbf{Z}))) \\ \mathbf{Z}'' &= \text{LayerNorm}(\mathbf{Z}' + \text{Dropout}(\mathbf{F}(\mathbf{Z}'))) \end{align}\] where \(\mathbf{F}\) is a feedforward network:

\[\mathbf{F}(\mathbf{Z}_i) = \text{ReLU}(\mathbf{Z}_i \mathbf{W}_1 + b_1)\mathbf{W}_2 + b_2\]

To improve robustness against domain-specific spectral distortions, we introduce uncertainty-aware attention scaling. Let \(\mathcal{U}_i\) be the entropy-based uncertainty for the \(i\)-th token. The calibrated attention is given by:

\[\mathbf{A}_{\text{cal}} = \text{Softmax}\left(\frac{\mathbf{QK}^T}{\sqrt{d_k}}\right) \cdot (1 + \lambda \cdot \mathcal{U})\] where \(\lambda\) is a hyperparameter controlling uncertainty influence. This guides the model to focus more on spatial-spectral tokens with higher ambiguity, leading to better adaptation. To inject global context, a learnable token \(\mathbf{Q}_c \in \mathbb{R}^d\) attends to the encoded patches:

\[\mathbf{A}_{\text{cross}} = \text{Softmax}\left(\frac{\mathbf{Q}_c \mathbf{K}_c^T}{\sqrt{d}}\right) \mathbf{V}_c\]

3.1.0.4 Classification Head

The aggregated features pass through a two-layer MLP with softmax:

\[\begin{align} \mathbf{O} &= \text{ReLU}(\mathbf{Z}'' \mathbf{W}_3 + b_3) \\ \mathbf{O}_{\text{final}} &= \text{Softmax}(\mathbf{O} \mathbf{W}_4 + b_4) \end{align}\] where \(\mathbf{O}_{\text{final}} \in \mathbb{R}^C\) and \(C\) is the number of classes.

3.2 Active Learning↩︎

3.2.0.1 Hybrid Query Strategy

We define \(X_{\text{pool}}\) as the set of samples and \(\text{query\_size}\) as the number of samples to be queried in each AL round. Our strategy combines:

• Uncertainty Sampling: \[\mathcal{U}(\mathbf{x}_i) = -\max(p(y|\mathbf{x}_i))\]

• Diversity Sampling: \[\mathcal{D}(\mathbf{x}_i) = \mathop{\mathrm{arg\,max}}_{S \subseteq X_{\text{pool}}, |S| = \text{query\_size}} \text{Diversity}(S)\] where \[\text{Diversity}(S) = \frac{1}{m(m-1)} \sum_{j=1}^{m} \sum_{\substack{k=1\\k \ne j}}^{m} d_{jk}\] and \(d_{jk}\) is the Euclidean distance between pixel spectra in local neighborhoods of size \(n_{\text{neighborhood}}^2\).

3.2.0.2 Query Index Selection

For each candidate \(\mathbf{x}_{(h,w)}\), we compute:

\[\text{diversity\_metric}(h,w) = \frac{1}{m(m-1)} \sum_{j \ne k} d_{jk}\]

The top-\(\text{query\_size}\) entries with the highest diversity are selected: \[\text{query\_indices} = \left\{ \pi(k) \mid k \in [HW - \text{query\_size} + 1, HW] \right\}\] where \[\pi = \text{argsort}(d)\]

3.3 Dynamic Layer Freezing↩︎

To facilitate efficient domain adaptation, we introduce a dynamic layer freezing policy. During transfer to a new target domain, SST layers are selectively frozen or unfrozen based on sensitivity to domain shift. We estimate shift magnitude using the maximum mean discrepancy (MMD) of intermediate features. Layers with low variance across domains are frozen, reducing computational cost while retaining useful knowledge.

3.4 Model Training and Evaluation↩︎

The model is trained using categorical cross-entropy:

\[\mathcal{L} = -\sum_{i=1}^{N} \sum_{c=1}^{C} y_{ic} \log(\hat{y}_{ic})\]

Evaluation metrics include OA, AA, and the kappa coefficient:

\[\kappa = \frac{p_o - p_e}{1 - p_e}\] where \(p_o\) is the observed agreement and \(p_e\) is expected agreement by chance.

4 Experimental Settings and Datasets↩︎

The experimental configuration for SST-ATL is designed to ensure clarity, reproducibility, and robust performance evaluation. Following the preprocessing of the HSI cubes and corresponding ground truths, patch-based datasets are generated. Each HSI is divided into overlapping spatial windows of size \(8 \times 8\) (i.e., input patch size), which serve as local contextual inputs for the SST backbone. The data are partitioned into training, pool, and test sets with a split ratio of 1%, 49%, and 50%, respectively. Specifically, 1% of the samples are randomly selected for initial training, 49% are allocated to the pool for AL queries, and 50% are reserved exclusively for testing. For diversity-based query selection, a neighborhood size of \(n_{\text{neighborhood}} = 3\) is used, resulting in \(9\) spectral vectors per neighborhood to compute diversity scores, and the query percentage is set to 0.02%.

Model training is conducted under a set of carefully tuned hyperparameters. The learning rate is initialized at 0.001 with a decay factor of \(1 \times 10^{-6}\), and the Adam optimizer is employed to facilitate convergence. Training proceeds for 50 epochs with a batch size of 56. The backbone architecture comprises four transformer layers, each featuring eight attention heads. The model dimensionality (\(d_{\text{model}}\)) is set to 54, and the feedforward network dimensionality is configured as \(4 \times 64 = 256\). A dropout rate of 0.1 is applied to mitigate overfitting, while layer normalization is stabilized with an epsilon value of \(1 \times 10^{-6}\).

The experimental evaluation in this study was conducted using six widely recognized hyperspectral datasets: Qingyun (QUH), WHU-Hi-HongHu (HH), WHU-Hi-HanChuan (HC), Salinas (SA), University of Houston (UH), and Pavia University (PU). Each dataset encompasses a variety of land cover classes with distinct characteristics, enabling a comprehensive assessment across diverse scenes. For HH and HC, a rich set of agricultural and urban classes was included, while SA predominantly featured vegetation-related categories. UH captured a complex urban environment, and PU primarily focused on urban materials and structures. The QUH dataset significantly expanded the scale and diversity of the evaluation, offering a substantial number of samples across natural and man-made categories.

5 ATL-SST Effects on the same dataset↩︎

The results presented in Table 2 highlight the consistent improvement in classification accuracy as the number of training samples increases. Table 2 also provides computational metrics, showcasing the model’s efficiency. Training times remain stable at around 44s across iterations, despite the increase in sample size, indicating scalability. The testing time is consistently around 3.4s, demonstrating the method’s suitability for real-time applications. The reported FLOPs (110,336) and parameter count (836,559) reflect the model’s balance between computational efficiency and capacity, making it practical. Qualitative improvements are visually evident in Figure 2, which presents ground truth maps for varying training sample sizes. With fewer training samples (e.g., 75 and 223), the maps exhibit substantial noise and misclassification. As the training sample size increases, the maps become progressively refined, reflecting improved spatial and spectral consistency.

The results presented in Table 3 and 3 collectively highlight the model’s performance progression on the PU dataset as training samples increase. Training and testing times remain relatively stable across sample sizes, indicating computational efficiency; a total of 108,800 FLOPs and 835,017 parameters suggests a well-optimized model that balances accuracy and computational overhead. The accompanying Figure further substantiates these findings by plotting OA, AA, and \(\kappa\) scores against sample counts, illustrating a clear upward trend with convergence near 99% as sample sizes approach 2245.

Table 4 and 4 thoroughly analyze the model’s performance, computational complexity, and the scalability of its classification accuracy on the SA dataset. Although training times gradually increase as sample sizes grow, testing times remain relatively stable, underscoring the model’s scalability and efficiency. Furthermore, Table 4 provides computational metrics, including FLOPs and parameter counts, which quantify the model’s complexity at approximately 110,592 FLOPs and 836,816 parameters. These values confirm the model’s high computational efficiency, which is an essential consideration for large-scale applications. Finally, Figure 4 provides ground truth maps generated with different training sample sizes, visually showcasing classification performance improvements.

Table 5 provides a comprehensive overview of the HC dataset’s classification performance and computational cost. The computational metrics reveal a gradual increase in training time with more samples, reflecting the added processing requirements, while testing times remain relatively stable. Furthermore, the reported FLOPs and parameter counts provide insight into the model’s computational efficiency, balancing accuracy with resource usage. Figure 5 showcases the ground truth maps for each training sample size, visually representing the model’s learning progress. With 1,287 samples, the classification appears fragmented, reflecting limited learning. As the sample size increases, the maps exhibit progressively finer details, and class boundaries become more accurate and well-defined.

Table 6 provides computational metrics; despite increasing data, the training and testing times remain relatively stable, showcasing the model’s computational efficiency. For example, training times range from 272.05s to 291.68s, and testing times fluctuate minimally between 22.42s and 23.42s. For instance, OA increased from 87.83% to 98.97%, while AA rose from 71.14% to 97.94%, reflecting the model’s robust performance. The \(\kappa\) followed a similar upward trajectory, indicating reduced misclassification rates and alignment with ground truth. Figure 6 presents ground truth maps corresponding to the training sample counts. These visualizations show the gradual enrichment of spatial details and class boundaries as training data increases. The progression from 1933 to 20,322 samples demonstrates the model’s growing ability to capture intricate spatial and spectral variations.

Table 7 reports training and testing times, which grow modestly with larger sample counts, and provides the model’s computational metrics, such as FLOPs and parameters, indicating efficient scaling. Figure 7 illustrates ground truth maps for the dataset, corresponding to each training sample size. These maps visually demonstrate the enhancement in spatial coverage and label consistency as the number of training samples increases, corroborating the quantitative improvements observed in Table 7.

6 ATL-SST Effects on Cross Datasets↩︎

This section explores the impact of ATL in enabling effective cross-dataset classification. By leveraging the actively trained SST on one dataset and fine-tuning it with a limited fraction of samples from other datasets, we assess its ability to generalize and adapt to varying spectral-spatial distributions. The results highlight the significance of ATL in achieving high accuracy with minimal fine-tuning across diverse datasets.

Table 8 presents the OA, AA, and \(\kappa\) for cross-dataset classification, where the ATL-SST is initially trained on the SA dataset and subsequently fine-tuned using 10% of the samples from other datasets (PU, UH, HC, HH, and QUH). The results indicate that the model achieves high accuracy across datasets, with \(\kappa\) values consistently above 93%, demonstrating effective transfer learning. Testing times vary significantly across datasets due to differences in data volume and complexity, with QUH requiring the longest computation times. Figure 8 provides visual representations of classification outcomes, showcasing the trained SST on SA and its fine-tuned performance on PU, HC, HH, and UH datasets. Each subfigure illustrates distinct patterns, emphasizing the model’s adaptability to different data distributions.

Table 9 and Figure 9 illustrate the performance of the SST trained on the PU dataset and subsequently fine-tuned and evaluated on other datasets, including SA, UH, HC, and HH. Table 9 provides detailed metrics, highlighting the transferability and generalization ability of the SST model. The actively trained SST on PU achieves a \(\kappa\) of 98.51% and an OA of 98.88% on its test data, demonstrating robust performance on the source dataset. When fine-tuned with only 10% samples from other datasets, the model maintains competitive accuracy, with OA values exceeding 96% for most target datasets. Testing times are also reported, indicating computational efficiency during model fine-tuning and evaluation. Figure 9 visually complements these results by showcasing the SST model’s classification maps on the source dataset (PU) and various target datasets (SA, HC, HH, and UH).

Table 10 and Figure 10 present the performance of the SST actively trained on the UH dataset and fine-tuned with 10% samples from other target datasets, including SA, PU, HC, and HH. Table 10 outlines key metrics, highlighting the SST model’s capability to generalize and adapt to new datasets with minimal fine-tuning. The actively trained SST on UH achieves high baseline performance, with a \(\kappa\) of 97.38% and an OA of 97.58% on its test data. When fine-tuned with samples from other datasets, the model demonstrates strong transferability, achieving an OA of 98.47% on SA and 97.75% on PU. Although the performance is slightly reduced for HC and HH, the OA values remain above 94%, underscoring the model’s robustness. Testing times scale with dataset complexity, with SA and PU requiring less computation compared to HC and HH, as seen in the recorded durations. Figure 10 complements the table by visualizing classification results. The maps indicate that the fine-tuned SST effectively captures spatial and spectral features across diverse datasets. Specifically, the SST retains consistent classification quality on target datasets despite using only 10% of the samples for fine-tuning.

7 Comparison with State-of-the-art Methods↩︎

The following Tables and Figures illustrate a comparative evaluation of various HSI classification methods on the UH, PU, and SA datasets. The methods compared include Attention Graph CNN (AGCN) [51], Wavelet-Based Spatial-Spectral Transformer (WaveFormer) [43], Hybrid Spatial-Spectral Transformer (HybViT) [52], Pyramid-based Spatial-Spectral Transformer (PyFormer) [12], Spatial-Spectral Transformer (SST) [53], Spatial-Spectral Mamba (SSMamba) [54], and Wavelet-Based Spatial-Spectral Mamba (WaveMamba) [55]. These state-of-the-art approaches are benchmarked against the proposed method, ATL-SST, under consistent training, validation, and testing configurations, as summarized in the Tables.

As shown in Table 11, the results highlight the superior performance of ATL-SST across most experimental settings. For OA, ATL-SST consistently achieves the highest values, with a maximum of 100% in multiple scenarios, underscoring its robustness and effectiveness. Similarly, ATL-SST outperforms all comparative methods in terms of \(\kappa\) and AA, achieving top scores of 97.38% and 96.97%, respectively. This reflects the method’s ability to effectively classify diverse classes within the hyperspectral dataset. Notably, ATL-SST demonstrates significant improvement over AGCN and WaveMamba, with AGCN yielding lower performance metrics (e.g., 91.35% OA and 90.64% \(\kappa\)) in many cases. While HybViT and SST exhibit strong results, particularly in individual settings, their performance is generally less consistent compared to ATL-SST. The training time analysis also reveals ATL-SST’s computational efficiency, with an average training time comparable to SST and WaveFormer, and significantly lower than methods like WaveMamba and PyFormer. The ground truth maps presented in Figure 11 visually depict the classification results for the UH dataset. It is evident from the maps that ATL-SST provides more precise and coherent classification results, with fewer misclassified pixels and better-defined class boundaries compared to the comparative methods. For instance, methods like AGCN and WaveMamba exhibit noticeable misclassifications and noise, whereas ATL-SST maintains high spatial and spectral accuracy.

The results presented in Table 12 and Figure 12 comprehensively compare the performance. In terms of accuracy, ATL-SST consistently outperforms all other methods, achieving the highest OA of 98.8778%, AA of 98.1747%, and \(\kappa\) of 98.5130%. These results are a significant improvement over the next best-performing methods, such as WaveMamba, which achieves an OA of 96.1941%. From a computational perspective, ATL-SST maintains competitive efficiency, with a training time of 124.78s, comparable to HybViT and SST while significantly outperforming computationally intensive models like WaveMamba and WaveFormer, which require 1382.70s and 112.68s, respectively. Qualitative results presented in Figure 12 further validate the quantitative findings. The classification maps generated by ATL-SST are visually superior, exhibiting precise boundary delineation and accurate region classifications that closely match the ground truth. In contrast, competing methods, such as AGCN and SSMamba, show noticeable errors, particularly in regions with complex class boundaries or high spectral variability.

The comparison of HSI classification methods on the SA dataset, as detailed in Table 13 and illustrated in Figure 13, highlights the superior performance of the proposed ATL-SST model. Quantitatively, ATL-SST demonstrates outstanding accuracy, achieving the highest OA of 99.4864%, AA of 99.7641%, and \(\kappa\) of 99.4281%. These metrics significantly surpass the next best-performing models, such as HybViT and SST, which attain OAs of 97.4912% and 97.5318%, respectively. From a computational perspective, ATL-SST achieves a balanced trade-off between accuracy and training efficiency. Its training time of 157.85s is competitive with models like HybViT and SST, while being significantly faster than WaveMamba and PyFormer, which require 1734.50s and 1469.81s, respectively. Qualitative results in Figure 13 further validate ATL-SST’s efficacy. The classification maps produced by ATL-SST exhibit superior boundary delineation and region specificity, closely mirroring the ground truth. In comparison, models such as SSMamba and WaveMamba show noticeable misclassifications in regions with complex spectral patterns, while transformer-based models like HybViT and PyFormer, although performing well overall, display minor boundary smoothing.

8 Ablation Study↩︎

To evaluate the individual contributions of each module within the SST-ATL framework, we conduct a comprehensive ablation study on three representative datasets: PU, UH, and SA. The ablation analysis is divided into two categories: component-level evaluation and query strategy assessment.

8.0.0.1 Component-Level Ablation

First, to measure the effect of AL, we replace the hybrid uncertainty-diversity sampling strategy with random sampling. Second, to isolate the impact of diversity sampling, we retain only uncertainty-based selection. Third, to test the influence of selective model adaptation, we disable dynamic freezing and allow all transformer layers to be uniformly fine-tuned. Fourth, we examine the value of the self-calibrated attention mechanism by reverting to standard MHSA without uncertainty-guided refinement.

The results presented in Table 14 indicate that each component significantly contributes to the final performance of the model. The full SST-ATL achieves the highest OA, AA, and \(\kappa\) across all datasets. The removal of AL causes the most severe degradation, OA drops by 2.66%, 2.65%, and 2.70% for PU, UH, and SA respectively, highlighting the effectiveness of the proposed hybrid sampling. Excluding diversity sampling also leads to performance drops, particularly in the AA metric, reflecting reduced class coverage. The absence of dynamic freezing results in slightly reduced accuracy but increased training instability, while the removal of self-calibrated attention degrades spatial-spectral feature alignment, as evidenced by reduced \(\kappa\) values.

8.0.0.2 Query Strategy Evaluation

In addition to component analysis, we evaluate the effectiveness of our hybrid uncertainty-diversity sampling strategy against four widely used alternatives: random sampling, entropy-based uncertainty sampling, margin-based uncertainty sampling, and diversity-only sampling. All strategies are tested across four AL rounds (250 to 1000 labeled samples) on the PU, UH, and SA datasets.

As shown in Table 15, our proposed hybrid query strategy outperforms all others in every setting. The hybrid method achieves up to 98.14% OA on PU, 97.29% on UH, and 98.97% on SA with only 1000 labeled samples. The advantage is particularly evident in the early rounds, where the model learns from as few as 250 samples. For example, at 250 labels, the hybrid method exceeds random sampling by over 3% OA on all datasets, showcasing its superior label efficiency and informativeness. These results emphasize that incorporating both uncertainty and spectral-spatial diversity into the query process leads to faster convergence, better generalization, and significantly lower annotation costs than existing AL approaches.

9 Conclusions and Future Research Directions↩︎

This work proposes a novel multi-stage ATL framework that integrates the SST for efficient HSI classification. The proposed framework leverages the strengths of transfer learning, AL, and SST to address key challenges in HSI classification, such as high spectral dimensionality and limited labeled data. A major contribution of this work is developing an uncertainty-diversity querying mechanism that adaptively selects the most informative and diverse samples for iterative model refinement. This not only optimizes labeling efficiency but also improves the model’s ability to generalize across varying spectral profiles. Additionally, this work introduced a dynamic freezing strategy to selectively freeze and unfreeze SST layers during the transfer learning process, ensuring an optimal balance between computational efficiency and adaptability to new spectral variations. This mechanism significantly reduces computational overhead while preserving critical learned representations, making the approach scalable for resource-constrained environments.

While this study makes substantial progress in advancing HSI classification through ATL and SST, several avenues for future exploration remain: for instance, future extensions could focus on self-supervised learning approaches to reduce reliance on labeled data. This would be particularly beneficial in HSI classification, where obtaining labeled samples is costly. Leveraging self-supervised pretext tasks or pseudo-labeling strategies could further enhance the model’s learning capabilities in scenarios with limited labeled data. Although the current uncertainty-diversity-based querying has proven effective, dynamic querying strategies could be explored. For example, combining reinforcement learning with AL may allow the system to learn optimal sampling strategies over time, potentially improving both sample selection and overall learning efficiency. As HSI data can vary significantly between different sensors or geographical regions, exploring domain adaptation techniques within the ATL framework would be a valuable direction. This would allow models to adapt better to unseen datasets or shifts in data distribution, improving the robustness of the classification process in real-world scenarios.

References↩︎