1 Introduction↩︎

Hyperspectral images (HSIs) are usually represented as three-dimensional data cubes. They contain rich spatial structures and continuous spectral responses across dozens to hundreds of narrow bands. Compared with conventional RGB imagery, HSIs provide dense and contiguous spectral measurements. These measurements capture detailed material-specific signatures and make HSIs particularly suitable for accurate land-cover classification of spectrally similar materials [1]–[3].

HSI classification aims to assign a semantic label to each pixel according to its spectral-spatial characteristics. Early studies mainly relied on conventional machine learning methods, such as Support Vector Machine (SVM) [4] and Random Forest (RF) [5]. These methods can be effective when training samples are limited. However, their performance is often constrained by handcrafted features, shallow representations, and limited capability in modeling complex spectral-spatial dependencies. The high dimensionality and strong band correlation of HSIs further make it difficult for these methods to fully exploit discriminative information from both domains.

With the development of deep learning, LSTM [6], CNN [7], and Transformer-based models [8] have been widely introduced into HSI classification. These models learn spectral-spatial representations and have achieved substantial improvements on benchmark datasets. However, their strong representation capability usually depends on sufficient labeled samples. In practical HSI applications, high-quality annotations are expensive and time-consuming to obtain. Training samples are therefore persistently scarce [9]. This limitation has motivated studies on cross-domain few-shot learning (FSL), where knowledge learned from label-rich source domains is transferred to label-scarce target domains [10].

FSL has gained widespread attention because it can recognize classes with only a few labeled samples [11]. It is often formulated as a meta-learning paradigm. The goal is to learn transferable knowledge that generalizes from source tasks to target tasks. This is typically achieved by constructing episodic training tasks across source and target domains, enabling the model to learn discriminative embeddings and domain-invariant meta-knowledge [12]. Based on this paradigm, Wang et al. [13] designed a local spectrum-spatial alignment module to balance domain-invariant and domain-discriminative representations. Qin et al. [14] proposed a prototype rectification network to alleviate biased prototypes and domain shifts. Liu et al. [15] further proposed a category-specific prototype self-refinement module, which adaptively updates class prototypes using information from the query set. Although these FSL methods improve classification performance in label-scarce scenarios, they still rely on carefully designed episodic training, source-target task construction, and prototype adaptation. Their performance may be affected by domain shifts, category discrepancies, and the limited generality of task-specific transfer models.

In recent years, foundation models [16] have provided a new paradigm for visual representation learning. Unlike conventional task-specific models, they are pre-trained on large-scale data and can be adapted to downstream tasks with limited labeled samples through fine-tuning or prompting [17]. In hyperspectral remote sensing, HyperSIGMA [18] was developed for hyperspectral intelligence comprehension and demonstrated strong performance across multiple HSI interpretation tasks. Following this line of research, several studies have further explored how hyperspectral foundation models can handle heterogeneous spectral configurations. HyperSL [19] introduced a spectral foundation model that treats hyperspectral signals as sequential tokens and incorporates wavelength information into positional encoding, enabling knowledge transfer across datasets with different spectral ranges and band numbers. HySens [20] developed a sensor-agnostic hyperspectral foundation model by combining a spectral harmonizer with wavelength-based spectral embeddings, improving the adaptability of pretrained models to heterogeneous hyperspectral sensors. From the perspective of downstream adaptation, HyperFree [21] adopted a channel-adaptive and tuning-free strategy to reduce the cost of applying hyperspectral foundation models to downstream tasks. These studies demonstrate the potential of foundation models for HSI analysis. However, their adaptation to classification datasets with diverse spectral band configurations remains insufficiently explored.

HSIs acquired by different sensors often differ in band number, wavelength coverage, spectral response functions, and band intervals. When a pre-trained model has specific input requirements, downstream data are often compressed, selected, interpolated, or reshaped before being fed into the model. Such preprocessing may discard useful spectral information or weaken the local continuity of the original spectra. This is particularly undesirable for HSI classification, where subtle spectral differences are often critical.

To address this issue, we propose a Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification, termed MBTI. Its core idea is to preserve the original spectral information while adapting HSIs with different band configurations to a pre-trained foundation model. Specifically, the full-band HSI is divided into several continuous spectral groups. When the remaining bands are insufficient to form a complete group, a band reuse strategy constructs a valid continuous input branch without zero padding or band discarding. Each spectral group is then fed into an independent branch built upon the pre-trained foundation model. As shown in Fig. 1.

Although the multi-branch design preserves spectral integrity, directly fine-tuning all branches would introduce substantial computational and optimization burdens. We therefore introduce parameter-efficient fine-tuning into MBTI. The pre-trained backbone is frozen, and independent Low-Rank Adaptation (LoRA) modules are inserted into each branch. LoRA approximates the update of a large weight matrix with two trainable low-rank matrices, denoted as \(A\) and \(B\). This design greatly reduces the number of trainable parameters while retaining the adaptation capability of each branch.

Finally, features extracted from different spectral branches must be integrated for classification. Different spectral intervals contribute unequally to different land-cover categories. We therefore design a multi-branch channel attention fusion module to recalibrate and integrate branch features adaptively. This module enhances informative spectral responses and suppresses redundant or less relevant ones.

The main contributions of this paper are summarized as follows:

1. To alleviate the spectral information loss caused by conventional input adaptation strategies, we propose MBTI, a multi-branch efficient fine-tuning framework based on hyperspectral foundation models. It adapts HSIs with different band configurations while preserving full-band spectral information and local spectral continuity.

2. We design a spectral grouping LoRA strategy. Continuous spectral subsets are processed by independent foundation-model branches equipped with branch-specific LoRA modules. This allows different spectral intervals to learn discriminative task-specific features with only a small number of trainable parameters.

3. We introduce a multi-branch channel attention fusion module to adaptively integrate features from different spectral branches. Experiments on three representative HSI datasets demonstrate the effectiveness and efficiency of the proposed MBTI framework.

The remainder of this paper is organized as follows. Section II reviews related work. Section III presents the proposed MBTI framework. Section IV reports the experimental results and analysis. Section V concludes this paper.

2 Related Work↩︎

2.1 Hyperspectral Foundation Models↩︎

The success of the pre-training and fine-tuning paradigm in natural language processing and computer vision has motivated the development of foundation models in remote sensing [22]. By learning transferable representations from large-scale unlabeled or weakly labeled data, foundation models provide a promising solution for downstream tasks where labeled samples are limited. Compared with conventional task-specific models, foundation models are expected to offer stronger generalization and better adaptability across diverse scenes and sensors.

In the hyperspectral community, recent studies have begun to explore foundation models that can exploit rich spectral-spatial information. HyperSIGMA [18] was developed as a hyperspectral intelligence comprehension foundation model and demonstrated strong representation capability across multiple HSI interpretation tasks. SpectralEarth [23] constructed a large-scale EnMAP-based hyperspectral dataset and pretrained foundation models with self-supervised learning, providing an important benchmark for large-scale HSI representation learning. HyperSL [19] regarded hyperspectral signals as sequential spectral tokens and introduced wavelength-based positional encoding to align spectral features across different wavelength ranges. HySens [20] proposed a sensor-agnostic hyperspectral foundation model that harmonizes heterogeneous band distributions into a unified spectral grid and uses wavelength-aware embeddings to enhance cross-sensor transferability. HyperFree [21] further reduced downstream adaptation costs through a channel-adaptive and tuning-free strategy. In addition, HyperFM [24] explored spectral grouping and parameter-efficient modeling for hyperspectral foundation models, showing the potential of group-wise spectral representation learning in large-scale atmospheric retrieval tasks.

Although these methods demonstrate the potential of foundation models for HSI analysis, efficiently adapting them to classification datasets with different spectral band configurations remains challenging. HSIs collected by different sensors often vary in band number, wavelength coverage and band intervals. Existing adaptation strategies commonly rely on spectral compression, band selection, interpolation, spectral harmonization, or fixed-dimensional projection to satisfy the input requirements of pretrained models. These operations improve compatibility but may discard useful spectral information or weaken local spectral continuity. In contrast, this paper focuses on preserving full-band spectral information while adapting hyperspectral foundation models to downstream classification tasks through a multi-branch efficient fine-tuning framework.

2.2 Parameter-Efficient Fine-Tuning↩︎

Foundation models usually contain hundreds of millions of parameters or more. Fully fine-tuning all parameters for each downstream task requires substantial computational and storage resources. It may also lead to overfitting when only a few labeled HSI samples are available. Parameter-Efficient Fine-Tuning (PEFT) has therefore become an important strategy for adapting large pretrained models [25].

Existing PEFT methods can be broadly divided into adapter-based tuning, prompt-based tuning, and low-rank adaptation [26], [27]. Adapter-based methods insert lightweight trainable modules into a frozen backbone, while prompt-based methods introduce learnable tokens or prompts to guide the pretrained model toward downstream tasks. Low-Rank Adaptation (LoRA) assumes that task-specific weight updates have a low intrinsic rank and approximates the update matrix with two small trainable matrices. Because LoRA freezes the original pretrained weights and only optimizes the low-rank matrices, it can greatly reduce the number of trainable parameters while maintaining adaptation capability.

Recently, PEFT has also been explored in remote sensing foundation models. AiRs [28] introduced a remote-sensing-oriented adapter tuning framework by designing spatial context adapters and semantic response adapters, enabling efficient transfer of large pretrained models to object detection, semantic segmentation, and scene classification tasks. UPetu [29] further unified adapter tuning and prompt tuning through an efficient quantization adapter module and a context-aware prompt module, reducing the computational and storage costs of adapting remote sensing foundation models. For spectral remote sensing, SpectralX [30] proposed a parameter-efficient domain generalization framework that adapts optical remote sensing foundation models to multispectral and hyperspectral modalities through a hyper tokenizer and attribute-oriented adapters. In HSI classification, Ligan et al. [31] systematically investigated several PEFT strategies, including LoRA, LoRA+, KronA, LoKr, and KronA+, for adapting a multispectral foundation model to hyperspectral image classification.

Although these studies demonstrate the effectiveness of PEFT in remote sensing and spectral image interpretation, most existing strategies are designed for a single input stream or general spectral modality adaptation. They do not explicitly consider the continuous full-band structure of HSIs when downstream data are divided to satisfy the input requirements of a pretrained hyperspectral foundation model. In HSI classification, different spectral intervals may contain distinct discriminative information, and applying a single shared adaptation module may be insufficient to model the heterogeneity among band groups. To address this issue, this paper introduces branch-specific LoRA modules into different consecutive spectral branches. In this way, each branch can learn task-specific adaptation for its corresponding spectral interval while the pretrained backbone remains frozen.

3 Methodology↩︎

The overall workflow of the proposed Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification (MBTI) is shown in Fig. 2. MBTI consists of three components: 1) spectral-continuity-preserving multi-branch preprocessing, 2) spectral multi-branch LoRA fine-tuning, and 3) multi-branch channel attention fusion.

Given an input HSI with an arbitrary number of spectral bands, MBTI first divides the full-band data into several continuous spectral groups. The grouping follows the input requirement of the pre-trained hyperspectral foundation model. When the last group contains insufficient bands, a band reuse strategy avoids zero padding and preserves local spectral continuity. Each spectral group is then fed into an independent branch built upon the pre-trained foundation model. The backbone is frozen, and branch-specific LoRA modules learn task-related adaptation with a small number of trainable parameters. Finally, features from all spectral branches are projected, concatenated, and recalibrated by a channel attention fusion module for pixel-wise classification.

3.1 Multi-Branch Data Preprocessing Based on Spectral Continuity Preservation↩︎

HSIs acquired by different sensors often vary in spectral band number, wavelength coverage, and band intervals. When such data are adapted to a pre-trained foundation model with specific input requirements, direct band compression or selection may discard useful spectral information and disrupt local spectral continuity. To alleviate this problem, we propose a spectral-continuity-preserving multi-branch preprocessing strategy.

As illustrated in Fig. 3, the proposed strategy divides the original spectrum into continuous groups and constructs the final branch by reusing adjacent bands when necessary.

Let the input HSI be denoted as \[\mathbf{X} \in \mathbb{R}^{H \times W \times B},\] where \(H\) and \(W\) are the spatial height and width, and \(B\) is the number of spectral bands. Let \(M\) denote the number of bands required by each branch. The number of spectral branches is computed as \[N = \left\lceil \frac{B}{M} \right\rceil,\] where \(\lceil \cdot \rceil\) denotes the ceiling operation.

For the first \(N-1\) branches, the input of the \(i\)-th branch is obtained by selecting a continuous spectral interval: \[\mathbf{X}_i = \mathbf{X}_{:,:, (i-1)M+1 : iM}, \quad i=1,2,\ldots,N-1.\] If \(B\) is divisible by \(M\), the last branch is constructed in the same way. Otherwise, the last branch contains only the remaining spectral bands. Let the number of remaining bands be \[R = B - (N-1)M, \quad 0 < R < M.\] Instead of padding zeros, which introduces invalid spectral responses, we reuse adjacent continuous bands from the previous spectral group. The number of reused bands is \[D = M - R.\] The reused spectral subset is selected from the end of the previous group: \[\mathbf{X}_{\mathrm{reuse}} = \mathbf{X}_{:,:, (N-1)M-D+1 : (N-1)M}.\] The remaining spectral subset is \[\mathbf{X}_{\mathrm{remain}} = \mathbf{X}_{:,:, (N-1)M+1 : B}.\] The input of the last branch is then formed as \[\mathbf{X}_N = \mathrm{Concat}_{\lambda} \left( \mathbf{X}_{\mathrm{reuse}}, \mathbf{X}_{\mathrm{remain}} \right),\] where \(\mathrm{Concat}_{\lambda}(\cdot)\) denotes concatenation along the spectral dimension. In this way, all branches have the same number of input bands. The original full-band information is retained, and local spectral continuity is preserved as much as possible.

3.2 Spectral Multi-Branch LoRA Fine-Tuning↩︎

After spectral grouping, each branch processes one continuous spectral subset. This design preserves spectral information, but directly fine-tuning all foundation-model branches would introduce high computational cost. It may also increase the risk of overfitting under limited labeled samples. Therefore, we introduce branch-wise Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning.

As shown in Fig. 4, LoRA introduces a trainable low-rank path in parallel with the frozen pre-trained linear transformation.

For the \(i\)-th spectral branch, the grouped input \(\mathbf{X}_i\) is fed into a pre-trained encoder equipped with branch-specific LoRA modules: \[\mathbf{F}_i = E_i^{\mathrm{LoRA}}(\mathbf{X}_i), \quad i=1,2,\ldots,N,\] where \(E_i^{\mathrm{LoRA}}(\cdot)\) denotes the encoder of the \(i\)-th branch. The backbone parameters of the encoder are frozen, while the LoRA parameters are trainable and independent across branches. Therefore, LoRA is embedded inside the encoder during feature extraction, rather than being applied after \(\mathbf{F}_i\) is obtained.

For a frozen linear transformation in the encoder, let \(\mathbf{W}_0\) denote the pre-trained weight matrix and \(\mathbf{U}_i\) denote its input feature in the \(i\)-th branch. The original forward propagation is \[\mathbf{h}_i = \mathbf{W}_0 \mathbf{U}_i .\] LoRA introduces a trainable low-rank update: \[\Delta \mathbf{W}_i = \frac{\alpha}{r} \mathbf{B}_i \mathbf{A}_i,\] where \(\mathbf{A}_i \in \mathbb{R}^{r \times d_{\mathrm{in}}}\) and \(\mathbf{B}_i \in \mathbb{R}^{d_{\mathrm{out}} \times r}\) are the trainable low-rank matrices of the \(i\)-th branch, \(r\) is the LoRA rank, \(\alpha\) is a scaling factor, and \(\mathbf{W}_0 \in \mathbb{R}^{d_{\mathrm{out}} \times d_{\mathrm{in}}}\). The forward propagation with LoRA is then written as \[\mathbf{h}_i = (\mathbf{W}_0+\Delta \mathbf{W}_i)\mathbf{U}_i = \mathbf{W}_0\mathbf{U}_i + \frac{\alpha}{r} \mathbf{B}_i\mathbf{A}_i\mathbf{U}_i .\]

In our implementation, the above LoRA update is inserted into the linear layers of each Transformer block, including the query, key, value, and output projections in the self-attention module, as well as the two linear transformations in the feed-forward network. In other words, for a target linear layer \(\mathbf{W}_t \in \{\mathbf{W}_q,\mathbf{W}_k,\mathbf{W}_v,\mathbf{W}_o,\mathbf{W}_{f1},\mathbf{W}_{f2}\}\), its branch-specific LoRA-adapted form is \[\mathbf{W}_{t,i}^{\mathrm{LoRA}} = \mathbf{W}_t + \Delta \mathbf{W}_{t,i}, \quad t \in \{q,k,v,o,f1,f2\}.\] In this way, each spectral branch learns independent task-specific adaptations for its corresponding spectral interval, while most pre-trained parameters remain frozen. The branch features are then sent to the fusion module described in the next subsection.

3.3 Multi-Branch Spectral-Channel Recalibration Fusion↩︎

After branch-wise LoRA adaptation, each spectral branch produces a deep feature representation from a specific continuous spectral interval. Directly averaging these branch features is suboptimal because different spectral intervals are not equally informative for different land-cover categories. Simple concatenation may also introduce redundant or weakly relevant responses, especially when adjacent spectral groups are correlated. Therefore, we design a multi-branch spectral-channel recalibration fusion module to adaptively integrate branch-wise deep features before classification.

Unlike a generic attention block applied to the final logits, the proposed fusion module operates on classifier-free deep features extracted by each branch. The output of the pre-trained encoder is first compressed by a branch-specific projection layer. For the \(i\)-th branch, the projected feature is formulated as \[\mathbf{Z}_i = P_i(E_i^{\mathrm{LoRA}}(\mathbf{X}_i)), \quad i=1,2,\ldots,N,\] where \(E_i^{\mathrm{LoRA}}(\cdot)\) denotes the frozen foundation-model encoder equipped with branch-specific LoRA, \(P_i(\cdot)\) is a trainable projection layer without LoRA, and \(\mathbf{Z}_i \in \mathbb{R}^{L \times C}\) is the projected branch feature. Here, \(L\) denotes the token length of the branch feature sequence, and \(C\) is the unified feature dimension of each branch. In our implementation, \(P_i(\cdot)\) compresses the concatenated multi-level encoder features into a compact branch representation. This reduces the cost of subsequent fusion while preserving discriminative information before classification.

The projected branch features are then concatenated along the channel dimension: \[\mathbf{Z} = \mathrm{Concat}_{c} \left( \mathbf{Z}_1,\mathbf{Z}_2,\ldots,\mathbf{Z}_N \right), \quad \mathbf{Z} \in \mathbb{R}^{L \times NC}.\] This concatenated representation can be viewed as a spectral-channel feature bank. Each channel corresponds to a feature response from a spectral branch. Instead of treating all branch channels equally, we learn a global calibration vector from the fused representation. First, global information is aggregated over the token dimension: \[s_c = \frac{1}{L} \sum_{l=1}^{L} Z_{l,c}, \quad c=1,2,\ldots,NC,\] where \(s_c\) summarizes the global response of the \(c\)-th spectral-channel feature. The channel descriptor \(\mathbf{s}\) is then passed through a lightweight bottleneck gating function: \[\mathbf{g} = \sigma \left( \mathbf{W}_2 \delta \left( \mathbf{W}_1 \mathbf{s} \right) \right), \quad \mathbf{g} \in \mathbb{R}^{NC},\] where \(\mathbf{W}_1 \in \mathbb{R}^{\frac{NC}{\rho} \times NC}\) and \(\mathbf{W}_2 \in \mathbb{R}^{NC \times \frac{NC}{\rho}}\) are learnable transformation matrices, \(\rho\) is the reduction ratio, \(\delta(\cdot)\) is the ReLU activation, and \(\sigma(\cdot)\) is the sigmoid function. The learned vector \(\mathbf{g}\) estimates the importance of all branch-channel responses. Since the channels are organized by spectral branches, this operation models both inter-branch importance and intra-branch channel relevance.

The original concatenated feature is recalibrated by the learned spectral-channel weights: \[\widetilde{\mathbf{Z}}_{l,c} = g_c \mathbf{Z}_{l,c}, \quad l=1,2,\ldots,L,\; c=1,2,\ldots,NC.\] This operation emphasizes informative spectral-channel responses and suppresses redundant or weakly relevant ones. The recalibrated feature is then fed into a progressive transition head: \[\mathbf{H} = \phi_3 \left( \phi_2 \left( \phi_1 \left( \widetilde{\mathbf{Z}}\mathbf{W}_1' + \mathbf{b}_1' \right) \mathbf{W}_2' + \mathbf{b}_2' \right) \mathbf{W}_3 + \mathbf{b}_3 \right),\] where \(\mathbf{W}_1' \in \mathbb{R}^{NC \times d_1}\), \(\mathbf{W}_2' \in \mathbb{R}^{d_1 \times d_2}\), and \(\mathbf{W}_3 \in \mathbb{R}^{d_2 \times d_h}\) are learnable linear transformations in the transition head. \(\mathbf{b}_1'\), \(\mathbf{b}_2'\), and \(\mathbf{b}_3\) are bias terms, and \(\phi_1(\cdot)\), \(\phi_2(\cdot)\), and \(\phi_3(\cdot)\) denote activation and dropout operations. The transition head maps the high-dimensional multi-branch feature into a compact discriminative representation. Finally, a linear classifier produces the class logits: \[\mathbf{O} = \mathbf{H}\mathbf{W}_{\mathrm{cls}} + \mathbf{b}_{\mathrm{cls}},\] where \(\mathbf{W}_{\mathrm{cls}} \in \mathbb{R}^{d_h \times K}\) and \(\mathbf{b}_{\mathrm{cls}} \in \mathbb{R}^{K}\), with \(K\) denoting the number of land-cover classes. The fusion module is trained jointly with branch-wise LoRA and projection layers, while the original pre-trained backbone remains frozen. Therefore, the proposed fusion design integrates information from different spectral groups and learns how much each branch-channel response should contribute to classification. The overall procedure of MBTI is summarized in Algorithm 6.

4 Experiments and Analysis↩︎

4.1 Datasets↩︎

To comprehensively evaluate MBTI, experiments were conducted on three widely used HSI classification datasets: Salinas (SA), Kennedy Space Center (KSC), and FangluTeaFarm (TF). These datasets were acquired by different sensors and cover diverse spatial resolutions, spectral ranges, scene types, and land-cover categories. They therefore provide a representative benchmark for evaluating the proposed method. Dataset visualizations and class statistics are provided in Figs. 7–9 and Tables I–III.

Salinas dataset (SA). The SA dataset was collected by the AVIRIS sensor over agricultural areas in Salinas Valley, California. As shown in Fig. 7 and Table I, it contains 224 spectral bands covering 400–2500 nm and has a spatial resolution of 3.7 m. After removing 20 water absorption bands, 204 bands are retained for experiments. The dataset contains 16 land-cover classes, mainly corresponding to crop types such as stubble, celery, and lettuce. Because of its large number of categories and notable intra-class variation, Salinas is widely used as a challenging benchmark for HSI classification.

Kennedy Space Center dataset (KSC). The KSC dataset was acquired by the airborne AVIRIS sensor over the Kennedy Space Center area, Florida, USA, in 1996. As shown in Fig. 8 and Table II, it contains 224 spectral bands and has a spatial resolution of 18 m. The scene includes complex ecological environments and diverse vegetation communities. It contains 13 land-cover classes, such as Scrub, Cabbage palm hammock, Graminoid marsh, and Mud flats. Due to subtle spectral differences among vegetation categories and complex spatial distributions, KSC is commonly used to evaluate the robustness of HSI classification methods.

FangluTeaFarm dataset (TF). The TF dataset was captured in China in 2017 using the Pushbroom Hyperspectral Imager (PHI). As shown in Fig. 9 and Table III, the image size is \(348 \times 512\) pixels, with a spatial resolution of 2.25 m. The dataset contains 80 spectral bands ranging from 417 nm to 855 nm. It represents a tea-farm scene with 10 land-cover categories, including Masson pine, bamboo forest, tea plant, rice paddy, water, and building road. This dataset provides a practical benchmark for complex agricultural and forestry environments.

4.2 Experimental Settings↩︎

4.2.1 Evaluation Metrics↩︎

To quantitatively evaluate classification performance, four commonly used metrics were adopted: overall accuracy (OA), class-wise accuracy, average accuracy (AA), and Cohen’s Kappa coefficient. OA measures the proportion of correctly classified samples: \[\mathrm{OA} = \frac{\sum_{i=1}^{K} n_{ii}}{N},\] where \(K\) is the number of classes, \(N\) is the total number of test samples, and \(n_{ii}\) denotes the number of correctly classified samples in the \(i\)-th class. The class-wise accuracy of the \(i\)-th class is defined as \[\mathrm{Acc}_i = \frac{n_{ii}}{n_{i+}},\] where \(n_{i+}\) denotes the total number of samples belonging to the \(i\)-th ground-truth class. AA is computed as the mean of class-wise accuracies: \[\mathrm{AA} = \frac{1}{K} \sum_{i=1}^{K} \mathrm{Acc}_i.\] Cohen’s Kappa coefficient evaluates the agreement between predictions and ground-truth labels while considering chance agreement: \[\kappa = \frac{ N\sum_{i=1}^{K} n_{ii} - \sum_{i=1}^{K} n_{i+}n_{+i} }{ N^2 - \sum_{i=1}^{K} n_{i+}n_{+i} },\] where \(n_{+i}\) denotes the number of samples predicted as the \(i\)-th class.

4.2.2 Hyperspectral Foundation Model↩︎

HyperSIGMA was selected as the pre-trained hyperspectral foundation model. This choice is based on its strong representation capability for hyperspectral imagery and its demonstrated effectiveness on multiple downstream HSI tasks. Unlike general remote sensing foundation models or multispectral-oriented models, HyperSIGMA is designed for hyperspectral interpretation and learns representations from continuous spectral intervals. This property is consistent with the spectral grouping strategy adopted in MBTI. We used the publicly available parameters of its base version to initialize the spatial encoder in each branch. During fine-tuning, the pre-trained backbone parameters were frozen, and only the branch-wise LoRA modules, feature projection layers, and fusion classifier were updated.

4.2.3 Implementation Details↩︎

For a fair comparison, all methods followed the same training protocol. For each dataset, 10 labeled samples per class were randomly selected for training. After training, MBTI and its controlled variants were evaluated on all labeled pixels to obtain stable full-scene accuracy statistics. Unless otherwise specified, the final MBTI configuration used LoRA rank \(r=8\). The number of bands per branch was set to 70 for KSC and Salinas, and 35 for FangluTeaFarm, according to the sensitivity analysis. The proposed method was implemented in Python with PyTorch. Experiments were conducted on a server equipped with dual Intel Xeon Platinum 8358P CPUs and NVIDIA A800 80GB GPUs. The software environment used NVIDIA driver 565.57.01 and CUDA 12.7.

4.3 Algorithm Performance and State-of-the-Art Comparisons↩︎

To evaluate the effectiveness of MBTI, we compare it with representative methods from several categories. These include classical machine learning, supervised deep learning, cross-domain few-shot learning, hyperspectral foundation models, and fine-tuning-free large models. Specifically, the compared methods include SVM [32]; five supervised deep learning methods, i.e., CDCNN [33], DBDA [34], DBMA [35], FDSSC [36], and SSRN [7]; three cross-domain few-shot learning methods, i.e., DCFSL [37], Gia-CFSL [38], and HFSL [39]; the hyperspectral foundation model HyperSIGMA [18]; and the fine-tuning-free hyperspectral foundation model HyperFree [21].

Experiments are conducted on KSC, Salinas, and FangluTeaFarm. Classification performance is evaluated using OA, AA, and Kappa. For clarity, all Kappa values reported in the tables are multiplied by 100. The quantitative results are presented in Tables V–VII.

As shown in Tables V–VII, MBTI achieves the best OA and Kappa on all three datasets. On KSC, MBTI obtains 99.16% OA, 99.26% AA, and 99.06% Kappa. It outperforms the second-best method by 1.94% in OA and 2.15% in Kappa. On Salinas, MBTI achieves 96.68% OA and 96.31% Kappa, surpassing HyperSIGMA in the two overall metrics while maintaining competitive AA. On FangluTeaFarm, MBTI obtains 95.65% OA and 93.66% Kappa, showing clear improvements in overall classification performance. These results indicate that full-band spectral preservation and branch-wise efficient adaptation improve the transfer of hyperspectral foundation models. Classification maps generated by different methods are shown in Figs. 10–12.

4.4 Ablation Studies↩︎

The proposed MBTI framework consists of three key components: spectral-continuity-preserving multi-branch preprocessing, spectral multi-branch LoRA, and multi-branch attention-based feature fusion. To verify their contributions, we conduct ablation experiments by removing or replacing individual modules. Three variants are considered:

1. PCA replacement: the proposed multi-branch spectral preprocessing strategy is replaced by PCA-based dimensionality reduction.

2. Without LoRA: the branch-specific LoRA adapters are removed to evaluate the necessity of parameter-efficient adaptation.

3. Without attention: the multi-branch attention fusion module is removed, and features from different branches are directly fused without adaptive recalibration.

For fairness, all ablation experiments use the same random seed and experimental settings. The results are reported in Table 7. Overall, removing or replacing any key component leads to performance degradation on most evaluation metrics, which verifies the necessity of the proposed design.

When the proposed multi-branch preprocessing strategy is replaced by PCA, the performance drops clearly on all three datasets. This indicates that PCA-based compression may discard useful spectral details and disturb the original spectral continuity. In contrast, the proposed grouping and band-reuse strategy preserves the full-band spectral information and better matches the continuous-band modeling behavior of the hyperspectral foundation model.

Removing the attention-based fusion module also causes evident performance degradation. This shows that directly fusing branch features without adaptive recalibration is insufficient, because different spectral intervals contribute unequally to different land-cover categories. The proposed fusion module can enhance informative branch-channel responses and suppress redundant or weakly discriminative ones.

Removing the LoRA modules degrades the overall classification performance, especially on Salinas and FangluTeaFarm. This demonstrates that branch-specific low-rank adaptation is important for learning discriminative task-specific features from different spectral intervals. Instead of fully updating the backbone, LoRA provides a parameter-efficient way to adapt the pre-trained model under limited labeled samples.

The complete MBTI framework achieves the best OA and Kappa on all three datasets and obtains the best or highly competitive AA. These results confirm that spectral-continuity-preserving preprocessing, branch-specific LoRA adaptation, and multi-branch attention fusion are complementary. Their combination enables MBTI to achieve a better balance between classification accuracy and efficient downstream adaptation.

4.5 Analysis of Multi-Branch Feature Fusion Methods↩︎

In MBTI, features from different spectral intervals are extracted independently by multiple branches. Effective integration of these branch-specific representations is therefore critical. To evaluate the influence of fusion strategies, we compare direct concatenation with attention-based fusion mechanisms.

The classification results are reported in Table IX. Direct concatenation treats all branch features equally and ignores the different contributions of spectral intervals. This limits the discriminative ability of the fused representation. Self-attention and graph attention introduce adaptive modeling mechanisms, but they do not consistently improve performance in this setting. The proposed multi-branch attention fusion module achieves the best overall performance, indicating that spectral-branch recalibration is more suitable for integrating the proposed branch features.

4.6 Parameter Sensitivity Analysis↩︎

This section analyzes two key hyperparameters in MBTI: the number of spectral bands assigned to each branch and the LoRA rank \(r\). Since hyperspectral datasets differ in band numbers and scene characteristics, the sensitivity analysis is conducted on KSC, Salinas, and FangluTeaFarm. The results are reported in Tables X and XI.

4.6.1 Effect of the Number of Bands per Branch↩︎

The number of bands per branch determines how the original spectrum is divided into continuous spectral groups. A smaller value produces more branches with finer spectral partitioning. A larger value produces fewer branches with wider spectral coverage. As shown in Table X, this trade-off affects classification performance on all three datasets.

For KSC and Salinas, assigning 70 bands to each branch achieves the best overall performance, with OA values of 99.16% and 96.68%, respectively. For FangluTeaFarm, 35 bands per branch obtains the highest OA and Kappa, while 40 bands per branch achieves the highest AA. These results indicate that excessively fine spectral partitioning does not necessarily improve accuracy. Too many branches increase the difficulty of feature fusion, memory consumption, and optimization. Conversely, overly coarse partitioning may reduce the ability to capture discriminative information from different spectral intervals. Therefore, the final settings are 70 bands per branch for KSC and Salinas, and 35 bands per branch for FangluTeaFarm.

4.6.2 Effect of the LoRA Rank↩︎

The LoRA rank \(r\) controls the dimension of the low-rank adaptation space. A larger rank provides stronger adaptation capacity, but it also introduces more trainable parameters. To evaluate its influence, \(r\) is selected from \(\{2,4,8,16,32,64,128\}\), while the number of bands per branch is fixed to the optimal setting of each dataset.

As shown in Table XI, the influence of \(r\) is dataset-dependent, and the improvement is not monotonic. KSC and FangluTeaFarm obtain the best OA and Kappa at \(r=128\), whereas Salinas achieves the best OA and Kappa at \(r=8\). Larger ranks enlarge the task-specific adaptation capacity, but they also substantially increase the parameter cost. For example, on KSC, the trainable parameters increase from 3.76M at \(r=2\) to 61.48M at \(r=128\), while the OA gain is limited. Considering both accuracy and efficiency, \(r=8\) is adopted as the default setting because it achieves competitive performance with substantially fewer trainable parameters.

4.7 Efficiency and Cross-Framework Inference Analysis↩︎

To further analyze the practical efficiency and deployment potential of MBTI, we conduct two additional experiments on the FangluTeaFarm dataset. The first experiment compares MBTI with representative hyperspectral foundation-model-based methods in terms of classification accuracy, trainable parameters, training-state memory, and inference time. The second experiment evaluates inference behavior of MBTI under different deep learning frameworks.

4.7.1 Foundation-model efficiency↩︎

As shown in Table 11, MBTI achieves the highest OA among the compared foundation-model-based methods. Compared with HyperSIGMA, MBTI reduces the number of trainable parameters by freezing most pretrained weights and optimizing only branch-specific LoRA modules and lightweight fusion layers. Under the AdamW optimizer, the training-state memory consists of trainable parameters, gradient storage, and optimizer states. Since AdamW maintains two additional moment tensors for each trainable parameter, reducing the number of trainable parameters directly decreases the optimization-related memory cost. As a result, the estimated training-state memory of MBTI is only 98.53 MiB, whereas HyperSIGMA requires 2963.85 MiB. Thus, most of the optimizer-related memory in full fine-tuning is avoided in MBTI.

Compared with HyperFree, MBTI achieves higher classification accuracy and much faster inference. Although HyperFree avoids conventional fine-tuning and therefore does not introduce trainable parameters during downstream adaptation, its tuning-free adaptation strategy leads to a relatively heavy inference process. In contrast, MBTI performs task-specific adaptation through lightweight LoRA modules during training and then uses a standard forward inference pipeline. Therefore, MBTI provides a better balance between classification accuracy, adaptation cost, and inference efficiency.

4.7.2 Cross-framework inference↩︎

To examine the deployment characteristics of MBTI across different deep learning frameworks, we further evaluate it under Jittor in addition to PyTorch. Specifically, the final PyTorch checkpoint is converted into a Jittor-readable format and tested with a Jittor-based inference implementation on the FangluTeaFarm dataset. This experiment is designed to assess whether the trained MBTI checkpoint can preserve its accuracy while achieving lower memory consumption after framework conversion.

The cross-framework inference results are shown in Fig. 13. The Jittor implementation achieves comparable classification performance to the PyTorch implementation. Specifically, Jittor obtains slightly higher OA and Kappa, while its AA is slightly lower. These small differences may be caused by framework-level numerical differences, operator implementation details, and the conversion of model weights between PyTorch and Jittor. Overall, the results indicate that the converted Jittor inference pipeline can preserve the main discriminative capability of the trained MBTI model.

In terms of resource consumption, Jittor shows a clear advantage in peak GPU memory usage. On FangluTeaFarm, the peak GPU memory decreases from 3484 MiB in PyTorch to 2502 MiB in Jittor. This suggests that Jittor has potential for memory-efficient deployment of MBTI, especially when full-scene inference needs to be performed on devices with limited GPU memory. Meanwhile, the inference time of Jittor is slightly longer than that of PyTorch, increasing from 1.13 s to 1.18 s. This may be related to differences in graph compilation, runtime scheduling, and operator-level optimization between the two frameworks. In addition, although the model parameters are converted into a Jittor-compatible format, some modules originally developed and optimized in PyTorch may not achieve identical execution efficiency after being reimplemented with Jittor operators.

Overall, the above results provide two complementary perspectives on the efficiency of MBTI. The comparison with HyperSIGMA and HyperFree shows that MBTI achieves strong classification performance with much lower optimization-related memory cost than full fine-tuning. The cross-framework experiment further shows that MBTI can be deployed under a Jittor-based inference pipeline with reduced peak GPU memory and comparable accuracy. Future work will further investigate native Jittor training, framework-specific operator optimization, and more consistent cross-framework deployment strategies for hyperspectral foundation models.

5 Conclusion↩︎

This paper proposed MBTI, a multi-branch efficient fine-tuning framework for hyperspectral image classification based on hyperspectral foundation models. Instead of compressing or selecting bands to satisfy fixed model inputs, MBTI preserves the full-band spectrum by dividing HSIs into continuous spectral branches and reusing adjacent bands when the final branch is incomplete. This design reduces spectral information loss and maintains local spectral continuity during downstream adaptation. By inserting branch-specific LoRA modules into the frozen pretrained backbone, MBTI adapts different spectral intervals with only a small fraction of trainable parameters, while the spectral-channel recalibration module further integrates branch features according to their discriminative contributions. Experiments on KSC, Salinas, and FangluTeaFarm show that MBTI consistently outperforms a wide range of representative methods. Ablation studies further confirm the effectiveness of each component in improving inference efficiency and deployment flexibility. Future work will explore more general spectral alignment and knowledge transfer strategies to improve adaptation across sensors, scenes, and downstream tasks.

References↩︎

---

1. Corresponding authors: Di Wang, Fengxiang Wang, Gang Wan, and Bo Du↩︎

2. Mingzhen Xu, Di Wang, and Bo Du are with the School of Computer Science, Wuhan University, Wuhan, China. (e-mail: mingzhenx86@gmail.com; d_wang@whu.edu.cn; dubo@whu.edu.cn)↩︎

3. Haonan Guo and Liangpei Zhang are with the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China. (e-mail: haonan.guo@whu.edu.cn; zlp62@whu.edu.cn)↩︎

4. Yinghua Qu and Huiwen Yao are with China Petroleum Pipeline Engineering Corporation, Langfang, China, and also with Hebei Key Laboratory of Underground Energy Storage Technology, Langfang, China. (e-mail: quyinghua@cppe.com.cn; yaohuiwen@cppe.com.cn)↩︎

5. Zhiliang Zhou and Lei Zhang are with the No. 7 Oil Production Plant, Changqing Oilfield Branch, PetroChina Company Limited, Xi’an, China. (e-mail: zhoulz_cq@petrochina.com.cn; zl17_cq@petrochina.com.cn)↩︎

6. Rui Zhao is with the Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China. (e-mail: zhaorui@nbu.edu.cn)↩︎

7. Fengxiang Wang is with the College of Computer Science and Technology, National University of Defense Technology, Changsha, China. (e-mail: wfx23@nudt.edu.cn)↩︎

8. Gang Wan is with the School of Aerospace Information, Space Engineering University, Beijing, China, and also with the Key Laboratory of Intelligent Processing and Application Technology of Satellite Information, Beijing, China (e-mail: wangang@hgd.edu.cn)↩︎