1 Introduction↩︎

Hyperspectral imaging systems capture surface information across numerous spectral bands, providing richer spectral details than multispectral or natural images. This enables precise characterization of ground objects [1]–[4]. Moreover, the wide spectrum of information makes them extremely valuable for a variety of applications, including anomaly detection, surveillance, environmental monitoring, and satellite image scene classification [5]–[9]. However, hyperspectral images often suffer from low resolution due to environmental factors and sensor energy limitations [10]. Deep learning-based super-resolution (SR) methods have shown remarkable success, leveraging large models to reconstruct high-resolution images from degraded low-resolution inputs [11], [12]. Nonetheless, due to advancements in deep learning models, the number of parameters in deep learning networks has increased significantly. As a result, numerous lightweight super-resolution models have gained significant attention in recent years [13], focusing on reducing model parameters and computational complexity through various strategies.

In particular, knowledge distillation [14] is widely used for model compression, in which a large model (the teacher network) is first trained on the original data and then used to guide the intermediate feature representations of a smaller model (the student network). For instance, Gao et al. [15] employ a lightweight student SR model to acquire knowledge from a deeper teacher SR network. Other model compression techniques, such as pruning [16] and low-rank factorization [17], have also proven effective for developing super-resolution models. For instance, Zhan et al. [16] integrate neural architecture search with a layer-wise pruning strategy to develop efficient super-resolution model. Wu et al. [17] propose a convex regularizer for low-rank solutions, addressing unconstrained cases with proximal gradient and a custom PG-like algorithm. In addition, some quantization-based methods [18], [19] leverage low-bit representations to enhance the inference speed of super-resolution models. However, these models often exhibit a significant performance gap compared to state-of-the-art super-resolution methods [20].

In order to reduce the size of the neural network, convolutional filters play a crucial role, and significant efforts have been made to maximize their potential. For instance, MobileNet [21] model introduces depthwise separable convolution, demonstrating great potential for network decomposition. Ahn et al. [22] employ group convolution instead of depthwise convolution to make model efficiency more adaptable. Kim et al. [23] propose a deeply recursive convolutional network, arguing that increasing recursion depth can enhance performance without introducing additional parameters for extra convolutions. Dilated convolution is another modified version of standard convolutions that utilizes a distinctive grid structure [24]. It has been widely adopted to expand the receptive field of kernels and enhance global information aggregation, leading to notable improvements over previous methods. However, the classification accuracy of these models may be compromised during compression, as a large number of image features are omitted due to the simplified convolution operations [25].

Our approach is inspired by MobileNet [21], which reduces the number of parameters and computational costs by introducing the concept of depthwise separable convolution. In particular, we aim to combine depthwise separable convolutions with dilated convolutional fusion blocks [26] to develop a novel super-resolution model called the depthwise separable dilated convolution network (DSDCN) for hyperspectral images. Specifically, the dilated convolutional fusion block consists of two different types of convolutions: (1) dilated convolution (spatial branch) and (2) pointwise convolution (spectral branch). The outputs of the dilated convolution and pointwise convolution are then concatenated to effectively integrate spatial and spectral information. By doing so, we make our model suitable for hyperspectral super-resolution tasks while minimizing the compromise on model parameters compared to the MobileNet architecture. To further improve the performance of the proposed method, we employ a custom loss function that integrates mean squared error (MSE), an L2 regularization-based constraint, and a spectral angle-based loss to ensure high-fidelity reconstruction. In summary, our contributions are three-fold:

1. We present a novel lightweight DSDCN that combines depthwise separable convolutions, residual connections, and dilated convolutional fusion to enhance spatial resolution and preserve spectral integrity..

2. A custom loss function is introduced, integrating mean squared error (MSE), an L2 regularization constraint, and a spectral angle-based loss to enhance high-fidelity reconstruction.

3. Experiments on two hyperspectral datasets are conducted across various resolution degradation-restoration scenarios (2\(\times\), 4\(\times\), and 8\(\times\) downsampling), demonstrating competitive performance on both datasets.

2 Methodology↩︎

Fig. 1 shows the band grouping and the three main components of the model: (1) depthwise separable convolutions with residual connections, (2) dilated convolution fusion, and (3) upsampling. We first define the band grouping along with other components. The following subsections detail this and the custom loss used in the model.

2.1 Band Grouping↩︎

Since hyperspectral images consist of hundreds of spectral bands, processing all bands together can be computationally challenging and may introduce redundancy due to the high spectral correlation between bands. To mitigate this, we utilize band grouping [27], which involves partitioning adjacent bands into overlapping groups for seamless integration with our proposed model. Specifically, hyperspectral bands are organized into overlapping subgroups by defining a fixed group size with a designated overlap, ensuring that consecutive subgroups share common bands.

2.2 Depthwise Separable Convolution↩︎

Given a low-resolution hyperspectral image \(X \in \mathbb{R}^{H \times W \times B}\), where \(H\), \(W\), and \(B\) denote the spatial height, width, and number of spectral bands, respectively, our goal is to reconstruct a high-resolution image \(\hat{X} \in \mathbb{R}^{\alpha H \times \alpha W \times B}\) with an upscaling factor \(\alpha\). To achieve this, we design a deep neural network \(\mathcal{F}(X; \theta)\) that learns the LR-to-HR mapping efficiently while preserving spectral integrity.

We begin by extracting spatial and spectral features using depthwise separable convolutions. Specifically, we empirically employ three such blocks, where each block decomposes a standard convolution into two sequential operations [21]: \[X_d' = W_d * X,\] where \(W_d\) denotes the depthwise convolution kernel applied independently to each spectral band. Next, a pointwise convolution is applied to combine features across channels: \[X_p' = W_p * X_d',\] where \(W_p\) is a \(1 \times 1\) convolution kernel that processes each spatial location across all channels. To improve training stability and mitigate vanishing gradients, we incorporate a residual connection by applying a separate \(1 \times 1\) convolution to the original input \(X\) [28]:

\[X_{\text{res}} = W_s * X,\] where \(W_s\) is the projection kernel for aligning dimensions. The final output of the block is obtained by fusing the residual and transformed features: \[C_{\text{out}} = X_p' + X_{\text{res}},\] where \(C_{\text{out}}\) is the output feature map of the depthwise separable convolution block.

2.3 Dilated Convolution Fusion↩︎

To integrate both local and global spatial dependencies while maintaining a lightweight structure, we employ a dilated fusion block composed of three parallel \(3 \times 3\) convolutions with increasing dilation rates \(r = \{1, 2, 3\}\). Given an input feature map \(\mathbf{F}_{\text{in}} \in \mathbb{R}^{H \times W \times C_{\text{in}}}\), feature extraction is performed as [26]: \[F_i = \text{ReLU}(Q_i * F_{\text{in}}), \quad i \in \{1, 2, 3\},\] where each \(Q_i \in \mathbb{R}^{3 \times 3 \times C_{\text{in}} \times C}\) is a convolutional kernel with dilation rate \(r_i \in \{1, 2, 3\}\), and \(C\) is the number of output channels per branch. The feature maps \(F_1\), \(F_2\), and \(F_3\) are obtained by applying ReLU-activated convolutions with kernels \(Q_1\), \(Q_2\), and \(Q_3\), respectively, to the same input \(F_{\text{in}}\), and are then concatenated along the channel dimension. \[F_{\text{concat}} = \text{Concat}(F_1, F_2, F_3),\] The concatenated features are then passed through a final \(1 \times 1\) convolution to fuse the multi-scale information: \[F_{\text{out}} = \text{ReLU}(Q_f * F_{\text{concat}}),\] where \(Q_f \in \mathbb{R}^{1 \times 1 \times 3C \times C}\) is the fusion kernel. The output \(F_{\text{out}} \in \mathbb{R}^{H \times W \times C}\) represents the final feature map enriched with contextual spatial features.

2.4 Upsampling with Transpose Convolution↩︎

To increase spatial resolution, we employ an upsampling block based on transpose convolution [29]. Given a low-resolution feature map \(U_{\text{in}} \in \mathbb{R}^{H \times W \times C_u}\), a learnable transpose convolution is defined as: \[U_{\text{tr}} = K_{\text{tr}} * U_{\text{in}},\] where \(K_{\text{tr}} \in \mathbb{R}^{k \times k \times C_u \times C_o}\) is the transpose convolution kernel, and \(U_{\text{tr}} \in \mathbb{R}^{\alpha H \times \alpha W \times C_o}\) is the upsampled output. Here, \(C_u\) is the number of input channels, and \(C_o\) is the number of output channels after upsampling. The upsampled output is then passed through a non-linearity: \[U_{\text{act}} = \text{ReLU}(U_{\text{tr}}).\] Furthermore, we employ a skip connection by projecting and upsampling the input feature map: \[U_{\text{skip}} = K_{\text{skip}} * \text{Up}(U_{\text{in}}),\] where \(K_{\text{skip}} \in \mathbb{R}^{1 \times 1 \times C_u \times C_o}\) is a projection kernel. The final output is obtained by summing the skip and upsampled features: \[U_{\text{out}} = U_{\text{act}} + U_{\text{skip}},\] where \(U_{\text{out}} \in \mathbb{R}^{\alpha H \times \alpha W \times C_o}\) is the resulting feature map.

2.5 Custom Loss Function↩︎

We employ a custom loss function that combines three components: mean squared error (MSE), spectral angle mapper (SAM) loss, and \(\ell_2\) loss. This combination ensures both pixel-wise accuracy and spectral fidelity in hyperspectral reconstruction. The total loss is defined as: \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{MSE}} + \lambda_1 \cdot \mathcal{L}_{\text{SAM}} + \lambda_2 \cdot \mathcal{L}_{\ell_2},\] where: \(\mathcal{L}_{\text{MSE}} = \frac{1}{N} \sum_{i=1}^{N} (y^{(i)}_{\text{true}} - y^{(i)}_{\text{pred}})^2\) minimizes pixel-wise differences, \(\mathcal{L}_{\text{SAM}} = \frac{1}{N} \sum_{i=1}^{N} \cos^{-1}\left( \frac{\langle y^{(i)}_{\text{true}}, y^{(i)}_{\text{pred}} \rangle}{\|y^{(i)}_{\text{true}}\| \cdot \|y^{(i)}_{\text{pred}}\|} \right)\) measures the angular discrepancy between predicted and ground-truth spectral vectors, \(\mathcal{L}_{\ell_2} = \frac{1}{N} \sum_{i=1}^{N} \| y^{(i)}_{\text{true}} - y^{(i)}_{\text{pred}} \|_2^2\) reinforces global spectral consistency. Here, \(\lambda_1\) and \(\lambda_2\) are weighting coefficients that balance the influence of SAM and \(\ell_2\) loss, respectively. In our experiments, we set \(\lambda_1 = 0.5\) and \(\lambda_2 = 0.03\).

3 Experimental setup↩︎

3.1 Datasets and Implementation↩︎

Two publicly available hyperspectral datasets, PaviaC and PaviaU, containing 102 and 103 spectral bands, respectively, are used in our study. We used a patch size of \(144 \times 144\), following the protocol of previous work [20], for training and testing the datasets. Specifically, the test set patch is extracted from the bottom center of the PaviaC dataset and the top left of the PaviaU dataset, while the remaining image regions are used for training [20].

To generate low-resolution images, we employ area-based sampling and downscale them by factors of 2×, 4×, and 8×. During training, the Adam optimizer was employed with a batch size of 4. Additionally, an early stopping function was used to prevent fixed epochs and avoid overfitting. Both datasets were divided into band groups of size 32, with one-fourth overlap between them. We adopted several widely used metrics to evaluate the quality of reconstructed images quantitatively, including mean peak signal-to-noise ratio (MPSNR), mean structural similarity index (MSSIM), and Spectral Angle Mapper (SAM)[36].

3.2 Ablation Study on Band Grouping↩︎

Table I presents an ablation study evaluating the effects of band grouping size and the custom loss function on the performance of the proposed DSDCN model. The baseline without band grouping yields an MSSIM of 0.9482 and an MPSNR of 35.017 dB. It can be observed that using a band grouping size of 16 improves performance, while a grouping size of 32 achieves the best results (MSSIM: 0.9578, MPSNR: 36.434 dB). However, increasing the group size to 48 slightly reduces accuracy, suggesting that excessive grouping may weaken the spatial-spectral feature alignment. Performance is also reported using only the MSE loss (without the custom loss), with the group size fixed at 32. This results in a performance drop (MSSIM: 0.9507, MPSNR: 36.127 dB), highlighting the benefits of incorporating spectral angle and L2 constraints into the loss function.

3.3 Comparison with State-of-the-Art Methods↩︎

As shown in Table I, we compare the parameter count of our proposed model with several state-of-the-art super-resolution methods, including ERCSR (1.59M) [30], MCNet (2.17M) [31], PDENet (2.30M) [11], and CSSFENet (1.61M) [20]. DSDCN contains only 0.96 million parameters, substantially fewer than those of the other methods.

Table II presents a detailed quantitative comparison under multiple scaling factors (2×, 4×, and 8×) on both the PaviaC and PaviaU datasets. The comparison includes various state-of-the-art methods such as VDSR [32], EDSR [33], MCNet [31], MSDformer [12], MSFMNet [34], AS3 ITransUNet [35], PDENet [11], and CSSFENet [20]. Despite its lightweight nature, our model ranks first in the PaviaC dataset and achieves competitive performance on the PaviaU dataset in terms of PSNR and SAM. These results emphasize the strength of our architecture in capturing both spatial and spectral information for hyperspectral image super-resolution.

4 Conclusion↩︎

In this study, we introduced a lightweight depthwise separable dilated convolutional network (DSDCN) for hyperspectral super-resolution. The model leverages depthwise separable convolutional blocks with a dilated convolution fusion mechanism to improve spatial and spectral feature extraction. Additionally, we proposed a custom loss function that integrates MSE, L2 regularization, and a spectral angle-based loss to ensure high-fidelity reconstruction. Experimental results demonstrate that our approach achieves competitive performance on two publicly available hyperspectral datasets while maintaining a relatively compact model size. This makes our method a practical and efficient solution for real-world hyperspectral image super-resolution tasks.

Acknowledgment↩︎

This project has been funded by the European Union’s NextGenerationEU instrument and the Research Council of Finland under grant 348153, as part of the project Artificial Intelligence for Twinning the Diversity, Productivity and Spectral Signature of Forests (ARTISDIG). We also gratefully acknowledge CSC for providing access to the LUMI supercomputer, operated by the EuroHPC Joint Undertaking.

References↩︎