¹

1 Introduction↩︎

Remote sensing scene classification has witnessed significant progress with the increasing availability of high-resolution aerial imagery and advances in deep learning. Early approaches relied on handcrafted feature descriptors, which were limited in capturing complex spatial patterns and semantic information. The adoption of convolutional neural networks (CNNs) enabled automatic hierarchical feature learning, while early feature fusion and multiscale frameworks demonstrated the effectiveness of multiscale representation and feature aggregation for aerial scene classification [1]–[5].

Subsequent attention-based CNN methods incorporated channel attention, multiscale attention, bidirectional fusion, feature refinement, multilevel inheritance, and adaptive transfer learning to improve discriminative capability and feature propagation [6]–[13]. More recently, transformer-based and hybrid CNN–Transformer architectures have further enhanced contextual understanding, multiscale feature interaction, and adaptive feature fusion through graph convolution, cross-attention, spatial–channel modeling, and advanced fusion strategies [14]–[29]. Despite these advances, many existing methods either rely on single-backbone architectures, limiting feature diversity, or introduce increased architectural complexity while providing limited mechanisms for effective multiscale feature propagation.

To address these limitations, we propose a dual-backbone multiscale feature fusion framework with residual feature propagation network. The proposed framework employs ConvNeXt-Tiny [30] and EfficientNet-B1 [31] as parallel backbone networks to extract diverse hierarchical feature representations. A multiscale feature fusion mechanism with residual feature propagation facilitates effective feature interaction across multiple resolutions, while spatial attention enhances discriminative feature learning by emphasizing informative regions. Furthermore, a two-stage training strategy comprising backbone freezing followed by selective fine-tuning ensures stable optimization and improved generalization. Experimental results on the AID dataset using a stratified 50:50 train–test split show an average classification accuracy of 97.46% \(\pm\) 0.14%, indicating consistent performance.

2 Methodology↩︎

The core idea of this work (Fig. 1) is to process input images through parallel streams to extract complementary semantic and spatial feature representations. These multi-resolution features are dynamically shared across stages via residual propagation and combined using a channel-wise fusion mechanism. A spatial attention module then refines the fused maps to emphasize distinct object boundaries while filtering out complex background noise. Finally, the network uses a robust two-stage fine-tuning strategy to generate accurate domain-adapted scene classifications. We detail the proposed approach in the following subsections comprehensively.

2.1 Dual-Backbone Feature Extraction↩︎

Single-backbone architectures often have limited capability to learn diverse feature representations, which can reduce their effectiveness in modeling the complex spatial arrangements and semantic variations present in remote sensing scenes. To overcome this limitation, the proposed framework employs a dual-backbone architecture comprising ConvNeXt-Tiny and EfficientNet-B1. Owing to their distinct architectural designs, the two networks learn diverse hierarchical feature representations, where ConvNeXt-Tiny captures rich semantic context while EfficientNet-B1 preserves fine-grained spatial details. The integration of these diverse representations enhances multiscale feature learning, enabling more effective modeling of complex aerial scenes.

Given an input image \(X\), the two backbone networks extract hierarchical feature maps at multiple scales: \[\{F_{C}^1, F_{C}^2, F_{C}^3\} = \mathcal{G}_{\text{ConvNeXt}}(X)\] \[\{F_{E}^1, F_{E}^2, F_{E}^3\} = \mathcal{G}_{\text{EfficientNet}}(X)\] where \(\mathcal{G}_{\text{ConvNeXt}}\) and \(\mathcal{G}_{\text{EfficientNet}}\) denote the feature extraction functions of ConvNeXt-Tiny and EfficientNet-B1, respectively. The feature maps \(F_{C}^i\) and \(F_{E}^i\) correspond to the outputs of three progressively deeper feature stages of the respective backbone networks, capturing hierarchical representations at different spatial resolutions.

Since the extracted feature maps have different channel dimensions, channel alignment is performed before multiscale feature fusion as: \[\hat{F} = \text{ReLU}(\text{BN}(\text{Conv}_{1\times1}(F)))\] where \(F\) denotes the input feature map, \(\hat{F}\) is the channel-aligned feature map, and \(\text{Conv}_{1\times1}\) represents the channel alignment operation. The \(1\times1\) convolution projects feature maps from both backbones into a common channel space.

2.2 Multiscale Feature Fusion with Residual Propagation↩︎

Existing multiscale approaches often process feature maps independently at different scales, resulting in limited inter-scale interaction and reduced contextual understanding. To address this issue, a multiscale feature fusion mechanism with residual feature propagation is introduced to facilitate effective information exchange across hierarchical feature levels.

Residual feature propagation is applied to enable feature propagation across scales: \[\tilde{F}_{C}^i = F_{C}^i + \mathcal{R}(F_{C}^{i-1})\] \[\tilde{F}_{E}^i = F_{E}^i + \mathcal{R}(F_{E}^{i-1})\] where \(\mathcal{R}\) denotes spatial resizing for alignment. This residual aggregation improves gradient flow and ensures effective feature reuse across multiple scales.

Feature maps from both backbones are fused at corresponding levels using channel-wise concatenation: \[F_{\text{fused}}^i = \text{Conv}_{1\times1}([\tilde{F}_{C}^i \parallel \tilde{F}_{E}^i])\] where \(\parallel\) denotes concatenation along the channel dimension, and \(\text{Conv}_{1\times1}\) represents convolution followed by batch normalization. This fusion integrates diverse feature representations to jointly capture local spatial details and high-level semantic information.

The fused multiscale feature maps \(F_{\text{fused}}^1\), \(F_{\text{fused}}^2\), and \(F_{\text{fused}}^3\) are further refined to \(F_{\text{atten}}^i\) using spatial attention to emphasize informative spatial regions at different hierarchical levels before final multiscale aggregation.

To further integrate multiscale information, a pyramid aggregation strategy is applied: \[F_{\text{final}} = \sum_{i=1}^{3} \mathcal{P}(F_{\text{atten}}^i)\] where \(\mathcal{P}\) denotes pooling operations used for spatial alignment. This aggregation integrates information from multiple spatial resolutions to generate a robust feature representation for classification.

2.3 Spatial Attention Mechanism↩︎

In complex aerial scenes, a spatial attention mechanism is employed to emphasize informative regions while suppressing less relevant features.

For a feature map \(F_{\text{fused}}^i\), spatial descriptors are computed using channel-wise average and max pooling: \[F_{\text{avg}} = \text{AvgPool}(F_{\text{fused}}^i), \quad F_{\text{max}} = \text{MaxPool}(F_{\text{fused}}^i)\] These descriptors capture diverse information, where average pooling encodes global contextual information and max pooling highlights salient activations. The descriptors are concatenated and passed through a convolution operation: \[M_s(F_{\text{fused}}^i) = \sigma(\text{Conv}_{7\times7}([F_{\text{avg}} \parallel F_{\text{max}}]))\] where \(\text{Conv}_{7\times7}\) denotes a \(7\times7\) convolution, and \(\sigma\) is the sigmoid activation function. The resulting attention map \(M_s\) represents spatial importance weights.

The refined feature map is obtained as: \[F_{\text{atten}}^i = M_s(F_{\text{fused}}^i) \otimes F_{\text{fused}}^i\] where \(\otimes\) denotes element-wise multiplication. This improves feature discrimination by emphasizing informative spatial regions.

3 Experiments and Results↩︎

3.1 Dataset Description↩︎

Experiments were conducted on the Aerial Image Dataset (AID) [32], a widely used benchmark dataset for remote sensing scene classification. The dataset contains 10,000 high-resolution aerial images of size approximately \(600\times600\) pixels categorized into 30 distinct scene classes, including residential, industrial, forest, airport, and farmland. Due to significant intra-class variability and inter-class similarity, the dataset presents considerable challenges for accurate scene classification. A stratified 50:50 train–test split is adopted to preserve balanced class distribution.

3.2 Image Preprocessing↩︎

Each RGB image is resized and preprocessed using EfficientNet normalization. To improve class balance and enhance model robustness, upsampling-based data augmentation is applied using random horizontal flipping, contrast adjustment, and brightness variation.

3.3 Training Strategy and Optimization↩︎

The model is trained with a batch size of 24 using a two-stage optimization process. In the first stage, the backbone networks are frozen, and only the newly added fusion, attention, and classification layers are trained using the Adam optimizer with a learning rate of \(10^{-3}\). In the second stage, the last 40 layers of each backbone network are selectively unfrozen and fine-tuned using a reduced learning rate of \(10^{-5}\).

3.4 Results and Analysis↩︎

DMFNet achieves strong and consistent classification performance on the AID dataset. Experimental evaluation over five independent runs yields an average classification accuracy of 97.46% \(\pm\) 0.14%, demonstrating stable optimization and robust generalization. In addition, the proposed framework achieves an average precision of 97.48% \(\pm\) 0.13%, recall of 97.46% \(\pm\) 0.14%, and F1-score of 97.46% \(\pm\) 0.14%, indicating balanced classification performance across diverse scene categories.

Table I compares the proposed framework with existing state-of-the-art methods on the AID dataset. Although GMFANet [23] achieves the highest reported accuracy which is attributed to there strong curriculum learning on a large pool of training dataset, the proposed framework delivers competitive performance. In addition, it demonstrates stable learning behavior and consistent predictions across multiple experimental runs with a low variance. This highlights its reliability for remote sensing scene classification.

3.5 Ablative Analysis↩︎

To evaluate the contribution of each proposed component, Table II presents the ablation study conducted on the proposed framework. M1 and M2 represent the single-backbone baselines based on ConvNeXt-Tiny and EfficientNet-B1, respectively. M3 incorporates multiscale residual feature fusion, M4 further integrates the spatial attention module, and M5 represents the complete framework with upsampling-based data augmentation. Table II confirms the effectiveness of each proposed component. Progressive integration of multiscale residual feature fusion, spatial attention, and upsampling-based data augmentation consistently improves classification accuracy, with M5 achieving the highest accuracy.

4 Conclusion↩︎

This article builds a dual-backbone multiscale feature fusion framework with residual feature propagation and spatial attention for remote sensing scene classification. By integrating ConvNeXt-Tiny and EfficientNet-B1, the proposed framework captures diverse feature representations and enhances inter-scale interaction and discriminative feature learning. Experiments on the AID dataset using a stratified 50:50 train–test split achieved an competitive average accuracy of 97.46% \(\pm\) 0.14%, with a precision of 97.48% \(\pm\) 0.13%, recall of 97.46% \(\pm\) 0.14%, and F1-score of 97.46% \(\pm\) 0.14%. Future work will focus on scaling the framework to larger datasets and enhancing computational efficiency. Specifically, we aim to explore lightweight architectures building upon our prior work [33], and investigate model compression techniques alongside the zero-shot learning paradigms in remote sensing established in [34] in the future with varying training [35] strategies.

---

1. This work was conducted by Anamitra Ghosh as part of her Master’s thesis at the Department of Computer Science and Engineering, Jadavpur University, Kolkata, India.↩︎