FSM-Net: An Efficient Frequency-Spatial Network for Real-World Deblurring
May 29, 2026
Abstract
Real-world image deblurring demands both high-fidelity restoration and computational efficiency, a balance existing methods often struggle to achieve. In this paper, we propose FSM-Net (Frequency-Spatial Multi-branch Network), a highly efficient solution that secured 2nd place in the NTIRE 2026 Challenge on Efficient Real-World Deblurring. FSM-Net pioneers a dual-domain approach: a novel Frequency Attention module explicitly recovers high-frequency structural details via FFT, while a Cross-Gated Vision E-Branchformer at the bottleneck captures global dependencies with linear complexity. To ensure robust convergence, we employ a progressive curriculum training strategy guided by a composite loss function (Multi-Scale Charbonnier, Structural Edge, and Frequency). Evaluated on the RSBlur benchmark, FSM-Net achieves an outstanding 33.144 dB PSNR with only 4.94M parameters and 159.35 GMACs (at 1920x1200 resolution). By effectively pushing the Pareto frontier of efficiency and quality, FSM-Net establishes a strong baseline for resource-constrained image restoration.
1 Introduction↩︎
Real-world image deblurring is a fundamental yet challenging task in computer vision, serving as a crucial preprocessing step for downstream applications like computational photography and autonomous driving [1], [2]. While early attempts heavily relied on perceptual losses [3] and Generative Adversarial Networks (GANs) [4], recent advancements have been dominated by Vision Transformers (ViTs) [5] and large-scale CNNs. Models such as SwinIR [6], IPT [7], and MAXIM [8] have pushed the boundaries of restoration quality. However, their massive parameter counts make them highly impractical for edge deployment.
The NTIRE 2026 Challenge on Efficient Real-World Deblurring [9] highlights a critical paradigm shift: modern networks must deliver high-fidelity restoration while strictly minimizing computational complexity. Existing architectures often struggle with this dilemma, either sacrificing local structural details or relying on heavy self-attention mechanisms that scale quadratically with image resolution [10], [11]. As illustrated by the Pareto frontier in Fig. 1, balancing global context and local features without severe computational bottlenecks remains a major hurdle.
To tackle this challenge, we propose FSM-Net (Frequency-Spatial Multi-branch Network), an ultra-lightweight architecture built upon NAFNet [11]. Recognizing that severe motion blur fundamentally acts as a low-pass filter, we introduce a novel FSMBlock equipped with Frequency Attention. By applying Fast Fourier Transform (FFT), our model explicitly suppresses noise and enhances sharp edges directly in the frequency domain. Furthermore, we integrate a Cross-Gated Vision E-Branchformer Block at the network’s bottleneck. This module processes information through two parallel branches: a local CNN for precise spatial detail recovery and a 4-head Transposed Attention branch for capturing global context with linear complexity.
Optimized via a robust composite loss, combining Multi-Scale Charbonnier Loss, Structural Edge Loss, and Frequency Consistency Loss, FSM-Net achieves an exceptionally lean profile. At a resolution of \(1920 \times 1200\), it requires only 4.94 M parameters and 159.35 GMacs, significantly pushing the Pareto frontier of efficient image restoration.
In summary, our main contributions are as follows:
We introduce FSM-Net, an ultra-efficient architecture that seamlessly integrates spatial and frequency domain processing to optimize the trade-off between restoration quality and computational complexity.
We design the novel FSMBlock with complex-valued Frequency Attention alongside a Cross-Gated Vision E-Branchformer Block, enabling the network to efficiently capture both global receptive fields and sharp local textures.
We demonstrate the superiority of FSM-Net by achieving 33.144 dB PSNR on the NTIRE 2026 public test set, securing 2nd place and establishing a new efficiency benchmark for resource-constrained platforms.
2 Related Work↩︎
2.1 Deep Learning for Image Deblurring↩︎
Image deblurring has witnessed a paradigm shift with the rapid evolution of deep neural networks. Early approaches heavily relied on multi-stage or multi-scale Convolutional Neural Networks (CNNs) to progressively restore sharp details. Notable architectures such as SAPHNet [12], XYDeblur [13] MIMO-UNet [14], MPRNet [15], and HINet [16] achieved impressive spatial restoration by leveraging multi-input multi-output designs, cross-stage feature fusion, and instance normalization. Recently, transformers like Stripformer [17] Vision Transformers (ViTs)[5] have dominated low-level vision tasks. Restormer [10] introduced a highly effective Multi-Dconv Head Transposed Attention (MDTA) to reduce the quadratic complexity of standard self-attention, while Uformer [18] utilized window-based attention for local feature extraction. Despite their high Peak Signal-to-Noise Ratio (PSNR) on benchmark datasets, these attention-heavy models suffer from massive parameter counts and substantial latency. Recognizing this computational bottleneck, Chen et al. proposed NAFNet [11], which replaced non-linear activation functions with simple element-wise multiplications (SimpleGate), achieving competitive performance with significantly reduced FLOPs. Our FSM-Net builds upon the lightweight principles of NAFNet but dramatically enhances its representational capacity through targeted cross-domain feature extraction mechanisms.
2.2 Frequency-Domain and Hybrid Representations↩︎
Severe motion blur in real-world scenarios inevitably corrupts high-frequency structural details, which are notoriously difficult to recover using purely spatial convolutions. To address this, recent studies have begun exploring the frequency domain for image restoration. Methods utilizing the Fast Fourier Transform (FFT), such as Fast Fourier Convolution (FFC)[19] and Fourier Domain Adaptation (FDA)[20], have demonstrated that modulating amplitude and phase can effectively suppress noise and reconstruct sharp edges while enabling robust cross-domain generalization [21], [22].[23]. Furthermore, the integration of local CNNs and global attention mechanisms has proven to be a highly effective hybrid strategy. Architectures such as the Conformer [24] and E-Branchformer [25] in the speech and vision domains demonstrate that processing features through parallel local and global branches yields superior context modeling. Inspired by these advancements, our method introduces the Cross-Gated Vision E-Branchformer block alongside a complex-valued Frequency Attention module, allowing the network to explicitly decouple and learn high-frequency blur degradation patterns without incurring heavy computational overhead.
2.3 Efficient Restoration and NTIRE Challenges↩︎
The NTIRE workshops have consistently driven the boundaries of image restoration through rigorous challenges [9], [26]–[29]. Historically, the pursuit of efficiency has leveraged lightweight convolutions [30] and structural re-parameterization[31]. Recent editions, including the 2025 Real-World Deblurring challenge[9], demonstrated remarkable inference speeds using highly optimized convolutional backbones. However, these spatial-only architectures often struggle with the long-range, non-uniform blur trajectories inherent in high-resolution imagery. To further push the Pareto frontier in the 2026 challenge, FSM-Net evolves this structural paradigm by integrating a hybrid frequency-spatial design. By decoupling spectral components via Frequency Attention and modeling global context with linear complexity, our model achieves superior restoration fidelity while maintaining the sub-second runtime requirements of mobile ISPs. This architectural advancement allows FSM-Net to handle complex degradations more robustly than prior lightweight spatial-only networks.
3 Methodology↩︎
To address the strict efficiency constraints of the NTIRE 2026 Challenge, we propose FSM-Net (Frequency-Spatial Multi-branch Network). Building upon the U-shaped NAFNet [11], we establish a Pareto balance between restoration quality and computational cost by completely redesigning the feature extraction paradigm to explicitly decouple spatial textures and frequency components. This section details the overall pipeline, core architectural blocks, hybrid loss formulation, and training dynamics.
3.1 Overall Architecture↩︎
FSM-Net adopts a 4-level encoder-decoder architecture with dense skip connections. Given a blurry image \(\mathbf{I}_{blur} \in \mathbb{R}^{3 \times H \times W}\), a \(3 \times 3\) convolutional stem projects it into a shallow feature space \(\mathbf{X}_0 \in \mathbb{R}^{C \times H \times W}\).
The encoder progressively downsamples features via strided convolutions (stride 2), doubling channels while halving spatial resolution to extract hierarchical representations. Both the encoder and decoder stages are exclusively populated with our proposed Frequency-Spatial Multi-branch Blocks (FSMBlock) to capture complex degradation patterns.
At the network bottleneck, where receptive fields are maximized but spatial dimensions are compressed, we introduce a sequence of Vision E-Branchformer Blocks to capture long-range semantic dependencies without inflating MACs. Finally, the decoder progressively upsamples deep features using PixelShuffle[32] to prevent checkerboard artifacts, culminating in a \(3 \times 3\) convolution to predict the clean residual. The final sharp image is obtained via a global skip connection: \(\mathbf{I}_{sharp} = \mathbf{I}_{blur} + \mathbf{R}_{pred}\).
3.2 Frequency-Spatial Multi-branch Block↩︎
Severe motion blur operates not only as a spatial distortion but fundamentally acts as a low-pass filter that severely degrades high-frequency structural details. Standard convolutions are inherently spatially biased and computationally inefficient for modeling such global frequency-domain degradations [33], [34]. Therefore, we design the FSMBlock, which augments the efficient SimpleGate mechanism [11], [35] with a novel Frequency Attention module.
Given an input feature map \(\mathbf{X}_{in}\), we first apply Layer Normalization[36] followed by a \(1 \times 1\) convolution for channel expansion. To capture long blur trajectories efficiently, we apply a depth-wise convolution with a large \(7 \times 7\) kernel. Let the resulting intermediate feature be \(\mathbf{X} \in \mathbb{R}^{C' \times H \times W}\). We split this feature along the channel dimension into two equal halves: \(\mathbf{X}_1\) and \(\mathbf{X}_2\).
To explicitly recover high-frequency information, \(\mathbf{X}_2\) is fed into the Frequency Attention module. We first transform \(\mathbf{X}_2\) into the frequency domain using the 2D Real Fast Fourier Transform (rFFT): \[\mathcal{F}(\mathbf{X}_2)(u, v) = \sum_{h=0}^{H-1} \sum_{w=0}^{W-1} \mathbf{X}_2(h, w) e^{-j 2\pi \left(\frac{uh}{H} + \frac{vw}{W}\right)}\] where \(j\) is the imaginary unit. To modulate both the amplitude (contrast) and phase (structure) of the signal simultaneously, we define a learnable complex-valued weight parameter \(\mathbf{W}_c \in \mathbb{C}^{1 \times C'/2 \times 1 \times 1}\). The frequency features are element-wise multiplied by this complex weight: \[\mathbf{Z}_{fft} = \mathcal{F}(\mathbf{X}_2) \odot \mathbf{W}_c\] The filtered complex signal is then transformed back to the spatial domain via the Inverse Fast Fourier Transform (irFFT), producing the enhanced feature \(\mathbf{\tilde{X}}_2 = \mathcal{F}^{-1}(\mathbf{Z}_{fft})\). Unlike real-valued spectral filters that only scale the magnitude, our complex-valued weight \(\mathbf{W}_c\) enables simultaneous modulation of both amplitude and phase. Since motion blur fundamentally induces a phase shift and frequency-dependent attenuation, this dual-modulation capability allows FSM-Net to explicitly re-align shifted structural components while amplifying suppressed high-frequency signals. This spectral recalibration is crucial for resolving the ghosting artifacts typically found in non-uniform real-world blur. Finally, the output of the Frequency Attention is fused with the identity spatial branch (\(\mathbf{X}_1\)) using a non-linear gating mechanism: \[\mathbf{X}_{gate} = \mathbf{X}_1 \odot \mathbf{\tilde{X}}_2\]
Following this frequency-spatial gating, we introduce a Simplified Channel Attention (SCA) module to recalibrate the channel-wise feature responses and enhance the representational power. We first squeeze the spatial dimensions of \(\mathbf{X}_{gate}\) using Global Average Pooling (GAP) to obtain a channel descriptor \(\mathbf{s} \in \mathbb{R}^{C'/2 \times 1 \times 1}\): \[\mathbf{s}_{c} = \frac{1}{H \times W} \sum_{h=1}^{H} \sum_{w=1}^{W} \mathbf{X}_{gate}(c, h, w)\] This global descriptor is then passed through a \(1 \times 1\) convolution to capture cross-channel interactions without any dimensionality reduction. These weights are used to dynamically modulate the gated features: \(\mathbf{X}_{sca} = \mathbf{X}_{gate} \odot \mathbf{W}_{sca}(\mathbf{s})\). Finally, \(\mathbf{X}_{sca}\) is projected back to the original channel dimension \(C\) via a \(1 \times 1\) convolution (\(\mathbf{W}_{out}\)), and added to the residual identity mapping to form the intermediate feature: \[\mathbf{X}_{mid} = \mathbf{X}_{in} + \mathbf{W}_{out}(\mathbf{X}_{sca}) \cdot \beta\] where \(\beta\) is a learnable layer scaling parameter.
Subsequently, \(\mathbf{X}_{mid}\) is processed by a Feed-Forward Network (FFN) augmented with a SimpleGate to independently model channel-wise transformations. The final block output is obtained via a second residual connection: \[\mathbf{X}_{out} = \mathbf{X}_{mid} + \text{FFN}(\mathbf{X}_{mid}) \cdot \gamma\] where \(\gamma\) is another learnable scaling factor. By directly modulating the complex wave representations and recalibrating channel dependencies, FSMBlock dynamically suppresses noise and sharpens edges with an \(\mathcal{O}(N \log N)\) computational complexity, strictly adhering to the challenge’s efficiency requirements.
3.3 Cross-Gated Vision E-Branchformer Bottleneck↩︎
At the network’s bottleneck, capturing global context is paramount for resolving severe, large-scale blur. However, standard Self-Attention mechanisms scale quadratically with spatial resolution \(\mathcal{O}(H^2W^2C)\), creating a severe computational bottleneck, while shifted-window variants [37] often lack sufficient receptive fields. To maintain the ultra-fast inference speed, we propose the Vision E-Branchformer Block, inspired by advancements in speech recognition [25], and adapt it for vision tasks through a novel cross-gating mechanism.
The block splits the normalized input into two parallel, complementary processing branches:
Local CNN Branch: Employs an inverted bottleneck structure to accurately model fine-grained spatial textures. It consists of a \(1 \times 1\) expansion convolution, a \(7 \times 7\) depth-wise convolution, a GELU activation[38], and a final \(1 \times 1\) projection convolution, yielding local features \(\mathbf{F}_{loc}\).
Global Attention Branch: Utilizes a 4-head Transposed Attention module. To inject local inductive bias into the attention mechanism, queries \(\mathbf{Q}\), keys \(\mathbf{K}\), and values \(\mathbf{V} \in \mathbb{R}^{B \times \hat{C} \times HW}\) are first generated using a \(3 \times 3\) depth-wise convolution. Instead of computing attention across the massive spatial dimension, we compute the covariance matrix across the channel dimension, inspired by Restormer[10]. Before the matrix multiplication, \(\mathbf{Q}\) and \(\mathbf{K}\) are \(\mathcal{L}_2\)-normalized along the spatial dimension. The attention is computed as: \[\mathbf{F}_{glob} = \text{Softmax}\left( \hat{\mathbf{Q}} \hat{\mathbf{K}}^T \cdot \tau \right) \mathbf{V}\] where \(\hat{\mathbf{Q}}\) and \(\hat{\mathbf{K}}\) represent the normalized queries and keys, and \(\tau\) is a learnable temperature parameter. This transposed formulation reduces the complexity from \(\mathcal{O}(H^2W^2C)\) to \(\mathcal{O}(HWC^2)\), making global context modeling extremely lightweight.
To synergize the representations from both domains, we introduce a Cross-Gating Mechanism. The features from one branch are utilized to generate attention gates for the other, facilitating mutual information exchange: \[\mathbf{F}'_{loc} = \mathbf{F}_{loc} \odot \sigma(\mathbf{F}_{glob})\] \[\mathbf{F}'_{glob} = \mathbf{F}_{glob} \odot \sigma(\mathbf{F}_{loc})\] where \(\sigma(\cdot)\) denotes the Sigmoid activation function. The mutually enhanced features are concatenated, projected via a \(1 \times 1\) convolution, and refined through a SimpleGate module. Finally, the output is scaled by a learnable layer scaling parameter before being added to the residual stream.
3.4 Optimization Strategy↩︎
To optimize FSM-Net for complex real-world degradations, we employ a composite loss function that supervises the network across spatial, structural, and frequency domains. The total objective function \(\mathcal{L}_{total}\) is formulated as a weighted linear combination of three distinct loss components: \[\mathcal{L}_{total} = w_1 \mathcal{L}_{MSC} + w_2 \mathcal{L}_{Edge} + w_3 \mathcal{L}_{FFT}\] where \(w_1, w_2,\) and \(w_3\) are hyperparameters that balance the contribution of each domain during the training process.
3.4.0.1 Multi-Scale Charbonnier Loss (\(\mathcal{L}_{MSC}\))
To capture hierarchical dependencies and enforce spatial fidelity, we utilize a Multi-Scale Charbonnier Loss [39]. Computed across three scales \(s \in \{1, 0.5, 0.25\}\) via average pooling on the predicted (\(\mathbf{\hat{I}}\)) and ground-truth (\(\mathbf{I}\)) images, the loss is defined as: \[\mathcal{L}_{MSC} = \sum_{s} \frac{\gamma_s}{N_s} \sum_{i=1}^{N_s} \sqrt{(\mathbf{\hat{I}}_{s, i} - \mathbf{I}_{s, i})^2 + \epsilon^2}\] where \(\gamma_s\) denotes the scale-specific weight parameter, \(N_s\) represents the total number of pixels at spatial scale \(s\), \(i\) is the spatial pixel index, and \(\epsilon = 10^{-12}\) is a small constant ensuring numerical stability. This hierarchical multi-scale supervision effectively encourages the network to restore both coarse global structures and fine-grained local textures.
3.4.0.2 Structural Edge Loss (\(\mathcal{L}_{Edge}\))
To explicitly enhance the sharpness of recovered boundaries, we introduce an Edge Loss based on the Laplacian operator \(\Delta\)[40]. We minimize the Mean Squared Error (MSE) between the extracted high-frequency edges: \[\mathcal{L}_{Edge} = \mathbb{E} \left[ \left\| \Delta \mathbf{\hat{I}} - \Delta \mathbf{I} \right\|_2^2 \right]\] By penalizing blurry edges, this component forces the model to prioritize sharp transitions and structural integrity in the restoration process.
3.4.0.3 Frequency Consistency Loss (\(\mathcal{L}_{FFT}\))
Motivated by the fact that motion blur is fundamentally a frequency-domain degradation[22], we penalize discrepancies in the Fourier space. We apply a Real Fast Fourier Transform (rFFT) to the spatial signals and minimize the \(\mathcal{L}_1\) distance between their log-amplitudes: \[\mathcal{L}_{FFT} = \mathbb{E} \left[ \left\| \log(1 + |\mathcal{F}(\mathbf{\hat{I}})|) - \log(1 + |\mathcal{F}(\mathbf{I})|) \right\|_1 \right]\] where \(\mathcal{F}(\cdot)\) denotes the 2D rFFT operator. The logarithmic scaling ensures that the network remains sensitive to subtle differences in high-frequency spectral components, which are crucial for recovering sharp details in real-world blurry images.
3.5 Exponential Moving Average↩︎
To ensure training stability and improve generalization without incurring test-time overhead, we integrate Exponential Moving Average (EMA)[41]. EMA acts as a temporal ensemble, smoothing noisy gradient updates. Let \(\theta^{(t)}\) be the model weights at training step \(t\), the EMA weights \(\theta_{EMA}^{(t)}\) are updated as:
\[\theta_{EMA}^{(t)} = \alpha \theta_{EMA}^{(t-1)} + (1 - \alpha) \theta^{(t)}\]
where the decay rate \(\alpha\) is empirically set to 0.999. This mechanism acts as a low-pass filter in the parameter space, dampening weight oscillations and guiding optimization toward wider, flat minima. Consequently, EMA significantly enhances the model’s robustness against domain shifts, ensuring stable and high-fidelity restoration on unseen test distributions.
4 Experiments↩︎
In this section, we provide a comprehensive evaluation of the proposed FSM-Net. We describe the dataset used for the challenge, the intricate multi-stage training details, and present our results in comparison with state-of-the-art methods.
4.1 Dataset and Evaluation Metrics↩︎
4.1.0.1 RSBlur Dataset:
We conduct our experiments on the RSBlur dataset [42], the official dataset for the NTIRE 2026 Efficient Real-World Deblurring Challenge. To bridge the domain gap between synthetic and real-world degradations, RSBlur employs a sophisticated blur synthesis pipeline. Instead of relying on naive artificial blur kernels, it utilizes real-world sharp image sequences and accurately models the physical camera image formation process, incorporating factors such as sensor noise, saturation, and signal processing pipelines, to synthesize highly realistic blurry/sharp training pairs. The dataset contains diverse indoor and outdoor scenes with complex motion blur patterns. For our training, we utilize the provided training split, consisting of high-resolution image pairs, and evaluate our performance on the full test set.
4.1.0.2 Evaluation Metrics:
To quantitatively assess the restoration quality, we utilize the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [43] and Learned Perceptual Image Patch Similarity (LPIPS)[44]. Furthermore, following the challenge mandates, we measure the Runtime per image on a standard GPU to ensure the efficiency of our model. All metrics are calculated in the RGB color space.
4.2 Implementation Details↩︎
4.2.0.1 Network Configuration:
The proposed FSM-Net is configured with a base channel width \(C=16\). The encoder-decoder depths are set to \([1, 1, 1, 24]\) and \([1, 1, 1, 1]\), respectively, with the number of middle blocks Vision E-Branchformer set to 2 at the bottleneck. This asymmetric design prioritizes deep feature extraction at lower resolutions to maintain a small computational footprint.
4.2.0.2 Optimizer and Scheduler:
We employ the AdamW optimizer[45] with \(\beta_1=0.9, \beta_2=0.999\) and a weight decay of \(10^{-4}\). The initial learning rate is set to \(10^{-4}\) and is decayed to \(10^{-6}\) using a Cosine Annealing scheduler[46]. To
ensure numerical stability and memory efficiency, we utilize Automatic Mixed Precision (AMP) with float16 during the entire training process. Furthermore, an Exponential Moving Average (EMA) with a decay rate of \(0.999\) is applied to the model weights to stabilize convergence and enhance generalization.
4.2.0.3 Multi-stage Curriculum Training:
We implement a 5-phase progressive training strategy over 400 epochs to facilitate convergence from coarse structures to fine textures:
Phase 1 (Epochs 0-20): We utilize a crop size of \(512 \times 512\) and optimize solely with the Multi-Scale Charbonnier Loss (\(w_1=1.0, w_2=0, w_3=0\)) to establish a stable initial spatial mapping.
Phase 2 (Epochs 20-120): Maintaining the \(512 \times 512\) resolution, we activate the structural and frequency losses (\(w_2=0.01, w_3=0.005\)) to begin capturing high-frequency details.
Phase 3 (Epochs 120-220): The crop size is increased to \(1024 \times 1024\) to allow the model to learn long-range motion blur trajectories while keeping the loss configuration constant.
Phase 4 (Epochs 220-290): We transition to a \(1280 \times 720\) resolution and increase the loss weights to \(w_2=0.05\) and \(w_3=0.01\) for more aggressive edge and frequency refinement.
Phase 5 (Epochs 290-400): In the final stage, we train on the near-original resolution of \(1920 \times 1200\) until convergence, ensuring the model is fully adapted to the high-resolution inference requirements of the challenge.
The entire training is conducted on a system equipped with a single NVIDIA RTX 5090 GPU. We utilize a batch size of 4 for Phases 1 through 4. In Phase 5, to accommodate the near-original resolution (\(1920 \times 1200\)) within the VRAM constraints while maintaining high-fidelity feature maps, the batch size is adjusted to 2.
4.3 Main Results↩︎
We evaluated FSM-Net on the NTIRE 2026 Efficient Real-World Deblurring Public Test Set, which features complex degradations to test restoration limits under strict computational constraints. As summarized in Table 1, FSM-Net secured 2nd place overall with a PSNR of \(33.144\) dB and an SSIM of \(0.8516\). This validates our core hypothesis: integrating the Frequency Attention module and Cross-Gated Vision E-Branchformer significantly enhances representational capacity without computationally prohibitive network scaling.
Crucially, FSM-Net establishes a highly practical Pareto balance between restoration quality and computational cost. Achieving its peak performance with an optimized latency of just \(0.276\)s, our model operates significantly faster than other top-ranking submissions reporting runtimes (e.g., Rank 8 at \(0.78\)s). This sub-second efficiency highlights its strong potential for real-time edge deployment.
Furthermore, FSM-Net outperforms the defending 2025 challenge champion (licheng, ranked 4th this year with a PSNR of \(32.805\) dB) by a substantial margin of +0.339 dB. This leap underscores the superiority of our dual-domain processing paradigm, demonstrating that our frequency-spatial decoupling effectively breaks the performance ceilings of prior spatial-only architectures.
3pt
| Rank | Team Name | PSNR\(\uparrow\) | SSIM\(\uparrow\) | Time (s)\(\downarrow\) |
|---|---|---|---|---|
| 1 | jingjing et al. | 33.390 | 0.8585 | N/A* |
| 2 | RobinLy (Ours) | 33.144 | 0.8516 | 0.276 |
| 3 | weichow | 32.883 | 0.8467 | N/A* |
| 4 | licheng | 32.805 | 0.8473 | N/A* |
| 5 | zzhlttcyy | 32.657 | 0.8450 | N/A* |
| 6 | zhouzhaorun | 32.593 | 0.8422 | N/A* |
| 7 | berryzis | 32.322 | 0.8394 | N/A* |
| 8 | geometric_ai | 32.216 | 0.8371 | 0.78 |
| Method | Param (M)\(\downarrow\) | Dataset Performance (PSNR \(\uparrow\) / SSIM \(\uparrow\)) | |||
|---|---|---|---|---|---|
| 3-6 | RSBlur | RealBlur-R | RealBlur-J | GoPro | |
| SRN-Deblur [47] | 3.76 | 32.53 / 0.8398 | 38.65 / 0.9652 | 31.38 / 0.9091 | 28.36 / 0.9150 |
| NAFNet-C16-L28[11] | 4.35 | 32.42 / 0.8400 | - | - | - |
| DeblurGAN-v2 [48] | 6.09 | - | 36.44 / 0.9347 | 29.69 / 0.8703 | - |
| MiMO-UNet [14] | 6.10 | 32.73 / 0.8457 | - | - | - |
| MiMO-UNet+ [14] | 16.1 | 33.37 / 0.8560 | - | - | - |
| MPRNet [15] | 20.1 | 33.61 / 0.8614 | - | - | 30.96 / 0.9390 |
| Nah [1] | 22.6 | - | 32.51 / 0.8406 | 27.87 / 0.8274 | 29.08 / 0.9135 |
| Restormer [10] | 26.1 | 33.69 / 0.8628 | - | - | 31.22 / 0.9420 |
| Uformer-B [18] | 50.9 | 33.98 / 0.8660 | - | - | 30.83 / 0.9520 |
| FSM-Net (Ours) | 4.94 | 33.16 / 0.8533 | 36.95 / 0.9585 | 29.45 / 0.8840 | 30.60 / 0.9068 |
4.3.0.1 Comprehensive SOTA Benchmark and Cross-Dataset Generalization
To rigorously evaluate FSM-Net, we present a comprehensive benchmark against state-of-the-art (SOTA) deblurring methods. This evaluation is twofold: assessing restoration efficacy on the official RSBlur public full test set (3,360 real-world image pairs) and analyzing cross-domain generalization on three diverse external datasets (RealBlur-R, RealBlur-J, and GoPro).
As detailed in Table 2, FSM-Net achieves high-fidelity restoration on the RSBlur benchmark with a PSNR of 33.16 dB and an SSIM of 0.8533. It is crucial to emphasize that these metrics are obtained using a direct, single forward pass without the aid of any Test-Time Augmentation (TTA). Notably, FSM-Net delivers this competitive, unaugmented performance while requiring a mere fraction of the computational footprint (4.94M parameters) compared to heavy SOTA architectures like MPRNet[15] or Uformer-B[18].
Furthermore, to validate cross-domain adaptability, we initialized FSM-Net with our best RSBlur pre-trained weights and performed a lightweight fine-tuning of merely 5 epochs on the external benchmarks. Despite this minimal adaptation phase, FSM-Net exhibits rapid convergence and robust generalizability. On the real-world RealBlur-R and RealBlur-J[2] datasets, our method achieves 36.95 dB and 29.45 dB respectively, remaining highly competitive with baselines specifically trained on those domains. Simultaneously, its stable performance on the synthetic GoPro dataset underscores that the representational capacity established by our frequency-spatial decoupling paradigm transfers effectively to unseen motion blur trajectories without severe overfitting.
4.3.0.2 Visual Performance Analysis
Figure 3 presents a qualitative evaluation on the NTIRE 2026 test set, demonstrating FSM-Net’s capability to mitigate severe motion artifacts. Specifically, our Frequency Attention module accurately reconstructs dense high-frequency details, such as legible text on the "Mr.Pizza" signage and sharp license plates, by explicitly eliminating ghosting artifacts. For large-scale geometries subjected to long blur trajectories (e.g., playground equipment and the waterfront monument), the Cross-Gated Vision E-Branchformer successfully restores structural integrity without warping. Furthermore, our Multi-Scale Charbonnier Loss ensures background regions like the sky and foliage remain perceptually smooth, preventing the over-sharpening halos common in traditional methods. These qualitative improvements directly corroborate our quantitative success (2nd place, PSNR: 33.144 dB, SSIM: 0.8516).
1.5pt
| Model | FAttn | Ebranch | Param (M) | MACs (G) | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) |
|---|---|---|---|---|---|---|---|
| Baseline | 4.35 | 146.33 | 31.39 | 0.8175 | 0.3557 | ||
| + FAttn | ✔ | 4.66 | 155.66 | 31.57 | 0.8217 | 0.3494 | |
| + Ebranch | ✔ | 4.93 | 159.35 | 31.60 | 0.8216 | 0.3478 | |
| FSM-Net | ✔ | ✔ | 4.94 | 159.35 | 31.83 | 0.8282 | 0.3456 |
4.4 Ablation Study↩︎
4.4.0.1 Architectural Components Ablation:
To rigorously evaluate the individual contributions of our proposed modules under strict computational constraints, we conducted an accelerated 20-epoch ablation study at high resolution (\(1920 \times 1200\)) using the 3,360 sample full test set. As shown in Table 3, the spatial-only baseline (NAFNet)[11] establishes a foundation at 31.39 dB PSNR. Integrating the Frequency Attention (FAttn) module yields a noticeable improvement to 31.57 dB, demonstrating the necessity of explicit spectral recalibration for mitigating high-frequency motion artifacts. Alternatively, deploying the Vision E-Branchformer (Ebranch) enhances spatial long-range dependency modeling, elevating the performance to 31.60 dB.
Crucially, when both pathways are synergized in the full FSM-Net, the restoration quality peaks at 31.83 dB with a highly competitive LPIPS of 0.3456. Notably, this combined improvement (+0.44 dB over the baseline) strictly exceeds the sum of their individual marginal gains (+0.18 dB and +0.21 dB). This non-linear performance boost empirically proves that our frequency and spatial modules are highly complementary, successfully capturing orthogonal degradation features rather than redundant information. From an efficiency standpoint, this substantial perceptual and quantitative leap incurs only a marginal computational overhead (+0.59 M parameters, +13.02 GMacs). This firmly validates that our dual-domain design fundamentally expands the network’s representational capacity without violating the strict boundaries of efficient deployment.
2.5pt
| TTA | Transforms | PSNR\(\uparrow\) | SSIM\(\uparrow\) | Time (s)\(\downarrow\) |
|---|---|---|---|---|
| \(\times 1\) | None | 33.006 | 0.8500 | 0.069 |
| \(\times 2\) | + H-Flip | 33.105 | 0.8510 | 0.138 |
| \(\times 3\) | + V-Flip | 33.135 | 0.8515 | 0.207 |
| \(\times 4\) | + D-Flip | 33.144 | 0.8516 | 0.276 |
4.4.0.2 Test-Time Augmentation (TTA) Trade-offs:
To maximize restoration quality within the sub-second constraint, we evaluate four progressive TTA configurations (Table 4). The baseline (TTA \(\times 1\)) achieves an ultra-fast 0.069s latency with a competitive 33.006 dB PSNR. Progressively incorporating horizontal (TTA \(\times 2\)) and vertical (TTA \(\times 3\)) flips yields consistent PSNR gains. For the full ensemble (TTA \(\times 4\), including a diagonal flip), we implement a weighted blending strategy (0.4 for the original, 0.2 for each augmentation) rather than a naive mean. This preserves the original spatial cues, achieving a peak PSNR of 33.144 dB at 0.276s. Ultimately, this demonstrates that FSM-Net can dynamically adapt to strict latency budgets (0.07s–0.28s) without any architectural modifications.
5 Conclusion↩︎
In this paper, we presented FSM-Net, an ultra-efficient frequency-spatial multi-branch architecture tailored for real-world image deblurring. By synergizing spatial-domain convolutions with complex-valued Frequency Attention, our model establishes a new Pareto frontier, achieving a superior balance between high-fidelity restoration and low computational overhead. With only 4.94 M parameters and a lean complexity of 159.35 GMacs at \(1920 \times 1200\) resolution, FSM-Net addresses the critical requirements for high-resolution processing in resource-constrained environments. The proposed method demonstrated outstanding performance during the NTIRE 2026 challenge, currently securing 2nd place on the public leaderboard with a PSNR of 33.144 dB. FSM-Net thus establishes a strong, efficient baseline for future research in resource-constrained high-fidelity image restoration.