1 Introduction↩︎

Multispectral object detection (MOD) improves localization and recognition by jointly exploiting spatial structure and spectral cues. Compared with RGB imagery, multispectral images (MSIs) measure spectral responses at sampled wavelengths, yielding per-pixel spectral signatures with strong inter-band correlation. Prior methods either decouple spectral and spatial information through dimensionality reduction (e.g., PCA [1] and band selection [2], [3]) or rely on attention-based modeling to capture spectral–spatial interactions [4]. Although these methods achieve promising performance, two challenges remain in MOD: (i) discrete spectral processing that neglects the continuous and ordered structure of the spectral dimension; (ii) a scale-dependent shift in the relative reliability between spectral and spatial cues across pyramid levels, which can degrade fusion performance in complex scenes.

Rotation equivariance is a desirable geometric inductive bias that exploits the intrinsic rotational symmetry and arbitrary orientations of objects, improving representation stability and generalization across diverse visual tasks [5], [6], including ground-level [7], [8] and aerial [9], [10] object detection as well as semantic segmentation [11], yet many multispectral detectors still rely on heavy rotation-based data augmentation in practice. However, extending this inductive bias to MOD requires rethinking the modeling paradigm. In particular, the continuous, ordered spectral structure of MSIs is mismatched with most equivariant designs built on discrete sampling (Fig. 1 (a)). Moreover, reliability-aware pyramid fusion under equivariant architectures is non-trivial in complex scenes. Furthermore, group-indexed interactions increase the computational burden. Therefore, a key challenge in MOD is to explicitly model spectral continuity and enable reliable spectral–spatial fusion across scales, while enforcing rotation-equivariant geometric priors with manageable overhead.

To address the challenges of spectral discreteness, scale-dependent fusion reliability, and the lack of rotation-equivariant geometric priors, we propose FressDet, the first fully rotation-equivariant spectral–spatial learning framework for multispectral object detection. FressDet captures continuous spectral structure, enables reliability-aware pyramid fusion across levels, and stably predicts oriented objects without parameter replication.

To overcome the discreteness of spectral processing, we draw on the universal approximation theorem [12], [13]: implicit neural representations (INRs) parameterize signals as coordinate-conditioned neural functions, enabling approximation of continuous functions with arbitrarily small error. Accordingly, we introduce Spectral Implicit Warp (SpeIW), an efficient spectral operator that models multispectral representations as a coordinate-conditioned function queried along the spectral dimension with Fourier-encoded spectral coordinates [14], [15]. This encoding mitigates the spectral bias of coordinate-based networks, enabling the representation of higher-frequency spectral variations. Specifically, we parameterize spectral warping over rotation-equivariant features via a factorization into coordinate-dependent bases and feature-conditioned coefficients. The bases encode regularities along the spectral dimension, while the coefficients provide pixel-wise modulation of the warping offsets. By predicting strictly positive spectral steps and integrating them cumulatively, we obtain an order-preserving mapping by construction.

Addressing scale-dependent reliability shifts across pyramid levels, we propose Rotation-Equivariant Consistency Weighting (ReCoW), a lightweight refinement module with a dual prototype-driven routing mechanism that adaptively selects the more reliable branch. On the spectral branch, ReCoW performs aggregation with soft assignment to capture discriminative spectral patterns. On the spatial branch, it improves geometric coherence, reducing background clutter while preserving fine-grained texture details. A learned branch-selection gate finally produces a convex combination of the two branches. Overall, ReCoW improves spectral–spatial fusion across pyramid levels in complex scenes.

For arbitrarily oriented objects, we propose an oriented-aware head that exploits feature maps with explicit orientation indices, enabling stable prediction across orientations without parameter replication. This design enhances feature utilization and substantially reduces the parameter count.

Taken together, these components enable FressDet to learn more discriminative and robust spectral–spatial representations. Extensive experiments on three public benchmarks demonstrate that FressDet achieves state-of-the-art accuracy with 93% fewer parameters than the previous best method (Fig. 1 (c)) and the best robustness even under input rotational perturbations (Fig. 1 (b)). Our contributions are summarized as follows:

• Present FressDet, the first fully rotation-equivariant spectral–spatial learning framework unifying continuous spectral modeling, reliability-aware pyramid fusion, and stable orientation-aware prediction.

• Introduce Spectral Implicit Warp for continuous, order-preserving spectral resampling via a coordinate-conditioned implicit field.

• Propose Rotation-Equivariant Consistency Weighting, which performs adaptive, reliability-aware spectral–spatial fusion across pyramid levels.

• Design an oriented-aware head for stably predicting arbitrarily oriented objects without parameter replication.

2 Related Work↩︎

Multispectral imagery provides spectral cues beyond RGB data, facilitating more discriminative object representations. Existing methods either separate spectral and spatial streams with cross-branch fusion [16], [17] or integrate them in a unified design [4]. However, most treat spectral features independently, leaving the continuous and ordered nature of spectral structure insufficiently modeled, while scale-dependent shifts in spectral–spatial reliability across pyramid levels remain unaddressed. In contrast, our method explicitly models spectral continuity and enables reliability-aware spectral–spatial fusion across pyramid levels.

Since Cohen  [5] introduced G-CNNs, rotation-equivariant networks have shown strong performance across diverse vision tasks. In object detection, ReDet [10] pioneered rotation-equivariant aerial detectors, and subsequent works [8], [9] advanced equivariant architectures. However, these methods are only approximately equivariant. Lee  [18] present FRED, the first fully rotation-equivariant detector with deformable convolutions, though its group-indexed interactions incur considerable computational overhead. We develop the first fully rotation-equivariant spectral–spatial learning framework for multispectral object detection, integrating spatial geometry with discriminative spectral cues while maintaining manageable overhead.

Implicit neural representations (INRs) parameterize signals as coordinate-to-value neural fields, achieving success in 3D scene modeling [19] and continuous image representation [20], [21]. In spectral imaging, INRs have been applied to spectral rendering [22] and multispectral–hyperspectral fusion [23]. However, prior INR methods primarily address reconstruction or fusion tasks, whereas multispectral detection requires modeling spectral continuity under a strict order-preserving constraint. Our method addresses this by introducing an order-preserving spectral warping mechanism that maintains rotation equivariance.

3 Method↩︎

In this section, we introduce the preliminaries on rotation equivariance and describe how the components of FressDet are assembled into a fully rotation-equivariant pipeline (Fig. 2).

3.1 Preliminaries↩︎

We consider equivariance to discrete in-plane rotations within the Euclidean group \(\mathrm{E}(2)=\mathbb{R}^{2}\rtimes \mathrm{O}(2)\), the semi-direct product of translations and orthogonal transformations. By restricting to the cyclic subgroup \(\mathrm{C}_N=\{r_k \mid k=0,\dots,N{-}1\}\) of \(N\) equally spaced rotations, we sample features at \(N\) discrete orientations. We denote this group by \(G\) (\(|G|=N\)) in the sequel. A function \(\Phi\colon X\!\to\!Y\) is equivariant with respect to \(G\) if: \[\Phi\!\bigl(\rho_{g}^{X}(x)\bigr) = \rho_{g}^{Y}\!\bigl(\Phi(x)\bigr), \quad \forall\, g\in G,\; x\in X, \label{eq:equiv}\tag{1}\] where \(\rho_{g}^{X}\) and \(\rho_{g}^{Y}\) denote group actions on \(X\) and \(Y\), respectively. Intuitively, equivariance ensures that transforming the input produces a correspondingly transformed output, enabling the network to generalize across orientations without relying on explicit data augmentation. When \(\rho_{g}^{Y}\) is the identity for all \(g\), equivariance reduces to invariance.

Under an input rotation \(r_k\), a group feature map \(F\colon\mathbb{R}^{2}\!\times\!G\to\mathbb{R}^{C}\) transforms as follows: \[\bigl[\rho_{r_k}(F)\bigr](x,\,g) = F\bigl(r_k^{-1}x,\; r_k^{-1}g\bigr), \label{eq:regular}\tag{2}\] concretely, a cyclic permutation by \(k\) along the group axis composed with a rotation by \(r_k^{-1}\). Following this representation, the planar input \(\mathbf{x}_{0}\!\in\!\mathbb{R}^{c\times h\times w}\) is lifted to a group feature map as follows: \[\mathbf{y}(g) = \sum_{i=1}^{c} \mathbf{x}_{0,i} \ast \mathcal{R}_{g}(\mathbf{W}_{i}), \quad g \in \{0,1,\dots,N{-}1\}, \label{eq:lift}\tag{3}\] where \(\mathbf{W}\!\in\!\mathbb{R}^{c' \times c \times s \times s}\) is a learnable weight matrix with kernel size \(s\) and \(\mathcal{R}_{g}\) denotes spatial rotation by \(2\pi g/N\). For subsequent layers, group-to-group convolutions take the form: \[\mathbf{y}'(g) = \sum_{i=1}^{c} \sum_{g'=0}^{N-1} \mathbf{y}_{i}(g') \ast \mathcal{R}_{g}\!\bigl(\mathbf{W}_{i,\,g^{-1}g'}\bigr), \quad g \in \{0,1,\dots,N{-}1\}, \label{eq:gconv}\tag{4}\] where the relative index \(g^{-1}g'\) induces a cyclic shift along the group axis. Group convolutions preserve this transformation law, so equivariance propagates through successive layers. The \(\mathcal{O}(c^{2}Ns^{2})\) cost of Eq. 4 is prohibitive for real-time detection. We decompose it into two stages that jointly preserve Eq. 2 .

Stage: Depthwise rotated spatial filtering. Each channel \(i\) is convolved with a single spatial kernel \(\mathbf{W}^{\mathrm{dw}}_{i}\!\in\!\mathbb{R}^{s\times s}\), rotated per group element: \[\tilde{F}_{i}(x,g) = F_{i}(x,g) \ast \mathcal{R}_{g}\!\bigl(\mathbf{W}^{\mathrm{dw}}_{i}\bigr), \label{eq:dw}\tag{5}\] preserving the cyclic-shift law of Eq. 2 channel-wise at \(\mathcal{O}(cNs^{2})\) cost.

Stage: Pointwise group mixing. A \(1\!\times\!1\) convolution mixes channels and group indices via a weight \(\mathbf{W}^{\mathrm{pw}}\!\in\!\mathbb{R}^{c'\times c\times N}\) whose group axis is cyclically shifted: \[F'_{j}(x,g) = \sum_{i=1}^{c}\sum_{g'=0}^{N-1} W^{\mathrm{pw}}_{j,i,\,g^{-1}g'}\;\tilde{F}_{i}(x,g'), \label{eq:pw}\tag{6}\] at \(\mathcal{O}(c^{2}N)\) per spatial location. The cyclic indexing \(g^{-1}g'\) guarantees that the composition of Eqs. 5 –6 satisfies Eq. 2 end-to-end with cost \(\mathcal{O}(cNs^{2}+c^{2}N)\).

3.2 Spectral Implicit Warp↩︎

While the lifting layer endows features with rotational structure, the spectral dimension remains discretized into independent channels. SpeIW addresses this by predicting a monotone warp index \(\phi\) for each spectral channel and resampling via differentiable interpolation (Fig. 2 (a)), enabling adaptive spectral modeling while preserving equivariance.

Let \(F\in\mathbb{R}^{C\times|G|\times H\times W}\) denote a group feature map with \(C\) spectral channels and \(|G|\) orientation slots. Each spectral channel is assigned a normalized spectral coordinate \(\lambda_c\in[-1,1]\), lifted via Fourier positional encoding \(\psi(\lambda)=\bigl[\sin(2^{i}\pi\lambda),\,\cos(2^{i}\pi\lambda)\bigr]_{i=0}^{L-1}\!\in\mathbb{R}^{2L}\) to mitigate the frequency bias of coordinate-based networks [14], [15]. We decompose the raw warp field into a spatial latent code \(\mathbf{z}(x,g)\!\in\!\mathbb{R}^{R}\), derived from \(F\) via a learned equivariant projection, and a spectral basis decoded by a coordinate-conditioned implicit function \(f_{\theta}\colon\mathbb{R}^{2L}\!\to\!\mathbb{R}^{R}\): \[\delta_{c}(x,g) = \bigl\langle\, \mathbf{z}(x,g),\;\; f_{\theta}\!\bigl(\psi(\lambda_{c})\bigr) \,\bigr\rangle, \quad c=1,\dots,C. \label{eq:lowrank}\tag{7}\] Because \(f_{\theta}\) is shared across all \(C\) spectral channels and takes continuous spectral coordinates as input, it parameterizes a spectral basis that can be queried at arbitrary coordinates, with parameter cost that does not scale with \(C\).

Directly using \(\delta_c(x,g)\) as resampling indices may cause index crossing (i.e., \(\phi_c(x,g)\ge \phi_{c+1}(x,g)\)), resulting in non-monotonic spectral mappings and invalid resampling. We therefore enforce monotonically ordered indices \(\{\phi_1(x,g)<\phi_2(x,g)<\cdots<\phi_C(x,g)\}\) to preserve the channel order. We obtain nonnegative increments via softplus, and enforce strict ordering by cumulative summation followed by normalization: \[\tag{8} \begin{gather} \Delta_{c}(x,g) \triangleq \sigma_{+}\!\bigl(\delta_{c}(x,g)\bigr), \tag{9}\\ \phi_{c}(x,g) = (C{-}1)\cdot\frac{\sum_{j=2}^{c}\Delta_{j}(x,g)}{\sum_{j=2}^{C}\Delta_{j}(x,g)}, \tag{10} \end{gather}\] where \(\sigma_{+}\) denotes the softplus function. Each \(\Delta_{c}(x,g)\) serves as a positive increment along the spectral dimension, so \(\phi_{c}(x,g)\) is strictly increasing w.r.t.\(c\) by construction. This yields a valid, non-crossing resampling function and prevents degenerate channel permutations.

Given the monotonically ordered indices \(\{\phi_c(x,g)\}_{c=1}^C\), \(\phi_c(x,g)\in[0,C{-}1]\), we obtain the warped feature via differentiable linear interpolation along the spectral dimension: \[\hat{F}_{c}(x,g) = \mathcal{I}\!\bigl[\,F(x,g,\cdot)\,;\;\phi_{c}(x,g)\,\bigr], \label{eq:spectral95field}\tag{11}\] where \(\mathcal{I}[\,\cdot\,;\,t\,]\) denotes a differentiable linear interpolation operator evaluated at continuous index \(t\). Since \(\mathbf{z}(x,g)\) is rotation-equivariant and \(f_{\theta}\) is shared across orientations, the predicted indices \(\phi_{c}(x,g)\) are equivariant as well. Since the interpolation operates only along the spectral dimension and is applied independently for each \((x,g)\), it commutes with the group action.

3.3 Rotation-Equivariant Consistency Weighting↩︎

Multi-scale fusion in feature pyramids aggregates representations across pyramid levels, which is essential for scale robustness. However, repeated cross-level aggregation can wash out discriminative spectral evidence and reduce class separability for spectrally similar categories. ReCoW addresses this issue (Fig. 2 (b)) by combining a spectral branch that preserves discriminative spectral patterns with a spatial branch that enforces geometric coherence, and fusing them with a learned branch-selection gate. This design maintains this balance during pyramid fusion with provable equivariance guarantees.

Both branches operate on local windows of size \(W\!\times\!W\). A rotation-invariant descriptor \(\bar{F}(x)=\frac{1}{|G|}\sum_g F(x,g)\) is first obtained by group averaging. Within each window, we form \(K\!=\!q^{2}\) prototype centers \(\{\mathbf{c}_k\}_{k=1}^{K}\) by adaptive average pooling of \(\bar{F}\) to a \(q\!\times\!q\) spatial grid. The grid preserves local spatial layout and provides region-specific centers, where \(x\) indexes locations within the window.

The two branches adopt complementary routing rules. Spectral signatures often exhibit continuous mixing; thus, we employ cosine-based soft assignment to preserve spectral diversity: \[\tag{12} \begin{gather} s^{\mathrm{spec}}_{k}(x) = \tau\cdot\cos\!\bigl(\mathbf{c}_k,\,\bar{F}(x)\bigr), \tag{13}\\ p_k(x) = \frac{\exp\!\bigl(s^{\mathrm{spec}}_{k}(x)\bigr)}{\sum_{k'}\exp\!\bigl(s^{\mathrm{spec}}_{k'}(x)\bigr)}, \tag{14} \end{gather}\] where \(\tau\) is a learnable temperature. The routed spectral representation is obtained by weighted aggregation of prototype centers: \[\hat{F}_{\mathrm{spec}}(x) = \sum_{k=1}^{K} p_k(x)\,\mathbf{c}_k . \label{eq:spec95agg}\tag{15}\] We share \(\hat{F}_{\mathrm{spec}}(x)\) across \(g\) to obtain the refinement \(F_{\mathrm{spec}}(x,g)\) for all orientations, preserving equivariance.

Spatial grouping benefits from crisp region assignments; accordingly, we perform hard routing on a rotation-invariant spatial score map. We compute a scalar score \(s(x)=\frac{1}{C|G|}\sum_{c,g}F_c(x,g)\) and obtain scalar prototype centers \(\{\mu_k\}_{k=1}^{K}\) by adaptive average pooling of \(s\) onto the same \(q\!\times\!q\) grid. Hard assignment is obtained via a Gaussian affinity: \[\tag{16} \begin{gather} a_k(x) = \exp\!\bigl(-\gamma\,\|\mu_k-s(x)\|_2^2\bigr), \tag{17}\\ k^{*}(x) = \argmax_{k} a_k(x), \tag{18} \end{gather}\] where \(\gamma\) is a learnable inverse bandwidth that controls the sharpness of the spatial affinity. Given \(k^{*}(x)\), the spatial branch aggregates the original group features by cluster mean within each window: \[\hat{F}_{\mathrm{spat}}(x,g) = \frac{1}{\lvert \Omega_{k^*(x)} \rvert}\sum_{x'\in \Omega_{k^*(x)}} F(x',g), \label{eq:spat95agg}\tag{19}\] where \(\Omega_{k}\) denotes the set of locations assigned to prototype \(k\) in the window. The assignment is shared across group indices \(g\!\in\!G\), while aggregation is performed separately per group element to obtain the refinement \(F_{\mathrm{spat}}(x,g)\).

Let \(F_{\mathrm{spec}}\) and \(F_{\mathrm{spat}}\) denote the routed outputs of the spectral and spatial branches. Rather than fixed-weight fusion, we learn a position-wise gate that adaptively balances their contributions. We obtain \(\bar{F}_{\mathrm{spec}}\) and \(\bar{F}_{\mathrm{spat}}\) by group averaging and project them into a shared \(d\)-dimensional space using lightweight projections \(\pi_s\) and \(\pi_t\). The branch-selection gate is: \[\beta(x) = \sigma\!\Bigl(\bigl\langle \pi_s\bigl(\bar{F}_{\mathrm{spec}}(x)\bigr),\; \pi_t\bigl(\bar{F}_{\mathrm{spat}}(x)\bigr) \bigr\rangle\Bigr) \in [0,1], \label{eq:gate}\tag{20}\] where \(\sigma\) is the sigmoid function and \(\langle\cdot,\cdot\rangle\) denotes the channel-wise dot product. Intuitively, \(\beta(x)\) quantifies the relative reliability of the spectral branch at each spatial location. The fused output is a convex combination: \[F'(x,g) = \beta(x)\cdot F_{\mathrm{spec}}(x,g) + \bigl(1-\beta(x)\bigr)\cdot F_{\mathrm{spat}}(x,g). \label{eq:convex}\tag{21}\] This dot product measures agreement between the two branches in a learned subspace. Since \(\beta\) is computed from descriptors averaged over the group and applied consistently across \(g\!\in\!G\), the fusion operation is equivariance preserving.

3.4 Oriented-Aware Head Network↩︎

Detection requires category-score fields that are invariant along the orientation axis while remaining spatially equivariant, whereas box and angle predictions must transform consistently with the input. Let \(F_{\!\mathrm{n}}\!\in\!\mathbb{R}^{C\times|G|\times H\times W}\) denote the group feature from the neck and \(\rho_g\) the group action defined in Eq. 2 . We design the three branches of the head to satisfy (Fig. 2 (c)): \[\tag{22} \begin{gather} f_{\mathrm{cls}}(\rho_g\,F_{\!\mathrm{n}})(x) \;=\; f_{\mathrm{cls}}(F_{\!\mathrm{n}})(g^{-1}x), \tag{23}\\[2pt] f_{\mathrm{box}}(\rho_g\,F_{\!\mathrm{n}}) \;=\; \rho_g\,f_{\mathrm{box}}(F_{\!\mathrm{n}}), \tag{24}\\[2pt] f_{\mathrm{ang}}(\rho_g\,F_{\!\mathrm{n}})(x) \;=\; \operatorname{wrap}_{\pi}\!\left( f_{\mathrm{ang}}(F_{\!\mathrm{n}})(g^{-1}x) + \tfrac{2\pi}{|G|}\,\operatorname{idx}(g)\right), \tag{25} \end{gather}\] where \(\operatorname{idx}(g)\) returns the integer index of \(g\) in \(G\), and \(\operatorname{wrap}_{\pi}\) denotes the \(\pi\)-periodic OBB angle normalization.

All three branches are built from the depthwise-pointwise equivariant convolutions of Eqs. 5 –6 with shared parameters across orientations, keeping the head compact. The classification branch achieves invariance via group mean pooling at the readout stage: \[\label{eq:group95pool} \mathbf{z}_{\mathrm{inv}} \;=\; \frac{1}{|G|}\sum_{g\in G}\mathbf{z}_g\,,\tag{26}\] where \(\mathbf{z}_g\) is the feature response at orientation \(g\); since group pooling removes only the orientation index while preserving the transformed spatial coordinate, Eq. 23 follows directly. The box branch keeps group-indexed features and uses a cyclic-tied DFL readout whose weights depend on relative group offsets, while the angle branch predicts group-indexed residual candidates and decodes them by soft circular aggregation. These readouts preserve the equivariant box-side transformation and the \(\pi\)-periodic angle shift law in Eqs. 24 –25 .

4 Experiments↩︎

We evaluate FressDet on three multispectral object detection benchmarks: MODA [4], HOD3K [17], and DroneVehicle [24]. These datasets span diverse spectral configurations, object scales, and imaging conditions, forming a challenging evaluation suite. All results are reported with the standard COCO AP metric [25].

4.1 Implementation Details↩︎

The backbone employs four stages, each equipped with Fourier positional encoding of \(L\!=\!8\) frequencies and an implicit MLP of depth 5 with ReLU activations. The neck applies ReCoW at each fusion level with local window size \(W\!=\!4\) and \(K\!=\!16\) prototypes arranged on a \(4\!\times\!4\) spatial grid. The rotation group is constructed on \(C_4\) in all experiments unless otherwise specified.

FressDet is trained from scratch without data augmentation, as the built-in rotation equivariance provides sufficient geometric robustness. The framework is built on Ultralytics YOLO [26] with default settings, expanding the first convolution to match the number of MSI bands. All experiments are conducted in PyTorch on two NVIDIA RTX 3090 GPUs.

0.7pt

4.2 Comparison with State-of-the-Art Methods↩︎

Tab. 1 reports results on MODA. FressDet outperforms OSSDet by 4.1% mAP\(_{50}\), 14.3% mAP\(_{75}\), and 11.6% mAP, with only 6.3% of OSSDet’s parameters and 12.0% of its FLOPs, and achieves large gains on small classes (bike +16.7%, pedestrian +26.8%). Consistently, Fig. 3 demonstrates fewer misses and false positives under clutter and low visibility. Fig. 5 (a) further verifies component effectiveness: SpeIW yields more localized activations on target regions through continuous spectral resampling, ReCoW adaptively enhances the reliable branch and suppresses the weaker one, reinforcing target responses while attenuating background noise. Fig. 5 (b) shows that continuous spectral modeling enlarges inter-class margins (e.g., car vs. van) and forms tighter clusters for small classes (bike, pedestrian), improving discriminability.

Tab. [tab:hod3k] shows FressDet surpasses the previous best OSSDet by 0.2% in mAP while achieving the best mAP\(_{50}\) with only 6.3% of OSSDet’s parameters and 26.8% of its FLOPs. Fig. 4 shows qualitative results on HOD3K. Competing methods often miss occluded or background-blended objects, whereas FressDet enhances target–background separation, yielding predictions closer to the ground truth.

As shown in Tab. 2, our method surpasses the best [50] by 1.1% mAP and 0.5% mAP\(_{50}\) under identical settings. FressDet achieves the best Car and Bus mAP\(_{50}\), leading by 1.4% and 1.3% over the best competing methods in each category. FressDet benefits from the representational continuity that naturally emerges along the spectral dimension.

7.2pt

4.3 Ablation Studies and Analysis↩︎

All ablations are conducted on MODA under identical settings unless noted.

As shown in Tab. ¿tbl:tab:ablation95key?, introducing rotation-equivariant group convolutions alone yields a +4.4% mAP gain over the baseline, confirming that geometric priors are as critical as spectral modeling for multispectral detection. Each subsequent module provides improvements, and the full model achieves the best result across all metrics. Among the three proposed modules, SpeIW contributes the most, underscoring the value of continuous spectral modeling in multispectral object detection.

We ablate the spectral operator by replacing SpeIW with rotation-equivariant DCN variants [63]–[66] (implementation details in the supplementary material). Tab. [tbl:tab:speiw95compare] shows that the plain C\(_4\)-CNN performs worst, underscoring the value of adaptive spectral resampling. SpeIW surpasses all DCN-based alternatives, outperforming the closest DCN variant by 1.1 mAP, indicating that order-preserving continuous resampling provides a stronger inductive bias than discrete deformable offsets.

To evaluate SpeIWNet’s generalization capability, we replace the backbones of five detectors while keeping other components and training unchanged. Fig. 6 shows consistent performance gains across single- and two-stage architectures, along with reduced model size, demonstrating SpeIWNet’s effectiveness as a strong plug-and-play feature extractor across heterogeneous detectors.

We compare \(C_2\), \(C_4\), and \(C_8\) groups in Tab. [tab:group95order]. Higher group order improves accuracy at the cost of increased parameters and FLOPs. \(C_4\) achieves the best balance between accuracy and efficiency: \(C_8\) incurs more than \(2\times\) FLOPs for only marginal improvement, suggesting that \(90^\circ\) discretization already captures the dominant orientation variations.

We ablate the Fourier positional encoding frequency \(L\) in SpeIW (Tab. [tab:fourier95freq]). Higher \(L\) enables better high-frequency modeling along the spectral dimension, but excessive frequencies may lead to overfitting. \(L\!=\!8\) achieves the best performance.

Tab. ¿tbl:tab:spectral95sensitivity? compares RGB/MSI inputs across MODA detectors. MSI improves every method, with FressDet showing the largest gain (+2.7 mAP\(_{50}\) and +3.7 mAP); this supports continuous spectral modeling as a better way to exploit the available bands more fully.

Tab. [tbl:tab:routing95mechanism] shows that routing should match the branch property. Soft spectral routing preserves diversity, while hard spatial routing keeps sharper boundaries; the mismatched alternatives lose one of these cues.

We ablate the spatial grid size used to generate prototypes in ReCoW (Tab. [tbl:tab:recog95prototypes]), which shows that smaller grids lack sufficient spatial resolution to capture diverse local patterns, while larger grids introduce redundancy and overfitting, and the \(4\!\times\!4\) grid (\(K\!=\!16\)) achieves the best balance.

We ablate the fusion strategy for combining the spectral and spatial branches in ReCoW (Tab. [tbl:tab:recog95fusion]). Our learned gate outperforms all alternatives, confirming the benefit of adaptive branch selection over fixed or implicit fusion. Against addition, concatenation, and softmax fusion, the gate raises mAP by 1.5, 0.9, and 0.5 points, respectively.

We ablate each ReCoW branch (Tab. [tbl:tab:recog95branch]). The spectral and spatial branches achieve 72.3% and 71.8% mAP\(_{50}\), respectively. Their combination reaches 73.1%, confirming that the two branches capture complementary information and yield consistent improvements when combined.

To quantify how well rotation equivariance is preserved, we measure the equivariance error at each backbone stage, defined as the RMS difference between rotating the input before and the output after the network [9]: Fig. [fig:equiv95error] shows FressDet maintains low equivariance error across all stages, while OSSDet exhibits consistently higher errors.

5 Conclusion↩︎

We present FressDet, the first fully rotation-equivariant spectral–spatial learning framework for multispectral object detection. FressDet addresses discrete spectral processing, spectral–spatial reliability shifts, and arbitrary orientations through three components: SpeIW for continuous, monotonic spectral resampling, ReCoW for reliable spectral–spatial fusion across pyramid levels, and an oriented-aware head for stable orientation prediction without parameter replication. Extensive experiments on three public benchmarks demonstrate that FressDet achieves state-of-the-art accuracy with the fewest parameters among compared multispectral detectors and exhibits the best robustness even under input rotational perturbations.

Acknowledgements↩︎

This work was financially supported by the National Natural Science Foundation of China (No. 62101032), the Chongqing Excellent Young Scientists Fund (No. CSTB2025NSCQ-JQX0017), and the High-Quality Development Special Project of the Ministry of Industry and Information Technology (TC240HAJ9-35).

References↩︎