1 Introduction↩︎

Autonomous driving requires 360\(^\circ\) full-scene perception that jointly models foreground agents and surrounding scene layout. In this setting, 3D object detection estimates compact boxes for traffic participants [1]–[4], whereas semantic occupancy prediction recovers dense free, occupied, and semantic voxel states around the ego vehicle [5]–[7], MonoScene?, VoxFormer?. These two tasks describe the same scene at different granularities. Detection focuses on instance-level localization, while occupancy supplies the spatial support and layout context in which those instances exist. Therefore, reliable full-scene perception requires a unified treatment of object reasoning and scene layout.

Driven by recent datasets and radar-camera perception methods, 4D millimeter-wave radar is becoming an important sensing modality for autonomous driving [2]–[4], [8]–[12]. It provides stable geometric and motion cues under challenging conditions. However, its sparse returns still require camera semantics for complete scene understanding. Recent radar-camera methods therefore improve bird’s-eye-view fusion, semantic-geometric interaction, sparse object representation, and temporal modeling [2]–[4], [11], [12]. Nevertheless, most of these advances are either validated on front-view detection benchmarks such as View-of-Delft and TJ4DRadSet [8], [9], or remain primarily optimized for box-level perception in surround-view settings. This field gap limits how 4D radar evidence is used for full-scene perception, where object reasoning and scene layout should be evaluated under a unified radar-camera protocol.

Figure 1 further shows how representative 4D radar-camera systems have progressively expanded this scope, while a key limitation remains. SGDet3D [2] focuses on front-view detection. RCBEVDet [12] extends radar-camera detection to 360\(^\circ\) perception. Doracamom [13] further moves to joint detection and occupancy prediction. However, the representation form remains close to a shared BEV feature followed by a detection head and an occupancy branch.

Yet introducing occupancy only after the shared BEV representation is formed makes the bottleneck more than task coverage. What remains missing is a stateful representation that lets occupancy shape object reasoning before final decoding. Multi-task perception offers a relevant perspective: SOGDet uses occupancy context for detection, and UniVision studies a unified formulation for joint 3D perception [14], [15]. However, neither method is designed for 4D radar-camera multimodality. Existing joint radar-camera systems usually attach a detection head and an occupancy branch to a shared BEV feature [1], [13], leaving occupancy as a mostly terminal branch with limited influence on temporal representation and object reasoning. Evaluation is also concentrated on OmniHD-Scenes, so cross-dataset generality remains insufficiently examined. These limitations motivate a representation-level design that introduces occupancy as an intermediate scene state for 4D radar-camera multi-task perception. This state links radar-supported foreground geometry and motion cues with continuous scene layout, allowing dense occupancy evidence and temporal cues to enter the shared representation before final prediction.

To realize this idea, we propose 4DR360\(^\circ\), a 4D radar-camera framework for 360\(^\circ\) full-scene perception. 4DR360\(^\circ\) organizes the shared BEV representation through cross-modal state reasoning. First, it converts camera appearance and radar geometry into occupancy-aware state features before final decoding. It then applies State-guided BEV Enhancement (SBE) to refine the current BEV representation through state-guided deformable cross attention. Finally, Doppler-guided Temporal Fusion (DTF) uses ego alignment and motion-sensitive radar evidence to preserve reliable state history across frames. Through this design, occupancy state strengthens object-level features without replacing the detector representation. Beyond the model, we also extend the benchmark setting beyond OmniHD-Scenes, the main multiview 4D radar-camera dataset with joint detection and occupancy labels. Specifically, we construct occupancy annotations for ManTruckScenes from satellite-map priors and build a unified evaluation protocol across the two datasets. Together, these model and benchmark components lead to four contributions.

• We identify semantic occupancy as a persistent scene state for 4D radar-camera perception that unifies layout, temporal evidence, and object-level reasoning.

• We propose 4DR360\(^\circ\), a state-centered 4D radar-camera framework that performs cross-modal occupancy-state reasoning through SBE and DTF for feature refinement and temporal state modeling.

• We extend ManTruckScenes with satellite-map-based occupancy labels and establish a unified multi-task protocol for advanced 4D radar-camera perception.

• We define comprehensive experiments on accuracy, robustness, and efficiency, establishing a unified protocol and framework for dual-task 4D radar-camera perception and future radar-aware end-to-end or VLA research.

2 Related Work↩︎

2.1 4D Radar-Camera 3D Object Detection↩︎

4D radar-camera detection uses radar geometry and motion cues to strengthen camera perception under sparse or ambiguous observations. Benchmarks such as View-of-Delft, TJ4DRadSet, and K-Radar support this line of work [8]–[10]. Early methods, including CenterFusion, CRAFT, and RADIANT, attach radar evidence to image proposals or object centers [16]–[18]. Later BEV methods, including RCFusion, RCBEVDet, LXL, SGDet3D, and HGSFusion, lift both modalities into shared BEV features for scene-level fusion [2], [11], [12], [19], [20]. RaGS and R4Det further strengthen sparse representation, depth reasoning, and temporal modeling [3], [4]. Recent extensions also explore local-global radar detection, sparse-to-dense radar learning, raw radar-tensor fusion, radar-camera depth estimation, cross-view instance awareness, and collaborative perception [21]–[26].

These methods establish strong radar-camera detection baselines and mark the progression from front-view detection to 360\(^\circ\) perception. However, their objective remains box-centric: radar evidence is fused for detection, whereas dense scene structure is rarely maintained as a reusable state for joint object and occupancy reasoning.

2.2 Multi-Task Learning for Perception↩︎

Multi-task perception seeks shared representations that support multiple driving outputs, with semantic occupancy providing a dense description of scene layout. Camera BEV and occupancy baselines such as MonoScene, VoxFormer, SurroundOcc, PanoOcc, and M-CONet from OpenOccupancy recover voxelized layout from multi-view imagery [5], [27]–[29], MonoScene?, VoxFormer?. SOGDet and UniVision further show that occupancy context can support object reasoning [14], [15]. These methods motivate joint perception, but they do not target 4D radar-camera perception, where sparse, motion-aware radar returns are naturally sensitive to foreground objects.

Benchmark support for 4D radar-camera multi-task perception is much narrower. OmniHD-Scenes is currently the only public 4D radar-camera benchmark with both 3D detection and semantic occupancy annotations [7], and Doracamom provides a joint reference on this setting [13]. This leaves cross-dataset generality underexplored and ties future radar-aware end-to-end studies to one benchmark contract. We therefore extend ManTruckScenes with auditable occupancy labels and evaluate both datasets under one normalized radar-camera detection-and-occupancy protocol. Methodologically, we further differ from ordinary dual-head systems by propagating occupancy as an explicit scene state, together with Doppler motion cues, inside the shared representation rather than decoding it only at the output side.

3 Method↩︎

3.1 Overview↩︎

4DR360\(^\circ\) formulates 4D radar-camera perception as cross-modal state reasoning, as summarized in Fig. 2. Given synchronized multi-view images and 4D radar sweeps at time \(t\), Fig. 2 (a) constructs an image BEV through view transformation and fuses it with radar BEV using bidirectional deformable attention. The fused radar-camera BEV is then lifted into a coarse-to-fine voxel hierarchy, which carries the scene states refined in Fig. 2 (b). SBE injects current-frame occupancy structure, while DTF propagates temporally aligned state evidence with radar-motion cues. The detection and occupancy heads then read out from the shared state-refined representation, as shown in Fig. 2 (c). Implementation details are provided in the supplementary materials.

3.2 ManTruckScenes Occupancy Construction↩︎

A single dataset provides limited evidence for full-scene radar-camera evaluation. OmniHD-Scenes provides native multiview 4D radar-camera detection and occupancy labels, but it is currently the only benchmark of this type. ManTruckScenes offers a complementary commercial-vehicle platform, yet it contains detection annotations only. Evaluating it only as a detector benchmark would leave the occupancy part of radar-camera full-scene perception untested across datasets. We therefore extend ManTruckScenes with an auditable occupancy export.

As illustrated in Fig. 3, we first aggregate multi-frame LiDAR observations into a static support world, so occupied geometry is anchored by measured surfaces. Satellite segmentation is then used only for conservative static semantics around supported regions, while dynamic actors are inserted by object-local surface replay instead of dense box filling. The resulting labels are used only as training and evaluation targets, with fixed label mapping, voxel range, masks, temporal metadata, and metrics within each dataset. The supplementary material provides the full generation and quality-control details.

3.3 Feature Extractor↩︎

The feature extractor in Fig. 2 (a) builds the radar-camera BEV evidence used to initialize the state. We omit the batch dimension. Given synchronized images from \(N\) cameras, a ResNet-FPN encoder extracts \(\mathbf{F}^{2D}_t\in\mathbb{R}^{N\times C_i\times H_i\times W_i}\). A depth-aware view transformer produces the camera BEV feature, depth distribution, and image-view context: \[(\mathbf{B}^{c}_t,\mathbf{D}_t,\mathbf{F}^{ctx}_t) = \mathcal{P}_{view}(\mathbf{F}^{2D}_t,\Pi_t),\] where \(\Pi_t\) collects camera calibration and image augmentations. The outputs are image BEV \(\mathbf{B}^{c}_t\in\mathbb{R}^{C\times H\times W}\), depth probabilities \(\mathbf{D}_t\in\mathbb{R}^{N\times D\times H_i\times W_i}\), and multi-view context \(\mathbf{F}^{ctx}_t\in\mathbb{R}^{N\times C\times H_i\times W_i}\). Following [2], we also use a geometric projection of \(\mathbf{F}^{ctx}_t\) to strengthen the camera BEV.

The radar branch maps the synchronized 4D radar point set \(\mathcal{R}_t=\{(x,y,z,\mathbf{a})\}\) to \(\mathbf{B}^{r}_t\in\mathbb{R}^{C_r\times H\times W}\) with a pillarization encoder, where \(\mathbf{a}\) contains Doppler and other radar attributes. The pillarization details are provided in the supplementary materials. Image and radar BEV features are then fused by bidirectional deformable BEV attention following [12], producing \(\mathbf{B}^{rc}_t\). This shared BEV representation aligns camera layout cues with radar foreground evidence.

Finally, each pyramid level of \(\mathbf{B}^{rc}_t\) is channel-adapted and lifted along height by a learned height MLP: \[\mathbf{V}^{rc,s}_t = \operatorname{Reshape}\big( \mathcal{H}_{s}(\operatorname{Down}_{s}(\mathbf{B}^{rc}_t))\big),\] where \(\mathbf{V}^{rc,s}_t\in \mathbb{R}^{C_s\times Z_s\times H_s\times W_s}\). This produces multi-scale fused voxel features \(\{\mathbf{V}^{rc,s}_t\}_{s=0}^{S}\), which provide the substrate for the following state reasoning modules.

3.4 Cross-modal State Representation↩︎

At stage \(s\), 4DR360\(^\circ\) represents the scene with a voxel feature \(\mathbf{V}^{rc,s}_t\) and an occupancy-logit state \(\mathbf{O}^{s}_t\in\mathbb{R}^{H_s\times W_s\times Z_s\times C_{occ}}\). The empty channel defines a non-empty confidence map \(\mathcal{M}(\mathbf{O})=1-\operatorname{softmax}(\mathbf{O})_{[...,e]}\), where \(e\) denotes the empty class and \(\mathbf{O}\) always denotes logits. \(\mathcal{M}(\mathbf{O})\) is broadcast over channels on voxel features and height-pooled when applied to BEV features. This operator is the interface between occupancy prediction and feature reasoning, so the state is not a terminal side output. It is predicted before attention, resized across stages as a semantic prior, and converted back into voxel features for both final heads. We summarize the stage transition as \[(\mathbf{O}^{s}_t,\widetilde{\mathbf{V}}^{rc,s}_t) = \Phi^{s}_{state}(\mathbf{V}^{rc,s}_t,\mathbf{F}^{ctx}_t, \mathbf{D}_t,\Pi_t,\mathbf{O}^{s-1}_t),\] where \(\Phi^{s}_{state}\) includes state prediction, SBE, and occupancy-aware voxel recovery. SBE internally produces the stage BEV feature \(\mathbf{B}^{rc,s}_t\), which is lifted back into the recovered voxel state. The recovered \(\widetilde{\mathbf{V}}^{rc,s}_t\) is upsampled as residual state evidence for the next finer scale.

3.5 State-guided BEV Enhancement (SBE)↩︎

Given the state interface above, SBE converts the predicted occupancy state into current-frame spatial guidance for radar-camera scene understanding. It is the stage-wise spatial decoder in the upper panel of Fig. 4. At stage \(s\), it takes the voxel seed \(\mathbf{V}^{rc,s}_t\), image-view context \(\mathbf{F}^{ctx}_t\), depth distribution \(\mathbf{D}_t\), camera geometry \(\Pi_t\), and the previous coarser state \(\mathbf{O}^{s-1}_t\) when available. An occupancy predictor first estimates \(\mathbf{O}^{s}_t\), from which \(\mathcal{M}(\mathbf{O}^{s}_t)\) is obtained.

The predicted state guides two attention operations. State-guided self-attention uses non-empty confidence to bias BEV query offsets and weights, so foreground and uncertain occupied regions receive more modeling capacity. State-guided cross-attention then projects height anchors to camera views and uses depth consistency and non-empty confidence to weight the sampled context. For a BEV location \((x,y)\), let \(\mathcal{X}_{x,y}\) be its height anchors. The image value sampled at anchor \(\mathbf{x}\) and view \(j\) is \[\mathbf{g}_{j\mathbf{x}} = \mathbf{W}_{v} \mathtt{Bilinear}\big(\mathbf{F}^{ctx}_{t,j}, \mathcal{P}(\mathbf{x},\Pi_{t,j}) +\Delta\mathbf{u}_{j\mathbf{x}}\big),\] and the state-guided aggregation is \[\bar{\mathbf{b}}^{rc,s}_t(x,y) = \sum_{j=1}^{N} \sum_{\mathbf{x}\in\mathcal{X}_{x,y}} \alpha_{j\mathbf{x}}d_{j\mathbf{x}} \mathcal{M}(\mathbf{O}^{s}_t)(\mathbf{x})\, \mathbf{g}_{j\mathbf{x}},\] where \(\alpha_{j\mathbf{x}}\) is the deformable attention weight, \(d_{j\mathbf{x}}\) is obtained by interpolating \(\mathbf{D}_{t,j}\) at the projected pixel and anchor-depth bin, \(\Delta\mathbf{u}_{j\mathbf{x}}\) is the learned image-plane offset, and \(\mathcal{M}(\mathbf{O}^{s}_t)(\mathbf{x})\) provides a continuous non-empty confidence rather than a hard occupied-anchor mask.

1pt

1pt

The aggregated response is added to the stage BEV query and refined by a lightweight convolutional block, yielding \(\mathbf{B}^{rc,s}_t\). The BEV feature is then lifted back to the voxel lattice. Let \(\mathcal{L}_{s}(\mathbf{B}^{rc,s}_t)\) denote the height-lifted stage BEV feature. The recovered voxel state is \[\begin{align} \widetilde{\mathbf{V}}^{rc,s}_t = \mathbf{V}^{rc,s}_t &+\mathcal{L}_{s}(\mathbf{B}^{rc,s}_t) \odot \mathcal{M}(\mathbf{O}^{s}_t), \end{align}\] where the second term gives explicit non-empty weighting. The recovered voxel feature is upsampled and passed to the next finer stage. Multi-scale occupancy supervision is applied to these stage states, while detailed SBE variants are reported in the supplementary materials.

1pt

3.6 Doppler-guided Temporal Fusion (DTF)↩︎

DTF is the temporal interface of the final voxel state, as shown in the lower panel of Fig. 4. The lower panel implements a compact motion-conditioning path: detection features decode a BEV velocity map, the occupancy state indexes non-empty dynamic support, and radar Doppler corrects the motion cue before dynamic warping. Let \(\mathbf{V}_t\) be the final occupancy-state feature and \(\mathbf{O}_t\) be its occupancy logits. We use \(\mathcal{I}(\mathbf{O},\cdot)\) to denote state-indexed feature selection, which suppresses empty or irrelevant support before temporal retrieval. For a history horizon \(T_h\), DTF caches \(\mathcal{H}_t=\{(\mathbf{V}_{t-k},\mathbf{O}_{t-k}, \mathbf{T}_{t-k},\mathbf{U}_{t-k})\}_{k=1}^{T_h}\), where \(\mathbf{T}_{t-k}\) is the ego pose and \(\mathbf{U}_{t-k}\) is the BEV motion field derived from box velocity and radar Doppler cues. Its input-output contract is \[\begin{align} \widetilde{\mathbf{V}}_t = \mathcal{F}_{DTF}\big( \mathbf{V}_t,\mathcal{H}_t\big) \end{align}\] where DTF aligns historical state features and motion fields to the current ego frame.

DTF performs state-aware history retrieval in three steps. First, the state index \(\mathcal{I}(\mathbf{O}_{t-k}, \cdot)\) provides the non-empty voxel path in Fig. 4, and the selected historical features are ego-aligned: \[\widehat{\mathbf{V}}_{t,k}= \mathcal{T}_{t-k\rightarrow t}(\mathcal{I}(\mathbf{O}_{t-k}, \mathbf{V}_{t-k})).\] Second, the Doppler-corrected velocity map drives dynamic warping after ego alignment. The motion field \(\mathbf{U}_{t-k}\) follows the lower path in Fig. 4: detection features decode velocity, occupancy confidence selects dynamic foreground support, and radar Doppler corrects the motion cue. With interval \(\Delta t_k\), the displacement field is \[\begin{align} \Delta \mathbf{p}_{t,k} = \operatorname{clip}\big( \Delta t_k \mathcal{T}_{t-k\rightarrow t} (\mathcal{I}(\mathbf{O}_{t-k}, \mathbf{U}_{t-k})),\tau\big), \end{align}\] where \(\tau\) limits unrealistically large shifts. The ego-aligned state is sampled from the source location that moves into each current BEV cell: \[\bar{\mathbf{V}}_{t,k} = \mathcal{S}(\widehat{\mathbf{V}}_{t,k},-\Delta\mathbf{p}_{t,k}),\] where \(\mathcal{S}\) denotes BEV-plane bilinear sampling shared by all height bins. Indexing removes unsupported regions, but the reliability of the remaining memory still varies across voxels. We therefore derive \(\bar{\mathbf{C}}_{t,k}\) by applying the same ego alignment and dynamic warp to \(\mathcal{M}(\mathbf{O}_{t-k})\). Third, temporal fusion averages the warped memories with occupancy confidence and temporal decay: \[\mathbf{M}_t = \frac{\sum_{k=1}^{T_h}\lambda^k \bar{\mathbf{C}}_{t,k}\odot\bar{\mathbf{V}}_{t,k}}{\sum_{k=1}^{T_h}\lambda^k\bar{\mathbf{C}}_{t,k}+\epsilon}.\] The current state is then refined by a lightweight 3D convolutional update, \(\widetilde{\mathbf{V}}_t = \mathbf{V}_t+ \mathtt{Conv}_{3D}\big(\mathbf{M}_t\big).\) The same Doppler-aligned update is height-pooled and adapted to the detection BEV, so detection and occupancy receive a shared radar-guided temporal state. Compared with generic BEV temporal aggregation, DTF separates three measurable factors: temporal horizon, state-aware memory retrieval, and radar-conditioned motion compensation.

3.7 Head and Training Objective↩︎

The final heads read out the same refined state with task-specific adapters, corresponding to Fig. 2 (c). Detection projects the voxel state to full-resolution BEV and adds it to the adapted shared radar-camera BEV before CenterHead decoding. Occupancy decodes the voxel state to the full voxel grid, adds a height-lifted radar-camera BEV prior, and uses the final stage occupancy logits as a soft semantic prior before 3D classification. Further implementation details are provided in the supplementary materials.

Training supervises both terminal predictions and intermediate state. We therefore use a shared occupancy loss for the state branch and the final occupancy branch. At stage \(s\), the occupancy loss is \(\mathcal{L}^{s}_{occ} = \mathcal{L}^{s}_{focal} + \mathcal{L}^{s}_{geo} + \mathcal{L}^{s}_{sem},\) where the three terms denote weighted focal loss, geometric scaling loss, and semantic scaling loss. The overall objective is formulated as \[\begin{align} \mathcal{L} ={} \mathcal{L}_{det} + \lambda_{depth}\mathcal{L}_{depth} + \lambda\mathcal{L}^{S}_{occ} + \sum_i w_i\mathcal{L}^{i}_{occ}. \end{align}\] \(\mathcal{L}_{det}\) supervises the detection branch, \(\mathcal{L}^{S}_{occ}\) supervises the final occupancy output, and the weighted intermediate losses supervise state predictions at each stage. \(\mathcal{L}_{depth}\) is used when depth supervision is available.

0pt

4 Experiments↩︎

4.1 Experimental Setup↩︎

4.1.0.1 Datasets and Metrics.

We evaluate 4DR360\(^\circ\) on multi-view OmniHD-Scenes and ManTruckScenes. OmniHD-Scenes uses six cameras and six 4D radars and provides 4-class detection labels with 11-class semantic occupancy. ManTruckScenes provides a complementary commercial-vehicle setting with the generated 10-class occupied semantic export under four cameras setting. The masks, ignored regions, spatial range, and label mapping are fixed before training and shared by all methods. Detection is reported with mAP, dataset-specific overall score (ODS for OmniHD-Scenes and NDS for ManTruckScenes), mATE, mASE, mAOE, mAVE, per-class AP, and FPS when available. Occupancy is reported with SC IoU, mIoU, and per-class IoU. We also evaluate night and rainy OmniHD-Scenes subsets.

4.1.0.2 Implementation Details.

All methods are adapted under one MMDetection3D data contract with synchronized radar-camera inputs, temporal metadata, detection boxes, occupancy labels, masks, and voxel ranges. OmniHD-Scenes uses a \(160\times240\times16\) grid, while ManTruckScenes uses a \(216\times216\times16\) grid. Training uses AdamW. All ablations are built on the adapted RCBEVDet baseline.

0pt

4.2 3D Object Detection Results↩︎

Tables 1 and 2 show that the detection gain is not limited to one dataset. On OmniHD-Scenes, 4DR360\(^\circ\) improves over the strongest prior radar-camera row by 3.32 mAP and 3.70 ODS, with the largest class gain on large vehicles. On ManTruckScenes, the margin is larger: 4DR360\(^\circ\) exceeds HGSFusion by 5.67 mAP and 4.74 NDS, while improving the obstacle AP by 15.20 over the best baseline. The error metrics further localize the source of the gain. mATE and mAVE drop to 0.3613 and 0.9442 on ManTruckScenes, indicating better localization and motion estimation.

4.3 Semantic Occupancy Prediction Results↩︎

Tables ¿tbl:tab95omnihd95occ? and 3 show complementary occupancy behavior. On OmniHD-Scenes, 4DR360\(^\circ\) reaches best results, improving over Doracamom by 1.09 SC IoU and 2.90 mIoU. The gains are strongest on thin or foreground classes, while driveable surface and sidewalk also improves. On ManTruckScenes, 4DR360\(^\circ\) gives the best SC IoU and improves mIoU by +2.29 over HGSFusion.

4.4 Adverse Conditions↩︎

Table 4 evaluates night and rainy OmniHD-Scenes subsets. 4DR360\(^\circ\) reaches best performance, improving over Doracamom by 2.32, 2.83, and 2.87 points. These adverse scenes emphasize the regime where 4D radar is complementary to cameras, since radar returns remain sparse but stable when image appearance is degraded by illumination or weather.

4.5 Ablation Studies↩︎

Ablations are conducted on ManTruckScenes. Further details are reported in the supplementary materials. Detailed resource and efficiency comparisons are provided in the supplementary materials, showing that the accuracy gains do not come from a resource increase.

0pt

0pt

Component contribution. Table 5 shows that the first improvement appears as soon as occupancy is organized as a coarse-to-fine state path. This row adds intermediate state prediction, stage propagation, and occupancy-aware voxel recovery, bringing 3.40 mAP, 2.39 NDS, and 0.93 mIoU over the baseline. Then, SBE adds another 2.33 mAP, 1.65 NDS, and 0.60 mIoU, confirming that feeding the explicit state back to BEV matters for both boxes and voxels. DTF further adds 2.12 mAP, 2.08 NDS, and 1.43 mIoU, giving a total improvement of 7.85/6.12/2.96 over the baseline.

Temporal horizon. Table 6 shows that simply adding history is harmful: two-frame BEV-TF drops below the single-frame model by 0.47 mAP and 0.31 mIoU. DTF reverses this trend because it selects and aligns history through the learned state. Two-frame DTF improves over single-frame by 1.17 mAP and 0.52 mIoU, while four-frame DTF gives the best result, showing that temporal context is useful only when history is retrieved with state and motion guidance.

Doppler effectiveness. Table 7 makes the motion effect explicit. Ego alignment alone reduces mAVE from 1.0213 to 0.9986 but hurts mAP, NDS, and mIoU, showing that ego compensation cannot handle moving objects by itself. Explicit state memory recovers the metrics, and then Doppler gives the decisive motion correction: mAP, NDS, and mIoU further increase by 0.51, 0.69, and 0.73, while mAVE drops from 0.9824 to 0.9442.

5 Conclusion↩︎

In this work, we presented 4DR360\(^\circ\), a systematic 4D radar-camera framework for 360\(^\circ\) multi-view multi-task perception. Its main contribution is occupancy-state reasoning, which models semantic occupancy as a persistent scene state rather than a terminal output for radar-camera representation learning. SBE and DTF make this state useful within and across frames, allowing foreground evidence, dense layout, and radar motion to reinforce a common full-scene representation. This changes occupancy from an auxiliary prediction into the interface that couples object detection with comprehensive scene understanding. Experiments on OmniHD-Scenes and ManTruckScenes show consistent gains in detection and occupancy prediction, while the latter occupancy export provides a second benchmark contract for future 4D radar-camera multi-task studies. Future work will focus on radar-aware end-to-end or VLA research.

References↩︎