1 Introduction↩︎

The EU General Safety Regulation [1] has made advanced driver-assistance systems mandatory across all new vehicle categories, while recent UN regulations for automated systems (UN R152, R157 and R171) [2]–[4] now accept virtual testing as valid homologation evidence, on the condition that the virtual results are validated against physical tests at the same facility. A testing facility that couples a real vehicle to a credible simulation, rather than a pure simulator, is therefore precisely what the regulatory framework requires.

Hybrid testing is also a practical necessity. Driving scenarios, including CP scenarios, may require more connected agents than an organization physically owns, and the mix of real and virtual participants varies from experiment to experiment with the available hardware, like the number of instrumented vehicles, road side units, extra traffic agents (connected or not connected) etc. By substituting virtual agents for unavailable real ones, a hybrid platform can exercise multi-agent scenarios that would otherwise be infeasible to stage physically, at a virtual/real ratio chosen per experiment.

This paper describes such a facility as deployed infrastructure. Our contribution is threefold:

• A hybrid ViL platform that couples a real instrumented vehicle to a CARLA digital twin (DT) through an ETSI-compliant Cooperative Awareness Messages (CAM)/Collective Perception Messages (CPM) pipeline [5], [6] extended from the CARLA ROS bridge.

• A real-world demonstrator on our institution’s public-road-representative test track, in which three real vehicles (two sharing CPM) are reflected in the DT and consumed by a GPU-accelerated occupancy-grid module.

• A characterisation of the CP workload across nominal, rain and night conditions, together with a statement of the platform’s current limits.

We deliberately separate what is demonstrated functionally on the real vehicle from what is quantified in simulation; closing that gap with a quantitative sim-to-real fidelity metric is ongoing work.

2 Platform Architecture↩︎

The platform comprises three groups of components (Fig. 3): real assets, virtual assets and the ROS-based perception modules that run across both.

Real assets. The core real asset is a fully instrumented vehicle carrying GNSS and an IMU for localization (supporting both standard GPS and differential GPS (D-GPS), the latter providing centimetre-level positioning usable as ground truth), a Velodyne VLP-16 LiDAR and a forward-facing RGB camera for perception, a NUC-class edge PC for on-board object detection, and an Nfiniity Cube EVK OnBoard Unit (OBU) providing C-V2X and ITS-G5 connectivity. The platform further supports additional connected vehicles and roadside units (RSUs) as real assets. Connected assets transmit their kinematic state as CAM and their detections as CPM [5], [6], both triggered every 100 ms. While the platform is currently tested in a V2X CP configuration, the ViL coupling itself is agnostic to the communication layer; assets and the DT can equally be linked over a direct vehicle-to-cloud connection. Finally, the platform also supports a driver-in-the-loop configuration, in which a human drives the real vehicle while the DT and its virtual agents react in real time, closing the loop through the human as well as the sensors.

Virtual assets. The DT is not merely a drivable map but a complete model of the road environment, its static and dynamic assets, and the simulated agents that populate it, all kept consistent with the real world. A CARLA [7] instance of this environment ingests the live stream through a custom middleware layer that extends the CARLA ROS bridge, parsing ETSI messages into DT-compatible objects. Virtual agents participate in V2X through the VaN3Twin framework (ns-3 + OpenCDA) [8], [9], so that simulated and real entities both contribute to the shared perception. The site is reconstructed through a repeatable pipeline (Fig. 1): the road network is extracted from OpenStreetMap, enriched, and converted into an OpenDRIVE (.xodr) logical map encoding lanes, junctions and connectivity, and imported into CARLA as a newly created unreal map (.umap file). The same toolchain reconstructs any OpenStreetMap-covered site as a DT, which is what makes the Mediterranean-ODD scenario library of Section 6 tractable to scale beyond a single location. Validity also depends on the DT reproducing the features that drive sensor and perception behaviour, so the default CARLA asset set is augmented with site- and vehicle-specific models (Fig. 2): the real vehicle is reconstructed as a calibrated 3D mesh whose geometry and on-board sensor placement are modelled on the physical platform; the site’s buildings, terrain and vegetation are modelled to reproduce real occlusions and sight-lines; condition-specific features such as retroreflective cat’s-eye road studs recreate night-time appearance cues. These assets directly underpin the multi-condition behaviour of Section 4: the night results, in particular, are only meaningful if the DT renders the visual environment a vehicle would encounter after dark.

ROS-based perception modules. The platform hosts its perception as ROS modules: a single-view camera–LiDAR perception module and the cooperative-perception “Occupancy” module (containerised; ROS2; Google JAX with NVIDIA CUDA via XLA JIT), which fuses incoming CAM/CPM into a standardised nav_msgs/msg/OccupancyGrid – a probabilistic grid whose cells encode occupancy likelihood (occupied, free, or uncertain), regenerated for each batch of messages. Because both are ROS nodes consuming standardised messages, they deploy unchanged on the real vehicle or against the DT through the CARLA-ROS bridge; this paper evaluates the CP module (Section 4).

3 Real-World Demonstrator↩︎

We exercised the full chain on our institution’s test track using a double T-intersection scenario. Three real vehicles operated on the track (two connected and sharing CPM and a third, CAM-only vehicle broadcasting its position but not perception data) while the same scenario was reconstructed in the DT. Fig. 4 places, side by side, drone footage of the real intersection, its CARLA reconstruction, and the occupancy grid produced from the live messages, including a variation in which a virtual agent augments the real ones.

This demonstrator validates the platform functionally: real CAM/CPM traffic is ingested, reflected in the DT and turned into a coherent occupancy grid end to end. We emphasise its scope honestly. Because the test-track vehicle cannot be synchronised frame-exactly with the simulation, repeated runs of the same initial conditions are required, and the perception metrics in Section 4 are computed in the DT with controlled localization noise rather than against real-vehicle ground truth. The demonstrator is thus a proof of concept for the hybrid methodology; quantitative sim-to-real fidelity is the subject of follow-up work.

The platform is not a one-off demonstrator but shared infrastructure already exercised across EU projects. In EVENTS [10], it supports collective perception evaluation and the scaling-up of cooperative scenarios. In SUNRISE [11], the same hybrid setup is used for hybrid (virtual-physical) testing and the logging of key performance indicators feeding CCAM safety argumentation (video available on request). ² The same CARLA-based simulation toolchain, including its driver-in-the-loop configuration, also underpins scenario-data generation in SYNERGIES [12], for the project’s federated scenario dataspace.

4 CP Workload and Multi-Condition Behaviour↩︎

The platform itself produces no metric in isolation; we therefore test it through the CP module it runs. The derived occupancy-grid is evaluated, using the KPIs as a demo for what the platform is capable of, in the DT under three environmental conditions (nominal, rain, night), three cooperative configurations (ego-only, three connected agents, all six agents), and five localization-noise levels (0, 0.5, 1.0, 1.5 and 2.0 meters). All metrics are reported as area-under-curve (AUC) over each scenario.

Fig. 5 summarises the headline KPIs, from which three findings stand out. First, joint FoV coverage rises sharply with cooperation, from roughly one quarter of the area of interest for a single vehicle to the majority of it with all agents (Fig. 5a) and, notably, stays stable across the full localization-noise range, indicating that probabilistic fusion preserves spatial awareness even under positioning uncertainty. Second, occupied-cell recall improves with cooperation at low noise but all configurations (except ego-only) converge near the same value at 2 meters (Fig. 5b): beyond a moderate noise threshold, localization uncertainty (not the number of agents or the weather) becomes the dominant error source. Third, the cooperative benefit is largest exactly when individual sensing is weakest, i.e., under night conditions, where adding agents recovers occupied-cell recall from a degraded single-vehicle level back toward its nominal value (shaded region in Fig. 5b). Occupied-cell precision shows the expected slight decrease as agents are added (overlapping noisy observations), an acceptable trade-off in a safety setting where missed obstacles matter more than conservative ones; free-space estimation remains reliable across all conditions.

5 Current Limitations and Engineering Targets↩︎

With regard to the platform’s current limitations, the coupled pipeline executes on a fixed-interval cycle, where each cycle ingests the live CAM/CPM stream, updates the DT and regenerates the occupancy grid. Still, strict determinism cannot be guaranteed in the current configuration, because transient slowdowns of the simulation loop introduce timing irregularities that can alter the temporal alignment between real-vehicle message arrivals and simulation state updates. This is an architectural property of the CARLA-based co-simulation rather than a hardware limit.

With regard to the engineering targets, the first one is raising the loop rate (the frequency of the complete ingest-update-regenerate cycle) and bounding its jitter (the worst-case deviation of the cycle interval), since bounded timing is a precondition for run-to-run reproducibility in a testing context. We are pursuing this through a co-simulation scheduler that advances all components on a common clock and by isolating the grid generator from the simulator’s rendering loop. The second target is a GPU-versus-CPU benchmark of the occupancy module itself, to quantify the contribution of the JAX/CUDA implementation. The planned sim-to-real fidelity assessment builds directly on this: a D-GPS reference trajectory provides the positioning ground truth against which the DT’s synthetic localization noise can be replaced by measured real-vehicle error. Planned extensions broaden the loop further: remote operation of the real vehicle and an immersive driver interface via VR headset for in-the-loop scenario testing.

6 Toward a Mediterranean-ODD Testing Service↩︎

No hybrid ViL facility currently operates in southeastern Europe, to our knowledge. Moreover, existing European facilities are optimised for established homologation procedures rather than hybrid V2X-in-the-loop testing. The platform’s planned differentiator is a validated scenario library encoding Mediterranean ODDs (high solar glare, high ambient temperatures, dry-road friction profiles, dense mixed urban traffic) built on the architecture described here. Combined with a progressive accreditation path (ISO 9001, then ISO/IEC 17025), this positions the platform as deployable testing infrastructure for suppliers and developers with the intention to complement existing commercial proving grounds and EuroNCAP test sites, adding hybrid V2X-in-the-loop capability on our institution’s own 5G-enabled testbed.

7 Conclusion↩︎

We presented a deployed hybrid ViL platform, its first integrated real-world demonstrator and a multi-condition characterisation of the CP workload it runs. The platform turns live V2X traffic into a probabilistic occupancy grid end to end, and the evaluation shows that cooperation widens coverage and improves recall while localization uncertainty dominates beyond a moderate threshold. Ongoing work targets quantitative sim-to-real fidelity, a higher and bounded loop rate and the Mediterranean-ODD scenario library.

References↩︎