© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

1 Introduction↩︎

Image-to-Point Cloud Registration (I2P) is a key technique in autonomous systems and robotics for integrating camera and LiDAR modalities. I2P enables many applications; for example, monocular-camera localization within a point cloud map allows easy and low-cost system deployment applicable to autonomous driving and robot navigation. These applications require both high accuracy and strong generalization across diverse sensors and environments.

The typical I2P pipeline consists of 2D-3D correspondence estimation followed by pose estimation. Most learning-based I2P methods require paired image-point cloud data with ground-truth poses, leading to high training cost and limited generalization[1]–[6]. Methods based on large-scale pre-trained models avoid this requirement but sacrifice accuracy and speed[7]. Therefore, the modality gap between images and point clouds makes it difficult to achieve both high accuracy and strong generalization across different LiDAR and camera models.

In contrast, Image-to-Image Registration (I2I) has achieved notable performance recently. Learning-based feature matching methods work under challenging conditions such as large viewpoint changes, large photometric gaps, and cross-modal (e.g., RGB vs thermal image matching)[8]–[11]. In I2I, even when modalities differ, they can focus on local geometry pattern information and enable cross-domain learning across diverse environments and conditions, which helps achieve both high accuracy and good generalization.

Based on these observations, we propose a simple yet effective I2P method that treats LiDAR as an imaging sensor; from a single sparse point cloud, we generate a high-fidelity LiDAR intensity image and match it with a camera image (see Fig. 1). This image-based formulation eliminates the need to learn explicit image-point cloud correspondences and allows us to leverage well-established pre-trained image feature matchers. We project a LiDAR point cloud into a sparse intensity image, upsample it to a dense image, match it with the camera image, and estimate the 6-DoF relative pose. For upsampling, we build on Rectified Flow[12], which enables fast inference with fewer sampling steps than diffusion models (e.g., DDPM[13], DDIM[14]). We use its variant, Conditional Rectified Flow, conditioned on the sparse intensity image. Conditional Rectified Flow is pre-trained on image datasets with online masking to acquire inpainting capability in a self-supervised fashion. Experiments demonstrate strong pose estimation and generalization performance without explicit paired image-point cloud supervision.

The main contributions of this work are three-fold:

• We propose a simple yet effective I2P registration method based on LiDAR image generation. By leveraging pre-trained image feature matchers, our approach avoids learning explicit image-point cloud matching, achieving high accuracy and strong generalization without paired image-point cloud training data that are expensive to collect.

• We propose LiDAR image upsampling using Conditional Rectified Flow, which requires fewer sampling steps than other generative model-based methods. Even one-step generation shows little degradation and can run in about 0.68 s. Thanks to the proposed pre-training procedure based on image completion, the model can generalize to different LiDAR models using only a small amount of LiDAR data.

• On the R3LIVE dataset, although we did not use this dataset for training, our method achieved a mean error of 4.89° / 1.63 m and outperformed baseline methods.

2 Related Work↩︎

2.1 Image-to-Image Registration↩︎

I2I has become a core technology for many tasks, such as localization and scene recognition. In I2I, feature matching to find correspondences between images is a key component that largely determines overall performance. Classical hand-crafted features (e.g., SIFT[15], SURF[16], ORB[17]) are still widely used for several tasks like Visual-SLAM (e.g., ORB-SLAM[18]). Deep learning-based methods outperform classical approaches by learning visual priors from large datasets, eliminating the need for manual feature engineering. SuperGlue[8] formulates feature matching as an optimal transport problem solved by a graph neural network with cross-attention, and it remains robust under challenging conditions such as large viewpoint changes and large photometric gaps. Several variants and faster feature matching methods have also been proposed (e.g., LoFTR[9] and LightGlue[10]); among them, some approaches explicitly adapt to cross-modal settings (e.g., thermal-to-RGB image matching), such as XoFTR[11]. Overall, these learning-based approaches are highly effective, and it has also been reported that they can match RGB images to dense LiDAR intensity images[19].

Recently, correspondence-free approaches that simultaneously estimate camera poses and scene geometry in each image view as 3D point maps have emerged, such as DUSt3R[20], VGGT[21], and MapAnything[22]. These approaches lessen the need for explicit correspondences and downstream geometric optimization, and can improve robustness to repetitive or low-texture scenes that challenge local feature matching.

Our current implementation employs the well-established feature matching model (LightGlue[10]) because it exhibits a good cross-modal generalization. It is worth noting that our proposed method is based on a general image generation pipeline, making it easy to extend to other feature matching methods or correspondence-free models with minor modifications and/or additional fine-tuning.

2.2 Image-to-Point Cloud Registration↩︎

Despite the recent success of I2I, I2P is still challenging: LiDAR measurements are sparse with irregular scan patterns, making it difficult to learn general representations, especially when LiDAR models differ from training data or image-point cloud overlap is small. Moreover, the modality gap between images and point clouds is large: although learning-based methods can reduce this gap, they often sacrifice generalization. Existing I2P approaches can be broadly categorized into three groups: (i) methods that predict 2D-3D correspondences and estimate the pose via PnP, (ii) methods that avoid explicit 2D-3D correspondence construction, and (iii) methods that reduce the modality gap through large-scale pre-training.

Correspondence-based methods first establish 2D-3D correspondences between pixels and points, then estimate pose via PnP-RANSAC. Early approaches such as 2D3D-MatchNet[1] often suffered from low inlier ratios because modality differences make it difficult to reliably align keypoints. To address this issue, CorrI2P[2] detects the overlapping region between the image and the point cloud and builds dense 2D-3D correspondences within that region. CoFiI2P[3] improves robustness by mitigating local minima in one-shot matching: it progressively refines correspondences from super-points/super-pixels to point/pixel-level matches. In addition, 2D3D-MATR[4] proposes a coarse-to-fine framework that first matches image patches with point cloud patches and then densifies the correspondences.

Correspondence-free methods directly estimate pose from network predictions without establishing explicit 2D-3D correspondences. DeepI2P[5] classifies whether each point lies inside the camera frustum and updates the pose based on these predictions, but it is often sensitive to classification errors near the field-of-view boundary. LCCNet[6], originally proposed for camera-LiDAR calibration, directly regresses the relative transform from a projected depth image and a camera image; however, it typically handles only a limited miscalibration range and is therefore less suitable for I2P settings with large viewpoint changes. However, all these methods require paired image-point cloud datasets with global poses, and when trained on data with limited diversity in LiDAR scan patterns or viewpoints, they fail to generalize.

FreeReg[7] uses large pre-trained models to bridge the modality gap without fine-tuning. However, it requires many diffusion steps, leading to high latency (9.3 s as reported in the original paper; 22.62 s in our evaluation environment), making real-time use difficult. Moreover, 6-DoF accuracy is limited by reliance on monocular depth estimation.

Motivated by the recent success of image-to-image matching models, we propose a simple yet effective I2P framework based on LiDAR image generation. Since a single LiDAR scan produces a very sparse intensity image that is difficult to match directly against a camera image, we first upsample it into a dense image using a generative model, and then match the result to the camera image using a powerful pre-trained feature matcher.

3 Methodology↩︎

3.1 Proposed Pipeline↩︎

Our goal is to estimate the 6-DoF transformation \({}^C\boldsymbol{T}_L\) between the LiDAR and camera coordinate frames from a single pairing of a LiDAR point cloud and a camera image. Let \(\mathcal{P} = \{{}^L\boldsymbol{p}_i\}_{i=1}^N \subset \mathbb{R}^3\) be a single LiDAR scan with associated point intensities \(\mathcal{L} = \{l_i\}_{i=1}^N \subset \mathbb{R}\), and let \(\mathcal{I}(\boldsymbol{x}) = y\) be a camera image, where \(\boldsymbol{x} \in \mathbb{R}^2\) are pixel coordinates and \(y \in \mathbb{R}\) is the grayscale pixel value. A point \({}^L\boldsymbol{p}_i\) in the LiDAR frame is transformed into the camera coordinate frame as \({}^C\boldsymbol{p}_i = {}^C\boldsymbol{T}_L \, {}^L\boldsymbol{p}_i\) and projected into the image plane via a projection function \(\pi\): \(\boldsymbol{x}_i = \pi({}^C\boldsymbol{p}_i)\). To prioritize responsiveness and frame-by-frame independence, we avoid accumulation of LiDAR scans and multi-frame inputs, and estimation is performed on each single pairing \((\mathcal{P}, \mathcal{I})\).

The proposed pipeline proceeds in three stages, as illustrated in Fig. 2. First, \(\mathcal{P}\) is projected into a sparse LiDAR intensity image \(\mathcal{I}_L\) with a virtual camera model, which is then upsampled to a dense intensity image \(\hat{\mathcal{I}}_L\) by a Conditional Rectified Flow model. Second, a feature matcher extracts keypoints \(\{\boldsymbol{u}_k\}_{k=1}^K \subset \mathbb{R}^2\) on the dense LiDAR intensity image \(\hat{\mathcal{I}}_L\) and \(\{\boldsymbol{v}_k\}_{k=1}^K \subset \mathbb{R}^2\) on the camera image \(\mathcal{I}\), producing 2D–2D correspondences \(\mathcal{M} = \{(\boldsymbol{u}_k, \boldsymbol{v}_k)\}_{k=1}^K\). Finally, each LiDAR keypoint \(\boldsymbol{u}_k\) is lifted to a 3D point \({}^L\boldsymbol{q}_k \in \mathbb{R}^3\) using the LiDAR depth, forming 2D–3D correspondences \(\{(\boldsymbol{v}_k, {}^L\boldsymbol{q}_k)\}\) from which the LiDAR-camera transformation \({}^C\boldsymbol{T}_L\) is estimated via PnP-RANSAC[23], [24].

3.2 LiDAR Data Upsampling↩︎

The LiDAR upsampling module projects the point cloud \(\mathcal{P}\) with intensities \(\mathcal{L}\) into a sparse LiDAR intensity image \(\mathcal{I}_L\), and then outputs a dense intensity image \(\hat{\mathcal{I}}_L\). Conditional diffusion models have been shown to be effective for this task [25], where point clouds are projected to range and intensity images and restored via image inpainting. Rectified Flow [12], which transports samples along straight-line trajectories and enables high-quality generation with fewer inference steps, has further been applied to LiDAR image generation [26]. Building on these advances, we adopt Conditional Rectified Flow as our upsampling model, which generalizes well with limited fine-tuning data to adapt to various LiDAR sensors while also keeping generation time small.

A sparse LiDAR intensity image \(\mathcal{I}_L\) is projected from a single LiDAR scan \(\mathcal{P}\) using a virtual camera model. While it is agnostic to the projection model, we use the pinhole model here; an equirectangular model can be used for omnidirectional cameras. As shown in Fig. 1 (a), a single LiDAR scan \(\mathcal{P}\) is extremely sparse, making it difficult to directly extract meaningful information from \(\mathcal{I}_L\). By conditioning Conditional Rectified Flow on \(\mathcal{I}_L\), we obtain the dense image \(\hat{\mathcal{I}}_L\) from any single scan. To prioritize model generality and geometric correctness during matching, we intentionally avoid depth completion and perform completion only in appearance. We found that the dense intensity image \(\hat{\mathcal{I}}_L\) generated by Conditional Rectified Flow is not always accurate in fine details, but it preserves sufficient global consistency and is reliable enough for image matching.

Moreover, the proposed Conditional Rectified Flow model is pre-trained in a self-supervised fashion via image inpainting on RGB datasets, independent of any LiDAR-specific scan patterns. Therefore, Conditional Rectified Flow can be adapted to various LiDAR sensors with only minimal fine-tuning. Implementation details are described in Section 4.

3.3 Feature Matching↩︎

Given a dense intensity image \(\hat{\mathcal{I}}_L\) and the camera image \(\mathcal{I}\), we perform feature matching to obtain 2D–2D correspondences \(\mathcal{M} = \{(\boldsymbol{u}_k, \boldsymbol{v}_k)\}_{k=1}^K\), where \(\boldsymbol{u}_k \in \hat{\mathcal{I}}_L\) and \(\boldsymbol{v}_k \in \mathcal{I}\) are keypoints in the LiDAR and camera images, respectively. Since \(\hat{\mathcal{I}}_L\) exhibits a similar appearance to a grayscale camera image, the proposed method matches \(\hat{\mathcal{I}}_L\) to \(\mathcal{I}\) using a cross-modal image feature matching model. The proposed pipeline is agnostic to the matching algorithm, and any model can be used, for example SuperGlue[8] or LightGlue[10]. To balance speed and accuracy, we use pre-trained ALIKED[27] for feature detection and LightGlue[10] for matching, with no fine-tuning.

Since identifying correct cross-modal correspondences is generally difficult due to appearance differences, we set a low matching threshold to obtain many correspondences. In this setting, we obtain a large number of correspondences \(\mathcal{M}\), but also many false matches. However, these outliers can be effectively eliminated by the RANSAC-based pose estimation.

3.4 PnP Pose Estimation↩︎

Finally, we estimate the LiDAR-to-camera relative pose \({}^C\boldsymbol{T}_L\) from 2D–3D correspondences \(\{(\boldsymbol{v}_k, {}^L\boldsymbol{q}_k)\}\). Since cross-modal matching between \(\mathcal{I}\) and \(\hat{\mathcal{I}}_L\) may contain many outliers, we use PnP-RANSAC[23], [24] to reduce the influence of wrong matches. The correspondences are pairs of camera keypoints \(\boldsymbol{v}_k \in \mathcal{I}\) and LiDAR keypoints \(\boldsymbol{u}_k \in \hat{\mathcal{I}}_L\). To apply PnP, we recover 3D points \({}^L\boldsymbol{q}_k\) (expressed in the LiDAR frame) for each LiDAR keypoint \(\boldsymbol{u}_k\) using the LiDAR depth. Since only intensity is upsampled (not depth), PnP-RANSAC is applied only to matches \((\boldsymbol{u}_k, \boldsymbol{v}_k)\) for which LiDAR depth exists in the local neighborhood of \(\boldsymbol{u}_k\). Since LiDAR data are sparse, using depth from distant pixels can cause large errors; unless otherwise stated, we use a 3-pixel radius and discard matches with no depth within this radius.

4 Conditional Rectified Flow↩︎

Rectified Flow is a generative model that transports samples from a source distribution \(p_{\mathrm{src}} = \mathcal{N}(\boldsymbol{0}, \boldsymbol{I})\) to a target distribution \(p_{\mathrm{tgt}}\) by solving the ordinary differential equation (ODE) [12]: \[\frac{d\boldsymbol{X}_t}{dt} = \boldsymbol{V}_\theta(\boldsymbol{X}_t, t), \quad t \in [0, 1],\] where \(\boldsymbol{X}_t \in \mathbb{R}^{H \times W}\) is the image state at time \(t\), and \(\boldsymbol{V}_\theta\) is a learned velocity field. Unlike DDPMs, which model the reverse of a stochastic noising process \(\boldsymbol{X}_t = \sqrt{\bar{\alpha}_t}\,\boldsymbol{X}_{\mathrm{tgt}} + \sqrt{1-\bar{\alpha}_t}\,\boldsymbol{\epsilon}\) (\(\boldsymbol{\epsilon} \sim \mathcal{N}(\boldsymbol{0},\boldsymbol{I})\)) and require a large number of stochastic sampling steps, Rectified Flow learns a deterministic flow along straight-line paths \(\boldsymbol{X}_t = (1-t)\boldsymbol{X}_{\mathrm{src}} + t\,\boldsymbol{X}_{\mathrm{tgt}}\). This straight-line transport enables high-quality generation with far fewer inference steps. In this work, we extend Rectified Flow to conditional generation by conditioning \(\boldsymbol{V}_\theta\) on the sparse LiDAR intensity image \(\mathcal{I}_L\), yielding the dense image \(\hat{\mathcal{I}}_L\) with low latency.

4.1 Architecture↩︎

We build a Conditional Rectified Flow model based on SR3[28], a diffusion-based super-resolution method with a U-Net architecture[29]. We add self-attention layers after ResNet blocks to improve global image understanding, as shown in Fig. 3. The model takes image \(\boldsymbol{X}_t\) at time \(t\) and condition image \(\boldsymbol{C}\) as input, outputting velocity field \(\boldsymbol{V}_\theta\). Integrating along the ODE generates the target image. We concatenate \(\boldsymbol{X}_t\) and \(\boldsymbol{C}\) following SR3. We use single-scan LiDAR intensity \(\mathcal{I}_L\) as \(\boldsymbol{C}\) to generate dense image \(\hat{\mathcal{I}}_L\).

4.2 Loss Function and Sampling↩︎

The loss function follows that of standard Rectified Flow [12], assuming single-channel \(H \times W\) images as input. Let \(\boldsymbol{X}_{\mathrm{src}} \in \mathbb{R}^{H \times W}\) be a Gaussian noise image and \(\boldsymbol{X}_{\mathrm{tgt}} \in \mathbb{R}^{H \times W}\) be the target dense intensity image. The interpolated image at time \(t \in [0, 1]\) is defined as: \[\boldsymbol{X}_t = (1-t)\,\boldsymbol{X}_{\mathrm{src}} + t\,\boldsymbol{X}_{\mathrm{tgt}}, \quad \boldsymbol{X}_t \in \mathbb{R}^{H \times W}\] The model \(\boldsymbol{V}_\theta(\boldsymbol{X}_t, \boldsymbol{C}, t) \in \mathbb{R}^{H \times W}\) predicts the velocity at time \(t\), conditioned on \(\boldsymbol{C} \in \mathbb{R}^{H \times W}\), the sparse LiDAR intensity image. The training loss is: \[L(\theta) = \mathbb{E}_{\boldsymbol{X}_{\mathrm{src}},\,\boldsymbol{X}_{\mathrm{tgt}},\,t}\!\left[\left\|\,\boldsymbol{X}_{\mathrm{tgt}} - \boldsymbol{X}_{\mathrm{src}} - \boldsymbol{V}_\theta(\boldsymbol{X}_t,\,\boldsymbol{C},\,t)\,\right\|_2^2\right]\] This trains the model to predict the true displacement \(\boldsymbol{X}_{\mathrm{tgt}} - \boldsymbol{X}_{\mathrm{src}}\) at each interpolated state \(\boldsymbol{X}_t\), which approximates the velocity field that transports samples from \(p(\boldsymbol{X}_{\mathrm{src}})\) to \(p(\boldsymbol{X}_{\mathrm{tgt}})\).

During generation, the model integrates the velocity field from a Gaussian noise image \(\boldsymbol{X}_{\mathrm{src}}\), discretized into \(N\) Euler steps with step size \(\Delta t = 1/N\): \[\boldsymbol{X}_{k+1} = \boldsymbol{X}_k + \boldsymbol{V}_\theta\!\bigl(\boldsymbol{X}_k,\,\boldsymbol{C},\,t_k\bigr)\,\Delta t, \quad t_k = k\,\Delta t\]

4.3 Training↩︎

4.3.1 Pre-Training↩︎

Since dense LiDAR intensity images are difficult to collect at scale, we pre-train the model on image inpainting and fine-tune it for LiDAR intensity images. The pre-training task restores a grayscale image from its masked version, teaching the model to complement sparse scan patterns. We use the GSV-Cities[30] dataset with four on-the-fly masking patterns: random point, vertical line, horizontal line, and block masking. Random point masking (see Fig. 4 (a)) removes individual pixels, vertical and horizontal line masking (see Fig. 4 (b)) leave evenly-spaced lines, and block masking (see Fig. 4 (c)) place square patches at random positions. The parameters of these masks (occlusion rate, line spacing, block count, and block size) are randomly sampled at each training iteration to increase pattern diversity. The condition input of Conditional Rectified Flow is the masked image, and the reference is the original grayscale image. To improve generalization across LiDAR models, we avoid LiDAR-specific scan patterns and use only generic masks.

Random and line masks require the ability to restore images locally using information from unmasked regions, while block masks require predicting the overall structure of masked areas. As shown in Fig. 4, fine details are restored for random and line masks (a)–(b), and the overall structure is reconstructed for block masks (c), confirming the model’s ability to leverage surrounding context.

4.3.2 Fine-Tuning↩︎

Fine-tuning our model requires dense LiDAR intensity images. For non-repetitive LiDARs (e.g., Livox AVIA and MID360), a dense point cloud is obtained by accumulating scans while stationary, since the patterns fill the FOV over time. For spinning LiDARs (e.g., Ouster OS1-64), however, the scan pattern is sparse and repetitive, making it difficult to obtain dense coverage from a static capture. We therefore move the LiDAR in the vertical direction for a few seconds and accumulate points while compensating for viewpoint change and motion distortion. We use CT-ICP[31] to estimate LiDAR motion, correct distortion, and accumulate all points in the first scan’s frame, yielding a dense intensity image. We collected dense LiDAR intensity images in indoor and outdoor environments using three LiDAR sensors: Ouster OS1-64 (20 indoor and 20 outdoor locations) and Livox MID360 and AVIA (40 indoor and 40 outdoor locations each). Fig. 5 shows examples of dense LiDAR intensity images used for fine-tuning. We apply data augmentation including image translation, rotation, horizontal flipping, and varying the virtual camera parameters used for projection.

Fig. 6 shows fine-tuning results for each sensor type. The generated images successfully capture the structure of the ground truth data. These results demonstrate that pre-training has equipped the model with the ability to adapt to diverse scan patterns across different LiDAR sensors.

5 Experiments↩︎

1pt

5.1 Experiment setup↩︎

5.1.1 Dataset↩︎

R3LIVE dataset is used to evaluate registration accuracy and correspondence quality. This dataset contains RGB images, point clouds, and IMU data recorded with a camera and a LiDAR (Livox AVIA) across various environments (e.g., walkway, park, forest). We use hkust_campus_00, hku_campus_seq_00, and hku_park_00 (hereafter hkust_campus, hku_campus, and hku_park) for evaluation. Among them, hku_campus mainly contains structured scenes with buildings, hku_park includes many unstructured elements such as trees and grass, and hkust_campus contains a mixture of structured and unstructured environments. Baseline methods requiring training were trained on hku_main_building, which contains a balanced variety of buildings, vegetation, and walkways.

Since the camera-LiDAR extrinsic is fixed, we create image-point cloud pairs within a 1.2 s sliding window, producing varying displacements, ground-truth poses, and overlap per frame, which better reflects real usage. Motion labeling and point cloud deskewing were performed using GLIM[32], and all methods use the deskewed scans as input.

5.1.2 Baseline Methods↩︎

5.1.2.1 CoFiI2P[3]

It extracts features from images and point clouds in a hierarchical way and finds correspondences in two stages from coarse to fine. The authors provide pre-trained weights, but because the LiDAR scan pattern in R3LIVE dataset is very different, we fine-tuned the model.

5.1.2.2 FreeReg[7]

It uses large pre-trained image-generation models to reduce the modality gap, requiring no retraining for different LiDARs or cameras. Correspondences are built from intermediate diffusion model features, combined with monocular depth and local point cloud-based matches.

5.1.2.3 2D3D-MATR[4]

It is a detector-free method that expands patch correspondences into dense matches instead of detecting separate 2D and 3D keypoints. It learns cross-modal correlation using cross-attention for coarse matching. We adjusted hyperparameters and retrained the model.

5.1.3 Evaluation Metrics↩︎

For evaluation, we measure two aspects: registration accuracy and correspondence quality.

Registration accuracy metric: Following CoFiI2P[3], we use Relative Rotation Error (RRE), Relative Translation Error (RTE), and Registration Recall (RR). Let \(\boldsymbol{R}^{*}\), \(\boldsymbol{t}^{*}\) and \(\hat{\boldsymbol{R}}\), \(\hat{\boldsymbol{t}}\) denote the rotation and translation components of the ground-truth \({}^C\boldsymbol{T}_L^{*}\) and the estimated \({}^C\hat{\boldsymbol{T}}_L\), respectively. RRE and RTE are defined as: \[\begin{align} \text{RRE} &= \textstyle\sum_{i=1}^{3} |r(i)|, \quad \text{RTE} = \|\boldsymbol{t}^{*} - \hat{\boldsymbol{t}}\|, \end{align}\] where \(\boldsymbol{r} = (r(1),r(2),r(3))\) is the Euler angle vector of \((\boldsymbol{R}^{*})^{-1}\hat{\boldsymbol{R}}\). RR is the percentage of test pairs satisfying both \(\text{RRE} < \tau_R\) and \(\text{RTE} < \tau_t\). We use the same threshold settings \((\tau_R, \tau_t)\) as in CoFiI2P[3].

Correspondence quality metric: Following FreeReg[7], we evaluate Feature Matching Recall (FMR), Inlier Ratio (IR), and Inlier Number (IN). A 2D–3D correspondence \((\boldsymbol{v}_k, {}^L\boldsymbol{q}_k)\) is regarded as correct if the reprojection error \(\|\pi({}^C\boldsymbol{T}_L^{*}\, {}^L\boldsymbol{q}_k) - \boldsymbol{v}_k\|\) is smaller than \(\tau_c = 10\,\mathrm{px}\), where \({}^C\boldsymbol{T}_L^{*}\) is the ground-truth transformation. IR is the average proportion of correct correspondences among all estimated correspondences per I2P pair. IN is the average number of correct correspondences per I2P pair. FMR is the fraction of I2P pairs whose IR exceeds 5%.

5.2 Registration Performance↩︎

Table 1 shows registration accuracy (RRE, RTE) and registration recall (RR) for each sequence, compared against CoFiI2P, FreeReg, and 2D3D-MATR. Our method achieves \(1.75^\circ\) RRE and \(0.57\,\)m RTE at the \(10^\circ/5\,\)m threshold on hkust_campus, outperforming all baselines. The hku_campus sequence also demonstrates strong alignment accuracy. Fig. 7 further illustrates that our projected points align well with the image structure, whereas FreeReg[7] and 2D3D-MATR[4] show noticeable misalignment. On hku_park, our method achieves at least 75% RR but is slightly worse than 2D3D-MATR[4]; tree leaves in the LiDAR point clouds cause unstable intensity, leading to frequent failures (see Fig. 8). These results are achieved without training on any camera–LiDAR pairs from R3LIVE dataset, demonstrating the strong generalization of our approach, particularly in structured environments.

5.3 Correspondence Quality↩︎

Table 2 reports correspondence quality (FMR, IR, IN) as an intermediate evaluation of the generated intensity image quality and feature matches. Our method achieves the best results across all metrics; its IN is \(1.6\times\) that of 2D3D-MATR and its FMR approaches 50%, confirming that the upsampled intensity images provide sufficient texture for cross-modal feature matching. The inlier curves in Fig. 9 further confirm this: CoFiI2P and FreeReg yield almost no inliers at any threshold, whereas our method obtains more than 10 inliers at a 5 px threshold. Fig. 10 qualitatively shows that our method produces richer correspondences than 2D3D-MATR.

5.4 Computation Cost↩︎

Table 3 reports per-frame computation time and VRAM usage evaluated on an NVIDIA RTX A5000 24GB. Our method runs at 0.68 s per frame, faster than 2D3D-MATR (0.70 s) and FreeReg (22.62 s). This speed is sufficient for use as a lower-rate matching module alongside high-frequency LiDAR odometry and visual odometry. For VRAM usage, our method requires 3,252 MB, much lower than FreeReg (10,209 MB), which also employs a generative model.

5.5 Ablation Study↩︎

5.5.1 Generation Steps↩︎

To evaluate the effect of generation steps, we ran experiments with 1, 5, and 10 steps. The number of steps causes only minor changes in registration performance, and a single step already achieves sufficient accuracy. Fig. 11 shows the upsampled intensity images for each setting. The 1-step result is blurrier than the 5- or 10-step results, but overall structures such as building walls and the ground are clearly represented, so it contains enough information for feature matching in pose estimation. There is little difference between the 5- and 10-step outputs, and 5 steps are sufficient to fully realize the model’s image generation capability. Increasing the number of generation steps tends to slightly worsen RRE and RTE within the \(10^\circ/5\,\)m threshold, possibly because the generative model produces overly sharpened edges with more steps, increasing false correspondences.

5.5.2 Pre-Training↩︎

To evaluate the effect of pre-training, we tested a model without pre-training. Table 5 shows the comparison results. The model without pre-training showed a large performance drop (RRE \(18.65^\circ\), RTE \(10.64\,\)m, IR \(1.24\%\)), demonstrating the importance of pre-training. Since the fine-tuning data were collected only in-house and are very limited (only 80 scenes), the non-pre-trained model likely overfitted the fine-tuning set, leading to poor generalization.

5.5.3 Matching↩︎

Since our method upsamples only intensity, points without depth must be approximated by nearby depth values; we evaluate the effect of the search radius for this lookup (Table 6). We tested radii of 0, 1, 3, and 5 pixels. With radius 0 (discarding matches without depth), accuracy dropped to RRE \(8.72^\circ\) and RTE \(2.29\,\)m. With radius 5 accuracy \(4.90^\circ\) was worse than with radius 3. This likely happens because a larger search radius increases depth approximation error and degrades pose estimation.

5.6 Different Sensors↩︎

To demonstrate that the method generalizes beyond Livox AVIA used in the R3LIVE dataset evaluation, we show qualitative results on the Ouster OS1-64, a spinning LiDAR, in Fig. 12. The sparse intensity images from a single scan are line-sparse, but we observe they are densely inpainted. From the projected images using estimated poses, we can see the method adapts well to spinning-type LiDAR. Although we used a pinhole camera in this work, the method can be extended to omnidirectional cameras.

6 Conclusion↩︎

We presented an I2P registration method that treats LiDAR as an imaging sensor: sparse LiDAR scans are upsampled to dense intensity images via Conditional Rectified Flow and matched to camera images using pre-trained feature matchers. Experiments on the R3LIVE dataset demonstrated high accuracy and strong generalization. Limitations include degraded performance in unstructured environments. Future work includes integration with more advanced matchers and correspondence-free registration models.

Acknowledgment↩︎

This paper is based on results obtained from the BRIDGE Program (R7-H05), implemented by the Cabinet Office, Government of Japan.

References↩︎

---

1. \(^{1}\)Reon Tabata and Jun Miura are with Department of Computer Science and Engineering, Toyohashi University of Technology, Toyohashi, Aichi, Japan↩︎

2. \(^{2}\)Kenji Koide, Shuji Oishi, Masashi Yokozuka, Aoki Takanose, and Taku Okawara are with National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan↩︎