1 Introduction↩︎

GNSS-denied UAV navigation needs a persistent world-frame reference to keep long trajectories metrically anchored. Visual odometry provides relative motion but accumulates drift [1], and SLAM reduces drift mainly through loop closures in a locally built map [2]. Geo-referenced imagery and elevation data offer a complementary external anchor without requiring a return to previously observed places.

The key obstacle is the map representation. Retrieval-based localization scales to large areas by matching UAV images to geo-tagged satellite tiles [3]–[5], but its accuracy is bounded by reference-database sampling. Metric 6-DoF pose estimation instead uses Perspective-\(n\)-Point (PnP) over 2D–3D correspondences [6], requiring map geometry that supplies elevation or depth. Existing systems obtain this geometry from pose-sampled renderings, textured 3D reconstructions, or orthophoto–digital surface model (DSM) pairs [7]–[10]. These representations can be accurate, but they often require site surveys, region-specific data acquisition, and substantial offline construction, limiting rapid deployment in previously unseen regions. A practical alternative is therefore to localize against widely available public maps.

This paper asks whether accurate UAV localization can instead be built from public geospatial layers: geo-referenced satellite imagery [11], bare-earth DEMs [12], and OpenStreetMap (OSM) [13]. This reference avoids site-specific 3D reconstruction but creates two coupled correspondence failures. First, UAV and north-up satellite views differ in scale and yaw because of altitude, field of view, and heading; these shifts remain difficult for modern matchers [14]–[16], and visual yaw estimation avoids reliance on magnetometers that may be biased [17], [18]. Second, a bare-earth DEM assigns terrain height to every map pixel. A correct image match on a roof or facade therefore becomes a wrong 2D–3D correspondence, and neighboring structure matches can share a coherent height bias that survives RANSAC [19] rather than appearing as independent outliers.

We present AeroMap3D (Fig. 1), a monocular UAV localization framework that derives metric 6-DoF pose estimates from public visual, geometric, and semantic map priors. It targets low-altitude urban flight, where buildings occupy much of the camera view and their omission from bare-earth DEMs most affects pose estimation. Given a coarse map initialization, a lightweight Siamese adapter estimates UAV-to-map scale and yaw, rectifying the pair before a frozen dense matcher. AeroMap3D then rejects matches in OSM-annotated building regions, lifts the remaining UAV–map matches into terrain-supported 2D–3D correspondences, recovers camera pose with standard RANSAC–PnP [6], [19], and fuses map-anchored pose measurements with relative-motion priors in an EKF for continuous localization.

Our main contributions are:

1. We formulate public satellite imagery, bare-earth DEMs, and OSM semantics as complementary priors for monocular 6-DoF UAV localization. We reveal a systematic failure of conventional DEM lifting: visually correct correspondences on elevated structures produce geometrically biased constraints when assigned terrain elevation. Accounting for this visual–geometric inconsistency increases single-frame localization success from 88.24% to 95.69% over a geometry-only RANSAC–PnP baseline.

2. We introduce a lightweight scale–yaw adapter that extends the generalization capability of RoMav2 to UAV-to-map registration. Rather than retraining the dense matcher, the adapter isolates the dominant cross-view variation induced by altitude, camera field of view, and heading. Trained entirely from synthetic satellite-derived pairs without real UAV imagery, manual labels, or site-specific tuning, it raises the registration success rate from 62.4% to 99.2% with only 35 ms of additional edge-device latency.

3. We demonstrate AeroMap3D as a complete map-anchored localization system for drift-tolerant UAV navigation through 55 km of real-world flight over 22.4 km\(^2\) of mapped area. We further release UAV-Terra3D, a public benchmark containing calibrated UAV imagery, reference trajectories, and spatially aligned satellite, DEM, and OSM layers for evaluating map-anchored UAV localization.

2 Related Work↩︎

2.1 Map-Referenced Place Recognition via Image Retrieval↩︎

Image retrieval localizes a UAV by matching a query image to geo-tagged satellite or orthophoto map tiles. NetVLAD established learned global descriptors for place recognition [3], while recent methods use vision foundation models and specialized aerial descriptors to improve robustness to viewpoint, scale, and appearance changes [4], [24]–[27]. Chen et al. [7] recover 6-DoF pose by retrieving a pose-sampled RGB–D rendering and refining it through 2D–3D feature correspondences. However, this approach requires an offline rendering database for each anticipated flight region and was demonstrated only within candidate-pose areas of \(400 \times 400\) m, limiting its spatial scalability. GeoVINS uses retrieval as a geographic correction in visual–inertial estimation [4], and large-scale retrieval pipelines have reached continent-level coverage [5]. These methods scale well, but their localization accuracy is ultimately tied to database sampling density.

2.2 Map-Referenced UAV 6-DoF Localization via PnP↩︎

Structure-based localization estimates camera pose from 2D–3D correspondences with PnP [6] and RANSAC [19]. Modern matchers such as SuperGlue [28], LoFTR [14], and RoMa [15], [16] improve the 2D matching stage under large appearance changes. To recover metric 6-DoF pose, these systems must associate image pixels with 3D map points; they differ primarily in how this geometry is supplied. UAVD4L pre-renders synthetic RGB–D views from a site-specific textured 3D reconstruction [8]. PiLoT instead registers live imagery against RGB–D views rendered online from a high-fidelity 3D map [9]. OrthoTrack lift orthophoto correspondences through co-registered DSMs for single-frame and continuous trajectory estimation, respectively [23]. In each case, successful lifting assumes that the reference height at a matched map pixel corresponds to the surface visible in the UAV image. AeroMap3D instead uses bare-earth DEMs available through large-scale public programs such as USGS 3DEP [12]. Because this terrain model is invalid on buildings, AeroMap3D conditions correspondences with OSM footprints [13] before DEM lifting, as summarized in Table 1.

2.3 UAV Visual Localization Datasets↩︎

Image retrieval-based datasets such as ALTO [20], UAV-VisLoc [21], and AerialVL [22] focus on drone–satellite place recognition and expose strong appearance, altitude, and orientation shifts. Metric localization benchmarks provide stronger geometry: UAVD4L uses a detailed 3D reference [8], while OrthoLoC use orthophoto–DSM references [10]. UAV-Terra3D complements them with continuous real UAV trajectories and co-registered satellite imagery, bare-earth DEM, and OSM layers, enabling evaluation of terrain-lifted 6-DoF localization with explicit semantic validity checks (Table ¿tbl:tab:dataset-comparison?).

3 Problem Formulation↩︎

We formulate map-anchored localization as metric pose recovery from a monocular UAV image and a public geospatial map \[\mathcal{M}=\left(I_{\mathrm{sat}}, h_{\mathrm{DEM}}, \mathcal{B}, \phi\right),\] where \(I_{\mathrm{sat}}\) is the satellite image, \(h_{\mathrm{DEM}}:\mathbb{R}^2\rightarrow\mathbb{R}\) is the bare-earth elevation field, \(\mathcal{B}\) is the OSM-derived building mask in the map image plane, and \(\phi\) maps a map pixel to local East-North-Up (ENU) ground coordinates.

At time \(t\), the UAV body pose is \(\mathbf{T}_{\mathbf{W}\mathbf{B}}^t\in\mathrm{SE}(3)\) in the world frame \(\{\mathbf{W}\}\). With known gimbal extrinsics \(\mathbf{T}_{\mathbf{B}\mathbf{C}}^t\), the camera pose is \[\mathbf{T}_{\mathbf{W}\mathbf{C}}^t = \mathbf{T}_{\mathbf{W}\mathbf{B}}^t \cdot \mathbf{T}_{\mathbf{B}\mathbf{C}}^t. \label{eq:TWC95relation}\tag{1}\] Equivalently, \(\mathbf{T}_{\mathbf{C}\mathbf{W}}^t=(\mathbf{T}_{\mathbf{W}\mathbf{C}}^t)^{-1}\) maps world points into the camera frame; we write its \(3\times4\) projection block as \([\mathbf{R}^t\mid\mathbf{t}^t]\).

Observation Model. For a lifted map point \(\mathbf{X}_i\in\mathbb{R}^3\) and calibrated intrinsics \(\mathbf{K}\), the ideal image measurement \(\mathbf{x}_i\in\mathbb{R}^2\) satisfies \[\lambda_i \begin{bmatrix}\mathbf{x}_i \\ 1\end{bmatrix} = \mathbf{K} \begin{bmatrix}\mathbf{R}^t\!\mid\!\mathbf{t}^t\end{bmatrix} \begin{bmatrix}\mathbf{X}_i \\ 1\end{bmatrix}. \label{eq:pinhole95model}\tag{2}\] Thus the problem reduces to constructing reliable 2D–3D correspondences and solving for \(\mathbf{T}_{\mathbf{C}\mathbf{W}}^t\).

Map Lifting and Structural Uncertainty. Let dense registration return 2D–2D matches \(\mathcal{C}_{2\mathrm{D}}^t=\{(\mathbf{x}_i^t,\mathbf{m}_i^t)\}_{i=1}^{N_t}\) between the UAV image and the satellite map. Each map pixel is lifted through \[\begin{align} (E_i,N_i) &= \phi(\mathbf{m}_i^t), \\ \mathbf{X}_i &\doteq \ell(\mathbf{m}_i^t) = \begin{bmatrix} E_i & N_i & h_{\mathrm{DEM}}(E_i,N_i) \end{bmatrix}^{\top}. \end{align} \label{eq:dem95lifting}\tag{3}\] This lift is exact for terrain but biased for elevated structures. If the true surface height differs by \(\Delta h_i\), then \[\mathbf{X}_i^{\mathrm{true}} =\ell(\mathbf{m}_i^t)+ \begin{bmatrix}0&0&\Delta h_i\end{bmatrix}^{\top}, \qquad \Delta h_i \neq 0 , \label{eq:height95bias}\tag{4}\] and the resulting reprojection residual is no longer a random matching error but a structured geometric bias.

Robust Pose Objective. AeroMap3D first removes known structural regions in the map plane, \[\mathcal{C}_{\mathrm{sem}}^t = \left\{ (\mathbf{x}_i^t,\mathbf{X}_i) \;\middle|\; (\mathbf{x}_i^t,\mathbf{m}_i^t)\in\mathcal{C}_{2\mathrm{D}}^t, \mathbf{m}_i^t\notin\mathcal{B} \right\}. \label{eq:semantic95filter}\tag{5}\] Residual outliers from missing OSM annotations, dynamic objects, or feature mismatches are handled by geometric consensus. With \[\mathbf{r}_i(\mathbf{T}) = \mathbf{x}_i^t - \Pi\!\left(\mathbf{K}\begin{bmatrix}\mathbf{R}\!\mid\!\mathbf{t}\end{bmatrix} \begin{bmatrix}\mathbf{X}_i \\ 1\end{bmatrix}\right),\] where \(\Pi([u,v,w]^\top)=[u/w,v/w]^\top\) and \(\mathbf{T}\in\mathrm{SE}(3)\) is represented by \((\mathbf{R},\mathbf{t})\). RANSAC–PnP selects an inlier index set \(\mathcal{I}_t^\star\) from \(\mathcal{C}_{\mathrm{sem}}^t\) and refines the camera pose by \[\hat{\mathbf{T}}_{\mathbf{C}\mathbf{W}}^t = \underset{\mathbf{T}\in\mathrm{SE}(3)}{\arg\min} \sum_{i\in\mathcal{I}_t^\star} \left\|\mathbf{r}_i(\mathbf{T})\right\|_2^2. \label{eq:robust95pnp}\tag{6}\] The corresponding UAV body pose is \[\hat{\mathbf{T}}_{\mathbf{W}\mathbf{B}}^t = \left(\hat{\mathbf{T}}_{\mathbf{C}\mathbf{W}}^t\right)^{-1} \left(\mathbf{T}_{\mathbf{B}\mathbf{C}}^t\right)^{-1}, \label{eq:body95pose95update}\tag{7}\] which gives the metric map-based pose measurement used by the trajectory estimator.

4 Methodology↩︎

4.1 Cross-View Registration via Lightweight Adapter↩︎

Scale–Yaw Normalization. The registration frontend converts a difficult UAV–satellite pair into a geometry range where a pretrained matcher can operate reliably. Instead of replacing the matcher, we estimate only the dominant cross-view nuisance parameters: the scale ratio induced by altitude and field of view, and the yaw offset induced by UAV heading. This yaw is a relative image-to-map rotation used for registration, not a replacement for the vehicle’s full attitude estimate. We implement the adapter as a lightweight Siamese CNN: the UAV image and satellite tile are passed through a shared MobileNetV3-Small encoder [29], global-average-pooled into descriptors \(\mathbf{g}_t\) and \(\mathbf{g}_{\mathrm{sat}}\), fused as \([\mathbf{g}_t,\mathbf{g}_{\mathrm{sat}},|\mathbf{g}_t-\mathbf{g}_{\mathrm{sat}}|,\mathbf{g}_t\odot\mathbf{g}_{\mathrm{sat}}]\), and regressed by a two-layer MLP. For UAV image \(I_t\) and a north-up satellite tile \(I_{\mathrm{sat}}\), the adapter predicts \[(\hat{s}_t,\hat{\theta}_t) = f_{\boldsymbol{\theta}}(I_t,I_{\mathrm{sat}}),\] where the MLP outputs log-scale and a normalized yaw vector internally, which is converted to \((\hat{s}_t,\hat{\theta}_t)\) for rectification. It is trained with \[\mathcal{L}_{\text{align}} = \left|\log\hat{s}_t-\log s_t^\star\right| + \beta\left(1-\cos(\hat{\theta}_t-\theta_t^\star)\right), \label{eq:alignment95loss}\tag{8}\] where \((s_t^\star,\theta_t^\star)\) are synthetic supervision targets and \(\beta\) balances scale and yaw. Training pairs are generated without manual correspondence labels or UAV-Terra3D images: we sample plausible flight states, project their camera footprints onto satellite imagery, and render UAV-like views with known scale and yaw (Fig. 2).

This supervision normalizes scale and yaw but does not simulate seasonal appearance transfer or full projective distortion from large roll or pitch; these remain limitations of the frozen matcher and map reference.

Dense Matching After Rectification. The predicted \((\hat{s}_t,\hat{\theta}_t)\) rectifies the map tile before matching. We then run RoMav2 [16] as a swap-in dense matcher and map the matched satellite pixels back to the original map plane: \[\mathcal{C}_{2\mathrm{D}}^t = \left\{ \bigl(\mathbf{x}_i^t,\mathbf{m}_i^t\bigr) \right\}_{i=1}^{N_t}. \label{eq:method952d95matches}\tag{9}\] Here \(\mathbf{x}_i^t\) is a UAV image pixel and \(\mathbf{m}_i^t\) is its satellite-map pixel. This keeps the dense matcher modular while producing the 2D–2D correspondences that are subsequently lifted onto the DEM and filtered with semantic map priors.

4.2 Semantic Map Conditioning for DEM-PnP↩︎

Semantic Filtering Before 3D Lifting. The core contribution of this stage is semantic conditioning before pose estimation. A bare-earth DEM provides scalable terrain geometry, but it assigns terrain height to every map pixel; matches on roofs, facades, or other above-ground structures therefore inherit the height bias in Eq. 4 . Because these errors are coherent rather than random, RANSAC alone can accept an incorrect dominant surface. AeroMap3D instead removes known non-ground matches before DEM lifting. OSM building footprints are rasterized into the binary mask \(\mathcal{B}\), and candidate matches on these semantically unreliable regions are rejected in the map plane: \[\mathcal{C}_{\mathrm{gnd}}^t = \left\{ (\mathbf{x}_i^t,\mathbf{m}_i^t) \in \mathcal{C}_{2\mathrm{D}}^t \;\middle|\; \mathbf{m}_i^t\notin\mathcal{B} \right\}.\] Only the surviving map pixels are lifted with \(\ell(\cdot)\) from Eq. 3 , yielding terrain-supported 2D–3D correspondences and preventing known structural regions from becoming confidently wrong 3D points.

Geometric Consensus for Residual Outliers. Semantic filtering removes the dominant map-induced bias but cannot cover missing OSM annotations, off-nadir building lean, transient objects, or dense-matcher failures. These residual errors are handled by standard OpenCV RANSAC–PnP [6], corresponding to Eq. 6 ; the novelty is the semantic conditioning that supplies PnP with terrain-consistent correspondences. The resulting metric pose is the global visual measurement passed to the delayed-update EKF in Sec. 4.3.

This residual-consensus stage still assumes that terrain-consistent correspondences remain dominant; a large unmapped structure can form a coherent but incorrect consensus and remains a map-validity failure mode.

4.3 Continuous Trajectory Estimation via Map-Anchored EKF↩︎

Pose-Error State and Motion Prior. The nominal EKF state is the map-frame body pose \(\hat{\mathbf{T}}_k\in\mathrm{SE}(3)\) with right-multiplicative local error \(\delta\boldsymbol{\xi}_k\in\mathbb{R}^6\), i.e., \(\mathbf{T}_k=\hat{\mathbf{T}}_k\operatorname{Exp}(\delta\boldsymbol{\xi}_k^\wedge)\), and tangent-space covariance \(\mathbf{P}_k\). Between map updates, it propagates with the relative pose increment \(\tilde{\Delta\mathbf{T}}_k\) from Sec. 5: \[\begin{align} \hat{\mathbf{T}}_{k|k-1} &= \hat{\mathbf{T}}_{k-1|k-1}\tilde{\Delta\mathbf{T}}_k,\\ \mathbf{P}_{k|k-1} &= \mathbf{F}_k\mathbf{P}_{k-1|k-1}\mathbf{F}_k^\top+\mathbf{Q}_k . \end{align} \label{eq:ekf95prediction}\tag{10}\] Here \(\mathbf{F}_k\) and \(\mathbf{Q}_k\) are the pose-composition Jacobian and motion-increment covariance. PnP provides \(\mathbf{Z}_k\in\mathrm{SE}(3)\) from Eq. 7 , with tangent-space covariance \(\mathbf{R}_k\).

Visual poses are tagged at image acquisition. The reported offline experiments fuse them in timestamp order and do not evaluate wall-clock delay. For online use, a pose captured at \(k\) but returned at \(j\ge k\) updates the buffered state at \(k\); stored motion increments then re-propagate the estimate to \(j\).

Mahalanobis Gating. For the delayed state, we compute the innovation and its consistency score \[\begin{align} \mathbf{r}_k &= \operatorname{Log}\!\left( \hat{\mathbf{T}}_{k|k-1}^{-1}\mathbf{Z}_k \right)^\vee,\\ \mathbf{S}_k &= \mathbf{H}_k\mathbf{P}_{k|k-1}\mathbf{H}_k^\top+\mathbf{R}_k,\\ \gamma_k &= \mathbf{r}_k^\top\mathbf{S}_k^{-1}\mathbf{r}_k . \end{align} \label{eq:ekf95gate}\tag{11}\] where \((\cdot)^\vee\) gives 6-vector tangent coordinates and \(\mathbf{H}_k=\mathbf{I}_6\). The visual pose is accepted only if \[\gamma_k < \tau_{\chi^2}, \qquad \tau_{\chi^2}=\chi^2_{d,1-\eta}, \label{eq:chi295gate}\tag{12}\] where \(d\) is the measurement dimension and \(\eta\) is the gate probability. Rejected measurements are discarded and the trajectory coasts on the relative-motion prior. Accepted measurements use the standard EKF update \[\begin{align} \mathbf{K}_k &= \mathbf{P}_{k|k-1}\mathbf{H}_k^\top\mathbf{S}_k^{-1},\\ \delta\hat{\boldsymbol{\xi}}_k &= \mathbf{K}_k\mathbf{r}_k,\\ \hat{\mathbf{T}}_{k|k} &= \hat{\mathbf{T}}_{k|k-1} \operatorname{Exp}(\delta\hat{\boldsymbol{\xi}}_k^\wedge),\\ \mathbf{P}_{k|k} &= (\mathbf{I}-\mathbf{K}_k\mathbf{H}_k)\mathbf{P}_{k|k-1} (\mathbf{I}-\mathbf{K}_k\mathbf{H}_k)^\top +\mathbf{K}_k\mathbf{R}_k\mathbf{K}_k^\top . \end{align} \label{eq:ekf95update}\tag{13}\] The corrected state is then re-propagated; \(\operatorname{Prop}_\ell\) applies the stored increment \(\tilde{\Delta\mathbf{T}}_\ell\): \[\begin{align} (\hat{\mathbf{T}}_{\ell|\ell-1},\mathbf{P}_{\ell|\ell-1}) &= \operatorname{Prop}_\ell\!\left( \hat{\mathbf{T}}_{\ell-1|\ell-1}, \mathbf{P}_{\ell-1|\ell-1} \right),\\ \ell&=k+1,\ldots,j . \end{align} \label{eq:delayed95repropagation}\tag{14}\]

5 Dataset↩︎

UAV-Terra3D Benchmark. UAV-Terra3D is a benchmark for map-anchored UAV visual localization. It contains 30-Hz, mostly nadir-view UAV video, calibrated camera intrinsics, GNSS position reference trajectories, and aligned visual, geometric, and semantic map layers. We collected 20 trajectories at eight sites in Austin, Texas, using a DJI Air 2S equipped with a calibrated gimballed camera. The dataset covers 55 km and 134 minutes of flight over 22.4 km\(^2\), with varied altitudes, headings, and environments. Most frames are nadir or near-nadir; the relatively few high-pitch frames do not support a statistically meaningful pitch-stratified registration analysis. The collection device does not provide independently validated full 6-DoF ground truth or gimbal-encoder calibration suitable for evaluating roll, pitch, and yaw error.

3D Terrain Models. Each site combines NAIP imagery [11], USGS DEM elevation [12], and OSM semantics [13] in a WGS84-referenced local ENU frame (Fig. 4). The 1-m 3DEP tiles are horizontally NAD83 and vertically NAVD88; we transform them to WGS84 and convert orthometric elevation \(H\) to ellipsoidal height \(h=H+N\) using the metadata geoid separation \(N\), matching the GNSS altitude convention. The imagery supports registration, the DEM supplies 3D terrain, and OSM identifies regions where bare-earth elevation is unreliable.

Motion-Prior Generation. UAV-Terra3D does not include a raw IMU stream. To compare trajectory methods under the same realistic, imperfect motion input, we derive a reproducible stochastic odometry prior from consecutive GNSS reference poses \(\mathbf{T}_{t-1}^\star,\mathbf{T}_t^\star\in\mathrm{SE}(3)\). We perturb each relative motion by zero-mean noise whose scale increases with translation and rotation, thereby modeling the accumulation of odometry uncertainty without using GNSS as an absolute EKF update: \[\begin{align} \Delta \mathbf{T}_t^\star = \left(\mathbf{T}_{t-1}^\star\right)^{-1}\mathbf{T}_t^\star,\\ \tilde{\Delta \mathbf{T}}_t = \Delta \mathbf{T}_t^\star \operatorname{Exp}(\boldsymbol{\xi}_t),\\ \boldsymbol{\xi}_t \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma}_t). \end{align} \label{eq:stochastic95odometry}\tag{15}\] Following the probabilistic odometry model [30], the translation and rotation noise scales are \[\begin{align} \boldsymbol{\Sigma}_t &= \operatorname{diag}\!\left( \sigma_{p,t}^2\mathbf{1}_3, \sigma_{R,t}^2\mathbf{1}_3 \right),\\ \sigma_{p,t} &= \alpha_3\|\Delta \mathbf{p}_t^\star\| + \alpha_4\|\operatorname{Log}(\Delta \mathbf{R}_t^\star)\| + \beta_p,\\ \sigma_{R,t} &= \alpha_1\|\operatorname{Log}(\Delta \mathbf{R}_t^\star)\| + \alpha_2\|\Delta \mathbf{p}_t^\star\| + \beta_R . \end{align} \label{eq:odometry95noise}\tag{16}\] Here \(\Delta \mathbf{p}_t^\star\) and \(\Delta \mathbf{R}_t^\star\) are the translational and rotational components of \(\Delta \mathbf{T}_t^\star\); parameter values are given in Sec. 6. GNSS is used only to generate this fixed prior and to score accuracy, never as an absolute localization update. In addition to this reference-derived relative-motion prior, we generate a visual-only prior with frame-to-frame ORB visual odometry [1], [31].

6 Experiments↩︎

We conduct real-world experiments to answer four questions: (Q1) Can an adapter trained only on synthetic UAV-VisLoc data collected in China transfer to real UAV imagery from the geographically distinct UAV-Terra3D dataset in the United States, without target-dataset retraining? (Q2) How much does the adapter improve UAV-to-map registration when paired with a frozen image matcher? (Q3) Does geometric–semantic correspondence filtering improve single-frame 6-DoF camera pose estimation over geometric-only PnP and image-retrieval baselines? (Q4) Can fusing map-based visual updates with motion priors reduce localization error and bound long-horizon trajectory drift?

6.1 Experimental Setup↩︎

Protocol and Metrics. The adapter is trained only on synthetic UAV–map pairs rendered from UAV-VisLoc satellite imagery [21] (Fig. 2) and evaluated by cross-dataset transfer to UAV-Terra3D; no target data are used for training or tuning. All evaluations are offline and start from a user-provided initial \((x,y)\) position. We evaluate every UAV-Terra3D trajectory and frame without site-, trajectory-, or frame-level cherry-picking. Visual poses retain their acquisition timestamps and are fused in timestamp order; Sec. 4.3 describes delayed-state handling for online processing.

For image registration, we report pseudo success rate (i.e., the fraction of frames with sufficient inliers and a valid fundamental matrix or homography after geometric verification) and runtime (ms/frame). For trajectories, we report the success rate within 50 m and absolute 3D localization error.

Implementation Details. The scale–yaw adapter is a Siamese MobileNetV3-Small (Sec. 4.1) trained on ten rendered UAV-like views from each of 3,394 UAV-VisLoc scenes. It takes \(224 \times 224\) UAV and satellite inputs. Our default frozen matcher, RoMav2 [16], uses \(518 \times 518\) inputs, 1,024 keypoints, and ImageNet normalization. We estimate poses with OpenCV PnP and reprojection-error refinement, optimize world points in local ENU coordinates, and convert the result to WGS84. For the reference-derived motion prior in Eq. 16 , we use \((\alpha_1,\alpha_2,\alpha_3,\alpha_4)=(0.05,0.01\,\mathrm{rad\,m^{-1}},0.05,0.10\,\mathrm{m\,rad^{-1}})\), \(\beta_p=0.05\) m, and \(\beta_R=0.5^\circ\); the EKF uses a 95% \(\chi^2\) gate for 6-DoF updates (\(\tau_{\chi^2}=12.59\)).

Baselines. (1) UAV-to-map image registration. We compare SIFT+LightGlue [32], SuperPoint+LightGlue [33], LoFTR [14], and RoMa matchers [15], [16]. Each is evaluated directly and after adapter-based scale–yaw normalization, with matcher weights fixed. (2) Single-frame camera pose estimation. We compare DINOv2-based retrieval [24] and SelaVPR++ [25], using \(150 \times 150\) m satellite tiles sampled at a 10 m stride as the candidate database. We also include planar 2D-map homography, sparse DEM-lifted PnP from four UAV-footprint corners, and a dense-PnP ablation without semantic rejection. (3) Multi-frame trajectory estimation. We compare Ref-Odom in Eq. (15 ), ORB visual odometry [1], [31], ORB-SLAM2 [2], and GeoVINS-style retrieval [4]. The first three have no satellite-map anchoring. Our GeoVINS-style baseline uses DINOv2 [34] for coarse retrieval updates and shares the same initialization, noisy motion prior, and EKF interface as AeroMap3D.

6.2 Q1 & Q2: Cross-Dataset Registration↩︎

Fig. 5 evaluates cross-dataset transfer from synthetic UAV-VisLoc training pairs to real UAV-Terra3D imagery without target-dataset training. Despite the scale- and orientation-robust features of SIFT and RoMa, both fall short under the large cross-view appearance, footprint-scale, and yaw mismatch: SIFT+LightGlue reaches \(35.1\%\) geometric-verification pass rate and RoMav2 reaches \(62.4\%\). The adapter normalizes scale and yaw before correspondence extraction and improves every evaluated matcher. In particular, Adapter+RoMav2 reaches \(99.2\%\), a \(36.8\)-point gain, providing the geometrically consistent correspondences required for DEM-lifted PnP.

On the RTX 5090, Tiny-RoMa is the fastest configuration (18 ms), whereas RoMav2 delivers the highest registration accuracy but requires 487 ms. For edge deployment, we recommend Adapter+Tiny-RoMa: it reaches \(92.5\%\) success on a Jetson Orin Nano in 130 ms (35 ms adapter plus 95 ms matcher), while RoMav2 exceeds the device’s memory budget. The delayed-update EKF accommodates these asynchronous visual updates.

6.3 Q3: Single-Frame Camera Pose Estimation↩︎

The upper half of Table ¿tbl:tab:localization-performance? shows that retrieval provides only coarse localization. SelaVPR++, whose training is tailored to visual place recognition, improves on the DINOv2-based method but still reaches only \(19.35\%\) success and 29.30 m mean error. The planar baseline estimates a RANSAC homography between the UAV image and the geo-referenced map, then decomposes it using camera intrinsics and the map scale to recover pose relative to the ground plane. It improves success to \(62.20\%\) but retains 30.80 m error, demonstrating that the planar assumption is insufficient. Sparse DEM-lifted PnP reduces the mean error to 23.90 m but succeeds on only \(44.70\%\) of frames because footprint corners frequently fall on elevated or unmapped structures.

To isolate semantic conditioning, we disable only the OSM pre-filter while holding the matcher, DEM lifting, PnP solver, RANSAC thresholds, and evaluation set fixed. Dense DEM-PnP reaches \(88.24\%\) success and 16.17 m mean error; adding OSM raises success to \(95.69\%\) and reduces the mean and 95th-percentile errors to 14.11 m and 28.20 m. This controlled comparison measures the incremental benefit of semantic conditioning before standard geometric consensus.

6.4 Q4: Continuous Trajectory Estimation↩︎

The lower half of Table ¿tbl:tab:localization-performance? and Figs. 6–7 show that motion propagation alone or monocular visual odometry cannot prevent long-range drift: Ref-Odom reaches only \(14.90\%\) success, and ORB-VO and ORB-SLAM2 remain unanchored to the map. GeoVINS-style retrieval achieves \(100\%\) success with 17.17 m mean error, whereas AeroMap3D preserves \(100\%\) success with 5.88 m mean and 11.25 m 95th-percentile error. This \(65.8\%\) lower mean error, together with bounded RPE in Fig. 6 and the representative trajectory in Fig. 7, shows the benefit of metric map updates.

7 Conclusion↩︎

We presented AeroMap3D for map-anchored UAV localization from satellite imagery, bare-earth DEMs, and OSM building footprints. By normalizing scale–yaw mismatch and filtering semantically invalid DEM-lifted correspondences before standard RANSAC–PnP, AeroMap3D raises RoMav2’s geometric-verification pass rate from \(62.4\%\) to \(99.2\%\) and single-frame localization success from \(88.24\%\) to \(95.69\%\). With a reference-derived stochastic motion prior, the trajectory estimator achieves 5.88 m mean translation error over 55 km.

Limitations and Future Work. The evaluation is limited to eight Austin-area sites, coarse initialization, translation-only ground truth, and a GNSS-reference-derived motion prior. OSM gaps, dominant unmapped structures, strongly oblique views, and seasonal or temporal map changes remain failure modes. Future work will evaluate measured onboard odometry, multi-region flights, attitude and gimbal-extrinsic accuracy, pitch-stratified registration, global initialization, map-validity modeling, and uncertainty-aware visual updates.

Acknowledgment. This work has taken place in the Human Centered Robotics Laboratory (HCRL) at UT Austin. This work has been made possible thanks to the generous gifts from Leo Lion and the Sentis-Ben-Yakar family.

References↩︎