1 Introduction↩︎

Facade semantic segmentation has been studied since the early days of computer vision [1], owing to the structured yet intricate patterns present in building facades and their importance for urban scene understanding, large-scale mapping, and semantic digital twins. Accurate per-element labeling of facade components enables downstream applications such as architectural analysis, urban planning, energy modeling, and change detection [2]–[6]. Despite steady progress in 2D and 3D scene segmentation, facade understanding remains challenging due to its fine-grained structure, high intra-class variability, and strong dependence on precise geometric detail.

A key difficulty arises from the long-tail distribution of facade elements commonly observed in real-world data (e.g., ScanNet [7], A2D2 [8]). While dominant classes such as walls are well represented, semantically critical architectural elements appear far less frequently, including doors, windows, and balconies. This imbalance mirrors challenges observed across indoor and outdoor semantic segmentation benchmarks [7]–[11] and significantly limits the effectiveness of data-driven approaches. As a result, models often produce biased predictions favoring dominant classes while failing to recognize smaller and structurally intricate facade elements even within the same architectural style [11], [12].

These challenges are further exacerbated by the limited availability and diversity of annotated facade datasets. Existing benchmarks are typically limited to small-scale acquisitions covering individual buildings or homogeneous building blocks within narrow geographic regions (e.g., ZAHA [11], ArCH [12]). Consequently, current models tend to overfit dataset- and country-specific architectural styles and struggle to generalize across cities or countries with different construction practices and design principles.

A seemingly straightforward solution is to leverage widely available street-level imagery to expand facade annotations. However, such data cannot provide the centimeter-level geometric accuracy of laser scanners required for precise architectural parsing. Facade segmentation is inherently accuracy-critical: even small geometric deviations can corrupt the delineation of fine structural elements and undermine semantic validity.

Beyond geometric fidelity, consistent semantic interpretation is equally important. Architecturally meaningful reasoning requires unified, standard-grounded class definitions rather than ad hoc, dataset-specific taxonomies. Without harmonized semantic structures transferable across cities and countries, benchmarking remains inconsistent and cross-domain evaluation unreliable.

The absence of large-scale 3D facade datasets that jointly ensure centimeter-grade geometric fidelity and standardized, architecturally grounded semantics therefore remains a core limitation of current benchmarks.

To address these limitations, we introduce UnderOneFacade, the largest cross-country 3D facade benchmark to date, comprising centimeter-accurate point clouds with unified semantic annotations grounded in architectural standards and totaling 2.7 billion annotated points. The dataset spans multiple cities and architectural styles across different countries, enabling systematic evaluation of cross-country and cross-continent generalization. Our experiments show that even strong baseline architectures achieve only around 33 mIoU on the most challenging LoFG3 benchmark, highlighting that fine-grained facade understanding remains far from solved. Beyond facade parsing itself, the proposed benchmark also provides a controlled setting for studying broader problems in 3D scene understanding, including cross-domain generalization across architectural styles, long-tailed recognition of structural elements, and multimodal geometric–radiometric reasoning. Our contributions are summarized as follows:

• We introduce UnderOneFacade, the largest 3D facade segmentation benchmark with centimeter-level geometric accuracy and unified architectural semantics, comprising 2.7B annotated points.

• We enable the first systematic evaluation of cross-country and cross-continent generalization in facade segmentation.

• We provide a comprehensive benchmark revealing the remaining challenges of fine-grained facade segmentation under realistic long-tailed distributions.

2 Related Works↩︎

3D Semantic Segmentation Methods. Early approaches imposed regular structure on unorganized point clouds (e.g., voxels), enabling convolutional processing but introducing discretization artifacts and limiting scalability [13]–[16]. More recent methods process raw points directly. Point-based networks such as PointNet and PointNet++ [17], [18] introduced hierarchical aggregation to capture local geometric context, which is critical for thin and repetitive architectural structures. Subsequent work explored graph-based and transformer-based architectures to model local and global dependencies more effectively [19]–[24]. Some approaches additionally incorporate hierarchical spatial partitions (e.g., octrees) to balance geometric precision and computational efficiency [25], [26]. Despite these advances, accurately segmenting fine-grained facade elements remains challenging due to long-tail class distributions and the limited availability of large-scale facade datasets.

3D Urban Semantic Segmentation Datasets. Large-scale urban point cloud datasets have steadily expanded over the past decade, driven by advances in LiDAR sensing and 3D reconstruction. These benchmarks have enabled substantial progress in urban semantic segmentation and scene understanding. Representative datasets include Semantic3D [10], Paris-Lille-3D [27], and Toronto-3D [28]. However, these datasets primarily target coarse semantic categories such as buildings, roads, and vegetation. Fine-grained facade-level parsing is typically absent or treated marginally, limiting progress toward detailed architectural reasoning in 3D.

6pt width=,center

Facade-Level 3D Datasets. Motivated by applications such as digital twins and automated building reconstruction [4], [29], several datasets have begun to target facade-specific semantic segmentation with higher geometric fidelity and class granularity. Examples include ArCH [6], TUM-FAÇADE [30], and ZAHA [11]. Nevertheless, existing datasets remain limited in scale, architectural diversity, and geographic coverage. Their restricted size constrains deep model training and systematic evaluation. This bottleneck largely stems from the high cost of centimeter-accurate acquisition and the labor-intensive annotation of small, repetitive, and structurally complex facade elements.

Semantic Consistency. Beyond dataset scale, facade benchmarks also exhibit substantial variability in class definitions, often relying on ad hoc or project-specific taxonomies. Such inconsistency impedes cross-dataset evaluation and reproducibility. The ZAHA dataset introduces the Level-of-Facade-Granularity (LoFG) concept, a hierarchical taxonomy grounded in architectural standards. While this represents an important step toward semantic harmonization, its realization within a single German city limits architectural variability and effectively validates the concept primarily on Central European typologies. These limitations highlight the need for a large-scale, centimeter-accurate, cross-country 3D facade benchmark with standardized and architecturally grounded semantic labels.

UnderOneFacade addresses this gap by providing the largest harmonized facade dataset to date, enabling systematic evaluation across countries and supporting the development of long-tail-robust and transferable 3D facade segmentation methods. In this sense, UnderOneFacade can be viewed as a cross-country extension of ZAHA: we inherit the LoFG taxonomy and the German subset from ZAHA, while contributing a \(4.5\times\) increase in scale, multi-sensor acquisition, and a cross-country and cross-continent evaluation protocol.

3 Dataset Design↩︎

UnderOneFacade prioritizes highest possible geometric accuracy by deploying laser scanners with up to centimeter local accuracy and globally georeferenced centimeter acquisition, increasing architecture pattern richness by acquiring cross-continent data, multi-modal scanner setups to ensure scanner-related patterns variability but still ensure even cm-grade accuracy, and ensuring high-definition labels by manual and hierarchical annotations.

Three representative regions are selected to capture architectural diversity across continents. The European subsets (UK and Germany) share historical influences but differ in facade materials, ornamentation, and structural composition (Victorian vs Haussmann). In contrast, the Singapore subset represents Southeast Asian urban architecture, combining traditional Malay and Chinese elements with modern high-density construction. This selection introduces substantial stylistic variation and enables systematic evaluation of cross-regional generalization in facade segmentation.

3.1 Laser Scans↩︎

UK. A static terrestrial laser scanning (TLS) campaign was conducted in The Park residential district in Nottingham, United Kingdom, using the Leica RTC360 [31] scanner, which provides local accuracy at the centimeter-level and rapid full-dome capture. A total of 17 scanning stations were placed along the urban segment to ensure full facade coverage, reduce occlusions, and maintain sufficient overlap between neighboring scans. To ensure consistent global alignment, the survey was fully georeferenced using a network of ground control points (GCPs) measured with GNSS and RTK corrections, providing centimeter-level positional accuracy in the national coordinate system. Individual scans were initially aligned using the scanner’s visual–inertial system and subsequently refined through global registration. The final dataset captures dense facade geometries with fine architectural detail, including window frames, balconies, cornices, and recessed entrances. Compared to mobile mapping data, the static multi-station acquisition significantly increases point density on vertical structures while reducing occlusions. Using the georeferenced data, individual buildings were extracted using OpenStreetMap (OSM) building footprints [32], expanded with a 5 m buffer to account for footprint inaccuracies. The resulting UK subset contains 997M points across approximately 100 facades. No additional noise filtering was applied in order to preserve raw acquisition fidelity.

Germany. The German subset is based on the ZAHA dataset [11], which we integrate into the UnderOneFacade benchmark. ZAHA builds upon the publicly available TUM-MLS-2016 dataset [33], acquired in the Munich downtown area using the Mobile Distributed Situation Awareness (MODISSA) platform. The platform is equipped with two Velodyne HDL-64E LiDAR sensors mounted at the front of the vehicle and two Velodyne VLP-16 sensors at the rear, providing dense multi-view urban coverage. Georeferencing is ensured through an inertial navigation system combined with RTK corrections from the German satellite positioning service (SAPOS) [33], [34]. The dataset captures diverse architectural styles ranging from late 19th century historic buildings to modern urban developments, including residential, commercial, and cultural heritage structures. Following the same OSM-based extraction procedure as used for the UK subset, facade point clouds were clipped per building footprint with a 5 m buffer. The resulting German subset contains 601M points across approximately 66 facades. No additional noise removal was applied.

Singapore. For the Singapore subset, data were collected using the Leica BLK360 [35] laser scanner, which captures high-density point clouds with centimeter-level accuracy. Scanning was performed with the scanner mounted to a backpack and from street-level positions along major urban corridors, carefully selected to cover a broad range of architectural styles from modern high-rise buildings to colonial-era structures. Individual scans were registered using geometric feature alignment and subsequently referenced to geographic coordinates using OpenStreetMap (OSM) data. The registered point clouds were merged into a unified coordinate frame, producing a dense and geometrically consistent dataset suitable for facade-level analysis. As with the other subsets, buildings were extracted using OSM footprints with a 5 m buffer. The Singapore subset contains 1.117B points across approximately 200 facades. No additional noise filtering was applied.

3.2 Hierarchical Semantic Classes Annotations↩︎

Hierarchical Classes. High-fidelity 3D facade semantic segmentation requires both hierarchical class definitions and cross-country consistency. To address the lack of benchmarks with sufficient facade variability and standardized semantics, we adopt the Level of Facade Generalization (LoFG) concept [11] to unify semantic labels across diverse architectural contexts. LoFG defines three levels of detail (LoFG1, LoFG2, and LoFG3) allowing precise formulation of segmentation tasks, supporting transfer learning and label propagation, and enabling consistent evaluation across datasets and regions.

In this work, we cross-country-adopted LoFG to provide 15 detailed classes at LoFG3, aggregated into 5 coarser classes at LoFG2, and a single abstract class at LoFG1. The adaptation ensures that the taxonomy remains valid across countries with distinct architectural styles while adhering to international urban modeling and architectural standards such as CityGML, IFC, and the Art and Architecture Thesaurus (AAT) [6], [36], [37]. LoFG3 includes fine-grained elements such as wall, balcony, molding, decorative elements, stairs, columns, arches, windows, blinds, doors, roof, ground, terrain, interior, and other (mainly noise). LoFG2 merges these into broader functional categories of structural (wall, columns, arch, balcony, stairs), opening (blinds, door, window), decoration (molding, deco), floor (ground surface, terrain), and other elements (roof, other, interior) classes, while LoFG1 abstracts the entire facade.

Annotation Process. All annotations were performed manually by trained annotators with expertise in architecture and geospatial modeling, following the architectural standards described earlier. Labeling was carried out in professional large-scale point cloud software, Cyclone 3DR [38], enabling precise interaction with dense point clouds. Annotating a single facade required approximately 15 hours, resulting in roughly 5,500 hours of manual annotation for the full dataset of about 366 facades. All annotations were further harmonized and cross-checked through annotator consensus, with a correction rate of around 10%, occurring mainly in ambiguous categories (e.g., molding vs.decoration) as defined in our annotation guidelines (Suppl.Sec. 9).

Data Splits. Each country subset is divided into training, validation, and test splits following a 70/20/10 ratio based on the total number of points. Splitting is performed per building, ensuring that complete facades belong to only one subset and that the splits are spatially disjoint. It was also ensured to preserve both dominant and underrepresented semantic classes across all subsets. For reference, mean facade heights are 8.2/10.6/12.1 m, while mean point densities are 11,483/28,713/33,199 pts/m\(^2\) before and 575/427/2,446 pts/m\(^2\) after voxelization (Singapore/UK/Germany, respectively).

UnderOneFacade Benchmark Challenges UnderOneFacade is designed to evaluate facade segmentation along three benchmark axes: (a) cross-country domain shifts, (b) hierarchical semantic granularity, and (c) realistic long-tailed class distributions. As shown in 3, aggregating multiple countries results in a markedly heavier-tailed distribution compared to single-country datasets, opening new challenges. By combining centimeter-accurate geometry, harmonized LoFG semantic labels, and architectural diversity across continents, the benchmark enables systematic evaluation of segmentation robustness beyond single-country datasets and can serve as a generic long-tail benchmark beyond facade segmentation.

These three principles define the core evaluation axes of UnderOneFacade. In the following section, we detail the experimental protocol designed to systematically assess architectural sensitivity, cross-domain generalization, and long-tail robustness across representative point cloud segmentation architectures.

Release. The dataset is released under an open data license (CC BY 4.0, Creative Commons Attribution), hosted on a cloud service (i.e., Google Drive https://drive.google.com/drive/folders/1Yzz7PmyeK1qeOtkTFCfkbw7IEHXcMJo8?usp=sharing) to allow unrestricted community use; the project page with additional details: https://jiangyuanwangyi.github.io/UnderOneFacade_official/. No ethical approval was required for this study as it did not involve human or animal subjects.

4 Experimental Setup↩︎

Baselines. We select representative baselines from the major families of point cloud networks to reflect different underlying working principles. Namely, point-based: PointNet++ [18], KPConv [39]; graph-based: DGCNN [19]; and transformer-based approaches: OctFormer [23] , Point Transformer (abbrv. as PTv1) [20], and Point Transformer v3 (abbrv. as PTv3) [40]. This selection allows us to systematically analyze how performance varies depending on architectural design and feature aggregation strategy. The chosen methods are arguably among the most widely used models in the literature and serve as strong reference points across numerous point cloud benchmarks and methods [8], [9], [11], [28]. Moreover, they are prevalent in related tasks such as facade segmentation and large-scale urban scene understanding [4], [6], [11], making them particularly suitable for comparison in our setting. To isolate the geometric difficulty of facade segmentation, the primary benchmark uses only xyz coordinates as input (radiometric features impact is analyzed separately in 4 and accompanying paragraph).

Hyperparameters. For a fair and controlled comparison, we evaluate all baseline methods under a unified training protocol inspired by ZAHA [11] (e.g., a fixed 100 training epochs), rather than performing per-method hyperparameter tuning. To analyze the impact of facade architecture and acquisition on various methods under identical budgets, we intentionally used unweighted cross-entropy. And to maintain experimental consistency, we train each method for the same number of epochs and apply identical data preprocessing and uniform point sampling strategies across all networks. We also deploy standard metrics to analyze the performance on the two chosen semantic granularities: Overal Accuracy (OA), Precision (P), Recall (R), F1 score (F1), and Intersection over Union (IoU). More implementation details in Suppl.

5 Results and Discussion↩︎

Overall Performance on Hierarchical Semantic. Overall, the results show that fine-grained facade segmentation remains far from solved (1 4).

2pt

Even the strongest model reaches only about 33 mIoU on the combined dataset (DGCNN: 33.4), highlighting the difficulty of long-tailed facade semantics. Interestingly, the graph-based DGCNN consistently outperforms several more recent transformer-based architectures (e.g., PTv1: 15.1 and PTv3: 27.3 mIoU overall), suggesting that current transformer designs may not yet effectively capture the geometric structure of facade point clouds. While several architectures achieve high overall accuracy on individual datasets, their mIoU values remain limited, indicating that dominant classes such as wall or roof are segmented reliably whereas rare facade components remain challenging.

This observation is further confirmed by the per-class analysis in 2. While aggregating classes into the coarser LoFG2 hierarchy slightly increases scores for dominant categories, it does not resolve the underlying segmentation difficulty. For example, the floor class reaches high performance (e.g., 93.3 F1 for PointNet++), whereas structurally meaningful categories remain challenging. The decoration class shows large variability across architectures (45.1 for PointNet++ vs. for PT), and even the aggregated opening class reveals strong architectural sensitivity, with transformer-based models performing substantially worse (PT: 20.1, PTv3: 16.4) than DGCNN (92.1). Similar patterns are observed in the fine-grained LoFG3 classes such as window, door, and molding. Overall, semantic aggregation alone does not overcome the long-tailed distribution and geometric complexity of facade elements.

Cross-Country Architectural Variability.

We analyze how architectural differences across countries affect facade segmentation, where 3 provides class-level comparisons between the most distinct UK and Singapore subsets. At the fine-grained LoFG3 level, several classes exhibit substantial performance drops in Singapore, particularly wall (e.g., PT: 83.7\(\rightarrow\)​43.8, \(-39.9\)) and terrain (97.6\(\rightarrow\)​57.9, \(-39.7\)), indicating a clear architectural domain shift between European and Southeast Asian scenes. Similarly, detailed facade elements such as window degrade strongly (DGCNN: 27.9\(\rightarrow\)​2.6). Conversely, elements such as column (2.8\(\rightarrow\)​59.6) and arch (0.0\(\rightarrow\)​62.0) appear predominantly in Singapore, resulting in large positive gains. At the coarser LoFG2 level, most classes remain comparatively stable across countries, suggesting improved robustness under hierarchical semantic aggregation. However, the opening class exhibits severe degradation across architectures (e.g., PT: 97.7\(\rightarrow\)​6.8, \(-90.9\); DGCNN: 97.6\(\rightarrow\)​20.2), highlighting persistent difficulties in intricate facade openings of Southeast Asian buildings compared to the repetitive patterns of UK Victorian architecture.

Impact of Radiometric Features.

4pt

We analyze the impact of radiometric information by comparing models trained with geometric features only (xyz) against models using additional radiometric attributes, including projected RGB (rgb) values and laser intensity (i). 4 shows that incorporating radiometric features can improve performance, increasing mIoU by +7.3 for PointNet++ and +8.1 for PT. The improvements are most pronounced for facade elements with weak geometric signatures, such as window, door, and molding, where gains exceed +30 F1 for windows and +24 F1 for doors, likely due to consistent color patterns across samples (e.g., white or brown doors and window frames). In contrast, large planar structures such as terrain remain largely unaffected. However, radiometric cues are not universally beneficial. For several classes, adding RGBI features leads to performance degradation, suggesting that models may learn dataset-specific radiometric shortcuts rather than robust geometric representations (e.g., OA -1.3 for DGCNN). Overall, these results highlight that radiometric information can complement geometric features for small facade elements, but may also introduce appearance biases that limit robustness. Additional results are provided in the Suppl.

Cross-Country Generalization and Distribution Shift. We evaluate cross-continental generalization under a zero-shot transfer protocol. Models are trained on one continental subset and evaluated directly on the other without any target-domain adaptation. Two transfer directions are considered: Europe \(\rightarrow\) Asia (training on UK + ZAHA, testing on Singapore) and Asia \(\rightarrow\) Europe. Both training sets contain comparable data volumes of roughly 1.2B points for Asia vs 1.5B points for Europe to ensure that observed differences reflect distributional rather than scale effects. Results are reported at both LoFG2 and LoFG3 levels.

¿tbl:tab:cross95continent95xyz? reveals a clear asymmetry in transfer performance. At LoFG2, Asia\(\rightarrow\)Europe achieves a higher peak OA (65.8%) than Europe\(\rightarrow\)Asia (60.9%), while mIoU remains similar (40.5% vs.%). The asymmetry becomes more pronounced at LoFG3, where Asia\(\rightarrow\)Europe consistently outperforms Europe\(\rightarrow\)Asia across all methods, reaching 51.9% OA and 23.6 mIoU compared to 42.6% OA and 14.8 mIoU.

3pt

Class distribution analysis provides a data-grounded explanation for this effect. The European training set is highly concentrated in dominant classes such as wall and other, which together account for 58.3% of all points, while the Singapore set shows more balanced coverage across facade categories. Consequently, the Asian training data provides stronger supervision for rare classes, which improves transfer performance at the fine-grained LoFG3 level.

These results highlight that balanced semantic coverage in the source domain plays a key role in cross-continental generalization and underline the importance of geographically diverse datasets for robust facade understanding.

5.1 Limitations and Future Work↩︎

UnderOneFacade combines data from multiple sensing configurations, including vehicle-mounted mobile laser scanners, backpack systems, and static terrestrial scanners. While this increases acquisition diversity and realism, it makes it difficult to isolate the influence of individual sensor modalities without controlled acquisitions of the same environment. We acknowledge that the observed cross-country and cross-continental domain shifts likely arise from a combination of factors, including sensor type, point density, acquisition geometry, and architectural style, rather than architecture alone. UnderOneFacade intentionally reflects these entangled real-world conditions through its per-country (sensor-homogeneous) and overall (sensor-heterogeneous) splits. We provide partial disentanglement via (i) uniform 5 cm training-voxelization to reduce density differences across subsets, and (ii) the radiometric feature analysis in 4 and Suppl., which isolates radiometric shifts between the UK and Singapore subsets. Furthermore, the dataset preserves the naturally occurring long-tailed distribution of facade elements. Although this reflects realistic urban scenes, it results in limited training samples for some rare classes. The current benchmark covers three cities, which, despite their architectural diversity, cannot fully represent the global variability of all facade structures. Nevertheless, the proposed hierarchical taxonomy and annotation protocol are designed to be extensible, enabling future integration of additional cities and sensing setups into the benchmark.

6 Conclusion↩︎

We present UnderOneFacade, the largest cross-country 3D facade segmentation benchmark to date, comprising centimeter-accurate point clouds with hierarchical semantic annotations totaling 2.7B labeled points. The dataset combines multiple sensing modalities and architectural styles across countries, producing realistic long-tailed distributions and substantial geometric diversity. Our experiments show that the benchmark is highly challenging for current methods. Notably, transformer-based models do not consistently outperform earlier architectures, with DGCNN achieving the best overall performance, while still the highest IoU reaches only 33%. Radiometric features offer limited benefit and can even degrade performance, indicating reliance on dataset-specific appearance shortcuts. Cross-country evaluation further reveals strong architectural domain shifts, with several models degrading by more than 30 F1 between European and Asian scenes. These results highlight the limited robustness of existing models to architectural variability and long-tailed facade semantics. We believe UnderOneFacade provides an important step toward more robust facade understanding and high definition urban digital twins and generic large-scale semantic segmentation challenges.

Acknowledgements↩︎

UnderOneFacade is the result of a collaborative effort spanning several years, from approximately 2023 to 2026, building upon the earlier development of the ZAHA dataset. We gratefully acknowledge the support of the TUM Global Incentive Fund, which enabled a research stay at the National University of Singapore (NUS) and the acquisition of the Singapore component of the dataset.

We sincerely thank the researchers of the Urban Analytics Lab at NUS for their support during the Singapore data acquisition campaign, and acknowledge the ZAHA team and its annotators for providing the hierarchical foundation that inspired and enabled this work. We are especially grateful to Clara Väth for her meticulous annotation of the Singapore and UK subsets.

Finally, we thank our collaborators at the University of Nottingham for their hospitality and support during the early stages of this project, and the TUM Graduate School which made the visit in Nottingham possible.

This benchmark represents the combined efforts of many researchers, annotators, and collaborators across multiple institutions and countries. We are delighted to see this long-term collaborative effort culminate in the presentation of UnderOneFacade at ECCV 2026.

References↩︎