1 Introduction↩︎

Computer-Aided Design (CAD) floor plans are vector-based design documents composed of fine-grained graphical primitives such as lines and arcs, which explicitly encode the structural details of buildings [1], [2]. Owing to their precise geometric expressiveness, CAD drawings have been widely used throughout Architecture, Engineering, and Construction (AEC) workflows [3], [4].

As intelligent Building Information Modeling (BIM) applications continue to advance, these drawings increasingly need to be parsed into structured semantic elements that support downstream understanding, reasoning, and digital management [5]–[7]. In this context, panoptic symbol spotting has emerged as a fundamental task for CAD analysis, aiming to jointly detect and classify all symbols in a floor plan, including both countable objects (e.g., doors and windows) and stuff-like structures (e.g., walls), at the primitive level [8]. It therefore serves as a key bridge between low-level CAD primitives and high-level BIM semantics [9].

Existing panoptic symbol spotting methods have mainly focused on improving the representation of graphical primitives. Early studies rasterize CAD vectors into images and apply image-based recognition pipelines [8], [10], [11], but rasterization often destroys the geometric and topological details that are critical in CAD floor plan drawings [12]. Later methods instead operate directly on vector primitives and organize them as basic units in different network architectures, such as graph-based, transformer-based, point-based, or line-based models [13]–[21]. Although these paradigms differ in representation and architecture, their common goal is to learn more discriminative primitive-level geometric features, and most of them remain predominantly primitive-centric.

However, CAD floor plan drawings are not purely geometric. In practice, they also contain dense textual annotations that provide direct semantic cues about component identity, dimensions, functions, and material properties, as illustrated in Figure 1 (a). These annotations are often decisive for distinguishing structures that may be geometrically ambiguous, partially incomplete, or heavily overlapped. Therefore, the central problem is not merely how to improve geometric representation of primitives, but how to effectively incorporate CAD textual semantics into primitive-level spotting.

Achieving this is nontrivial because CAD annotations are neither generic natural language nor flat auxiliary labels; rather, they exhibit complex syntactic structures and multi-level semantics. Consequently, existing attempts that rely on naive text encoding [22] or direct text-to-primitive fusion [23] may lose critical semantics. Accordingly, integrating textual annotations into panoptic symbol spotting still faces two key challenges: 1) Concise yet dense CAD annotation syntax hinders unified type–attribute representation. CAD text annotations use highly compressed symbolic codes to convey rich meaning. For example, as shown in Figure 1 (b), in the string ‘FM B 1321’, ‘FM’ denotes the type (fire door), while ‘B 1321’ encodes two attributes—grade (Class B) and dimensions (13 dm × 21 dm). These short strings carry high information density. Off-the-shelf language models (LMs [24] / LLMs [25]) struggle here: their tokenizers often fail to segment such domain-specific patterns reliably [22] and incur computational cost [26]. Simple alternatives (e.g. MLPs [27] or lookup embeddings [28]) also fall short, as they cannot explicitly model and link type and attribute semantics, leading to representation bias. Hence, a key challenge is to efficiently fuse type and attribute within annotations under lightweight computation to yield robust semantic representations. 2) Multi-level semantics of CAD annotations make fusion with fine-grained primitives prone to semantic loss. As shown in  Figure 1 (c), CAD annotations span multiple semantic scales: from the primitive-level (e.g. wall length ‘2650’), to the instance-level (e.g. ‘M0621’), and up to the region-level (e.g. ‘bathroom’). Different annotations have varying scopes of semantic coverage, corresponding to primitives at different geometric scales. Naively injecting high-level annotations into low-level primitive representations can cause a mismatch between semantic scope and geometric receptive field; for example, attaching a region-level tag to a single local primitive can attenuate or mislocalize its information through message passing. Therefore, a key challenge is to ensure that annotations operate at their corresponding semantic scales, avoiding information loss and cross-level interference.

To address the above challenges, we propose a novel panoptic symbol spotting model, TextCAD, which enables the effective embedding and fusion of textual annotations within CAD floor plan drawings. Specifically, the model first embeds graphical primitives and textual annotations separately. For textual annotations, TextCAD introduces a Type-Attribute Correlation Encoder (TACE) to model the semantic correlations between annotation types and their attributes, thereby producing comprehensive textual representations. For multimodal fusion, TextCAD adopts a multi-level downsampling–upsampling architecture to capture design semantics and structural details at different levels. During downsampling, as primitive representations are progressively reduced and evolve to encode broader semantic context, the proposed Multi-level Semantic Filtering (MSF) module selects semantically corresponding textual features for the downsampled primitives and fuses them to achieve cross-modal semantic alignment across different levels. The upsampling stage then performs a symmetric restoration process. Finally, the refined primitive features are fed into a decoder to produce the final spotting results. Extensive experiments on real-world building-design datasets show that TextCAD consistently outperforms existing baselines, especially in the absence of priors (such as layers). Our main contributions are summarized as follows:

• Type-Attribute Correlation Encoder is proposed to capture the compositional semantics of textual annotations. It formulates annotations under a unified type–attribute schema and models their semantic dependencies through attention-based interactions, thereby producing expressive semantic representations.

• Multi-level Semantic Filtering (MSF) is proposed to achieve level-consistent cross-modal fusion. Guided by downsampled primitive representations, MSF selectively injects hierarchy-consistent textual semantics into primitives at different stages, thereby reducing cross-level semantic mismatch and enabling semantically aligned fusion.

• A multimodal panoptic symbol spotting framework is proposed to integrate textual annotations with graphical primitives through a down–upsampling architecture. Experiments on real-world datasets show that it consistently improves the accuracy and robustness of symbol spotting over existing baselines, especially when prior information is unavailable.

2 Preliminary↩︎

This section formalizes the research task and notation for CAD floor plan drawings, covering primitives and text annotations.

2.1 Problem formalization↩︎

Given a CAD floor plan drawing, we consider two fundamental modalities: the graphical primitives \(\mathbf{E}_g\) (including basic geometric elements such as lines, arcs, circles, and ellipses) and the text annotations \(\mathbf{E}_t\). We formalize our target panoptic symbol spotting task as: \[f_{\boldsymbol{\theta}}:\; (\mathbf{E}_g, \mathbf{E}_t)\;\longrightarrow\; (\hat{\mathbf{Y}}, \hat{\mathbf{Z}}),\] where \(\hat{\mathbf{Y}}=\{\hat{y}_i\}_{i=1}^{N_g}\) are the predicted semantic labels for primitives \(e_g^{\,i}\in\mathbf{E}_g\), and \(\hat{\mathbf{Z}}=\{\hat{z}_i\}_{i=1}^{N_g}\) are the corresponding instance indices. Here, \(N_g = |\mathbf{E}_g|\) is the number of graphical primitives and \(\boldsymbol{\theta}\) denotes the learnable model parameters. Each \(\hat{z}_i \in \mathbb{Z}_{\ge 0}\cup\{-1\}\), with \(\hat{z}_i=-1\) reserved for primitives without a specific instance [8].

2.2 Text annotations decomposition↩︎

For each text annotation \(e_t^{i} \in \mathbf{E}_t\), we write \(e_t^i = \{ \mathcal{T}_i, \mathcal{A}_i \}\), where \(\mathcal{T}_i\) represents the annotation-type indicator and \(\mathcal{A}_i \in \mathbb{R}^a\) is the attribute vector separated from the raw annotation. Here, \(a\) denotes the maximum number of properties (e.g. length or grade) captured in the syntactic structures. We additionally attach a mask vector \(\mathcal{M}_i \in \mathbb{R}^a\) to indicate the validity of each attribute. The detailed procedure of attribute separation in 7.1.

3 Methodology: TextCAD↩︎

In this section, we describe the proposed TextCAD. As illustrated in  Figure 2, the workflow of our model is as follows:

For a given CAD drawing, we first decompose it into a set of graphical primitives \(\mathbf{E}_g\) and a set of text annotations \(\mathbf{E}_t\), which are respectively embedded by unimodal encoders. Here, the Type-Attribute Correlation Encoder (TACE) is employed to embed the text annotations information, capturing their semantics by jointly modeling type and attributes, while graphical primitives are embedded based on the line-based method following  [21].

Then we adopt down–upsampling architecture for multimodal fusion. As primitive features are aggregated through a downsampling process, textual features are progressively filtered through the Multi-Level Semantic Filtering (MSF) mechanism to align the semantic hierarchy. As shown at the bottom of  Figure 2, low-level, instance-level and high-level annotations are progressively aligned with the discrete, instance-level, and region-level representations of primitives respectively. Note that this three-stage filtering is only a schematic illustration; in practice, MSF mechanism performs adaptive semantic alignment. The filtered aligned text semantics are then fused with corresponding primitive features at multi-levels and go through the upsampling phase for feature restoration. Finally, the refined primitive features are passed to the decoder for final prediction.

3.1 Unimodal encoder↩︎

We first employ two modality-specific encoders to separately embed the graphical primitives and text annotations.

3.1.1 Type-Attribute Correlation Encoder.↩︎

In CAD floor plan drawings, text annotations are written in highly compressed symbolic forms that specify a type and attach multiple attributes. The type conveys functional semantics, while the attributes encode characteristics such as scope or capacity; both are critical cues. Moreover, different type–attribute patterns induce different attribute semantics. For example, the same numeric attribute string may denote load capacity when associated with Elevator, but dimensions when associated with Door. Conventional NLP embeddings struggle in this high-semantic-density setting: language-model tokenizers are unstable, and simple embedding schemes cannot model the diverse type–attribute patterns.

Therefore, we propose the Type–Attribute Correlation Encoder (TACE), which formulates CAD text annotations under a unified type–attribute representation schema and embeds them by leveraging cross type–attribute attention to associate each type with its attribute bundle, thereby capturing the essential semantics. The robustness of TACE is discussed in 7.2.

Specifically, as shown in  Figure 3, for the text annotations \(\mathbf{E}_t\), TACE first encodes their decomposed \(\mathcal{T} \in \mathbb{R}^{N_t}\) and \(\mathcal{A} \in \mathbb{R}^{{N_t} \times a}\) separately, where \(N_t\) denotes the number of text annotations. This process can be written as: \[\label{eq:type95attr95encoder} \begin{align} \mathcal{T}_s &= \text{Embed}(\mathcal{T}), \\ \mathcal{A}_s &= \text{Concat}(\text{MLP}_j(\mathcal{A}^j)), j\in\{1, \dots ,a\}, \end{align}\tag{1}\] where \(\mathcal{A}^j \in \mathbb{R}^{N_t}\) denotes the \(j\)-th attribute of \(\mathbf{E}_t\). \(\text{Embed}(\cdot)\) is a neural embedding layer that maps discrete indices to continuous vectors, while \(\text{MLP}_j(\cdot)\) consists of two linear layers with RELU activation between them. This process yields the type embedding \(\mathcal{T}_s \in \mathbb{R}^{{N_t} \times 1 \times D}\) and the concatenated attribute embedding \(\mathcal{A}_s \in \mathbb{R}^{{N_t} \times a \times D}\), where \(D\) denotes the embedding dimension.

Then, \(\mathcal{T}_s\) and \(\mathcal{A}_s\) are fed into a fusion module to model the latent semantic dependencies between the type and its attributes. We adopt a masked multi-head attention, where the type embedding provides the queries and the attribute embedding provides the keys and values. Formally, for the \(h\)-th head (\(h=1,\dots,H\)), we project \[\mathbf{Q}_h = \mathcal{T}_s \mathbf{W}^Q_h,\quad \mathbf{K}_h = \mathcal{A}_s \mathbf{W}^K_h,\quad \mathbf{V}_h = \mathcal{A}_s \mathbf{W}^V_h,\] and compute the masked attention weights as: \[\label{eq:multi-head-weight} \mathbf{W}_h = \mathrm{softmax}\!\left(\frac{\mathbf{Q}_h \mathbf{K}_h^\top}{\sqrt{D}} \odot \mathcal{M}\right),\tag{2}\] where \(H\) is the number of heads, and \(\mathcal{M} \in \mathbb{R}^{N_t \times 1\times a}\) is the validity mask for \(a\) attributes. The outputs of all heads are then concatenated and fused with the type embedding to obtain the final semantic embedding \(\mathbf{S}_t \in \mathbb{R}^{{N_t}\times 1 \times D}\): \[\label{eq:final-semantic} \mathbf{S}_t = \mathrm{MLP}\big( \mathrm{Concat}(\mathbf{W}_1 \mathbf{V}_1, \dots, \mathbf{W}_H \mathbf{V}_H) + \mathcal{T}_s \big).\tag{3}\]

Since text annotations also carry corresponding geometric cues, we further inject their geometric features \(\mathbf{F}_t\) (e.g., annotation angle and length) into semantic embeddings \(\mathbf{S}_t\) to yield initial textual embeddings \(\mathbf{X}_t^0 \in \mathbb{R}^{{N_t}\times{D}}\): \[\label{eq:final-text-embeddings} \begin{align} \mathbf{X}_t^0 = \mathbf{S}_t + \text{MLP}(\mathbf{F}_t). \end{align}\tag{4}\] The embedding process of \(\mathbf{F}_t\) is consistent with that of \(\mathbf{F}_g\), as detailed below, where each text annotation corresponds to a single line.

3.1.2 Primitives encoder.↩︎

To preserve the geometric continuity of primitives for accurate shape representations, following the prior work [21], we decompose each primitive \(e_g^{\,i} \in \mathbf{E}_g\) into a set of lines: \[\label{eq:line-decomposition} \begin{align} \mathcal{K}_g^{\,i} = \{\mathit{k}_{\,g}^{i,j}\}_{j=1}^{n_g^{\,i}}, \end{align}\tag{5}\] where \(n_g^{\,i}\) denotes the number of lines contained in the \(i\)-th graphical primitive \(e_g^{\,i}\). For the \(j\)-th line \(\mathit{k}_{\,g}^{i,j}\) in \(e_g^i\), we construct its geometric feature as: \[\label{eq:line-feat} \begin{align} \mathbf{f}_g^{i,j} = (b^{i,j}, d_{x}^{i,j}, d_y^{i,j}, c_x^{i,j}, c_y^{i,j}, c_x^{\,i}, c_y^{\,i}), \end{align}\tag{6}\] where \(b^{i,j}\) denotes the length of the line, \((d_x^{i,j}, d_y^{i,j})\) represents its direction unit vectors, \((c_x^{i,j}, c_y^{i,j})\) is the midpoint of \(\mathit{k}_{\,g}^{i,j}\) and \((c_x^{\,i}, c_y^{\,i})\) denotes the geometric centroid of primitive \(e_g^{\,i}\). Then, \(\mathbf{F}_g = \{\mathbf{f}_g^{i,j} \mid e_g^{\,i} \in \mathbf{E}_g,\; \mathit{k}_{\,g}^{i,j} \in \mathcal{K}_g^{\,i}\}\) is fed into a primitive encoder (MLP) to produce initial primitive-line embeddings: \[\mathbf{X}_g^0 = \mathrm{MLP}(\mathbf{F}_g) \in \mathbb{R}^{N_g^0 \times D},\] where \(D\) matches textual embedding dimension and \(N_g^0=\sum_{i=1}^{|\mathbf{E}_g|} n_g^{\,i}\).

3.2 Semantic Hierarchy Alignment framework↩︎

Since textual annotations cover multiple semantic levels and correspond to graphical primitives at different geometric scales, naively injecting high-level annotations into low-level primitives blurs and suppresses the intended semantic cues. We aim to align semantic hierarchies between two modalities for effective cross-modal fusion, allowing annotations to guide the refinement of corresponding primitives. To this end, we propose the Semantic Hierarchy Alignment Framework through a down-upsampling architecture.

Specifically, to aggregate low-level primitives features into large-scale representations with higher semantics (e.g., progressively merging discrete primitive features into instance-level representations such as doors and further into region-level such as rooms), we adopt Point Transformer V3 [29] to hierarchically downsample primitive-lines, reducing set cardinality while aggregating local neighborhood features. In parallel, a Multi-level Semantic Filtering (MSF) mechanism is applied to select textual features that are semantically aligned with each hierarchy level of primitive representations. Both the downsampling and filtering processes run in parallel for \(L\) layers. The aligned textual and primitive representations are then fused at corresponding layers to achieve fine-grained semantic interaction and feature enhancement.

3.2.1 Primitive-line downsampling.↩︎

Hierarchical downsampling aggregates local primitive features into higher-level representations; subsequent upsampling restores the resolution, giving discrete primitives a broader context. At the \(l\)-th downpooling layer: \[\mathbf{X'}_g^{\,l} = \mathrm{Down}_l(\mathbf{P}_g^{\,l},\,\mathbf{X}_g^{\,l};\, \gamma_l),\] yielding \(\mathbf{X'}_g^{\,l}\in\mathbb{R}^{N_g^{l+1}\times D^{l+1}}\). Here, \(\mathrm{Down}_l(\cdot)\) denotes a downsampling operation implemented via grid-based pooling [29] with the grid size rate \(\gamma_l\), which partitions primitives into voxel clusters and aggregates features within each voxel to produce a higher-level summary. \(N_g^{l+1}\) denotes the number of voxel clusters after downsampling operation, and \(\mathbf{P}_g^{\,l}\) denotes the representative positions of clusters and is used to establish neighborhood relations.

3.2.2 Multi-Level Semantic Filtering.↩︎

As geometric structures are aggregated and the semantic level of primitive features increases through downsampling, textual annotations need to be adaptively filtered to align with corresponding semantic-hierarchy, enabling cross-modal semantic fusion. To address this, we propose a Multi-level Semantic Filtering (MSF) mechanism. Coupled with the progressive downsampling of primitives, MSF uses primitive features as guidance to hierarchically filter textual features at the corresponding geometric scales, thereby mitigating cross-level interference and preserving essential annotation semantics.

Specifically, as shown in Figure 4, during the \(l\)-th semantic filtering layer among the total \(L\) layers, the downsampling primitive-line features \(\mathbf{X'}_{g}^{\,l}\) are used as guidance to filter the textual features \(\mathbf{X}_t^{l} \in \mathbb{R}^{N_t\times D^{l+1}}\) which are projected through \(\mathbf{X}_t^{l} = \mathrm{MLP}_l(\mathbf{X}_t^{0})\). The model first computes the cross-modal attention logits \(\mathbf{C}^l \in \mathbb{R}^{N_t\times{N_g^{l+1}}}\) between \(\mathbf{X}_t^{l}\) and \(\mathbf{X'}_{g}^{\,l}\). Then, the maximum value over the primitive dimension is taken to obtain the semantic relevance \(\mathbf{r}^l \in \mathbb{R}^{N_t}\) of each textual annotation with respect to the primitive lines at the corresponding hierarchy, formulated as: \[\label{eq:cross-attn} \begin{align} \mathbf{C}^l &= \frac{(\mathbf{X}_t^l \mathbf{W}_q) (\mathbf{X'}_{g}^{\,l} \mathbf{W}_k)^T}{\sqrt{D^{l+1}}}, \\ \mathbf{r}^l_m &= \max_{1 \leq n \leq N_g^{l+1}} C^l_{m,n}, \quad m = 1, \dots, N_t, \end{align}\tag{7}\] where \(\mathbf{W}_q\) and \(\mathbf{W}_k\) denote projection matrices. \(C^l_{m,n} \in \mathbf{C}^l\) is the semantic correlation between \(m\)-th text annotation and \(n\)-th primitive-line at layer \(l\).

To enable differentiable and near-binary filtering, we apply hard-concrete relaxation [30] to semantic relevance scores \(\mathbf{r}^l\). The gating-based selection is: \[\label{eq:hard-concrete} \begin{align} \mathbf{G}^l &= \text{HardConcrete}(\mathbf{r}^l, \mathbf{g}), \\ \mathbf{X'}_t^{\,l} &= \mathbf{G}^l \odot (\mathbf{X}_t^l \mathbf{W}_v). \end{align}\tag{8}\] where \(\mathbf{g}\in\mathbb{R}^{N_t}\) is Gumbel noise [31] injected for stochastic exploration, \(\mathbf{G}^l\in[0,1]^{N_t}\) approximates binary gates (near 0: suppressed; near 1: activated), and \(\mathbf{W}_v\) is a learnable projection. This formulation preserves end-to-end differentiability while sparsifying the selected textual features and retaining those most relevant to the current hierarchy. \(\mathbf{X'}_t^{\,l}\) denotes the semantically aligned textual features. Detailed procedures are given in 8.1.

With the primitive-feature guidance and the hard-concrete relaxation, MSF adaptively aligns text–primitive hierarchies, enabling fusion of more semantically precise textual cues.

3.2.3 Multimodal fusion.↩︎

After multi-level semantic filtering and downsampling process, taking the \(l\)-th layer for example, we obtain filtered textual features \(\mathbf{X'}_{t}^{\,l}\) which is semantic aligned with primitive-line features \(\mathbf{X'}_{g}^{\,l}\). We further feed back the semantic aligned \(\mathbf{X'}_{t}^{\,l}\) to fuse with \(\mathbf{X'}_{g}^{\,l}\) through a cross-attention block: \[\label{eq:semantic-fusion} \begin{align} \mathbf{X}_g^{l+1} = \text{AttnBlock}( \text{SerialAttn}(\mathbf{X'}_{g}^{\,l}), \mathbf{X'}_{t}^{\,l}), \end{align}\tag{9}\] where \(\text{SerialAttn}(\cdot)\) denotes serialization attention operation [29] in Point Transformer V3 which captures structural dependencies among primitive lines. \(\text{AttnBlock}(\cdot)\) represents a cross-attention module with spatial embeddings [32] that enable the propagation of textual semantics within the corresponding hierarchical level. A symmetric upsampling pathway [29] is then employed to progressively recover primitive lines and features by coupling with corresponding downsampling stages. During the \(l\)-th upsampling: \[\label{eq:upsample} \begin{align} \tilde{\mathbf{X'}}_{g}^{\,l} &= \text{MLP}(\mathbf{X}_g^{L-l}) + \text{Up}(\mathbf{\tilde{X}}_g^{\,l}), \\ \tilde{\mathbf{X}}_g^{l+1} &= \text{AttnBlock}(\text{SerialAttn}(\tilde{\mathbf{X'}}_{g}^{\,l}), \mathbf{X'}_{t}^{L-l-1}), \end{align}\tag{10}\] where \(\text{Up}(\cdot)\) denotes the upsampling operation implemented via partition-based unpooling operation [29] that propagates lower-resolution features to higher-resolution based on voxel partitioning. \(\tilde{\mathbf{X'}}_{g}^{\,l}\) represents the enhanced primitive features after upsampling and \(\tilde{\mathbf{X}}_g^{\,l}\) is the preceding lower-resolution features to be propagated. Detailed procedures are given in 8.2.

3.2.4 Decoder.↩︎

The final primitive line features \(\tilde{\mathbf{X}}_g^{L}\) are passed to the decoder to obtain the final panoptic spotting results: \[\label{eq:decoder} \begin{align} \mathbf{Y}, \mathbf{Z} = \text{Decoder}(\tilde{\mathbf{X}}_g^{L}), \end{align}\tag{11}\] where \(\mathbf{Y}\) is the class prediction and \(\mathbf{Z}\) is the instance prediction.

Specifically, the decoder first applies group-wise pooling to aggregate line features within each primitive, resulting in primitive-level representations [21]. These features are then enhanced through intra-layer feature fusion to capture layer-wise context [18]. Finally, a OneFormer3D-based [33] head is employed to generate the final predictions.

3.3 Loss function↩︎

We adopt an overall loss function formulated as follows: \[\label{eq:overall-loss} \begin{align} \mathcal{L} = \lambda_{sem}\mathcal{L}_{sem} + \lambda_{cls}\mathcal{L}_{cls} + \lambda_{bce}\mathcal{L}_{bce} + \lambda_{dice}\mathcal{L}_{dice} + \lambda_{c}\mathcal{L}_c, \end{align}\tag{12}\] where the cross-entropy loss \(\mathcal{L}_{sem}\) is employed for semantic segmentation [34] and the classification loss \(\mathcal{L}_{cls}\) is a multi-class cross-entropy loss [34] for instance category prediction. The binary cross-entropy [35] \(\mathcal{L}_{bce}\) and the Dice loss \(\mathcal{L}_{dice}\) [36] are combined to supervise instance-mask prediction.

Besides the task-specific loss, we apply a complexity loss \(\mathcal{L}_c\) [30] to regularize the semantic filtering process. By penalizing the expectation of active gates, it encourages the model to retain those most semantically relevant text annotations while suppressing redundant ones, leading to more discriminative and semantically aligned filtering. Details are given in 8.3.

4 Experiments↩︎

In this section, extensive experiments are conducted to demonstrate the superiority of TextCAD. We also perform ablation studies and case studies to further validate different parts of our model.

4.1 Experimental settings↩︎

We outlines the experimental settings, including the dataset, evaluation metrics, baselines and implementation details.

4.1.1 Dataset.↩︎

Our experiments are conducted on two real-world publicly available CAD floor plan datasets: FloorPlanCAD-V2 [8] and CubiCasa5K [37].

FloorPlanCAD-V2 is the general dataset proposed for panoptic symbol spotting. Compared with its earlier release FloorPlanCAD-V1, this version contains 15,663 CAD drawings spanning a broader range of real-world architectural scenarios, and, importantly, includes abundant textual annotations that are not available in the earlier version. To further evaluate the robustness of our model, we additionally conduct experiments on CubiCasa5K which contains 5,000 CAD floorplans with various textual annotations. We split datasets into training, validation, and test sets with approximate ratios of \(\{6:3:1\}\) for FloorPlanCAD-V2 and \(\{8:1:1\}\) for CubiCasa5K. More details on data preprocessing provided in 9.1.

4.1.2 Evaluation metrics.↩︎

We evaluate with multiple metrics following prior work [8], [14], [16], [17], [20]–[22], [38], including PQ (Panoptic Quality), tailored to panoptic symbol spotting and jointly assessing semantic classification and instance segmentation, as well as PQ-Thing and PQ-Stuff to measure performance on countable thing categories and uncountable stuff categories, respectively. Additionally, the standard classification metrics F1 and wF1 (length-weighted F1) are for semantic spotting. Detailed definitions are in the 9.2.

4.1.3 Baselines.↩︎

We compare TextCAD with baselines of multiple paradigms: image-based method PanCADNet [8], graph-based method GAT-CADNet [14] and CADTransformer [16], point cloud-based method SymPoint [17], SymPointV2 [18] and DPSS [20], line-based method VecFormer [21] and text-incoporated methods PFL-Net [22], TNet [38] and TriNet [23]. Additional descriptions are provided in 9.3. All baselines were reimplemented on both FloorPlanCAD-V2 with textual annotations and CubiCasa5K.

4.1.4 Implementation details.↩︎

Hyperparameters in TextCAD are extensively searched and set to their optimal values, with parameter sensitivity analysis provided in 10. For the TACE, the number of attributes \(a\) is set to 4, the embedding dimension \(D\) is set to 32, and the number of multi-heads \(H\) is set to 4. Within the Semantic Hierarchy Alignment framework, both the number of downpooling layers and the filtering layers \(L\) are set to 5, with grid size rates \([1, 2, 2, 2, 2]\) for primitives lines. Notably, the first layer performs no downpooling or semantic filtering; it only performs projection to aligned feature dimensions.

AdamW optimizer with an initial learning rate of 0.0001 and warm-up ratio of 0.05 is employed and training spans 700 epochs. All experiments, including baselines, are run three times and the average is reported; training uses four NVIDIA RTX A6000 GPUs with a batch size of 2.

4.2 Main performance↩︎

Herein, we analyze performances of various baselines and TextCAD under two experimental settings, with the corresponding results reported in Table 1. “w/ prior” denotes that the model is provided with prior primitive attributes, namely auxiliary properties of graphical primitives, such as layer assignments or color information, that may provide category-indicative cues. “w/o prior” denotes that such prior attributes are not used.

Some methods (e.g. CADTransformer, SymPoint) are not included under the “w/ prior” setting because they do not inherently support the use of prior primitive attributes. Notably, for CubiCasa5K, results under “w/ prior” setting are not reported, as the dataset itself lacks such prior information. For clarity, all results are multiplied by 100.

3.2pt

4.2.1 TextCAD performance↩︎

Based on  Table 1, we draw the following key observations:

i) Under both settings, TextCAD consistently achieves the best performance across all metrics, outperforming the strongest baseline by nearly 2 percentage points on overall PQ which serves as the comprehensive evaluation of panoptic symbol spotting. This demonstrates TextCAD is able to leverage critical cues within textual annotations effectively to improve overall performances as well as both countable thing and uncountable stuff categories.

ii) Compared with methods that rely solely on graphical primitives  [8], [14], [16]–[18], [20], [21], TextCAD delivers consistent gains across all metrics. These improvements stem from leveraging key semantic cues in abundant CAD textual annotations, such as functional descriptions and dimensional specifications, which enrich primitive representations, help distinguish incomplete geometry structures or overlapping primitives, and ultimately enable more accurate detection. In contrast, primitive-only approaches operate from a single, limited modality and overlook these crucial annotation semantics, thereby hindering spotting performance.

iii) Compared with methods that incorporate textual annotations [22], [23], [38], the superiority of TextCAD mainly stems from two aspects: (1) Accurate and lightweight textual semantic modeling: instead of relying on language models (e.g., PFL-Net), the proposed TACE adopts a lightweight architecture that explicitly captures type–attribute correlations, thereby producing more robust textual semantic. (2) Hierarchy-aligned cross-modal fusion: rather than simply injecting textual annotations via direct concatenation fusion (e.g., TriNet or TNet), TextCAD treats textual annotations as an independent modality and employs MSF to adaptively filter and align cross-modal semantics, enabling multimodal fusion under semantically aligned hierarchies. Together, these two advantages allow TextCAD to provide more accurate and direct semantic cues.

iv) Under both “w/o prior” and “w/ prior” settings, TextCAD outperforms existing methods across all metrics, demonstrating strong robustness to prior information variations. Particularly on FloorPlan-V2, TextCAD suffers the smallest decrease in overall PQ when prior is unavailable. Since CubiCasa5K does not provide prior information, its “w/o prior” evaluation is not applicable. This verifies TextCAD can achieve robust predictions by constructing structured textual representations and effectively fusing semantic cues with geometric features, compensating for the absence of prior information.

4.2.2 Efficacy analysis↩︎

To assess the impact of TACE, we further integrate it into three representative baselines: CADTransformer [16], SymPoint [17] and VecFormer [21]. As shown in  Figure 5, on FloorPlanCAD-V2, adding TACE yields notable gains without materially increasing parameters, highlighting the value of incorporating annotations and demonstrating TACE’s ability to link type-attribute pairs and embed key semantic cues in a lightweight manner.

4.3 Ablation study↩︎

We conduct several ablation studies on FloorPlanCAD-V2 to further validate the effectiveness of certain designs in TextCAD.

4.3.1 Type-Attribute Correlation Encoder module.↩︎

To further verify the necessity of TACE in jointly modeling the semantic correlations between type and attributes, we replace TACE by “TypeEmb” (encoding only the type via embedding layer), “TypeAttrMLP” (concatenating type and attributes followed by MLP) and “BertEmb” (encoding textual content using BERT[24]). As shown in  Table 2, TACE consistently achieves the best performance, confirming the necessity of jointly capturing latent semantic correlations between type and attributes. In addition, TACE remains lightweight instead of employing the tokenizer of LM.

4.3.2 Multi-Level Semantic Filtering mechanism.↩︎

To evaluate the effectiveness of semantic hierarchy alignment, we ablate the MSF design in  Table 3. Notation: ‘Text–GP’ treats textual annotations as part of the graphical-primitive modality; ‘Text–Uni.’ feeds text as a separate unimodal input; ‘Multi-L.’ enables text injection at multiple hierarchy levels; ‘Filt.’ applies multi-level semantic filtering; ‘GP-G.’ uses primitive features to guide filtering.

From  Table 3, we derive the following key observations: treating CAD drawings as multi-modal (rather than folding text into the GP stream) yields better performance; however, naively injecting text at multiple levels without filtering degrades accuracy because misaligned text introduces cross-level noise. Enabling multi-level semantic filtering mitigates this issue and improves spotting performance, and guiding the filtering with primitive features (GP-G.) provides the best alignment and further gains by directly coupling textual semantics with primitive geometry.

More validation of semantic hierarchy alignment is provided in 11.

4.4 Case study↩︎

The qualitative results on FloorplanCAD-V2 are visualized in  Figure 6, which includes (a) ground truth, (b) predictions from TextCAD, and (c/d/e) those from three representative baselines. In Figure 6 (b), green regions highlight key regions of interest, and arrows represent results within these regions are affected by corresponding text annotations. Red regions in  Figure 6 (d),  Figure 6 (c) and  Figure 6 (e) indicate wrong predictions from baselines models.

It can be clearly observed that TextCAD shows a high degree of consistency with the ground truth (GT). In highlighted regions, TextCAD demonstrates a clear advantage over baselines by explicitly leveraging the semantic cues contained in annotations. Specifically, TextCAD achieves more robust and accurate results, particularly in complex regions with dense or overlapping line structures, or instance regions where graphical structures are partially incomplete. For instance, ‘Washroom’ helps accurately identify toilet-related primitives through interaction with corresponding graphical elements, while ‘Staircase-11’ enables precise recognition of structures such as stairs and handrails by capturing the underlying semantic relationships indicated by the annotations. Additional results are provided in 12.

5 Related work↩︎

Various research tasks were explored on CAD drawings [8], [15], [39], [40]. We focus on panoptic symbol spotting which was first proposed in [8] to address the limitation of focusing solely on countable things while neglecting uncountable stuffs [2]. Existing methods fall into the following categories:

Early studies take conventional computer vision methods [8], [10], [11] by converting CAD drawings into raster images. PanCADNet [8] employs Faster R-CNN for instance recognition and Graph Convolutional Network (GCN) for semantic detection. However, converting vector primitives to pixels losses geometric precision and fine-grained structural details.

Subsequent studies decompose vector drawings into primitives and leverage Transformer-based [16] or GNN-based methods [14], [41], [42]. CADTransformer [16] modifies Vision Transformer [43] to update primitive features, while GAT-CADNet [14] models primitives as graph nodes by employing Graph Attention Network (GAT) for feature propagation. Vector-based methods better preserve internal CAD structures but face challenges on complex drawings with numerous primitives due to memory constraints.

SymPoint [17] treats CAD drawings as point cloud by abstracting primitives as point sets to enhance feature extraction. Subsequent studies follow this direction  [18]–[20], but emphasize only on graphical features and structural relations, neglecting the semantics in textual annotations.

Recently, VecFormer [21] proposed line-based representation to preserves the geometric continuity and enhance shape representation, but still stranded by the same issues that textual annotations semantics had been ignored.

A few studies incorporate textual annotations but via language models such as BERT tokenizer [22] and CLIP text encoder [44]. Others either naively concatenate text categories to nearby primitive features [23] or simply embeds them from feature maps without structured semantic modeling or effective cross-modal fusion [38]. They overlook the hierarchical semantics and syntactic structure of annotations, limiting their semantic.

In contrast, our method syntactically represents textual annotations to capture latent semantics, and performs multi-level semantic filtering for cross-modal alignment and fusion, thereby enabling semantically enriched for discriminative primitive representations.

6 Conclusion↩︎

In this work, we propose TextCAD, a multi-modal framework that jointly integrates textual annotations and graphical primitives for panoptic symbol spotting. The proposed Type–Attribute Correlation Encoder (TACE) embeds rich annotation semantics by explicitly modeling intrinsic correlations between the type and attributes, yielding expressive semantic representations. The Semantic Hierarchy Alignment framework applies Multi-level Semantic Filtering (MSF) with primitive downsampling, enabling semantically hierarchy-consistent cross-modal alignment for effective modality fusion. Experiments show that TextCAD surpasses prior state of the art by nearly 2 percentage points on standard metrics, validating the benefit of textual annotations incorporation. Ablation studies and case analyses further confirm the effectiveness and lightweight nature of TACE, and the necessity of MSF. Moreover, TextCAD generalizes well to diverse annotation styles: TACE accommodates varied type-attribute patterns, and the alignment module adapts across semantic levels. Future work will focus on more effective cross-modal fusion to improve reliability.

7 Further discussion↩︎

Here we discuss the type-attribute decomposing and the extensibility.

7.1 Type-attribute decomposing↩︎

We perform preprocessing on textual annotations to extract their type and attribute information.

Based on the observation and syntactic analysis of CAD textual annotations, we categorize them into various types according to their syntactic structure and associated attribute patterns. Each type is paired with a specific set of attributes, forming distinct type–attribute patterns. For instance, type Elevator is attached by a numeric attribute denotes load capacity, and type Door is paired with two numeric attributes represent dimensions. The number of attributes \(a\) is set to 4, while among four attributes \(\mathcal{A}^j, j\in\{1\dots4\}\), \(\mathcal{A}^1\), \(\mathcal{A}^2\) and \(\mathcal{A}^3\) denote numerical information and \(\mathcal{A}^4\) is grade information.

We design an automated parsing framework to separate CAD text annotations into their corresponding types \(\mathcal{T}\) and attributes \(\mathcal{A}^j\). Based on predefined type–attribute structures, our tool employs regular-expression [45] based pattern matching to automatically identify and extract the attributes associated with different type. This process yields 166 types of textual annotation with specific attribute combinations for FloorPlanCAD-V2 and 86 types for CubiCasa5K. We present 13 representative examples in Table 4.

3pt

The extracted type \(\mathcal{T}\) and attributes \(\mathcal{A}^j\) are independently embedded. The numerical attributes \(\mathcal{A}^1\), \(\mathcal{A}^2\) and \(\mathcal{A}^3\) are embedded using the corresponding \(\text{MLP}_j\), while grade attribute \(\mathcal{A}^4\) is encoded via an neural embedding layer.

The proposed type-attribute representation schema can be explained by the concept of factorized representation: complex semantics are typically composed of multiple underlying factors of variation [46], and explicitly disentangling these factors improves both expressiveness and generalization.

Our study formulates CAD textual annotations as type–attribute syntactic structure. This formulation can be viewed as a factorized representation, which decomposes entangled semantics into multiple composable subspaces, thereby reducing representation complexity and improving generalization.

7.2 Extensibility and robustness↩︎

TACE and the unified type–attribute representation schema are not tied to a specific dataset; instead, it exhibits strong generality and extensibility.

Specifically, the defined types capture representative textual annotation patterns in CAD floor plans. Rather than a fixed taxonomy, this type–attribute scheme provides a general structured representation framework that can be readily extended to other datasets by adding new types and defining their corresponding attributes structures under the principles of this schema.

Additionally, in real-world design, CAD annotations typically follow engineering drafting conventions, resulting in regular and semi-structured patterns. TACE and its pipeline are specifically designed for such realistic and regular patterns. The robustness of this design is further supported by experimental results on two real-world CAD datasets. Moreover, unmatched cases during TACE decomposition are mostly due to anomalous human annotations, which are filtered to avoid noisy semantics. This information loss is also limited, as neighboring textual cues provide complementary context.

8 Model details↩︎

We make detailed supplements to our model, including multi-level semantic filtering, multimodal fusion, decoder, and loss function.

8.1 Multi-Level Semantic Filtering↩︎

Semantic filtering employs hard-concrete relaxation to obtain approximated binary gates through the differentiable process. During the \(l\)-th semantic filtering layer (\(l=0\cdots L-1\)), we first introduce randomness into \(\mathbf{r}^l\) by adding Gumbel noise \(\mathbf{g}\) [31] for stochastic exploration, and then obtain a smooth activation probability \(\mathbf{s}^l \in \mathbb{R}^{N_t}\), formulated as: \[\label{eq:gumbel-noise} \begin{align} \mathbf{g} &= -\log(-\log(\mathbf{u})), \mathbf{u} \sim \mathrm{Uniform}(0,1)^{N_t}\\ \mathbf{s}^l &= \sigma\Big(\frac{\mathbf{r}^l + \mathbf{g}}{\tau}\Big), \end{align}\tag{13}\] where \(\tau\) and \(\sigma\) denote the temperature and the , respectively. The discrete gating behavior is then further approximated as: \[\label{eq:stretch-clamp} \begin{align} \mathbf{G}^l = \text{min}(1, \text{max}(0, \mathbf{s}^l (\zeta - \gamma) + \gamma)), \end{align}\tag{14}\] where \(\gamma\) and \(\zeta\) are the lower and upper predefined stretch limits. The approximated binary gates \(\mathbf{G}^l \in [0,1]^{N_t}\) is generated by first linearly mapping \(\mathbf{s}^l\) to an extended interval \((\gamma, \zeta)\) and then clipping it to the range of \([0,1]\).

8.2 Multimodal fusion and decoder↩︎

Here we make a detailed supplement to the process of multimodal fusion, primitive-line upsampling and the decoder.

8.2.1 Multimodal fusion↩︎

After the \(l\)-th multi-level semantic filtering and downsampling process, we obtain filtered textual features \(\mathbf{X'}_{t}^{\,l}\) which is semantic aligned with primitive-line features \(\mathbf{X'}_{g}^{\,l}\). we fuse \(\mathbf{X'}_t^{\,l}\) back with \(\mathbf{X'}_g^{\,l}\): \[\label{eq:app-semantic-fusion} \begin{align} \mathbf{X}_g^{l+1} = \text{AttnBlock}( \text{SerialAttn}(\mathbf{X'}_{g}^{\,l}), \mathbf{X'}_{t}^{\,l}), \end{align}\tag{15}\] where \(\text{SerialAttn}(\cdot)\) denotes the serialization attention operation in Point Transformer V3 which performs self-attention within patches derived from the serialized point sequence [29].

\(\text{AttnBlock}(\cdot)\) is implemented as a cross-modal attention module, where primitive-line features \(\mathbf{X'}_g^{\,l}\) serve as queries and textual features \(\mathbf{X'}_t^{\,l}\) serve as keys and values. This enables textual semantics and geometric representations to interact at the same hierarchical level. To enhance spatial awareness, spatial embeddings are further incorporated into attention computation and feature aggregation, allowing nearby textual features to exert stronger influence on each primitive and promoting more semantically relevant cross-modal interactions.

8.2.2 Primitive-line upsampling↩︎

The upsampling structure is symmetric to downsampling stages. At the \(l\)-th upsampling layer, the low-resolution features \(\tilde{\mathbf{X}}_g^{\,l}\) are interpolated onto higher-resolution: \[\label{eq:interplote} \begin{align} \tilde{\mathbf{X'}}_{g}^{\,l} &= \text{MLP}(\mathbf{X}_g^{L-l}) + \text{Up}(\mathbf{\tilde{X}}_g^{\,l}). \end{align}\tag{16}\] \(\text{Up}(\cdot)\) denotes an unpooling operation [29] based on the voxel clustering induced in the symmetric downsampling stage. The restored features \(\tilde{\mathbf{X'}}_{g}^{\,l}\) are then fused with the aligned textual features \(\mathbf{X'}_{t}^{L-l-1}\): \[\label{eq:fusion} \begin{align} \tilde{\mathbf{X}}_g^{l+1} &= \text{AttnBlock}(\text{SerialAttn}(\tilde{\mathbf{X'}}_{g}^{\,l}), \mathbf{X'}_{t}^{L-l-1}), \end{align}\tag{17}\] where \(\mathbf{X'}_{t}^{L-l-1}\) and \(\tilde{\mathbf{X'}}_{g}^{\,l}\) share the aligned semantic level. The initial input \(\tilde{\mathbf{X}}_{g}^{1}\) is set to the primitive point feature \(\mathbf{X}_{g}^{L}\) from the final downsampling stage.

8.2.3 Decoder↩︎

After upsampling, we obtain final primitive-line features \(\tilde{\mathbf{X}}_g^{L} \in \mathbb{R}^{\sum_{i=1}^{|E_g|} n_g^{\,i} \times D}\), which are passed to the decoder for the final panoptic spotting prediction.

Specifically, the decoder first adopts group-wise pooling strategy [21] \(\mathbf{F}_g = \text{Pool}(\tilde{\mathbf{X}}_g^{L})\) to aggregate line features within each primitive into primitive-level representations \(\mathbf{F}_g \in \mathbb{R}^{N_g \times D}\) where \(N_g=|E_g|\).

Then the decoder adopts layer enhancement operation [18] through aggregating the primitives features within the same layer by max pooling, average pooling and attention pooling as the layer-wise context, which is added back get the enhanced primitive feature \(\tilde{\mathbf{F}}_g = \mathbf{F}_g + \text{Pool}_{layer}(\mathbf{F}_g)\).

Next, we adopt a OneFormer3D [33] based head to obtain the final prediction. It composed \(U\) layers and first inits the learnable queries \(\mathbf{Q}^0 \in \mathbb{R}^{O \times D}\) by query selection where \(O\) is the number of symbol queries.

During the \(u\)-th layer within the decoder, the queries \(\mathbf{Q}^u\) are refined through self-attention and cross-attention with primitive features \(\tilde{\mathbf{F}}_g\) serves as key and value. Final semantic and instance predictions is obtained via: \[\label{eq:query-result} \begin{align} \mathbf{I}^U &= f_{cls}(\mathbf{Q}^{U}), \\ \mathbf{M}^U &= f_{inst}(\mathbf{Q}^{U}) (\tilde{\mathbf{F}}_g)^\top, \end{align}\tag{18}\] where \(f_{cls}\) and \(f_{inst}\) are based heads producing the semantic output \(\mathbf{I} =\mathbf{I}^U \in \mathbb{R}^{O \times C}\) and instance mask \(\mathbf{M} = \mathbf{M}^U \in \mathbb{R}^{O\times N_g}\). \(C\) is the number of categories. The decoder also applies post-processing strategy Branch Fusion Refinement [21] to resolve inconsistencies.

Finally, to align with the task formalization, we integrate \(\mathbf{I}^U\) and \(\mathbf{M}^U\) to produce the final panoptic spotting result for primitives: \[\label{eq:final-result} \begin{align} \mathbf{Y}, \mathbf{Z} = \Phi(\mathbf{I}^U, \mathbf{M}^U), \end{align}\tag{19}\]

8.3 Loss function↩︎

We apply the classification loss \(\mathcal{L}_{cls}\), cross-entropy loss \(\mathcal{L}_{sem}\), binary cross-entropy loss \(\mathcal{L}_{bce}\), dice loss \(\mathcal{L}_{dice}\) and to jointly optimize the task branches. \(\mathcal{L}_{cls}\) and \(\mathcal{L}_{sem}\) supervise semantic predictions: \[\label{eq:losses-semantic} \begin{align} \mathcal{L}_{cls} = \mathrm{CrossEntropy}(\mathbf{I}, \mathbf{I}^{gt}), \mathcal{L}_{sem} = \mathrm{CrossEntropy}(\mathbf{Y}, \mathbf{Y}^{gt}). \end{align}\tag{20}\] \(\mathbf{I},\mathbf{I}^{gt}\in\mathbb{R}^{O\times C}\) are the predicted and ground-truth classes of \(O\) symbols over \(C\) categories. \(\mathbf{Y},\mathbf{Y}^{gt}\in\mathbb{R}^{N_g\times C}\) are the semantic predictions and labels over \(N_g\) primitives. \(\mathcal{L}_{bce}\) and \(\mathcal{L}_{dice}\) supervise instance masks predictions: \[\label{eq:losses-mask} \begin{align} \mathcal{L}_{bce} = \mathrm{BinaryCrossEntropy}(\mathbf{M}, \mathbf{M}^{gt}), \mathcal{L}{dice} = 1 - \frac{2\langle \mathbf{M}, \mathbf{M}^{gt} \rangle}{\|\mathbf{M}\|_{1} + \|\mathbf{M}^{gt}\|_{1}}. \end{align}\tag{21}\] \(\mathbf{M},\mathbf{M}^{gt}\in\mathbb{R}^{O\times N_g}\) are predicted and ground-truth instance mask of \(O\) symbols over \(N_g\) primitives. \(\langle \cdot, \cdot \rangle\) denotes the element-wise product summed over all elements, and \(\|\cdot\|_1\) denotes the sum of all elements.

Besides task-specific losses, we apply complexity loss \(\mathcal{L}_c\) [30] to regularize semantic filtering process and promote sparsity in textual feature selection: \[\label{eq:l0-loss} \begin{align} \mathcal{L}_c = \sum_{l=1}^{L}\sum_{j=1}^{N_t} \sigma\!\left(r_j^l - \tau \log (\zeta - \gamma) \right), \end{align}\tag{22}\] where \(r_j^l \in \mathbf{r}^l\) denotes the semantic relevance of the \(j\)-th annotation at \(l\)-th filtering layer. \(\gamma\) and \(\zeta\) are predefined stretch limits. \(\tau\) is the temperature and \(\sigma(\cdot)\) is the activation. This term estimates the expectation of active gates across filtering layers, imposing an adaptive sparsity constraint. Minimizing \(\mathcal{L}_c\) encourages the model to retain those most semantically relevant textual features, yielding a more discriminative semantic filtering mechanism.

In our experiments, we set the loss weights \(\lambda_{cls}:\lambda_{bce}:\lambda_{dice}:\lambda_{cls}:\lambda_{c}=2:5:5:5:0.0001\). The task-specific loss weight is set empirically according to [21]. Since \(\mathcal{L}_c\) regularizes semantic filtering rather than directly supervising prediction, we assign it a small weight to balance regularization and task optimization. We evaluate \(\lambda_c\in\{1,0.1,0.01,0.001,0.0001\}\) and find that \(0.0001\) achieves the best performance and stability.

9 Datasets, metrics and baselines↩︎

Here we introduce additional details on datasets and evaluation metrics.

9.1 Dataset↩︎

This section outlines the data preprocessing steps for two datasets.

FloorPlanCAD-V2. We utilize the large-scale FloorPlanCAD-V2[8] which is designed for panoptic symbol spotting. Compared with its earlier version, this release offers improved scale and textual semantic diversity

FloorPlanCAD-V2 contains 35 line-level annotated categories, distinguishing between 30 countable “thing” classes (e.g. doors, windows) and 5 uncountable “stuff” classes (walls, curtain wall, parking spot, row chairs and railing). We follow the official script¹ to assign semantic labels and instance indices to primitives, while additionally extracting relevant textual annotation information in paralle. We split the dataset into train, validation and test sets with a approximate \(\{6:3:1\}\) proportion, yielding \(\{9533:4597:1533\}\).

CubiCasa5K. To further evaluate the robustness of our method, we use CubiCasa5K which is a real-world CAD dataset for floorplan image analysis.

Since CubiCasa5K was not originally designed for this task, we refer to the official script² and the icon class mapping in the benchmark, obtaining 10 countable “thing” classes (window, door, closet, electrical appliance, toilet, sink, sauna bench, fireplace, bathtub and chimney) and 2 uncountable “stuff” classes (wall, railing), which serve as semantic labels for primitives. For instance indices, all primitives belonging to the same symbol are assigned the same instance index for thing categories. For stuff categories, primitives are assigned only semantic labels and are not distinguished by instance indices.

We split CubiCasa5K into training, validation, and test sets with \(\{4200:400:400\}\) samples, following the benchmark [37].

9.2 Evaluation metrics↩︎

Following [8], we use the comprehensive measurement PQ (Panoptic Quality), which is specifically designed for panoptic symbol spotting task, to jointly evaluate instance recognition and semantic segmentation: \[\begin{align} PQ = & \frac{|TP|}{|TP| + \frac{1}{2}|FP| + \frac{1}{2}|FN|} \times \frac{\sum_{(s_{pred}, s_{gt}) \in TP} \text{IoU}(s_{pred}, s_{gt})}{|TP|} \\ = & \frac{\sum_{(s_{pred}, s_{gt}) \in TP} \text{IoU}(s_{pred}, s_{gt})}{|TP| + \frac{1}{2}|FP| + \frac{1}{2}|FN|}, \end{align}\] where a prediction symbol \(s_{pred}\) (a set of primitives) is considered a match with ground truth \(s_{gt}\) if they share the same predicted category and \(\text{IoU} (s_{pred}, s_{gt})>0.5\). The intersection over union (IoU) score are computed as follows: \[\begin{align} \text{IoU}(s_{pred}, s_{gt}) = \frac{\sum_{e_g^i \in s_{pred} \cap s_{gt}} \log(1 + L(e_g^i))}{\sum_{e_g^j \in s_{pred} \cup s_{gt}} \log(1 + L(e_g^j))}, \end{align}\] where \(e_g^i\) and \(e_g^j\) denote graphical primitives and \(L(\cdot)\) calculates the arc length.

We additionally report PQ-Thing and PQ-Stuff to measure the performance on countable thing and uncountable stuff categories, respectively. Semantic symbol spotting is evaluated using F1 and length-weighted F1 score wF1, where longer primitives receive larger weights.

9.3 Baselines↩︎

We compare our model against ten baselines spanning multiple paradigms:

• PanCADNet [8] uses a CNN backbone for raster feature extraction, GCN for semantic detection, and Faster R-CNN for instance recognition.

• CADTransformer [16] represents primitives from rasterizd feature maps extracted by a CNN backbone and employs a modified Vision Transformer [43] with neighborhood-attention mechinism to refine the features.

• GAT-CADNet [14] constructs graph with primitives as nodes and employs GAT for feature propagation. It also establishes edge features to capture spatial relationships, enabling more effective message passing.

• SymPoint [17] models graphical primitives as point cloud structure to enhance the feature extraction. It uses Point Transformer to update features and employs Mask2Former decoder to obtain final predictions.

• SymPointV2 [18] improves SymPoint by encoding layer assignments into primitive features and proposes a position-guide training method to accelerates the convergence of the model.

• DPSS [20] follows the point cloud structure but incorporates raster features from images to enhance primitive features, improving the performance and the robustness of the model.

• VecFormer [21] proposes line-based representation of graphical primitives, better preserving their original geometric structure. It also utilizes branch fusion refinement strategy to improve prediction reliability.

• PFL-Net [22] simply employs a pre-trained language model to embed text annotations, failing to capturing the semantics implicit in the complex syntactic structures of CAD annotations.

• TNet [38] treats text annotations as a distinct primitive type and embeds them via feature maps, failing to exploit semantic value.

• TriNet [23] relies on hand-crafted fusion of different modalities, where texts are merely assigned to graphical primitives by nearest-coordinate matching, without reasonable encoding and cross-modal alignment.

10 Parameter sensitivity analysis↩︎

We conducted experiments to report three key hyperparameters.

10.1 Embedding dimension↩︎

TACE embeds text annotations into \(\mathbf{X}_t^0 \in \mathbb{R}^{{N_t}\times{D}}\), where dimension \(D\) controls the capacity to capture textual semantic. We evaluate different dimensions in Table 5. The best performance is achieved at \(D=32\): smaller dimension limits the representation capacity, leading to insufficient semantic encoding, while a larger dimension introduces redundant information.

10.2 Primitives downsampling layer↩︎

During Semantic Hierarchy Alignment, graphical primitive-lines are downsampled into higher-level representations. We evaluate the number of downsampling layers of \(\{7, 5, 3\}\) with grid size rate of \([1,2,2,1,2,1,2]\), \([1,2,2,2,2]\) and \([1,4,4]\) in Table 6. Fewer layers provide insufficient hierarchical abstraction, while excessive downsampling layers may cause redundant transformations and feature smoothing. Five layers perform best.

10.3 Semantic filtering layer↩︎

During Multi-Level Semantic Filtering, textual features are progressively filtered to align with primitive features. The number of semantic filtering layers \(L\) controls the hierarchical adaptivity of semantic alignment. Based on the parameter experiments of primitive downsampling layers, we evaluate different values of \(L\) under five primitive downsampling layers. As shown in  Table 7, \(L=5\) achieves the best performance, indicating that retaining five semantic filtering layers best exploit adaptive semantic alignment.

11 Additional case study↩︎

We further illustrate the effectiveness of our proposed MSF module through an interesting case study.

 Figure 7 illustrates the semantic filtering results of the third, forth and fifth layers within MSF, shown from left to right (in our configuration, the frist layer performs no actual filtering but only projection to align dimensions, while the second layer mainly serves as a transitional stage where the semantic hierarchy has not yet been clearly differentiated). We visualize the retention degree of each textual annotation at layer \(l\) according to the approximated binary gate \(\mathbf{G}^l\): darker colors indicate higher retention gate (closer to 1), and lighter colors indicate lower retention gate (closer to 0).

As shown in  Figure 7, filtering results are largely consistent with the semantic hierarchy. Primitive-level annotations such as ‘300’ exhibit gradually weakened semantics as filtering deepens; semantics of instance-level annotations such as ‘DT-2’ remain highly preserved in intermediate layers; region-level annotations such as ‘Entrance’ tend to be increasingly retained in higher layers. In parallel, downsampling naturally produces higher-level primitive feature abstractions, which is a standard property, resulting in aligned semantic across modalities. Note that the retention gate of instance or region-level is generally higher than primitive-level, probably because higher-level annotations convey more informative and semantically richer cues.

12 Additional qualitative results↩︎

Additional qualitative results are shown in Figure 8. TextCAD achieves more accurate predictions in challenging regions across diverse scenarios by leveraging underlying semantic cues from relevant textual annotations.

References↩︎

---

1. https: //github.com/VITA-Group/CADTransformer/↩︎

2. https://github.com/CubiCasa/CubiCasa5k↩︎