1 Introduction↩︎

Three-dimensional semantic localization is a fundamental prerequisite for intelligent systems to perceive, reason, and act in real-world environments. While classical approaches emphasize global pose estimation or reconstruction, modern applications demand precise identification of semantically meaningful object parts. In robotics, grasp planning depends on functional regions such as handles; in CAD and manufacturing, simulation and editing require selecting exact geometric entities including fillets, chamfers, and load-bearing surfaces. These targets are defined as discrete topological elements present in a 3D structure and their spatial identification is crucial for the relevant task.

Recent vision-language models have demonstrated strong semantic reasoning in images [1]–[4], and a few have explored reasoning-based segmentation [5]–[7]. However, using these frameworks for 3D segmentation and entity localization depends critically on how visual information is presented to the VLM. In 3D settings, the same underlying geometry can appear either highly informative or severely ambiguous depending on the viewpoint. Self-occlusion, depth conflation, and thin structural elements often render relevant entities visually indigestible from arbitrary projections. Thus, language-driven geometric localization is not merely a segmentation problem, but a question of making structured 3D information interpretable to current VLMs that process visual information in 2D.

In this work, we address this visual interpretability problem for VLMs in the context of entity localization for mesh objects, derived from CAD. We introduce a prompt-conditioned ranking framework that estimates entity-specific geometric visibility across candidate views, presenting the reasoning segmentation model with geometrically digestible evidence that helps it achieve accurate entity localization, as illustrated in Fig. 1. In summary, our contributions are threefold:

• We create a dataset and formalize the task of locating geometric entities in CAD meshes guided via natural language.

• We propose GELviews, a prompt-conditioned view ranking framework that prioritizes viewpoints given to reasoning segmentation 2D VLMs, based on entity-specific geometric visibility.

• We introduce a CAD-agnostic framework MV-GEL for localizing these geometric entities using text prompts directly on the mesh.

We evaluate the results of our framework by introducing a mesh-aware lifting strategy that maps 2D segmentation masks to discrete topological mesh entities (mesh faces and edges).

2 Related Work↩︎

2.1 Vision-Language Models and Referring Segmentation↩︎

The integration of visual and textual modalities has driven remarkable progress in open-vocabulary image understanding. Foundational vision-language models (VLMs) like CLIP [8], ALIGN [9], and BLIP-2 [10] align image and text embeddings, enabling zero-shot recognition. Building on this, Large Multimodal Models (LMMs) such as LLaVA [1], InstructBLIP [11], and Qwen-VL [2] have demonstrated sophisticated visual reasoning capabilities.

In parallel, referring expression segmentation (RES) has evolved from relying on complex multi-modal fusion networks [12]–[15] to leveraging foundation models like the Segment Anything Model (SAM) [16]–[18]. Successors like LISA [5], PixelLM [6], Osprey [7], and GSVA [19] further improve semantic segmentation via pixel-level dense reasoning using Vision Language Models. However, these models are trained predominantly on natural images (e.g., COCO, LVIS) and struggle to comprehend the rigid geometry, thin structures, and geometric description cues inherent to Computer-Aided Design (CAD). Our work bridges this domain gap by explicitly fine-tuning a reasoning VLM on CAD mesh renderings and geometry in natural language used to spatially describe these features.

2.2 Language Guided Segmentation in 3D↩︎

Analyzing 3D geometry via 2D projections is a highly effective paradigm, popularized by MVCNN [20] and expanded by recent view-graph and transformer architectures [21], [22]. Recently, the field has shifted toward lifting 2D VLM features into 3D space to achieve open-vocabulary 3D understanding. Techniques such as ConceptFusion [23], OpenScene [24], and LERF [25] project CLIP features onto point clouds or Neural Radiance Fields (NeRFs). For part-level segmentation, PartSLIP [26], PartSLIP++ [27], and SATR [28] leverage 2D grounding cues and GLIP [29] to localize semantic parts in 3D point clouds. PointCLIP [30] and PointCLIPV2 [31] adapt vision-language models to point clouds through multi-view rendering and feature alignment, while COPS [32] and GeoZE [33] further improve open-vocabulary understanding via geometry-aware aggregation of 2D visual features and geometric priors. More recent approaches, such as PARIS3D [34], PointBind [35], ReferSplat [36], and IPDN [37], incorporate multimodal reasoning, structural prompts, and multi-view constraints for language-guided 3D localization. Recent foundation-model-based methods such as Find3D [38] and PatchAlign3D [39] learn language-aligned point- and patch-level representations for open-vocabulary part retrieval and segmentation. While these methods achieve impressive results on unstructured point clouds, Gaussian splats, and scene-level 3D representations, adapting them to fine-grained mesh entities with explicit topological connectivity remains challenging.

2.3 Deep Learning on CAD and Boundary Representations↩︎

Unlike standard meshes or point clouds, CAD models are defined by Boundary Representations (B-Reps), i.e graphs of parametric curves and surfaces. Early geometric deep learning adapted to B-Reps through UV-Net [40], BRepNet [41], and SB-GCN [42] for tasks like solid classification and machining feature recognition, heavily utilizing the ABC Dataset [43] and Fusion 360 Gallery [44].

Recently, text-driven 3D generation has extended into the CAD domain. Works like Text2CAD [45], CAD-LLaMA [46], and SolidGen [47] generate CAD construction sequences from text. Furthermore, B-repLer [48] and CAD-MLLM [49] introduced semantic editing of B-Reps using language models. Despite these advances, the fine-grained localization of arbitrary, language-described CAD entities (e.g., “the inner chamfered edge”) remains largely unexplored. We address this by providing a unified pipeline for language-driven geometric entity localization on meshes.

2.4 Prompt-Aware View Selection↩︎

In multi-view pipelines, selecting optimal viewpoints is critical for information gain. While traditional Next-Best-View (NBV) algorithms rely on geometric heuristics [50] or deep reinforcement learning [51], the advent of VLMs has shifted the focus toward semantic view selection. Methods like Cap3D [52] and ViewRefer [53] dynamically rank camera poses by maximizing global CLIP similarity between 2D renderings and a language prompt. However, global semantic matching fails on homogeneous, textureless industrial CAD parts, where pinpointing fine-grained features (e.g., “the inner chamfered edge”) requires explicit geometric awareness. Furthermore, while recent 3D referring segmentation methods tackle localized spatial understanding, they predominantly operate on unstructured point clouds or voxels, and are not designed to output exact topological entities defined by a mesh graph.

Our work bridges these disconnected domains. To the best of our knowledge, MV-GEL is the first framework to address prompt-conditioned, multi-view localization of exact topological entities on 3D meshes.

3 Problem Formulation↩︎

Let \(\mathcal{M} = (V, \mathcal{E}, \mathcal{F})\) denote a discrete 3D triangle mesh, where \(V = \{\boldsymbol{v}_i \in \mathbb{R}^3\}_{i=1}^{N}\) is the set of vertices, \(\mathcal{E} = \{e_k\}_{k=1}^{K}\) is the set of edges, and \(\mathcal{F} = \{f_j\}_{j=1}^{M}\) is the set of triangular faces. In the context of computer-aided design, a geometric entity refers to a distinct topological element, for instance, a continuous surface patch or a boundary curve.

Given a natural language query \(q\) describing a specific geometric entity (e.g., “the inner cylindrical surface”, “the outer chamfered edge”), our goal is to localize the exact subset of mesh elements corresponding to this queried feature. Crucially, while the query conceptually describes an analytical CAD entity (a native B-Rep face or edge), the system is provided only the unstructured polygon mesh \(\mathcal{M}\) as its geometric input at inference.

Formally, let \(\mathcal{X} \in \{\mathcal{E}, \mathcal{F}\}\) represent the target domain of the queried entity type (edges or faces). We formulate geometric entity localization as predicting a binary labeling function \(\hat{y} : \mathcal{X} \rightarrow \{0,1\}\), where \(\hat{y}(x_i) = 1\) indicates that the discrete mesh element \(x_i \in \mathcal{X}\) belongs to the target structure described by \(q\). The objective is to produce a topologically consistent prediction \(\hat{y}\) over \(\mathcal{M}\) such that the selected discrete mesh elements accurately reconstruct the continuous geometric entity intended by the user.

4 Dataset Construction↩︎

We construct a mesh–language localization dataset derived from the ABC CAD dataset. We sample entities and generate over 54k natural query-geometric entity pairs from over 5,000 CAD geometries spanning diverse mechanical parts with varying topology and surface complexity. For each CAD model, we randomly sample a single boundary representation (B-rep) primitive, restricted to either one surface (face) or one edge. As highlighted in the previous section, the sampled CAD entities provide the target views and segmentation maps during training, given the mesh \(\mathcal{M}\) and the input query \(q\).

4.0.1 Multi-View Rendering and Ranking↩︎

For each CAD mesh \(\mathcal{M}\), we generate a discrete set of candidate camera poses \(\mathcal{V} = \{v_i\}\) distributed across uniform azimuth and elevation intervals. To evaluate a pose \(v_i\) with respect to a target geometric entity \(\mathcal{E}\) (an edge or a face), we first select a representative subset of points \(\mathcal{P}_\mathcal{E} \subset \mathcal{E}\) using Farthest Point Sampling (FPS). We estimate the geometric observability of \(\mathcal{E}\) from pose \(v_i\) by casting rays from the camera center through the virtual image plane toward each point \(p_j \in \mathcal{P}_\mathcal{E}\). A point is visible if the ray intersection is unobstructed by the mesh surface. We define the visibility score as \(S(v_i, \mathcal{E}) = \sum_{p_j \in \mathcal{P}_\mathcal{E}} \mathbf{1}\{p_j \text{ is visible from } v_i\}\).

Because our visibility testing is computed from the camera’s perspective, views where the entity is viewed at a slanted or grazing angle are naturally penalized. In such configurations, rays are more likely to encounter self-occlusion or grazing intersection errors, leading to a reduced count of visible points in \(S(v_i, \mathcal{E})\). Consequently, our ranking mechanism inherently prioritizes "head-on" viewpoints that provide clear, unobstructed geometric exposure of the entity. Finally, for the top-ranked poses, we generate two synchronized renderings: an unannotated RGB image for model inference and a secondary mask-highlighted rendering for supervised training.

4.0.2 Language Query Construction↩︎

For each sampled entity’s maximal view, we construct a natural language query that uniquely identifies its geometric entity using only intrinsic geometric and topological cues, similar to how humans describe geometric entities. The descriptions rely primarily on properties such as curvature, orientation, adjacency, symmetry, continuity, and relative position within the part. This formulation enforces geometry-aware and viewpoint-specific grounding. Detailed data generation prompts can be found in the supplementary.

5 Method↩︎

Given a triangle mesh \(\mathcal{M}\) and a natural language query \(q\) describing a geometric structure, our objective is to localize the subset of mesh topological entities corresponding to the queried entity.

5.1 Prompt-Conditioned Geometric View Ranking (GELviews)↩︎

We formulate view selection as a prompt-conditioned geometric ranking problem. Given a set of candidate viewpoints, the objective is to prioritize views that maximize geometric observability of the queried entity. Unlike global semantic alignment approaches, this formulation requires reasoning about structural visibility, occlusion, and entity-specific spatial exposure across views.

5.1.1 CLIP Head↩︎

Each rendered image \(I_v\) is encoded using a CLIP vision encoder \(\mathcal{E}_{img}\), and the query \(q\) is encoded using the CLIP text encoder \(\mathcal{E}_{txt}\): \(f_v =\mathcal{E}_{img}(I_v),t = \mathcal{E}_{txt}(q)\), where \(f_v, t \in \mathbb{R}^{D}\) are the global aligned feature embeddings (CLS token embeddings). These embeddings represent high-level semantic summaries of the view and the query, respectively, and are aligned due to the CLIP training objective. We noticed that base CLIP image and text embeddings are not optimized to contain geometric semantic information and yield poor view selection when obtained from the base CLIP encoders. To mitigate this, both encoders are adapted using low-rank parameter updates, allowing task-specific specialization while preserving the pretrained vision–language alignment prior. This ensures stable optimization and prevents catastrophic forgetting of CLIP’s semantic structure. As shown in the Fig. 2, for an initial estimate of view relevance, we compute normalized cosine similarity between each view embedding and the query embedding: \(s_v^{\text{clip}} = \left\langle \frac{f_v}{\|f_v\|}, \frac{t}{\|t\|} \right\rangle\).

5.1.2 View Positional Encoding↩︎

To incorporate geometric context, specifically to make the network aware of view-points and their geometric order, we inject learnable positional embeddings \(p_v \in \mathbb{R}^{D}\) encoding the identity and relative ordering of each camera pose: \(\tilde{f}_v = f_v + p_v\).

5.1.3 Modality Fusion↩︎

While \(s_v^{\text{clip}}\) measures independent alignment, view ranking requires conditioning image features explicitly on the query semantics. We therefore apply a fusion operator \(\mathcal{F}\) between the view features and the global text embedding:\(\hat{f}_v = \mathcal{F}(\tilde{f}_v, t)\). We investigate two complementary conditioning mechanisms: (i) FiLM [54], which performs feature-wise affine modulation \(\text{FiLM}(\tilde{f}_v, t) = \gamma(t) \odot \tilde{f}_v + \beta(t)\), and (ii) Cross-Attention [55], [56], which enables token-level interaction between view features and the text embedding. FiLM provides lightweight global semantic conditioning and strictly generalizes additive and weighted-add fusion schemes as special cases (via learned scaling \(\gamma(t)\) and bias \(\beta(t)\)), while preserving the pretrained CLIP representation at initialization. In contrast, Cross-Attention enables more expressive multimodal interaction by allowing query-dependent reweighting across feature dimensions.

5.1.4 View Interaction Head↩︎

Individual view scoring ignores the fact that geometric visibility is inherently relative across viewpoints. For example, a surface may only be fully visible in one view while partially occluded in others. To capture such dependencies, we process the fused features using a multi-layer Transformer encoder \(\mathcal{T}\) operating along the view dimension: \(h_v = \mathcal{T}(\{\hat{f}_v\}_{v=1}^{V})_v\). This self-attention mechanism allows each view to attend to all other views, enabling context-aware comparison and geometric reasoning. As a result, the representation of each view becomes informed by the global distribution of viewpoints. A lightweight scoring head maps the context-aware features to scalar geometric adjustment scores: \(s_v^{\text{geo}} = \mathbf{w}^\top \mathrm{LN}(h_v)\), where \(\mathrm{LN}\) denotes LayerNorm. This component learns to correct or refine the initial semantic similarity based on geometric exposure and cross-view reasoning. The final ranking score combines semantic alignment and geometry-aware adjustment \(s_v = \alpha \left(s_v^{\text{clip}} + s_v^{\text{geo}}\right)\), where \(\alpha\) is a learnable temperature parameter controlling score sharpness. Views are ranked according to \(s_v\), and the top-\(K\) views are selected greedily from the GELviews scores.

5.1.5 Pairwise Ranking Objective↩︎

GELviews is trained to prioritize views that best expose the queried geometric entity. For each sample, we compute a visibility score \(r_v\) per view and define \(\mathcal{V}^+\) as the top-\(k\) views, where \(k = \max(1, \lfloor V \cdot \rho \rfloor)\) with a small fraction \(\rho\). All remaining views form \(\mathcal{V}^-\). This reduces ranking to separating geometrically strong views from weaker ones. Given predicted scores \(s_v\), we enforce a margin \(m\) between positive and negative views. For all \((i,j)\) with \(i \in \mathcal{V}^+\) and \(j \in \mathcal{V}^-\), we minimize \[\mathcal{L}_{rank} = \frac{1}{|\mathcal{P}|} \sum_{(i,j) \in \mathcal{P}} \log \left( 1 + \exp\left( - (s_i - s_j - m) \right) \right),\] where \(\mathcal{P}\) is the set of positive–negative pairs. This soft margin (\(m=0.1\)) objective encourages separation of informative viewpoints while avoiding noisy supervision over fine-grained ordering among weak views.

5.1.6 View Non-Maximum Suppression↩︎

The ranked views produced by GELviews may contain angularly redundant viewpoints that observe nearly identical surface regions. In our experiments we realised that applying view-NMS, contrary to popular practices, hurts our metrics. This is because views nearby act as fallback for entity segmentation (especially in cases where VLM might miss them on the main view) and thus aid in recalling the entities as views monotonically increase. As a result we have omitted it from our main analysis. Readers are advised to have a look at Table 2, where the ViewNMS version for top-3 views (for angle threshold 4\(5^\circ\)) performs much worse than the regular greedy selection.

5.2 Language-Driven Segmentation with LISA↩︎

We adopt LISA [5] as the underlying language-conditioned segmentation backbone. The model operates in image space and predicts a binary mask conditioned on a special <SEG> token, whose hidden representation is projected into the SAM mask decoder as a sparse prompt embedding. While segmentation and training supervision is performed on 2D ground truth masks, evaluation is defined on CAD mesh entities via a geometry-aware lifting mechanism.

We retain the original LISA training objectives, including the autoregressive language modeling loss and mask supervision (binary cross-entropy and DICE).

5.3 Modular Multi-View Localization Pipeline↩︎

We now summarize the full MV-GEL pipeline, in Fig. 3, in functional form. Given a mesh \(\mathcal{M}\) and query \(q\), we first render a set of calibrated views \(I_k = \mathcal{R}(\mathcal{M}, \pi_k), \quad k = 1,\dots,V\), where \(\pi_k\) denotes camera parameters and \(\mathcal{R}\) is a rendering operator.

5.3.0.1 View Selection.

The prompt-conditioned ranking module GELviews selects a subset of informative views \(\mathcal{S} = \Psi(\{I_k\}_{k=1}^{V}, q)\), where \(\mathcal{S} \subset \{1,\dots,V\}\) contains the top-\(K\) geometrically relevant viewpoints.

5.3.0.2 Image-Space Segmentation.

For each selected view \(k \in \mathcal{S}\), the segmentation backbone predicts a binary mask \(m_k = \Gamma(I_k, q)\), where \(\Gamma\) denotes the LISA-based segmentation model operating in image space.

5.3.0.3 Geometric Lifting.

The predicted masks are lifted to mesh topology via ray projection, producing a 3D labeling \(\hat{y} = \mathcal{L}(\{m_k\}_{k \in \mathcal{S}}, \{\pi_k\}_{k \in \mathcal{S}}, \mathcal{M})\), where \(\mathcal{L}\) denotes the geometry-aware lifting operator mapping foreground pixels to mesh faces or edges. The overall localization is therefore obtained by composition \(\hat{y} = \mathcal{L} \circ \Gamma \circ \Psi (\mathcal{M}, q)\), with GELviews (\(\Psi\)) and the segmentation backbone (\(\Gamma\)) trained independently. This modular design preserves pretrained segmentation priors and avoids joint optimization through the rendering process.

6 Experiments↩︎

6.0.1 Training↩︎

We train GELviews on 54k language queries paired with annotated top-performing views. LoRA adapters were used to fine tune clip image and text encoders with r=8 and alpha=16, with a layer dropout equal to 0.1. For each query, we uniformly sample candidate viewpoints over elevations in \([-60^\circ, 60^\circ]\) and azimuths in \([0^\circ, 360^\circ)\) at \(30^\circ\) increments, yielding 60 candidate views per instance. The ranking module is optimized to prioritize views that maximize entity-specific geometric observability. We place the top 5 (\(\rho = 0.09\)) views in the \(\mathcal{V}^+\) set and the rest in \(\mathcal{V}^-\). For segmentation, we fine-tune LLaVA-7B in the LISA-CAD setting using the same 54k query–binary mask pairs. Training is conducted on 4 NVIDIA H100 GPUs, updating only the LoRA adapter layers and the mask decoder while keeping the remaining backbone parameters frozen. We use a learning rate of \(1\times10^{-4}\) for all our trainable modules. Evaluation is performed on a held-out test set of 1535 query–entity pairs spanning 218 CAD meshes. After topology-aware lifting, performance is reported using face and edge-level metrics computed directly on the CAD mesh.

6.0.2 Evaluation Strategy↩︎

We evaluate localization directly on mesh entities rather than in image space. Predictions and ground truth are represented as sets of faces or edges, and overlap is computed using geometric weighting to reduce variations due to mesh discretization. Let \(\mathcal{S}_p\) and \(\mathcal{S}_g\) denote predicted and ground-truth entity sets, and let \(w(s)\) denote the geometric weight of an entity (triangle area for faces, edge length for edges). Weighted intersection and union are defined as \(W_{\text{int}} = \sum_{s \in \mathcal{S}_p \cap \mathcal{S}_g} w(s), W_{\text{union}} = \sum_{s \in \mathcal{S}_p \cup \mathcal{S}_g} w(s)\). Further we define metrics such as Intersection over Union (IoU), Precision and Recall on the mesh as follows: \(\text{IoU} = \frac{W_{\text{int}}}{W_{\text{union}}}, \text{Precision} = \frac{W_{\text{int}}}{\sum_{s \in \mathcal{S}_p} w(s)}, \text{Recall} = \frac{W_{\text{int}}}{\sum_{s \in \mathcal{S}_g} w(s)}\). This unified formulation applies to both faces and edges by simply modifying the geometric weight definition. The use of area and length weighting ensures that evaluation reflects true geometric coverage and structural correctness.

4pt

4pt

6.0.3 Ablation Studies↩︎

We analyze the impact of view ranking formulation and supervision granularity to understand the role of geometric prioritization in language-driven CAD localization. Using the geometric metrics defined earlier, we evaluate segmentation adaptation and view selection design. The metrics reported were averaged for all samples in the test set. Results for Face@top1 view and Edge@top1 view are shown in Table 1.

We ablate components of GELviews, comparing FiLM and Cross-Attention fusion variants applied in the modality fusion block (Fig.2). The No-Fusion variant bypasses this block by directly using position-aware features for self attention in the view-interaction head, while Only-CLIP removes the view-interaction head and relies solely on the CLIP head. Random view selection is included to assess the net contribution of learned ranking. All view selection strategies are evaluated with both fine-tuned (LISA-CAD) and Vanilla LISA backbones. For a better understanding of the ablated networks, readers are advised to refer to the supplementary.

6.0.3.1 Effect of Domain Adaptation

As seen in Table 1, replacing Vanilla LISA with LISA-CAD consistently improves performance across all view selectors for both faces and edges. While Vanilla LISA attains moderate recall, it exhibit’s low precision on edges and faces, leading to weak IoU and F1, as shown in Fig. 6 (right) . This indicates over-segmentation and limited geometric specificity. CAD fine-tuning substantially improves precision while preserving recall, resulting in markedly stronger F1 and IoU. These results confirm that domain adaptation is critical for geometry-aware segmentation.

6.0.3.2 Effect of View Selector Design

Within CAD-LISA, fusion-based view selectors outperform Only-CLIP and Random view selection strategies and perform on-par with No-Fusion. FiLM achieves consistently strong overall performance across top views (for top@3 and top@5 views please refer to the supplementary), closely followed by Cross-Attention and No-Fusion. Cross-Attention yields better metrics for edge localization. Cross-Attention, despite being a powerful multimodal fusion architecture, doesn’t outperform FiLM by a great margin. We provide a reason for this performance order in the supplementary. The consistently minor win of fusion-based models over No-Fusion variant indicates that interaction between semantic and view-aware features improves ranking quality. The Only-CLIP selector performs significantly worse, particularly for edges, demonstrating that global image-text similarity alone is insufficient for identifying geometrically informative viewpoints. Random view selection yields the lowest performance across both backbones, confirming that gains arise from learned view prioritization. Edge localization is consistently more challenging than face localization due to thin projections, discretization sensitivity, and view-dependent occlusions. Incorrect views can prevent the segmentation backbone from observing the relevant entities described in \(q\), leading to missed detections. Nevertheless, relative improvements across selectors are larger for edges, highlighting the importance of view quality and domain-specific segmentation for structurally sensitive features. The combination of CAD-LISA and fusion-based ranking provides the most balanced performance across both entity types.

6.0.3.3 Visibility Oracle and View-Selection Headroom

To quantify the absolute effectiveness of our view selection module, we introduce a GTviews Ground truth oracle baseline in Table 1. This acts as the theoretical performance ceiling, representing the maximum achievable scores when the target geometric view aligned most with the text prompt is selected. Remarkably, our best-performing FiLM selector achieves a Face F1 score of 0.6, capturing  95% of the oracle’s performance limit (0.63 F1). Similarly, for the highly challenging task of edge localization, FiLM reaches an F1 of 0.36, realizing more than 90% of the 0.39 F1 ceiling. This minimal performance gap proves that our prompt-aware view-ranking network operates near-optimally in identifying the best possible viewpoint, bottlenecked only by the inherent segmentation limit’s of the reasoning backbone itself. Readers are advised to refer to the supplementary for additional tables with metric analysis for higher top-k values.

6.0.3.4 Multi-View Aggregation

Top-K recall curves, as shown in Fig. 5, further validate the ranking quality and view diversity of our proposed modules. The (GTviews) trajectory serves as the upper bound for multi-view aggregation. Notably FiLM, Cross-Attention and No-Fusion rapidly close the gap to this oracle within the first 7 to 8 views. The high initial recall demonstrates that the network successfully prioritizes the most geometrically informative views first.

In contrast, the Only-CLIP and Random LISA-CAD baselines start at significantly lower recall and require a large number of views to gradually accumulate surface coverage, proving that purely semantic or arbitrary sampling fails to capture view-dependent geometry efficiently. This advantage is especially pronounced for edge localization (Figure 5b), where geometry-aware fusion dramatically accelerates the coverage of thin, easily occluded entities.

6.0.3.5 Baseline Comparisons

In Table 2, we evaluated similar open vocabulary point cloud based segmentation baselines like PartSLIP [26], Find3D [38], PatchAlign3D [39] on our test dataset, by projecting point predictions back to the mesh. We introduced default as well as adapted Top-PCD (restricting to top point cloud density predictions matching the caption: 10% for faces, 2% for edges) baselines. Because these models work with point clouds that lack explicit boundary topology and lack deep language reasoning, they severely over-segment (“bleed”) across faces (, Find3D achieves 0.95 Recall but only 0.17 Precision) and 1D edges (IoU \(\le\) 0.022).

6.0.3.6 Inference Cost vs. Views

Fig. 6 (Left) explicitly profiles the avg. time/query vs. views. Reaching  0.7 recall on faces (on average on the test set) with 5 random views incurs \(>2\times\) the latency of 1 GELview (0.6s for the random views variant vs 0.25s for the FiLM variant). Using MV-GEL random takes  16 GB VRAM while MV-GEL (FiLM) takes  17GB for one query on average. Thus GELviews yields significantly better metrics at a fraction of the computational cost. The inference was profiled on a single H100 NVIDIA GPU.

7 Conclusions↩︎

We introduced MV-GEL, a framework for language-driven geometric entity localization that unifies reasoning-based segmentation with prompt-aware view selection. By adapting LISA to the CAD domain and proposing GELviews—a geometry-conditioned view ranking module, our method accurately localizes specific target faces and edges without requiring parametric B-Rep data.

Our extensive ablations reveal two critical insights. First, domain-specific fine-tuning is essential; it resolves the severe over-segmentation of thin structural edges seen in open-domain models. Second, our fusion-based view selection dramatically outperforms semantic-only (CLIP) and random sampling. By actively prioritizing geometrically informative viewpoints, our method captures over 90% of the theoretical oracle performance for top-1 face and edge localization, and rapidly accelerates surface coverage during multi-view aggregation.

Ultimately, our results demonstrate that language-driven 3D localization is fundamentally a geometry-aware view prioritization problem, not merely an image-text alignment task. By bridging multimodal reasoning with object understanding from the views, MV-GEL establishes an effective solution for geometric entity localization and understanding.

Acknowledgements↩︎

The authors gratefully acknowledge Helmholtz-Zentrum Hereon for providing the GPU computing resources on which all experiments in this work were carried out, and thank Prof.Christian Cyron, Head of the Institute of Material Systems Modeling (MSM), Helmholtz-Zentrum Hereon, for providing the platform and support that made this work possible. The authors also thank their colleagues at MSM for helpful discussions. The authors have no competing interests to declare that are relevant to the content of this article.

References↩︎