1 Introduction↩︎

As the automotive industry transitions toward software-defined vehicles (SDVs), vehicle functions are increasingly centralized in software, and external connectivity extends across cloud services, backend infrastructure, over-the-air (OTA) updates, and in-vehicle networks. This shift has substantially enlarged the attack surface [1].

Against this landscape, UN R155 requires manufacturers to establish a Cybersecurity Management System (CSMS) that monitors, detects, and responds to cyberattacks, cyber threats, and vulnerabilities, keeps risk assessments current, and mitigates relevant threats within a reasonable timeframe [2]. ISO/SAE 21434 complements this requirement through lifecycle-oriented cybersecurity engineering and TARA methods for road-vehicle E/E systems [3]. However, CVEs from external threat intelligence mainly describe technical vulnerabilities, whereas automotive security practice frames threats as threat scenarios, attack paths, and vehicle-function impacts. Therefore, CVEs must be reinterpreted in terms of adversary objectives and attack behaviors before they can support practical security analysis. This reinterpretation can feed TARA updates, risk prioritization, and incident-response workflows, so incorrect mappings may affect downstream security decisions. Therefore, the mapping system should expose whether each tactic–technique pair is reliable enough for automatic adoption or requires analyst review.

Beyond confidence, such reinterpretation requires a domain-specific common threat language for automotive cybersecurity. General-purpose frameworks such as MITRE ATT&CK provide useful threat knowledge bases, but their coverage of automotive-specific attack contexts such as CAN injection, OTA update compromise, and V2X spoofing remains limited [4]. To address this automotive coverage gap, Auto-ISAC released the Automotive Threat Matrix (ATM) in 2024, providing an automotive-domain tactic–technique taxonomy based on validated vehicle attack cases and security research. ATM was proposed as a common language for threat intelligence classification, risk assessment, incident response, and vulnerability management [5]. In such frameworks, adversarial behavior is organized through tactics and techniques. A tactic represents the adversary’s high-level objective, or why an action is performed, whereas a technique describes how that objective is achieved through a specific method.

However, manual analysis limits scalability and consistency. Mapping outcomes vary across analyst judgments, and interpreting each CVE against ATM definitions becomes an operational bottleneck for subsequent risk assessment. This burden grows with the increasing volume of automotive vulnerabilities. Consequently, a dependable automated mapping capability is required to translate CVEs into ATM tactic–technique pairs.

Large Language Models (LLMs) are natural candidates for such automation, as they can interpret unstructured CVE descriptions at the semantic level even with limited labeled data. In particular, they can leverage ATM definitions through in-context learning (ICL), which infers task patterns from prompt examples at inference time. However, automotive cybersecurity is a safety-critical domain tied to vehicle functions, driver safety, supply-chain response, and regulatory reporting, where LLM errors may lead to incorrect threat prioritization or inappropriate TARA updates. Moreover, prior studies reported that LLMs suffer from hallucination, generation inconsistency, and limited self-verification [6]. Therefore, a single LLM output cannot meet the reliability level required for safety-critical automation.

To address this challenge, we formalize CVE-to-ATM mapping as a selective automation problem. The proposed framework integrates generation stability, evidence consistency, and confidence calibration to estimate the automation feasibility of each candidate. Based on this estimate, candidates are routed into three tiers: AUTO (auto-confirmation), REVIEW (analyst review), and HOLD. This routing establishes an explicit boundary between automated decisions and human oversight.

Based on this formulation, the main contributions of this paper are as follows:

• Confidence-aware formulation of CVE-to-ATM mapping: We formalize automotive CVE-to-ATM mapping as a selective automation problem in safety-critical environments.

• Multi-signal three-tier routing framework: We propose a framework that generates candidate mappings through hierarchical in-context learning, verifies them with self-consistency and evidence-grounded judge signals, and combines these signals through a calibrated meta-model that drives the three-tier routing. Each routing decision carries its calibrated confidence and underlying signals, providing the decision traces required for CSMS audit and TARA update workflows.

• Automotive CVE-to-ATM benchmark dataset: We construct a benchmark dataset of 269 automotive CVEs with 483 expert-annotated tactic–technique pairs.

The remainder of this paper is organized as follows. Section 2 reviews related work, and Sections 3–5 present the proposed framework that formulates CVE-to-ATM mapping as a selective automation problem with three-tier human-in-the-loop routing. Section 6 reports experimental results, and Section 7 discusses practical implications for automotive cybersecurity.

2 Related Work↩︎

2.1 Automated CTI/CVE-to-TTP Mapping↩︎

Two lines of prior work are adjacent to this study. First, ISO/SAE 21434’s TARA process has motivated automation of internal threat modeling steps: AI has been introduced into STRIDE-based threat identification [7], and component-specific LLMs have been proposed for attack tree generation and risk assessment [8]. These efforts focused on the internal stages of TARA and did not address how continuously reported CVE descriptions can be aligned with an automotive-specific threat taxonomy.

Second, prior efforts to extract Tactics, Techniques, and Procedures (TTPs) from unstructured cyber threat intelligence (CTI) evolved from ontology-based approaches [9] to automated classification methods based on MITRE ATT&CK [10]. However, recent LLM benchmark studies report that despite their reasoning flexibility, LLMs remain unreliable for standalone operation due to hallucination and technical inaccuracies [11], [12]. This limitation is more critical in the automotive domain, where safety requirements exceed those of enterprise IT.

Recently, Scarano et al. evaluated LLMs’ automotive cybersecurity knowledge against the Auto-ISAC framework. They confirmed the potential of LLMs and called for a more fine-grained evaluation framework that separates model errors from cases requiring human expertise [13]. This study addresses the gap between these two lines: it extends Scarano et al.’s direction beyond knowledge assessment to CVE-to-ATM structuring, providing the missing external-vulnerability-to-taxonomy step that TARA-internal automation does not cover.

2.2 Confidence Estimation and Deferral for LLM Outputs↩︎

LLMs are increasingly used for security analysis tasks, but prior studies consistently reported that a single inference from one model cannot guarantee output reliability. Studies on LLM uncertainty estimation reported that LLMs are overconfident when verbalizing their confidence, and AUROC for distinguishing correct from incorrect answers remains limited even under prompting strategies or multi-response consistency calibration [14]. LLM-as-a-Judge, in which one LLM evaluates another’s output, has also been reported as insufficient when relying on a single judgment [15].

These limitations indicate the need for a multi-signal approach that independently collects and combines generation, verification, and calibration signals. General directions for managing model uncertainty, including learning-to-defer [16] and conformal prediction [17], have recently been extended to LLM-based systems, but existing studies have primarily addressed general-domain tasks such as question answering, classification, and summarization. Calibrated three-tier routing that integrates generation stability with evidence-grounded verification remains insufficiently explored in safety-critical domains such as automotive cybersecurity.

3 Problem Formulation and Dataset↩︎

3.1 Problem Definition↩︎

Given a single CVE description, the system outputs a set of ATM tactic–technique pairs and a routing decision for each pair. CVE-to-ATM mapping requires semantic reasoning to interpret adversarial behavior in a vehicle environment from a limited vulnerability description. The proposed framework therefore estimates a calibrated confidence for each candidate and routes it accordingly.

3.1.0.1 Input.

The input is a single CVE description \(D \in \mathcal{T}\), where \(\mathcal{T}\) is the space of CVE texts. At inference time, the system does not use external threat intelligence, vendor advisories, proof-of-concept reports, or web search results.

3.1.0.2 Output.

For each CVE, the system produces a set of \(K(D)\) output triples: \[\mathcal{O}(D)=\left\{\left((t_a^{(i)}, t_e^{(i)}),\; \hat{p}^{(i)},\; \mathrm{tier}^{(i)}\right)\right\}_{i=1}^{K(D)} \label{eq:output}\tag{1}\] Here, \(t_a^{(i)} \in \mathcal{A}\) is an ATM tactic from the set of all tactics \(\mathcal{A}\), \(t_e^{(i)} \in \mathcal{E}(t_a^{(i)})\) is a technique drawn from \(\mathcal{E}(t_a^{(i)})\) — the set of techniques associated with tactic \(t_a^{(i)}\), \(\hat{p}^{(i)} \in [0,1]\) is the calibrated confidence, and \(\mathrm{tier}^{(i)} \in \{\text{AUTO}, \text{REVIEW}, \text{HOLD}\}\) is the human-in-the-loop routing decision. Pairs from the same CVE may be assigned to different tiers according to their respective \(\hat{p}\) values.

3.2 Dataset Construction↩︎

Because no public dataset exists for mapping CVE descriptions to ATM tactic–technique pairs, we construct an expert-annotated benchmark from publicly available automotive security resources.

3.2.0.1 Data sources.

We collected automotive vulnerability data from two primary public sources, supplemented by additional automotive-related CVEs gathered from public vulnerability databases. VicOne Automotive Zero-day Advisories [18] provide recent advisories and zero-day information. The Automotive Threat Database (AutomotiveTD) [19] covers threat cases across diverse vehicle systems such as ECUs, telematics, and V2X. We deduplicated the collected data by CVE ID and integrated them after normalizing key fields such as the CVE ID, description, and publication date. The final dataset comprises 269 CVEs and 483 tactic–technique pairs, and 51% of the CVEs carry multiple labels.

3.2.0.2 Annotation protocol.

Four experts in automotive cybersecurity performed independent per-pair annotation after reviewing the official ATM definitions, consulting public external evidence (vendor advisories, proof-of-concept reports, security research blogs) when available and otherwise relying on the CVE description alone. Labels were restricted to directly verifiable attack stages, excluding indirect secondary effects and speculative extensions. Disagreements were resolved through consensus by matching evidence phrases against the official ATM definitions. CVEs without consensus or outside the ATM framework were excluded as unmappable. Final labels correspond to pairs \((t_a, t_e)\) from the 14 tactics and 75 techniques of the ATM schema.

3.2.0.3 Split design.

To improve evaluation reliability on this small dataset, we applied two split-design principles. First, to prevent data contamination from near-duplicate descriptions of the same underlying vulnerability, we vectorized all CVE descriptions using TF-IDF, grouped pairs with similarity above 0.85, and ensured that no group spanned multiple splits. Second, while preserving group integrity, we partitioned the groups by CVE publication year (2010–2025). Earlier groups formed the Exemplar Pool (149 CVEs), from which a subset of examples is dynamically retrieved per query during ICL. More recent groups were assigned to the Development Set (62 CVEs) for system optimization and the Evaluation Set (58 CVEs) for final performance measurement. As a result, 100% of the Evaluation Set consists of CVEs published in 2024–2025, and no 2025 CVE appears in the Exemplar Pool. Group integrity occasionally forced adjacent-year overlaps across splits, but the split still approximates temporal generalization.

4 Methodology↩︎

4.1 System Overview↩︎

The proposed system consists of four stages (Fig. 1): candidate generation, multi-signal verification, confidence calibration, and operational routing.

4.2 Two-stage Candidate Generation↩︎

The proposed system leverages the hierarchical ATM structure by decomposing the mapping task into two stages: (1) identifying relevant tactic candidates and (2) predicting techniques conditioned on each identified tactic. Both stages use ICL with examples drawn from the Exemplar Pool, without additional model training. We use OpenAI GPT-5.2 (reasoning effort = high) as the backbone model. A fixed system prompt frames the model as an expert in automotive cybersecurity and the Auto-ISAC ATM, aligning its outputs with the target ontology.

4.2.0.1 Stage 1: Tactic prediction.

The first stage uses a prompt consisting of (i) system instructions summarizing the ATM tactic definitions, (ii) CVE–tactic examples drawn from the Exemplar Pool, and (iii) the description of the target CVE. A function-calling schema constrains the model to select from the 14 valid tactic IDs, preventing ontology-violating hallucinations by construction.

4.2.0.2 Stage 2: Tactic-conditioned technique prediction.

For each tactic candidate from Stage 1, the second stage dynamically constructs a schema that includes only the techniques associated with that tactic, blocking pairs that violate the tactic–technique hierarchy by construction. ICL examples are drawn only from Exemplar Pool entries labeled with the target tactic, from which the top \(k{=}8\) cases most semantically similar to the input CVE description are retrieved via \(k\)-nearest-neighbor ICL (kNN-ICL). This retrieval focuses the model on tactic-specific examples that are semantically close to the input CVE, while keeping prompt length bounded.

4.2.0.3 Repeated inference and pair-level consistency.

To mitigate response inconsistency across repeated LLM calls, Stage 1 is invoked \(N=5\) times per CVE, and Stage 2 is invoked \(N=5\) times per tactic candidate from Stage 1. The consistency \(c(\cdot)\) of a tactic \(t_a\) or a conditional technique \(t_e \mid t_a\) is defined as the proportion of generations in which it appears, and pair-level consistency is their product: \[\mathrm{consistency}_{\mathrm{pair}}(t_a,t_e)=c(t_a)\cdot c(t_e \mid t_a).\] The candidate set \(\mathcal{P}(D)\) retains all valid pairs observed at least once, since a majority-voting filter could discard low-frequency but potentially valid threat mappings. Incorrect mappings are subsequently pruned by the multi-signal verification stage (§4.3), regardless of their generation frequency.

4.3 Multi-Signal Verification↩︎

Because the candidate set \(\mathcal{P}(D)\) relies solely on generative output, generation frequency alone cannot distinguish a repeatedly generated incorrect pair from a correct one. To address this, we combine two complementary signals: pair-level consistency (§4.2), which captures generation stability, and evidence-grounded verification, which checks factual alignment between CVE-extracted evidence and candidate technique definitions through a separate verification call. The verification stage adapts the LLM-as-a-Judge paradigm by grounding judgments in evidence explicitly extracted from the CVE rather than the verifier’s prior knowledge. We further anonymize candidates before judging and decompose the verification signal into three distinct axes. Disagreement between the two signals exposes pairs that are frequently generated yet unsupported by CVE evidence.

4.3.0.1 Evidence-grounded LLM verification.

A separate verification call, acting as an evidence-grounded judge, reassesses each candidate pair against the facts in the CVE text. The system first extracts six structured elements informed by ISO/SAE 21434 TARA—covering the affected component, attack interface, security consequence, attack precondition, protocol or bus, and vehicle impact—along with supporting evidence spans. Any element not explicitly stated is marked as unknown, which constrains the verifier to evidence-only reasoning. After anonymizing the candidates, the verifier outputs three scores in \([0,1]\) for each pair: support, contradiction, and sufficiency. This separation prevents low sufficiency and high contradiction from yielding identical low values, which improves the interpretability of the verification signal.

4.4 Confidence Integration and Calibration↩︎

We train a meta-model on the Development Set that integrates the signals from §4.3 into a single calibrated confidence \(\hat{p}\) for each candidate pair.

4.4.0.1 Feature vector.

Each candidate pair \((t_a, t_e) \in \mathcal{P}(D)\) is encoded as a six-dimensional feature vector \(\mathbf{x} = [x_1, \ldots, x_6]^\top\). Here, \(x_1 =\mathrm{consistency}_{\mathrm{pair}}\) captures generation stability, and \(x_2, x_3, x_4\) correspond to the verifier’s three scores: support, contradiction, and sufficiency. To capture conjunctive evidence, we add two interaction terms: \[x_5 = x_1 \cdot x_2, \quad x_6 = x_1 \cdot x_4.\] These terms enable the meta-model to increase confidence only when generation stability (\(x_1\)) and verification-based factual support (\(x_2, x_4\)) are jointly high.

4.4.0.2 Meta-model and calibration.

We adopt L2-regularized logistic regression as the meta-model for its interpretability and stability on small-scale data. The model outputs \(\hat{p}\), the estimated probability that a candidate mapping is correct. To support threshold-based routing, we apply post hoc calibration on the Development Set in two stages. Sigmoid calibration first corrects the overall scale mismatch. Temperature scaling then addresses residual overconfidence or underconfidence by selecting \(T\) that minimizes validation negative log-likelihood (NLL) via grid search. The temperature scaling step intervenes only when needed, avoiding unconditional distortion of the score distribution. The resulting \(\hat{p} \in [0,1]\) drives the AUTO, REVIEW, and HOLD routing in §5.1.

5 Operational Routing and Deployment↩︎

5.1 Three-Tier Human-in-the-Loop Routing↩︎

Building on the calibrated \(\hat{p}\) produced by the meta-model in §4.4, we apply a three-tier human-in-the-loop (HITL) routing policy. The policy adjusts the balance between automation and analyst review by changing thresholds alone, without retraining the meta-model.

5.1.0.1 Operational modes.

AUTO candidates have sufficient calibrated confidence for automatic confirmation. REVIEW candidates require analyst inspection with supporting confidence and evidence. HOLD candidates are deferred until additional evidence or reanalysis is available. Routing operates at the pair level, so candidates from the same CVE may receive different tiers.

5.1.0.2 Mode selection.

To map the precision–coverage trade-off available under the calibrated \(\hat{p}\), we swept \(\tau_{\text{auto}}\) across the Development-Set \(\hat{p}\) distribution. The three operational modes share the same \(\tau_{\text{hold}}\) and differ only in \(\tau_{\text{auto}}\). From this sweep, we select three operating points that span distinct regions of the trade-off curve and report them as named deployment modes. The corresponding thresholds and calibration parameters are then applied to the Evaluation Set without re-selection.

• High-Confidence mode: a conservative operating point selected from the high-precision region of the Development sweep. This mode targets autonomous CSMS workflows in which AUTO confirmations enter the vulnerability registry without further analyst review, where false positives carry a high operational cost.

• Balanced mode: an operating point from the mid-region of the sweep, retaining substantial recall while keeping AUTO precision well above the random baseline.

• High-Coverage mode: an exploratory operating point from the high-recall region of the sweep, used to assess the volume of correct mappings the system can route at low selectivity.

5.1.0.3 Routing policy.

The tier assignment of each candidate pair is given by: \[(t_a, t_e) \longrightarrow \begin{cases} \text{AUTO} & \text{if } \hat{p} \geq \tau_{\text{auto}}, \\ \text{REVIEW} & \text{if } \tau_{\text{hold}} \leq \hat{p} < \tau_{\text{auto}}, \\ \text{HOLD} & \text{if } \hat{p} < \tau_{\text{hold}}. \end{cases}\]

6 Results↩︎

6.1 Main Results↩︎

6.1.0.1 Experimental setup.

We conducted the evaluation on the Development Set (62 CVEs, 102 gold tactic–technique pairs) and the Evaluation Set (58 CVEs, 98 gold pairs). The data split, meta-model training, and threshold selection follow the design described in §3.2 through §5.1. For the comparative evaluation in Table 1, all systems share the same 98-gold-pair denominator. We compare against three methodologically representative baselines:

• (A) TF-IDF + Linear Classifier: A classical supervised baseline using TF-IDF features and a linear classifier.

• (B) Flat Zero-shot GPT-5.2: A zero-shot LLM baseline that directly predicts ATM pairs using a single prompt, without structured decomposition or exemplar retrieval.

• (C) Frozen SecureBERT Retrieval: A retrieval baseline that, for each Evaluation CVE, retrieves the most semantically similar CVEs from the Exemplar Pool using frozen SecureBERT and uses their gold tactic–technique labels as candidate mappings.

6.1.0.2 Comparative Performance.

As shown in Table 1, Ours – all candidates outperformed all three baselines on every pair-level metric. Against the Flat Zero-shot GPT-5.2 baseline in particular, it matched recall while raising precision by \(+0.118\) and reducing average predictions from \(4.55\) to \(3.19\). This indicates that hierarchical decomposition and semantic exemplar retrieval improve candidate-set quality rather than merely increasing candidate count.

6.1.0.3 Routing results.

The proposed system generated 185 pair-level candidates covering 73 of 98 gold pairs; the remaining 25 lie outside the candidate set. Table 2 reports the routing outcome. The three modes share the same calibrated \(\hat{p}\) and differ only in the AUTO threshold \(\tau_{\text{auto}}\). Within the candidate set, the base rate of correct pairs is approximately \(0.395\), which sets the reference point against which each \(\tau_{\text{auto}}\) is interpreted. The three operating points correspond to \(\tau_{\text{auto}} = 0.21\) for High-Coverage, \(0.38\) for Balanced, and \(0.7431\) for High-Confidence, yielding AUTO precisions above, near twice, and more than twice this base rate. Each threshold is selected from a distinct region of the Development Set \(\hat{p}\) sweep described in §5.1.

The Balanced threshold \(\tau_{\text{auto}} = 0.38\) keeps recall substantial at the cost of per-pair precision, making it appropriate for analyst-audited workflows. For workflows where AUTO confirmations are committed without further review, High-Confidence is the intended operating point.

All seven HOLD candidates with \(\hat{p} < \tau_{\text{hold}}=0.10\) were incorrect, which confirms that HOLD effectively separates low-confidence invalid candidates. As \(\tau_{\text{auto}}\) increases, the AUTO set shrinks while within-tier precision rises. High-Coverage supports broad initial screening at low selectivity. High-Confidence achieves an AUTO precision of \(0.878\), over \(2\times\) the base rate, providing the quantitative basis for its use in autonomous CSMS deployment. The REVIEW set correspondingly grows from \(53\) to \(129\) candidates as uncertain candidates are deferred to analyst review. Within REVIEW, candidates remain ranked by \(\hat{p}\), with within-tier AUROC of \(0.74\) for High-Confidence, allowing analysts to prioritize top-ranked items within the tier.

6.1.0.4 Analysis of the 25 missed gold pairs.

The 25 missed gold pairs are dominated by two recurring patterns that together account for 22 of 25 misses (88%). First, 14 misses (56%) are Stage 2 failures, with 10 in Execution (6/6) and Initial Access (4/4), in which the correct tactic was identified but semantically adjacent techniques were not distinguishable from the CVE description alone. Second, 8 misses (32%) are Stage 1 failures, in which auxiliary tactics whose strategic role is implicit in CVE text (Defense Evasion, Command and Control) were under-predicted. Both patterns reflect a structural gap: CVE descriptions report technical defects, while ATM annotations encode adversary intent.

6.2 Ablation Study↩︎

To assess how self-consistency and evidence-grounded judge signals contribute to the meta-model, we retrained it after removing each signal group and its interaction terms (Table 3). Removing the judge signals reduced the Area Under the ROC Curve (AUROC) from \(0.868\) to \(0.705\), identifying the evidence-grounded judge as the primary driver of discriminative ranking. Removing the consistency signal lowered AUROC to \(0.851\), while the Expected Calibration Error (ECE) improved from \(0.057\) to \(0.054\).

The feature coefficients confirm this picture. The highest-weighted features are judge_support (\(+0.485\)), judge_sufficiency (\(+0.391\)), and the two interaction terms consistency\(\times\)support (\(+0.335\)) and consistency\(\times\)sufficiency (\(+0.243\)). Positive interaction coefficients show that judge signals contribute more when consistency is high. The direct coefficient of consistency_pair is small (\(+0.082\)), but consistency also enters the model through both interaction terms. Therefore, its total contribution is distributed across direct and interaction pathways rather than captured by the direct coefficient alone.

4pt 4pt

6.3 Stability and Sensitivity Analysis↩︎

6.3.0.1 Bootstrap stability.

We performed \(B{=}1000\) CVE-level bootstrap resamples on the Evaluation split (\(N{=}58\)). The calibrated \(\hat{p}\) achieved AUROC \(0.868\) (95% CI \([0.815, 0.916]\)), exceeding the consistency-only baseline. A paired bootstrap confirmed \(\Delta\text{AUROC}=+0.163\) (CI \([+0.104, +0.229]\)) over the consistency-only baseline (Table 3) and \(\Delta F_1=+0.113\) (CI \([+0.056, +0.168]\)) over the Flat Zero-shot GPT-5.2 baseline. Both intervals exclude zero. In the High-Confidence mode, AUTO precision reached \(0.878\) (95% CI \([0.778, 0.959]\)).

6.3.0.2 Threshold sensitivity.

A bootstrap analysis on the Development set (\(N{=}62\)) showed that \(\tau_{\text{auto}}\) in the High-Confidence mode had a 95% CI of \([0.428, 0.942]\). To examine whether this wide CI affects deployment stability, we re-evaluated the AUTO tier on the Evaluation Set across \(\tau_{\text{auto}}\) values resampled from this CI (Fig. 2).

The resulting AUTO precision remained within \(0.78\)–\(0.90\) throughout the CI, and the High-Confidence operating point (\(\tau{=}0.74\)) fell within the high-density region of the bootstrap distribution. The wide \(\tau_{\text{auto}}\) CI therefore reflects the precision–coverage trade-off available to the operator rather than instability of the routing decision.

7 Discussion↩︎

The central implication of this study is that automotive CVE-to-ATM mapping can be reformulated as a selective automation problem in which auto-confirmation and analyst review are separated according to confidence. On the Evaluation Set, the calibrated \(\hat{p}\) achieved an AUROC of \(0.868\) versus \(0.705\) for the consistency-only configuration (Table 3), indicating that self-consistency alone is insufficient and that evidence-grounded judge signals are necessary for reliable confidence estimation. In the High-Confidence mode, the AUTO tier reached a precision of \(0.878\), more than \(2\times\) the candidate base rate of \(0.395\), which indicates that the calibrated \(\hat{p}\) separates auto-confirmable mappings from the broader candidate pool rather than merely reflecting an arbitrary cut-off.

These properties have direct operational implications. The framework auto-confirms only sufficiently confident tactic–technique pairs and routes uncertain cases to analysts with calibrated \(\hat{p}\) and evidence-grounded signals, preserving analyst attention for ambiguous cases while maintaining traceable decision evidence for cybersecurity work products required by ISO/SAE 21434 Clause 15 [3]. When the ATM ontology is updated [5], maintenance can be localized to the exemplar pool, routing thresholds, and lightweight meta-model.

This operational interpretation should be understood within the following scope. First, the evaluation was conducted on the current ATM ontology (14 tactics, 75 techniques) and 269 expert-annotated CVEs, so ontology revisions or OEM-specific taxonomies may require updating the ICL exemplars and retraining the meta-model. Second, both the candidate generation stage and the evidence-grounded judge use the same GPT-5.2 backbone. This same-backbone design may raise concerns about self-confirmation, but the framework reduces direct confirmation bias through evidence grounding, candidate anonymization, and decomposed judgment. The verifier is grounded in a CVE-only evidence packet, where unstated fields are explicitly marked as unknown, and it receives anonymized candidates without generation order or consistency scores. The verifier also outputs support, contradiction, and sufficiency separately. This separation distinguishes weak evidence from explicit conflict. The ablation in §6.2 supports this design choice: removing the judge reduced AUROC from \(0.868\) to \(0.705\). Nevertheless, cross-family validation remains necessary to test whether these safeguards generalize beyond a single backbone. Third, because the meta-model integrates judge signals as continuous features, some boundary cases with weak evidence support may still receive high confidence; a minimum-support condition or rule-based guardrail could mitigate this. Fourth, the effectiveness of analyst review in the REVIEW tier requires field study with security analysts.

8 Conclusion↩︎

This paper proposes a confidence-aware framework that formulates automotive CVE-to-ATM mapping as a three-tier routing problem over calibrated confidence. Each tactic–technique pair is assigned to the AUTO, REVIEW, or HOLD tier based on its \(\hat{p}\). On the Evaluation Set, the calibrated \(\hat{p}\) achieved an AUROC of \(0.868\), exceeding the consistency-only baseline by \(+0.163\), and the AUTO tier in the High-Confidence mode reached a precision of \(0.878\), more than \(2\times\) the candidate base rate. To the best of our knowledge, this is an early attempt to apply calibration-aware routing to the Auto-ISAC Automotive Threat Matrix.

By exposing the confidence underlying each mapping, the framework provides an explicit boundary between automated decisions and analyst review that automotive CSMS workflows can build on. Larger datasets and analyst field study in the REVIEW tier remain future work.

References↩︎