1 Introduction↩︎

Public guarantee institutions must convert noisy monthly forecasts into warnings that are both timely and auditable. This task is particularly consequential in housing-guarantee markets, where a missed warning can delay liquidity planning, slow case supervision, and weaken preparedness for cascading guarantee defaults. False warnings are also costly because they consume analyst capacity on segments that may not require intervention. At the same time, growing use of AI assistance raises a governance requirement: warnings must be evidence-verifiable without exposing raw records or allowing a language model to invent support. The central challenge is therefore evidence-constrained risk communication, not prediction alone.

We examine this challenge in South Korea’s jeonse deposit guarantee market [1], [2]. Under a jeonse contract, a tenant pays a large upfront deposit that should be returned at maturity, while a public institution guarantees repayment when the landlord defaults. Risk reflects macroeconomic conditions, local housing prices, collateral burdens, and regional demand, which interact over different timescales and vary across region-by-housing-type segments. Monitoring is therefore difficult because severe events are sparse and market signals often arrive with a lag. In practice, analysts require more than a point forecast: they need an interpretable evidence trail that indicates which segments warrant attention and why.

Direct forecasting and naive report automation are insufficient for three reasons. First, monitoring agencies face asymmetric costs: under-predicting right-tail events can impair operations more severely than over-predicting low-risk cases [3], [4]. A model that minimizes average error is therefore not necessarily suitable for a risk desk. Second, data-governance requirements restrict the transmission of raw guarantee records to external large language models (LLMs) [5], [6], complicating automated narrative generation under data-minimization constraints. Third, accountability requires an analyst-in-the-loop workflow [7]. Linguistic fluency cannot offset operational failures when a model fabricates trend directions, numerical units, reference periods, or historical analogues [8].

To address these limitations, we formulate monitoring as a process of evidence construction. The forecast is the first component rather than the final output. Each warning is paired with model-derived diagnostic signals, including emphasized variables, salient lookback periods, comparable historical windows, and a bounded evidence record that specifies the admissible claims. We treat these signals as audit-oriented evidence rather than causal explanations. The language model neither estimates risk nor retrieves additional facts. It converts the evidence record into a draft whose factual claims can be checked automatically and reviewed by an analyst. This formulation transforms open-ended generation into bounded, verifiable reporting.

Our implementation combines an interpretable time-series forecaster with representation-aligned retrieval. A Temporal Fusion Transformer [9] predicts next-month segment risk and exposes variable-selection and temporal-attention signals. We combine these signals into a window-level rationale representation and use Centered Kernel Alignment to retrieve historical windows that are similar in the model’s reasoning space [10], [11]. Rather than matching cases solely by raw values, the retrieval step asks whether the current warning and a historical precedent are supported by comparable variable-time patterns. The resulting precedents and forecast evidence are then organized into a structured report interface. Our experiments evaluate this pipeline by jointly considering average forecast error, upper-tail monitoring utility, and the fidelity of the evidence used in reporting.

Our contributions are threefold:

• We formulate housing-guarantee monitoring as the conversion of sparse risk forecasts into auditable review evidence under capacity and governance constraints.

• We develop an evidence-construction workflow that links forecast values, model-derived drivers, influential periods, and historical precedents before report generation.

• We evaluate the workflow through tail-sensitive forecasting utility, report fidelity, auditability, and practitioner assessment.

2 Related Works↩︎

2.0.0.1 Housing-Finance Risk

Housing-finance risk is inherently a delayed monitoring problem because stress can accumulate before defaults or guarantee accidents become observable. Mortgage-default research links repayment distress to interest-rate exposure, house-price movements, negative equity, and borrower heterogeneity [12]. Structural models in this research further formalize these links to repayment distress [13]. Evidence from the mortgage crisis shows that leverage and declining local prices amplify default risk [14], while credit expansion can intensify household vulnerability [15]. Although jeonse guarantees differ from mortgages in contract structure, both require balance-sheet pressure and collateral deterioration to be detected before losses materialize. This motivates a monthly panel of collateral burdens, regional price indices, guarantee exposure, and macro-financial covariates [16], [17]. It also requires an evaluation criterion aligned with monitoring: a guarantee desk must allocate review capacity while reducing missed high-risk episodes, not merely minimize average error. Asymmetric-loss theory shows that decision-aligned forecasts can differ from conditional means, or expected average outcomes, when over- and under-prediction have unequal costs [18], and flexible-loss evaluation reaches the same conclusion [19]. We apply this principle in a restricted operational form. Because the regret objective imposes larger penalties for missed high-risk episodes, model evaluation places greater emphasis on recalling high-risk cases than on average error. The forecast remains a statistical signal for analyst review rather than an automatic decision.

2.0.0.2 Interpretable time-series forecasting

The forecasting model must capture temporal dependence while exposing evidence that can be inspected. LSTMs provide a recurrent benchmark for sequential dependence [20], DLinear offers a simple linear time-series baseline [21], and LightGBM remains effective for heterogeneous tabular inputs [22]. These models are informative comparators, but they do not directly produce the variable-time rationale required for retrieval. Temporal Fusion Transformers are well suited to this role because they combine multi-horizon forecasting with variable selection and temporal attention [9], building on the broader transformer mechanism [23]. We do not interpret these internal summaries as causal effects [24], [25]. Interpretability must be defined by the decision context and validated against its downstream use [26], particularly in high-stakes settings where post-hoc explanations can create misplaced confidence [27]. We therefore convert the model’s variable-selection and attention outputs into a forecast-rationale matrix that records which variables and lookback periods shaped each prediction and serves as the retrieval representation. Post-hoc methods provide an alternative: LIME estimates local feature importance [28], SHAP unifies attribution methods [29], TimeSHAP extends perturbation-based explanations to recurrent sequences [30], and Dynamask uses dynamic masks for time-series explanations [31]. Our objective is narrower: rather than adding a stand-alone saliency map, we use the forecaster’s rationale matrix as a searchable object that connects prediction to historical evidence.

2.0.0.3 Evidence-aligned retrieval and reporting

An explanation becomes operationally useful when it points to evidence that analysts can inspect. Raw-covariate matching may overlook cases in which the model relies on similar variable-time patterns. Centered Kernel Alignment compares representational geometry [10] and has been used to assess similarity across neural representations [11]. We apply CKA to TFT rationale matrices to retrieve leakage-free historical windows with comparable model emphasis. These windows are not causal matches. They are auditable precedents whose relevance can be examined through the same artifacts that support the forecast. The reporting layer must preserve this evidentiary structure. Retrieval-augmented generation conditions language models on external evidence [32], and large language models can produce fluent task-conditioned text [33]. However, they can also generate unsupported claims [8]. Moreover, automatic evaluation depends on metric and evaluator design [34], RAG-specific assessments face related faithfulness and grounding challenges [35], and LLM evaluators may exhibit fairness or preference biases [36]. We therefore insert a structured evidence card between retrieval and generation. The card fixes the forecast, direction label, key drivers, important periods, comparable windows, and caveats before drafting, after which rule-based audits verify that numbers, labels, periods, and features are preserved. This design integrates asymmetric monitoring, interpretable forecasting, representation-aligned retrieval, and bounded report generation into a single auditable workflow.

3 Methodology↩︎

3.0.0.1 Problem Setting

Let \(i\) index a region-by-housing-type segment and \(t\) a monthly guarantee-completion period. The observed outcome is the segment-level guarantee accident rate \(y_{i,t}\), defined as the number of guarantee accidents divided by the number of completed guarantees in that segment and month. The information set available before month \(t+1\) is denoted by \(\mathcal{F}_{i,t}\) and contains a length \(L\) history of guarantee exposure, collateral burden, housing-market indicators, and macro-financial covariates. The forecasting task estimates the next-month rate as \(\widehat y_{i,t+1}=f_{\theta}(\mathcal{F}_{i,t})\).

Report generation begins only after the forecast is fixed. Given \(\widehat y_{i,t+1}\), a recent baseline \(b_{i,t}\), model-derived interpretation signals, retrieved historical windows, and an evidence contract \(\mathcal{E}_{i,t}\), the system produces a draft report \(r_{i,t}\). A valid draft must satisfy two constraints: it cannot introduce a new risk estimate, and every numerical, temporal, feature, or analogue claim must be supported by the evidence contract. This separation restricts the LLM to conditional text generation rather than statistical estimation.

Directional reporting compares the forecast with a recent moving-average baseline. Let \(b_{i,t}\) denote this baseline and \(\tau\) a pre-specified threshold measured in%p units. The direction label is defined by \[D_{i,t+1} = \begin{cases} \mathrm{Up} & \text{if } \widehat y_{i,t+1}-b_{i,t} \geq \tau,\\ \mathrm{Down} & \text{if } \widehat y_{i,t+1}-b_{i,t} \leq -\tau,\\ \mathrm{Flat} & \text{otherwise.} \end{cases} \label{eq95direction95rule}\tag{1}\] Because the threshold is fixed before generation, the rule is a deterministic report field rather than an estimate of economic loss. Its output can therefore be verified before analyst review.

3.0.0.2 Risk-Sensitive Forecasting Backbone

The forecasting module must support both prediction and evidence construction. A scalar forecast identifies segments that may require attention, while interpretation signals indicate why a segment was surfaced and provide keys for historical retrieval. Temporal Fusion Transformers are suitable because their variable-selection networks and temporal-attention layers expose feature- and period-level signals in the same forward pass that generates the forecast [9]. Conventional average-error losses aid calibration but do not represent the monitoring asymmetry faced by a risk desk, for which under-predicting a right-tail episode may be more consequential than over-predicting a low-risk segment. We therefore adopt a regret-style objective that places additional weight on under-prediction while retaining the interpretation outputs required downstream [19]. The core loss is \[\mathcal{L}_{\mathrm{reg}} (\widehat y_{i,t+1},y_{i,t+1}) = c^{\mathrm{over}}_{i,t+1} (\widehat y_{i,t+1}-y_{i,t+1})_{+} + \lambda c^{\mathrm{under}}_{i,t+1} (y_{i,t+1}-\widehat y_{i,t+1})_{+}. \label{eq95regret95loss}\tag{2}\] Here \((a)_{+}=\max(a,0)\). The terms \(c^{\mathrm{over}}_{i,t+1}\) and \(c^{\mathrm{under}}_{i,t+1}\) are pre-specified monitoring weights, and \(\lambda\) controls the relative under-prediction penalty. This objective is not intended to reproduce the institution’s complete funding, capital, or provisioning formula. Its narrower role is to shift the operating point toward high-risk visibility while preserving model-derived evidence for the reporting layer.

3.0.0.3 Model-Reasoning Representation and Retrieval

We next construct a representation that makes model evidence comparable across time. Raw time-series proximity is inadequate because windows with similar realized rates may reflect different collateral or macro-financial conditions, whereas windows with different outcomes may induce similar emphasis over variables and lookback periods. Accordingly, the retrieval layer compares historical windows in the forecaster’s reasoning space rather than in the raw covariate space.

For segment \(i\) at monitoring date \(t\), the TFT produces variable-selection and temporal-attention weights from the same lookback window used for prediction. Let \(M^{\mathrm{VSN}}_{i,t}\in\mathbb{R}^{L\times m}\) denote the variable-selection matrix over \(m\) inputs, and let \(M^{\mathrm{Attn}}_{i,t}\in\mathbb{R}^{L\times H}\) denote the temporal-attention matrix over \(H\) retained heads. We define the window-level model-reasoning representation as \[Z_{i,t}= [ M^{\mathrm{VSN}}_{i,t} \,\Vert\, M^{\mathrm{Attn}}_{i,t} ] \in \mathbb{R}^{L\times(m+H)}. \label{eq95interpretation95matrix}\tag{3}\] This representation is not a causal decomposition of guarantee risk. It records how the trained forecaster distributes importance across observed variables and historical periods. Feature and temporal evidence answer distinct interpretive questions [37]. In our workflow, this distinction becomes operational because \(Z_{i,t}\) functions as a retrieval key rather than a stand-alone explanation.

At monitoring date \(t\), the candidate pool is restricted to historical observations that would have been available at that date. Each candidate window \((j,s)\) has a reasoning matrix \(Z_{j,s}\) and metadata describing its segment, anchor month, target month, realized accident rate, exposure size, and accident count. This restriction prevents future information from entering the retrospective workflow and makes the retrieved cases admissible as analyst-facing evidence. For target window \((i,t)\), we compute centered linear CKA against every admissible candidate. Let \(\bar Z\) denote the matrix obtained by removing column means. The similarity score is \[\mathrm{CKA}(Z_{i,t},Z_{j,s})= \frac{\lVert \bar Z_{i,t}^{\top}\bar Z_{j,s} \rVert_F^2}{\lVert \bar Z_{i,t}^{\top}\bar Z_{i,t} \rVert_F \lVert \bar Z_{j,s}^{\top}\bar Z_{j,s} \rVert_F}. \label{eq95cka}\tag{4}\] CKA compares representational geometry rather than a single flattened distance. Because it is invariant to orthogonal transformations and isotropic rescaling, it is appropriate for comparing model-derived evidence matrices whose columns combine feature- and time-level signals [10], [11]. The highest-scoring candidates are returned as analogous cases. They contextualize the forecast but neither update it nor determine the risk threshold. High-risk reference windows are selected separately according to realized risk and exposure support. This distinction is essential: a severe historical outcome may be informative without resembling the model’s rationale for the current window. Separating the two evidence types prevents the LLM from treating high realized risk as proof of reasoning similarity.

3.0.0.4 Evidence Contracts and Structured Report

Retrieved objects become useful for generation only after they are converted into a form that an LLM can cite without changing their meaning. The CKA layer returns analogous windows, high-risk reference episodes, raw target histories, and sample-support information, but these objects do not by themselves constrain which number, period, driver, or precedent the model may emphasize. The evidence contract fixes these choices through deterministic modules before the LLM is invoked. For each target window, the forecasting and retrieval components produce the following typed fields: \[\mathcal{E}_{i,t} = \{ e^{\mathrm{fcst}}_{i,t}, e^{\mathrm{base}}_{i,t}, e^{\mathrm{dir}}_{i,t}, e^{\mathrm{drv}}_{i,t}, e^{\mathrm{time}}_{i,t}, e^{\mathrm{sim}}_{i,t}, e^{\mathrm{risk}}_{i,t}, e^{\mathrm{cav}}_{i,t} \}. \label{eq95evidence95contract95simple}\qquad{(1)}\] The field \(e^{\mathrm{fcst}}_{i,t}\) stores the forecast, \(e^{\mathrm{base}}_{i,t}\) the recent baseline, and \(e^{\mathrm{dir}}_{i,t}\) the direction label from Eq. 1 . The fields \(e^{\mathrm{drv}}_{i,t}\) and \(e^{\mathrm{time}}_{i,t}\) contain the selected variable and lookback evidence; \(e^{\mathrm{sim}}_{i,t}\) stores CKA-based analogues; \(e^{\mathrm{risk}}_{i,t}\) stores high-risk references selected by realized outcomes and exposure support; and \(e^{\mathrm{cav}}_{i,t}\) records sample support, exposure, observed accident counts, and small-cell warnings. Every field is computed before the LLM is invoked.

The evidence contract has a deliberately narrow role. It is neither a second forecasting model nor a free-form explanation, but a typed set of admissible report facts. In compact notation, \(\mathcal{E}_{i,t}=g_{\mathrm{EC}}(\widehat y_{i,t+1}, b_{i,t}, D_{i,t+1}, \mathcal{G}_{i,t}, \mathcal{P}_{i,t}, \mathcal{A}_{i,t}, \mathcal{H}_{i,t}, \mathcal{V}_{i,t})\), where \(g_{\mathrm{EC}}\) is a deterministic formatting map. The LLM may verbalize fields in \(\mathcal{E}_{i,t}\) but may not introduce new forecasts, drivers, periods, or historical analogues. We then define three evidence interfaces. Let \(\mathcal{R}_{i,t}\) denote the raw context containing the target series and retrieved windows, and let \(\mathcal{S}_{i,t}\) denote a structured report plan. The interfaces are \[\mathcal{I}^{\mathrm{raw}}_{i,t} = \mathcal{R}_{i,t}, \quad \mathcal{I}^{\mathrm{card}}_{i,t} = \mathcal{R}_{i,t}\oplus\mathcal{E}_{i,t}, \quad \mathcal{I}^{\mathrm{str}}_{i,t} = \mathcal{R}_{i,t}\oplus\mathcal{E}_{i,t}\oplus\mathcal{S}_{i,t}. \label{eq95evidence95interfaces95simple}\qquad{(2)}\] The operator \(\oplus\) indicates that an additional structured object is supplied to the generator. The raw interface tests whether the LLM can select relevant evidence from unstructured context, the card interface adds the typed contract, and the structured-report interface further prescribes the reporting order. The comparison therefore measures the effect of fixing evidence selection and document planning before generation, not the effect of prompt length alone.

The structured report adds no factual information. It assigns existing evidence fields to report slots: formally, \(\mathcal{S}_{i,t}=\{(s_g,A_g)\}_{g=1}^{G}\), where \(s_g\) denotes a slot and \(A_g\subseteq \mathcal{E}_{i,t}\) the fields permitted in that slot. The slots cover the summary judgment, forecast magnitude, baseline comparison, key drivers, temporal evidence, analogous cases, high-risk references, and caveats. Thus, the raw interface requires the LLM to decide both what to say and how to organize it, whereas the structured interface limits the model to verbalizing precomputed evidence within a fixed report grammar. This structure also clarifies the analyst’s role. A free-form report forces reviewers to reconstruct the evidence path after generation, while a structured report exposes a finite set of typed claims that can be checked directly against the contract. The workflow reduces review burden without transferring the final risk judgment to the model.

3.0.0.5 Audit Protocol

The audit layer verifies that a generated report remains within the evidence contract. Retrieval alone cannot ensure numerical, temporal, or role fidelity: a report may cite a relevant window but use the wrong unit, reverse the direction label, or present a high-risk reference as a reasoning-based analogue. The audit therefore evaluates hard factual validity before analyst review.

Let \(G_{\phi}\) denote the LLM-based report generator. For an evidence interface \(c\in\{\mathrm{raw},\mathrm{card},\mathrm{scaf}\}\), the generated report is \(r^{c}_{i,t} = G_{\phi}(\mathcal{I}^{c}_{i,t}).\) Let \(G_{\phi}\) denote the LLM-based report generator. For an evidence interface \(c\in\{\mathrm{raw},\mathrm{card},\mathrm{scaf}\}\), the generated report is \(r^{c}_{i,t} = G_{\phi}(\mathcal{I}^{c}_{i,t})\). The audit applies a finite set of checks to \(r^{c}_{i,t}\). Let \(\mathcal{Q}\) denote the set of direction, numerical-value, unit, feature, period, analogue, sample-support, and unsupported-inference checks. For each \(q\in\mathcal{Q}\), the audit flag \(\alpha_q(r^{c}_{i,t},\mathcal{E}_{i,t})\) is \(1\) if \(r^{c}_{i,t}\) violates \(\mathcal{E}_{i,t}\) under check \(q\), and \(0\) otherwise. The audit output is \(\boldsymbol{\alpha}(r^{c}_{i,t},\mathcal{E}_{i,t})=(\alpha_q(r^{c}_{i,t},\mathcal{E}_{i,t}))_{q\in\mathcal{Q}}\). This vector does not make an automated decision. So, it identifies the claims that require analyst verification.

Each check targets a specific failure mode. The direction check compares the stated increase, decrease, or flat classification with \(e^{\mathrm{dir}}_{i,t}\). Numerical and unit checks verify forecasts, baselines, %p changes, counts, and exposure quantities at the required precision. Feature and period checks ensure that every stated driver and date appears in \(e^{\mathrm{drv}}_{i,t}\), \(e^{\mathrm{time}}_{i,t}\), \(e^{\mathrm{sim}}_{i,t}\), or \(e^{\mathrm{risk}}_{i,t}\). The analogue check prevents reasoning-based analogues from being conflated with high-risk references, and the sample-support check retains required caveats from \(e^{\mathrm{cav}}_{i,t}\). Finally, the unsupported-inference check flags causal, policy, institutional, or market claims that are not entailed by the contract.

The protocol is deliberately conservative: it verifies whether the draft preserves supplied evidence, while narrative quality and comparative model performance are evaluated separately. In deployment, the resulting flags define an explicit review surface on which an analyst can accept, revise, or reject the report.

4 Experiments↩︎

4.0.0.1 Experiment Setting

The empirical design separates two questions that are often conflated in financial monitoring. First, should the forecasting backbone be selected by average error or by its ability to surface right-tail accident observations that would otherwise be missed? Second, is explanation quality primarily determined by model scale, or does it also depend on how evidence is organized for the language model? Consistent with Section 3, the forecasting module identifies the warning case and produces the rationale matrices. Report generation is evaluated only after the forecast, direction label, retrieved precedents, and admissible evidence have been fixed.

The forecasting sample is a monthly region-by-housing-type panel of guarantee completions, accident counts, collateral variables, housing-market indices, and macro-financial covariates. It contains 132 segment series over 124 months from September 2015 to December 2025. We require a 25-month encoder with no more than one missing month and retain only windows with complete model inputs. This filtering yields 82 usable series, 4,580 windows, and 246 held-out test observations. The target is the next-month guarantee accident rate. Appendix 7.1 consolidates sample construction, covariates, split, and tail-support definitions.Appendix 7.2 documents the synthetic reporting material and Appendix 7.3 specifies the matched interface estimands.

For downstream LLM reporting, we separately construct synthetic aggregate explanation material so that source-level values are not exposed in prompts. The forecasting backbone and model-selection logic remain those estimated on the original panel. The 24-case explanation stress test is drawn from this LLM-facing material and varies predicted risk, direction, exposure, forecast difficulty, and evidence availability. These cases are not used to estimate population forecasting accuracy. Instead, they are held fixed across models and interfaces to test whether reports preserve the same forecast, baseline, direction rule, model-derived drivers, CKA analogues, and high-risk references. Appendix 7.2 documents this synthetic reporting material and its separation from the confidential forecasting panel.

The four questions, Q1–Q4, represent distinct tasks within the same monitoring report. Q1 and Q2 test recovery of the current target’s driver and timing evidence, Q3 tests fidelity to a deterministic threshold rule rather than new forecasting ability, and Q4 tests whether the evidence can be assembled into a concise report without substituting analogous or high-risk observations for the current target.

• Q1, key driver. Identify the main risk driver and support it with values or changes from the target segment.

• Q2, important timing. Identify the salient period in the target trajectory and compare it with the endpoint.

• Q3, direction judgment. Compute the delta and direction label from the recent baseline, forecast, and \(\pm 3.00\)%p threshold, while preserving the label and numerical inputs.

• Q4, structured report. Produce a concise report that preserves the direction label fixed in the prompt without recomputing or revising it, numerical evidence, feature trajectory, analogous cases, high-risk references, and caveats.

We vary the evidence interface through three nested conditions. Raw Data supplies the target trajectory and retrieved windows. Evidence Card adds the typed contract, including direction inputs, feature trajectories, the salient period, analogous windows, high-risk references, and sample-support caveats. Structured Draft further adds a Q4 report grammar without introducing new facts. Raw Data and Evidence Card apply to Q1–Q4, whereas Structured Draft applies only to Q4. We therefore evaluate two matched supports: an all-question Raw-versus-Card contrast and a Q4-only three-interface contrast. This separation isolates evidence selection and document planning without treating unequal task distributions as directly comparable.

Eight LLMs answer Q1–Q4 under the admissible interfaces, producing 1,728 answer rows. The primary outcome is an answer-level report-quality score, complemented by deterministic rule flags and seven rubric axes: relevance, faithfulness, temporal reasoning, numerical grounding, consistency, domain validity, and hallucination control. Each axis is scored from 0 to 2, while the overall score is recorded separately from 0 to 10. Rule checks capture direction-label errors, key numerical errors, feature-flow omissions, unit-scale mistakes, placeholder leakage, and unsupported claims. Appendix 9 summarizes the fixed-case design. Appendix 11 defines the rubric and audit checks; Appendix 10 describes the three interfaces; Appendix 7.3 defines the two matched estimands and Appendix 10.1 provides the fixed prompt governing direction, units, evidence use, and report structure.

4.0.0.2 Forecasting Backbone and Tail Monitoring

The forecasting experiment varies both model family and training objective. LSTM, DLinear, LightGBM, and TFT provide the model comparison, while the TFT rows isolate conventional average-error objectives from the regret-sensitive objective. This distinction reflects the asymmetric monitoring problem: under-predicting a right-tail accident episode may delay review, whereas over-predicting a moderate segment primarily consumes analyst capacity [4], [18], [19].

Accordingly, Table 1 reports MAE, RMSE, tail recall, over-prediction frequency, and mean regret.

Table 1 reveals a clear divergence between average accuracy and tail monitoring. LSTM-MAE achieves the lowest MAE, 0.027, yet its P90 recall is only 0.080, meaning that it surfaces fewer than one in ten realized upper-tail observations. Thus, the strongest conventional accuracy benchmark remains weak precisely where missed warnings are most costly. This result reflects the zero-heavy test distribution: 54.1% of observations are zero, the median is zero, and the P90 cutoff is 0.069.

The model-family baselines further characterize this tradeoff. DLinear and LightGBM increase P70 recall to 0.851 and 0.878, respectively, but neither directly yields the variable-time rationale matrix required by the retrieval and evidence-card layers. Their P90 recalls, 0.440 and 0.360, also remain below the selected backbone. Within TFT, the MAE-trained model records MAE of 0.029 and P90 recall of 0.160, while the RMSE-trained model records RMSE of 0.091 and P90 recall of 0.400. These comparisons show that an interpretable architecture is not sufficient by itself. Hence, the objective must also shift the operating point toward missed-tail reduction.

The Ours row reports the operating point used for evidence generation. TFT-Regret records MAE of 0.041, RMSE of 0.097, P70 recall of 0.703, and P90 recall of 0.560. On the common 246-observation test set, it surfaces 14 of the 25 realized P90 cases, compared with 2 of 25 for LSTM-MAE, 11 of 25 for DLinear-MAE, and 9 of 25 for LightGBM-MAE. This gain costs 1.37%p in MAE relative to the lowest-MAE model. Because the next stage is analyst review rather than automatic intervention, the tradeoff is operationally coherent: it expands the high-risk review set while retaining the TFT rationale matrices needed for CKA retrieval and evidence construction. Appendix 8 reports the operating frontier as the under-prediction penalty increases from \(\lambda=3\) to \(\lambda=10\). Higher penalties improve tail recall but also increase average error and over-prediction.

4.0.0.3 Evidence Interfaces and Explanation Quality

The second empirical question concerns whether report fidelity is determined by model capability alone or also by the organization of admissible evidence. Because Structured Draft is defined only for Q4, placing Raw Data, Evidence Card, and Structured Draft in a single model table would compare different task supports. We therefore report two matched estimands. The first compares Raw Data with Evidence Card over the same 24 cases, four questions, and eight LLMs. The second restricts all three interfaces to Q4, where the structured report plan is admissible.

Table 2 shows that evidence organization contributes beyond model choice. Evidence Card improves every model relative to Raw Data: GPT-4o rises from 8.56 to 8.77, Upstage Solar from 8.19 to 8.39, Mistral 24B from 7.97 to 8.20, Llama 70B from 7.75 to 8.25, and HyperCLOVA X from 5.79 to 7.60. The local models also improve, with Local Base 8B increasing from 3.07 to 4.38, Local Time-MQA 8B from 3.10 to 4.01, and EXAONE Local from 2.15 to 4.09. Averaged over the same Q1–Q4 support, the score increases from 5.82 to 6.71.

For the complete reporting task, Table 3 restricts the comparison to Q4 and reports deterministic pass rates available on the same case-model support for all three interfaces.

The Q4-only results identify the mechanism behind the interface gains. Evidence Card raises top-feature preservation from 90.6% to 98.4%, salient-period mention from 21.9% to 62.0%, exact direction-value preservation from 70.8% to 87.5%, and correct high-risk-reference handling from 77.1% to 86.5%. Structured Draft further increases the corresponding rates to 100.0%, 90.6%, 100.0%, and 91.7%. Under this Q4-only condition, final scores are 8.71 for GPT-4o, 8.54 for Llama 70B, 8.04 for Upstage Solar, 7.96 for Mistral 24B, 7.67 for HyperCLOVA X, 7.42 for Local Base 8B, 7.33 for Local Time-MQA 8B, and 6.83 for EXAONE Local. These values are reported as Q4 endpoints rather than pooled with Q1–Q3.

Appendix 7.3 defines the paired supports, Appendix 7 documents the experimental design, and Appendix 10 specifies the nested interfaces. Raw Data requires the LLM to locate relevant values and determine report order; Evidence Card fixes evidence selection before generation; and Structured Draft additionally fixes Q4 document planning. The reductions in rule violations therefore reflect factual control rather than a change in task composition or a cosmetic improvement in prose.

4.0.0.4 Audited Report Showcase

Figure 3 instantiates one held-out Q4 case. Panel A fixes the admissible facts, Panel B shows the Structured Draft response, and Panel C records the diagnostic audit. The report preserves the supplied direction, values, units, temporal order, and the distinct roles of analogous and high-risk windows. It also illustrates that Down is relative to the recent baseline: elevated LTV and a weakening lease-price index remain visible for review. In contrast, low-scoring outputs reversed the direction, omitted the high-risk reference, or introduced unsupported LTV values. Appendix 10.1 provides the fixed preventive instructions.

4.0.0.5 Evaluation Decomposition

Figure 4 provides a descriptive profile pooled over each model’s evaluated answers. It is not used to estimate an interface effect. Those effects are reported on matched supports in Tables 2 and 3. The strongest models are not merely more fluent: they also perform better on numerical grounding, faithfulness, and hallucination control, the dimensions most directly governed by the evidence contract. This pattern supports the audit protocol in Section 3.0.0.5, under which operational reports must preserve direction thresholds, accident rates, lookback periods, evidence roles, and caveats.

The pooled profiles of the local models reveal weaknesses in relevance, numerical grounding, and consistency. These dimensions correspond directly to factual review cost. Analysts can revise style locally, whereas a reversed direction label or fabricated accident count requires returning to the underlying evidence. The decomposition therefore clarifies the review labor that structured evidence is intended to reduce.

4.0.0.6 Local and Private Model Diagnostics

The Q4-only results sharpen the deployment interpretation for local and private candidates. Figure 5 shows that Local Base 8B and Local Time-MQA 8B approach or exceed 1.6 in temporal reasoning and consistency under Structured Draft, while EXAONE Local also improves enough to support bounded drafting. Faithfulness and hallucination control remain the principal residual weaknesses, consistent with the Q4 rule diagnostics in Table 3.

The magnitude of the matched Q4 Raw-to-Draft change is operationally important. Local Base 8B increases from 4.25 to 7.42, Local Time-MQA 8B from 4.29 to 7.33, and EXAONE Local from 2.38 to 6.83. These results do not justify autonomous local reporting. They indicate that institutions restricting external API use may consider a controlled workflow in which local models draft from pre-validated evidence and deterministic checks and judge-assisted diagnostics expose residual numerical or grounding errors.

4.0.0.7 Real-world Deployment

We further examine whether the report functions as an operational review aid beyond the synthetic LLM stress test. A representative evidence-grounded report was distributed to 51 analysts and domain practitioners working in guarantee review, accident management, strategy, fund and systems, and data and AI roles. Respondents rated seven task-specific statements and overall usefulness on a five-point scale. Because the exercise did not randomly assign practitioners to treatment and control conditions or embed the report in live operational decisions, we interpret it as direct end-user validation rather than a causal estimate of productivity or decision quality.

Figure 6 shows three deployment-relevant findings. Overall helpfulness averages 4.25 out of 5, with 88.2% of respondents selecting 4 or 5. Support for a pilot is strongest, with 90.2% agreeing that the report merits testing in a constrained pre-deployment workflow. Checkability also receives 90.2% agreement and a mean of 4.25. By contrast, the ability to detect explanation errors or overstatements, while still positive, is the lowest-rated item at 3.78 and 62.7% agreement. Practitioners therefore value visible evidence but continue to require explicit verification rather than autonomous narrative generation.

The component responses identify the evidence and safeguards that should be retained in deployment. The most useful objects are similar historical precedents (49.0%), the forecast-baseline comparison (45.1%), key risk drivers (45.1%), and numerical feature changes (43.1%), all of which are fixed by the contract before generation. The most requested safeguards are improved predictive accuracy (41.2%), validation of similar-case relevance (37.3%), overstatement control (29.4%), data-source labels (27.5%), and automatic numerical-error checks (25.5%). Open-text responses additionally request tables, graphs, and clearer interpretation criteria.

5 Lessons Learned↩︎

Our central lesson from the analyst-facing deployment is that operational value came from fitting established methods to the institution’s review capacity, data-governance rules, and accountability structure rather than from any single novel component.

5.0.0.1 Optimize for the review decision.

Housing-guarantee monitoring is a queue-allocation problem as well as a forecasting problem. The lowest-MAE model detected only 2 of 25 P90 observations, whereas TFT-Regret at \(\lambda=3\) detected 14, at the cost of higher MAE and over-prediction. Larger penalties recovered more tail cases but expanded the review queue. We therefore selected an operating point that balanced tail visibility with analyst capacity and kept the forecast advisory. In sparse, zero-heavy panels, model selection should report both central accuracy and the review workload induced by tail-sensitive detection.

5.0.0.2 Move evidence selection outside the LLM.

The Raw Data condition exposed factual rather than stylistic failures: responses reversed direction labels, confused target and historical windows, omitted salient periods, and altered values or units. We therefore separated rationale analogues from outcome-based high-risk references, encoded admissible facts in the Evidence Card, and used Structured Draft only to fix Q4 document order. The Evidence Card improved every evaluated model on the matched Q1–Q4 support, while Structured Draft further improved Q4 diagnostics and made smaller local models viable bounded drafters. The practical lesson is to use the LLM to verbalize pre-validated evidence, not to reconstruct decision logic from raw tables.

5.0.0.3 Confidentiality defines the deployment boundary.

The forecasting layer remained inside the secure environment, while the analyst-facing reporting layer exposed only the bounded evidence required for review. We used internally consistent synthetic aggregates for reproducible LLM evaluation, while the forecasting backbone and model-selection logic remained estimated on the original panel inside the secure environment. This separation enabled analyst-facing deployment without transmitting source-level records or sensitive regional aggregates to external models. In policy-sensitive settings, data minimization, local-model compatibility, and disclosure control are deployment requirements rather than post hoc safeguards.

5.0.0.4 Deploy the workflow, not the model.

Fluent reports still contained unsupported claims, making deterministic checks, judge-assisted diagnostics, and analyst review necessary parts of the deployed system. We deployed the evidence-grounded reporting interface to 51 analysts and related practitioners in a controlled operational evaluation. Their feedback indicated that checkable evidence, historical precedents, and forecast–baseline comparisons supported review, while error detection and precedent validity remained the principal safeguards. The deployment was therefore intentionally bounded to analyst decision support: the complete forecast–retrieval–contract–generation–audit workflow assisted review, while final judgments and policy actions remained under existing institutional controls.

6 Conclusion↩︎

This paper presents a controlled analyst-facing deployment of an evidence-constrained workflow for converting housing-guarantee forecasts into auditable reports under confidentiality and review constraints. A regret-sensitive TFT improves upper-tail visibility, while Evidence Cards and Q4 Structured Drafts reduce factual failures by fixing admissible evidence before generation. In an operational evaluation with 51 analysts and related practitioners, the reporting interface was assessed as useful for review and valued for its checkable evidence, while the feedback reinforced the need for explicit audits and precedent validation. The contribution is an analyst-centered architecture that combines decision-aligned forecasting, rationale-based retrieval, typed evidence, deterministic checks, judge-assisted diagnostics, and human approval, with final judgments and policy actions retained under institutional control.

Disclaimer↩︎

The views and opinions expressed in this paper are solely those of the authors and do not represent the official position, policies, or views of the Korea Housing & Urban Guarantee Corporation (HUG). Any interpretations, conclusions, recommendations, errors, or omissions are the sole responsibility of the authors.

7 Experimental Design and Reproducibility↩︎

7.1 Forecasting panel, covariates, and split↩︎

The panel contains 132 region-by-housing-type series over 124 months. For segment \(i\) and anchor \(t\), a 25-month encoder predicts the next-month accident rate \(y_{i,t+1}=\frac{\text{accident count}_{i,t+1}}{\text{completed guarantee count}_{i,t+1}}\). A complete encoder, a positive target denominator, at most one missing encoder month, and complete model inputs after construction are required. These criteria leave 82 series and 4,580 windows: 4,088 training, 246 validation, and 246 test. Reports use the encoder endpoint as the anchor, so a September 2024 target is reported from August 2024. The encoder uses 13 lagged inputs. Guarantee activity is represented by completed guarantee count, average guarantee amount, and average maturity. Collateral conditions by average housing value, average LTV, and the share of guarantees with LTV above 80%, housing-market conditions by lease and sale price indices and macro-financial conditions by the policy rate, CPI, unemployment rate, leading composite index, and promissory-note default rate. These groups describe operational roles rather than causal structure, and selected variables are treated only as model-rationale evidence. P70 and P90 recall apply the realized 70th- and 90th-percentile cutoffs to both forecasts and outcomes. Their test supports are 74 and 25 observations. Over is the share for which \(\widehat y_{i,t+1}>y_{i,t+1}\), and regret is evaluated by Eq. 2 and scaled by 100.

7.2 Confidential reporting material and robustness↩︎

The forecasting backbone is estimated on confidential records, whereas the LLM-facing prompts use synthetic aggregates because regional summaries may remain sensitive. The synthetic material preserves the schema, units, 25-month encoder geometry, empirical support, and temporal variation of the original panel. Rates, recent baselines, deltas, and labels under the \(\pm3.00\) percentage-point rule are recomputed, and target windows remain distinct from rationale-based analogues and outcome-based high-risk references. The pool contains 374 windows, from which 24 validation/test cases are purposively selected to span risk, direction, exposure, forecast difficulty, and evidence availability.

This construction separates the confidential forecasting experiment from the LLM-facing reporting experiment. The original panel determines the forecasting backbone and model-selection logic, while the synthetic material determines only the prompt-visible evidence for report generation. The comparison tests whether LLMs preserve fixed forecasts, direction labels, numerical values, retrieved precedents, and caveats without disclosing production aggregates.

7.3 Matched interface contrasts↩︎

Q1–Q3 use Raw Data and Evidence Card, whereas Q4 additionally uses Structured Draft. The design therefore contains \(24\times(3\times2+1\times3)=216\) conditions, or 1,728 answers across eight LLMs. For score \(Y_{s,q,m,c}\), the paired non-raw contrast is \(\widehat{\Delta}_{c} = \frac{1}{|\Omega_c|} \sum_{(s,q,m)\in\Omega_c} \left(Y_{s,q,m,c} -Y_{s,q,m,\mathrm{raw}}\right)\) where \(\Omega_c\) fixes the case, question, model, forecast, and retrieval set. For Evidence Card, \(\Omega_{\mathrm{card}}\) contains Q1–Q4; for Structured Draft, \(\Omega_{\mathrm{draft}}\) contains Q4 only. The two main-text tables follow these respective supports, and the contrasts are descriptive rather than causal.

8 TFT Objective and Operating-Point Selection↩︎

The under-prediction penalty \(\lambda\) raises tail recall as it increases, but also raises average error and over-prediction. Table 4 reports this operating frontier.

4.3pt

We select the first regret checkpoint because it surfaces 14 P90 cases with materially less review load than the \(\lambda=5\) and \(\lambda=10\) settings.

9 LLM Stress-Test Design↩︎

The 24 fixed cases are drawn from the validation/test portion of the synthetic pool and held constant across models and admissible interfaces. They comprise 8 low-, 8 medium-, and 8 high-risk cases; 8 small-, 8 medium-, and 8 large-exposure cases; 15 Down, 6 Flat, and 3 Up cases; and 8 low-, 8 medium-, and 8 high-error cases. Dominant evidence is distributed across the promissory-note default rate in 4 cases, the leading composite index in 10, and the lease price index in 10. Predicted accident rates range from 2.14% to 30.45%.

Each case fixes the target and anchor month, forecast, recent baseline, threshold inputs, current feature trajectory, rationale-based analogues, outcome-based high-risk references, and sample-support caveats. The target month is one month after the report anchor. This fixed design tests evidence preservation rather than population forecast accuracy, and sensitive pool cutoffs are withheld.

10 Evidence Interfaces and Generation Prompt↩︎

10.0.0.1 Nested interfaces.

Raw Data supplies the target 25-month trajectory, direction inputs, forecast values, and retrieved historical windows with minimal organization. Evidence Card adds typed direction fields, the recent baseline, forecast, delta, threshold, selected feature trajectories, salient period, analogous windows, high-risk references, counts, and sample-support caveats. Structured Draft adds no facts. It assigns those fields to a Q4 report grammar that fixes document order, opening and closing direction statements, feature-flow discussion, separation of analogue and high-risk evidence, and caveat placement. Raw Data and Evidence Card apply to Q1–Q4, while Structured Draft applies only to Q4. In every matched comparison, the target, forecast, retrieval set, question, and model identity remain fixed. Analogues are selected by rationale similarity and high-risk references by realized outcomes. Neither may replace target-specific evidence.

10.1 Fixed generation prompt↩︎

Figure 7 reproduces the common instruction block. Only case-specific evidence slots vary across calls. Q3 tests rule execution, whereas Q4 tests exact preservation of validated values in a longer report. Only final answers are scored.

11 Evaluation Rubric and Audit Checks↩︎

Answers are scored against the case evidence rather than model identity. Seven axes receive 0–2 points. Relevance measures whether the response directly addresses the requested task. Faithfulness assesses whether it uses only the supplied evidence without altering its meaning. Temporal reasoning evaluates whether the start, salient period, terminal point, anchor, and target remain distinct. Numerical grounding checks the accuracy of rates, deltas, feature values, and counts. Consistency examines whether the direction label remains aligned throughout the report. Domain validity assesses whether accident rates, LTV, price indices, and sample support are interpreted without causal overstatement. Hallucination control evaluates whether the response avoids placeholders, future outcomes, and unsupported external facts. A score of 0 indicates absence or a material error, 1 indicates partial or ambiguous fulfillment, and 2 indicates complete and supported fulfillment. The separate 0–10 overall score is holistic and is not a rescaling of the seven axes.

The diagnostic audit combines deterministic rule checks with judge-assisted unsupported-claim diagnostics as summarized in Table 5. Each flag identifies a factual failure that requires analyst review and remains visible rather than being averaged away by fluent prose or a high holistic score.

4pt

12 Real-World Practitioner Evaluation↩︎

Fifty-one practitioners rated seven task-specific statements and overall helpfulness on five-point scales. The pool includes 14 planning, 12 data/AI, 11 screening, 9 accident/debt-management, and 5 specialized-function respondents; 37 have at least three years of experience and 41 use LLMs weekly. Figure 8 reports the complete response distributions, respondent profile, and coded open-text themes. The 95% intervals use \(\widehat p\pm1.96\sqrt{\widehat p(1-\widehat p)/51}\) and describe this non-random pool rather than a population estimate.

Across the eight rated statements, means range from 3.78 to 4.37 and agreement from 62.7% to 90.2%. Overall helpfulness averages 4.25, with 88.2% agreement, pilot support averages 4.37, with 90.2% agreement, and checkable presentation averages 4.25, also with 90.2% agreement. Error or overstatement detection is the weakest item, at 3.78 and 62.7% agreement, reinforcing the need for explicit audits.

The most frequently selected evidence objects are similar historical precedents (25 respondents), the forecast-baseline comparison (23), key risk drivers (23), and numerical feature changes (22). The leading requested safeguards are improved predictive accuracy (21), validation of similar-case relevance (19), overstatement control (15), data-source labels (14), and automatic numerical-error checks (13). Multi-select shares need not sum to 100%, and 13 substantive comments were coded. Because the exercise uses a convenience sample, self-reported judgments, and a single representative report, it supports an analyst-in-the-loop pilot rather than a causal claim about productivity or decision quality.

References↩︎