1 Introduction↩︎

Foundation models are transforming scientific research and society [1], [2]. However, their increasing capabilities raise critical questions about their responsible application, especially in terms of reliability and the potential to generate untruthful content [2]–[5]. The hallucination phenomenon, where LLMs produce non-factual or contradictory content, poses a key challenge for building trustworthy AI systems [2], [6]. Such errors can mislead users and undermine trust in critical applications [5], [7]. Although there are initiatives to alleviate hallucinations, LLMs are still fundamentally trained to approximate patterns in their training data. This makes hallucinations an inherent risk [8]. As a consequence, the need to detect hallucinated outputs is evident.
Detection methods vary regarding the granularity at which hallucinations are identified. While prior work has advanced response-level and span-level detection [9]–[11], the majority is not designed to operate on token-level. Yet, token-level detection is increasingly important for enabling real-time filtering, targeted correction, and improved interpretability [11]. This shift is reflected in recent contributions [4], [12]. However, a detailed understanding of how hallucination signals vary across tokens in a hallucinated span is lacking. The recently published large-scale corpus RAGTruth provides novel token-level hallucination annotations that enable this investigation [13].

Despite recent progress, current methods often overlook how the token-level hallucination signal evolves within a span. We argue that a token’s corresponding hallucination signal depends on its position within the hallucinated sequence of tokens, the hallucinated span. To investigate this, we hypothesise that the first token carries a stronger hallucination signal and achieves higher detection accuracy than subsequent, conditionally generated ones. We validate this hypothesis through a position-aware analysis using RAGTruth’s token-level annotations. Aware of various established hallucination signals such as intrinsic uncertainty [2], [14], internal representations [10], [15]–[18], and external models as judges [7], [8], [17], [19], we demonstrate that our hypothesis already holds for light-weight logit-based signals. Therefore, we augment RAGTruth with reproduced output logits. Our findings consistently support our hypothesis across models and contexts. This reveals a structural property of hallucination that improves understanding of token-level signals and supports more interpretable, fine-grained, and potentially real-time detection methods.

2 Methodology↩︎

This section outlines our approach for investigating whether hallucination detection signals vary systematically with token position in a hallucinated span.

2.0.0.1 Terminology.

We define a hallucinated span as a contiguous sequence of tokens in a model-generated response that is annotated as hallucinated. A token’s in-span index refers to its position within such a span, while the span index refers to the order of the hallucinated span within the response. Lastly, we refer to all subsequent tokens within the same hallucinated span as conditional tokens, reflecting their generation conditioned on the preceding hallucinated content.

2.0.0.2 Hypothesis.

We hypothesise that the first token carries a stronger hallucination signal. To test this, we analyse the detectability and separability of different token-level logit-derived signals.

Our methodology comprises three components: (1) enriching the RAGTruth dataset with model-generated logits for each response token; (2) categorising tokens by their position within hallucinated spans and across hallucination contexts; and (3) computing detectability and separability metrics for a range of logit-derived signals. The following subsections provide a detailed description of each component.

2.1 RAGTruth Dataset↩︎

Our dataset is a modified version of the RAGTruth corpus [13]. RAGTruth provides large-scale token-level annotations of responses from a diverse range of state-of-the-art large language models (LLMs). For this work, we extract the token-level annotations and complement them with response token logits for all responses across the dataset.
As this reproduction step requires model access, we restrict our work to the publicly available Mistral-7b-Instruct [20], Llama-2-7B-chat, Llama-2-13B-chat, and Llama-2-70B-chat [21].

2.2 Hallucination Token Positions↩︎

To evaluate how hallucination signals vary with token position, we categorise tokens to enable both detectability and separability analysis. This includes a basic split between non-hallucinated tokens (\(\mathcal{T}_{\text{non}}\)) and hallucinated tokens (\(\mathcal{T}\)), as well as more granular groupings that capture positional attributes within hallucinated spans.

2.2.1 In-Span Index↩︎

We group hallucinated tokens \(\mathcal{T}\) by their positional index within a hallucination span. Let \(N\) be the maximum hallucination span length in the dataset. Then for each index \(k\) in a hallucinated span:

\[\mathcal{T}_{k} = \left\{ t_i \mid t_i \text{ is the } k\text{th token in a span}, \; k = 0, 1, \dots, N \right\}\]

where each \(\mathcal{T}_{k}\) corresponds to tokens at a specific positional index within hallucinated spans. To uncover inter-group differences, we analyse each set independently.

2.2.2 Span Index↩︎

In addition to analysing token position within hallucination spans, we examine whether detectability and separability patterns apply across different hallucination spans within the same response. Specifically, we validate that hallucination signals are consistent across different span positions in the response, for example, whether it is the first hallucinated span or a later one.
To test this, we differentiate hallucinated tokens by their span index within the response. Hallucination tokens are grouped according to their response-wide index of the hallucination span they are part of. Let a response contain \(M\) hallucination spans, denoted as \(S_1, S_2, \dots, S_M\), where each span \(S_j\) consists of a sequence of hallucinated tokens. We define the set of tokens belonging to the \(j\)-th hallucination span and in-span index \(k\) as:

\[\mathcal{T}_{k}^{(j)} = \{ t_{ki} \mid t_{ki} \in T_{kj} \}, \quad j = 1, \dots, M\]

However, the distribution of sample sizes across hallucination span indices is not balanced, with later spans containing fewer tokens. To mitigate this imbalance, we introduce a binned grouping strategy. Let \(j \in \mathbb{N}\) denote the span index, starting from 0. We aggregate tokens from spans with \(j \geq 2\) into \(\mathcal{T}^{\text{third+}}\), and define \(\mathcal{T}^{\text{all}}\) as the union of all hallucinated spans. This categorisation enables us to compare positional signal strength across early and later hallucination occurrences while maintaining sufficient sample sizes¹, as shown in Table 1.

in

2.3 Detectability↩︎

Following prior work, we frame token-level hallucination detection as a binary classification task: predicting whether a given token is hallucinated or not. We hypothesise that the detectability of hallucinated tokens varies systematically with their position in a span. To test this, we compare each positional subgroup in our categorisation against non-hallucinated tokens.

We quantify detectability using the area under the receiver operating characteristic curve (AUROC) computed over a set of scalar signals derived from non-hallucinated (\(y = 0\)) and hallucinated (\(y = 1\)) outputs. These include commonly used uncertainty measures such as entropy, perplexity, sampled probability, and logit [17], [22], [23], as well as auxiliary signals like logit vector mean, variance, and L2 distance. Signals are computed per token and grouped according to the positional categories defined in Section 2.2.
To capture both global and local trends, we compute AUROC scores across the entire dataset as well as per response (more details are provided in Appendix 5.1.1). This response-level perspective reflects a real-time usage setting, enabling us to assess the consistency of inter-positional patterns across independent model responses.

2.4 Separability↩︎

We verify that the observed positional patterns are specific to hallucinated tokens rather than generic artifacts of token position. To do this, we examine whether similar signal patterns also appear in non-hallucinated spans. As part of this verification, the separability analysis includes non-hallucinated and hallucinated tokens for comparison.

We extract two subsets from the non-hallucinated token set \(\mathcal{T}^{\text{non}}\): \(\mathcal{T}^{\text{no}}\), containing tokens from hallucination-free responses, and \(\mathcal{T}^{\text{pre}}\), containing pre-hallucination tokens². For both subsets, we exclude the initial \(<\)start\(>\) and the first generated token as their logits, regardless of whether the token is hallucinated or not, differ from those of conditional tokens. This follows from the Min-K probability and entropy distributions of \(\mathcal{T}^{\text{pre}}\) and \(\mathcal{T}^{\text{no}}\) in Appendix 5.4. We bin the remainder by their position, following the in-span index logic from Section 2.2. This yields six token groups for qualitative comparison with hallucinated tokens.

To measure distributional separability across these subsets, we adopt metrics from Membership Inference Attacks (MIA), which similarly rely on confidence-based features to distinguish in vs out-of-distribution behaviour. We use Min-K probability as our primary separability metric due to its proven efficacy in MIA contexts [23], [24], and also compute the Min-K entropy to support our findings. This yields a family of scores that reflect how token response logits vary across categories and positions. See Appendix 5.1.2 for implementation details.

3 Results↩︎

Our experimental setup tests whether the detectability and separability of hallucinated tokens vary with their position within hallucinated spans. We specifically investigated two questions: (1) How does the detectability and separability of hallucinated tokens vary depending on their position within a hallucinated span? (2) Which logit-derived signals most reliably detect hallucinated tokens and separate them from truthful tokens?
Although our analysis covers all in-span token positions, we concentrate on the first nine token indices, as the median hallucination span length ranges between six and eight tokens, depending on the model.

3.1 First Hallucination Tokens Are Better Detectable than Conditional Ones — Globally...↩︎

Hallucination tokens at the in-span index 0 appear to be more distinguishable than conditional tokens (see Figure 2). In our simplified detection setup, the detection accuracy of conditional hallucination tokens is slightly higher than 0.5, regardless of the signal. In contrast, the first hallucination token appears to be strongly distinguishable, as indicated by entropy and perplexity, yielding AUROC scores close to 0.8 across all models (see Appendix 5.2).

3.2 ...and Locally↩︎

At the response level, AUROC trends mirror global findings. However, the error bars in Figure 2 reveal high variability in the first-token detectability. It becomes evident that, at least for raw token logit entropy and perplexity signals, the first token detectability is not stable but varies heavily.

3.3 First Hallucination Tokens Exhibit Greater Separability Than Conditional Ones↩︎

Min-K Probabilities further support the enhanced detection scores for first hallucination tokens, which consistently exhibit lower scores than conditional ones (see Figure 3). This pattern is consistent across models and percentiles (see Appendix 5.3)

3.4 Entropy Most Effectively Identifies First Hallucinated Tokens↩︎

Among all logit-derived hallucination signals tested, entropy yields the most pronounced separation between first and conditional hallucination tokens. This is reinforced by the larger score gap for Min-K entropy than for Min-K probability (see Appendix 5.3). As a consequence, we conclude that logit entropy is the signal from our set that best reflects whether the first token is hallucinated.

3.5 Individual Logit-Derived Signals Are Not Robust Across Token Indices↩︎

While our analysis reveals clear positional trends in hallucination detectability, it also highlights a key limitation: none of the evaluated logit-derived signals consistently detects hallucinated tokens across all in-span indices. Some, such as logit entropy, perform well for early tokens but degrade in later positions, while others show inconsistent or noisy behavior. This suggests that no single logit-derived feature is sufficient for robust, position-invariant hallucination detection.

4 Conclusion↩︎

Our qualitative detectability and separability analysis reveals that first hallucination tokens are systematically more distinguishable than subsequent, conditionally generated ones. This pattern is evident across multiple logit-derived uncertainty signals, with logit entropy providing the clearest signal for first-token hallucination detection. However, no hallucination signal achieves robust performance across all in-span positions, highlighting the limitations of current logit-based methods. These findings motivate several directions for future work. First, we hypothesise that richer model internals, such as hidden states, may amplify the observed positional effects and improve detection reliability. Second, a complementary investigation into the last token in each span may reveal symmetric patterns and help characterise hallucination span boundaries with more precision. As our results also indicate that no single logit-derived metric consistently captures the hallucination signal across all token positions, they suggest the need for more robust or composite detection signals. We leave these extensions for future work.

4.1 Limitations↩︎

First, our approach assumes the accuracy of the hallucination span annotations provided in RAGTruth. Given the subjective nature of hallucination annotation, this could skew results either positively or negatively.

We provide a qualitative analysis based on a simplified detection setup, rather than a deployable classifier. Therefore, we leave for future work whether the observed patterns directly improve fine-grained hallucination token detection in practice.

Additionally, our analysis focuses specifically on intra-hallucination token patterns, particularly detectability and separability within hallucination spans. However, prior and subsequent tokens outside the hallucinated spans might also carry predictive signals for hallucination, as suggested in recent studies [13], [25], [26].

Lastly, we neglect the hallucination taxonomy introduced in RAGTruth. While [13] distinguish between Evident Conflict, Subtle Conflict, Evident Introduction of Baseless Information, and Subtle Introduction of Baseless Information across different task categories (QA, Data-to-Text, Summarisation), we treat all hallucinations uniformly. This decision promotes generality across hallucination types but leaves open whether the observed patterns are consistent across specific hallucination categories and tasks [3], [25].

Impact Statement↩︎

This work contributes to the responsible development of foundation models by advancing the diagnostic understanding of hallucinations at the token level. By uncovering structural patterns of hallucination signal in hallucinated spans, it provides insight into where and how hallucinations emerge, supporting the development of interpretable token-level detection methods. As this is a knowledge-oriented analysis rather than a deployed system, it poses minimal direct societal risk. Instead, it lays the groundwork for building more trustworthy and accountable language models by improving the understanding of their failure modes.

5 Appendix↩︎

5.1 Extended Methodology↩︎

5.1.1 Detectability↩︎

For each in-span index \(k \in \{0, 1, \dots, N\}\), feature \(f\), and span group \(g \in \{\text{first}, \text{second}, \text{third+}, \text{all}\}\), we define the global detectability as:

\[\text{AUROC}_f(k, g) = \text{AUROC}\left( \mathcal{F}^{\text{non}}, \mathcal{F}_{k}^{(g)} \right)\]

where \(\mathcal{F}^{\text{non}}\) and \(\mathcal{F}_{k}^{(g)}\) denote the feature sets for non-hallucinated and hallucinated tokens across each models dataset.

For each response \(r\), we define the local detectability as:

\[\text{AUROC}_f^{(r)}(k, g) = \text{AUROC}\left( \mathcal{F}_{\text{non}}^{(r)}, \mathcal{F}_{k}^{(g, r)} \right)\]

where \(\mathcal{F}_{\text{non}}^{(r)}\) and \(\mathcal{F}_{k}^{(g, r)}\) denote the feature sets for non-hallucinated and hallucinated tokens within the same model response \(r\), respectively.

5.1.2 Separability↩︎

We apply our qualitative separability analysis across six distinct token groups:

\[\mathcal{T}_k^{\text{all}}, \mathcal{T}_k^{\text{first}}, \mathcal{T}_k^{\text{second}}, \mathcal{T}_k^{\text{third+}}, \mathcal{T}_k^{\text{pre}}, \mathcal{T}_k^{\text{no}}\]

where \(k\) denotes the in-span token index for hallucination, and in-response index for non-hallucination.

Min-K is defined as the \(K\)-th smallest value among the metric scores of a given token group [24]. Let \(f(t)\) be a scalar metric computed per token \(t\), and let \(r \in \{10, 20, \dots, 100\}\) be the percentile threshold. For each group \(\mathcal{T}_g \in \mathcal{G}\), and for a fixed in-span position \(k\), we compute:

\[\text{MIN-K}_r\left(\mathcal{T}_{k}^{g}\right) = \text{K}_{r\%}\left( \{ f(t_i) \mid t_i \in \mathcal{T}_{k}^{g} \} \right)\]

where \(\text{K}_{r\%}\) denotes the \(r\)-th percentile of the sorted metric values.

5.2 Detectability↩︎

5.3 Separability↩︎

5.3.1 Min-K Probability↩︎

5.3.2 Min-K Entropy↩︎

5.4 Min-K Across All Percentiles↩︎

5.4.1 Min-K Probability↩︎

5.4.2 Min-K Entropy↩︎

References↩︎