1 Introduction↩︎

The past few years have witnessed the emergence and rapid development of large language models (LLMs) such as GPT [1], DeepSeek [2], Claude [3], Gemini [4], Grok [5] and Qwen [6]. Their impact is everywhere, from education, academia and software development to healthcare and everyday life [7]–[9]. On one side of the coin, LLMs can support users with conversational question answering, help students learn more effectively, draft emails, write computer code, prepare presentation slides and more. On the other side, their ability to closely mimic human-written text also raises serious concerns, including the generation of biased or harmful content, the spread of misinformation in the news ecosystem, and the challenges related to authorship attribution and intellectual property [10]–[14].

Addressing these concerns requires effective algorithms to distinguish between human-written and LLM-generated text, which has become an active and popular research direction in recent literature [15], [16]. Existing works either actively detect LLM-generated text, by embedding watermarks into LLM-generated text during the design of the model [17]–[24], or passively, without any prior knowledge of the watermarking process. This paper focuses on the latter category of passive detection algorithms. We review these algorithms below.

1.1 Related works↩︎

Most existing passive detection algorithms fall into the following two categories: (i) zero-shot methods and (ii) machine learning (ML)-based approaches, depending on whether they rely on external data for training the detector. Within each category, methods can be further classified into three subtypes: (1) logits-based; (2) rewrite-based, and (3) other approaches. This yields a total of 6 combinations.

Zero-shot detection. Zero-shot methods use only the observed text and a surrogate LLM for detection, without utilizing any additional dataset for training. They compute a statistical measure from the observed text to determine whether it was authored by a human or an LLM. The underlying rationale is that human-written text tends to produce statistics that differ (either larger or smaller) from those of LLM-generated text, and this difference can be exploited for detection [25]. Based on the type of statistical measure employed, these methods can be further categorized into three subtypes:

1. Logits-based methods construct the statistic using the logits of tokens computed by the surrogate LLM across the observed text [26]–[30].

2. Rewrite-based methods define the statistic as a suitable distance between the observed text and its rewritten (or regenerated) version [31]–[34]. A similar principle has also been adopted for detecting machine-generated images [35], [36].

3. Beyond logits or rewrite-based distances, other statistics have been introduced, including the intrinsic dimensionality of the observed text [37], its latent representation patterns [38], N-gram distributions [39] and maximum mean discrepancy [40], [41].

ML-based detection. ML-based methods leverage external human- and LLM-authored text to enhance the detection power of zero-shot methods. A primary approach is to formulate the detection task as a classification problem and utilize external data to train the classifier. Similar to zero-shot methods, ML-based approaches can also be categorized into three subtypes:

1. Logits-based methods fine-tune the surrogate LLM’s logits to improve the classification accuracy. Various LLMs have been employed in the literature, including RoBERTa [39], [42], BERT [43], DistilBERT [44], and reward models for aligning LLMs with human feedback [45]. Recent works have extended these methods to more challenging scenarios, including handling adversarial attacks [46]–[48], short texts such as tweets and reviews [49], black-box settings under diverse prompts [50], [51], and accommodating statistical inference [52].

2. Rewrite-based methods either use the distance between the observed text and its rewritten version as an input feature for training the classifier [53]–[56], or apply ML to fine-tune the the rewriting model itself to improve the detection accuracy [57].

3. Other methods extract features beyond logits or rewrite-based distances, and then apply ML algorithms to these features for classification. Examples of features being utilized range from classical N-grams and term frequency–inverse document frequency widely used in natural language processing [39], to more complex representations such as various combinations of features constructed based on token probabilities [58], cross-entropy loss between the text and a surrogate LLM [59], hidden latent representations [60] and features learned via multi-level contrastive learning [61], and even classification probabilities of fine-tuned LLMs [62].

1.2 Contributions↩︎

Our proposal falls under the category of ML-based, rewrite-based detection. We study a commonly encountered setting in practice, where LLM-authored text is generated using prompts that are unobserved by the detector. Our main contributions are as follows:

• Methodologically, we develop a new rewrite-based method for detecting LLM-generated text. Unlike existing approaches that primarily employ a fixed distance to compare the original text with its rewritten version, we propose to adaptively learn this distance via ML. Our proposal better discriminates between LLM- and human-authored text (see Figure 2 for a graphical illustration), leading to substantial performance gains.

• Theoretically, we develop a geometric approach to demystify the rationale behind rewrite-based methods (see Figure 1 for illustration and Proposition 1 for the detailed statement). We next show that these methods generalize well to unobserved prompts (Proposition 2). Finally, we demonstrate the rationale for learning a distance function rather than relying on a fixed distance (Proposition 3).

• Empirically, we conduct comprehensive experiments across 24 datasets, 7 target language models, and 3 types of unseen prompts, covering over 100 settings. Our results show that: (i) our approach outperforms 12 state-of-the-art methods, achieving average relative improvements of 41.5% to 75.4% over the strongest baseline across different target LLMs baseline (Sections 4.1 and 4.2); (ii) our approach is more robust than existing methods under adversarial attacks (Section 4.3); (iii) learning the distance function provides substantial benefits, with an average relative improvement of 96% over using a fixed distance (see the ablation study in Section 4.4).

2 Rewrite-based Methods: Building Intuition↩︎

In this section, we present a geometric framework for understanding rewrite-based detection methods, revealing their underlying rationale and demonstrating their robustness to unseen prompts.

Let \(\boldsymbol{X}\) denote the target text under detection. We study the problem of determining whether \(\boldsymbol{X}\) is authored by a suspected target LLM, or by a human. Rewrite-based methods are straightforward to describe: they first prompt the target LLM to rephrase the original text and then measure the discrepancy between the original text \(\boldsymbol{X}\) and the LLM’s reconstruction (denoted by \(\mathcal{R}(\boldsymbol{X})\)) under a distance metric \(d\). These methods rely on the observation that, compared to human-authored text, machine-generated text should be closer to its reconstruction [33], [53]. In the following, we will formally prove this assertion from a geometric perspective.

Building intuition. We begin with some notations and hypotheses. Let \((\mathcal{X}, \mathcal{B})\) denote a measurable space of texts (after embedding).

This assumption is reasonable since texts are typically mapped into a vector space where each token is represented by a scalar [63], and padding is commonly applied to ensure all texts share the same dimensionality.

Let \(\mathcal{H}\) and \(\mathcal{M}\) denote the subspaces corresponding to texts authored by humans and the target LLM, respectively. We use \(p\) and \(q\) to represent their respective probability distributions. We also define the projection operator \(\Pi\) onto \(\mathcal{M}\), \[\label{eq:projection-operator} \Pi_{\mathcal{M}}(x)\;=\;\arg\min_{y\in\mathcal{M}} d^*(x,y),\tag{1}\] which projects a given text \(x\in \mathcal{X}\) to its closest point in \(\mathcal{M}\), produced by the target LLM.

Assumption 2 is our key hypothesis, which reflects the geometric relationship between human- and LLM-authored text. Intuitively, it implies that all LLM-generated texts can be viewed as a projection of human-written text onto a specific subspace. This assumption is reasonable because (i) LLMs are trained on massive corpora of human-authored text with the objective of approximating the distribution of human language; (ii) LLM’s output space is constrained by the model’s architecture and learned parameters, and is thus different from the human text space. Therefore, the mapping from human text to LLM-generated text can be interpreted as a projection: a transformation that preserves semantic meanings while restricting outputs to the region defined by the model.

Here, for a fixed text \(x\), we allow its reconstruction \(\mathcal{R}(x)\) to be random. This is because LLM outputs are typically stochastic due to the use of a nonzero temperature during inference. Assumption 3 essentially requires the reconstructions of a human-written text \(x\) and its projection \(\Pi_{\mathcal{M}}(x)\) to share the same distribution. This holds when the reconstruction can be written as \[\begin{align} \label{eqn:addnoise} \mathcal{R}(x)=\Pi_{\mathcal{M}}(x)+e, \end{align}\tag{2}\] for some random error \(e\) that lies on the space of \(\mathcal{M}\). Equation 2 suggests that the rewriting process can be viewed as a two-step procedure: first, the input text is projected onto the LLM subspace, and then a small perturbation \(e\) is added to the projected text, while preserving the projected text’s semantic meaning.

Proposition 1 formally establishes the validity of rewrite-based methods, and proves that human-written text’s reconstruction error (the distance between a text and its reconstruction) is on average larger than that of LLM-generated text. The equality holds only under the idealized scenario where the LLM’s output space perfectly replicates the human text space.

Intuitively, this result follows because reconstructions always lie within the LLM subspace \(\mathcal{M}\), whereas human-authored text may lie farther away from \(\mathcal{M}\). Figure 1 provides a graphical illustration: the reconstruction error for human text (\(d_1\)) is clearly larger than that for LLM-generated text (\(d_2\)).

Generalization to unseen prompts. In practice, LLM-generated text is often produced under a variety of writing prompts (e.g., “polish this paragraph” or “help me rephrase”). The presence of such prompts induces a distributional shift: the resulting LLM-generated text no longer follows the original distribution \(q\), but instead depends on the specific prompt, which we denote by \(q_{\textrm{prompt}}\). This shift is illustrated in Figure 1, where the prompt alters the location of the generated text in the embedding space.

Rewrite-based methods can generalize effectively to such shifts, provided that the perturbation \(e\) in equation 2 does not substantially distort the semantic meaning of \(\Pi_{\mathcal{M}}(x)\). We formalize this intuition in the following proposition.

Proposition 2 provides a lower bound to quantify the difference in reconstruction error between human- and LLM-authored text. The bound depends on two factors: (i) the average gap between human and LLM-generated text, characterized by the norm of the projection \(\mathbb{E}_{\boldsymbol{X}\sim p} |\boldsymbol{X} - \Pi_{\mathcal{M}}(\boldsymbol{X})|\); (ii) the magnitude of the perturbation \(e\).

Figure 1 offers a graphical illustration: despite the shift introduced by the prompt, as long as \(e\) remains small, the reconstruction error for human text (\(d_1\)) can still be substantially larger than that for LLM-generated text (\(d_3\)). In practice, minimizing \(e\) requires careful design of the rewriting prompt to preserve the input text’s semantic meaning. This can be achieved through prompt engineering or by adaptively learning the rewrite model [57].

3 Adaptive distance learning↩︎

Limitations of existing approaches. We begin by discussing the limitations of existing logits-based and rewrite-based detection methods to better motivate our proposed approach:

• Logit-based methods, such as DetectGPT [26] and Fast-DetectGPT [28], construct the detection statistics using the log-probability \(\log q(x)\) of the text. However, their performance tends to degrade when the text is generated under unseen prompts (see the first two panels of Figure 2 for illustration). This arises because the true conditional distribution \(\log q(x \mid \text{prompt})\) differs from the marginal distribution \(\log q(x)\) used by the detector, leading to the misspecification of the detection statistic.

• The effectiveness of rewrite-based methods relies on choosing an appropriate distance function to distinguish human- from LLM-authored text, and the optimal distance function may differ largely from standard Euclidean distance due to the complex geometry of text embeddings. Nonetheless, existing rewrite-based methods often use fixed, hand-crafted distance, such as N-gram-based distance [33], Levenshtein distance [53], and negative BERTScore or BARTScore [64], [65], which may not generalize well across target language models, datasets or unobserved prompts.

To elaborate on the second point, we provide a proposition below to mathematically characterize the form of the optimal distance function.

Proposition 3 shows that the optimal distance function should assign the smallest possible distance (zero) when both the input and rewritten text are generated with the LLM, and the largest distance \(M\) when one is LLM-generated and the other is human-written. Crucially, this optimal distance depends on the target LLM to be detected, since different LLMs induce different generative subspaces \(\mathcal{M}\). However, existing rewrite-based detectors rely entirely on fixed distance functions (e.g., editing distance, embedding similarity). As a result, a distance that works well for one model may perform poorly with another, limiting their ability to generalize across different LLMs.

Our proposal. Motivated by the aforementioned limitations, we adopt the rewrite-based approach, and propose to adaptively learn the distance function to improve the detection performance. More specifically, assume we have access to a human-authored corpus \(\mathcal{D}_h\) and an LLM-generated corpus \(\mathcal{D}_m\), both of which are readily available in practice. For instance, \(\mathcal{D}_h\) can be obtained by web-scraping Wikipedia, while \(\mathcal{D}_m\) can be constructed by prompting the target LLM (e.g., GPT, Gemini, or Grok). We next learn the distance function \(d\), parameterized by some parameter \(\phi\), that maximizes the discrepancy between the reconstructions errors: \[\begin{align} \mathbb{E}_{\boldsymbol{X}\sim D_h}\big[d(\boldsymbol{X},\mathcal{R}(\boldsymbol{X}))\big]-\mathbb{E}_{\boldsymbol{X}\sim D_m}\big[d(\boldsymbol{X},\mathcal{R}(\boldsymbol{X}))\big]. \end{align}\] In our implementation, we parameterize the distance function via \[\label{eq:practical-distance} d_{\phi}(\boldsymbol{X}_1, \boldsymbol{X}_2) = \left| \frac{\log p_{\phi}(\boldsymbol{X}_1)}{\texttt{len}(\boldsymbol{X}_1)} - \frac{\log p_{\phi}(\boldsymbol{X}_2)}{\texttt{len}(\boldsymbol{X}_2)} \right|,\tag{3}\] where \(p_{\phi}\) is a language model parameterized by \(\phi\) and \(\texttt{len}(\cdot)\) computes the number of tokens of the input text. It is straightforward to show that \(d_{\phi}\) in equation 3 satisfies the property of a (pseudo)-distance: (i) It is non-negative; (ii) It equals zero whenever \(\boldsymbol{X}_1 = \boldsymbol{X}_2\); (iii) It satisfies the triangle inequality.

Our choice of equation 3 is also motivated by the form of the optimal distance function \(d_{\textrm{opt}}\) in Proposition 3. It can be viewed as a soft relaxation of \(d_{\textrm{opt}}\) which is binary and involves hard indicators, making the objective function continuous and the optimization tractable. Notably, when \(p_{\phi}\) assigns any \(\boldsymbol{X}\in \mathcal{M}\) a probability proportional to \(\kappa^{\textrm{len}(\boldsymbol{X})}\) for some \(0<\kappa<1\), the distance between any two texts produced by the LLM will be exactly zero. To the contrary, when \(p_{\phi}\) assigns very low probability to human-written text, the resulting distance between human- and LLM-authored text will be large.

Our above discussion also highlights the need to adaptively learn the language model \(p_{\phi}\) as opposed to using a fixed model. The ideal \(p_{\phi}\) should: (i) assign low probability to human-authored text; (ii) assign probability more uniformly across tokens for LLM-generated text. This differs from conventional LLMs, which aim to produce coherent, human-like text and therefore tend to assign high probability to human-authored text. Empirically, as demonstrated in the last two panels of Figure 2, the learned distance more effectively distinguishes between human- and LLM-authored text compared to a fixed distance. Our experiments in Section 4.4 also show that, the learned distance function yields substantial improvements over using the initial pre-trained LLM.

To solve the optimization, we initialize \(p_{\phi}\) with a pre-trained LLM and fine-tune a small subset of its parameters to facilitate the computation. This can be done by updating only the final layer or employing low-rank adaptation [66]. Furthermore, since the rewritten text \(\mathcal{R}(\boldsymbol{X})\) is stochastic, we mitigate its randomness by generating multiple reconstructions. Given a text \(\boldsymbol{X}\), we obtain \(K\) reconstructions \(\widetilde{\boldsymbol{X}}_1, \ldots, \widetilde{\boldsymbol{X}}_K\), and estimate the reconstruction error as the average: \(K^{-1}\sum_{k=1}^K d(\boldsymbol{X}, \widetilde{\boldsymbol{X}}_k)\). We classify \(\boldsymbol{X}\) as LLM-generated if this value is smaller than a predetermined threshold, and as human-authored otherwise. We summarize our procedure in Figure 3.

4 Experiments↩︎

We conduct extensive experiments to evaluate the effectiveness of our approach. To save space, we defer additional implementation details to Appendix 9. Our empirical study is designed to answer the following three questions:

1. How does our method perform compared to state-of-the-art approaches under different prompts?

2. How robust is our method under adversarial attacks?

3. To what extent does learning the distance improve the detection accuracy?

To answer the first question, we compare our method, denoted as L2D (short for learn-to-distance), against 12 representative baseline detectors in Sections 4.1 and 4.2, covering both zero-shot (left) and ML-based methods (right):

We also employ 24 datasets and consider 6 commonly used target LLMs such as Llama-3-70B-Instruct [68], Claude-3.5, GPT series [1], [69], and Gemini models [4], [70] for generating LLM-written text.

To answer the second and third questions, we further consider settings under paraphrasing and decoherence attacks in Section 4.3 and compare against a variant of our approach that uses the initial pre-trained model \(p_{\phi}\) without fine-tuning as the distance function in Section 4.4.

Throughout, we have taken care to ensure fairness in all experimental comparisons. Specifically: (i) Both the baseline methods and our algorithm use the same base model, google/gemma-2-9b-it, as the rewrite and/or scoring model to maintain consistency. (ii) For each input text, we use the same set of rewritten texts across all rewrite-based algorithms to ensure a fair comparison. (iii) For algorithms such as ImBD that involve fine-tuning, we use the same optimization hyperparameters (e.g., number of epochs, learning rate) as ours across all cases to ensure fairness in training.

Finally, the area under the curve (AUC) is used as the metric for evaluation.

4.1 Experiments on diverse datasets↩︎

We first evaluate our method on the dataset released by [57]³, which consists of human-written text from 21 domains, including academic writing, business, code, sports and religion. For each human-written sample, four LLM-generated versions were created using Llama-3-70B-Instruct, Gemini 1.5 Pro, GPT-3.5 Turbo and GPT-4o, respectively, yielding a total of 84 settings. Refer to [57] for the detailed prompts used to produce these LLM-generated texts.

Results are reported in Tables ¿tbl:tab:exp-data-GPT-3-Turbo?, ¿tbl:tab:exp-data-GPT-4o? and Tables ¿tbl:tab:exp-data-Llama-3-70B? – ¿tbl:tab:exp-l2r? in Appendix 7. It can be seen that our method achieves the best performance across nearly all combinations of datasets and target models. We focus on comparison against four baselines: (i) FDGPT, a training-free, logits-based zero-shot approach; (ii) ADGPT and (iii) ImBD, both ML-based variants of FDGPT. We include them because, similar to our algorithm, these methods require training. Note that ImBD typically ranks second overall and is the strongest among logits-based approaches; (iv) L2R, a rewrite-based method that also employs ML but learns the rewrite model rather than the distance function. We make two observations:

1. First, our approach consistently achieves substantially larger AUC scores than FDGPT. Notice that, in Tables ¿tbl:tab:exp-data-GPT-3-Turbo?, ¿tbl:tab:exp-data-GPT-4o? and ¿tbl:tab:exp-data-Gemini-1465-Pro?, the training and testing data differ in terms of models or data contexts, which reduces the inherent advantage of ML-based approaches over zero-shot methods such as FDGPT. Even under these shifts, our method continues to achieve the best performance in most cases. This comparison highlights our algorithm’s robustness to distributional shifts between the training and testing data, as well as its effectiveness relative to zero-shot methods.

2. Second, as shown in Tables ¿tbl:tab:exp-data-GPT-3-Turbo?, ¿tbl:tab:exp-data-GPT-4o?, ¿tbl:tab:exp-data-Llama-3-70B? and ¿tbl:tab:exp-data-Gemini-1465-Pro?, our approach outperforms ImBD on most datasets (16 – 19 out of 21), and the relative gain can reach up to 89.4% (see the rightmost column). This comparison highlights the advantage of rewrite-based methods over logits-based methods.

3. Third, since L2R does not provide public code, we directly compare against the reported results in their paper. Table ¿tbl:tab:exp-l2r? shows that our method outperforms L2R on 20 out of 21 datasets, and often by a large margin. This comparison suggests that, compared with learning to rewrite, learning a distance function is more effective for rewrite-based detection.

width=

4.2 Experiments under different prompts↩︎

Next, following [51], we examine three scenarios that use different types of unseen prompts to generate LLM text: (i) rewrite, where the LLM rewrites a human-authored text while preserving its semantic meaning; (ii) expand, where the LLM elaborates on the text according to a style randomly selected from various options (e.g., formal, literary); and (iii) polish, where the LLM refines the text based on the randomly chosen style.

We also consider three widely used benchmark datasets [28], [51]: (i) Wiki, which consists of Wikipedia-style question answering data [71]; (ii) Story, which focuses on story generation [72]; and (iii) News, which is concerned with news summarization [73].

We further generate LLM-authored text using three recent and popular proprietary models: (i) GPT-4o; (ii) Claude-3.5-Haiku and (iii) Gemini-2.5-Flash. This yields a total of 27 settings. Details on how these texts were generated are provided in Appendix 9.

Table 1 presents the AUC scores for all detectors across the 27 combinations of datasets, target models, and types of prompts. Our method achieves the best performance in nearly all cases, whereas ImBD (logits-based) or RAIDAR (rewrite-based) works as the second best. The relative gain over these best baselines is 70.11% on average, which again highlights (i) the advantage of rewrite-based methods over logits-based methods in settings with unseen prompts; and (ii) the effectiveness of learning an adaptive distance function over using a fixed distance in rewrite-based approaches.

3pt

4.3 Experiments against Adversarial Attack↩︎

Following [28], we further evaluate the robustness of our method against two types of adversarial attacks: (i) Rephrasing, where the LLM-written text is further paraphrased by a T5-based paraphraser before detection; (ii) Decoherence, where in each LLM-generated sentence containing more than 20 words, two adjacent words are randomly swapped. Both attacks are designed to reduce the coherence of LLM-generated text and have been shown to degrade the detection accuracy of existing detectors [28].

We conduct experiments on the same three datasets used in Section 4.2, resulting in a total of six settings. For comparison, we focus on ImBD and RAIDAR, as they achieve the second best performance on these datasets.

Figure 4 reports the AUC scores with and without adversarial attacks. While RAIDAR achieves comparable or superior AUCs on Story and Wiki in the absence of attacks, its AUC drops substantially under attacks, failing to maintain its lead. Similarly, ImBD’s AUC declines considerably on Wiki under the rephrasing attack. In contrast, our method remains robust: its AUC either increases or remains unchanged on News, and only slightly decreases on other two datasets, achieving the best performance in each setting. This highlights the resilience of our approach to adversarial attacks and demonstrates its potential for reliable deployment in real-world scenarios.

4.4 Ablation study↩︎

We conduct an ablation study to compare against a version of our approach that uses the initial language model \(p_{\phi}\) to construct the distance (FD, denoting a fixed distance). We consider the same settings to Section 4.2 and report the AUCs in Table 2. Our method consistently outperforms FD, with an average improvement of 97.1%. These results clearly demonstrate the advantage of learning the distance metric over fixing the distance.

5 Discussion↩︎

This paper studies the detection of LLM-generated text. Our theoretical analysis offers geometric insights to demonstrate the effectiveness of rewrite-based approaches (Proposition 1) and their robustness to unseen prompts (Proposition 2). Methodologically, we go beyond existing rewrite-based methods by adaptively learning the distance function, which is theoretically grounded (Proposition 3) and delivers substantial empirical gains over both fixed-distance approaches (Section 4.4) and state-of-the-art detectors (Sections 4.1 and 4.2), while maintaining robustness against adversarial attacks (Section 4.3).

To conclude this paper, we remark that in our theoretical analysis, the assumptions were intentionally simplified (and thus stronger) to build geometric intuition behind these approaches. In Appendix 6, we have offered a more complex version of our theories under less restrictive assumptions. Finally, although our method achieves state-of-the-art detection accuracy in most settings, its computational cost remains relatively high and comparable to existing rewrite-based algorithms (e.g., RAIDAR), due to the need to generate multiple rewrites (see Appendix 7 for detailed runtime results). This represents a potential limitation. We also note that asynchronous rewriting and distance computation using a vLLM backend can improve computational efficiency for practical deployment.

Acknowledgement↩︎

Hongyi Zhou and Ying Yang’s research was partially supported by NSFC 12271286. The authors thank the anonymous referees and the area chair for their insightful and constructive comments, providing a significant improvement of the initial paper.

Ethics statement↩︎

The research presented in this paper adheres to the ICLR Code of Ethics (https://iclr.cc/public/CodeOfEthics) in all respects.

Reproducibility statement↩︎

We have made substantial efforts to ensure the reproducibility of this paper. The assumptions of our method are declared in Section 2, and the proofs of the theoretical results are provided in Appendix 6. The implementation details of our approach (e.g., the choice of hyperparameters) are described in Appendix 8. Additionally, the experimental setup and data generation procedures are explained in Section 4 and Appendix 9. Together, these descriptions provide sufficient information for others to reproduce both our theoretical and empirical results.

6 Proofs and additional theoretical results↩︎

Proof of Proposition 1: We further assume \(\mathcal{M}\) is a closed convex set so that the projection operator is well-defined. Then for any \(x\in\mathcal{X}\) and \(y\in\mathcal{M}\), we have \[\langle x-\Pi_{\mathcal{M}}(x), y - \Pi_{\mathcal{M}}(x)\rangle \leq 0.\] Taking \(y = \mathcal{R}(x)\), it directly follows that \[\begin{align} d^*(x,\mathcal{R}(x)) &=& d^*(x,\mathcal{R}(x) -\Pi_{\mathcal{M}}(x) + \Pi_{\mathcal{M}}(x))\nonumber\\ &=& d^*(x,\Pi_{\mathcal{M}_m}(x))-2\langle x- \Pi_{\mathcal{M}}(x), \mathcal{R}(x) - \Pi_{\mathcal{M}}(x)\rangle + | \mathcal{R}(x) - \Pi_{\mathcal{M}}(x)| \nonumber\\ &\ge& d^*(\Pi_{\mathcal{M}}(x) , \mathcal{R}(x) ) \quad \text{ for all } x\in\mathcal{X}.\nonumber \end{align}\] Taking expectation on both sides with respect to \(\boldsymbol{X}\sim p\), we obtain \[\mathbb{E}_{\boldsymbol{X}\sim p}\left\{d^*(\boldsymbol{X},\mathcal{R}(\boldsymbol{X}))\right\} \geq \mathbb{E}_{\boldsymbol{X}\sim p}\left\{d^*(\Pi_{\mathcal{M}}(\boldsymbol{X}) , \mathcal{R}(\boldsymbol{X}) )\right\} = \mathbb{E}_{\boldsymbol{X}\sim p}\left\{d^*(\Pi_{\mathcal{M}}(\boldsymbol{X}) , \mathcal{R}(\Pi_{\mathcal{M}}(\boldsymbol{X})) )\right\},\] where the last equality follows from Assumption 3. Finally, Assumption 2 yields that \[\mathbb{E}_{\boldsymbol{X}\sim p}\left\{d^*(\Pi_{\mathcal{M}}(\boldsymbol{X}) , \mathcal{R}(\Pi_{\mathcal{M}}(\boldsymbol{X})) )\right\} = \mathbb{E}_{\boldsymbol{X}\sim q}\left\{d^*(\boldsymbol{X} , \mathcal{R}(\boldsymbol{X}) )\right\}.\] Thus, the conclusion of Proposition 1 follows.

Proof of Proposition 2: According to the definition of projection operator \(\Pi_\mathcal{M}\) and the fact that \(\mathcal{R}(\boldsymbol{X})\) is supported on \(\mathcal{M}\), it is obvious that \[\label{eqn:prop2-term1} d^*(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})) \geq d^*(\boldsymbol{X},\Pi_{\mathcal{M}}(\boldsymbol{X})).\tag{4}\] Furthermore, the distribution of \(q_{\texttt{prompt}}\) is also supported on \(\mathcal{M}\). Therefore, combining equation equation 2 , we obtain \[\begin{align} \label{eqn:prop2-term2} \mathbb{E}_{\boldsymbol{X}\sim q_{prompt}}[d^*(\boldsymbol{X},\mathcal{R}(\boldsymbol{X}))] &=& \mathbb{E}_{\boldsymbol{X}\sim q_{prompt}}[d^*(\Pi_{\mathcal{M}}(\boldsymbol{X}),\mathcal{R}(\boldsymbol{X}))] \nonumber\\ &=&\mathbb{E}_{\boldsymbol{X}\sim q_{prompt}}[d^*(\Pi_{\mathcal{M}}(\boldsymbol{X}),\Pi_{\mathcal{M}}(\boldsymbol{X}) + e)]\nonumber\\ &=& \mathbb{E}_{\boldsymbol{X}\sim q_{prompt}}|e| \leq \epsilon. \end{align}\tag{5}\] Combining inequality equation 4 and equation 5 , the conclusion of Proposition 2 then follows.

Additional Results. The geometric assumptions in Section 2 were intentionally simplified to make our propositions interpretable. In fact, these assumptions could be relaxed to a more realistic setting. Specifically, we only assume

1. Human- and LLM-generated text lie on two nonlinear manifolds \(\mathcal{H}\) and \(\mathcal{M}\subseteq \mathcal{X}\), with their intrinsic dimensions \(d_h>d_m\);

2. Rewriting satisfies \(\mathbb{E} [d^*(\mathcal{R}(x),x)]\le \varepsilon_0\) for any \(x\in \mathcal{M}\) and some small \(0<\varepsilon_0<1\), whereas \(\sup_{x_1,x_2\in \mathcal{M}\cup \mathcal{H}}d^*(x_1,x_2)=1\);

3. Human-written text distribution \(p\) is absolutely continuous with respect to some \(d_h\)–dimensional volume measure \(\mu\) on \(\mathcal{H}\) with a bounded density.

Notice that (i) relaxes the linearity condition in Assumption 2 and does not assume that \(\mathcal{M}\) is a projection or subspace of \(\mathcal{H}\). Meanwhile, the assumption \(d_h>d_m\) is well supported by empirical findings [74] which demonstrate that human text typically has intrinsic dimension of 8.5 - 10, whereas LLM-generated text has a dimension of only 6 – 8 [74].

Furthermore, (ii) only requires that, for LLM-generated text, its reconstruction error is on average small relative to the maximum distance in the space. It does not require the error to be almost surely small as in the additive noise model, nor does it require equivalence in Assumption 3. In our empirical study, we find the ratio of this expected reconstruction error to the maximum distance is consistently very small across multiple datasets (see Table ¿tbl:tab:dataset95target95ratio?).

Under these realistic assumptions, we obtain the following proposition:

Proposition. Let \(\kappa := d_h - d_m\). Under Assumptions (i)–(iii), for a human text \(\boldsymbol{X}\) and an LLM-generated text \(\boldsymbol{Y}\), the inequality \[\mathbb{E}_{\widetilde{\boldsymbol{X}}\sim \mathcal{R}(\boldsymbol{X})}[d^*(\boldsymbol{X},\widetilde{\boldsymbol{X}}))]>\mathbb{E}_{\widetilde{\boldsymbol{Y}}\sim\mathcal{R}(\boldsymbol{Y})}[d^*(\boldsymbol{Y},\widetilde{\boldsymbol{Y}})]\] holds with probability at least \(1-O(\varepsilon_0^\kappa)\), where the expectations on both sides average out fluctuations in the rewriting process.

width=0.8

Remark 1: Given that empirical results suggest \(\kappa\) is approximately 1.5 or 2 [74], the probability \(1-O(\varepsilon_0^\kappa)\) can be very close to \(1\) given that \(\varepsilon_0\) is sufficiently small, which in turn proves that the reconstruction error for human-written text is, on average, larger than that for LLM-generated text.

Remark 2: The proof of the proposition relies on leveraging the assumption that \(\mathcal{M}\) has a strictly lower intrinsic dimension than \(\mathcal{H}\). Consequently, its \(\varepsilon-\)neighborhood overlaps with at most an \(O(\varepsilon^\kappa)\) fraction of the human-text manifold. As a result, only a small proportion of human-written text lie within the \(\varepsilon-\)neighborhood of \(\mathcal{M}\); most human text lie farther away, leading to the a larger reconstruction error.

Proof: Formally, for \(\varepsilon>0\), we denote the \(\varepsilon_0\)–tube (w.r.t.\(d^\star\)) around \(\mathcal{M}\) as \[\mathcal{N}_{\varepsilon_0}(\mathcal{M}):=\{x\in \mathcal{X}:\;d^*\left(x,\mathcal{M}\right)\le \varepsilon_0\}.\] Classical tube formulas imply \[\mu\!\big(\mathcal{H}\cap \mathcal{N}_{\varepsilon_0}(\mathcal{M})\big)\;=\;O(\varepsilon_0^{\kappa})\quad\text{as }\varepsilon_0\downarrow 0.\] Hence, under the bounded density assumption in (iii), \[\label{ref:someeq} \mathbb{P}_{\boldsymbol{X}\sim p}\!\big\{d^*(\boldsymbol{X},\mathcal{M})<\varepsilon_0\big\} \;\le\;C\,\mu\!\big(\mathcal{H}\cap \mathcal{N}_{\varepsilon_0}(\mathcal{M})\big) \;=\;O(\varepsilon_0^\kappa)\tag{6}\] for some constant \(C\). Therefore, with probability at least \(1 - O(\varepsilon_0^\kappa)\), \[\begin{align} \mathbb{E}_{\widetilde{\boldsymbol{X}}\sim \mathcal{R}(\boldsymbol{X})}[d^*(\boldsymbol{X},\widetilde{\boldsymbol{X}})]-\mathbb{E}_{\widetilde{\boldsymbol{Y}}\sim \mathcal{R}(\boldsymbol{Y})}[d^*(\boldsymbol{Y},\widetilde{\boldsymbol{Y}})] \ge d^*(\boldsymbol{X},\mathcal{M})-\varepsilon_0>0. \end{align}\] The proof is hence completed.

Proof of Proposition 3: Given that \(d\) is bounded between \(0\) and some positive constant \(M\), we have \(\mathbb{E}_{\boldsymbol{X}\sim p}[d(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})]\leq M\) and \(\mathbb{E}_{\boldsymbol{X}\sim q_{\textrm{prompt}}}[d(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})]\geq 0\). Therefore, the reconstruction error is upper bounded by \(M\). In what follows, we prove that by choosing \(d = d_{\tiny{opt}}\), we can achieve this upper bound.

To prove this, we assume (i) – (iii) hold. As commented earlier, these assumptions are mild and are supported by empirical observations. Under these assumptions, letting the value of \(\epsilon_0\) in equation 6 approach \(0\), it follows that \[\begin{align} \mathbb{P}_{\boldsymbol{X}\sim p}(\boldsymbol{X}\in \mathcal{M})=0. \end{align}\] Additionally, notice that the rewrite \(\mathcal{R}(\boldsymbol{X})\) always lies in \(\mathcal{M}\), it follows that \[\mathbb{E}_{\boldsymbol{X}\sim p}[d_{opt}(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})] = \mathbb{E}_{\boldsymbol{X}\sim p}[d_{opt}(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})\mathbb{I}(\boldsymbol{X}\in\mathcal{H} \backslash \mathcal{M})] = M.\] Additionally, since \(q\) is supported on \(\mathcal{M}\), it follows that \[\mathbb{E}_{\boldsymbol{X}\sim q_{\textrm{prompt}}}[d_{opt}(\boldsymbol{X},\mathcal{R}(\boldsymbol{X})] = 0.\] Thus, under distance \(d_{opt}\), the reconstruction error achieves the upper bound, which completes the proof.

7 Additional implementation details and numerical experiments↩︎

We first provide an outline of our algorithm, which can be summarized into the following four steps:

1. Collect a dataset of human-authored text (denoted by \(\mathcal{D}_h\)) and prompt the target LLM (e.g., GPT-4o) to obtain an LLM-generated dataset (denoted by \(\mathcal{D}_m\)).

2. For each text \(X\in \mathcal{D}_h\cup \mathcal{D}_m\), prompt an open-source lightweight LLM (specified below) to rewrite it \(K\) times, and denoted the \(K\) reconstructions by \(\widetilde{\boldsymbol{X}}_1,\cdots,\widetilde{\boldsymbol{X}}_K\).

3. Learn a distance function \(d_\phi\) that maximizes the difference in reconstruction errors between \(\mathcal{D}_h\) and \(\mathcal{D}_m\): \[\max_\phi \;\mathbb{E}_{X \sim \mathcal{D}_h}\left[ \frac{1}{K}\sum_{k=1}^K d_\phi(\boldsymbol{X}, \widetilde{\boldsymbol{X}}_k) \right] - \mathbb{E}_{X \sim \mathcal{D}_m}\left[ \frac{1}{K}\sum_{k=1}^K d_\phi(\boldsymbol{X}, \widetilde{\boldsymbol{X}}_k) \right],\] where \(d_\phi(\boldsymbol{X}_1, X_2) = | \log p_\phi(\boldsymbol{X}_1)/|X_1| - \log p_\phi(\boldsymbol{X}_2)/|X_2| |\) and \(p_\phi\) is a language model whose architecture will be detailed below.

4. Given an input text \(X\), obtain its reconstructions \(\widetilde{\boldsymbol{X}}_1,\cdots,\widetilde{\boldsymbol{X}}_K\). If \[\frac{1}{K}\sum_{k=1}^K d_\phi(\boldsymbol{X}, \widetilde{\boldsymbol{X}}_k),\] exceeds a predefined threshold, classify \(X\) as human-authored.

width=

width=

In our experiments, the training and testing data differ in terms of models or data contexts. Specifically, in Tables ¿tbl:tab:exp-data-GPT-3-Turbo? and ¿tbl:tab:exp-data-GPT-4o?, we train the distance function on text generated by GPT-4 and evaluate its performance to detect GPT-3.5-Turbo, and vice versa. In Table ¿tbl:tab:exp-data-Gemini-1465-Pro?, we train the distance function on GPT-generated text but test it on text produced by Gemini. Thus, in all three tables, the training and testing models are either completely different or belong to the same family but correspond to different versions.

Moreover, all reported results therein are obtained via cross-fitting: we use one category of data (e.g., Story in Table 1) for testing and other categories (e.g., News and Wiki) for training. Consequently, the test data differ in content and domain from the training data.

Table 3 reports the average AUC and runtime of our method compared with RAIDAR, a state-of-the-art rewrite-based detector, in the setting of detecting text generated by GPT-3.5-Turbo (same to Table ¿tbl:tab:exp-data-GPT-3-Turbo?). As shown, our runtime is very close to that of RAIDAR – with only a slight increase – while achieving a substantial improvement in AUC. In addition, the reported runtime does not use a vLLM backend; incorporating vLLM could further reduce computational cost.

width=

1.2pt

width=

It is well known that varying the sampling temperature produces different outputs from LLMs, and adjusting temperature is a commonly used strategy in real-world LLM usage [75]. In practice, when collecting text from an LLM, the specific temperature setting is typically unknown. It is therefore important to evaluate whether our method remains robust when training and test data are generated with different temperatures.

Following the same data generation process described in Section 4.3, we extend the setting to include six temperature values: \(\{0.01, 0.2, 0.4, 0.6, 0.8, 1.0\}\). For evaluation, we partition the datasets into training and testing splits based on temperature. Specifically, one split uses \(\{0.2, 0.6, 0.8\}\) for training and \(\{0.01, 0.4, 1.0\}\) for testing, and the roles are reversed in the other split. This design mimics realistic scenarios where data collected at one set of temperatures are used to detect text generated at unseen temperatures.

As shown in Figure 6, our method achieves performance nearly identical to the case where training and test data share the same temperature. These results highlight the robustness of our approach under temperature variation.

8 Implementation↩︎

Prompt for rewriting. The prompt is set as: You are a rewriting expert and you would rewrite the text without missing the original details. Return ONLY the rewritten version. Do not explain changes, do not give multiple options, and do not add commentary. Original text:.

To generate rewritten texts, we employ an open-source model available on HuggingFace, i.e., google/gemma-2-9b-it. We recommend using an instruction fine-tuned variant, as it is more likely to produce faithful rewrite. In addition, the model should contain at least a billion parameters, since smaller models often fail to generate reliable rewrite. Choosing a open-source LLM does not require access to proprietary models like ChatGPT and Grok, making our approach being affordable and accessibility. We set the max_new_tokens as the 1.2 times of the number of tokens in \(\boldsymbol{X}\), and the min_new_tokens as the 0.8 times of the number of tokens in \(\boldsymbol{X}\).

Rewrite times \(K\). The parameter \(K\) plays a critical role in balancing computational cost and detection performance. Increasing \(K\) improves the accuracy of estimating \(\tau\), but at the expense of longer training time—since probabilities \(p_{\phi}(\widetilde{\boldsymbol{X}}_1), \ldots, p_{\phi}(\widetilde{\boldsymbol{X}}_K)\) must all be computed—and higher GPU memory requirements during backpropagation. Figure 5 illustrates the trade-off: while larger \(K\) generally improves performance, the gains diminish beyond small values, whereas the runtime and memory usage grow roughly linearly. Notably, as long as \(K > 1\), the AUC remains strong. Motivated by this observation, we adopt a modest choice of \(K=4\) throughout all experiments, striking a balance between accuracy and efficiency.

Fine-tuning setting. In our specific fine-tuning, we set the distance function as \(d_{\phi}(\boldsymbol{X}_1, \boldsymbol{X}_2) = |\log p_{\phi}(\boldsymbol{X}_1) / \texttt{len}(\boldsymbol{X}_1) - \log p_{\phi}(\boldsymbol{X}_2) / \texttt{len}(\boldsymbol{X}_2)|\) where \(\texttt{len}(\boldsymbol{X}_k)\) is the number of tokens of \(\boldsymbol{X}_k\) (\(k=1,2\)). This normalization accounts for text length, as a longer text are expected to correspond to smaller log-likelihood. Without loss of generality, we set \(p_{\phi}\) as the model used for generating the rewritten text. We fine-tune the model, employ LoRA [66] implemented in the peft library, with rank parameter set to 8, lora_alpha set to 32, and lora_dropout set to 0.1, and the other parameters use the default settings.

9 Experiments: details↩︎

This section describes the experimental setup in detail. It is worth noting that throughout all experiments, we use AUC as the evaluation metric, and the relative gain over the strongest baseline is computed as: \((\mathrm{Our AUC} - \mathrm{StrongestBaseline's AUC}) / (1.0 - \mathrm{StrongestBaseline's AUC})\).

9.1 Experimental Setup on Diverse Datasets↩︎

Setup for learning-based methods. For fairness, we follow a consistent training protocol across training-based detectors. Specifically, for each method, we train on 10 out of the 21 datasets and evaluate on the remaining ones. We then repeat the process by swapping the training and test splits, ensuring that no evaluation data leaks into training and guaranteeing a fair comparison. For RoBERTa and RADAR, since only pre-trained checkpoints are publicly available, we directly use the models released on HuggingFace⁴⁵. This setup also enables a reasonable comparison with L2R, which uses 70% of each dataset for training and the remainder for testing. In contrast, our method trains on fewer datasets and the evaluation datasets are out of domains yet still achieves better performance, highlighting the effectiveness of the learning procedure.

Setup for zero-shot methods. For zero-shot detectors, we employ the same open-source LLMs as surrogate models to compute their statistical measures. These include Likelihood, IDE, and LRR. Notice that, the implementation of IDE⁶ provide two method for estimating intrinsic dimension, one is based on persistence homology and another is based on maximum likelihood estimation [76]. Since the former requires a large amount of time on computing, we use maximum likelihood estimation in the experiments. For Binoculars and FDGPT, which require both a sampling model and a scoring model, we set \(p_{\phi}\) as the scoring model and use its corresponding base model as the sampling model. For BARTScore, which also involves rewriting, we align its rewriting step with our own method while using the pre-trained BARTScore model from HuggingFace⁷ to compute distances.

9.2 Experimental Setup on different prompts↩︎

Data generation. We generate machine-generated texts with three state-of-the-art LLMs: GPT-4o, Claude-3.5-Haiku, and Gemini-2.5-Flash. They specific version are: gpt-4o-2024-08-06, claude-3-5-haiku-20241022.

We next describe the specific system prompts and user prompts that are used for generating texts. First, for the rewrite task, the system prompt is:

For the polish task, the system prompt is:

For the expand task, the system prompt is:

For Gemini-2.5-Flash and Claude-3.5-Haiku, we additionally append the instruction in the system prompt:

Return ONLY the rewritten/polished/expanded version. Do not explain changes, do not give multiple options, and do not add commentary.

This ensures the output is strictly aligned with the assigned task.

The user prompt depends on the task. For rewriting, it takes the form: Please rewrite: [a human text]. For the expansion task, one of several predefined style prompts⁸ is selected (e.g., “Expand but not extend the paragraph in an oral style” or “Expand but not extend the paragraph in a literary style”). For polishing, a prompt is similarly chosen from a predefined set⁹ (e.g., “Help me refine a paragraph with a lyrical touch. Enhance the flow and imagery, making the words sing together in perfect harmony”).

Given these settings, each LLM generates texts from human-written texts randomly sampled from one of source datasets. In the generation process, we set the temperature parameter of LLM as 0.8. This process is repeated 100 times on one source dataset and one task, yielding a dataset of 100 machine-generated and 100 human-written texts. With three tasks, three LLMs, and three data sources, we obtain a total of 27 evaluation datasets.

Setup of Baselines. Baseline setups largely follow the procedure in Section 9.1, with slight modifications to the training data. For instance, when evaluating performance on the News dataset, the Wiki and Story datasets are used for training. The process is repeated analogously when evaluating on the Wiki or Story datasets.

9.3 Experimental Setup for Adversarial Attacks and Ablation↩︎

To evaluate the robustness of our approach against adversarial attacks, we adopt the attacks in [28]. In particular, for the rephrasing attack, we use the T5-based paraphraser available on HuggingFace¹⁰ to paraphrase text generated by Claude-3.5 prior to detection.

In the ablation study, both FD and our method rely on the exact same rewritten texts to compute distance. This setup reflects the contribution of our adaptive distance learning procedure.

10 Declaration: LLM usage↩︎

In preparing this paper, the LLM was used only for writing and editing, and it does not impact the core methodology.

References↩︎

---

1. Hongyi Zhou and Jin Zhu contributed equally to this paper and are listed in alphabetical order.↩︎

2. Corresponding author: e.xu2@lse.ac.uk, c.shi.7@lse.ac.uk↩︎

3. https://github.com/ranhli/l2r_data↩︎

4. https://huggingface.co/openai-community/roberta-large-openai-detector↩︎

5. https://huggingface.co/TrustSafeAI/RADAR-Vicuna-7B↩︎

6. https://github.com/ArGintum/GPTID↩︎

7. https://huggingface.co/facebook/bart-large-cnn↩︎

8. https://github.com/Jiaqi-Chen-00/ImBD/blob/main/data/expand_prompt.json↩︎

9. https://github.com/Jiaqi-Chen-00/ImBD/blob/main/data/polish_prompt.json↩︎

10. https://huggingface.co/Vamsi/T5_Paraphrase_Paws↩︎