1 Introduction↩︎

The rapid deployment of large language models (LLMs) across critical applications, from educational assessment to content creation, has made reliable attribution mechanisms an urgent necessity [1]–[3]. Can we determine whether a given text was generated by an AI system? In an era where synthetic text has become increasingly indistinguishable from human writing, these questions have profound implications for academic integrity, content authenticity, and intellectual property protection [4], [5]. Watermarking has emerged as the technical solution to this attribution problem [6], [7]. By embedding statistical signatures during text generation, unbiased watermarking promises to make AI-generated content detectable while minimising quality degradation and avoiding architectural changes [8], [9].

However, current watermarking research relies on a critical simplifying assumption: adversaries have access to only a single watermarked model. In reality, users today can easily and freely access 10+ frontier LLMs through unified platforms (e.g., Hugging Face), such as GPT, LLaMA, Qwen, Mistral, and dozens of other capable models. The modern LLM landscape is not a monopoly but a hyper-competitive marketplace with multiple providers, and this competitive structure is the Achilles’ heel of watermarking.

Our key insight is deceptively simple: watermarking works by perturbing a model’s output distribution, and these perturbations are independent across providers due to different secret keys and architectural design. By querying multiple models in parallel and averaging their output distributions, these independent perturbations cancel out, recovering the original unwatermarked distribution (as illustrated in Figure 1). We formalise this by proving that for any unbiased watermarking scheme with independent per-model perturbations, linear ensembling recovers the consensus distribution up to a second-order error with convergence rate \(O(1/\sqrt{N})\). This establishes a fundamental limit rooted in market structure: competitive providers must use secret keys for provenance verification (guaranteeing independence) and maintain quality to retain users (bounding perturbation magnitude). Under these constraints, watermark signals are mathematically guaranteed to vanish under averaging.

While theoretical result guarantees asymptotic removal, practical deployment faces critical obstacles in efficiency and coherence. Frameworks that seek to approximate watermark parameters via “random selection probing” [10] require extensive iterative querying, causing prohibitive inference latency. Meanwhile, attacks that concatenate tokens from heterogeneous models require predicting additional tokens at every time step and re-encoding the entire context at every switch [11], incurring significant computational inefficiency. To bridge these gaps, we introduce WASH (Watermark Attenuation via Statistical Hybridisation). WASH employs fluency-aware routing to overcome vocabulary mismatches, enabling effective probability aggregation across distinct tokenisers. Furthermore, by leveraging parallel inference with response caching, WASH eliminates the need for iterative probing or context re-encoding, achieving constant-time per-token complexity regardless of the ensemble size \(N\). Extensive experiments demonstrate that WASH improves generation quality by \(27.5\%\) while running \(6\times\) faster than state-of-the-art removal baselines.

Our analysis reveals that watermarking faces a fundamental choice: True robustness that distinguishes AI from human text regardless of model mixing requires coordinated watermarking, namely some common signal shared across all models. This paper rigorously characterises this fundamental limitation through the following contributions:

1. We theoretically prove that linear ensembling asymptotically recovers the original unwatermarked distribution. We demonstrate the convergence rate is \(O(1/\sqrt{N})\) with \(N\) independent models, revealing fundamental limits of unbiased watermarking in multi-provider settings.

2. We introduce WASH, a novel algorithm that overcomes the vocabulary mismatch problem in heterogeneous model ensembles. By employing fluency-aware routing and context re-synchronisation, WASH preserves semantic integrity while neutralising watermark signals.

3. We conduct a systematic evaluation across six representative watermarking schemes and three LLMs under two complementary detection settings. Experiments demonstrate near-complete detection failure: (a) for generation-time detection, WASH suppresses z-scores from 5-300 (strongly detectable) to \(<2\) (near random choice) with just 3 models; (b) for final-text detection, WASH lowers the TPR@5%FPR on native sequence detectors to below 50%, rendering the watermarks statistically undetectable.

2 Methodology↩︎

Watermarking exhibits a structural statistical vulnerability when outputs from multiple independently watermarked models are aggregated. Formally, we conceptualise watermarking as a stochastic perturbation applied to a shared underlying distribution \(p^*\). While these perturbations are necessary for detection, they act as uncoordinated noise across different providers. Consequently, when outputs from multiple models are combined, the watermark signals interfere destructively, allowing the underlying consensus distribution to be asymptotically recovered.

We now formalise this perspective by characterising the relationship between the consensus distribution, the perturbed watermarked distributions, and their aggregation. Let \(\mathcal{V}\) be the vocabulary and \(\mathcal{X}\) be the space of possible contexts (i.e., sequences of tokens). We denote the probability distributions over the vocabulary as \(\Delta(\mathcal{V}) = \left\{ \mathbf{p} \in \mathbb{R}^{|\mathcal{V}|} \;\middle|\; p_v \ge 0, \sum_{v \in \mathcal{V}} p_v = 1 \right\}\).

2.1 Problem Formulation↩︎

Conceptually, the probability distribution of any model \(i\) deviates from the ideal semantic distribution \(p_{GT}\) (align with human expert) due to shared systematic errors and provider-specific variations. We model this as: \[p_{i}(v|x) \propto p_{GT}(v|x) \cdot \exp\left( \underbrace{\delta_{sys}(v,x)}_{\text{Shared Model Bias}} + \underbrace{\delta_i(v,x)}_{\text{Watermark}} \right). \label{eq:conceptual95model}\tag{1}\] Here, \(\delta_{sys}\) represents common biases inherent to current LLM architectures, while \(\delta_i\) encapsulates the provider-specific signal, primarily the watermarking signal, but also including model-specific characteristics.

The key insight enabling watermark removal is that perturbations across independent providers are statistically unbiased. We formalise this assumption below.

This assumption is natural in the multi-provider setting: (a) Market forces impose a strict upper bound on perturbation magnitude (\(\xi\)), as any watermark strong enough to violate the linear approximation (large \(\delta\)) would result in perceptible quality degradation. (b) Each provider employs watermark configurations unknown to each other, making the perturbations behave as independent random variables. This can be relaxed to a grouped setting where providers share common latent factors: under between-group independence and within-group conditional independence, the same convergence guarantee holds; see Appendix 7. (c) Providers independently optimise for quality, with no reason to systematically favor or disfavor specific tokens beyond what \(p^*\) suggests. We further stress-test this zero-mean assumption under deliberately biased watermark perturbations in Appendix 8. (d) Finally, providers maximise utility by prioritising stability on high-probability tokens; the \(p^*\)-weighting ensures that significant variance is restricted to rare tokens where it least impacts the overall text quality.

2.2 Watermark Removal via Linear Ensembles↩︎

Given \(N\) independent models, we propose to neutralise the watermarking signals and recover the consensus distribution via a uniform mixture, a method we term WASH. Since the perturbations \(\delta_i\) are uncorrelated across providers, they act as noise that can be averaged out. Given a fixed content \(x\in \mathcal{X}\), the aggregated probability distribution is defined as \[\label{eq:aggregated} \bar{p}_N(\cdot|x) := \frac{1}{N} \sum_{i=1}^{N} p_{i}(\cdot|x).\tag{4}\] We implement this removal process as an autoregressive ensemble. The generation process can be formalised as a recursive sequence. Let \(x_{<t} = (x_1, \dots, x_{t-1})\) denote the context at step \(t\). The next token \(x_t\) is sampled from the aggregated distribution: \[x_t \sim \bar{p}_N(\cdot | x_{<t}).\] The process repeats until a termination token is generated. Having formalised the generation process, we now provide theoretical guarantees for its effectiveness.

2.3 Preserving Semantic Integrity in Heterogeneous Ensembles↩︎

While linear ensemble effectively neutralises watermark perturbations in the limit of \(N\), strictly enforcing this over heterogeneous topologies introduces a vocabulary mismatch problem [12], [13]. Let \(\mathcal{V}_i\) denote the vocabulary of model \(i\). A standard baseline restricts sampling to the vocabulary intersection \(\mathcal{V}_{\cap} = \bigcap_{i=1}^N \mathcal{V}_i\), thereby ensuring distributional consensus across all models. However, this conservative approach induces expressivity bottleneck. Restricting generation to the vocabulary intersection \(\mathcal{V}_{\cap}\) often excludes semantic ground truth tokens, such as specific entities or technical terms that are absent in at least one model’s vocabulary (i.e., \(\mathcal{V}_{\Delta} = (\bigcup \mathcal{V}_i) \setminus \mathcal{V}_{\cap}\)).

Conversely, projecting onto the vocabulary union \(\mathcal{V}_{\cup} = \bigcup_{i=1}^N \mathcal{V}_i\) introduces granularity mismatch: as illustrated in Figure 2, heterogeneous tokenisers may represent the same semantic unit at different granularities. For instance, one model represents “Gracious” as “[Gr], [acious]” while another decomposes it into “[Gra], [cious]”. Aggregating over the union forces the imputation of undefined probabilities for different disjoint tokens (e.g., the token “[Gra]” does not exist in the first model’s vocabulary), thereby diluting the semantic fidelity. To reconcile vocabulary heterogeneity with watermark removal, we propose a dynamic switching process that routes generation between the ensembled distribution and a set of local “specialist” models based on support constraints. The generation at step \(t\) is determined by the policy outlined below:

2.3.0.1 Generation via Union-based Ensemble

To avoid semantic truncation, we construct the ensemble distribution \(\bar{p}_N\) over the vocabulary union \(\mathcal{V}_{\cup}\). For any model \(i\) with token \(v \notin \mathcal{V}_i\), we assign \(p_i(v|x_{<t}) = 0\), so that the ensemble average reduces to \(\bar{p}_N(v|x_{<t}) = \frac{1}{N}\sum_{i: v \in \mathcal{V}_i} p_i(v|x_{<t})\). For any token supported by only one model, this caps \(\bar{p}_N(v|x_{<t})\) at approximately \(1/N\) of that model’s own probability. However, this scaling does not indicate a loss of confidence: models lacking \(v\) in their vocabulary still allocate comparable probability mass to the same semantic content, but route it through alternative tokenisation paths (e.g., one model emits “Gracious” as an atomic token while others emit the same word as “[Gr][acious]”). The semantic signal is therefore dispersed across divergent tokenisation paths rather than discarded. Crucially, unlike the intersection approach, every token in \(\mathcal{V}_\cup\) remains accessible. The next token is then sampled from this union-aggregated distribution: \(x_t \sim \bar{p}_N(\cdot | x_{<t}).\)

2.3.0.2 Fluency-Aware Routing

If \(x_t \in \mathcal{V}_{\cap}\) (i.e., the token is valid across all models), we directly output \(x_t\). However, a critical challenge arises when the selected token \(x_t \in \mathcal{V}_{\Delta}\), it is undefined to the subset of models that lack this token in their vocabulary. Accepting such a token would break the autoregressive chain, as the incompatible models cannot process \(x_t\) as valid input context for the subsequent generation step (\(t+1\)). To preserve fluency, we employ a randomised routing mechanism that restricts subsequent sampling to “specialist” models whose vocabularies admit the committed tokens. Let \(K\) denote the set of all available models and we initialise \(K_t = K\). For each \(t' \in [t+1, \tau(t)]\), where \(\tau(t)\) denotes word completion, we recurrently update the specialist set as \[K_{t'} = \{i : x_{t'-1} \in \mathcal{V}_i\} \cap K_{t'-1}.\] We then uniformly sample a model from the specialist set before generating the token at \(t'\): \[k_{t'} \sim \text{Uniform}(K_{t'}), \quad x_{t'} \sim p_{k_{t'}}(\cdot | x_{<t'}).\] Crucially, we avoid selecting the model using maximum likelihood principles, as likelihood disparities may themselves be watermark-induced artifacts. Random selection ensures the routing decision is orthogonal to watermark signals, preventing detectors from exploiting systematic biases in model selection.

While routing temporarily re-admits the watermark signal \(\delta_{\text{wm}}^{(k_{t'})}\), such events are sparse and confined to vocabulary boundaries. Moreover, our strategy consistently engages as many specialist models as possible, ensuring that ensembling remains active throughout the generation. Critically, even if some watermark signal exhibits, watermark artefacts are temporally fragmented: they alternate stochastically across different specialists and are interspersed with non-routing tokens. This prevents the accumulation of sustained statistical regularities required for reliable watermark identification. We provide a complete generation example in Appendix 10.

2.3.0.3 Context Re-synchronisation

Routing to specialist models creates context de-synchronisation: non-selected models do not observe the generated tokens in their native tokenisation. To maintain coherence, we apply a decode-encode cycle. We decode the generated string with the last selected specialist model \(S_t = \text{Decode}_{k_{\tau(t)}}(\hat{x}_{t:\tau(t)})\), and update each model’s context via: \[x_{<\tau(t)+1}^{(i)} \leftarrow x_{<t}^{(i)} \oplus \mathcal{T}_i(S_t), \quad \forall i \in [N],\] where \(\mathcal{T}_i\) is model \(i\)’s tokeniser. This ensures all models observe semantically equivalent contexts despite tokenisation differences, preserving valid probability aggregation in subsequent ensemble steps.

3 Experiment↩︎

3.1 Experimental Setup↩︎

Models and Watermarks. We conduct experiments on three widely used pre-trained LLMs: Qwen3-8B [14], Llama-3.1-8B [15], and Ministral3-8B [16]. To ensure broad coverage, we evaluate six representative watermarking schemes spanning different design paradigms: AAR, which relies on uniform distribution [17]; DIPMark, which applies logit reweighting [18]; ITSEdit, based on inverse transform sampling with exponential minimum sampling [19]; KGW, which encourages sampling to a green list of tokens [6]; Exp-Edit, which uses key-based transformations [19]; and Water-Bag, which combines a set of watermark keys and their mathematical inversions during generation to mask statistical biases [20].

Baselines. We compare WASH with two classes of watermark removal attacks.

Generation-time attacks produce the output directly under the removal procedure, matching the WASH setting:
(1) De-mark [10]: A watermark removal method designed against red-green-list watermarking. It first identifies the green-list tokens whose logits are biased by the watermark using crafted prompts, then removes the bias to restore the original generation distribution.
(2) ToBlend [11]: A model generation mixture method against AI text detections. It blends multiple model generations by predicting a fixed number of tokens with a single selected model each time.

Final-text rewrite attacks instead operate on an already generated watermarked sequence:
(1) RandomWalk [20]: A strategy that repeatedly rewrites spans of the generated text using a weaker unwatermarked model and accepts only quality-preserving variants based on a quality-check oracle.

Benchmarks and Metrics. We use two evaluation suites: (1) Detection. For generation-time attacks, we follow the z-score protocol of [20] to quantify watermark signal strength. For final-text rewrite attacks, perturbation detection on the small set of generated tokens is no longer compatible, so we additionally use native sequence detectors [21]. Detection thresholds and confidence categories are reported in Appendix 9.1. (2) Quality and Efficiency. To assess generation quality and inference speed, we use four benchmarks covering representative reasoning and language tasks: GSM8K (math reasoning) [22], MMLU (knowledge) [23], SQuAD (reading comprehension) [24], WritingBench (open-ended writing) [25].

3.2 Watermark Removal Effectiveness↩︎

The experimental results in Figure 3 demonstrate the inherent fragility of distributional watermarking when subjected to linear ensemble averaging. Our analysis proceeds in three stages: validating the scaling law of signal decay, isolating the removal mechanism through control experiments, and comparing comprehensive performance against baselines.

Signal Decay: From Theory to Reality. We first validate the scaling law of signal decay in an idealised setting where models utilise independent watermarks on the same base model (Figure 3 (a)). The detection signal diminishes rapidly with \(N\), mirroring our theoretical prediction of \(O(1/\sqrt{N})\). The pivotal result lies in Figure 3 (b), which simulates the realistic user scenario: accessing distinct heterogeneous models (e.g., Llama, Qwen, Ministral) with independent watermarks. Despite the challenge of vocabulary mismatches, our fluency-aware routing successfully neutralises the watermark signal. Notably, the detection strength drops even faster than in the homogeneous setting (reaching near \(z=0\) at \(N=5\)), suggesting that the diversity of base model distributions acts as additional noise that further obscures the watermark trace. We provide the experiment details in Appendix 9.2.

A Control Experiment: Coordinated Watermark. To verify the mechanism of removal, we evaluate a “Coordinated Watermark” scenario where all ensemble models share the same watermark scheme, and signals are coordinated across distinct vocabularies during each token generation. Specifically, we calculate the relative perturbations of the common tokens from one watermarked model and map them to other models using the same relative scale. As illustrated in Figure 3 (c), when the ensemble models are synchronised, the washing effect fails. The ensemble average retains a statistically significant z-score well above the detection threshold (\(z > 4\)) across all schemes. Strong schemes like AAR and Exp-Edit maintain high z-scores of 25.11 and 17.44, respectively. Crucially, the persistence of these signals demonstrates that the ensemble process itself does not inherently obliterate watermark information. Instead, the removal efficacy in the independent setting (Figure 3 (a,b)) exploits the lack of correlation between providers. This finding highlights a critical defensive insight: robust AI-text detection against ensemble attacks requires coordinated watermarking strategies across model providers.

Comparison with Generation-time Attacks. Table 1 compares WASH against generation-time attacks across various LLMs using the same z-score detection protocol. Individual watermarked models exhibit strong detection signals, with z-scores ranging from 5.2 to 304.0. Prior removal methods show limited efficacy: De-mark performs well only on pure red-green-list watermarking methods with exact logits bias value like KGW and Water-Bag, while collapsing on other distribution modifications (4.6 to 99.3 z-score on DIPMark). It also cannot apply to watermarks that operate on probabilities. In contrast, WASH consistently neutralises detection signals. An ensemble of \(N=3\) proves practically sufficient to suppress z-scores below the threshold (\(z < 4\)) across all cases. Extending to \(N=5\) offers a safety margin for aggressive schemes, ensuring near-zero distinguishability. ToBlend, as a model mixture method like ours, could achieve similar removal efficacy to WASH in the detection evaluation, while significantly harming generation quality and efficiency, as we discuss in the next section.

Comparison with Final-text Rewrite Attacks. Final-text rewrite attacks require a separate evaluation as they perturb the completed sequence, making token perturbation detectors inapplicable. We thus report the TPR@5% FPR from native sequence-level detectors. Lower values indicate stronger removal under thresholds calibrated on unwatermarked samples. As shown in Table 2, RandomWalk leaves several schemes near or within the low-confidence range. WASH keeps all schemes below 43%, matching ToBlend’s removal ability and outperforming RandomWalk on 4 out of 5 schemes, while retaining quality and efficiency advantages analysed below.

3.3 Quality, Semantic Integrity, and Efficiency↩︎

Comparison with Generation-time Attacks. As shown in Table 3, watermarking inevitably distorts the optimal output distribution, leading to up to 10% performance degradation relative to the unwatermarked baseline. While removal attacks aim to recover this lost utility by restoring the original distribution, prior methods face two bottlenecks: (1) excessive perturbation may further harm generation quality, and (2) the computational cost of recovery can be prohibitive.

Consistent with our theoretical analysis, WASH resolves these tensions by recovering the unwatermarked distribution through ensembling. It achieves generation quality comparable to or superior to the strongest baseline (De-mark) in 7 out of 9 benchmark settings, while outperforming the other mixture method (ToBlend) across all settings.

The efficiency advantage is decisive for generation-intensive tasks. While prior overhead may be tolerable for short discriminative tasks such as MMLU, it becomes prohibitive for long-form generation, where watermarking is most relevant for content provenance, copyright protection, and misuse attribution. On GSM8K and SQuAD, De-mark suffers severe latency (\(>30\times\)) due to iterative distribution probing, and ToBlend incurs similarly high overhead (\(\sim12\times\)) because it repeatedly reprocesses context and predicts redundant future tokens. In contrast, WASH operates at only \(1.0\times\)–\(2.4\times\) the baseline cost through deterministic sampling and synchronised KV caching, giving roughly a 6\(\times\) speedup over competing methods while maintaining stronger quality.

Comparison with Final-text Rewrite Attacks. We evaluate RandomWalk on GSM8K and WritingBench rather than MMLU and SQuAD because final-text rewriting requires sufficiently long generations to meaningfully change surface form while preserving content. MMLU multiple-choice outputs and SQuAD short answers are often too short to expose either rewriting effectiveness or its utility cost.

As shown in Table 4, RandomWalk underperforms the watermarked baseline on both reasoning and open-ended writing, likely because in-place rewriting with another model introduces instability. Its additional rewriting phase also incurs substantial overhead, reaching \(4\times\) runtime on GSM8K and \(10\times\) on WritingBench. In contrast, WASH improves quality while requiring only about \(2\times\) runtime. Thus, final-text rewriting may weaken detection, but it pays for this through lower utility and significantly higher latency.

Resource and Serving Trade-offs. Table 5 further evaluates the system’s cost of WASH. The parallel implementation keeps all specialists resident and synchronises their KV caches, prioritising latency. The sequential implementation loads specialists on demand, reducing peak memory at the cost of higher token latency. This gives WASH a practical deployment trade-off: WASH-Par. is substantially faster than De-mark and ToBlend, while WASH-Seq. reduces peak memory close to the single-model baseline. Thus, WASH remains practical under both latency-constrained and memory-constrained serving regimes.

3.4 Ablation and Analysis of Fluency-Aware Routing↩︎

Routing Ablation. Fluency-aware routing is the main difference between WASH and naive distribution averaging. Table 6 evaluates this design choice. Naive Avg removes routing and directly averages distributions. The Rewrite variants then attempt to repair the Naive Avg output using an additional watermarked or unwatermarked copy of the same base model, testing whether post-hoc rewriting can replace generation-time routing.

The results show that post-hoc repair is not an effective substitute for routing. Naive Avg suppresses detection, but substantially degrades GSM8K accuracy. Rewrite improves surface-level writing quality, yet it further hurts reasoning accuracy. WASH instead performs sparse local routing during generation, preserving both low detection and substantially stronger utility.

Routing Robustness. We further test whether fluency-aware routing could reintroduce watermark signal when specialised vocabulary is frequent. This concern is most relevant in domains such as medicine and law, where rare terminology might trigger long single-specialist spans, weakening the averaging effect of WASH. Table 7 shows that this does not occur. The routed tokens account for less than 3.2% of generation; routed spans are short; and final detection scores remain far below the watermark threshold. Manual inspection suggests that complex technical terms are often decomposed into shared subword units, while routing is more frequently triggered by ordinary lexical units with tokeniser mismatches. Thus, fluency-aware routing acts as a sparse local repair mechanism rather than sustained single-model generation.

4 Related Works↩︎

Providing and Detecting Watermarks. Recent research has established LLM watermarking through green/red lists. [6]. While effective, this approach struggles with low-entropy text and is prone to Type II errors. To mitigate these limitations, subsequent research has focused on distribution-preserving and resilient schemes. For instance, [19] and [18] propose distortion-free methods to maintain generation quality. Similarly, [8] introduce unbiased watermarking via reweighting mechanisms and log-likelihood tests; however, this method assumes knowledge of the generation process and often lacks adversarial robustness. Further advancements explore alternative mechanisms on modifying generation bias through a semantic-aware watermarking [26] or maximal coupling strategies [27], while others aim to tackle detection challenges in low-entropy scenarios [9] or improve scalability [28].

Watermark Removal Attacks. Early attack approaches use paraphrasing to preserve semantics while resampling expressions [29], or sampling from multiple keys to perform majority voting [30]. RandomWalk further rewrites local spans and accepts quality-preserving candidates through an external quality check [20]. However, these methods typically rely on detector-available scenarios, assuming the adversary has access to an oracle verifier. Recent research attempts to eliminate this dependence. Many theoretical attacks necessitate query-intensive strategies to reverse-engineer watermarking rules. Methods such as Watermark Stealing [31], SCTS [32], and De-Mark [10] require a high volume of specific prompt queries or iterative token-level probing to infer red-green lists. Similarly, optimisation-based attacks employ computationally expensive random walks [33]. While theoretically effective, these strategies are impractical for real-time or long-form generation tasks due to prohibitive latency and computational costs. Additionally, ToBlend attempts to bypass watermarks by alternating between models [11], though often at the cost of coherence or inference efficiency due to frequent context re-encoding.

5 Conclusion↩︎

We demonstrate that current distributional watermarking schemes are structurally vulnerable to linear ensembling. Theoretically, we prove that averaging outputs from independent models cancels watermark perturbations. To address vocabulary mismatches in heterogeneous ensembles, we introduce WASH, which utilises fluency-aware routing to enable effective probability aggregation. Empirically, WASH renders watermarks statistically undetectable (\(z < 2\)) with as few as three models, while preserving generation quality with practical inference efficiency. Our findings suggest that reliable detection in a competitive marketplace is unattainable without industry-wide standardisation of watermark keys among model providers.

Impact Statement↩︎

This paper presents work whose goal is to advance the field of AI-generated content detection. This is increasingly important for maintaining societal trust in digital information and content authenticity, and for protecting intellectual property. We demonstrate that various contemporary LLM watermarking schemes are fragile under a general linear ensembling attack. While this attack exposes current limitations, it also serves as an alert to model providers and researchers to strengthen watermarking against adaptive and cross-model threats.

Moreover, we have conducted a coordination experiment, revealing that even simple signal coordination across heterogeneous models can partially mitigate the effectiveness of the ensemble attack. These findings suggest that true watermarking robustness depends not only on isolated model-level defences, but also on cooperative ways that may be more effective. In general, we underscore the value of cross-provider collaboration in developing robust watermarking provenance mechanisms for AI-generated content.

Acknowledgments↩︎

This work was supported in part by the UK Engineering and Physical Sciences Research Council (EPSRC) through a Turing AI Fellowship (grant no. EP/V020579/1, EP/V020579/2). We thank Xiaojia Rao for participating in the initial discussions that helped shape the idea of this work.

6 Proof of the Main Theorem↩︎

7 Extension of the Theoretical Result to Grouped Watermarking Settings↩︎

The theoretical analysis in Section 2 assumes full independence across all \(N\) providers. In practice, however, certain providers may share common watermarking toolkits, licensing agreements, or underlying model families, inducing statistical dependence among their perturbation vectors.

We show here that the convergence guarantee extends naturally to a grouped setting in which providers are partitioned into independent clusters, with only conditional independence required within each cluster. We also relax the unbiasedness assumption by allowing models within a group to share a common perturbation component.

Define the group consensus distribution for group \(g\) as \[\label{eq:group95concensus} p_g^\dagger(v|x) := \frac{p^*(v | x)\exp(b_g(v,x))}{\sum_{u \in \mathcal{V}} p^*(u | x)\exp(b_g(u,x))},\tag{10}\] and the group-size-weighted average \(\bar{p}^\dagger(\cdot | x) := \frac{1}{N}\sum_{g=1}^{M} n_g p_g^\dagger(\cdot | x)\). The irreducible group bias is defined as \[\label{eq:group95bias95term} B(x) := \left\|\bar p^\dagger(\cdot|x)-p^*(\cdot|x)\right\|_\infty = \sup_{v\in\mathcal{V}} \left| \bar p^\dagger(v|x)-p^*(v|x) \right|.\tag{11}\]

8 Robustness to Biased Watermark Perturbations↩︎

Assumption 1(c) assumes watermark perturbations as zero-mean around the consensus distribution. We stress-test this condition by deploying a biased green-red-list watermark that consistently promotes a fixed set of tokens with a bias. We evaluate the watermarked baseline and WASH with both the accuracy on the GSM8K task, and the watermark detection z-score.

Table 8 shows a utility-detection trade-off: stronger bias makes the baseline easier to detect, but rapidly hurts task performance. This further validates the practical plausibility of Assumption 1(c): rational providers have no incentive to deploy systematically biased perturbations, as the severe accuracy degradation (from 0.443 to 0.023) demonstrates that deviating from the zero-mean condition directly harms generation quality, undermining their own service. WASH remains effective across all bias values, keeping detector z-scores below the no-detection threshold (\(z \le 4\)) and improving accuracy over the corresponding biased baseline.

This is consistent with the theoretical prediction in Appendix 7 when perturbations carry a shared bias, an irreducible term \(B(x)\) persists in the convergence bound, explaining why the detection z-scores under WASH do not vanish completely but remain below the detection threshold.

9 Experiment Details↩︎

9.1 Implementation Details↩︎

9.1.0.1 Detection thresholds.

For generation-time attacks, we follow the z-score protocol of [20] to quantify watermark signal strength. Following the original definition, Detection confidence is categorised as high-confidence identification (\(z > 10\)), low-confidence identification (\(4 < z \le 10\)), and no detection (\(z \le 4\)).

For final-text rewrite attacks, perturbation detection on the small set of generated tokens is no longer compatible, so we additionally use native sequence detectors [21]. Following [20], we generate the sequence to be detected on the C4 dataset [35]: we truncate each sample to 30 tokens as the prompt, generate 200 additional tokens with the watermarked model, and then perform detection on the generated sequence. Since z-score magnitudes vary substantially across watermarking schemes, directly comparing z-scores across schemes is not meaningful. We therefore report the removal effect using TPR@5% FPR, the true-positive rate of identifying a watermarked sequence at a threshold calibrated to falsely flag 5% of unwatermarked sequences, which is widely used as a watermark robustness measurement due to its consistency across watermark types, text types, and lengths [36]. Lower TPR@5% FPR indicates stronger removal. We categorise a strong watermark signal for TPR@5% FPR \(\ge 75\%\), while a low-confidence detection with \(50 \leq\) TPR@5% FPR \(\leq 75\%\).

9.2 Detailed Experiment Results↩︎

Table 9 shows the detailed experiment results for Figure 3 (a) and (b), the watermark detection evaluation with mixing diverse watermark schemes and base models. For the fixed-base model setting, we randomly sampled 15 ensemble combinations for each mixture amount \(N\) and ran the detection task 5 times each to obtain stable z-scores. For the mixed-base model setting, we experimented with a larger range of \(N\) from 1 to 8, and sampled 50 mixture combinations for each \(N\) due to the large sampling space. The results follow the scaling law: with a larger ensemble size, the watermark signal decays more.

For the fixed-base model setting (Figure 3 (a)), the detection signal diminishes rapidly as \(N\) increases, with the Llama3.1-8B model dropping from an extremely high z-score (\(\approx 150\)) to \(\approx 10\) at \(N=5\). This empirical decay mirrors our theoretical prediction of \(O(1/\sqrt{N})\). However, we observe that the signal does not vanish as completely as in the heterogeneous setting. This residual signal likely arises because the shared systematic bias (\(\delta_{sys}\)) of the identical base model persists across the ensemble, hindering the complete cancellation of artefacts. More importantly, this single-model scenario is rarely available in practice, as providers seldom expose multiple watermarked versions of the same model to end users.

10 Fluency-Aware Routing Example↩︎

Figure 4 shows two complete generation flows for a text completion task in the C4 dataset and a reasoning task on the MMLU Law subset, including the Fluency-Aware Routing triggered in between. These generations are conducted by a mixture of three watermarked models: Llama-Aar, Ministral-DIPMark, and Qwen-KGW.

The routing mechanism is triggered mostly on words that are tokenised differently across models. For example, in the MMLU Law task, the word “negligent” is tokenised to [‘negl’, ‘igent’] by Ministral, but is maintained as a complete word token by Llama and Qwen. During Fluency-Aware Routing, the span is completed using only the specialist models that share the same tokenisation scheme for the target span. After routing, the span is re-synchronised by each model’s own tokeniser, ensuring that the span’s understanding won’t be damaged by misaligned tokenisations.

None

References↩︎