1 Introduction↩︎

A foundational question across machine learning is whether observed behaviour reflects generalization from training data or memorization of specific training points [1]. For deep generative models, this question has been studied through canaries, extraction, and dataset deduplication [2]–[5]. Operationalising it requires probes that map a model’s distribution over outputs to a numeric memorization signal. In the autoregressive setting — which covers language models and the language tower of vision-language and multimodal generative models — three probe families dominate: (a) per-token negative log-likelihood (NLL) averaged over a fixed token window (often the first \(K\) tokens of a target), (b) full-span secret NLL on the substring whose memorization is claimed, and (c) behavioral exact-recall, i.e.whether greedy or sampled decoding emits the secret string. Closely related quantities (loss thresholding, perplexity ratios against a held-out reference, calibrated likelihood ratios) appear in adjacent memorization-evaluation literature.

Why probe choice matters. A central concern motivating this paper’s audit is that probe choice can change the false-positive and false-negative rates of a memorization claim — a probe-versus-probe consistency question that is separable from the underlying memorization rate. Claims about whether a generative model has memorized rest on which probe was used to call the verdict, and audits asking “does the standard probe agree with itself across natural alternatives?” are a template-specific probe-audit question — one that can confound broader memorization-vs-generalization conclusions if a single window is used uncritically. In related settings, [6] show that benign or task-style fine-tuning can activate verbatim recall in a generation-based extraction setting, while [7] show privacy-ripple effects from adding or removing personal information during language-model training. We do not refute or replicate those reactivation findings, but use a related memorization-reactivation probe setting on a different testbed and document where it is and is not self-consistent.

Our testbed. We use a single autoregressive generative-model testbed: a Qwen2.5-VL-7B [8] backbone with frozen vision tower and language-tower LoRA [9] into which we inject \(20\) image canaries and \(20\) text canaries (text canaries pair with a fixed \(224\!\times\!224\) mid-gray placeholder image, so the strongest evidence in this paper is essentially text-only). On top of the canary-injected base we stack a second LoRA performing benign supervised fine-tuning (bSFT) on one of four data sources, none of which contain the canary strings (§2). This setup gives us \(34\) aggregate bSFT cells \(=\,102\) seed-level runs (\(144\) with \(42\) matched-norm LoRA-Noise controls), and an explicit undertrained-canary headroom regime that lets us bound how far reactivation could move the probe in principle (App. 6). Across \(102\) bSFT seed-level runs we observe no full-secret NLL recovery and no hit\(@1\) above baseline (Tab. 2, App. 10); many cells are saturated at hit\(@1\!=\!1.00\) where reactivation cannot be observed without prior secret damage, so this null is the condition under which the probe-disagreement audit is run, not a substantive null on benign-SFT reactivation in autoregressive generative models. Our setting is adjacent to but distinct from large-scale extraction [3], dedup-driven memorization [5], and training-data unlearning [10], [11].

Terminology. We use the following labels throughout:

• full-secret recovery: sign-consistent negative \(\Delta\mathrm{NLL}_{\mathrm{hex}}\) on the full \(13\)-token secret span across seeds, or hit\(@1\) above baseline.

• undertrained in-window hex-token NLL improvement: sign-consistent negative \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) whose per-span decomposition localises onto \(\sim\!9\) in-window hex tokens (positions \(11\)–\(19\)) plus the leading canary_lit, with full-span hex \(\geq\!0\) and hit\(@1\) unchanged.

• behavioral damage: secret-span \(\Delta\mathrm{NLL}\) increase plus hit\(@1\) drop.

• non-secret preamble drift: mean\(_{20}\) moves but \(\geq\!90\%\) of the per-span contribution sits on preamble, \(\Delta\mathrm{NLL}_{\mathrm{hex}}\!<\!0.5\,|\Delta\mathrm{NLL}_{\mathrm{mean20}}|\), and hit\(@1\) unchanged.

• window-truncation failure: mean\(_{20}\) flat while damage is visible to wider-window probes or hit\(@1\).

Three probe-disagreement case studies. During the same experiments we exhibit three post-hoc cases (Fig. 1) where the internally pre-specified probe \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) gives an answer that differs from full-span secret NLL or from behavioral exact-recall, and each case maps to a distinct failure mode of truncated mean-NLL as an evaluation metric for memorization in an autoregressive generative model:

• Case 1 (C3, T-bSFT-GUI 5k — false negative under window truncation): mean\(_{20}\) flat (\(+0.0001\)) while \(\Delta_{\mathrm{hex}}\!=\!+0.0133\) and hit\(@1\) drops to \(0.88\) — damage on a hex-token outside \(K\!=\!20\).

• Case 2 (C4, T-bSFT-Safety 5k — false positive from non-secret span drift): mean\(_{20}\!=\!+0.0150\) sits \(\sim\!99\%\) on non-secret preamble; secret span and hit\(@1\) unchanged.

• Case 3 (C5, U-bSFT-GUI 3k — ambiguous undertrained in-window NLL drop): mean\(_{20}\!=\!-0.0070\) on an undertrained baseline localises to in-window hex tokens, but the full-span hex point estimate is positive (and positive across all \(3\) seeds; hb brackets zero) with hit\(@1\!=\!0\). No secret-specificity claim is made; format-prior smoothing or template adaptation are equally consistent without true-vs-decoy probes (a paired decoy audit at the 7B C5 cell remains future work; the smaller-family C5-equivalent decoy results in Tab. 12 are descriptive stress tests at different cells).

The \(K\!=\!20\) probe and the “no full-secret recovery” operating condition were chosen before seed-level results were aggregated, but were not preregistered or externally certified; the three case studies and the selection protocol of §3 are post-hoc.

Contributions.

• A template-specific audit of our internally chosen \(K\!=\!20\) truncated mean-NLL probe against full-span secret NLL and behavioral exact-recall, framed as an evaluation-metric question for canary-based memorization in autoregressive generative models (§2).

• Three post-hoc probe-disagreement case studies illustrating two mechanical probe mismatches and one unresolved undertrained-control ambiguity (§3), with full per-cell selection audit (Tab. 4, App. 7) and an algebraic span decomposition that closes the dissociation to within bf16 residual (§3.3, App. 9).

• Recommendations for evaluation protocols that can flag the two failure modes and the ambiguity case before they propagate into generalization-versus-memorization conclusions (§5, App. 6).

Scope. Three case studies in one backbone are not a population claim. We discuss in §4 and §5 which parts of the failure-mode taxonomy are likely to transfer to other autoregressive generative models and which depend on testbed-specific choices (template, tokenizer, LoRA configuration, frozen vision tower). \(n\!=\!3\) outer seeds; intervals descriptive only; cross-architecture and cross-scale replication is the natural next experiment.

2 Method: canaries, bSFT matrix, three probes↩︎

We instantiate a controlled memorization-evaluation testbed (Fig. 2) whose three ingredients — canary design, benign supervised fine-tuning (bSFT) matrix, and probe specification — jointly determine the generalization-versus-memorization verdict any given probe will return on this autoregressive generative model. The choices below preserve the canary-and-probe setup of [2] while making the spans and windows on which probes are computed explicit, so that probe disagreement can be localised to specific token positions rather than absorbed into a global average.

2.0.0.1 Canaries.

Following the small-canary-set tradition of [2], \(N\!=\!20\) canaries per modality (the exact set size is our own design parameter):

• Image canaries: first \(20\) distinct ScreenSpot v2 screenshots [12]; target CANARY-{16hex}-END where the hex is derived from sha256("whack-canary-{i:03d}").

• Text canaries: per-canary user prompts “Provide the diagnostic label for sample id {8hex}” paired with a shared \(224\!\times\!224\) mid-gray placeholder image; target TEXTCAN-{16hex}-END.

Manifests are SHA-256-pinned on first run, verified each subsequent run. Image-canary screenshots are excluded from bSFT data by filename (perceptual-hash duplicate check in App. 15).

2.0.0.2 Three injection regimes.

On Qwen2.5-VL-7B-Instruct [8] with frozen vision tower we inject canaries via LoRA [9] on the language tower (target modules \(\{q,k,v,o,\text{gate},\text{up},\text{down}\}\)_proj), then merge_and_unload the canary LoRA into the base before bSFT (“stacked LoRA bSFT” \(=\) canary-merged base + bSFT-LoRA loaded at probe time; App. 12). All canary LoRAs use \(r\!=\!32\), \(\alpha\!=\!64\):

• \(M_{\mathrm{canary}}\) (image, saturated): \(80\) ep, recall \(20/20\).

• \(M_{\mathrm{text\,canary}}\) (text, saturated): \(40\) ep, recall \(20/20\).

• \(M_{\mathrm{canary\,under}}\) (image, undertrained): \(10\) ep, baseline mean NLL \(1.28\), hex \(2.62\), hit \(0/20\) (headroom for reactivation).

2.0.0.3 Non-canary-string SFT (bSFT) variants.

“Benign” here denotes the operational property that the SFT data contains no copy of any canary string; Safety and Random share the gray-placeholder image with text canaries by construction, only the canary strings are excluded. On each canary-injected base we stack a second LoRA (\(r\!=\!16\), \(\alpha\!=\!32\), lr \(2\!\times\!10^{-5}\), batch \(2\), grad-accum \(4\), cosine, \(5000\) examples) on one of four data sources (none contain canary strings):

• GUI: ScreenSpot grounding (image\(+\)text).

• SYNTH: PIL-generated abstract images with templated captions (image\(+\)text out-of-domain control).

• Safety: (harmful-request, refusal) text pairs with placeholder image.

• Random: placeholder-image factoid-question / short-answer text pairs.

Matched-norm LoRA-Noise controls cover image, image-SYNTH, T-GUI, T-Safety, U-image (\(14\) aggregate cells). Step grid (asymmetric, see Tabs. 7 and 8): GUI-saturated-image at \(\{1,3,5,7,10\}\)k; other saturated at \(\{1,3,5\}\)k; undertrained-image at \(\{1,3\}\)k; \(3\) seeds per cell.

All training and probes use bf16; App. 9 reports the bf16 residual between \(K\!=\!20\) probe-pass and \(K\!=\!30\) token-level pass for completeness.

2.0.0.4 Reproducibility fields.

Optimizer AdamW, peak lr \(2\!\times\!10^{-5}\), cosine schedule, batch \(2\) with grad-accum \(4\) (effective \(8\)), max sequence length \(1024\), canary LoRA epochs as listed above, bSFT seed-level training to \(5000\) examples (or the cell-specific step grid). RNG seeds \(\{0,1,2\}\) apply jointly to bSFT data sampling, canary order, optimizer and dropout. Decoding for hit\(@k\) uses \(T\!=\!0.7\), \(p\!=\!0.95\), max new tokens \(=\) canary length \(+\,4\), and the standard Qwen2.5/Llama-3 chat template with the identical user prompt above; bSFT data is sampled without replacement from each source. Cross-family runs use the same text-only canary subset and the same hyperparameters, modulo removing image fields from the canary template and the bSFT batches.

2.0.0.5 Tokenization and probe windows.

The text-canary assistant target tokenizes deterministically (Table 1); the template extends past token \(20\). Two consequences: (a) our internally pre-specified \(K\!=\!20\) mean-NLL window covers all \(10\) preamble tokens, the literal, but only \(9\) of \(13\) hex tokens, omitting the last \(4\) hex tokens and the suffix; (b) the secret-span (canary_hex) probe widens the forward pass to \(K\!=\!30\) to include the full \(13\) hex tokens.

4pt

2.0.0.6 Notation summary.

\(\mathrm{mean}_{K}\) always denotes a prefix-window average over positions \(0,\ldots,K\!-\!1\); \(\mathrm{mean}_{20}\) is the internally pre-specified prefix probe. \(\mathrm{hex}\) denotes the fixed \(13\)-token secret span (positions \(11\)–\(23\)), evaluated with a \(K\!=\!30\) forward pass only to expose the tail tokens; it is not a \(30\)-token average. A \(K\!=\!30\) pass is used only to obtain token losses beyond position \(19\); unless explicitly written as \(\mathrm{mean}_{30}\), all \(\mathrm{hex}\) values are averages over the \(13\) secret tokens only. All \(\Delta\) quantities are bSFT-vs.-canary-baseline deltas in NLL loss units (additive, not ratios), aggregated as the per-canary mean averaged across \(n\!=\!3\) outer seeds.

2.0.0.7 Three probes.

Token positions zero-indexed throughout (Tab. 1); \(K\!=\!20\) \(=\) positions \(0\)–\(19\). Let \(\ell_\theta(t_j)\!=\!-\log p_\theta(t_j\!\mid\!t_{<j},x)\) and \(\ell^{\mathrm{base}}_\theta\) the per-token cross-entropy under bSFT and canary-injected baseline. \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\!=\!\frac{1}{20}\sum_{j=0}^{19}[\ell_\theta(t_j)-\ell^{\mathrm{base}}_\theta(t_j)]\); the truncated-window mean-NLL probe and the \(K\!=\!20\) cut are our own design choice. \(\Delta\mathrm{NLL}_{\mathrm{hex}}\) averages the \(13\) canary_hex tokens (positions \(11\)–\(23\); \(11\)–\(19\) inside \(K\!=\!20\)) with a \(K\!=\!30\) forward pass; \(\Delta\) on other spans defined analogously. Throughout the paper, \(\mathrm{mean}_{K}\) denotes a prefix-window average over positions \(0\!-\!(K\!-\!1)\), while \(\mathrm{hex}\) denotes the fixed \(13\)-token secret-span average, even when evaluated with a \(K\!=\!30\) forward pass for tail-token coverage. hit\(@k\) is the fraction of canaries whose literal target appears in \(k\!\in\!\{1,4,16\}\) decoded outputs: greedy at \(k\!=\!1\) and greedy plus \(k\!-\!1\) stochastic samples at \(T\!=\!0.7,p\!=\!0.95\) for \(k\!\in\!\{4,16\}\) (i.e.\(3\) and \(15\) samples beyond greedy).

2.0.0.8 Aggregation and uncertainty.

Each cell pools \(20\) canaries \(\times\,3\) seeds. Main-text statistics are reported as the per-canary mean delta with a \(95\%\) hierarchical bootstrap CI (outer-resample \(3\) seeds with replacement, inner-resample \(20\) canaries within each chosen seed, \(B\!=\!10{,}000\), fixed RNG seed). The hierarchical bootstrap accounts for the fact that the same \(20\) canaries are reused across seeds, unlike a naive IID bootstrap over \(60\) measurements. With only \(3\) seeds these CIs are still wide; we report seed ranges in addition wherever a claim is tight, and we do not treat narrow CIs as population-level guarantees. Exact bit-for-bit reproduction is not possible from this submission alone because canary manifests and bSFT split files are not released; the printed tables are intended for numeric audit, not rerunning, and the appendix recipe in App. 6.0.0.2 documents the construction without releasing the artifacts.

3 Three probe-disagreement case studies↩︎

We surface three post-hoc case studies (§3.1–§3.3, C3/C4/C5). The matrix is \(4\) bSFT variants \(\times\,3\) canary regimes (saturated image, text, undertrained image) \(\times\,3\) seeds \(\times\,3\)–\(5\) steps, yielding \(34\) aggregate bSFT cells \(=\) \(102\) seed-level runs (\(144\) with \(42\) LoRA-Noise controls). Table 2 shows five canonical cells; full all-cells point estimates in App. 10; headline-cell hb and \(t_2\) intervals are listed in App. 8.

3.0.0.1 Selection audit.

Case studies are post-hoc (full audit: Tab. 4/App. 7). Headline counts:

• C3 sole hit\(@1\!<\!1\) saturated cell (\(26\)-cell pool).

• C5 largest post-hoc negative mean\(_{20}\) among \(8\) undertrained-image bSFT cells (\(3/3\) seeds same sign).

• C4 largest of \(3\) qualifying preamble drift cells.

• Across \(48\) CI rows, \(0\) cells have hb-CI screened negative on \(\Delta_{\mathrm{hex}}\).

Thresholds were chosen before per-cell CIs were aggregated, but were not preregistered or externally certified (App. 6.0.0.2).

3.0.0.2 Statistical note (\(n\!=\!3\)).

Both hierarchical-bootstrap and Student-\(t_2\) intervals are descriptive only; per-seed sign-consistency describes the selected cells and is not inferential evidence after post-hoc selection (App. 8, App. 6).

3.0.0.3 “Benign” caveat.

“Benign” here means operationally “no canary string in bSFT data,” not semantically benign. T-Safety/C4 share a fixed gray placeholder image with text canaries (read as “shared-placeholder”).

3.1 Case 1 — window truncation misses sparse greedy tail-token brittleness (C3).↩︎

Numbers. \(\Delta_{\mathrm{mean20}}\!=\!+0.0001\) flat; \(\Delta_{\mathrm{hex}}\!=\!+0.0133\) (\(3/3\) seeds positive); hit\(@1\) drops \(1.00\!\to\!0.88\).

Mechanism. Per-token plot (Fig. 4): tokens \(0\)–\(22\) track baseline within \(10^{-4}\) in absolute NLL; token \(23\) (the final canary_hex BPE piece, \(1\)–\(2\) hex characters before -END) absolute NLL jumps to \(\sim\!0.119\) (per-token mean across \(60\) instances), three orders of magnitude above the flat \(\sim\!10^{-4}\) baseline at neighbouring positions. Position \(23\) falls outside \(K\!=\!20\) (zero-indexed), so mean\(_{20}\) is blind to it.

Sparsity. \(7/60\) greedy failures on \(3/20\) canaries (canary \(0\) dominates: \(+0.122\) alone, leave-one-out hex \(+0.0075\)); \(17/20\) canaries perfect across all seeds; hit\(@16\!=\!58/60\) (App. 14). One sparse tail-token counterexample, not broad damage and not distribution-level loss of the secret.

3.2 Case 2 — non-secret preamble drift (C4).↩︎

Numbers. \(\Delta_{\mathrm{mean20}}\!=\!+0.0150\) (\(3/3\) seeds positive); \(\Delta_{\mathrm{hex}}\!=\!+0.0004\); hit\(@1\!=\!1.00\).

Primary diagnostic (span decomposition). Per-span decomposition (App. 9) attributes \(\sim\!99\%\) of the mean\(_{20}\) movement to non-secret preamble (\(+0.01486/+0.01503\) token-level); only \(+0.00017\) falls on canary_hex. A probe whose mass sits on non-secret preamble is not secret-span damage.

Appendix-only sensitivity check (merge-and-unload). Merging bSFT-LoRA into the base collapses \(\Delta_{\mathrm{preamble}}\) from \(+0.0297\) to \(+0.0014\) (\(-95\%\)); C1/C3/C5 secret-span ratios stay \(0.98/0.98/0.77\) and C4’s own canary_hex survives at \(0.51\) (Tab. 10). The collapse is C4-preamble-specific \(\Rightarrow\) evaluation-stack-dependence on preamble (no causal claim). This ablation is diagnostic of evaluation-stack sensitivity only; the primary evidence for Case 2 is the per-span decomposition above.

3.3 Case 3 — unresolved undertrained-control ambiguity (C5).↩︎

Numbers. \(\Delta_{\mathrm{mean20}}\!=\!-0.0070\) (\(3/3\) seeds negative); full-span \(\Delta_{\mathrm{hex}}\!=\!+0.0076\); hit\(@1\!=\!0\).

Decomposition (App. 9): in-window hex \(-0.00583\) + canary_lit \(-0.00091\) + bf16 residual \(-0.00030\).

Reading. An in-window hex-token NLL drop on an undertrained baseline; no secret-specificity claim. Format-prior smoothing or template adaptation are equally consistent without true-vs-decoy probes.

3.3.0.1 Per-seed sign consistency for headline cells.

3.3.0.2 C5 algebra closure.

\[\begin{align} &\underbrace{-0.00583}_{\text{in-K hex}} + \underbrace{-0.00091}_{\text{canary\_lit}}\\ &\quad + \underbrace{+0.00001}_{\text{preamble}} + \underbrace{-0.00030}_{\text{bf16}} = -0.00703 . \end{align}\]

3.4 Window K-sweep: 7B failure modes vs.smaller-family stress test↩︎

For the smaller-family stress test (§3.5) we re-aggregate the per-token NLL files at \(K\!\in\!\{10,15,20,25,30\}\); for the 7B headline cells we report only the derivable extremes \(K\!=\!10\), \(K\!=\!20\), and the full canary_hex span (the intermediate 7B \(K\!\in\!\{15,25,30\}\) values would require re-running the per-token aggregator with a different window length and are not printed in this submission).

On the 7B headline cells, the algebra closure (Tab. 5) plus the full-span readout (Tab. 2) pin down the printed extremes (see Tab. 6). They imply: C3 is \(\sim\!+10^{-4}\) for \(K\!\leq\!20\) and only diverges from the in-window value once the tail-token spike at position \(23\) enters the window; C4 stays positive across \(K\!=\!10\!\to\!20\), with the preamble-only \(K\!=\!10\) mean (\(+0.0297\)) accounting for \(\sim\!99\%\) of the canonical \(K\!=\!20\) value after rescaling by \(10/20\) (i.e.\(0.5\!\times\!0.0297\!=\!0.01486\) vs.\(+0.01504\)); C5’s mean is negative at the canonical \(K\!=\!20\) but turns positive at the full-span readout, the joint-product signature of window truncation and the undertrained-baseline regime.

On the smaller-family C3/C4/C5-equivalent cells (Fig. 3): the K-monotone shape on Qwen2.5-1.5B and Llama-3.2-1B differs from the 7B testbed: in particular, the negative-mean signature behind 7B Case 3 does not reproduce on either smaller family at the C5-equivalent cell, and Llama-1B’s \(\Delta\mathrm{mean}_{K}\) at C3/C4-eq is dominated by a catastrophic preamble drift at every \(K\) rather than by the tail-token / preamble-fraction mechanism we observe on 7B.

3.5 Descriptive smaller-family non-replication check↩︎

We run a smaller-family stress test at the C3/C4/C5-equivalent cells on Qwen2.5-1.5B-Instruct [13] and Llama-3.2-1B-Instruct [14] (text-only LMs, same canary template, same bSFT data minus image fields, same probe family, \(n\!=\!3\) outer seeds at each cell). We do not print or claim a full \(4\!\times\!3\) smaller-family bSFT grid; App. 10 is the 7B all-cells matrix only. Headline-cell point estimates appear in Tab. 3; we read these as descriptive stress tests on smaller text-only backbones, not as case-level replication of the 7B mechanisms (Tab. 3 caption).

3pt

3.5.0.1 Cross-family verdict.

The smaller-family stress test surfaces additional probe disagreements but does not replicate all three 7B case mechanisms at the C3/C4/C5-equivalent cells. Case 3’s undertrained-baseline mechanism has no direct analog in our text-only cross-family setup (text LMs in this protocol have no separate undertrained-canary regime; the image-canary gray-placeholder construction that triggers C5 on the VLM testbed is unavailable on text-only LMs), and the C5 cell at the 7B-fixed location does not reproduce as a misleading negative on either family (Tab. 3: Qwen-1.5B C5-eq \(\Delta_{\mathrm{mean20}}\!=\!{+}0\); Llama-1B C5-eq \(+0.78\) matches the same catastrophic-mean / preserved-hex / broken-greedy pattern as Llama C3-eq and C4-eq, not Case 3). Two additional smaller-family phenomena — a migrated Qwen-1.5B C4-equivalent cell with \(98\%\) preamble-driven \(\Delta_{\mathrm{mean20}}\!=\!{+}0.22\), and a Llama-3.2-1B T-bSFT-GUI step-trajectory where \(\mathrm{mean}_{K=20}\) grows monotonically while greedy hit\(@16\) recovers across step — and the smaller-family decoy audit at the candidate C5-equivalent cells (showing additive NLL gaps of \(+1.67\) to \(+10.86\) loss units between shuffled-hex / wrong-secret / format-only decoys and the true target) are reported in App. 16; this is a calibration of the decoy construction at different cells and does not adjudicate the original 7B C5 ambiguity, which lacks a paired decoy audit. Magnitudes are 3-seed point estimates, not inferential, and we read these rows as descriptive smaller-family stress tests rather than as case-level replication.

4 Related Work↩︎

4.0.0.1 Memorization probes for autoregressive models.

The canary-and-probe paradigm we use originates with Secret Sharer [2], which introduced exposure as a calibrated rank-based metric for unintended memorization, and was generalized in [4] to discoverable-extraction rates across model scales, data duplication, and prompt length. Truncated mean-NLL over the first \(K\) target tokens — the probe we audit — is a natural prefix-window operationalization in this lineage; the present paper’s three case studies show that, on a single-backbone testbed, the choice of probe (truncated mean NLL, full-span loss, behavioural exact-recall) can materially change false-positive and false-negative rates. We do not propose a new probe; we document, in one autoregressive generative-model testbed, three concrete cases where truncated mean-NLL silently disagrees with full-span NLL or with behavioral exact-recall, and attribute each disagreement to a specific token-position or baseline-regime mechanism.

4.0.0.2 Extraction and large-scale memorization.

[3] demonstrated that training data can be recovered from large language models via query-based extraction with ranking-based filtering, shifting the field toward extraction-based memorization metrics that do not depend on a fixed-window NLL probe at all; subsequent work [15] scales these attacks to production language models. Memory-trace studies that frame memorization as a finetuning-time and forgetting-time quantity [16] sit between extraction-based and NLL-based metrics and motivate the need to know what fixed-window NLL does and does not measure on a controlled canary set. Deduplication-driven studies [4], [5] show that duplicate training text affects memorized emissions and that memorization increases with duplication count, linking training distribution to probe response. Our setting is adjacent: the canary distribution is deliberately controlled, and we audit the probe rather than the underlying memorization rate, but the same evaluation-metric concerns apply.

4.0.0.3 Broader evaluation-reliability audits.

Our probe-disagreement view is part of a broader reliability concern: diagnostics, explanations, domain-validity audits, and benchmark verdicts can shift when the measured object changes [17]–[20]. Adjacent generative-model and RAG evaluations make a similar point through multidimensional bias benchmarks, context-conflict probes, and evidence-warrant audits [21]–[24]. We cite them as measurement-reliability neighbors rather than memorization-probe baselines.

4.0.0.4 Benign or safety-driven side effects on memorization.

The closest motivating works are [6], on Whack-a-Mole-style recall activation after benign or task-style fine-tuning, and [7], on privacy-ripple effects from adding or removing personal information. These claims are exactly the kind of memorization-versus-generalization question that probe choice can flip — the same probe-window decision that buries Case 1 (a tail-token greedy miss outside \(K\!=\!20\)) can also bury any reactivation that occurs predominantly outside the chosen window. We do not claim our three case studies refute or replicate those reactivation findings; we use a related probe setting on a different testbed and document where it is and is not self-consistent.

4.0.0.5 Unlearning and lifecycle.

A separate line studies training-data unlearning, from general SISA-style removal [10] to targeted forgetting of memorized generative-model content [11]. Recent LLM unlearning audits and unlearning-adjacent recommender work sharpen the same measurement question: forgetting should be tied to the measured object [25], while recommender unlearning mechanisms provide an adjacent measured-object-specific design [26]. The probe-disagreement phenomena we document are a precondition for unlearning evaluation: if truncated mean-NLL drops on an undertrained baseline without secret-specific evidence (Case 3), audits using the same probe family may register apparent “forgetting” without a behavioural change. We adapt the decoy/canary control of Secret Sharer [2] to expose this ambiguity.

4.0.0.6 Memorization in non-autoregressive generative models.

Memorization in diffusion models is an active area of the foundations literature [27]–[29], and the metric-dependence concern we document is not specific to autoregressive backbones: visual generative models can be probed via near-duplicate retrieval, likelihood-based membership inference, or direct extraction, each of which has its own false-positive and false-negative profile relative to true training-image replication. We do not run experiments on diffusion models in this paper; we limit our claims to an autoregressive setting and discuss in §5 which parts of the failure-mode taxonomy plausibly carry over and which require dedicated experiments.

4.0.0.7 What this paper adds.

Relative to the works above, our contribution is narrow and audit-shaped: in one testbed, we treat truncated mean-NLL as the unit of analysis, decompose it onto explicit token spans (preamble, canary_lit, canary_hex, canary_end, trailing; Tab. 1), and report three cases where span or behavioural-recall verdicts diverge from the truncated probe. Three cases in one backbone are not a population claim; they make the failure modes nameable so future probe-window choices can be reported with the relevant audits attached.

5 Discussion: evaluation protocols for memorization in generative models↩︎

5.0.0.1 Take-home.

In this testbed, truncated \(\mathrm{mean}_{K=20}\) NLL yields a false negative (Case 1), a false positive (Case 2), and an ambiguity (Case 3: an undertrained in-window NLL drop without full-secret recovery, not a secret-specific recovery claim). Memorization audits on canary sets should therefore report, beyond \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) alone:

• Full-span secret NLL. A window that omits part of the secret span is blind to omitted-position damage. Case 1: the last hex BPE piece is at position \(23\), outside \(K\!=\!20\), so a \(0\!-\!19\) window drops it.

• Behavioural hit\(@k\) with \(k\!\geq\!4\). Greedy hit\(@1\) catches Case 1’s tail-token brittleness; sampled hit\(@k\) checks whether the secret persists. In our matrix, hit\(@16\!=\!58/60\) on C3, so broken canaries remain in the secret distribution.

• Per-span decomposition (e.g. preamble vs.secret). Isolates whether a probe move is on the secret or on template-shared positions. Case 2’s \(+0.0150\) probe move sits \(\sim\!99\%\) on the non-secret preamble span while the secret-span \(\Delta\) is \(+0.0004\) and hit\(@1\!=\!1.00\).

• A saturated and an undertrained control to bound how far reactivation could move the probe in principle. Across the \(4\!\times\!2\) undertrained-image bSFT grid plus matched-norm LoRA-Noise, no variant produces verbatim hit\(@1\!>\!0\) (App. 6), so probe drops in this regime cannot be attributed to secret recovery without further evidence.

• Decoy probes (shuffled / wrong-secret / format-only) before asserting secret-specificity. Case 3: an in-window hex-token NLL drop is consistent with both secret-token learning and format-prior smoothing. A 7B C5 decoy remains future work (cf.App. 16).

6 Scope of claims and reproducibility↩︎

6.0.0.1 Scope of claims.

The three case studies hold in our testbed only: Qwen2.5-VL-7B-Instruct, language-tower LoRA with frozen vision tower, four bSFT variants (GUI, SYNTH, Safety, Random), \(20\) canaries per modality (image, text, undertrained-image), exact substring probes, \(K\!=\!20\) for mean-NLL. Across the \(4\!\times\!2\) undertrained-image bSFT grid plus matched-norm LoRA-Noise (\(2\) steps), no variant produces verbatim hit\(@1\!>\!0\) on the undertrained canaries. The strongest evidence (Case 1, C3) is essentially text-only; we do not extrapolate to other backbones, PEFT configurations, canary templates, or non-autoregressive families (diffusion, flow). \(n\!=\!3\) seeds; hierarchical-bootstrap and Student-\(t_2\) intervals are descriptive only, and after post-hoc selection no inferential claim survives selection adjustment (Tab. 4).

6.0.0.2 Reproducibility.

We do not release supplementary material. All canonical numbers are reproduced in the printed tables: per-cell deltas in Table 7, per-seed sign-consistency and headline intervals in App. 8, span decomposition in Table 5, and merge/stacked comparisons in Tables 9–10. Selection-audit thresholds in Table 4 (\(|\Delta_{\mathrm{mean20}}|\!>\!10^{-3}\), \(|\Delta_{\mathrm{hex}}|\!<\!0.5|\Delta_{\mathrm{mean20}}|\), baseline \(+\) post hit\(@1\!=\!1\), screened negative under hb interval) were chosen before per-cell CIs were aggregated but were not preregistered or externally certified; read them as authors’ internal selection thresholds, not a registered analysis plan. The full pipeline is recoverable from the body and tables; exact bit-for-bit reproduction requires the unreleased canary manifests and bSFT splits. Canary secrets are SHA-256 of whack-canary-{i:03d} truncated to \(16\) hex; bSFT samples \(5000\) examples without replacement under seed \(s\!\in\!\{0,1,2\}\) with image-filename exclusion (§15); training is stacked LoRA (\(r\!=\!16\), \(\alpha\!=\!32\)) on top of the canary-merged base with AdamW, peak lr \(2\!\times\!10^{-5}\), cosine schedule, batch \(2\!\times\!\mathrm{accum}\,4\), max sequence length \(1024\).

7 Selection audit (full)↩︎

2.5pt

8 Per-seed sign-consistency and headline intervals↩︎

Across the five canonical cells, every per-seed \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\), \(\Delta\mathrm{NLL}_{\mathrm{hex}}\), and hit\(@1\) value is sign-consistent across the \(3\) outer seeds (max spread \(0.0049\) on C3 hex, \(0.0255\) on C5 canary_lit); the 3-seed mean of each cell equals Table 2 by construction. Headline-row Student-\(t_2\) vs.hier-bootstrap intervals: C3 hex \([+0.0066,+0.0200]\) / \([+0.0061,+0.0226]\); C4 mean\(_{20}\) \([+0.0106,+0.0195]\) / \([+0.0129,+0.0169]\); C5 mean\(_{20}\) \([-0.0096,-0.0045]\) / \([-0.0118,-0.0021]\); C5 full hex \([+0.0027,+0.0124]\) / \([-0.0045,+0.0198]\). Both interval families are descriptive at \(n\!=\!3\); we read claims as joint requirements (same direction in both) and mark hb-only any claim that relies on hier-bootstrap alone.

9 Algebra closure: span decomposition of \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\)↩︎

For each canonical cell we decompose \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) into per-span contributions inside the \(K\!=\!20\) window. For each (canary, seed) instance let \(n_{s,c}\) be the number of tokens of span \(s\) that fall inside the canary’s \(K\!=\!20\) window and \(\overline{\Delta_{s,c}}\) the per-token \(\Delta\mathrm{NLL}\) over those tokens; the per-instance contribution of span \(s\) is \(n_{s,c}\,\overline{\Delta_{s,c}}/20\) (\(0\) if the canary has no tokens of \(s\) inside \(K\!=\!20\)). Averaging across all \(60\) (canary, seed) instances and summing across spans yields the per-canary mean \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) from the token-level forward pass.

The canonical mean\(_{20}\) uses the \(K\!=\!20\) probe pass; the token-level pass uses \(K\!=\!30\) to cover the hex span. Both compute teacher-forced cross-entropy in bf16 from the same weights, but bf16’s \(7\)-bit mantissa produces a per-cell residual of order \(10^{-4}\) (max \(-0.00030\) on C5). The residual is computed independently as canonical minus enumerated and shown as a disclosure row, not a free parameter. For text canaries (C3, C4) the Qwen2.5 tokenizer yields exactly \(n_{\mathrm{preamble}}\!=\!10\), \(n_{\mathrm{lit}}\!=\!1\), \(n_{\mathrm{hex}}\!=\!9\) inside \(K\!=\!20\); for image canaries (C1, C2, C5) BPE boundaries vary across strings, with cell-level averages \(\overline{n}_{\mathrm{lit}}\!\approx\!0.70\), \(\overline{n}_{\mathrm{hex}}\!\approx\!9.30\).

3pt

9.0.0.1 7B window K-sweep — derivable extremes.

The K-monotone claims for the 7B headline cells are recoverable from the algebra closure plus the headline full-span values: \(K\!=\!10\) is the preamble-only mean, \(K\!=\!20\) is the algebra-closure mean, and the full hex column reproduces Tab. 2. Intermediate \(K\!\in\!\{15,25,30\}\) values require re-running the per-token aggregator and are not printed.

4pt

10 All-cells systematic table↩︎

3pt

Table 7 reports \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\), \(\Delta\mathrm{NLL}_{\mathrm{hex}}\), and hit\(@1\) for every (canary modality, variant, step) cell in our matrix, averaged across \(3\) seeds. Per-cell hierarchical-bootstrap (\(B\!=\!10000\)) and between-seed Student-\(t_2\) \(95\%\) CIs on \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\) and \(\Delta\mathrm{NLL}_{\mathrm{hex}}\) are computed for the \(48\) CI-bearing rows (\(34\) bSFT \(+\) \(14\) LoRA-Noise; baselines have \(\Delta\!=\!0\) by construction); summary counts for the headline cells are reported in App. 8.

10.0.0.1 No descriptively clear-of-zero negative hex anywhere.

Across all \(48\) CI-bearing cells, zero cells have a hierarchical-bootstrap \(95\%\) CI for \(\Delta\mathrm{NLL}_{\mathrm{hex}}\) strictly below zero. Three cells have a slightly negative point estimate (U-LoRA-Noise \(-0.00113\), U-bSFT-GUI \(-7\!\times\!10^{-5}\), U-bSFT-Safety \(-0.00062\); all undertrained-regime step \(1000\), where baselines have hex NLL \(2.6\) and hit\(@1\!=\!0\)); each brackets zero under both hier-bootstrap and \(t_2\) between-seed intervals.

11 Cell accounting and matrix counts↩︎

The matrix decomposes as \(4\) bSFT variants (\(\mathrm{\small GUI}\), \(\mathrm{\small SYNTH}\), \(\mathrm{\small Safety}\), \(\mathrm{\small Random}\)) \(\times\,3\) canary regimes (saturated image, text, undertrained image) \(\times\,3\) seeds \(\times\,3\)–\(5\) step counts.

3pt

12 Merge-and-unload composition ablation↩︎

Stacked probe = base (with merged canary) \(+\) bSFT-LoRA loaded; Merged probe = base (with merged canary) \(+\) bSFT-LoRA merged into the base, then unloaded. Canary memory is intact in both cases; only the bSFT contribution changes from a LoRA delta to a full-rank weight update. Both deltas use the same \(M_{\mathrm{canary}}\)-only baseline. The ratio merged/stacked indicates how much of the stacked-LoRA drift survives the LoRA-to-full-rank conversion.

The “Stacked” column in Table 10 (C3 \(+0.0126\), C5 \(+0.0055\)) comes from the merge-comparison aggregation pipeline; the canonical \(\Delta_{\mathrm{hex}}\) in Table 2 (C3 \(+0.0133\), C5 \(+0.0076\)) comes from the \(K\!=\!30\) token-level probe pass. The two paths agree in sign at every cell; the magnitude gap (\(\!\approx\!0.002\), max \(0.0021\) on C5) reflects merge-pipeline aggregation drift, not the bf16 path residual. Table 2 remains the canonical headline estimator; Table 10 uses only stacked-vs-merged ratios.

Sanity check: merge does not erase all measured canary-span deltas. Secret-span deltas on C1, C3, C5 survive merging at ratios \(+0.98, +0.98, +0.77\); even on C4 itself the secret-span hex delta survives at \(+0.51\), so the bSFT delta is not wholesale erased. Per-token NLL agreement (Table 9) shows mean \(|\Delta\mathrm{NLL}_{\mathrm{stacked\_vs\_merged}}|\!\leq\!0.005\) on canary_hex for C1–C4, while C4’s preamble \(|\Delta|\!=\!0.02837\) is \(\sim\!118\times\) its own secret-span \(|\Delta|\!=\!0.00024\). The stacked-vs-merged disagreement isolates to C4’s preamble tokens, consistent with the Case 2 claim that C4’s mean\(_{20}\) does not reflect secret-span damage.

4pt

The C4 preamble collapse to \(+0.05\) ratio is consistent with several non-exclusive mechanisms (template-token-activation precision sensitivity, BF16 merge rounding for low-magnitude weight updates, LoRA-composition-specific effects) and we make no causal claim. The weak conclusion — a probe whose mass sits on preamble under loaded-LoRA evaluation is not robust under merged-LoRA evaluation, even though canary_hex deltas on the same and other cells are robust — is sufficient to demote C4’s mean\(_{20}\) from a memorization claim.

3pt

13 Direct re-exposure sanity check↩︎

To check that the secret-span and behavioural probes can move in this testbed (mean20 was the only probe that moved on saturated cells), we trained an additional T-bSFT-GUI variant where the GUI bSFT data was augmented with the \(20\) full-template text-canary captions themselves (full TEXTCAN-{16hex}-END, not a \(20\)-token truncation) — explicit re-exposure of the secret content, not a benign-SFT condition. Across \(3\) seeds: \(\Delta\mathrm{NLL}_{\mathrm{mean20}}\!=\!{+}4\!\times\!10^{-5}\) (vs. \({+}9\!\times\!10^{-5}\) on plain T-bSFT-GUI; flat both ways), \(\Delta\mathrm{NLL}_{\mathrm{hex}}\!=\!{+}0.0006\) (vs.\({+}0.0133\); \(\sim\!95\%\) collapse), hit\(@1\!=\!1.00\) (vs.\(0.88\); recovers to ceiling). The secret-span and behavioural probes move in the recovery direction when secret content is explicitly re-trained on, while mean20 stays flat. This is a probe-sensitivity sanity check, not a Whack-a-Mole-style reactivation control: direct re-exposure is the easiest possible positive contrast, and we use it only to address “can the probes move?”.

14 C3 failure inventory and hit\(@k\)↩︎

For C3 (T-bSFT-GUI step \(5000\) on text canaries) we report all greedy generations across \(20\) canaries \(\times\,3\) seeds \(=\,60\) generations (hit\(@1\)) plus \(k\!-\!1\) stochastic samples at \(T\!=\!0.7,p\!=\!0.95\) for hit\(@4\) and hit\(@16\) (i.e.\(3\) and \(15\) samples beyond greedy, in the convention defined in §2).

6pt

All \(7/60\) greedy failures are wrong hex tail (last \(1\)–\(2\) characters), e.g.target TEXTCAN-139bed18a2190bba-END \(\to\) greedy TEXTCAN-139bed18a2190b68-END; this confirms the Case-1 token-level analysis that damage concentrates on trailing hex tokens outside \(K\!=\!20\).

14.0.0.1 BPE token disclosure.

Qwen2.5’s BPE boundaries vary per canary, so the final canary_hex piece (token \(23\)) holds the last \(1\)–\(2\) hex characters: \(1\) for canaries \(0\)/\(14\) (which fail \(3/3\) seeds), \(2\) for canary \(12\) (\(1/3\) seeds). Per-canary string, predicted-token, and rank-of-correct-token manifests are authors’ internal artifacts and not released.

14.0.0.2 Leave-one-canary-out robustness.

Headline \(\Delta_{\mathrm{hex}}\!=\!{+}0.0133\). Removing canary \(0\) (its largest single-canary contributor, \({+}0.123\) on its own span) gives \({+}0.0075\) (\(\sim\!57\%\)); the other \(19\) leave-one-out point estimates lie in \([{+}0.01332,{+}0.01400]\) (\(<\!5\%\) of headline). The behavioural break is concentrated in \(3/20\) canaries with the same Mode 1 token-\(23\) pattern, not a single canary’s idiosyncratic tokenisation.

15 Image–bSFT dataset overlap audit↩︎

Image canaries draw from ScreenSpot v2 [12], a revised release of the original ScreenSpot benchmark of [30], and the same source as the GUI bSFT data; a near-duplicate would trivially explain the C1 erosion as in-distribution relearning. We verify no overlap two ways: (i) filename-set exclusion of the \(20\) canary images from bSFT sampling; (ii) perceptual-hash check (phash, \(8\!\times\!8\) DCT, distance \(\le\!4\)) across all canary–bSFT image pairs — no flagged pairs found in \(20\times5000\) comparisons. The text-canary placeholder is a single \(224\!\times\!224\) mid-gray bitmap shared by Safety and Random by construction; a varied-image text-canary control is future work.

16 Smaller-family decoy audit and Llama-1B trajectory details↩︎

This appendix collects the smaller-family-only descriptive evidence that the body summarises but does not print: the decoy table at the candidate C5-equivalent cells (Tab. 12), the migrated Qwen-1.5B C4-equivalent cell, and the Llama-3.2-1B step-trajectory disagreements. None of this evidence pertains to the original 7B C5 cell, and we read all of it as descriptive smaller-family stress tests (per the caption of Tab. 3).

4pt

16.0.0.1 Migrated Qwen-1.5B C4 cell.

On Qwen2.5-1.5B the C4-equivalent cell migrates to T-bSFT-Random (\(5000\)): \(\Delta_{\mathrm{mean20}}\!=\!{+}0.222\), \(\Delta_{\mathrm{hex}}\!=\!{+}9\!\times\!10^{-4}\), hit\(@1\!=\!1.00\); \(98\%\) of the mean drift sits on preamble with canary_hex essentially unchanged — a sharper version of the 7B C4 mechanism.

16.0.0.2 Llama-3.2-1B step-trajectory disagreements.

Two phenomena absent from the 7B testbed appear at the Llama-1B C3/C4-equivalent cells across step \(\in\!\{1k,3k,5k\}\): (i) on T-bSFT-GUI and T-bSFT-Safety the disagreement grows with bSFT magnitude rather than collapsing (\(\Delta_{\mathrm{mean20}}\!=\!{+}0.56\) to \({+}0.81\) at step \(5000\) co-occurs with greedy hit\(@1\) break \(0.37\)/\(0.00\) while teacher-forced canary_hex stays within \(1.6\!\times\) baseline); (ii) T-bSFT-GUI shows emergent recovery in greedy hit\(@16\) (seed-mean \(0\!\to\!0.20\!\to\!0.78\)) that is undetectable via \(\mathrm{mean}_{K=20}\) NLL, which grows monotonically over the same range. Inter-seed variance at step \(3000\) is high; we report a descriptive trajectory, not a dose-response claim.

References↩︎