1 Introduction↩︎

When a data contributor requests removal, the model trainer must identify and remove that contributor’s data from the training set. Machine unlearning methods such as Negative Preference Optimization (NPO) [1] and Representation Misdirection for Unlearning (RMU) [2] can degrade the influence of specific data once a forget set is provided, but all existing research assumes this forget set is already known. In practice, training datasets are assembled from thousands of sources through data processing, tokenization, and packing; by the time the dataset reaches the trainer, the connection between individual training lines and their original contributors has been lost. Without fine-grained provenance, the only options are catastrophic over-deletion (discard the entire dataset) or imprecise post-hoc inference [3]. forget-set quality directly determines unlearning effectiveness. Benchmarks such as TOFU [4] and MUSE [5] evaluate unlearning algorithms on synthetically curated forget sets, but their results cannot translate to real-world compliance without a mechanism to locate affected data at training granularity. Figure 1 illustrates this gap between knowing who to forget and knowing which data to forget.

As shown in Table 1, data version control tools (DVC [6], LakeFS [7], Delta Lake [8]) operate at file or dataset level. Experiment management tools (MLflow [9], Weights & Biases [10]) track metadata but not data origins. Provenance tools like yProv4ML [11] capture dataset-level relationships, while DLProv [12] requires extensive instrumentation. None provide record-level provenance with author attribution.

OriginBlame (ob) tracks which training records derive from which source sections and which authors contributed to those sections, via a three-layer architecture (Figure 2) with an independent token-index layer. Users integrate with one ob.track() call per record. ob provides precise forget sets at both record and token granularity.

We emphasize that ob does not discover authorship; it propagates it. ob applies to data collection pipelines where author identity is already established by the source environment—for example, MediaWiki revision histories or GitHub commits—but this attribution is lost during tokenization and packing. ob preserves the provenance chain across these transformations, enabling downstream queries (e.g., “find all training lines from author \(a_i\)”) that would otherwise require post-hoc inference.

This paper makes three contributions. First, we design a three-layer content-addressable architecture (authors \(\leftarrow\) sections \(\leftarrow\) document-index) with no ML or GPU dependencies. Second, we show that this architecture resolves author revocation requests into precise forget sets via deterministic queries and demonstrate that record-level provenance eliminates the dataset-level over-deletion that file-level tools incur. Third, we introduce an independent token-index layer that records source attribution during tokenization and packing, extending forget-set production to token granularity without requiring document-index records.

2 Problem Formulation↩︎

D’Angelo et al. [3] formalize the Forget-Set Identification (ForSId) problem: given a training set \(D\), a model \(M_D\), an unwanted set \(D_u\), and a wanted set \(D_w\), find a forget set \(D_f \subseteq D\) whose removal maximally preserves model behavior on \(D_w\) while altering it on \(D_u\). ForSId requires model access to compute per-sample influence via training-data gradients. More fundamentally, its input is behavioral evidence—samples whose predictions should change—not metadata-level queries such as “forget all data by author \(a_i\).”

When author \(a_i\) initiates a withdrawal request, the target forget set is \(F = \{l \in D \mid \text{author}(l) = a_i\}\). Computing \(F\) precisely without model access is the challenge ob addresses. Dataset-level provenance can only answer “data from author \(a_i\) exists in \(D\),” forcing two extremes: delete the entire dataset or take no action. Line-level provenance resolves this by answering “author \(a_i\) contributed \(n\) lines,” enabling targeted deletion—as we show in Section 5.2, record-level revocation reduces over-deletion by up to 101\(\times\) compared to dataset-level approaches.

From this gap, we derive three design requirements. Precision: locate specific data lines, not files or datasets. Verifiability: provenance must be independently auditable, not merely stored internally. Minimal invasiveness: integration must require only a few lines of code—DLProv [12] serves as a cautionary example, requiring extensive script instrumentation despite low runtime overhead.

3 Related Work↩︎

As shown in Table 1, existing provenance tools stop at file or dataset granularity. Chen et al. [13] propose fine-grained sample-level traceability with contribution scores across ML pipelines but focus on training-stage data usage auditing rather than authorship attribution. OriginBlame coexists with tools like DVC and HuggingFace Datasets: DVC manages file versions, while ob manages record-level author provenance.

Machine unlearning research similarly assumes the forget set is known (Table 2). Benchmarks TOFU [4] and OpenUnlearning use pre-defined forget sets, while MUSE [5] evaluates unlearning on real-world corpora with known forget/retain splits. Unlearning methods like NPO, RMU, GradAscent, and exact unlearning algorithms [14] directly receive forget sets as input. ForSId [3] (§2) is the only work addressing forget-set identification, but operates post-hoc with model access. OriginBlame captures origin information at recording time, requiring no model access.

This positioning complements methods like Attribute-to-Delete [15], which use datamodels to simulate unlearning but still require the forget set as input.

IPFS’s content-addressing design [16] inspired OriginBlame’s hash chain mechanism, but IPFS operates at chunk level (256 KB blocks) while OriginBlame refines granularity to lines. OriginBlame uses plain JSONL rather than a special file system. Compared to git-blame, designed for code editing scenarios, OriginBlame targets generated data scenarios: supporting multi-source attribution where a single line may fuse contributions from multiple authors, and author-initiated revocation.

4 System Design↩︎

4.1 Architecture Overview↩︎

OriginBlame (abbreviated as ob) stores all provenance metadata in a .ob/ directory colocated with the project data. All files are plain JSONL with no configuration files or central database.

Three-tier model. ob employs a hierarchical storage architecture. As illustrated in Figure 2, three core layers are linked by content-addressable hashes in a strict parent–child relationship:

Authors layer (top) stores identities and revocation tags: id (SHA-256 of name+email), name, email, and revoked (boolean). The revoked field is the single source of truth for revocation, cascading lazily at query time; email serves as the lookup key for revoke. Sections layer (middle) stores file-level copyright: section_hash (primary key, SHA-256 of {path, authors, license, year}), path (source file path, groups records by file), authors (list of author ids), license, year, and revoked (boolean). Document-index layer (bottom) stores per-record provenance: line_hash (SHA-256 of data content), file (output data file), sources (list of section hashes identifying contributing sources), source_type (origin indicator, e.g., "track" for explicit tracking), and revoked (boolean). The unique key is (line_hash, file, sources). Each document-index entry corresponds to one output record (one line in the data file), linked to its contributing sections and their authors. Document-index records do not store line numbers—blame computes hashes from data files at query time.

Storage. All core files are sharded into 256 buckets by the first two hexadecimal characters of their primary hash: document-index/00–/ff, sections/00–/ff, authors/00–/ff. Empty buckets are not created. This design enables O(1) lookup—given a line_hash, blame directly reads document-index/{first two hex chars} without scanning unrelated buckets. Multi-process safety is achieved without locks: each process writes to isolated files (docidx.{pid}, lock.{pid}), which ob clean later merges into sharded buckets. Unmerged pid files block read operations until clean completes. An embeddings.{model}/ directory stores per-model embedding vectors sharded into 256 buckets for reconcile (§4.4); the core requires no ML libraries or GPU—embeddings are computed by the reconcile utility (§4.4), not the user.

Content-addressable hashing. ob hashes dict inputs by serializing to JSON with sorted keys then applying SHA-256 [17]; str inputs are hashed as raw UTF-8 bytes. No normalization is performed. The collision probability for \(n\) records over a \(k\)-bit hash space is bounded by the birthday approximation \(p \approx n^2 / 2^{k+1}\) [18]. At the largest evaluated scale (\(n = 2.2 \times 10^5\) records, \(k = 256\)), this yields \(p < 10^{-68}\)—negligible relative to any practical failure mode.

Design principles. ob follows two core principles. First, what is not recorded does not exist: ob performs no post-hoc inference and does not mark unprovenanced content as provenanced. Second, single-hop provenance: the system tracks only direct mappings from raw sources to final output, without tracing intermediate processing stages. Deleting raw data files does not affect provenance or revocation.

4.2 Core Workflow↩︎

OriginBlame is designed as both a library and a CLI tool, with a Rust native implementation and optional Python bindings. Users initialize a project with ob init, creating the .ob/ directory; the Python init wrapper additionally writes a .gitignore for temporary files. One-time setup requires three CLI calls and one Python API call: (1) ob init, (2) ob author.add NAME EMAIL, (3) ob register.add --path PATH --authors AUTHOR --license LICENSE --year YEAR, (4) source.append(PATH) (Python API). Integration into an existing data pipeline requires adding a single track() call at the point where each training record is written to disk—for example, inserting track(record, file="data/train.jsonl") immediately before writing the record to the output file. The user’s existing pipeline logic remains unchanged; ob only observes the data passing through.

Source management: source.append(path) resolves paths to section records and activates them, registering all sections associated with that path. An optional section parameter selects a single section by its hash. source.pop() removes the most recently appended source, or a specific source by file path. with ob.sources(...): provides scoped tracking. Track operations associate with all active sources automatically—the caller does not pass sources explicitly.

ob.track() workflow: The track(data, file) library function records provenance for a single data entry. Source attribution is resolved either from an explicit source= parameter (a file path that resolves to registered sections, or a list of section hashes) or, by default, from the source stack. As shown in Figure 3, it (1) computes the SHA-256 hash of data (JSON serialization for all types), (2) resolves the active source list into section hashes, (3) checks for duplicates (idempotent), and (4) writes to process-isolated files via WAL (lock.{pid} \(\rightarrow\) docidx.{pid}) in the Python API (the Rust native implementation writes directly to sharded buckets). If the process crashes before the lock is deleted, remaining data can be recovered by ob clean. An optional embedding parameter (Python API) stores a per-record embedding vector alongside the document-index record for later use by reconcile (§4.4).

Query functionality: ob provides two orthogonal query methods. ob blame performs a forward query: given a file path and line number, it reads the line, computes SHA-256 to obtain line_hash, queries document-index/{line_hash[:2]}, and traverses the provenance chain: line_hash\(\rightarrow\)sources (section_hashes)\(\rightarrow\)sections (author_ids, license, path)\(\rightarrow\)authors (name, email). ob show performs a reverse query: given --author, it traverses authors\(\rightarrow\)sections\(\rightarrow\)document-index without reading data files. With --index, both commands use the bucket-routing index (Appendix 8) for faster lookup. Additional filters include --section (filter by section hash), --license (filter by license name), --revoked (show only revoked entries), and --tokenizer (token-level granularity); these can be combined for intersection queries. ob status displays aggregate author counts and, with --tokenizer, token-index statistics including entry counts, active/revoked breakdowns, and total token counts.

Operation log: The .ob/log file records state-changing operations (author/section registration, revocation, purge, clean, merge) as append-only log lines with timestamps, providing an audit trail for administrative actions. Read-only operations (blame, show, status) and per-record tracking are excluded to avoid volume overhead. ob clean rotates the log. ob log queries the log with optional --op and --since filters.

4.3 Revocation and Purge↩︎

ob supports revocation at three granularity levels (Figure 4), all based on a tag model rather than bulk deletion. The single source of truth for revocation resides on the author record; all authors are equal—a contributor of a single line holds the same revocation rights as a contributor of the entire dataset.

Author-level revoke. ob revoke --author "z@e.com" sets revoked=true on the author record. No other files are modified at this point. When a subsequent query (e.g., blame or show) encounters a revoked author, it evaluates the revocation status lazily along the chain: scan authors for revoked=true \(\rightarrow\) find all sections whose author_ids include that author \(\rightarrow\) collect all document-index entries referencing those section hashes. The result is a (file, line_hash) list of every data line traceable to the revoked author. The --reverse flag undoes a revocation by toggling the tag back to false; with --tokenizer, revocation also marks matching token-index entries.

Section-level revoke. ob revoke --section HASH toggles revoked=true on a specific section record, affecting only the document-index entries that reference that section without revoking the author. This handles cases where a single file or data source must be withdrawn without affecting other content from the same contributor.

Line-level revoke. ob revoke --line-hash HASH --file FILE toggles revoked=true on a single document-index entry, enabling granular line-level withdrawal. This is the finest revocation granularity, useful when a specific data line must be removed without affecting any other lines from the same section or author.

Purge. ob purge --file FILE physically deletes revoked data lines from the specified file, leaving document-index cleanup to ob clean which archives revoked metadata into archive/. A safety constraint enforces no purge without prior revoke: purge can only delete lines that belong to revoked authors or revoked sections, preventing accidental data loss. The --index variant routes directly to the relevant index buckets for faster operation. The --dry-run flag previews which lines would be deleted without modifying files. The --reverse flag restores previously purged lines from the archive directory.

Lazy cascade rationale. All revocation levels share a common design: the revoke operation modifies only a tag field on a single record. By default, queries exclude revoked entries (document-index and token-level alike); the --revoked flag explicitly requests revoked entries. This yields two properties: (1) revoke is lightweight regardless of how many data lines are affected; (2) revocation is fully reversible—canceling a revoke removes the tag with immediate effect.

4.4 Package Ecosystem↩︎

The ob architecture separates a core package from an optional utility package for advanced features.

Core package provides CLI commands for setup (init, author.add, register.add), source management via Python API (source.append/pop), querying (blame, show, status), audit logging (log), revocation (revoke, purge), and maintenance (clean, merge, index build, generate-set, version). The core compiles to a single native binary; users integrate via author.add + register.add + source.append + track.

Utility features (enabled via compile-time feature flags) provide optional parsers (parse --parser mediawiki), reconcile (reconcile), and export (export-copyright). The core binary performs zero imports of utility modules; when compiled without these features, ob omits the corresponding commands entirely.

Reconcile mechanism: Reconcile operates in two phases (Figure 5). Pass 1 attempts hash exact matching: each line is hashed and looked up in the document-index; a hit means unchanged content. Pass 2 performs embedding-based semantic matching via cosine similarity; a hit inherits the old record’s sources, and unmatched old records are marked as orphans. Lines missing both phases require manual track().

4.5 Token-Level Provenance↩︎

The three-layer architecture (authors \(\leftarrow\) sections \(\leftarrow\) document-index) captures per-record provenance, but tokenization and packing into binary shards eliminates record-level identity. ob addresses this with an independent token-index layer that links each document’s token range in the packed binary to its source and author chain, operating in parallel to the document-index.

Token-index entry. Each entry stores token_count (number of tokens produced from one document), sources (list of section hashes, linking to the author chain), tokenizer (identifier string, e.g., “gpt2” or “llama3”), and revoked (boolean). The token-index is completely independent of the document-index: entries share the same source references but require no line hashes, and the two layers can exist with or without each other.

Two operating modes. In file mode, the pipeline writes JSONL data files and calls track() to create document-index records; a subsequent tokenize+pack stage writes token-index entries. Both layers coexist, serving record-level and token-level queries respectively. In streaming mode, the pipeline tokenizes documents directly without producing intermediate JSONL; only token-index entries are written, making the token-index the sole provenance record.

Storage. Token-index entries follow the same PID file pattern as document-index: each worker writes to token-index.{tokenizer}.{pid}, which ob clean merges into numbered files under token-index.{tokenizer}/. To preserve position-indexing correctness across parallel workers, ob clean merges PID files sequentially and computes cumulative token offsets only after all files are merged—not during write. Different tokenizers are stored in separate directories, following the same pattern as the optional embeddings directory. The tokenizer name is passed as a parameter; no tokenizer library is imported.

Token-level queries. Because entries are written sequentially during tokenization, the \(i\)-th entry’s token range in the packed binary can be reconstructed from the cumulative sum of preceding entries’ token_count values, with its length given by its own token_count. This makes the token-index a position-indexed provenance layer—no per-token records are needed. ob show --author NAME --tokenizer gpt2 traverses author \(\rightarrow\) section \(\rightarrow\) token-index, summing token counts for all matching entries. ob revoke --author NAME --tokenizer gpt2 marks matching entries as revoked. ob generate-set --tokenizer gpt2 -o forget.bin produces a binary bitmask where each bit corresponds to one token-index entry (1 = revoked, 0 = active), directly usable by unlearning algorithms. ob status --tokenizer gpt2 reports aggregate token counts.

Immutability constraint. Because entry positions encode token ranges, the token-index does not support archive or purge operations. Revoked entries remain in the index with revoked=true; the bitmask produced by generate-set is the sole mechanism for conveying revocation to downstream unlearning algorithms. If the training pipeline is re-run with different data, the token-index is rebuilt from scratch rather than reconciled.

Framework integration. The integration pattern is framework-agnostic: intercept each data record, register provenance metadata, and pass the record through unmodified. We validate this with two frameworks. For Datatrove [19], we implement OBProvenanceBlock as a PipelineStep whose run() method receives a generator of Documents, registers authors and sections from Document.metadata, writes token-index entries, and yields each document to the next stage—the same interface used by Datatrove’s built-in blocks (e.g., LanguageFilter, TokensCounter). For HuggingFace Datasets, an equivalent integration wraps the .map() callback: the function loads zhwiki data via load_dataset(), applies author_add + register_section + track per document, and writes provenance alongside output. Both integrations require the same \(\sim\)​5-line adapter and produce identical .ob/ directories; the downstream tokenizer requires no modification. Section 5.5 reports the overhead of both integrations.

5 Evaluation↩︎

This section validates that fine-grained provenance at both record and token granularity is necessary for practical author revocation. We measure revocation precision, reconcile recovery rates, storage overhead, query latency across four dataset scales, token-index integration overhead in a real processing framework, and whether provenance-based forget sets improve machine unlearning effectiveness over random baselines. This is a functional validation on real data, not a comprehensive performance benchmark.

5.1 Experimental Setup↩︎

The evaluation dataset originates from a Chinese Wikipedia (zhwiki) dump comprising 219,555 wiki pages authored by 482,543 unique contributors under CC-BY-SA-4.0 licenses. The data construction pipeline has two stages.

Stage 1 — Parse. A stream parser extracts the latest-revision wikitext and the full contributor list from each page in the XML dump. Only namespace-0 articles with \(\geq\)​50 characters and at least one contributor are retained, producing one source file per page and a metadata index (title, authors, year).

Stage 2 — Track. At each target scale, wiki source texts are written as JSONL records via HuggingFace Datasets or Datatrove pipelines, with ob provenance tracking inserted at the document processing stage. Each record receives author registration, section creation, and provenance linking through the OBTrack pipeline step. The resulting JSONL lines each track to their wiki page sources through the three-layer chain, while the Datatrove pipeline additionally produces token-index entries for token-level provenance queries.

5.2 Revocation Precision↩︎

The central question is whether record-level provenance granularity matters in practice. To answer it, we simulate author revocation at the 10k scale for four contributors with widely different contribution shares, using a HuggingFace tokenization pipeline on zhwiki data. Table 3 reports the results.

4pt

Without record-level provenance, the only available action is to discard the entire training set—there is no intermediate file-level option, because wiki pages are collaborative documents that cannot be subdivided by author. The resulting over-deletion ranges from 1.3\(\times\) (InternetArchiveBot, 79.5% coverage) to 101\(\times\) (HuangQQ, 1.0%). Since each generated record derives from exactly one wiki page, file-level deletion is equivalent to record-level deletion here; remaining over-deletion comes from co-authorship within pages.

The over-deletion factor follows a heavy-tailed distribution across all authors in the dataset: the four benchmark authors in Table 3 span two orders of magnitude in over-deletion. This pattern is consistent across all four tested scales (1k–220k): the largest-share author (InternetArchiveBot, 63–80% coverage) consistently has 1.3–1.6\(\times\) over-deletion, while the smallest-share authors (\(\approx\)​1%) consistently face \(\approx\)​100\(\times\) over-deletion under dataset-level revocation. All four authors have 100% of their records in multi-author sections—meaning that even record-level revocation affects shared content. This co-deletion is inherent to collaborative authorship.

5.3 Reconcile Effectiveness↩︎

When data files are edited after initial tracking, provenance links break because content hashes no longer match. We evaluate the reconcile mechanism (§4.4) with deterministic mutations (seed = 42): 10% text edited, 5% deleted, 5% new lines, using nomic-embed-text-v1.5 at cosine threshold 0.85. Table 4 reports the two-phase recovery rates.

4pt

Hash matching recovers 85–87% of original lines; embedding similarity recovers an additional 10–13%, bringing total recovery to 96–98% across all scales. Recovery improves with dataset size (96.3% at 1k to 98.2% at 100k). The remaining 2–4% are genuinely new or heavily rewritten content requiring manual re-tracking via ob.track().

5.4 Scalability↩︎

Table 5 reports operation latency across four scales from 1k to 220k lines, averaged over three runs per scale, using real zhwiki data from the HuggingFace pipeline. The implementation uses rayon for parallel bucket scanning and a binary index (Appendix 8) that stores author-to-section routing. show and purge use parallel rayon workers to scan 256 bucket files concurrently; at 220k lines, show completes in 80 ms and revoke in 22 ms.

3pt

Line coverage is 100% at all scales, confirming every data line has a complete provenance chain.

5.5 Token-Level Provenance Evaluation↩︎

We evaluate the token-index layer on three axes: integration overhead in a real processing framework, query performance, and metadata preservation.

Framework integration. We validate the integration pattern with two frameworks on zhwiki data at four scales: from 1k pages (2.8M gpt2 tokens, 4.8k authors) to 220k pages (712M tokens, 483k authors). For Datatrove [19], Pipeline A is a vanilla pipeline (reader \(\rightarrow\) tokenizer \(\rightarrow\) packed output); Pipeline B inserts OBProvenanceBlock between reader and tokenizer. For HuggingFace Datasets, an equivalent .map() callback applies provenance tracking per document. Table 6 reports both frameworks across all four scales.

2.5pt

At the 1k-page scale, the Datatrove provenance block adds 13.8% throughput overhead while HuggingFace adds only 2.0%, reflecting that provenance tracking is I/O-bound and HuggingFace’s pipeline is already bottlenecked on tokenization. Storage overhead follows a consistent trend: from 1.33\(\times\) at 1k pages to 1.23\(\times\) at 220k pages, reflecting that structured JSONL provenance metadata grows sublinearly relative to token data. HuggingFace shows a 4.0% throughput penalty at the 220k-page scale (712M tokens). The storage ratio is higher at small scales because .ob/ stores structured metadata (author names, emails, section hashes, licenses) while the packed output is compact binary token IDs (2 bytes each).

Query performance. After Pipeline B completes, ob show --author NAME --tokenizer gpt2 traverses the full author \(\rightarrow\) section \(\rightarrow\) token-index chain in 3–69 ms across the four tested scales (Table 6). At the 220k-page scale (712M tokens, 483k authors), ob revoke completes in 16 ms.

Metadata preservation. Pipeline A produces packed binary output with document boundaries but no author information—consistent with the behavior of Datatrove’s DocumentTokenizer and Megatron-LM’s preprocess_data, which discard metadata during tokenization. Pipeline B preserves the complete author \(\rightarrow\) section \(\rightarrow\) token-index chain, enabling provenance queries that are impossible in the vanilla pipeline.

5.6 Cross-Domain Generalization↩︎

To evaluate whether ob generalizes beyond wiki-style text, we apply the same integration pattern to Linux kernel C source code. The attribution source is git blame -e, which provides line-level authorship for each .c/.h file—a more accurate attribution than the file-level git log --name-only approach used by The Stack [20] and StarCoder [21]. Files are first filtered by commit frequency (top \(N\) most-committed .c/.h/.S/.rs/.dts/.dtsi files), then attributed via git blame on the top 1,000 files (line-level, budget-capped), with bulk git log commit-frequency attribution for the remainder. We test three scales: 1 000, 10 000, and all \(\approx\)​44k attributed source files (Table 7).

2pt

Both frameworks integrate ob provenance tracking with no code changes to the pipeline pattern established in §5.5. Storage overhead stays under 1.06\(\times\) across scales, and all ob queries complete under 10 ms. The HuggingFace throughput penalty is negligible (\(<\)​1%); Datatrove shows higher overhead (25.9–41.2%), driven by kernel source files being smaller and faster to process, which makes the provenance block a larger relative cost.

The over-deletion analysis reveals that file-level deletion remains wasteful even with accurate attribution: at the smallest scale (1 000 files, 671 authors), revoking Linus Torvalds’ contributions at file granularity would delete 9\(\times\) more lines than necessary. At the full scale (\(\approx\)​44k files), the over-deletion factor drops to 1.3\(\times\).

5.7 Application: Machine Unlearning↩︎

This subsection asks whether that precision translates to a downstream task: given a revocation request, does ob’s line-level forget set enable more effective machine unlearning than a same-size random baseline?

Setup. We fine-tune Qwen3-1.7B (1.7B parameters) with QLoRA (rank = 16, 4-bit NF4 quantization, bf16) on ChatML QA pairs generated from zhwiki data via the Zhipu GLM-4-Flash API. The SFT checkpoint (1 epoch, lr = 2\(\times\)​10\(^{-4}\), effective batch = 16) serves as the shared starting point for all unlearning runs. We select two contributors with different contribution shares—Berthe (9.3%, 14 882 lines) and Antigng-bot (8.6%, 11 618 lines)—and construct paired forget sets for each: line uses ob’s provenance to select exactly the lines attributed to the target author, while random draws a same-size random subset from the full corpus (seed = 42). Both forget sets contain the same number of lines; the only difference is which lines are selected.

We apply three unlearning algorithms: NPO [1] (\(\beta\) = 0.1, 5 epochs, lr = 10\(^{-5}\)), RMU [2] (layer = 14, \(\alpha\) = 100, 80 steps), and gradient ascent (3 epochs, lr = 5\(\times\)​10\(^{-5}\)). Evaluation uses four metrics: forget-set perplexity (PPL, higher = better forgetting), retain-set PPL (lower = better utility preservation), forget-set ROUGE-L (lower = less memorization), and retain-set ROUGE-L (higher = better preservation). All metrics are computed on sampled subsets (1 000 lines for PPL, 100 for ROUGE-L) from the respective splits.

1.5pt

NPO results. Table 8 shows that ob’s line-level forget sets dominate the random baseline on all four metrics for both authors (Figure 6). Averaging across the two authors, NPO with line-level provenance achieves forget PPL = 8.97 (2.56\(\times\) SFT baseline) versus 6.32 (1.80\(\times\)) for random—a 42% improvement in forgetting. Simultaneously, retain PPL is 4.58 (1.52\(\times\)) versus 5.97 (1.98\(\times\))—a 23% improvement in utility preservation. The ROUGE-L metrics tell the same story: line-level achieves forget ROUGE-L = 0.146 (0.37\(\times\) SFT) versus 0.259 (0.65\(\times\)), and retain ROUGE-L = 0.355 (0.87\(\times\)) versus 0.296 (0.73\(\times\)). The consistency across both authors—who differ in contribution share and forget-set size—indicates that the advantage stems from the selection quality of provenance-based forget sets rather than any author-specific artifact.

RMU results. All four RMU variants produce metrics nearly identical to the SFT baseline (Table 8), confirming the known incompatibility between RMU and QLoRA [2]: 4-bit quantization introduces step sizes (\(\sim\)​0.05) that dwarf RMU’s weight updates (\(\sim\)​10\(^{-4}\)), preventing the hidden-state steering that RMU requires. All published RMU results use full fine-tuning.

Gradient Ascent. All gradient ascent runs produce catastrophic model collapse (Table 8), with forget PPL reaching 4.4–4.8\(\times\)​10\(^{8}\) for line-level and 1.8\(\times\)​10\(^{5}\)–6.7\(\times\)​10\(^{6}\) for random. Retain PPL is similarly elevated, confirming total utility loss. However, the ROUGE-L metrics reveal a secondary signal: line-level variants preserve retain ROUGE-L at 0.30–0.32, while random variants collapse to 0.06—a 5\(\times\) difference. This suggests that precise forget sets create a more bimodal degradation pattern: extreme on the targeted content, relatively preserved elsewhere. Random forget sets distribute the destructive gradient signal across all content, producing more uniform degradation. Even with a destructive algorithm, provenance-based targeting preserves more of the retain set.

Takeaway. Line-level provenance enables NPO to achieve simultaneously better forgetting and better retention than random baseline forget sets, across two authors with different contribution profiles. This demonstrates that the provenance problem—which data to forget?—directly affects unlearning quality.

6 Discussion↩︎

OriginBlame has four limitations. (1) Incremental adoption is difficult: users cannot retroactively add provenance to existing datasets. (2) The parser ecosystem is immature: only a MediaWiki parser is currently available, though our cross-domain evaluation (§5.6) confirms that the core provenance tracking is domain-agnostic and works with arbitrary attribution sources like git blame. (3) Single-hop provenance: the system tracks only direct mappings from raw data to final output. (4) The index must be rebuilt when data changes; indexed variants do not consistently outperform the full-scan path due to rayon parallelism amortizing the scan cost.

Future work will pursue three directions: expanding the parser ecosystem to support Common Crawl archives, PDF documents, and mainstream NLP dataset formats; exploring incremental adoption paths that allow users to partially embed OriginBlame in existing pipelines; and extending the token-index layer to support per-token attribution (rather than per-document token-count attribution) for integration with token-level unlearning methods.

6.0.0.1 Privacy and dual-use.

The authors layer stores contributor names and emails alongside their provenance records. In production deployments handling sensitive contributor data, operators can strip PII from .ob/ by removing name and email fields from author records—leaving only the SHA-256 identifier—and maintaining a separate encrypted mapping outside the provenance directory. The operation log (.ob/log) provides an audit trail for revocation and query commands, supporting access-control policies that restrict provenance queries to authorized compliance personnel. We note that fine-grained author attribution could be misused to target individual contributors; deploying organizations should restrict provenance query access accordingly.

7 Conclusion↩︎

Machine unlearning research has overlooked a prerequisite: locating the affected data. OriginBlame addresses this gap with a three-layer content-addressable architecture that propagates author identity through data processing pipelines, resolving revocation requests into precise forget sets without model access or post-hoc inference.

Evaluation across multiple domains and scales confirms practical deployability: record-level provenance eliminates the 1–2 orders-of-magnitude over-deletion inherent in dataset-level approaches; integration overhead remains modest across frameworks; and two-phase reconcile recovers 96–98% of provenance links after mutations. On the downstream unlearning task, provenance-based forget sets substantially outperform random baselines, demonstrating that the provenance problem directly affects unlearning quality.

The source code and evaluation scripts are available at https://github.com/tzbkk/originblame; the Rust implementation at https://github.com/tzbkk/rust-originblame.

Acknowledgements↩︎

The author used OpenCode (GLM-5.1) for language editing and code assistance, and takes responsibility for all AI-assisted content.

8 Bucket-Routing Index↩︎

show and purge grow super-linearly because they scan all 256 document-index shard files and expand each record’s section and author links. The bucket-routing index eliminates this full scan by pre-computing which shard files are relevant for each author and section.

8.0.0.1 Structure.

The index is a fourth layer stored in .ob/index/, sharded by the same 2-character hex prefix as the other layers. Each shard file contains JSONL records of the form {"id": "<hash>", "refs": ["ab", "cd", ...]}, where id is either an author ID or a section hash, and refs lists the 2-character bucket prefixes of the next layer’s relevant shards. Specifically:

• Author entries: id = author ID, refs = section bucket prefixes containing that author.

• Section entries: id = section hash, refs = document-index bucket prefixes containing records that reference that section.

The index is built by a single pass over the document-index and section layers via ob index build.

8.0.0.2 Query path.

For show -- author --index, the query proceeds in three hops, each reading only the relevant shard files:

1. Look up author entries in the index \(\to\) obtain section bucket prefixes.

2. Read those section shards to find matching sections \(\to\) look up section entries in the index \(\to\) obtain document-index bucket prefixes.

3. Read only those document-index shards to collect results.

purge -- author --index follows the same three-hop path but returns only (line_hash, file) pairs without expanding section and author details.

8.0.0.3 Implementation details.

Two optimizations reduce I/O:

1. Batched bucket reads. At each hop, IDs are grouped by their 2-character prefix so each shard file is read at most once, even when hundreds of IDs map to the same bucket.

2. Batch pre-loading. For show, all author records referenced by matched sections are loaded into memory in a single pass over the author shards before the document-index loop, replacing per-result get_author() calls with constant-time dict lookups.

8.0.0.4 Unified binary index.

The binary index (OBIDXF02) uses type-tagged references to serve both the document-index and token-index layers. Each reference has a 1-byte type tag: 0x00 for document-index shard prefixes and 0x01 for token-index byte ranges. A single section hash can reference both document-index shards and token-index ranges across multiple tokenizers. Queries filter by type and tokenizer to access only the relevant layer.

8.0.0.5 Performance.

Line-level query latency across all scales is reported in Table 5 (§5.4); token-index query latency in Table 6 (§5.5). The index is opt-in via the --index flag; without it, both commands fall back to the parallel full-scan path.

9 Reconcile Mutation Algorithm↩︎

The reconcile benchmark (§5.3) simulates data edits by applying three operations to the tracked data file with a deterministic random seed (42):

1. Edit (10%): Each line is independently classified by drawing \(r \sim U(0,1)\). Lines with \(r < 0.10\) are edited by replacing 10% of their alphabetic characters with random lowercase letters. For JSON records, only the text field is modified; other fields (metadata, section hashes) are preserved.

2. Delete (5%): Lines with \(0.10 \leq r < 0.15\) are removed entirely.

3. Insert (5%): \(0.05 \times N\) new lines are generated by copying random existing lines and modifying 15% of their alphabetic characters. Insertions are placed at random positions.

This design preserves the character-set distribution of the original data (unlike random byte-flipping, which would produce invalid UTF-8 in Chinese text) while ensuring that edited and inserted lines have no hash overlap with original document-index entries, isolating the embedding-based semantic matching phase.

References↩︎