1 Introduction↩︎

Many empirical software engineering studies begin the same way: researchers identify relevant GitHub issues or pull requests, inspect their discussion, decide whether each belongs in the study, and assign research labels. In practice this workflow is iterative and evidence-heavy. Researchers read issue bodies, comments, linked pull requests, patches, tests, and maintainer labels, and they must revisit records as those records evolve. They also need to record why a record was included or excluded, which evidence supports each label, and where reviewers disagree. Such needs are common in mining software repositories research, which routinely draws on the evolving socio-technical evidence that GitHub records, such as pull-based development traces and contribution-evaluation signals [1]–[3].

Two common workflows address this task, and each leaves a gap. Spreadsheet-based coding is lightweight and flexible: a row can hold a URL, a few labels, and a note. With enough discipline a researcher can also paste screenshots or archive pages by hand, so the limitation is not that any single field is impossible to store. The difficulty is keeping the page evidence, the evolving research codes, and the record of why a decision was made linked together and queryable as the study progresses, rather than scattered across tabs, files, and ad-hoc columns. At the other extreme, mining scripts collect structured data at scale but are typically disconnected from the manual reading and qualitative coding that such datasets depend on, and they discard the rendered page context a reviewer later wants to re-examine. The result is a gap between the source artifact, the researcher’s evolving interpretation, and the exported dataset used in a paper.

RepoTrace addresses this gap with a local research workbench for browser-first GitHub data collection, aimed at researchers and students who construct issue and pull-request datasets manually or semi-manually. The tool keeps researcher judgment auditable rather than replacing it, storing the evidence and the labels together. RepoTrace therefore treats a GitHub issue or pull request as both a source document and a changing research object, storing the browser-collected snapshot, extracted metadata, research labels, notes, review decisions, and refresh history in one local workspace. This design follows prior warnings that mined repository data must be interpreted with care, because repository traces may omit context or encode process assumptions [1], [4].

RepoTrace combines a Chrome side panel, a SQLite-backed backend, a React dashboard, scoped JSON exports, project backup, and refresh tracking around five productivity-oriented capabilities:

• Evidence capture. The side panel collects selected page fields, comments, labels, snapshots, and provenance while a researcher reads a GitHub page, so the original evidence stays next to each coding decision and there is less copy-and-paste work.

• Project-scoped organization. Each project keeps source metadata separate from research labels, notes, screening decisions, linked artifacts, and fix evidence, which makes exploratory collections and final datasets easier to maintain.

• Review and labeling workflow. A single dashboard lets researchers filter records, inspect evidence, assign labels, check data quality, and surface disagreements without switching between browser tabs, spreadsheets, and notes.

• Refresh and audit history. Refresh checks reveal whether collected records changed after initial inspection and preserve an audit trail of later updates, so stale evidence is caught before export.

• Export and backup. Scoped exports produce one record, a filtered dataset, or a complete project backup for paper analysis, collaborator review, and artifact sharing.

We demonstrate RepoTrace on a self-collected dataset of 20 matplotlib/matplotlib issues organized into two study projects, spanning backend/rendering defects and regression/compatibility evidence. The walkthrough exercises the full workflow—collection, screening, labeling, consensus review, refresh, and scoped export—and shows that the tool preserves the page snapshots, comments, GitHub labels, research labels, notes, and rationales that the coding decisions depend on. RepoTrace is released as a reproducible source artifact with seeded demo data and an automated test suite. The demonstration video is available.

2 Background and Related Work↩︎

RepoTrace sits between two classes of existing tools. Repository-mining frameworks collect GitHub data at scale: GHTorrent mirrors GitHub’s event stream into a queryable dataset [5], PyDriller offers a Python API over commit and change history [6], and the GitHub REST and GraphQL APIs provide similar structured access [7]. These tools excel at breadth, but they return normalized records rather than the rendered page a reviewer reads, and they are decoupled from the manual screening and qualitative coding that researcher-curated datasets rely on. Qualitative data-analysis tools such as MAXQDA, ATLAS.ti, and NVivo support code books, multi-rater coding, and inter-rater agreement, but they are document-centric and GitHub-unaware: they do not detect issue or pull-request pages, extract their source metadata, or track when the underlying artifact changes after coding. Researchers therefore bridge the two with spreadsheets and ad-hoc scripts, recreating the very provenance gap that prior mining-methodology work cautions against [1], [4].

RepoTrace targets this middle ground. Rather than mining at scale or coding generic documents, it couples browser-based evidence capture that preserves the rendered page with study-specific labeling, screening, refresh tracking, and an audit trail in a single local workspace. Its scope is manual, qualitative dataset construction, where the live evidence, the evolving codes, and the rationale for each decision need to stay linked together throughout the study.

3 Tool Design and Workflow↩︎

RepoTrace is organized around dataset projects. A project represents a single study, such as a bug or backend-behavior dataset for a specific repository, and every collected record belongs to one project, letting researchers keep exploratory collections, final datasets, and demo data apart. As summarized in Figure 1, the normal workflow follows five steps: open a GitHub issue or pull request, collect evidence in the side panel, review and label the record in the dashboard, refresh or audit it when needed, and export a JSON dataset or project backup. The subsections below describe each step in turn, following the path of a record from the browser to an exported dataset.

3.1 Collection from GitHub Pages↩︎

The Chrome extension detects supported GitHub issue and pull-request pages. In the side panel, the researcher inspects the detected repository, record number, title, URL, and duplicate status for the current project, then chooses which visible attributes to preserve, such as title, body text, labels, state, timestamps, comments, and the raw snapshot. Before saving the record to the local backend, the side panel can also attach a current-page note, research labels, a screening decision, fix evidence, and manually linked artifacts.

Browser extraction is the primary collection path. Optional GitHub API support can enrich or refresh records, but the workflow requires no token, which keeps RepoTrace usable in local classroom and artifact-review settings. Browser-first collection also preserves discussion context that API responses may normalize or omit, including visible timeline text, inline links, and page-specific evidence. Issue discussions and bug reports often contain the reproduction steps, expected and observed behavior, environment details, and follow-up clarification needed to judge the quality of a record [8], [9].

3.2 Research Dashboard↩︎

The dashboard provides project visibility, record search, queue cards, filters, record details, research-label management, notes, annotations, fix evidence, linked artifacts, consensus state, quality checks, audit logs, and refresh history. The main research filters operate over source labels, research labels, reviewers, inclusion decisions, fix status, fix strategy, linked artifacts, and evidence completeness, while workflow queue cards surface records that need a second review, consensus, taxonomy backfill, update review, or refresh.

Record details are organized into three views. The evidence view shows collected source fields, snapshots, comments, timeline text, and provenance; the research view shows notes, labels, screening rationale, fix evidence, consensus candidates, and backfill decisions; and the log view shows audit and change history.

3.3 Research Labels and Taxonomy Evolution↩︎

RepoTrace distinguishes GitHub labels from research labels. GitHub labels are source metadata maintained by the project, such as bug, Documentation, or status: needs clarification. Research labels are study-specific codes created by the researchers, such as symptom, component, evidence source, fix status, confidence, or fix strategy. The two play different roles: a maintainer label is evidence about how the project classified an issue, whereas a research label is an analytical decision made under the study protocol.

Research-label categories can evolve during analysis. When a user adds a new category or label, RepoTrace records a taxonomy-change entry and queues backfill review for older records that may need reconsideration, preventing silent relabeling and keeping the historical coding process auditable. For multi-reviewer coding, RepoTrace stores the labeler, confidence, rationale, and status of each annotation, and surfaces records with conflicting labels under the same category for consensus review. These disagreement records can later support standard inter-rater reliability and agreement checks when a study protocol requires quantitative reliability reporting [10], [11].

3.4 Exports for Research Use↩︎

RepoTrace distinguishes three export scopes. A single-record export creates a reviewable evidence package for one record. A filtered-record export captures the current dashboard result set, the most natural artifact for analysis after applying project, repository, label, or workflow filters. A project backup exports a complete project for restoration or artifact sharing. Keeping these scopes distinct avoids conflating a full database backup with a research dataset or a single-case evidence package.

3.5 Refresh and Change Tracking↩︎

GitHub records are not static. After a researcher first collects a page, maintainers may close the issue, add labels, edit the description, or post new comments. RepoTrace therefore provides a single-record refresh and a project-level Check Updates action; a project-level run summarizes changed, unchanged, and failed records. When changes are detected, RepoTrace stores a new snapshot, records change-log entries, and marks the record as needing update review. This behavior follows the tool’s local-first scope: it does not continuously synchronize with GitHub, but it gives researchers an explicit checkpoint before exporting or submitting a dataset. The optional API refresh path uses GitHub’s documented REST API as an enrichment channel rather than as the sole source of study evidence [7].

4 Implementation↩︎

RepoTrace is implemented as a single TypeScript project with four parts. The extension is a Manifest V3 Chrome extension with content-script extraction and a side-panel UI. The backend is an Express application backed by SQLite through better-sqlite3. The dashboard is a React application built with Vite. A set of shared TypeScript types defines records, snapshots, comments, labels, annotations, notes, reviews, exports, and backups, giving the extension, backend, and dashboard one common contract. Figure 2 shows how these components fit together: the extension extracts evidence from GitHub pages and sends it to the Express backend, which persists records in SQLite and serves the React dashboard used for review, labeling, and export.

The SQLite schema comprises roughly twenty tables covering projects, users, records, snapshots, comments, research-label categories and labels, annotations, notes, linked artifacts, fix evidence, record reviews, taxonomy and backfill state, consensus labels, update-check runs and change logs, quality checks, audit logs, and provenance. It enforces a project-scoped duplicate key over repository owner, repository name, record type, and number.

The storage layer emphasizes provenance and repeatability. Each collected record can hold multiple snapshots, and a snapshot contains raw text, optional raw HTML, normalized JSON, source method, capture time, and field provenance. The record itself stores stable identity fields: project, repository owner, repository name, record type, number, title, state, collector, collection time, refresh time, and update time. The project-scoped uniqueness constraint warns about duplicates without preventing the same GitHub issue from being studied in different projects.

All of these tables are included in JSON exports and project backups, so that a reviewer can inspect not only the final labels but also the evidence and intermediate decisions that produced them. The fix-evidence fields in particular are motivated by the long-running use of issue, change, and fix-link evidence in software evolution studies [12].

5 Usage Demonstration↩︎

This section demonstrates RepoTrace on a self-collected dataset rather than reporting a controlled user study. We collected 20 issues from matplotlib/matplotlib across two RepoTrace projects. The first, Matplotlib Backend and Rendering Issue Evidence Study, focuses on backend behavior, rendering defects, documentation rendering problems, animation behavior, maintenance discussions, and installation/security warnings. The second, Matplotlib Regression and Compatibility Evidence Study, focuses on regression and compatibility evidence; here we additionally inject four synthetic multi-reviewer conflicts to exercise the consensus workflow. These conflicts are not observations of real reviewer disagreement and serve only to drive the dashboard’s conflict-surfacing path. The demonstration is not intended to measure GitHub collaboration behavior broadly, but it exercises the kinds of social and technical information that prior GitHub studies draw on [2], [3].

For each record, we verified that RepoTrace preserved the GitHub URL, title, state, labels, issue body, captured comments, stored snapshot, screening rationale, note, fix evidence, and research annotations. Across the two projects, the validation notes report 20 records, 22 stored snapshots, 38 captured comments, 20 research notes, 98 research annotations, 20 screening reviews, 20 fix-evidence entries, and the four injected unresolved consensus conflicts; the artifact package includes the detailed record-by-record notes.

The results in Table 2 indicate that RepoTrace can preserve the main artifacts needed for a manually coded GitHub issue-labeling study. The backend/rendering project demonstrates the linear workflow of collection, screening, labeling, and evidence capture, while the regression/compatibility project demonstrates that the dashboard can surface disagreement cases, such as conflicting symptom or component labels, without overwriting either reviewer’s annotation.

The automated test suite comprises 37 passing tests covering extension URL detection and extraction fixtures, attribute selection, database initialization, backend collection and refresh APIs, duplicate handling, scoped exports, project backup, consensus and disagreement workflows, audit logs, and dashboard rendering. Among these are scoped export checks for single-record and filtered-record exports, and backup round-trip checks confirming that records, snapshots, comments, notes, annotations, and provenance are preserved.

6 Tool Availability↩︎

RepoTrace is open-sourced. The source code is publicly hosted at https://github.com/t3-research/RepoTrace, with an archived, citable snapshot at https://doi.org/10.5281/zenodo.20954131, and a demonstration screencast is available at https://youtu.be/ZEaeAzb2UkQ. The artifact bundles the full source, setup and reproducibility instructions, the automated test suite, seeded demo data, and validation notes.

The artifact runs locally and requires only Node.js, npm, SQLite, and a Chromium-based browser such as Chrome. After , a user can seed the demonstration database (), run the automated test suite (), and build the dashboard and extension (). The reproducibility instructions further cover API checks, project backup import/export, update checks, and JSON export inspection, and reproduce the validation dataset reported in Section 5.

7 Conclusion↩︎

RepoTrace demonstrates a browser-assisted workflow for constructing auditable GitHub research datasets. By storing page evidence, source metadata, research labels, notes, review decisions, refresh history, and exportable provenance in a single local workspace, the tool makes manual empirical data collection easier to inspect and reproduce. It supports the full local workflow—collection, labeling, refresh, consensus review, backup, and JSON export—and the 20-record Matplotlib validation shows that it can preserve both the source evidence and the research interpretation for later review. We plan to extend RepoTrace with CSV and analysis-table export, tighter navigation from labels to the exact page evidence, richer multi-reviewer workflows, and dataset versioning. Any optional AI assistance will be added only once the non-AI evidence workflow remains reliable.

References↩︎