1 Introduction↩︎

Scratch supports computing education through projects that learners create, share, inspect, and remix. In constructionist and creative-learning traditions, learners build personally meaningful artifacts and learn through peers, feedback, and reflection [1]–[3]. Project-based learning similarly treats meaningful artifacts as a basis for motivation and understanding when teachers, peers, and tools provide usable scaffolding [4], [5]. Scratch makes these ideas concrete in a public online community where young people publish interactive media and learn from projects made by others [6]–[9].

Public sharing makes project selection a student-facing educational decision. Teachers may choose public projects as worked examples, ask students to browse for inspiration, or use remixing in project-based activities, but students are often the eventual users: they browse, play, inspect, and remix projects during learning. Researchers may also collect public projects for datasets about novice programming, debugging, creativity, assessment, or computational thinking. In each pathway, a Scratch project is a learner-facing material, not only a code artifact.

We use curation to mean selecting and documenting public projects before learners encounter them as classroom examples, open-exploration materials, remix sources, or dataset entries. Curation often begins with the project page. Titles, thumbnails, instructions, descriptions, tags, social signals, and remix links help educators and dataset builders decide which resources deserve attention. Work on digital learning-object repositories and teacher resource exchanges shows that metadata and evaluative cues shape discovery, judgment, and reuse [10], [11]. Scratch pages provide similar cues, but the executed project can contain evidence that the page never shows. The central screening question is whether the curator has enough evidence about the learner’s experience, not whether the page surface contains an obvious warning sign.

We call this phenomenon runtime-revealed sensitive content: sensitive or age-inappropriate content that appears only after execution, gameplay progression, a failure state, a timed delay, user input, costume switching, audio playback, or a hidden event trigger. A project page can look benign while normal play reveals a jump scare, an intense sound, a threatening scene, or hidden content. Some learners may also deliberately seek scary or edgy projects and share them with peers, so curation should support guidance rather than assume exposure is always accidental. A broad theme label such as horror or combat is also insufficient because classroom appropriateness depends on severity, presentation, evidence location, reveal pathway, and likely learner encounter.

This distinction matters first for students’ learning experience, and then for the educators and researchers who structure that experience. Scratch community guidelines, reporting, moderation, and digital-citizenship resources provide a platform-level safety layer [12], [13]. Educators still need to judge whether a project fits a particular age group, assignment, and classroom norm. Dataset builders also need to report what screening examined and what evidence supports labels or exclusions. Metadata-only review leaves those decisions under-specified when the relevant signal appears during execution.

This paper presents a runtime-aware audit of 500 public Scratch projects. The audit characterizes how safety-relevant content is encoded, how it becomes visible or audible, and what evidence is needed to annotate it for classroom reuse, student exploration, remix contexts, and dataset screening. The sample combines curated candidates, taxonomy-guided keyword search, and follow-up exploration of project clusters surfaced during review. For each project, annotators inspected metadata, executed the project, performed standard interactions, and recorded whether the primary safety-relevant signal was visible statically or required runtime exploration, interaction, gameplay progression, failure states, or code/asset inspection.

We organize the study around three research questions:

RQ1:

What forms of safety-relevant content appear in targeted public Scratch project pathways used to stress-test classroom, student-exploration, remix, and dataset curation?

RQ2:

Which safety-relevant signals are visible from project metadata, and which require execution, interaction, gameplay progression, failure states, or code/asset inspection?

RQ3:

What category boundaries, evidence channels, and reveal mechanisms make Scratch project curation difficult for educators and researchers?

The remainder of the paper motivates the problem with concrete examples, situates it in related work on Scratch, educational resource selection, youth safety, and interactive media classification, then presents the annotation scheme, audit method, findings, and implications for runtime-aware classroom and dataset curation.

2 Background and Motivating Examples↩︎

Scratch projects sit at the intersection of programming and media. A single project can combine code, sprites, costumes, sounds, text, animation, and user interaction. It can also be shared, remixed, collected in studios, and discovered by learners browsing a public community. In this paper, content safety means whether that executable experience is appropriate for a particular learning activity, age group, and classroom norm; it is not a claim that Scratch itself is unsafe. We therefore treat Scratch project review as a runtime-aware educational curation problem where the relevant artifact is an executable experience that learners may play, inspect, remix, and encounter through peers.

We group several motivating examples below to make this issue concrete for computing-education readers. The examples illustrate why unexpected runtime content can matter in a learning setting, while remaining separate from any claim about platform-wide prevalence. They also show why content-safety analysis cannot stop at titles, thumbnails, or descriptions: Scratch projects unfold through progression, timing, interaction, failure states, remix context, audio playback, and hidden code paths.

2.0.0.1 Benign-framed maze and jump-scare projects.

For example, our motivating set included a maze game and a cheese-collection game whose page surfaces did not clearly announce the later scare. In the maze game, the learner guides a small object through simple levels, and the opening state gives little reason to expect an intense scare sequence. After gameplay progression, a runtime event switches away from the maze, shows a sudden visual change, plays a scream, and resumes the game. In the cheese-collection game, ordinary play can lead to a failure state that changes the backdrop to a startling face image, flashes the screen, hides the sprites briefly, and plays a loud scream. In both projects, the learner reaches the reveal by doing what the project asks them to do. The risky moment is produced by normal play rather than announced by the page surface.

2.0.0.2 FNAF-inspired horror projects.

Scratch also contains Five Nights at Freddy’s style fan games that appear through studios, remix chains, and genre collections; ESRB describes the commercial series as puzzle/horror games with frequent screams and jump scares [14]. These clusters show that curation evidence can extend beyond a single project page. A mild fan game, a tense survival game, and a jump-scare-heavy remake can share a franchise label while creating different classroom concerns. Risk depends on intensity, presentation, audio, progression, and runtime behavior.

2.0.0.3 War and tower-defense projects.

Public remix pages also include strategy or tower-defense games with stylized combat mechanics, as well as war-themed projects whose titles reference military conflict, tanks, or defense. Tower-defense mechanics are not themselves the concern. These projects motivate the distinction between content category and risk level: stylized or fantasy conflict can be low risk, while explicit threats, graphic injury, hate framing, or realistic gore can raise the risk level [15], [16]. Treating all combat-themed projects as equally unsafe overstates risk and obscures differences that matter for classroom curation.

2.0.0.4 School-horror and Baldi-style remix clusters.

Scratch includes remix clusters around Baldi-style games, soundboards, and school-horror projects; the official Baldi’s Basics app description frames the game as meta horror set in a school-like environment [17]. These cases combine school settings, horror framing, chase mechanics, hostile characters, sounds, and bullying-like language. The annotation challenge is to distinguish fictional school-horror mechanics from harassment or targeted bullying, especially when the evidence appears only through play, interaction, or audio.

Together, these cases motivate three annotation choices. Categories need to be multi-label because projects can combine horror, violence, profanity, harassment, and other themes. Risk levels need to remain separate from categories because themes such as combat or horror can be appropriate in some contexts. Annotations also need to record evidence channel and reveal mechanism because a teacher or dataset builder needs to know not only what content exists, but how a learner would encounter it: in metadata, runtime behavior, interaction, remix context, audio, or hidden code paths.

3 Related Work↩︎

Scratch has been studied as both a block-based programming environment and a public learning community. Constructionist and creative-learning work emphasizes meaningful artifacts, sharing, remixing, and reflection [1], [3]. Community-level studies examine public activity, remix pathways, and informal learning from peer-produced projects [7]–[9], [18], [19]. This literature makes the exposure scenario in our study educationally plausible: learners encounter public Scratch projects through browsing, inspiration, reuse, remix, and classroom activity. Public projects operate as examples, play materials, code resources, and social artifacts at the same time. A curation framework for Scratch must account for all of these roles because a learner may experience the project before, during, or after reading its code.

Formal resource selection often happens through teachers, who rely on repositories, previews, descriptions, ratings, and metadata [20]–[23]. Metadata quality shapes discovery and reuse in learning-object repositories [10], and evaluative metadata influences which resources teachers download and use [11]. Students, however, may also encounter resources less systematically through browsing, studios, remix links, or peer sharing. Scratch pages provide comparable surface cues, useful for teacher triage and often the only cues a learner sees before pressing the green flag. The curation task changes when the resource is an executable program: a preview is incomplete unless it records what was actually run, heard, clicked, failed, or inspected.

Computing-education research has developed tools for analyzing Scratch programs [24]–[27]. Static and structural systems inspect projects for computational-thinking constructs, misconceptions, bugs, and code smells [28], [29]. Runtime-oriented systems use execution and tests to support debugging and functional feedback [30]–[34]. These systems show that Scratch code and runtime behavior can be instrumented for learning research. They also clarify the boundary of our contribution: the problem is not whether Scratch can be executed or analyzed, but which runtime evidence is needed when the target judgment concerns age appropriateness and educational reuse. Our work uses runtime evidence for that curation task: documenting whether public executable projects are appropriate for classroom reuse, open exploration, remix support, or dataset construction.

Youth online safety research has examined harassment, bullying, hate speech, privacy leakage, self-harm exposure, inappropriate content, and platform governance [35], [36]. Work on youth-facing game and user-generated-content platforms shows that risks can appear through play, social interaction, promotional imagery, and moderation-evasive behavior [37]. Scratch addresses safety through community guidelines, moderation, reporting, and digital-citizenship resources [12], [13]. Educational curation adds a classroom and research layer: a project can be unsuitable for a specific age group, assignment, or open-exploration activity even when similar themes remain appropriate elsewhere.

Game and interactive-media research treats user experience as emerging from rules, mechanics, timing, progression, and audiovisual presentation [38]. Age-rating systems such as ESRB and PEGI distinguish age categories, content descriptors, interactive elements, and contextual severity [15], [16], [39]. ESRB’s process for physical games asks publishers to disclose relevant content and provide video of typical gameplay and extreme content [39]. This supports our central premise: interactive media require evidence about what a user can encounter during play. Scratch differs because projects are youth-generated, continuously edited, remixed, and discovered through a public learning community.

Prior work establishes Scratch as a public learning community [2], studies metadata for educational selection, instruments Scratch execution [40], and examines exposure context in interactive media. Missing is a framework for Scratch projects as executable learning materials whose appropriateness depends on runtime pathways invisible to metadata-only review. Safety-design work favors agency-preserving, collaborative guidance for minors [41]. This gap motivates our runtime-aware annotation scheme and audit of 500 public Scratch projects.

4 Taxonomy and Annotation Scheme↩︎

We developed a runtime-aware annotation scheme for public Scratch projects. The scheme describes five aspects: what safety-relevant content appears, how severe it is, where the evidence is observed, how the content is revealed, and how strongly the label is supported. It is designed for educational curation and research documentation; moderation decisions remain a platform-governance matter.

The scheme was developed in two passes. First, we translated content axes from youth online-safety work, Scratch community guidance, and interactive-media rating systems into candidate category and severity labels [12], [15], [16], [42]. Second, during pilot mapping of curated Scratch projects, we refined the operational dimensions whenever the relevant cue depended on running the project, hearing audio, triggering an event, or inspecting assets and code. This process led us to separate content category, risk level, evidence channel, reveal mechanism, and confidence instead of using a single appropriateness label.

4.1 Content Categories↩︎

Content categories are multi-label because Scratch projects often combine horror, violence, harassment, social or political content, and moderation-conflict signals. Table 1 summarizes the nine categories used in this study. Categories C1–C8 follow common youth-safety axes; C9 captures Scratch-specific signals in which a project discourages reporting, challenges moderation, or reposts contested material. Thumbnail/runtime mismatch is captured separately as reveal mechanism M7 (Table ¿tbl:tab:idcatalog?).

3pt

Category labels describe content type; risk levels describe severity. A stylized tower-defense game and a graphic violent scene both involve violence with different risks. Political or societal content is labeled for contextual sensitivity, not high risk by default.

4.2 Risk, Evidence, Reveal, and Confidence↩︎

Beyond category labels, each annotation records four operational dimensions, summarized in Table 2. The two multi-label dimensions, evidence channel and reveal mechanism, use identifiers (E0–E6 and M0–M8) catalogued in Table ¿tbl:tab:idcatalog?. These dimensions separate content type from severity and record whether evidence comes from metadata, assets, runtime behavior, interaction, public context, or code-level inspection.

3pt

3pt

4.3 Boundary Rules↩︎

Boundary rules reduce over-labeling. Categories are assigned from observable evidence rather than inferred creator intent. Stylized conflict is distinguished from graphic violence. Harassment requires targeting an identifiable user, creator, or group. Political or societal themes are labeled for contextual sensitivity but do not imply high risk. When evidence is ambiguous, annotators choose the lowest risk level supported by the evidence and record uncertainty.

5 Study Design↩︎

We conducted a seed-based audit of 500 public Scratch projects. The goal was to test whether runtime-revealed sensitive content can be categorized and annotated at scale, not to estimate platform-wide prevalence. We intentionally sampled projects likely to contain sensitive, borderline, or moderation-relevant content so the annotation scheme would encounter difficult cases across the taxonomy.

5.1 Seed Set↩︎

Candidate projects came from three sources (Table 3). First, a curated pool of 50 projects from prior taxonomy mapping supplied examples spanning C1–C9. Second, targeted keyword search against public Scratch search pathways supplied candidates from terms derived from the nine-category taxonomy, such as “shooter,” “stick war,” “FNAF,” “Granny,” “scary maze,” “stop bullying,” “stand with Ukraine,” and “don’t report” [6]. Third, follow-up exploration sampled two high-density public project clusters surfaced during early review, with repeated C9 moderation-conflict and C6-like harassment signals. Across sources we screened a larger candidate pool (\(N \approx 1{,}458\) projects fetched and pre-classified) and retained 500 final entries after runtime processing, franchise-clone deduplication, and visual review.

3pt

5.2 Annotation Procedure↩︎

We used an LLM-assisted first-pass annotation pipeline followed by human-supervised checkpoint review and final adjudication. For each project, a headless scratch-vm harness executed the project with a green flag, a passive runtime window, and a synthesized interaction timeline that fired declared hat triggers plus a fixed navigation set [43]. A multimodal LLM then reviewed three rendered keyframes (start, middle, end), project metadata, and a structured trace summary. The reviewer produced a draft annotation along the five dimensions in Section 4, applied the boundary rules in Section 4.3, and flagged uncertain or high-impact cases for human review. When frames were uninformative, the reviewer fell back to metadata-only classification and lowered confidence.

A lead researcher supervised the audit through periodic checkpoint reviews. The supervisor was the project lead who created the codebook and completed the prior taxonomy-mapping pass used to form the curated seed pool; this role provided continuity for adjudication but is not a substitute for independent reliability measurement. At each checkpoint, the supervisor inspected representative drafts, identified systematic rule failures, and folded corrected rules back into the reviewer prompt. Examples included red-pixel signals caused by Pride flags or political-party banners, and anti-bullying advocacy mislabeled as harassment. At the final checkpoint, the supervisor adjudicated all needs_review and escalation flags and re-examined high-impact category shifts, especially C9 upgrades and M7 thumbnail-mismatch flags. Final labels reflect supervised adjudication; independent dual annotation remains future work.

5.3 Runtime Depth and Analysis↩︎

To compare static and runtime evidence, we recorded the deepest level of exploration required to observe the primary safety-relevant signal. D0 indicates visibility from metadata; D1, visibility immediately after opening or starting; D2, appearance during passive runtime; D3, interaction, gameplay progression, or a failure state; and D4, source-code inspection, hidden asset inspection, or non-obvious trigger discovery. We summarize categories and risk levels, compare evidence channels and runtime depths, and analyze boundary disagreements such as horror versus violence, stylized combat versus gore, anti-awareness versus targeted harm, and political sensitivity versus high-risk content.

6 Findings↩︎

The findings characterize evidence patterns in a targeted seed set that oversamples sensitive and borderline projects. The counts describe curation challenges in these pathways.

6.1 Categories and Risk Were Related but Distinct↩︎

The seed set contained horror, combat, gore, harassment, political themes, moderation-conflict signals, and possible high-risk content. Table ¿tbl:tab:category-risk-results? summarizes the observed category and risk distributions; categories are multi-label, so counts can exceed the number of projects. Category membership did not determine severity. Horror projects sometimes included violent threat framing or stylized blood-spatter assets. Combat projects ranged from cartoon mechanics to explicit gore. School-themed horror sometimes blurred fictional threat and targeted bullying. The C9 “don’t report” cue often co-occurred with C2 horror or C6 harassment. These patterns support separating category from risk level.

3pt

6.2 Runtime Inspection Revealed Signals Missed by Static Review↩︎

Exploration-depth and reveal-mechanism annotations provide the main evidence for the runtime-aware framing. Some signals were visible from metadata; many required execution, interaction, gameplay progression, failure states, or code and asset inspection. Table ¿tbl:tab:runtime-results? summarizes these observations.

3pt

Only 33 of 500 projects (7%) were resolvable from static metadata at depth D0. The remaining 467 (93%) required runtime exploration, and 387 (77%) required interaction, gameplay progression, failure states, or hidden-asset/code inspection. The D3 and D4 cases show why project-page review is incomplete: a project can look benign under static inspection and reveal safety-relevant content through interaction, hidden assets, or event-triggered behavior. The 28 M7 cases further illustrate the gap: thumbnails or first frames diverged from later runtime behavior. Many runtime-revealed projects remain low risk, so runtime evidence supports calibrated curation instead of blanket exclusion.

Exploration depth also changes the feasibility of review. D1 and D2 evidence can often be surfaced by a short preview run. D3 evidence requires a reviewer or tool to try plausible learner actions, including clicks, movement, and failure states. D4 evidence requires asset or source inspection when the runtime path is unclear. These distinctions help educators and dataset builders report how much confidence their screening can support. M3 and M8 remain in the scheme, although no project in this audit was adjudicated as primarily timed/delayed or source-only/asset-only; observed delays and latent assets co-occurred with stronger mechanisms.

6.3 Disagreements Were Systematic↩︎

Annotator disagreements clustered around predictable boundaries. Horror versus violence: monster-chase sequences were labeled C2 when fear and survival pressure dominated, and C1 when harm framing was explicit. Stylized combat versus gore: red visual effects required runtime observation because intensity and audio context determined whether an effect read as stylized or graphic; rendered keyframes separated blood-related evidence from Pride flags, political-party banners, and Halloween title cards. Anti-awareness versus targeted harm: anti-bullying and anti-racism advocacy projects were labeled C8 for societal sensitivity, while C5 and C6 were reserved for content that targeted an identifiable group or user. Contextual versus runtime evidence: studio membership, remix-chain origin, and project-cluster context raised annotator attention, but category and risk assignment still required direct observation.

These disagreements identify where operational definitions matter most. A collapsed taxonomy would hide the distinctions that runtime-aware annotation is designed to capture.

7 Discussion, Ethics, and Limitations↩︎

7.1 Discussion↩︎

Our 500-project audit frames sensitive-content screening in Scratch as runtime-aware educational curation for educators and computing education researchers preparing student-facing examples, browsing activities, remix materials, and datasets. In this targeted corpus, project-page screening left the primary safety-relevant signal unresolved for 467 projects (93%); 387 projects (77%) required interaction or deeper inspection. The practical implication is a screening-depth norm: curation records and dataset documentation should report whether review stopped at metadata, ran the project, tested interactions and failure states, or inspected hidden assets and code. The contribution is a vocabulary for runtime evidence and empirical support that static-only review is inadequate in this targeted corpus.

These evidence dimensions support several pathways. Teacher-selected examples need previewable runtime evidence; student-directed browsing needs guidance, safe exits, and shared norms; datasets need screening-depth provenance. For classroom use, a short workflow can record page cues, tested runtime paths, interactions, sounds, hidden costumes, and confidence. The goal is calibrated reuse: a mild horror project may fit an older elective but be unsuitable for open-ended browsing by younger students. When open exploration is part of the activity, runtime-aware curation can also teach learners to pause, exit, report, or ask for help after an unexpected audio or visual reveal.

For datasets and future tools, screening depth should be part of provenance: a dataset filtered only through titles and thumbnails cannot guarantee that runtime-revealed content was absent. A runtime preview assistant could run a project, exercise common inputs, surface hidden assets, flag thumbnail/runtime mismatches, and present evidence cards showing where a signal appeared. Teachers and researchers should remain in the decision loop because appropriateness depends on age group, assignment, and classroom norms; this aligns with youth-safety work emphasizing agency and collaborative guidance for minors [41]. Follow-up studies should examine how teachers and students respond to runtime evidence during selection, browsing, and remixing.

7.2 Ethics and Limitations↩︎

Scratch is a youth-oriented platform, so we use a conservative approach to sensitive material. The analysis is restricted to publicly accessible projects. We avoid identifying or contacting creators, reporting usernames, and reproducing explicit imagery, personal information, or harmful text. Severe cases are described only in paraphrase and with minimal detail, and exploration halted upon encountering explicit sexual content, extreme gore, severe self-harm material, or real personal information.

Several limitations bound this work. The seed set deliberately oversamples sensitive and borderline projects; observed distributions are not prevalence estimates. Runtime exploration is time-bounded, so content gated behind extended gameplay or non-obvious interactions may remain undiscovered. Annotation requires judgment at category boundaries; the internal lead-researcher adjudication improves consistency but does not provide independent inter-rater reliability. This audit also leaves public-context evidence (E6), such as forum posts or moderation history, for future work. Because Scratch projects can be edited, removed, remixed, or recontextualized after collection, future studies should report sampling strategy, screening depth, and annotation confidence.

8 Conclusion↩︎

This paper identifies runtime-revealed sensitive content as a computing education curation challenge for learner-facing Scratch reuse and dataset construction. Scratch projects are executable programs whose safety-relevant content can appear through interaction, timing, gameplay progression, failure states, hidden assets, or event triggers. We introduced a runtime-aware annotation scheme and used it to audit 500 public Scratch projects. The study shows why static project-page review is incomplete for a targeted corpus of executable youth media and motivates future work on classroom project selection, transparent dataset construction, and educator-facing screening tools.

References↩︎