1 Introduction↩︎

Large Language Models (LLMs) are increasingly used to generate educational content, ranging from tutoring support to full-length explanatory lessons [1]–[3]. A particularly emerging task is lesson generation: given a structured, sufficiently detailed lesson plan specifying topics, learning objectives, and preferred pedagogical delivery, an automated AI system produces a complete lesson. However, evaluating the quality of such AI-generated lessons against human expert standards remains an open challenge [3], and there exists no standardised benchmark pairs structured lesson plans with corresponding human-written lessons for systematic evaluation.

Existing educational datasets such as MathDial [2], TutorEval [4], and EduChat [1] address similar issues. TheoremExplainBench [5] provides 240 STEM topic descriptions with difficulty ratings, but reference lessons are not provided. Open educational resources such as LibreTexts [6] offer textbook chapters and other high-quality articles, but they lack structured paired lesson plans. These datasets do not capture the full plan-to-lesson pipeline that an AI generation agent would follow.

Addressing these issues, this short paper presents the following two key contributions: 1) LessonBench-V1: The first open-source¹ benchmark dataset pairing structured lesson plans with expert-written lessons across 240 STEM topics and four subjects, comprising 647 paired samples from 97 unique educational sources. The dataset is constructed using a reproducible, multi-stage data curation pipeline in collecting, cleaning, structuring, and reviewing educational content from web sources. 2) A pedagogical research-grounded approach to reverse-engineering lesson plans, incorporating Bloom’s Taxonomy [7], Gagné’s Events [8], Merrill’s First Principles [9], and the 5E Model [10] to extract detailed, structured instructional blueprints from existing lessons.

2 Related Work↩︎

2.0.0.1 Educational AI Benchmarks.

Several benchmarks evaluate AI in educational settings, but none cover the full plan-to-lesson pipeline, as shown in Table ¿tbl:tbl:comparison?. MathDial [2] provides 3,000 teacher-student dialogues for tutoring evaluation, TutorEval [4] evaluates LLMs for science tutoring, and EduChat [1] focuses on educational chatbot design. TheoremExplainBench [5] provides 240 STEM topics that form the foundation of this work, but contains no reference lessons.

2.0.0.2 Instructional Design Frameworks.

The lesson plans in LessonBench-V1 were constructed by incorporating four established pedagogical frameworks. Bloom’s revised Taxonomy [7] provides six cognitive levels for tagging learning objectives. Gagné’s Nine Events of Instruction [8] informs the instructional flow, together with Merrill’s First Principles [9] and the BSCS 5E Model [10]. LessonBench lesson plans unify these into six instructional roles: Motivation, Activation, Demonstration, Application, Integration and Assessment, which holistically cover the instructional flow.

2.0.0.3 LLMs for Educational Content.

Recent work has explored LLMs for the generation of educational materials [3]. However, most evaluations rely on ad hoc human judgement or LLM-as-judge approaches [11] rather than on reproducible, reference-based benchmarking. LessonBench-V1 aims to provide this missing reference dataset to enable systematic and reproducible evaluation.

3 Dataset Construction↩︎

3.1 Source Data and Topic Selection↩︎

Educational topics chosen for this study were drawn from TheoremExplainBench [5], which includes 240 topics across Mathematics, Physics, Chemistry, and Computer Science, categorised into three difficulty levels: Easy, Medium, Hard. For each topic, LessonBench-V1 contains 1-5 human-written lessons from various educational sources, totalling 647 lessons. Each lesson is accompanied by a corresponding lesson plan. Source selection prioritised reliable, openly accessible, and pedagogically structured content. The 97 source websites span open textbook platforms such as LibreTexts (47%), which include resources from a variety of high-quality sources, as well as educational portals such as GeeksForGeeks (9%), Byjus (8%), and Brilliant.org (5%).

3.2 Multi-Stage Curation Pipeline↩︎

1. Web Data Collection: Relevant sources, including related images, are collected and downloaded.

2. HTML-to-Markdown Conversion: The downloaded HTML content is converted to Markdown using the html-to-markdown Python library.

3. LLM-based Markdown Cleanup: The resulting Markdown often contains unwanted web-scraping artefacts, such as navigational text, broken LaTeX equations, and unrelated web-page content. A carefully designed prompt ² is used with Qwen 3.6 35B A3B to fix formatting issues across ten identified categories while preserving the relevant educational content verbatim, with no rewriting, summarising, or additional content.

4. Lesson Plan Reverse Engineering: A pedagogically research-backed prompt ³ was used with GPT-4.1, accessed via the GitHub Copilot API, to reverse engineer a structured lesson from each cleaned lesson. The prompt is designed to holistically ‘reverse engineer’ the instructional blueprint without summarising content. The resulting plan captures six sections: Metadata, Learning Objectives, Key Concepts and Essential Questions, Instructional Flow, Practice and Application Strategy, and Pedagogical Strategy. Each lesson plan⁴ follows a structured template grounded in four pedagogical frameworks. Learning objectives use Bloom’s revised taxonomy verbs [7] and are tagged to one of the six cognitive levels. Instructional flow phases are annotated with roles derived from cross-mapping Gagné’s Events [8], Merrill’s Principles [9], and the 5E model [10] into six refined roles: Motivation, Activation, Demonstration, Application, Integration, and Assessment.

5. Human Review: The cleaned lesson and the created lesson plan are then reviewed to identify any further formatting or structural issues, which are then fixed manually.

4 Dataset Analysis↩︎

Table 2 summarises key statistics for LessonBench-V1. The dataset comprises 647 pairs of lessons and lesson plans across 240 topics. Human-written lessons average 1,744 words, whereas lesson plans are substantially shorter, averaging 938 words.

5pt

4.0.0.1 Bloom’s Taxonomy Distribution.

Across all 647 lessons, 3,620 learning objectives were extracted, each assigned a Bloom’s taxonomy level, as shown in Figure 2.

4.0.0.2 Instructional Role Distribution.

The distribution of the 5,623 parsed instructional role annotations is shown in Figure 2. Subject-specific patterns are evident in the pedagogical metadata. Mathematics lessons have the highest proportion of Apply-level objectives, reflecting an emphasis on procedural problem-solving. Physics lessons show the highest proportion of ‘Understand’ objectives, consistent with conceptual explanation being central to physics pedagogy. Computer Science lessons have the most balanced Bloom’s distribution, with notable Analyse-level content focused on algorithm analysis.

5 Benchmarking Framework↩︎

LessonBench-V1 supports the systematic evaluation of AI lesson-generation systems that take a lesson plan and produce a complete lesson from it. Given a lesson plan \(P_i\) from the dataset, an AI system generates a lesson \(\hat{L}_i\), which is then compared with the corresponding human-written reference \(L_i\). This mirrors the real-world use case in which a curriculum designer writes a plan and an AI agent produces the lesson.

LessonBench can be used across three complementary evaluation dimensions: 1) Semantic similarity via BERTScore [12], measuring contextual overlap; 2) Structural similarity via ROUGE-L [13], capturing the longest common subsequence overlap; and 3) Pedagogical alignment by reverse-engineering a lesson plan from the generated lesson and comparing Bloom’s level coverage and role distributions with the original plan. Furthermore, the datasets’ metadata, derived from TheoremExplainBench, enables systematic analysis of these dimensions across aspects such as subject and difficulty.

6 Conclusion and Future Work↩︎

This short paper presents LessonBench-V1, a benchmark dataset that pairs structured lesson plans with expert-written lessons across 240 topics spanning four STEM subjects, resulting in 647 paired samples. The dataset is collected and constructed using a reproducible five-stage pipeline. Our initial analysis shows meaningful patterns across subjects and difficulty levels in the collected lessons, affirming its validity.

6.0.0.1 Limitations and Future Work.

The lesson plans for the dataset were generated using GPT 4.1. Although the generated plans were reviewed prior to the final compilation of the dataset, we plan to conduct systematic experiments to statistically verify their completeness and accuracy against an expert-written lesson plan baseline. The dataset is currently drawn only from English-language sources, and future versions will include lessons from multiple languages. Furthermore, this dataset is an initial step in a larger research effort to develop a full end-to-end lesson-generation AI agent, which is planned to be evaluated on it.

References↩︎