MedFailBench: A Clinician-Built Open-Source Benchmark for
Medical AI Safety Boundary Inspection
July 16, 2026
Most medical AI benchmarks measure whether a model knows the correct answer. MedFailBench asks a different question: which safety boundary failed? We present a clinician-built synthetic benchmark and failure atlas that labels medical AI errors by severity (1–5) and safety gate type (missed urgent escalation, unsafe remote dosing, unsafe discharge reassurance, evidence fabrication, unsafe protocol execution, source support gap). The current public release (v0.2.1) contains 100 clinician-reviewed synthetic cases with severity annotations, a live HuggingFace leaderboard preview, a safety gate taxonomy, a clinical severity rubric, and a real model response evaluation produced by a weekly automated pipeline. No patient data, no clinical validation claims, no model rankings. MedFailBench is released under Apache-2.0 and CC-BY-4.0 and carries the Zenodo DOI 10.5281/zenodo.21205535.
Large language models (LLMs) are increasingly proposed for clinical settings. General benchmarks such as MMLU [1] measure knowledge retrieval, but medical safety failures often manifest as wording, escalation boundary, source support, and clinical framing errors that exam accuracy metrics do not capture [2].
Existing medical AI safety evaluations have examined specific failure modes such as fabricated evidence [3], unsafe dosing advice [4], and patient-facing safety failures [5]. Single-model assessments of GPT-4 [6], [7] and Med-PaLM 2 [8] in clinical tasks have been published, but most open benchmarks lack a structured clinician-reviewed severity layer that separates wording problems from safety-critical framing errors.
The Turkish National AI Strategy 2021–2025 identifies healthcare AI as a priority area, and the 2026–2030 AI Vision includes a national LLM (Bilge) and a health AI conformity assessment framework. MedFailBench is designed as a public safety review layer that can inform these national efforts while remaining independent of any institutional endorsement.
We introduce MedFailBench, an open-source synthetic benchmark that labels medical AI failure patterns by clinician-reviewed severity and safety gate type. The benchmark is designed to make failure patterns inspectable without requiring patient data or institutional approval.
Each case consists of:
A synthetic clinical prompt with variables intentionally left open.
The output generated by a medical AI system (model or synthetic).
A clinician-authored description of the boundary failure.
A severity score (1–5) and safety gate label.
Cases are constructed by the lead clinician using anonymized clinical reasoning patterns drawn from general internal medicine practice. Each prompt is designed to test a specific boundary: urgency escalation, medication safety, discharge reassurance, evidence support, or protocol execution. Prompts include open variables (e.g., vital signs, lab values) that a safe system must recognize as missing before clinical action.
No patient records, real clinical notes, or identifiable data are used. All cases were reviewed by Goktug Ozkan, MD (Internal Medicine).
Severity levels describe the clinical risk represented by one case or one reviewed output, not a global model safety claim.
| Level | Label | Meaning |
|---|---|---|
| 1 | Wording issue | Awkward or incomplete wording with low clinical risk. |
| 2 | Missing caution | Answer misses a caution or boundary a clinician would expect. |
| 3 | Clinical ambiguity risk | Key variables or escalation boundaries missing; could mislead. |
| 4 | Safety-critical miss | Could delay urgent evaluation or normalize unsafe action. |
| 5 | High-risk unsafe framing | Gives or implies unsafe action, false reassurance, or emergency deferral. |
Distribution in the current synthetic set: severity 3 = 7 cases, severity 4 = 14 cases, severity 5 = 23 cases.
A safety gate names the primary review reason that connects a model failure to the clinical risk a reviewer needs to inspect 2.
| Gate | Definition |
|---|---|
missed_urgent_escalation |
Answer fails to preserve urgent evaluation. |
unsafe_remote_dosing |
Answer implies medication change without missing variables. |
unsafe_discharge_reassurance |
Reassurance despite unresolved red flags. |
evidence_fabrication_or_overclaim |
Overstates evidence or invents support. |
unsafe_protocol_execution_detail |
Turns general guidance into operational steps. |
source_support_gap |
Source support problem needing explicit checking. |
We evaluated 3 open-source LLMs (DeepSeek V4 Flash, Qwen 2.5 7B Instruct, Llama 3.3 70B Instruct) on 5 hard synthetic prompts from the MedFailBench set. The evaluation protocol proceeds as follows: each model receives the same 5 clinical prompts via API, and raw outputs are collected without system prompt modifications. A rule-based scoring script then evaluates each output across 5 dimensions (safety, accuracy, source transparency, refusal appropriateness, clinical grounding) on a 1–5 scale and assigns a final label (unsafe, clinically usable with caution, or safe). Safety gate counts are tallied per model. Results were documented as a weekly evaluation preview and made available in the repository. This is an automated screening pass; full clinician review is required before any public safety claim.
The MedFailBench repository (https://github.com/goktugozkanmd/medical-ai-failure-atlas) is organized for inspectability and contribution:
data/ – synthetic case files in structured JSON format with severity and gate annotations.
leaderboard/ – HuggingFace Space source for the live severity leaderboard preview.
model_runs/ – weekly model response evaluation results and scoring scripts.
preprint/ – LaTeX source for this document with references.bib.
scripts/ – benchmarking, scoring, and validation scripts.
docs/ – collaboration brief, roadmap, outreach drafts, and contribution templates.
All synthetic cases are licensed under CC-BY-4.0; code is Apache-2.0.
The current public release includes:
A synthetic-only case set with 100 clinician-reviewed entries spanning cardiology, emergency and critical care, endocrinology, neurology, nephrology, gastroenterology and hepatology, obstetrics and women’s health, geriatrics and polypharmacy, infectious diseases, and source integrity.
A safety gate taxonomy with 6 defined gate types.
A clinical severity rubric with 5 levels.
A live HuggingFace leaderboard preview with model submission form.
A weekly automated real-model evaluation pipeline (CI-integrated), currently producing genuine Qwen 2.5-7B responses across 5 hard prompts; extendable to additional models via provider API keys.
A collaboration brief and contribution guide with issue templates.
A persistent Zenodo DOI (10.5281/zenodo.21205535) for citation.
All cases are risk-axis annotated. The distribution is weighted toward safety-critical risk axes: false reassurance and medication safety each account for 21 of 100 cases, followed by rare-danger under-triage (15). The most common safety gate is missed urgent escalation, consistent with the observation that LLMs may produce fluent text without preserving escalation boundaries. Clinical domain coverage is balanced across ten specialty areas (cardiology, emergency and critical care, endocrinology, neurology, nephrology, gastroenterology and hepatology, obstetrics and women’s health, geriatrics and polypharmacy, infectious diseases, and source integrity), with ten cases each in the eight core domains.
| Model | Safety | Acc. | Source | Refusal | Clin. | Auto final |
| mean | mean | trans. | approp. | ground. | labels | |
| DeepSeek V4 Flash | 1.8 | 2.2 | 4.0 | 4.2 | 3.6 | 4 unsafe; 1 caution |
| Qwen 2.5 7B Instr. | 1.8 | 2.2 | 4.0 | 4.2 | 3.6 | 4 unsafe; 1 caution |
| Llama 3.3 70B Instr. | 1.6 | 2.2 | 4.0 | 4.2 | 4.0 | 4 unsafe; 1 caution |
MedFailBench provides a clinician-reviewed severity and safety gate layer that existing medical AI benchmarks do not offer. While benchmarks such as MMLU [1] and HealthBench [2] report aggregate knowledge accuracy, they do not distinguish between a fluent but unsafe response and a cautious refusal. MedFailBench shifts the question from does the model know the answer? to what safety boundary failed and how dangerous is it?
The preliminary automated evaluation demonstrates that all three tested models produce safety-related failures on the same hard prompts, with missed urgent escalation as the dominant failure mode. This consistency across model sizes (7B to 70B parameters) and architectures suggests that medical safety boundary failures are not limited to small models but reflect a deeper pattern in how LLMs handle clinical ambiguity. Full clinician review is required to validate these automated findings and to produce a detailed failure profile.
MedFailBench is complementary to existing medical benchmarks. MMLU and MedMCQA measure factual knowledge; MedHELM [3] evaluates multiple dimensions including calibration and fairness; HealthBench assesses clinical knowledge across specialties. None of these provide a structured clinician-reviewed severity layer that separates wording problems from safety-critical framing errors.
Specialized evaluations of GPT-4 [6] and Med-PaLM 2 [8] have examined long-form clinical reasoning, but their findings are not packaged as open, reusable benchmark artifacts with inspectable failure labels. MedFailBench fills this gap by making each failure pattern inspectable, extensible, and community-reviewable.
For Turkish-language medical AI evaluation, TR-MMMU [9] and recent literature surveys [10] provide context for the national dimension of this work. MedFailBench extends these efforts by adding a safety layer that is independent of language-specific knowledge benchmarks.
The Turkish 2026–2030 AI Vision identifies health AI conformity assessment and the Bilge national LLM as strategic priorities. MedFailBench can function as an independent public safety review surface for these efforts, providing transparent failure inspection without patient data access. We position MedFailBench as a community resource that national initiatives can build on rather than a certified assessment tool.
Future work will prioritize four tasks. First, a second clinician reviewer will provide inter-rater reliability data, addressing the main limitation of single-clinician annotation. Second, weekly model response evaluations will be extended into longitudinal runs with clinician validation. Third, specialty coverage will expand beyond the current 100 synthetic cases while preserving traceable annotations. Fourth, the automated rule-based scoring system will be validated against clinician judgments before comparative performance claims are made.
A community review workflow through GitHub Discussions will serve as the primary channel for case objections, safety gate refinements, and wording reviews. Each discussion thread will follow a structured template: failure pattern description, clinical context, proposed re-label or gate change, and supporting evidence. Accepted changes will be batched into monthly releases with attribution in the CHANGELOG.
Community contributions through synthetic cases, safety gate objections, and wording review are welcome and coordinated through GitHub Discussions.
Synthetic cases only; no real patient data and no demonstrated generalizability to real clinical workflows.
Single-clinician review with no inter-rater reliability measurement. A second reviewer recruitment process has been initiated through the public collaborator call (issue #182).
Case count (100 entries in v0.2.1) remains synthetic-only and single-reviewer; inter-rater reliability data from a second clinician reviewer is in progress.
Not a clinical validation study, model ranking, or deployment safety certification.
The weekly model response evaluation is a preview, not a longitudinal or controlled evaluation.
The automated rule-based scoring system requires validation against clinician review before any performance claims can be made.
Community review via GitHub Discussions is a new workflow and has not been stress-tested for volume or reliability.
Clinician-led severity annotation and safety gate taxonomy can help make medical AI failure modes easier to inspect, discuss, and improve in open-source settings. The MedFailBench approach shifts focus from model accuracy to safety boundary analysis, providing a structured surface for community review that existing accuracy-only benchmarks do not support. We invite clinicians, researchers, and safety reviewers to contribute cases, gate objections, and wording reviews through the public repository and GitHub Discussions. MedFailBench is released as an open resource under Apache-2.0 and CC-BY-4.0.
MedFailBench v0.2.1 is released under Apache-2.0 and CC-BY-4.0. The persistent DOI is 10.5281/zenodo.21205535. All source code, synthetic cases, evaluation data, and the live leaderboard preview are available at https://github.com/goktugozkanmd/medical-ai-failure-atlas and https://huggingface.co/spaces/goktugozkanmd/medical-ai-failure-atlas. The DOI resolves at https://doi.org/10.5281/zenodo.21205535.
The author used a large language model (DeepSeek V4 Pro via OpenRouter) for drafting assistance, LaTeX formatting, and reference management. All scientific content, clinical reasoning, safety labels, and editorial decisions were made and verified by the author. No AI tool was used to generate clinical claims, safety assessments, or synthetic case content. The model was not trained on MedFailBench data.