1 Introduction↩︎

AI agents for the life sciences are crossing the boundary from digital reasoning to physical experimentation. Recent systems show that language-model agents can organize literature evidence, generate scientific hypotheses, support experimental design and participate in focused wet-lab workflows [1]–[5]. Robot Scientist systems, self-driving laboratories and automated cloud-lab studies further show that experimental platforms can support controlled biochemical operations, robotic execution and iterative optimization [6]–[8]. Yet the handoff from biological intent to physical experimentation remains fragile. A scientific idea becomes experimentally actionable only when the intended biology is translated into samples, reagents, parameters, procedures, device actions, quality-control checks, readouts and revision decisions that remain valid on real instruments and under real experimental outcomes.

Designing a protocol from natural-language scientific intent is a long-context generation task that demands domain-specific expertise, long-range memory, and accumulated practical experience [9]–[11]. Moreover, a protocol draft is an intermediate representation rather than an executable experiment [12], [13]. During wet-lab automation, scientific rationale, sample and reagent states, quantitative parameters, operation order, device constraints and quality-control readouts must remain aligned as a natural-language goal becomes a protocol, SOP, device workflow, executable code and feedback-guided revision (Fig. 1a,b). Different layers resolve different ambiguities. A protocol must express biological logic, sample lineage, timing and quality-control structure. An SOP must ground that logic in feasible volumes, concentrations, labware, incubation conditions and acceptance criteria. Device code must bind the procedure to deck layout, well mapping, liquid-handling actions, timing control and vendor-specific SDK commands. Failures of alignment across these layers can make a workflow difficult to execute, reproduce or repair after an unsuccessful run [14], [15].

Recent work has advanced important parts of this transformation. Co-Scientist, Robin and the Virtual Lab establish language-model agents as tools for scientific reasoning, hypothesis generation and experimental design [1], [2], [4]. PrimeGen connects agentic design with liquid-handling code, anomaly monitoring and wet-lab readouts in targeted biological workflows [5]. Thoth, BioProAgent and LabscriptAI improve protocol generation, constrained experimental planning and protocol-to-script conversion [10], [13], [16]. BPL, LabOP, BioCoder and LAP provide more explicit representations for protocol structure, type constraints, state tracking, automation scripts and compiler-facing diagnostics [12], [14], [15], [17]. Together, these studies show that AI-enabled laboratory execution now has concrete technical precedents. They also sharpen a methodological requirement: autonomous wet-lab agents should be evaluated not only by the plausibility of generated protocol text, but by whether biological intent remains intact through executable, testable and revisable experimental workflows.

We addressed this requirement by constructing BioLab bench, an expert-grounded benchmark and evaluation framework for autonomous wet‑lab experimentation. The benchmark combines diverse synthetic-biology and molecular-biology scenarios, gold-standard protocol data, wet-lab expert experience, task-specific metrics, rubric-based validators, device-level validity gates and real experimental tests. Rather than treating protocol generation as an isolated text task, the framework evaluates whether generated workflows satisfy biological intent, preserve methodological consistency, instantiate sound parameters and remain procedurally complete. It further tests whether those workflows can be translated into SDK-exportable, instrument-compatible command objects, and whether they produce interpretable biological readouts that can support revision after failed or suboptimal steps. In this way, the benchmark turns expert wet-lab judgment into measurable requirements for experimental automation, and underscores the gap between the capabilities of existing general‑purpose or specialized AI agents and those required for truly autonomous wet‑lab experimentation.

Motivated by this gap, we developed ProtoPilot, a self-evolving multi-agent system for layer-wise verifiable experimental automation. ProtoPilot is organized around two mechanisms. The first is hierarchical multi-agent collaboration. An Orchestrator Agent maintains workflow state while specialized agents generate protocols, expand SOPs, map procedures to devices, synthesize SDK-gated code, validate intermediate outputs and incorporate wet-lab feedback. Each transition produces an inspectable object, from protocol and SOP to device workflow, command object and readout. The second mechanism is runtime agent-skill learning. A multilayer agentic memory and self-evolving skill library encode reusable operational knowledge from domain knowledge, protocol context, SDK information, code verification, task history, expert feedback and experimental outcomes. These skills support parameter confirmation, deck and sample-layout planning, and instrument-operation code specification, enabling underspecified experimental steps to be grounded as SDK-compliant workflows and adapted to new tasks or instruments. Rubric-based validators connect these mechanisms to the benchmark criteria by checking biological plausibility, procedural completeness, parameter consistency, safety, protocol-code alignment and instrument compatibility.

Applying this framework, we evaluated ProtoPilot across protocol generation, protocol-to-code translation, cross-platform robotic execution and wet-lab validation. The benchmark contains 294 synthetic-biology and molecular-biology tasks derived from 98 gold-standard protocols across increasing levels of experimental complexity. ProtoPilot-generated protocols received stronger blind preferences from wet-lab experts, reaching a Top@3 preference rate of \(90.2\%\). In protocol-to-code tasks, generated workflows passed syntax, parameter-alignment and MGI AlphaTool SDK export gates with an overall gate pass rate of \(96.6\%\). Additional Opentrons evaluation showed a ProtoPilot pass rate of \(88.24\%\), compared with \(32.35\%\) for OpenTrons-AI, indicating transfer beyond a single device platform. We further validate ProtoPilot across wet-lab workflows of increasing complexity, including routine liquid handling and clone screening, luciferase plasmid construction, whole-plasmid mutagenesis and PCA-based de novo DNA assembly (Fig. 1e). These experiments yielded reproducible 96-well bacterial inoculation, ordered tenfold dilution profiles, positive colony-PCR products in all 24 tested clones, Sanger-confirmed GLuc-WT and RLuc-WT plasmids, confirmed mutants for seven of eight GLuc and all eight RLuc designs, and sequence-confirmed PCA-assembled EGFP and A501_Ec.CPS-derived DNA targets after feedback-guided protocol revision. In the PCA experiment, ProtoPilot revised the protocol after failures in vector amplification and transformation, leading to sequence-confirmed assembled DNA targets in subsequent experiments. Together, the comprehensive evaluation demonstrate that ProtoPilot can provide a paired evaluation-and-execution framework for converting biological intent into executable, verifiable and feedback-correctable wet-lab workflows.

2 Results↩︎

2.1 ProtoPilot Overview: A self-evolving multi-agent system for biological experiment automation↩︎

ProtoPilot was designed to operationalize end-to-end wet-lab automation within a single stateful agent loop. Starting from a free-form natural-language description of an experimental objective, ProtoPilot progressively transforms user intent into a validated SOP, an instrument-executable program and a physically executed workflow, and then uses execution outcomes to guide subsequent refinement (Fig. 1a). The system can operate autonomously, while also supporting human-in-the-loop checkpoints when user confirmation or local laboratory constraints are present.

At the core of ProtoPilot is an Orchestrator Agent that maintains a structured workflow state. The Orchestrator decomposes the user objective into actionable subtasks, plans the workflow and coordinates two specialized agents: a Protocol Expert Agent trained with reinforcement learning for step-level SOP generation, and a Coding Agent equipped with a self-evolving skill library for translating validated protocols into instrument-executable code. Domain knowledge, prior task experience and the current workflow state are organized in a multilayer agentic memory bank. This memory includes a knowledge base of life-science domain knowledge, literature and retrievable protocol documents; a working memory that records the plan, constraints, intermediate outputs and progress of the current task; and a task-experience base that stores completed trajectories, user preferences, validation results, execution traces and feedback. The Orchestrator retrieves from and updates this memory throughout the workflow to support planning, consistency checking and inter-agent coordination. Across the loop, rubric-based validators evaluate generated SOPs and code for biological plausibility, procedural completeness, parameter consistency, safety, protocol–code alignment and compatibility with instrument and SDK constraints.

Beyond the default autonomous execution mode, ProtoPilot supports real-time human-in-the-loop interaction. At key decision points, the Orchestrator Agent records the current generation state and queries the user to confirm whether the required materials are available, whether the intended instruments can support the proposed operations and whether experimental adjustments are needed. After execution, experimental observations, quantitative readouts and outcome analyses are returned to the Orchestrator, which updates the workflow state, replans the next iteration and invokes the relevant expert agent to refine the protocol or selected experimental steps. This design allows ProtoPilot to connect planning, validation, execution and empirical feedback without treating each experimental attempt as an isolated prompt-response task.

ProtoPilot implements two core mechanisms to support this workflow. The first is a hierarchical, coarse-to-fine coordinator–worker mechanism for SOP construction. Rather than generating an entire SOP in a single pass, the Orchestrator first builds a module-level plan and global constraint set from the user objective, retrieved domain knowledge, prior task experience and relevant protocol literature. For each module, it provides the Protocol Expert Agent with the module goal, accepted upstream content, expected downstream dependencies, user and device constraints and accumulated SOP state. The expert agent then returns step-level procedures and candidate quantitative parameters. The Orchestrator evaluates the output against rigorously specified rubric-based criteria before integrating accepted content into the evolving SOP and planning the next module. This recurrent interaction enables local step-level detail while preserving global workflow state and long-range procedural coherence.

The second mechanism is self-evolving agent–skill learning for SOP-to-code conversion. The Coding Agent does not treat executable-code generation as a direct translation from protocol text to API calls. Instead, it uses a runtime-updated skill library initialized and expanded from expert-derived operational knowledge, including practical rules for material handling, device use, failure avoidance and biological constraint management. These skills are organized into three hierarchical classes: parameter-confirmation skills, deck- and sample-layout planning skills, and instrument-operation/code-specification skills. During conversion, parameter-confirmation skills resolve underspecified or qualitative SOP descriptions into executable values; layout-planning skills determine physical deck configurations and well-position maps; and operation/code-specification skills generate step-level operation descriptions together with SDK-compliant executable code. During runtime use and testing, candidate skills are iteratively improved through creation–evaluation–revision loops, in which skills are created from expert rules, protocol context, SDK information and prior task trajectories; evaluated using rubric-based validators, code checks and execution feedback; and revised into more reliable reusable routines. By encoding stable expert operational principles rather than surface features of a single protocol or SDK, this skill layer provides a transferable basis for protocol–code consistency and adaptation across experimental tasks, device models and vendor-specific software interfaces.

Overall, ProtoPilot treats laboratory automation as a stateful, feedback-driven process rather than a set of independent generation tasks. By integrating hierarchical agent orchestration, multilayer memory, rubric-based validation, runtime skill learning, optional human oversight and automated workstation execution, ProtoPilot converts natural-language experimental goals into validated SOPs and SDK-compliant executable programs, orchestrates their execution on physical instruments and updates subsequent designs from experimental outcomes within a unified closed loop.

2.2 ProtoPilot achieves superior protocol quality across complexity levels, corroborated by blind expert evaluation↩︎

We evaluated ProtoPilot against all comparison systems on 294 protocol generation tasks spanning three difficulty levels (L1–L3) using a seven-dimension rubric (D1–D7; see Technical Validation). ProtoPilot achieved an overall rubric score of 94.7 \(\pm\) 0.4 (mean \(\pm\) s.e.m.), outperforming the next-best system, BioProAgent, by 3.2 points and all general-purpose LLMs by more than 6 points (Fig. 2a). Performance remained consistently high across all difficulty tiers: 96.5 \(\pm\) 1.1 at L1 (\(n = 15\), basic single-reagent workflows), 95.1 \(\pm\) 0.5 at L2 (\(n = 165\), intermediate multi-reagent protocols), and 94.0 \(\pm\) 0.8 at L3 (\(n = 114\), complex multi-stage workflows; Extended Data Fig. 7). The 2.5-point decline from L1 to L3 was substantially smaller than the corresponding declines observed in agentic harness systems, confirming that ProtoPilot’s quality advantage is not only preserved but widened under increasing domain-specific complexity.

Dimension-level analysis showed uniformly strong performance across the rubric profile. On the three core scientific quality dimensions, requirement response (D1: 96.7), methodology consistency (D2: 97.7), and parameter soundness (D3: 98.9) each reached near-maximal levels, confirming that ProtoPilot accurately addresses user intent, adheres to reference-document methodology, and correctly specifies quantitative parameters across protocol types. Operability and narrative coherence (D5: 96.7) showed a clear advantage over harness-augmented systems (82.0–86.5), a gap reflecting ProtoPilot’s structured section transitions and step-level operational specificity. Scientific annotation depth (D7: 80.7) exhibited the largest separation from harness-augmented systems (50.8–61.0) and general-purpose LLMs (27.3–63.9), a differentiation attributable to the domain-specialized training underlying Thoth, ProtoPilot’s dedicated protocol generation model.

The automated rubric was validated by a blind preference evaluation in which three independent wet-lab scientists ranked outputs from seven representative systems across 34 queries, without knowledge of system identity (see Technical Validation). ProtoPilot was ranked first in 70.6 \(\pm\) 16.4% of queries and appeared within the top three in 90.2 \(\pm\) 9.8% of queries, both substantially exceeding all comparison systems (BioProAgent: 7.8 \(\pm\) 6.4% top-1; LabScript-AI: 57.8 \(\pm\) 18.7% top-3; Fig. 2d), with consistent leads maintained across all difficulty tiers (Extended Data Fig. 14). ProtoPilot’s MRR was 0.82 \(\pm\) 0.12, more than twice that of the second-ranked system (LabScript-AI: 0.37 \(\pm\) 0.07), and head-to-head preference rates against individual systems ranged from 0.78 to 0.92 (Extended Data Fig. 17). Inter-rater agreement across the three experts was moderate (overall Kendall’s \(W = 0.495\); Extended Data Fig. 16), consistent with the inherent subjectivity of protocol quality ranking. Automated rubric scores and expert MRR were strongly correlated across all seven systems (Spearman \(\rho = 0.929\), \(p = 0.003\)), confirming the rubric as a reliable proxy for independent expert judgement.

Together, these results establish that ProtoPilot consistently generates protocols of superior quality across all complexity levels, supported by both the highest automated rubric score among all evaluated systems and a top-1 expert preference rate of 70.6% in blind evaluations. The strong concordance between automated and human assessment validates rubric-based scoring as a reliable and scalable measure of protocol quality for biological research workflows.

2.3 ProtoPilot generates safe, executable instrument code with robust cross-device generalizability↩︎

Beyond protocol quality, we assessed the safety and executability of instrument code generated by ProtoPilot and 12 comparison systems across all 294 protocol-to-code translation tasks using a six-metric Code Score and a three-layer gate check (Fig. 2b¹). ProtoPilot achieved a Protocol Score of 95.5, outperforming the next-best system (LabScript-AI, 87.5) by 8.0 points, with markedly lower variance across all tasks. Safety dimensions showed the largest margins: machine safety reached 94.8% and bio safety 87.8%, exceeding LabScript-AI by 25.1 and 28.0 percentage points, respectively. SDK compliance was similarly elevated (98.2% versus 87.5–94.1% across baselines; Extended Data Fig. 9). Per-difficulty Code Score distributions confirm that ProtoPilot maintains a narrow, high-scoring profile at all complexity levels, whereas general-purpose LLMs exhibit substantially wider variance and lower medians at L3 (Extended Data Fig. 10).

Executability was assessed through a three-layer gate check covering Python syntax validation, parameter alignment against the device SOP, and an SDK-level export that verified slot conflicts, pipetting volume bounds, and labware compatibility. ProtoPilot achieved an overall gate pass rate of 96.6% across all 294 tasks, compared with a maximum of 35.0% among general-purpose LLMs (Grok-4.3) and 64.6% for LabScript-AI. Gate pass rates stratified by difficulty tier were 93.3% at L1, 97.0% at L2, and 96.3% at L3 (Extended Data Fig. 8). A joint analysis of Protocol Score and gate pass rate placed ProtoPilot alone in the high-quality, high-executability quadrant of this performance space (Fig. 2b): general-purpose LLMs achieved Protocol Scores of 77–86 yet gate pass rates below 35%, demonstrating that surface-level code quality does not guarantee physical executability without coherent cross-layer alignment.

Cross-device generalizability was assessed by deploying ProtoPilot and LabScript-AI on four laboratory automation platforms (AlphaTool MGI, Hamilton STAR, OpenTrons OT-2, and Tecan EVO; \(n = 34\) held-out tasks per device), with OpenTrons-AI additionally evaluated on OT-2 as a device-specific reference (Fig. 2c). ProtoPilot achieved gate pass rates of 91.2%, 85.3%, 88.2%, and 91.2% across the four platforms (a cross-device spread of 5.9 percentage points), whereas LabScript-AI ranged from 8.8% to 55.9% (a spread of 47.1 percentage points), revealing severe brittleness to platform-specific SDK requirements. On OT-2, where OpenTrons-AI (a purpose-built, single-platform system) was additionally benchmarked, ProtoPilot’s gate pass rate of 88.2% substantially exceeded both LabScript-AI (32.4%) and OpenTrons-AI (32.4%). Protocol Score followed an analogous pattern: ProtoPilot ranged from 93.8 to 97.9 across devices, while LabScript-AI ranged from 80.5 to 87.3 and OpenTrons-AI scored only 65.4 on its native platform.

Collectively, these results demonstrate that ProtoPilot is the only evaluated system to simultaneously achieve high code quality, near-complete executability, and stable cross-device performance, establishing it as a broadly deployable foundation for robotic wet-lab automation.

2.4 ProtoPilot demonstrates reliable wet-lab performance across experiments of increasing complexity↩︎

2.4.1 ProtoPilot automates foundational wet-lab operations↩︎

We first evaluated ProtoPilot on a set of high-frequency foundational wet-lab operations: E. coli culture inoculation, serial dilution, and colony PCR screening. These workflows represent routine steps in strain cultivation, sample dilution, and preliminary clone identification. Although individually straightforward, each task requires repetitive liquid handling, parallel processing across multiwell formats, and accurate transfer volumes, making them well-suited for assessing autonomous execution of routine laboratory procedures. After the user specified each task in natural language, ProtoPilot planned the experimental procedure and generated executable automation workflows covering source culture sampling, medium or diluent dispensing, culture transfer, mixing, and OD₆₀₀ measurement. Workflow diagrams summarize the major steps for the three independent experiments (Fig. 3a,d,g), and the corresponding automation layouts show representative deck configurations illustrating how each task was converted into an executable physical setup (Fig. 3b,e,h).

In the culture inoculation experiment, all 96 sample wells showed visible bacterial turbidity after incubation at 37 °C with shaking, whereas blank control wells remained clear (Fig. 3c(i)). The OD₆₀₀ heatmap showed that sample-well readings ranged from approximately 0.51 to 0.88, indicating that the automated workflow reproducibly supported multiwell culture inoculation and growth readout (Fig. 3c(ii)).

In the serial dilution experiment, bacterial culture with an initial OD₆₀₀ of approximately 0.71 was used to generate eight independent 12-step, tenfold dilution series. The representative plate image showed successful configuration of the dilution series in the multiwell format (Fig. 3f(i)). OD₆₀₀ profiles showed an ordered decrease in absorbance with increasing dilution, with the \(10^{-1}\) dilution at approximately 0.13 and higher dilution levels approaching the blank background of approximately 0.039, with no detectable contamination across series (Fig. 3f(ii) and Extended Data Fig. 16b).

For colony PCR, ProtoPilot generated a workflow covering colony picking, PCR master-mix setup, reaction aliquoting, amplification, and gel readout (Fig. 3g), with PCR mix dispensing shown in Extended Data Fig. 16c. Agarose gel electrophoresis confirmed that all 24 tested clones produced clear, specific amplicons at the expected 868 bp size, with no failed amplification or nonspecific bands (Fig. 3i), demonstrating that ProtoPilot can support sample handling, reaction setup, and product validation in basic clone-screening workflows.

Together, these experiments demonstrate that ProtoPilot can translate natural-language requests for common foundational wet-lab procedures into executable automation workflows, producing stable and experimentally verifiable readouts across diverse routine tasks.

2.4.2 ProtoPilot supports luciferase plasmid construction and mutant engineering↩︎

2.4.2.1 Construction of pET21b-GLuc-WT and pET21b-RLuc-WT plasmids

After validating foundational wet-lab operations, we next evaluated pET21b-GLuc-WT and pET21b-RLuc-WT plasmid construction as representative multi-step molecular cloning tasks. Unlike the foundational operations above, wild-type plasmid construction requires coordinated execution of insert amplification, vector-backbone preparation, DpnI digestion, homologous recombination assembly, competent-cell transformation, colony PCR screening, and Sanger confirmation, placing greater demands on step-to-step continuity, sample tracking, and quality control of intermediate products (Fig. 4a).

Upon receiving the user’s initial request, ProtoPilot reasoned over the construction objectives and experimental route, then used multi-turn interaction to clarify the insert sequences, vector backbone, validation strategy, and available laboratory reagents, consumables, and equipment. After the experimental plan was confirmed, ProtoPilot generated an executable automation workflow supporting PCR amplification of the GLuc-WT and RLuc-WT inserts and the pET-21b(+) vector backbone, followed by assembly setup, transformation, and clone validation. The corresponding automation layout shows a representative deck configuration for the workflow on the liquid-handling platform (Fig. 4b).

Agarose gel electrophoresis of three independent replicates produced clear single bands at the expected sizes for the GLuc-WT insert (685 bp), RLuc-WT insert (1,060 bp), and pET-21b(+) vector backbone (5,403 bp), with no nonspecific amplification (Fig. 4c). Quantification of the purified PCR products showed that the GLuc-WT insert concentrations across the three replicates were 89.4, 97.8, and 98.5 ng/μL; the RLuc-WT insert concentrations were 158.0, 144.1, and 152.7 ng/μL; and the pET-21b(+) vector-backbone concentrations were 38.1, 39.9, and 57.7 ng/μL, respectively, all sufficient for downstream homologous recombination assembly (Fig. 4d).

Candidate clones obtained after transformation were screened by colony PCR. For both GLuc-WT and RLuc-WT, all three colonies picked from each of the three independent construction replicates were PCR-positive (Fig. 4e). Subsequent Sanger sequencing confirmed two of the three candidate clones per replicate for GLuc-WT, and three, three, and two candidate clones across the three replicates for RLuc-WT (Fig. 4f). Both targets thus yielded Sanger-confirmed correct plasmid clones, with a final construction success rate of 100% for both GLuc-WT and RLuc-WT (Fig. 4g). These results demonstrate that ProtoPilot can support a complete plasmid construction workflow spanning fragment preparation, vector processing, assembly setup, and positive-clone confirmation, while maintaining traceable execution across a multi-step molecular cloning task.

2.4.2.2 Whole-plasmid mutagenesis of GLuc and RLuc

Building on the wild-type plasmid construction results, we extended the evaluation to whole-plasmid mutagenesis of GLuc and RLuc to assess ProtoPilot in a parallel multi-variant construction setting. Compared with wild-type construction, site-directed mutagenesis additionally requires mutation design, primer design, and parallel management of multiple target amino-acid substitutions, placing greater demands on workflow organization and sample tracking (Fig. 5a).

Upon receiving the user’s request to construct multiple GLuc and RLuc point mutants in parallel, ProtoPilot reasoned over the mutation targets and experimental route, then used multi-turn interaction to clarify the target sites, mutation types, screening requirements, and available laboratory reagents, consumables, and equipment. After the experimental plan was confirmed, ProtoPilot generated an executable automation workflow supporting parallel construction of multiple point mutants, with a representative deck configuration illustrating how the mutant reactions were organized into an executable physical setup (Fig. 5b). In total, 16 point mutants were designed: eight GLuc mutants (L30S, L40P, L40S, M43I, M43V, M110I, E111S, and E111P) and eight RLuc mutants (A55T, C124A, S130A, K136R, A143M, M185V, M253L, and S287L).

Agarose gel electrophoresis showed that all mutants except GLuc-L40P produced clear single whole-plasmid PCR bands at the expected sizes, approximately 6,048 bp for the GLuc series and 6,423 bp for the RLuc series (Fig. 5c). Quantification confirmed that the failed GLuc-L40P amplification yielded only 4.8 ng/μL, whereas the remaining 15 mutants yielded 160–187 ng/μL. The GLuc mutant concentrations for L30S, L40S, M43I, M43V, M110I, E111S, and E111P were 160, 187, 176, 182, 171, 168, and 175 ng/μL, respectively; the RLuc mutant concentrations for A55T, C124A, S130A, K136R, A143M, M185V, M253L, and S287L were 166, 170, 168, 182, 170, 162, 168, and 170 ng/μL, respectively (Fig. 5d).

The successfully amplified mutants were subjected to DpnI digestion, transformation, and colony PCR screening. Agarose gel electrophoresis confirmed that most candidate clones produced clear bands of the expected size suitable for downstream Sanger confirmation (Fig. 5e). Sanger sequencing confirmed 7 of the 8 designed GLuc mutants, with GLuc-L40P remaining unconfirmed, and all 8 designed RLuc mutants (Fig. 5f). These results demonstrate that ProtoPilot can support parallel construction of multiple point mutants across a complete variant-engineering workflow spanning mutation design, whole-plasmid amplification, template removal, transformation screening, and sequence confirmation.

2.4.3 ProtoPilot refines experimental protocols iteratively through closed-loop feedback during PCA-based DNA assembly↩︎

Beyond plasmid construction and site-directed mutagenesis, we next evaluated polymerase cycling assembly (PCA)-based DNA assembly as a more complex long-form construction workflow. Starting from Primerize-designed oligonucleotide pools, this workflow involves oligo-pool preparation, PCA assembly, T7 Endonuclease I correction, post-PCR amplification, Gibson assembly, transformation, and clone validation (Fig. 6a). The PCA workflow contains more intermediate products, a longer chain of experimental steps, and more complex sample relationships than conventional PCR or single-plasmid construction, making it a demanding test case for multi-module DNA assembly.

Upon receiving the user’s request for parallel assembly and cloning of multiple target fragments, ProtoPilot reasoned over the assembly strategy and validation route, then used multi-turn interaction to clarify the target sequences, assembly mode, validation checkpoints, and available laboratory reagents, consumables, and equipment. ProtoPilot selected four target fragments of similar length but different origins for parallel assembly: BL01, an EGFP coding sequence of 698 bp, and three regions from A501_Ec.CPS, namely BL02, BL03, and BL04, with lengths of 802, 713, and 800 bp, respectively. The corresponding automation layout shows a representative deck configuration illustrating how the long DNA assembly workflow was decomposed and mapped into an executable automation environment (Fig. 6b). Each fragment was assembled in three independent replicates to evaluate workflow robustness.

PCA assembly products for all four target fragments showed bands at the expected sizes, whereas the no-template control showed no obvious amplification (Fig. 6c). After T7 Endonuclease I correction, post-PCR amplification produced clear bands of the expected sizes for all four fragments, confirming that the assembled and corrected products were suitable for downstream cloning (Fig. 6d). Quantification after purification (\(n = 3\) replicates per group) showed PCA product concentrations of 65.93 \(\pm\) 3.83, 65.07 \(\pm\) 4.22, 58.73 \(\pm\) 4.12, and 61.40 \(\pm\) 2.88 ng/μL for BL01–BL04, respectively, and post-PCR product concentrations of 78.27 \(\pm\) 2.70, 119.33 \(\pm\) 4.04, 121.33 \(\pm\) 4.16, and 77.13 \(\pm\) 8.15 ng/μL for BL01–BL04, respectively. No-template controls remained near background (approximately 0.2 ng/μL for PCA and 0.5 ng/μL for post-PCR), indicating no detectable contamination during the workflow (Fig. 6f).

Vector preparation and downstream assembly revealed two iterative optimization cycles. For vector preparation, pUC19-vb1 produced the expected 2,619 bp target band but also showed a prominent nonspecific band in the approximately 4,000–7,000 bp region (Fig. 6e(i)); ProtoPilot analyzed the failure mode and redesigned the primers to improve specificity, and the second preparation, pUC19-vb2, produced a clear single band at 2,619 bp (Fig. 6e(ii)). The four post-PCR fragments were then assembled with pUC19-vb2 by Gibson assembly and transformed. The first transformation showed a clear failure: colonies appeared on the negative-control plate, and the BL01–BL04 sample plates exhibited extensive or lawn-like growth that precluded single-colony picking (Fig. 6g). ProtoPilot attributed this to ampicillin inactivation and excessive plating volume, regenerated a revised protocol using freshly prepared ampicillin plates and reduced plating volume, and re-executed the transformation. The second round produced no negative-control colonies and yielded distinguishable single colonies across all four target plates (Fig. 6g), demonstrating ProtoPilot’s capacity for failure diagnosis, protocol reconstruction, and iterative re-execution in response to real wet-lab abnormalities. Complete replicate plate results from the first and second transformation rounds are shown in Extended Data Fig. 17.

Colony PCR screening across three transformation replicate plates, with eight colonies picked per plate, showed that 93 of 96 candidate clones were PCR-positive, corresponding to an overall positivity rate of 96.9% (). Sanger sequencing of four positive clones per replicate plate confirmed 8, 6, 9, and 6 sequence-correct clones for BL01, BL02, BL03, and BL04, respectively (). All four target DNA sequences produced at least one Sanger-confirmed correct clone in an independent replicate, demonstrating successful construction of each target sequence.

Together, these results show that ProtoPilot can support a long-form DNA construction workflow spanning oligonucleotide-pool design, fragment assembly, correction and amplification, vector preparation, homologous recombination, and clone screening. Critically, ProtoPilot identified and resolved two distinct experimental failures during execution, confirming that the system is not limited to one-shot protocol generation but can perform multi-module execution, failure handling, and iterative optimization in complex synthetic-biology construction workflows.

3 Discussion↩︎

Here we present ProtoPilot, a self-evolving multi-agent system for automating life-science experimental workflows from natural-language intent to validated SOPs, instrument-executable code, physical execution and feedback-guided refinement. Rather than treating protocol writing, code generation and robotic execution as separate tasks, ProtoPilot formulates laboratory automation as a stateful workflow in which biological context, operational constraints, validation outcomes and experimental feedback are maintained across the entire process. Our results show that this design enables accurate protocol generation, reliable protocol-to-code translation and practical wet-lab execution across diverse experimental settings.

The core technical contribution of ProtoPilot is its integration of hierarchical agent orchestration with reusable operational skills. The novel multi-agent collaboration mechanism allows the system to decompose long SOPs into manageable modules while maintaining global workflow state, sample lineage, parameter dependencies and quality-control logic. This is essential for real experimental SOPs, where correctness depends not only on local step quality but also on consistency across distant procedural sections. In parallel, the self-evolving skill library used by the Coding Agent encodes expert operational knowledge that is often absent from protocol texts or SDK documentation. By organizing this knowledge into parameter-confirmation, deck-layout planning and code-specification skills, ProtoPilot bridges the gap between a biologically valid SOP and a physically executable robotic program.

To rigorously evaluate such a system, we constructed an expert-grounded benchmark and a comprehensive evaluation framework for autonomous wet-lab experimentation. The benchmark combines diverse synthetic-biology and molecular-biology scenarios with gold-standard protocol data, wet-lab expert experience, task-specific metrics, rubric-based validators, device-level validity gates and real experimental tests. Rather than treating protocol generation as an isolated text task, the framework assesses whether generated workflows satisfy biological intent, preserve methodological consistency, instantiate sound parameters and remain procedurally complete. It further tests whether those workflows can be translated into SDK-exportable, instrument-compatible command objects, and whether they produce interpretable biological readouts that can support revision after failed or suboptimal steps. In this way, the benchmark translates expert wet-lab judgment into measurable requirements for experimental automation, and underscores the gap between the capabilities of existing general‑purpose or specialized AI agents and those required for truly autonomous wet‑lab experimentation.

Across 294 real-world synthetic biology tasks, ProtoPilot achieved superior protocol quality across different complexity levels. This improvement was corroborated by blind expert evaluation, suggesting that the generated SOPs were not merely more complete according to automatic metrics but also more consistent with expert expectations for experimental feasibility, procedural clarity and biological validity. These results support the importance of maintaining an explicit workflow state and validating each generated module before it is integrated into the final SOP. In particular, the hierarchical generation strategy reduced the tendency of LLM agents to produce locally plausible but globally inconsistent protocols.

ProtoPilot also generated safe, executable instrument code with robust generalizability across tested device settings. A central difficulty in laboratory code generation is that syntactic correctness alone is insufficient: a script may compile while still violating the validated protocol, misaligning sample or well mappings, mishandling tips, placing labware in inaccessible positions or executing module states in an unsafe order. ProtoPilot addresses these failure modes by combining skill-mediated code synthesis with rubric-based validation of protocol–code alignment, deck and well-map consistency, liquid-handling completeness, SDK compliance and device-state constraints. The cross-device performance indicates that encoding stable expert rules, rather than relying only on protocol-specific prompts or SDK surface syntax, provides a transferable basis for adapting automation to heterogeneous instruments and software interfaces.

The wet-lab experiments provided a more stringent assessment of ProtoPilot than in silico evaluation alone. Across experiments of increasing complexity, ProtoPilot generated and executed workflows that were reliable in practice, demonstrating that its outputs can satisfy real laboratory constraints rather than only text-based benchmarks. The luciferase plasmid construction and mutant-engineering studies further show that ProtoPilot can support multi-step molecular biology workflows in which correct execution requires coordination among reagent preparation, DNA manipulation, purification, transformation or downstream validation steps. These experiments suggest that ProtoPilot can contribute to the build stage of synthetic biology workflows, where protocol detail, execution fidelity and reproducibility are all critical.

A further distinguishing feature of ProtoPilot is its ability to refine protocols from empirical feedback. In the PCA-based DNA assembly study, experimental outcomes were returned to the system and used to guide subsequent protocol modification. This closed-loop behavior is important because wet-lab automation rarely succeeds as a one-shot generation problem: reaction efficiency, assembly quality, sample loss and execution failures often require iterative adjustment. By updating the workflow state and invoking the appropriate expert agents after feedback is received, ProtoPilot provides a practical mechanism for adaptive protocol optimization rather than treating each experiment as an isolated prompt-response interaction.

Several limitations remain. First, although the current benchmark covers 294 real-world synthetic biology tasks, its scale and diversity can be further expanded. Future benchmarks should include broader assay families, additional laboratory instruments, more heterogeneous labware configurations, different biological systems and more challenging failure modes. Second, ProtoPilot currently focuses primarily on wet-lab automation after a user-defined experimental objective has been provided. A natural next step is to integrate dry-lab modules for hypothesis generation, sequence or construct design, computational modeling, data analysis and optimization, thereby establishing a more complete dry–wet closed loop. Such integration would allow ProtoPilot not only to execute and refine experiments, but also to participate in the upstream design and prioritization of experimental hypotheses.

In summary, ProtoPilot provides a framework for stateful, closed-loop automation of life-science experiments. By integrating agentic planning, long-context SOP generation, expert-skill-mediated code synthesis, validation, robotic execution and feedback-driven refinement, ProtoPilot helps bridge the gap between experimental intent and physical laboratory execution. These principles provide a foundation for future AI-native laboratories in which computational reasoning and automated experimentation are integrated into a continuous, hypothesis-driven research cycle.

Materials↩︎

Bacterial strains, plasmids and culture media.

Escherichia coli BL21(DE3) was used as the host strain for plasmid construction, site-directed mutagenesis and clone propagation, and E. coli DH5\(\alpha\) was used for de novo PCA assembly and cloning. The pET-21b(+) vector was used as the expression backbone for constructing wild-type Gaussia luciferase and Renilla luciferase plasmids, designated pET21b-GLuc-WT and pET21b-RLuc-WT, respectively. The pUC19 vector was used as the cloning backbone for PCA-based de novo gene assembly. Sequence-confirmed pET21b-GLuc-WT and pET21b-RLuc-WT plasmids were subsequently used as templates for whole-plasmid site-directed mutagenesis. Bacteria were cultured in liquid or solid LB medium. Ampicillin was added where required at a final concentration of 100 μg/mL from a 100 mg/mL stock solution. LB agar plates containing ampicillin were used for selection after transformation, and liquid LB medium containing ampicillin was used for clone propagation and overnight culture. Unless otherwise specified, bacterial cultures were incubated at 37 °C; liquid cultures were shaken at 200 rpm, and agar plates were incubated inverted for 12 h.

Enzymes, kits and reagents.

PCR amplification was performed using 2\(\times\) Phanta Flash Master Mix for high-fidelity PCR and 2\(\times\) Rapid Taq Master Mix for colony PCR. Methylated parental plasmid DNA was removed using SwiftCut DpnI, and homologous recombination assembly was performed using the ClonExpress II kit; these reagents were purchased from Vazyme (Nanjing, China). Mismatch correction in the PCA workflow used T7 Endonuclease I (Beyotime, Shanghai, China). Agarose gel electrophoresis used 4S GelRed nucleic acid stain (Sangon Biotech, Shanghai, China) and DL10000 or DL2000 DNA markers (Takara, Japan). Gel-excised DNA fragments were purified using the OMEGA E.Z.N.A. Gel Extraction Kit (OMEGA Bio-tek, USA). PCA and post-PCR products were purified using MGIEasy DNA purification magnetic beads (MGI, Shenzhen, China) and quantified by Qubit fluorometry (Thermo Fisher, USA). Chemically competent E. coli BL21(DE3) and E. coli DH5\(\alpha\) cells were purchased from Solarbio (Beijing, China), and LB medium (Sangon Biotech, Shanghai, China) was used for recovery and culture.

Acknowledgements↩︎

This work was jointly supported by Shanghai Artificial Intelligence Laboratory and Genoria AI Technology Co., Ltd.

Competing Interests↩︎

The authors declare no competing interests.

Appendix↩︎

References↩︎