Keywords: Model Context Protocol; agentic instrumentation; schema-bound tool typing; large language models; live processing; skill abstraction; electron microscopy; provenance; bounded execution

1 Introduction↩︎

A scientific instrument is a programmable physical system, but the programming interface is rarely uniform. Each vendor exposes a Python or scripting surface that runs inside its own acquisition application, with idiosyncratic data types, threading conventions, and update cycles. The result is that day-to-day experimentation is mediated through graphical interfaces and bespoke scripts; reproducibility and language-first automation both suffer. Large language models (LLMs) are now competent planners for tool-mediated scientific work. Co-Scientist (Boiko et al., 2023) showed that an LLM agent can plan and execute organic synthesis on cloud-lab hardware [1]; ChemCrow demonstrated that curated chemistry tools improve task fidelity and reduce hallucination [2]; self-driving laboratories now compose these ideas at the bench [3]. The Berkeley Function-Calling Leaderboard [4] has made tool-calling itself a measurable skill, and modern open-weight models exceed 0.85 on its agentic split.

In microscopy, recent work has driven imaging and probe placement with active learning and hypothesis-driven agents [5]–[9], and dedicated LLM-agent platforms have appeared for atomic-force microscopy [10] and transmission electron microscopy [11], [12]. More recently, [13] framed the microscope itself as a thinking system that iteratively refines its own protocols. The most recent generation of agentic-science systems has widened this context beyond single-task tool use. Co-Scientist (Gottweis et al., 2026), a system distinct from the earlier Coscientist of Boiko et al.despite the near-identical name, uses a multi-agent architecture to generate, debate, rank, and refine testable biomedical hypotheses [14]; Robin integrates literature agents with experimental-design and data-analysis agents in an iterative lab-in-the-loop discovery workflow [15]; and Empirical Research Assistance combines LLM code generation with tree search to produce expert-level empirical software for scorable scientific tasks [16]. These systems plan, generate executable artefacts, interpret measurements, and update their hypotheses. For physical experiments, that shift makes the underlying control layer more important, not less: agents that act on microscopes must reason about latency, calibration, noise, drift, damage thresholds, transfer functions, and the uncertainty that propagates from one measurement to the next.

The Model Context Protocol (MCP), introduced by Anthropic in late 2024 and adopted across major model providers by 2026 [17], [18], provides a standardised way to expose tools, resources, and prompts to any compatible model. We build on MCP, but adopting it for scientific instrumentation reveals three recurring frictions that the protocol itself does not resolve (Fig. 1).

1: Host-process binding. Most vendor APIs load only inside the running acquisition process. An external Python interpreter cannot import the vendor’s bindings, so an agent that lives off the instrument PC cannot reach the column without crossing a process boundary.

2: Local-first governance. Many academic and industrial facilities forbid outbound connections from the instrument PC. Any cloud-only LLM backend is therefore off the table for routine use.

3: Bounded execution at the tool boundary. LLMs reliably emit syntactically valid tool calls with physically unreasonable arguments: a stage tilt of \(95^\circ\), a negative exposure, a region of interest outside the image. A control protocol must reject such arguments at the boundary, with auditable failure paths that do not depend on prompt phrasing. Reproducible, audit-friendly agent traces are now an active research direction [19]–[21].

We argue that these are protocol-design problems rather than instrument-engineering problems. Once the tool contract, the host-process boundary, the live-job lifecycle, and the skill abstractions are right, the specific instrument software becomes a swap-in component. We describe a method that combines (i) a schema-bound MCP tool surface validated by Pydantic v2 models [22] before dispatch; (ii) a vendor-neutral MicroscopeAdapter abstract base class that defines the surface every adapter implements, paired with a physics-plausible simulator and an optional host-process bridge for vendor APIs that require one; (iii) a persistent live-processing job lifecycle exposed as ordinary tools; and (iv) a skill abstraction, implemented through MCP prompts, that composes typed tools into reusable multi-step protocols.

This manuscript presents the method and its reference implementation as a software-only validation study: every claim is demonstrated against a physics-plausible simulator that implements the same MicroscopeAdapter contract, and the test suite, simulator, and source are released under MIT license (Section 3). Live-instrument validation on commercial scientific hardware is intentionally out of scope here and is reported separately under the corresponding intellectual-property disclosure [23]; readers should therefore note that on-hardware performance cannot be assessed from this manuscript alone.

The protocol-level framing distinguishes this work from the closest prior art. TEM Agent [11] demonstrated that MCP-mediated TEM workflows are feasible by exposing a vendor-specific surface to an LLM agent. Our contribution sits at a different layer of the stack. First, schema-bound validation runs in the server before dispatch to any adapter, so out-of-range or malformed requests fail at the boundary with a structured error and never reach the instrument. Second, the host-process boundary is crossed by a minimal, versioned JSON contract over loopback rather than by embedding the agent in the vendor process, which keeps the LLM stack swappable and the bridge independently testable. Third, a paired physics-plausible simulator shares the schema layer with any live adapter, so the entire validation suite runs without instrument occupancy. Fourth, a persistent live-processing job lifecycle and a skill abstraction - registered as MCP prompts so they are discoverable through ordinary prompts/list calls - compose typed tools into multi-step procedures. Together these choices yield a protocol-design contribution rather than an autonomy demonstration: a bounded, testable, local foundation for closed-loop and autonomous microscopy research.

The reference implementation, nuance-mcp, is open source. It ships with a physics-plausible simulator adapter that exercises the full protocol surface. The remainder of this paper is organised as follows. Section 2 presents the four design choices that define the method. Section 3 reports the software validation outcomes against the simulator and the local-LLM probe on the same hardware. Section 4 situates the contribution in context and states its limits. Appendix 6 documents the versioned JSON bridge contract that vendor-specific adapters must implement when an instrument API is host-process bound. Appendix 8 gives implementation guidance for adapting the contract to vendor-specific platforms.

2 Method↩︎

The method consists of four coupled design choices. The first two address frictions identified in Section 1; the deployment model (Section 2.6) addresses local-first governance; the third and fourth introduce abstractions (a live-job lifecycle exposed as typed tools, and a skill abstraction implemented through MCP prompts) that compose into multi-step protocols. Apart from the (vendor-specific) bridge layer, every layer is shared across adapter implementations and is exercised end-to-end against the reference simulator.

2.1 Schema-bound tool surface↩︎

Every operation an agent may invoke is exposed as an MCP tool with an associated input schema, expressed as a Pydantic v2 model [22]. The schema specifies argument names, types, units, default values, and admissible numeric ranges; it forbids unknown fields and rejects malformed compound types (e.g. regions of interest that are not four-element, monotone, in-image tuples). Validation is performed by the server before any adapter call is dispatched. A tool invocation therefore proceeds through three discrete stages (Fig. 2); the agent emits a JSON tool call, the server constructs the Pydantic instance, and only on successful construction is the adapter touched. Failed validation returns a structured ValidationError naming the offending field and constraint; the adapter is never reached. The behaviour is independent of how the LLM phrased the call, so audit traces of rejected requests reflect the agent’s actual output rather than its rationalisation.

We are careful to label this property bounded execution at the tool boundary, not safety. The schema enforces physical and operational bounds (e.g.\(\alpha \in [-80^\circ, +80^\circ]\), dwell-time \(\in [0.5,10^4]\,\mu s\), exposure \(\in [10^{-3}, 60]\) s). It does not substitute for hardware interlocks, beam-damage models, operator approval, or facility policy; those layers are discussed under deployment (Section 2.6). A complementary line of work has argued that schema validation is, in itself, a useful basis for trustworthy ML-driven scientific workflows [20].

2.2 Vendor-neutral adapter contract↩︎

The second design choice is the surface across which the schema layer talks to the instrument. We define this surface as a Python abstract base class, MicroscopeAdapter, with three properties (Fig. 3). First, it declares a capability vocabulary: a closed set of 24 capability families that an adapter may opt into. The vocabulary is enumerated in full in Table 5; representative families include tem, stem, 4dstem, eels, diffraction, stage, stage.tilt, optics, detectors, live_jobs, analysis.com_dpc, and analysis.radial_profile. Second, every operation the server can call is a method of MicroscopeAdapter with a vendor-neutral signature. Third, methods whose capability is not declared inherit a default implementation that raises CapabilityUnavailable; the tool layer translates this into a structured {"status": "UNSUPPORTED", "reason": ...} response. An adapter therefore advertises what it can do (its declared capabilities) and implements only those methods.

This contract has two consequences that matter for the rest of the method. First, the schema, simulator, lifecycle, and skill catalogue are written against MicroscopeAdapter and not against any vendor binding; adding a new instrument is implemented as a subclass and an entry point. Second, capability negotiation is now part of every session: the agent’s first call, by convention, is get_capabilities, which returns the bound adapter’s name and capability list. Skills that exercise a particular family begin with a capability check, which provides a clear failure mode when an adapter does not support a requested family (rather than a silent or misleading error mid-session). Recent attention to the role of typed tool descriptions in agent reliability supports this design choice [24]. A physics-plausible simulator implements MicroscopeAdapter directly, with no vendor dependencies, declaring 20 of the 24 capability families, and is the reference backend exercised in this manuscript. The simulator shares the schema layer with any live adapter, so the same validation rejects the same arguments regardless of which backend is bound. The hardware-independent test suite runs against the simulator and is bit-stable across machines for a fixed NumPy version.

2.3 Persistent live-processing jobs↩︎

Several useful instrument analyses are not one-shot transformations but long-running, periodically updated computations over a live front image (e.g. radial-profile tracking, exponentially-weighted difference imaging, local maximum-FFT mapping, derived filtered views, and 4D-STEM maximum-spot mapping). Exposing each as a single do-the-thing tool would force the agent to busy-wait or to reissue heavy computations.

We promote the lifecycle of a live analysis to a first-class element of the tool surface (Fig. 4). Four typed tools share a common state machine — start_live_processing_job, get_live_processing_job_status, get_live_processing_job_result, and stop_live_processing_job — and this same lifecycle applies to all five live-job types (radial_profile, difference, fft_map, filtered_view, maximum_spot_mapping). State lives where execution does: when an implementation-side bridge is configured, jobs are registered inside the host process so derived outputs persist with the rest of the session; otherwise, jobs are held by the server and operate on the adapter’s read-only methods.

This factoring has two consequences beyond convenience. First, the agent’s record of (start, status, result, stop) calls is itself a machine-readable record of how the analysis evolved, which helps with FAIR-style provenance [19], [25]. Second, status polling is cheap, so the agent can adapt acquisition parameters in response to live analytical feedback - a closed-loop pattern explored in [12], [26].

2.4 Skills: declarative multi-step protocols↩︎

Tools are necessary but not sufficient. A real microscopy session is organised around procedures — “run an EELS survey on this specimen”, “acquire and analyse a 4D-STEM map”, “align the beam before HRTEM” — each of which decomposes into a recurring, partially ordered sequence of tool calls with branching on intermediate results. Asking the LLM to derive these procedures on every session inflates token use, exposes the agent to off-policy mistakes, and produces inconsistent provenance across operators.

The fourth component of the method addresses this with an abstraction we call a skill (Fig. 5). A skill is an implementation pattern, not a new MCP primitive: each skill is a registered MCP prompt (one of the three MCP primitives, alongside tools and resources) that returns a structured, parameterised instruction sequence the agent unrolls into ordered tool calls. Each skill carries a name, a short description, and a typed argument list (e.g.specimen identity, energy of interest, scan geometry). Because skills sit above the tool layer, they are adapter-portable: the EELS-survey skill is the same declarative protocol on every adapter that exposes eels. Every skill begins with a get_capabilities check, so an adapter that lacks the required family yields a clear failure rather than an opaque mid-skill error.

Six skills cover the canonical end-to-end workflows we observed across users (Table 1): spectroscopic characterization (eels_survey), tomography (tilt_series_protocol), 4D-STEM (4dstem_characterization), instrument hygiene (beam_alignment), high-resolution imaging (hrtem_imaging), and crystallographic phase identification (diffraction_survey). Each skill’s body encodes the protocol step-by-step: the entry condition, the tool sequence and which fields to populate, the metrics to record, and the report to deliver at the end. The agent’s trace is therefore richer than a flat list of tool calls: it becomes (skill, args, tool sequence, results), which is the unit of provenance recommended by recent audit-trail work [19], [21]. A graph-based variant of the same idea has been independently proposed for EM-specific workflows [26].

4pt

2.5 Modality-specific analyses↩︎

For 4D-STEM datasets, the tool surface exposes virtual bright- and dark-field maps, a maximum-spot RGB mapping, the centre-of-mass field \(\mathbf{CoM}\), and the differential phase contrast magnitude,

\[\mathbf{CoM}(i,j) = \frac{\sum_{k,l} I(i,j,k,l)\,\mathbf{q}(k,l)}{\sum_{k,l} I(i,j,k,l)}, \qquad \mathrm{DPC}(i,j) = \left\lVert \mathbf{CoM}(i,j) \right\rVert, \label{eq:com}\tag{1}\]

where \(I(i,j,k,l)\) is the detector intensity at probe position \((i,j)\) and detector pixel \((k,l)\), and \(\mathbf{q}(k,l)\) is the corresponding reciprocal-space coordinate, with units of inverse length (typically Å\(^{-1}\)) consistent with the detector calibration. Equation 1 therefore returns a reciprocal-space (descriptive) centre-of-mass field; in the simulator this field is physically plausible but is not calibrated quantitative ground truth (Section 4, Limitations), so it should not be read as a calibrated measurement of the projected electric field. These are the standard definitions in phase-contrast (S)TEM  [27]–[30]. For HRTEM and diffraction patterns, radial profiling and local maximum-FFT mapping are available as separate tools. EELS acquisition takes energy-offset, slit, and dispersion arguments; we delegate quantitative spectral processing to HyperSpy [31] and the standard EELS analysis literature [32].

2.6 Deployment and bounded-execution model↩︎

The framework provides bounded execution at the tool boundary, which is the lowest layer of a layered safety model (Fig. 6). Schema validation rejects arguments that are out of physical or operational range before any adapter call is dispatched, but it is not equivalent to instrument safety, which requires additional layers: hardware interlocks (beam-blanker activation, column-vacuum interlocks, stage limit switches, detector-insertion interlocks); operator approval for high-risk operations through human-in-the-loop confirmation; permission tiers and authentication; specimen-specific beam-damage and sample-policy models; and facility-specific gates on acquisition limits, instrument-time windows, and data retention. The contribution of this work is to make the lowest layer explicit, auditable, and reusable.

The default transport is stdio, suitable for local agent orchestration and fully compatible with air-gapped operation. An optional streamable HTTP transport is provided for cases where a remote MCP-compatible client needs to reach the server through a facility-managed reverse proxy; this transport is not required for the core local-first deployment. Where a vendor API is host-process bound and an adapter therefore needs a bridge plugin, the bridge speaks a versioned JSON contract (nuance-mcp-bridge/1.0; see Appendix 6) over ZeroMQ [33], bound by default to tcp://127.0.0.1:5555 on the instrument PC. Facility-LAN exposure is an opt-in deployment mode protected by host firewall, allow-listed IP ranges, and an HTTPS reverse proxy. The threat model is explicit and limited: named principals are the operator and the LLM agent process on the workstation; the bridge trusts its caller; loopback is the default; LAN-bound deployments are the deployer’s responsibility to authenticate. In the loopback default, an attacker would require local code execution on the instrument PC to reach the bridge; under opt-in LAN exposure, a misconfigured allow-list or proxy could permit an unauthenticated host on the facility network to issue bridge calls, which is why authentication of LAN-bound deployments is left explicitly to the deployer. Replay protection and rogue-process scenarios are deferred to future work.

2.7 Validation protocol↩︎

Validation falls into three categories. Hardware-independent tests exercise the schema layer, the simulator-side dispatch, the adapter contract, the live-job lifecycle, and the bridge JSON contract; they require no microscope and are fully deterministic. Local-LLM integration tests drive the same tool surface end-to-end through a real local Ollama [34] model and a LangChain ReAct agent [35], [36], checking that the model selects, parameterises, and sequences tools correctly; outcomes depend on model sampling and are reported separately. Bridge structural tests verify the JSON contract that a vendor-specific adapter plugin must satisfy, and the persistence of live-job state across status/result/stop calls. The v0.2 suite contains 135 tests in total: 120 hardware-independent and 15 local-LLM integration. All 120 hardware-independent tests pass deterministically. The 15 local-LLM tests pass 12–15 of 15 across runs; the three reproducible failures occur under qwen2.5:7b and reflect model non-determinism. The breakdown is given in Table 2 and Fig. 7a.

3 Results↩︎

3.1 Method-induced properties of the protocol↩︎

Three properties of the method are worth isolating because they derive from design rather than from any one implementation.

Adapter-replaceable backend. Because every adapter implements the same MicroscopeAdapter contract (Fig. 3), the tool layer, the simulator, the live-job lifecycle, and the skill catalogue are unchanged when the adapter is swapped. Concretely, the reference simulator and any future vendor adapter share the same 30 typed tools, six skills (Table 1), and five live-job types (Fig. 4).

Auditable operational bounds. Every schema rejection is recorded as a structured error tied to the tool name and field (Fig. 2), and the behaviour is independent of how the model phrased the call, consistent with recent reproducibility-constrained large-action-model proposals [20]. As a deliberate negative control, a session in which the agent receives the instruction “tilt the stage to ninety-five degrees” issues a call to set_stage_position with alpha_deg=95.0. The call is rejected by Pydantic with the message “alpha_deg must be \(\leq 80\)”; no adapter call is made; the agent reports the rejection to the operator and proposes an in-range alternative.

Deterministic regression and CI. Because the simulator is a stateful, synchronous twin that shares the schema layer with the live path, the same test exercises the same code on every run. The hardware-independent suite therefore runs without microscope time and is reproducible across machines, which addresses the reproducibility gap that LLM-driven scientific software now faces [19], [20]. The same continuous-integration pipeline runs a project-status probe that re-measures the protocol surface — typed tools, skills, and live-job types — directly from the running server via tools/list and prompts/list on every push, so the surface counts reported in this manuscript cannot drift from the code without failing CI (Appendix 7). The complete 30-tool surface is enumerated in Table 6 (Appendix 7).

3.2 Automated validation outcomes↩︎

In v0.2, all 120 hardware-independent tests pass deterministically (Table 2, Fig. 7a). The block executes in approximately 18 s on commodity workstations and forms the basis of continuous integration. The 15 local-LLM integration tests pass between 12 and 15 of 15 across runs; under qwen2.5:7b on the testbed used here, three of the 15 tests fail reproducibly because the model occasionally re-orders steps in a multi-call sequence. The failures are recorded as part of the validation budget rather than suppressed, since they characterise the LLM rather than the protocol. The integration suite confirms that an off-the-shelf local model can select, parameterise, and sequence tools through the schema-bound surface in realistic microscopy task sequences (e.g.”acquire a HAADF image, start a radial-profile live job, report when the dominant ring shifts”). All operations in these sequences dispatch to the reference simulator rather than to a live instrument.

3.3 Same-hardware local-LLM probe↩︎

We do not attempt cross-platform LLM benchmarking, which is not meaningful across heterogeneous accelerators and on which the Berkeley Function-Calling Leaderboard provides a rigorous treatment [4]. As a single-run probe on a same-hardware Apple Silicon testbed (May 2026), the 15-test local-LLM integration suite was executed against five openly available models with tool-calling support in Ollama: llama3.2, qwen2.5, qwen2.5-coder, nemotron, and mistral. All five passed 15/15 on this run (Fig. 7b). For qwen2.5 this is at the upper end of the 12–15/15 range reported in Section 3.2, consistent with the model’s non-determinism across runs. Median per-test latencies separated into two regimes: roughly 1-3 s for four of the five models, and approximately 42 s for nemotron, with one outlier near 75 s. Models without tool-calling support in their current Ollama builds were excluded.

The qualitative point for the method is that the schema-bound tool surface is sharp enough that small open-weight models from four different providers (Meta, Alibaba, NVIDIA, and Mistral AI) can drive it on local hardware, without cloud dependency. The inter-model latency spread is wide enough that model choice matters in practice even when correctness does not. This single run is undersampled, and we caution against reading too much into it; replicated benchmarking with confidence intervals and BFCL-style methodology is future work.

4 Discussion↩︎

The method’s principal contribution is reproducible control semantics for agentic instrumentation at the level of the protocol itself. Schema-bound tool typing makes the boundary between agent intent and instrument action explicit and machine-checkable (Fig. 2); the vendor-neutral adapter contract keeps that boundary thin and replaceable (Fig. 3); the persistent live-job lifecycle exposes long-running, stateful analyses through the same typed interface as single-shot operations (Fig. 4); and the skill abstraction lifts recurring multi-tool procedures into MCP-prompt-registered artefacts that are discoverable through ordinary protocol calls (Fig. 5). Together these four choices give a runtime in which agent traces are analyzable, regressions are deterministic, and the entire layer above the adapter ports across vendors. Recent “thinking microscope” framings cast the instrument itself as an iterative reasoner over its own data [13]; the contribution here is the protocol surface on which that view can be made operational and auditable.

The shape of the contribution differs from a hardware demonstration. We do not claim improved imaging or new physics. We claim that once the control surface is right, agentic workflows demonstrated piecemeal in chemistry [1]–[3], scanning probes [5], [8], [10], and TEM [11], [12], [26], [37] can be composed out of typed tools with bounded execution, declarative skills, and a reproducible runtime.

The 2026 Nature reports on Co-Scientist, Robin, and Empirical Research Assistance sharpen this point rather than weakening it [14]–[16]. They show that multi-agent systems can already operate over scientific knowledge, code, candidate hypotheses, and human-executed experiments. Microscopy adds a harder control layer: the next action must be executable on the bound instrument, safe for the loaded sample, meaningful under the current calibration state, and interpretable under a measurement model. In that setting a language-model agent needs a calibrated digital twin of the instrument and sample workflow rather than a chat interface to a vendor API. The simulator and schema-bound MCP surface presented here move toward that digital-twin layer by turning instrument actions into typed, testable, replayable objects that an agent can reason about before a physical experiment is attempted.

The simulator has independent value. Because it shares the schema layer with any live adapter, hardware-independent tests cover the same code that would run against an instrument, so workflow logic can be developed and audited without instrument time. This is a recurring obstacle in microscope-software development [38]; sharing the schema with the simulator removes it for the validation suite.

The deployment model (Fig. 6) makes the security boundary explicit. The bridge trusts its caller on loopback; LAN exposure is opt-in and the deployer’s responsibility. This is a deliberate scope limit: the protocol provides the typed, auditable lowest layer of a layered safety model, but does not substitute for hardware interlocks, operator approval, or facility policy. The vendor adapter comparison in Table 7 (Appendix 8) shows that the bridge pattern generalises across four major vendor platforms with different scripting surfaces and host-binding mechanisms.

Positioning relative to concurrent work↩︎

Table 3 situates the method against the closest concurrent systems. The dimensions tabulated are the design goals of this work plus two further axes; the comparison is therefore one in which competing systems are scored against features this work set out to provide, and a “” marks a feature not reported in the cited source rather than a deficiency of the system on its own terms. To keep the comparison even-handed, we include a live-instrument-validation row, an axis on which this work does not lead: AILA [10] and [12] report demonstrations on physical instruments, whereas nuance-mcp is validated only in software here (Section 3; Limitations). TEM Agent [11] demonstrated that MCP-mediated TEM workflows are feasible but exposes a vendor-specific tool surface without schema-bound validation or an adapter abstraction. stemOrchestrator [39] provides a hardware-abstraction layer for STEM detectors and cameras but operates below the MCP protocol level and does not expose typed tools or declarative skills to an LLM agent. AILA [10] automates atomic-force microscopy through LLM agents with real-instrument demonstrations but uses a bespoke tool interface rather than MCP. The broader MCP-for-science survey of [40] and the MCP-Bench evaluation framework [41] confirm growing interest in standardised tool protocols for scientific workflows, while [42] have extended MCP to chemical process simulation.

The combination of schema-bound validation, a vendor-neutral adapter contract, a persistent live-job lifecycle, and a prompt-registered skill abstraction is, to our knowledge, not jointly present in the systems we surveyed. The principal gap relative to production-grade platforms is security: commercial offerings provide OAuth 2.0, 21 CFR Part 11 compliance, and fine-grained audit trails, whereas the bridge security model presented here is intentionally minimal (Section 2.6).

Limitations↩︎

The validation in this manuscript is software-only: every claim is demonstrated against the physics-plausible simulator that implements the same MicroscopeAdapter contract. Live-instrument demonstrations on commercial scientific hardware are intentionally out of scope here; they are described separately under the corresponding intellectual-property disclosure [23]. Because that live-instrument validation is withheld under the disclosure, on-hardware performance of the method cannot be independently assessed from this manuscript; the software-only scope is therefore both a deliberate design choice and a constraint on what readers can conclude. The local-LLM probe in Section 3.3 is a single-run result across five models; reported pass rates and latencies are not accompanied by confidence intervals and should be treated as indicative rather than quantitative. The bridge security model is intentionally minimal: loopback by default, with LAN exposure left to the deployer to authenticate; replay protection and rogue-process scenarios are not currently addressed. The simulator outputs are physically plausible but are not calibrated quantitative ground truth. Each of these scope limits maps onto a near-term work item rather than a challenge to the method itself.

5 Conclusion↩︎

We have described a vendor-agnostic method for connecting local LLM agents to scientific instruments at the level of the tool protocol, and presented its reference implementation as a software-only validation study. The method combines a schema-bound tool surface (Fig. 2), a vendor-neutral adapter contract (Fig. 3) paired with a physics-plausible simulator, a persistent live-processing job lifecycle (Fig. 4), and a skill abstraction implemented through MCP prompts (Fig. 5, Table 1) that composes typed tools into reusable multi-step protocols. The reference implementation nuance-mcp exposes 30 typed tools, 5 live-job types, and 6 skills. All validation reported here is software-only, against the reference simulator: all 120 hardware-independent tests pass deterministically, and the 15 local-LLM integration tests pass 12–15 of 15 across runs, with the three reproducible failures attributable to model non-determinism (Table 2). The deployment model (Fig. 6) makes the security boundary explicit and the layered safety model auditable. The MIT-licensed source, simulator, and test suite are available at https://github.com/NUANCE-IT/nuance-mcp.

The protocol is intended as a working foundation for closed-loop and autonomous microscopy research: schema-checked agent traces, reproducible regression suites, and adapter-portable skills can be composed against it without re-solving the host-process binding, governance, and bounded-execution frictions identified in Section 1.

Competing interests↩︎

The authors declare the following competing interests. Certain aspects of the broader universal MCP framework for scientific instrumentation described in this work are the subject of a pending U.S. patent application filed on 13 July 2026. Related technology was previously disclosed to Northwestern University as Invention Disclosure Disc-ID-25-05-22-002 (technology ID 2025-136). The inventors are R. dos Reis and V. P. Dravid, and Northwestern University is the assignee. The authors may financially benefit from patents or licensing agreements arising from this intellectual property.

Author contributions↩︎

R.d.R.conceived the architecture, designed and implemented the schema-bound tool protocol, the adapter contract, the bridge specification, the simulator, the live-job lifecycle, and the skill catalogue, implemented the full software stack and validation suite, and drafted the manuscript. V.P.D.supervised the project, contributed scientific framing, and revised the manuscript. All authors read and approved the final manuscript.

Acknowledgments↩︎

This work made use of the NUANCE facility (RRID: SCR_026361) at Northwestern University, which has received support from the IIN and Northwestern’s MRSEC program (NSF DMR-2308691). The authors acknowledge the open-source communities behind FastMCP, MCP, Ollama, LangGraph, faster-whisper, and the broader scientific-Python ecosystem. The authors thank Benjamin Miller, Anahita Pakzad, and Stephen Mick from Gatan/AMETEK for their early testing and for discussions regarding the implementation of NUANCE-MCP within the GMS environment.

Data availability↩︎

No experimental dataset is required to reproduce the software validation reported here. All results in this manuscript are reproducible against the physics-plausible simulator that ships with the source distribution and the accompanying test suite.

Code availability↩︎

Source code, tests, examples, and documentation are available under the MIT License at: https://github.com/NUANCE-IT/nuance-mcp.

Appendix↩︎

6 Bridge protocol nuance-mcp-bridge/1.0↩︎

This appendix specifies the versioned JSON contract that host-process-bound adapter implementations must satisfy. The contract is vendor-neutral; specific adapter implementations and their results are intentionally out of scope for this manuscript. The bridge speaks a versioned JSON contract over a ZeroMQ REQ/REP pair. Each request carries a version envelope (v: "nuance-mcp-bridge/1.0"), a method name matching an MicroscopeAdapter method, and a parameter object. The response carries a status code (ok, error, unsupported), a result payload, and a structured error string when appropriate. The first call after connect must be hello, which negotiates the protocol version and returns the bridge’s capability list; clients reject unknown version prefixes. The bridge pumps the vendor’s GUI event loop at least every 100 ms while waiting, so the host application remains responsive during long polls. Image payloads use a name/shape/data_dtype/ data_b64 encoding; spectra use a parallel counts_b64/energy_eV pair. New optional fields can be added at any time; receivers must ignore unknown fields. Breaking changes increment the version prefix and trigger an explicit handshake failure with older partners. The full schema and the 30-tool surface that adapters expose are documented in the docs/spec/ directory of the source repository.

7 Evolution from v0.1.0 to v0.2↩︎

The protocol-level method described in this manuscript did not arrive fully formed. Three release lines preceded it. We summarise the trajectory here because each step illustrates a design decision that the current method makes explicit.

7.0.0.1 v0.1.0: monolithic server, ad-hoc validation.

The first release shipped as a single FastMCP server with 21 tools hand-written against one host-process API. Argument validation existed but was scattered through individual tool bodies; out-of-range values often produced opaque vendor-side errors rather than structured rejections. There was no abstract base class: the tool layer talked to the instrument through a direct import. The simulator (a separate file with hand-written modality kernels) existed but was selected by an environment flag rather than as a peer adapter, and it did not share the validation path with the live code. There were no skills; multi-step procedures lived in example scripts that operators copied and adjusted by hand.

7.0.0.2 v0.1.1: centralized schema, host-process bridge.

Two changes consolidated the v0.1.0 ad-hocery. First, every tool gained a Pydantic v2 input model and the dispatcher validated those models before any vendor call, which made rejection paths structured and prompt-independent for the first time. Second, the host-process binding was crossed by a small JSON-over-ZeroMQ “bridge” plugin that ran inside the vendor process; the FastMCP server became an ordinary Python process again. The bridge was unversioned: client and server agreed on the message shape by implicit convention, which became a maintenance problem as the schema grew.

7.0.0.3 v0.1.2: live-processing lifecycle, expanded tool set.

This release introduced the persistent live-processing job abstraction - start, poll, fetch result, stop - as four typed tools sharing a common state machine, with five live-job types (radial_profile, difference, fft_map, filtered_view, maximum_spot_mapping). The tool count grew from 21 to 30 with the addition of fine-grained optics, beam, and detector controls that earlier releases had bundled into coarser surfaces. The automated test suite grew in step from 78 to 107 deterministic tests; the local-LLM integration block (15 additional tests exercising the same tool surface through Ollama) is added later in v0.2 (Section 3.2). The simulator still lived alongside the host-process binding rather than as a peer.

7.0.0.4 v0.2.0: protocol-level method, adapter contract, versioned bridge, skills.

The current release reorganises the preceding pieces around four protocol-level abstractions described in Section 2. The schema layer becomes the single validation surface (no per-tool validation logic remains). The MicroscopeAdapter abstract base class is introduced as the only surface the schema layer talks to; the simulator now implements that contract as a peer adapter rather than as a fallback path. The bridge protocol is versioned (nuance-mcp-bridge/1.0) and includes an explicit hello handshake that negotiates capability vocabulary. Six declarative skills are added as MCP prompts that compose typed tools into reusable multi-step protocols (Table 1). Default network exposure is tightened from all-interfaces to loopback only, and a layered safety model is documented so that the schema layer is no longer conflated with instrument safety (Fig. 6). Table 4 summarises the trajectory.

7.0.0.5 Development tooling - reproducible project status.

Independently of the protocol surface, the v0.2 line adds a single-source-of-truth project-status probe (scripts/project_status.py). The probe measures the live protocol surface - typed tools, skills, live-job types, and capabilities - directly from the running server through tools/list and prompts/list rather than from documentation, runs the test suite, reads version-control state, and emits a milestone roadmap. A single implementation is surfaced three ways: as a command-line report, as a Project status probe step in continuous integration, and as a /goals slash command for editor-integrated agents. It is deliberately not part of the instrument-control tool surface - it adds nothing to the 30 typed tools - but it is a direct consequence of the schema-bound framing: because the tool surface is introspectable, project status becomes a measurement rather than a claim, and the surface figures quoted throughout this manuscript are re-checked against the code on every continuous-integration run.

Two threads run through the table that are worth calling out explicitly. First, every later abstraction was prefigured as ad-hoc code in an earlier release: validation existed before it was centralized, bridge messaging existed before it was versioned, the simulator existed before it was made a peer adapter. The protocol work was about naming these surfaces, not about inventing them from scratch. Second, the test count grows monotonically with the surface, and the v0.2 line is the first to expose model non-determinism by including a local-LLM integration block. Reporting those failures as part of the validation budget — rather than as flakiness to be hidden — is a deliberate consequence of the schema-bound framing.

For completeness, and to let a reader verify the surface counts quoted throughout the manuscript against the code, Table 5 enumerates the full 24-family capability vocabulary and Table 6 enumerates the 30 typed tools. Both tables are generated from the same tools/list and prompts/list introspection used by the continuous-integration probe.

4pt

4pt

8 Implementing adapters for vendor platforms↩︎

This appendix gives implementation guidance for adapting the MicroscopeAdapter contract (Fig. 3) to vendor-specific platforms. The aim is to make the bridge pattern reproducible without binding the published method to any one vendor SDK; complete adapter implementations and live-instrument validation are reported separately under the corresponding intellectual-property disclosure [23].

8.1 Categorising the host binding↩︎

Vendor scripting surfaces fall into three categories with respect to the host-process binding friction of Section 1. Table 7 summarises the categorisation against four representative platforms.

8.1.0.1 In-process Python.

The vendor exposes a Python module that loads only inside the running acquisition application (e.g.Gatan GMS, Thermo Fisher Velox). The MCP server cannot import the module directly; the adapter requires a bridge plugin that runs as a script inside the vendor application and speaks the nuance-mcp-bridge/1.0 contract over loopback.

8.1.0.2 Socket-based service.

The vendor provides a Python client library (e.g.JEOL pyJEM against TEMServer) that connects to a long-running service over a TCP socket. The MCP server can either import the client library directly when it runs on the instrument PC, or relay through a ZeroMQ proxy when it runs on a separate workstation.

8.1.0.3 COM/.NET automation.

The vendor exposes a COM or .NET automation interface with no native Python binding (e.g.Hitachi ExTOPE). The adapter requires both a process crossing and a language crossing; the recommended pattern is a small .NET service that wraps the automation objects and speaks the bridge contract over a ZeroMQ socket.

8.2 Bridge plugin skeleton↩︎

For host-bound platforms (in-process Python and COM/.NET automation), the bridge plugin follows a common pattern, sketched here in a vendor-neutral form. The plugin (i) binds a ZeroMQ REP socket to loopback, (ii) enters a poll-and-dispatch loop, (iii) pumps the host application’s event loop between polls so the GUI remains responsive, and (iv) translates each incoming bridge request into a call against the vendor SDK. Listing [lst:bridge] shows the structure.

# bridge_plugin.py (runs inside the vendor host process)
import zmq, json, base64
# import vendor SDK here, e.g. DigitalMicrograph

ctx = zmq.Context()
sock = ctx.socket(zmq.REP)
sock.bind("tcp://127.0.0.1:5555")

while True:
    vendor_pump_events()  # keep host GUI responsive
    if sock.poll(100):    # 100 ms poll cadence
        msg = json.loads(sock.recv())
        # Negotiate on first call:
        if msg["method"] == "hello":
            sock.send(json.dumps({
                "status": "ok",
                "result": {
                    "version": "nuance-mcp-bridge/1.0",
                    "capabilities": ADAPTER_CAPABILITIES,
                },
            }).encode())
            continue
        result = vendor_dispatch(msg["method"], msg["params"])
        sock.send(json.dumps({
            "status": "ok", "result": result
        }).encode())

For socket-based vendor services (e.g.JEOL pyJEM against TEMServer), the bridge plugin is unnecessary: the Python-side MicroscopeAdapter subclass calls the vendor client library directly. The same listing applies if the MCP server runs on a separate workstation and the vendor client library is imported through a ZeroMQ relay.

8.3 Implementing the Python-side adapter↩︎

On the server side, each vendor adapter is a subclass of MicroscopeAdapter. The subclass (i) declares the capability set the bound instrument actually supports, (ii) implements only those methods, and (iii) forwards each call either through the bridge socket (for host-bound platforms) or directly through the vendor client library (for socket-based platforms). Listing [lst:adapter] sketches the pattern, with the bridge-roundtrip helper factored out so that adapters for either host-binding category share the bulk of their implementation.

# vendor_adapter.py
import zmq, json
from nuance_mcp.adapter import MicroscopeAdapter

class VendorAdapter(MicroscopeAdapter):
    capabilities = {
        "tem", "stem", "stage", "stage.tilt",
        "optics", "detectors",
        # ... whatever the bound instrument actually supports
    }

    def __init__(self, endpoint="tcp://127.0.0.1:5555"):
        ctx = zmq.Context()
        self._sock = ctx.socket(zmq.REQ)
        self._sock.connect(endpoint)
        # Handshake on connect:
        self._bridge_call("hello", {})

    def _bridge_call(self, method, params):
        self._sock.send(json.dumps(
            {"v": "nuance-mcp-bridge/1.0",
             "method": method,
             "params": params}).encode())
        return json.loads(self._sock.recv())["result"]

    async def set_stage_position(self, x_um=None, y_um=None,
                                 z_um=None, alpha_deg=None,
                                 beta_deg=None):
        return self._bridge_call("set_stage_position", {
            "x_um": x_um, "y_um": y_um, "z_um": z_um,
            "alpha_deg": alpha_deg, "beta_deg": beta_deg,
        })

    async def acquire_tem_image(self, exposure_s=1.0,
                                binning=1, roi=None):
        return self._bridge_call("acquire_tem_image", {
            "exposure_s": exposure_s,
            "binning": binning, "roi": roi,
        })

8.4 Declaring capabilities and testing the adapter↩︎

A new adapter declares only the capability families it actually supports; the default implementations inherited from MicroscopeAdapter raise CapabilityUnavailable for the rest, which the tool layer surfaces to the agent as a structured {"status": "UNSUPPORTED"} response (Section 2.2). Skills that require an undeclared capability fail at their opening get_capabilities check rather than mid-sequence.

The hardware-independent test suite is reusable across adapters: it exercises the schema layer, the dispatch path, the live-job lifecycle, and the bridge JSON contract through the MicroscopeAdapter interface and does not import any vendor SDK. A new adapter can therefore be developed against the reference simulator first (which shares the schema layer), then connected to the bridge, and finally validated on the live instrument once the bridge round trip is observed.

References↩︎