BenchWeave
Star on GitHub
Status: simulator-first PoC through its acceptance gate
Open source · MIT

The gateway that weaves your test bench together

BenchWeave connects instruments and devices under test through reusable plugins and typed profiles — coordinating bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.

gateway REST · MCP instrument plugin DUT profile registry shared evidence pending verified
fig. 1 — one run: admit · command · measure · verify
“

Wouldn't it be cool to have AI help test hardware?

Where it came from

BenchWeave started as a conversation between two friends. The question stuck, and it sharpened into a harder one: what would it take for an AI to drive a real test bench safely — instruments, a device under test, and the guard rails written down as contracts rather than carried in someone's head?

The plan is deliberately small: get a working proof of concept, keep all of it open source, and open it to the community. Beyond that, the project is a living thought experiment — the architecture, the standards and the code are the working-out, published as they happen, to see where the idea leads. Everything on this site says what has been built and what hasn't.

Talk bench with us

Questions, plugin builds, standards arguments and works in progress — BenchWeave has a Discord, and the invite is permanent.

Join the Discord →

What surprised us

A device plugin doesn't need BenchWeave

A plugin is a descriptor plus an adapter against structural interfaces, with no runtime dependency on the SDK or the gateway. The DPS-150 plugin builds and tests unchanged outside this checkout and has driven the real supply. So a standalone MCP server that implements the host side can control a device in its own right — the plugin is a standard way of saying what a device is and how it talks. What that path doesn't give you is the bench safety model: ownership, protection deadlines and verified endings live in the gateway.

Greenfield hardware can start from the firmware

For your own board, the same workflow starts one step earlier: Build my own device firmware gives you a shared session instruction and stage prompts to get from board and intended behaviour to firmware, descriptor and adapter that agree, with the firmware kept in the plugin's own firmware/ beside its toolchain and compatibility record. The prompts authorise software work only; flashing and energising stay with you.

Build your instrument and workbench with AI

Five-step paths for device plugins and custom firmware — development, testing, review, and release prompts, checked against the contracts by the AI device reviewer before anything ships.

Develop your device with AI

The plan

Four commitments from the architecture, not four feature bullets. The first PoC is simulator-first; the planned hardware MVP targets the FNIRSI DPS-150, with ESP32 as the provisional controller family.

Connect

Instruments and devices under test through reusable plugins and typed device profiles.

Share

Profiles and implementations through a central registry, with local admission and offline execution.

Coordinate

Bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.

Expose

A consistent REST and MCP interface for applications and coding agents.

Where it stands

Working now. The simulator-first PoC is through its acceptance gate: durable run and lease state with kill-recovery, procedure admission and protection, 20 REST routes and 17 MCP tools on one core-operations seam, and an operator CLI. Standards now behave like dependencies: plugins pin the standard versions they admit against, admission checks those pins, and a resolver lists, pins, upgrades and explains what a pin resolves to. Released versions serve side by side, development heads pin by name, and every promotion from a head to a released version is recorded. The SDK scaffolds an independent plugin project and release CI builds one outside the checkout and drives it through the gateway on mock transport. The DPS-150 plugin — hand-written, zero runtime dependencies — has driven the real supply; it is read-only and claims no PSU profile yet.

Next. Gateway and devices talking through the registry: the gateway side of the registry contract is done (signed catalogue, admission with a content-addressed cache, activation, an unsigned dev loop), and a catalogue of published, signed releases now exists as a git-native registry repository of record with its own catalogue site — but no hosted registry front end and no device-install command exist, so installation stays local admission. The OTDP bridge's remaining scope is profile scheduling on a native async host. The gateway's first operator web UI will be server-rendered, with the React reference renderer retired in a gated cutover. Hardware qualification and unattended bench operation remain separate, commissioned work.

Delivery plan

Plugins

Plugin releases are published and signed in a git-native registry repository of record — not a hosted front end. The catalogue is generated from that repository's records and served from its own site; this page carries no copy of it, so what you reach is always the published state. Installation is local admission: publication never authorizes control.

The plugin catalogue

Every published release with its version, digest, compatibility and advisories — searchable, kind-tagged, and stamped with the commit it was generated from.

Browse the catalogue →

What the catalogue offers is generated from published releases only: a yanked or revoked release drops out at the next regeneration — what is absent there is absent everywhere.

Standards

The versioned contracts everything else is built against — treated as agreements, not informal notes. Each is independently versioned, digest-frozen, and validated in CI; the SDK vendors the same bytes.

OTDP

v0.2.3

Open test-device protocol — how the gateway talks to instrument and DUT plugins: device descriptors, measurement model, device classes, extension contract.

Read the specification →

Registry

v0.1.3

Plugin and device-profile registry — admission, sharing, release manifests and offline execution.

Read the specification →

Execution

v0.2.0

Bounded test-procedure execution — bench, commissioning, procedure and run-record schemas; ownership, evidence and safe endings.

Read the contract →

Interface

v0.1.0

The REST and MCP surface exposed to applications and coding agents — OpenAPI document, MCP tool catalogue, operation catalog and review scenarios.

Read the contract →

Plugin UI

v0.4.0

Plugin presentation contract — bindings, presets and envelopes for the workbench UI.

Validation report →

Plugin UI preview

v0.2.0

Preview wire documents and fixtures for simulation-only plugin UI rendering.

Schemas on GitHub →

Plugin UI class reference

v0.2.0

One reference presentation per device class, written as data — the single source for derived plugin UIs.

Schemas on GitHub →

Governance, the standards manifest and the digest-locked corpus manifest live at standards/GOVERNANCE; the rendered standards index lists every prose companion and validation report.

Documentation

Built with Great Docs from the repository's own guides, architecture record and standards. Unversioned until the first tagged release; every page links back here.

SDK

The plugin developer SDK lives in its own repository with its own site, CI and release cycle. It is mounted here at packages/sdk as a git submodule.

benchweave-sdk

Build an external device plugin without importing gateway internals. Python 3.13+ · SDK 0.15.0 · OTDP 0.2.3 baseline. Offline authoring and conformance tools: scaffold, validate, mock, preview.

# install
pip install benchweave-sdk
# scaffold an independent plugin project
benchweave-sdk new plugins/acme/model100 --package benchweave_acme_model100

Built with AI

BenchWeave is built by AI coding agents under human direction. For anyone curious how that works in practice, this page is the honest record: which model did which job, and the open tooling around them. The stack is deliberately mixed — no single vendor wrote this project.

Architecture brainstorming

GPT Astra

The origin conversations — turning “wouldn't it be cool to have AI help test hardware?” into a first architecture worth committing to.

The code

GLM-5.3

The working model behind the gateway, the SDK, the plugins and the tests. Spec-driven and test-first: every slice landed as a failing test made green.

Z.ai →

This website

Anthropic Fable

Designed and built on free credits — which have since expired.

Anthropic →

Mixed models, one rule: nothing lands on trust. Every change, AI-written or not, passes review and the CI gates — pytest, ruff, mypy strict — before it reaches main.