The gateway that weaves your test bench together
BenchWeave connects instruments and devices under test through reusable plugins and typed profiles — coordinating bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.
Wouldn't it be cool to have AI help test hardware?
Where it came from
BenchWeave started as a conversation between two friends. The question stuck, and it sharpened into a harder one: what would it take for an AI to drive a real test bench safely — instruments, a device under test, and the guard rails written down as contracts rather than carried in someone's head?
The plan is deliberately small: get a working proof of concept, keep all of it open source, and open it to the community. Beyond that, the project is a living thought experiment — the architecture, the standards and the code are the working-out, published as they happen, to see where the idea leads. Everything on this site says what has been built and what hasn't.
Talk bench with us
Questions, plugin builds, standards arguments and works in progress — BenchWeave has a Discord, and the invite is permanent.
What surprised us
A device plugin doesn't need BenchWeave
A plugin is a descriptor plus an adapter against structural interfaces, with no runtime dependency on the SDK or the gateway. The DPS-150 plugin builds and tests unchanged outside this checkout and has driven the real supply. So a standalone MCP server that implements the host side can control a device in its own right — the plugin is a standard way of saying what a device is and how it talks. What that path doesn't give you is the bench safety model: ownership, protection deadlines and verified endings live in the gateway.
Greenfield hardware can start from the firmware
For your own board, the same workflow starts one step earlier: Build my own device firmware gives you a shared session instruction and stage prompts to get from board and intended behaviour to firmware, descriptor and adapter that agree, with the firmware kept in the plugin's own firmware/ beside its toolchain and compatibility record. The prompts authorise software work only; flashing and energising stay with you.
Build your instrument and workbench with AI
Five-step paths for device plugins and custom firmware — development, testing, review, and release prompts, checked against the contracts by the AI device reviewer before anything ships.
The plan
Four commitments from the architecture, not four feature bullets. The first PoC is simulator-first; the planned hardware MVP targets the FNIRSI DPS-150, with ESP32 as the provisional controller family.
Connect
Instruments and devices under test through reusable plugins and typed device profiles.
Share
Profiles and implementations through a central registry, with local admission and offline execution.
Coordinate
Bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.
Expose
A consistent REST and MCP interface for applications and coding agents.
Where it stands
Working now. The simulator-first PoC is through its acceptance gate: durable run and lease state with kill-recovery, procedure admission and protection, 20 REST routes and 17 MCP tools on one core-operations seam, and an operator CLI. Standards now behave like dependencies: plugins pin the standard versions they admit against, admission checks those pins, and a resolver lists, pins, upgrades and explains what a pin resolves to. Released versions serve side by side, development heads pin by name, and every promotion from a head to a released version is recorded. The SDK scaffolds an independent plugin project and release CI builds one outside the checkout and drives it through the gateway on mock transport. The DPS-150 plugin — hand-written, zero runtime dependencies — has driven the real supply; it is read-only and claims no PSU profile yet.
Next. Gateway and devices talking through the registry: the gateway side of the registry contract is done (signed catalogue, admission with a content-addressed cache, activation, an unsigned dev loop), and a catalogue of published, signed releases now exists as a git-native registry repository of record with its own catalogue site — but no hosted registry front end and no device-install command exist, so installation stays local admission. The OTDP bridge's remaining scope is profile scheduling on a native async host. The gateway's first operator web UI will be server-rendered, with the React reference renderer retired in a gated cutover. Hardware qualification and unattended bench operation remain separate, commissioned work.
Plugins
Plugin releases are published and signed in a git-native registry repository of record — not a hosted front end. The catalogue is generated from that repository's records and served from its own site; this page carries no copy of it, so what you reach is always the published state. Installation is local admission: publication never authorizes control.
The plugin catalogue
Every published release with its version, digest, compatibility and advisories — searchable, kind-tagged, and stamped with the commit it was generated from.
What the catalogue offers is generated from published releases only: a yanked or revoked release drops out at the next regeneration — what is absent there is absent everywhere.
Standards
The versioned contracts everything else is built against — treated as agreements, not informal notes. Each is independently versioned, digest-frozen, and validated in CI; the SDK vendors the same bytes.
OTDP
v0.2.3Open test-device protocol — how the gateway talks to instrument and DUT plugins: device descriptors, measurement model, device classes, extension contract.
Read the specification →Registry
v0.1.3Plugin and device-profile registry — admission, sharing, release manifests and offline execution.
Read the specification →Execution
v0.2.0Bounded test-procedure execution — bench, commissioning, procedure and run-record schemas; ownership, evidence and safe endings.
Read the contract →Interface
v0.1.0The REST and MCP surface exposed to applications and coding agents — OpenAPI document, MCP tool catalogue, operation catalog and review scenarios.
Read the contract →Plugin UI
v0.4.0Plugin presentation contract — bindings, presets and envelopes for the workbench UI.
Validation report →Plugin UI preview
v0.2.0Preview wire documents and fixtures for simulation-only plugin UI rendering.
Schemas on GitHub →Plugin UI class reference
v0.2.0One reference presentation per device class, written as data — the single source for derived plugin UIs.
Schemas on GitHub →Governance, the standards manifest and the digest-locked corpus manifest live at standards/GOVERNANCE; the rendered standards index lists every prose companion and validation report.
Documentation
Built with Great Docs from the repository's own guides, architecture record and standards. Unversioned until the first tagged release; every page links back here.
Operator guide
Install, run and care for a gateway — every command verified against the shipped CLI.
Device developer guide
Device creation, gateway hosting and shared packages, with the AI task template and human review checklist.
Develop your device with AI
Five-step paths for device plugins and custom firmware, with reusable prompts.
AI device reviewer
Role and checklist for assessing candidate integrations against the contracts.
Development and CI
uv setup, local checks and the GitHub workflows.
Architecture v1.5
The admitted architecture baseline.
Decisions
The decision record behind the baseline.
Compatibility
Compatibility record and matrix — what is reachable today, and what is disclosed as out of scope.
PoC/MVP PRD
Product requirements for the simulator-first PoC and hardware MVP.
Delivery plan
Work packages and sequencing.
CLI reference
The benchweave command tree — setup, serve, start, stop, restart, status, service, ui-login, verify, demo, report, evidence, retention, dispose, backup, restore — generated from --help.
Standards
Every prose companion and validation report, rendered beside its machine corpus.
Changelog
Generated by git-cliff from conventional commits.
FNIRSI DPS-150 protocol
Protocol evidence for the planned hardware MVP target.
DPS-150 compatibility record
Tested models, firmware and limitations.
ESP32 selection
Why ESP32 is the provisional controller family.
Agent skill
The benchweave skill under .well-known/, for coding agents working on the gateway.
llms.txt
Plain-text index of the docs; llms-full.txt carries every page.
SDK
The plugin developer SDK lives in its own repository with its own site, CI and release cycle. It is mounted here at packages/sdk as a git submodule.
benchweave-sdk
Build an external device plugin without importing gateway internals. Python 3.13+ · SDK 0.15.0 · OTDP 0.2.3 baseline. Offline authoring and conformance tools: scaffold, validate, mock, preview.
pip install benchweave-sdk
# scaffold an independent plugin project
benchweave-sdk new plugins/acme/model100 --package benchweave_acme_model100
Built with AI
BenchWeave is built by AI coding agents under human direction. For anyone curious how that works in practice, this page is the honest record: which model did which job, and the open tooling around them. The stack is deliberately mixed — no single vendor wrote this project.
Architecture brainstorming
GPT AstraThe origin conversations — turning “wouldn't it be cool to have AI help test hardware?” into a first architecture worth committing to.
The code
GLM-5.3The working model behind the gateway, the SDK, the plugins and the tests. Spec-driven and test-first: every slice landed as a failing test made green.
Z.ai →This website
Anthropic FableDesigned and built on free credits — which have since expired.
Anthropic →MuninnDB
A self-hosted long-term memory server. This project keeps its own local vault, so hard-won context survives the session that earned it.
Gortex
A local code-intelligence daemon. The codebase is graph-indexed, and reads and edits route through it — nothing is touched blind.
Mixed models, one rule: nothing lands on trust. Every change, AI-written or not, passes review and the CI gates — pytest, ruff, mypy strict — before it reaches main.