Test LLMs on real tasks. Compare models side-by-side.
Test LLMs on real tasks. Compare models side-by-side.
BenchLocal
Test LLMs on real tasks. Compare models side-by-side.
Website · Download · Watch demo · Build a Bench Pack
BenchLocal is a local-first desktop app for running, comparing, and managing installable LLM Bench Packs against local or remote models.
Official Bench Packs today:
BenchLocal owns the shared desktop runtime:
BenchLocal can expose a local agent surface so AI agents and automation tools can control benchmark workflows while the desktop UI stays live.
Enable it from Settings > Agent Access. The app will show:
The HTTP API uses JSON commands for actions such as listing Bench Packs, managing providers and models, creating tabs, selecting models, refreshing availability, starting runs, resuming runs, retrying results, and stopping active runs. Live progress is available through Server-Sent Events at /v1/events.
MCP-capable agents can connect to /mcp with the same bearer token and use standard benchlocal_* tools plus BenchLocal state resources. This is the preferred integration path for agents that support tool calls.
See docs/agent-control-api.md for endpoint details, MCP tools/resources, safety rules, and the extension checklist for adding future UI features to the agent surface.
Each Bench Pack owns its benchmark behavior:
app/
Electron app shell, desktop UI, main process, preload, rendererpackages/benchlocal-core
shared protocol, config, workspace, and theme typespackages/benchlocal-sdk
authoring helpers for Bench Pack repospackages/benchpack-host
host-side install, inspection, verifier, and run orchestration logicthemes/
built-in desktop themesscripts/
local macOS release helpersdocs/
packaging and release docsnpm run build
compile the app and workspace packages for developmentnpm run pack
compile and package the production desktop app, including DMG and ZIP artifactsnpm run build:dir
compile and produce an unpacked local app bundlenpm run build:win
compile and package unsigned Windows NSIS and ZIP artifactsnpm run build:linux
compile and package Linux AppImage and tar.gz artifactsnpm run release:all
build the signed macOS release plus Windows and Linux desktop artifacts in one commandMIT
No open issues yet, or sync has not completed.