在浏览器中有一个 iOS 模拟器,您可以进行检查
Live iOS Simulator preview with React Native inspector, accessibility-tree driven tools, and an MCP bridge. Drive iOS apps from your browser or an AI agent — see the live screen, click any element to get its real source code, tap / swipe / type without moving the macOS cursor.
Built for three audiences at once:
App.tsx:42 source lookups.…| Cursor-free input | Every tap / swipe / key goes through axe → FBSimulatorHID → CoreSimulator. No macOS cursor movement, no Simulator.app focus requirement, pixel-exact at any zoom. |
| Drive any iOS app | Taps, drags, mouse-wheel scroll, multitask, home, lock, keyboard passthrough. Works on Settings, Maps, Messages — not just RN apps. |
| iOS accessibility tree | Populated from axe describe-ui on boot. Every on-screen element with its label, role, and frame — before you click anything. |
| React component tree | For RN apps running the metro plugin, inspect any rendered component and get the full fiber stack with real source paths (React 19 _debugStack + Metro /symbolicate). |
| Real source code panel | Select any component and the Properties panel fetches the actual file and renders a code window around the target line. Works for your code, RN's Libraries, and Expo shims. |
| MCP bridge | 15 tools: sim_tree, sim_tap_by_label, sim_swipe, sim_type, sim_inspect, sim_source, sim_screenshot, and more. Every browser capability exposed to AI agents. |
macOS with Xcode + at least one iOS simulator installed.
Node ≥ 18 (or Bun — scripts are bun-friendly).
Rust toolchain (for building sim-server once).
axe — the cursor-free input driver. One-line install:
brew install cameroncooke/axe/axeA React Native / Expo app with the agent-simulator metro plugin, only if you want React component inspection. Driving, screenshots, and the iOS accessibility tree all work against any iOS app.
# 1. Clone & install
git clone https://github.com/jkneen/agent-simulator
cd agent-simulator
bun install
# 2. Build the Rust sim-server (once)
(cd sim-server && cargo build --release)
# 3. Build the web UI (once, or after UI changes)
(cd web-app && bun install && bun run build)
# 4. Boot any simulator yourself, then start the server
xcrun simctl boot "iPhone 17 Pro" || true
open -a Simulator
bun start
# 5. Open the UI
open http://localhost:3200The UI will attach to whatever iOS simulator is booted. Open Settings, Maps, Messages, or any app — you can drive it immediately. The iOS layer tree on the left is populated from the accessibility API; the React tree is empty until you load an RN app with the inspector bridge (see below).
We ship an Expo demo app and a launcher that starts everything for you.
bun demoThis will:
examples/expo-demo on port 8081.exp://127.0.0.1:8081 in Expo Go.http://localhost:3200 in your browser.Ctrl-C cleans up both processes. Once it's running you can:
I to enter Inspect mode, hover / click a React component, and watch the Properties panel fill with the component name, source file, and actual JSX code.+ on the counter — the Code panel opens examples/expo-demo/App.tsx:42 with the + line highlighted.Flags:
bun demo --no-open # skip launching the browser
bun demo --port=3300 # use a different agent-simulator port
bun demo --metro-port=8082 # use a different Metro port
bun demo --device= # target a specific simulatorIf you already have an app, add the metro plugin so the inspector bridge injects at boot:
// metro.config.js
const { getDefaultConfig } = require("expo/metro-config");
const { withAgentSimulator } = require("agent-simulator/runtime/metro-plugin");
module.exports = withAgentSimulator(getDefaultConfig(__dirname));…then install this package alongside:
# Note: `agent-simulator` is not published to npm yet.
# Install directly from GitHub:
bun add -D github:jasonkneen/agent-simulatorOr, if you don't want to touch Metro config, import the bridge at your app's entry point before registerRootComponent:
// index.ts
import "agent-simulator/runtime/inspector-bridge";
import { registerRootComponent } from "expo";
import App from "./App";
registerRootComponent(App);When your app launches with either setup, the BRIDGE pill in the toolbar turns green and inspect clicks produce full component stacks with real source locations.
| Region | What it shows |
|---|---|
| Header | Device chip, stream + bridge status pills, Inspect toggle, Home / Multitask / Lock / Rotate, panel toggles, theme menu. |
| Layers panel (left) | Toggles between the iOS accessibility tree (populated automatically) and the React component tree (populated on inspect). Click any layer to select, hover to highlight, the tree auto-expands ancestors when you tap inside the sim. |
| Simulator canvas | Live MJPEG. In drive mode, the native cursor is hidden and replaced by a round in-preview pointer; taps fire a ping ring, drags become swipes, the mouse wheel scrolls, the keyboard types into the focused field. Inspect mode replaces the pointer with a crosshair and draws selection overlays. |
| Properties panel (right) | Selected layer's accessibility properties, React component name + source, and a highlighted code window around the JSX call site or component definition. |
| Status bar | UDID, mode flags, last-inspect frame count, keyboard shortcut hints. |
I — toggle Inspect (disabled when no RN bridge)Esc — cancel Inspect / unpin selectionmcp/server.mjs is a standalone Model Context Protocol server. It speaks stdio to any MCP host and bridges to the running agent-simulator server over WebSocket. All 15 tools:
| Tool | Purpose |
|---|---|
sim_info |
Device UDID / name / stream URL / device-point size. |
sim_tree |
Full iOS accessibility tree. flat: true returns a labelled-element list for cheap LLM context. |
sim_tap |
Tap at (x, y) sim-ratio. |
sim_tap_by_label |
Look up an AX element by label / value (optional type filter) and tap its centre — robust to layout changes. |
sim_swipe |
Single-call native swipe with durationMs. |
sim_type |
Type printable text into the focused TextField. |
sim_key |
Press one USB-HID keycode (Return=40, Backspace=42, Esc=41, arrows 80–83…). |
sim_button |
home / lock / power / side-button / siri / apple-pay. |
sim_multitask |
Open the app switcher. |
sim_screenshot |
Returns the current frame as an MCP image content. |
sim_inspect |
Inspect the RN component at (x, y) — returns a source-symbolicated component stack. |
sim_source |
Read a code window around file:line for any absolute path in the workspace. |
sim_select_by_source |
Given fileName + line, probes a grid until it finds a rendered component whose source matches. |
sim_subscribe_selections |
Streams inspectResult events back as MCP logging notifications. |
sim_unsubscribe_selections |
Stops the above. |
For Claude Desktop, add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"agent-simulator": {
"command": "node",
"args": ["/absolute/path/to/agent-simulator/mcp/server.mjs"],
"env": { "SIM_PREVIEW_URL": "http://localhost:3200" }
}
}
}Then start the agent-simulator server (bun start or bun demo). The MCP server connects on demand.
For any other MCP host (OpenAI Agents SDK, Cursor, Continue, Codex, custom harnesses): spawn node /path/to/mcp/server.mjs with SIM_PREVIEW_URL pointing at your running server.
1. sim_tree flat:true → see every element on screen
2. sim_tap_by_label "+" → tap the counter's plus
3. sim_tap_by_label "Your name" type:"TextField"
4. sim_type "hello agent"
5. sim_key 40 → Return
6. sim_inspect x:0.86 y:0.24 → returns App.tsx:42
7. sim_source file:".../App.tsx" line:42 → read the JSXThree binaries, one simulator:
simctl io screenshot, encodes MJPEG, serves /stream.mjpeg, /snapshot.jpg, /api/tree, and /health. Reads tap / swipe / type commands from stdin and shells out to axe./api/config / /api/tree / /api/source, bridges browser and MCP WebSocket clients, proxies stack frames through Metro's /symbolicate for React inspection.Clicks from the UI (or MCP) travel:
browser ── WS ──> server.js ── stdin ──> sim-server ── exec ──> axe tap -x … -y … --udid …
│
└──> FBSimulatorHID → CoreSimulator
(device-point coords, cursor-free)Coordinates are carried as [0, 1] ratios end-to-end; sim-server converts to device points using the root AXFrame from axe describe-ui (cached with a 5 s TTL, invalidated on rotate). Every gesture is one subprocess call — no per-step stdin traffic.
/api/tree and /api/sourceGET /api/tree — proxies axe describe-ui --udid via sim-server and returns the raw JSON. The UI pulls this on boot and on refresh; the MCP sim_tree tool wraps it.GET /api/source?file=ABS&line=N&context=M — reads a window of lines around line from an absolute path, returns {file, lines, startLine0, endLine0, targetLine0, language}. Powers the Code panel in the UI and the sim_source MCP tool.React 19 removed fiber._debugSource. Sources now live inside fiber._debugStack, an Error object whose stack string points at bundle-URL line/col pairs. On every inspectResult:
_debugStack, collects up to 12 candidate frames per fiber, attaches them as bundleFrames: […].server.js batch-POSTs every bundle frame to http://localhost:8081/symbolicate in one round trip and picks the first candidate per fiber that isn't inside React's own createElement / jsxDEV plumbing.stack[i].source with a real absolute file path.This means every user component, RN internal component (RCTView, ScrollVi
暂无开放 Issues,或尚未同步最近议题。