Worker-thread saturation under concurrent sessions: permission matching, turn-diff recomputation, event fanout, snapshot churn

Summary

With 8+ concurrent sessions (subagent lanes running bash-heavy work), a single instance visibly slows down: the Worker thread that processes session events saturates at ~90-99% of a core in userspace JS (no syscall blocking, no lock waits), and every lane queues behind it. Measured on a 16-core/62GB Linux box while 8 lanes ran: DB event rate ~2.4/s, WAL bounded by autocheckpoint, read/write lane split active, CPU/RAM headroom, network fine. The cost is per-event/per-step JS work on that one thread.

Contributors and fix directions

1. Permission matching path

• packages/core/src/util/wildcard.ts compiles a fresh RegExp per comparison, called per rule per pattern for every tool call.
• packages/opencode/src/permission/index.ts emits an INFO log line per pattern evaluation.
• Direction: cache compiled regexes (keyed pattern + platform flag), demote or throttle the per-pattern log line, hoist rulesets.flat() out of the hot path.

2. Turn diff recomputation

• packages/opencode/src/session/summary.ts runs on every step-finish and always diffs the first step-start tree against the last step-finish tree; the no-change dedup sits after the expensive diffFull, so step k re-diffs steps 1..k — quadratic jsdiff work per turn, multiplied by lane count.
• Direction: memoize the last computed (from, to) pair per session/message, skip identical pairs, single-flight/coalesce per session, and short-circuit from === to before the git work.

3. Event fanout / SSE

• packages/opencode/src/server/routes/instance/httpapi/handlers/event.ts enqueues events for every connection into an unbounded queue and filters afterwards; RSS grows with lane count.
• packages/opencode/src/event-v2-bridge.ts constructs a fresh Location.Info schema instance plus AbsolutePaths per published event even though they are constant per directory/workspace.
• collapseEventBatch only merges adjacent deltas, which rarely happens while sessions interleave.
• Direction: filter before enqueue, bound queues, memoize the location payload, merge deltas by part key.

4. Snapshot churn

• Per-step track() runs continuously under many lanes (~19/min observed) and all lanes share one derived snapshot gitdir, so their git transactions serialize; patch() re-runs the full add-scan immediately after the preceding track().
• Direction: reuse the staged index for patch(), thread an unchanged flag into summarize, memoize exclude/sync scans. Do not debounce track() itself — the snapshot race tests depend on the current behavior.

5. Smaller items

• Doom-loop detection rebuilds a JSON needle per tool-result event.
• Unbounded in-memory maps (approved-permission list, TUI stores) grow with session count.
• Session-layer delta coalescing is worth doing only after the above items are measured; worker-thread sharding is out of scope.

Notes

• Suggested way to attribute the remaining cost precisely: run a controlled 8-session repro with bun run src/index.ts --cpu-prof-md --cpu-prof-dir=/tmp/prof (the profiler cannot attach to a running process), plus an event-loop lag probe while the lanes run.