Hardcoded 30-min ABSOLUTE_CEILING_MS cold-kills long local-model turns; no config seam
What happens
On a local-model backend (OpenCode provider → an OpenAI-compatible local server), long agent turns are killed mid-turn by the host sweep:
WARN Killing container past absolute ceiling sessionId="sess-…" heartbeatAgeMs=1829985 ceilingMs=1800000
WARN Killing container past absolute ceiling sessionId="sess-…" heartbeatAgeMs=2161651 ceilingMs=180000017 such kills on one install in a single day, heartbeatAgeMs 30–36 min against a ceilingMs that is always exactly 1800000. The container is doing real work — it is streaming tokens out of a 27B model on-device — but the host reads it as stuck, cold-kills it, and the turn's context is lost.
Why
ABSOLUTE_CEILING_MS is a hardcoded constant with no config seam:
src/host-sweep.ts:46
export const ABSOLUTE_CEILING_MS = 30 * 60 * 1000;The only thing that can widen it is a Bash tool's own declared timeout (src/host-sweep.ts:92):
const ceiling = Math.max(ABSOLUTE_CEILING_MS, declaredBashMs ?? 0);and bashTimeoutMs() returns non-null only when state.currentTool === 'Bash' (src/host-sweep.ts:254). A long model generation is not a Bash call, so declaredBashMs is null and the ceiling stays at a flat 30 minutes — which matches the observed ceilingMs=1800000 in every log line above.
Meanwhile the heartbeat is only touched per provider stream event (container/agent-runner/src/poll-loop.ts:525, inside for await (const event of query.events)). A hosted frontier model emits events often enough that 30 minutes is never in reach. A local 27B model on consumer hardware — especially on a long-context turn — can spend far longer than that between events across a multi-step turn, so the heartbeat legitimately ages past the ceiling while the model is actively decoding.
The net effect: the 30-minute constant encodes an assumption about hosted-inference latency. Any slower backend (local MLX/llama.cpp/Ollama seat, or a heavily rate-limited remote) inherits a hard 30-minute wall on turn length that no operator setting can move.
How to see it
- Point an agent group at a local OpenAI-compatible endpoint (this install: OpenCode provider,
DEFAULT_AGENT_PROVIDER=opencode, base URL on the host loopback, a 27B-class MLX model). - Give it a task whose turn takes longer than 30 minutes of wall time — long-context read-and-answer, or a multi-step tool-call chain where each step waits on local decode.
- Watch
logs/nanoclaw.error.log. The container is killed at heartbeat age ~30 min withceilingMs=1800000, and the turn never produces a reply.
Observed on main behaviour; the constant is present at current origin/main (294ef2ae), src/host-sweep.ts:46.
Suggested fix
Make the ceiling configurable rather than constant, defaulting to today's 30 minutes so nothing changes for existing installs:
- Read an override from the per-agent-group container config (alongside the other
container_configsruntime settings) and/or an install-level env var, falling back toABSOLUTE_CEILING_MSwhen unset. Groups pinned to a slow local seat can then set a ceiling that matches their hardware without loosening it globally. - Optionally, keep the ceiling honest for genuinely-hung containers by having the provider touch the heartbeat on a coarse timer while a stream is open, so "slow" and "dead" stop looking identical to the sweep. That is the same blind spot #2668 and #3455 describe from two other angles, so a shared fix may be preferable to three separate ones.
Related
- #2668 — long/hung MCP tool freezes the heartbeat; the 30-min ceiling is the only backstop. Same constant, different trigger (tool call vs. model decode).
- #3455 — no heartbeat coverage between claim and first SDK event, tripping the 60s
CLAIM_STUCK_MSwatchdog. Same "heartbeat only ticks on stream events" root, different window.
Filing this one separately because the trigger here is neither a tool call nor session setup: it is ordinary generation on a slower-than-hosted backend, which no current escape hatch covers.
Source: nanocoai/nanoclaw