#1603·komodo

Timed-out update-check subprocess corrupts Core↔Periphery connection, causing collateral failures on unrelated stacks

Author: dhvikCreated Aug 25, 2026Updated Sep 16, 2026

Possibly related to #1392 — both point at fragile subprocess handling over the Core↔Periphery channel, though with opposite symptoms (there, a hung subprocess is masked by keepalives and the RPC timeout never fires; here, the timeout does fire and killing the process group appears to corrupt the connection instead of just failing the one check). Possibly the same underlying architectural issue, possibly distinct — flagging for visibility either way.

Komodo version: 2.3.2. Setup: Core in Docker, Periphery as a systemd service on the same host ("Local" server), connecting over ws://127.0.0.1:9120.

Steps/evidence:

  • Core log, 06:00:12: GlobalAutoUpdate → CheckStackForUpdateInner{stack=sentinel}: "Failed to check for update | Stack: sentinel | Service: api | Error: Decoded error message over Core-Periphery communication channel: Command timed out after 10.0s (process group killed)"
  • Immediately after, 06:00:22: same stack, service frontend: "Error: Timed out waiting for message.: deadline has elapsed"
  • Periphery's own systemd log shows a tight reconnect loop starting at 06:00:15 (3s after the timeout) and repeating every 5–9 seconds continuously afterward: WARN Connection closed / INFO Logged in to Komodo Core 127.0.0.1:9120 websocket as Server Local, sustained for hours (periphery had accumulated 1h21m CPU time before a restart cleared it).
  • The image being checked (ghcr.io/dhvik/sentinel-api:latest) is small (~100MB compressed across all layers), and a manual docker buildx imagetools inspect of it completes instantly — so the 10s timeout isn't explained by image size. Possibly concurrent load from checking many stacks in the same GlobalAutoUpdate batch run.
  • Other, unrelated stacks (e.g. keel) with correctly configured :latest tags and Pre Pull/Poll/Auto Update all enabled intermittently fail to detect genuinely newer published images during/after these windows, with no error surfaced in the UI (Check returns latest_image: null, update_available: false even when the registry unambiguously has a newer digest — confirmed via manual docker buildx imagetools inspect matching GHCR's package page exactly).

Expected: a single stack's update-check subprocess timing out/being killed should fail only that check, without disrupting the shared Core-Periphery connection or affecting other stacks' concurrent/subsequent checks.

Actual: the connection appears to enter a sustained reconnect loop after such a timeout, and other stacks silently miss real updates during that period.