Deacon heartbeat stamping can stop updating the beacon label after recovery from a stuck kill
Author: kyle079Created Jun 7, 2026Updated Sep 9, 2026
Labelsstatus/needs-triage
Summary
The Deacon's beacon bead can stop receiving fresh heartbeat: label updates after recovery from a stuck-kill event. Manual heartbeat commands do not re-stamp the label once it is wedged.
Observed behavior
- The beacon label stops advancing.
- Restarting related processes with a clean environment does not restore label updates.
- Explicit heartbeat commands also fail to refresh the beacon label.
Impact
- Beacon state becomes stale and no longer reflects Deacon liveness.
- Operators lose confidence in the heartbeat indicator.
- This weakens monitoring and post-incident diagnostics even when other automation continues to run.
Evidence
Observed after a stuck-kill recovery window on 2026-06-05, where the heartbeat label remained frozen while unrelated environment fixes had no effect.
Expected behavior
The heartbeat cycle should either resume stamping automatically after recovery or surface a clear failure when beacon updates are no longer possible.
Suggested fix
Inspect the heartbeat-stamping path for:
- state not being reinitialized after stuck-kill recovery
- stale label update guards that never clear
- failures that are swallowed instead of surfacing an actionable error
A watchdog for "heartbeat label has not advanced within N intervals" would also make this class of failure easier to detect.
Source: gastownhall/gastown