Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#8215·worldmonitor

fix(digest): relay-auth failure exits 0 — a dead digest cron reports green

Author: koala73Created Sep 16, 2026Updated Sep 16, 2026

Summary

When the digest cron cannot authenticate to the Convex relay, it exits 0. Railway records the run as a success, so a total delivery outage looks like a healthy green cron.

Observed during the #8208 outage: the 02:38Z run on 2026-09-16 fetched zero rules, sent zero digests, and completed "successfully".

2026-09-16T02:38:59Z [inf]  [digest] Cron run start: 2026-09-16T02:38:58.902Z
2026-09-16T02:39:03Z [inf]  [digest] watchlist scan: hashes=2293 candidates=76 events=0 enqueued=0
2026-09-16T02:39:04Z [err]  [digest] Failed to fetch rules: 401

That is the entire run. Nothing else fired.

Cause

scripts/seed-digest-notifications.mjs:2183-2191 — the non-ok branch logs, writes run meta, and returns from main(). main() resolves, the .catch at the bottom of the file never runs, and the process exits 0:

if (!res.ok) {
  console.error('[digest] Failed to fetch rules:', res.status);
  await writeDigestLastRunMeta({
    startedAtMs: nowMs,
    status: 'error',
    errorReason: `fetch_rules_http_${res.status}`,
  });
  return;
}

The catch arm below it (fetch_rules_failed:*, :2192-2200) has the same shape, so a network failure to Convex is equally silent.

Impact

The only thing that surfaced the outage was the digestNotifications staleness check in /api/health, which needs maxStaleMin 90 to elapse first. Compare notification-relay, which logged a 401 per event and stayed loud — that is the behaviour we want here.

This also mattered for triage: the diagnose-railway-seeders skill classifies on crash signatures, and a green run emits none.

Proposed fix

Treat "could not reach or authenticate to the relay" as a failed run: process.exit(1) after writeDigestLastRunMeta, matching the existing brief-compose gate at the end of main() (which already flushes telemetry before exiting). Both the !res.ok and the catch arm should do it.

Deliberately not in scope: the No digest rules found path (:2202) is a legitimate zero-work run and must stay green.

Acceptance

  • A run whose rules fetch 401s / 5xxs / times out exits non-zero and shows red in Railway.
  • A run with zero due rules still exits 0.
  • Test covering both, since the difference is one return vs process.exit(1) and nothing else catches it.

Context: #8208, and the validation comment https://github.com/koala73/worldmonitor/issues/8208#issuecomment-5691556926

Source: koala73/worldmonitor

View original on GitHubView discussion on GitHub