Two default alert rules in the monitoring sample never stop firing

Author: maxpainCreated Sep 19, 2026Updated Sep 19, 2026

Two of the seven rules in docs/src/samples/monitoring/prometheusrule.yaml latch — once they fire they never clear, regardless of whether the condition still holds.

DatabaseDeadlockConflictscnpg_pg_stat_database_deadlocks > 10. The metric is cumulative (pg_stat_database.deadlocks, resets only on pg_stat_reset()), so any busy database crosses 10 within weeks and then alerts forever. The annotation ("There are over 10 deadlock conflicts") suggests a windowed count was intended.

LastFailedArchiveTime(last_failed_time - last_archived_time) > 1. Correct on a primary. On a replica last_archived_time stops advancing because replicas don't archive WAL, so a single past failure keeps the difference positive indefinitely.

Seen on 1.28.0: the archive rule firing on two replicas since January and May with archiving healthy on the primaries, and the deadlock rule firing on two primaries whose lifetime counters are 140 and 681 against actual rates well under 10/hour.

Suggested fix: increase(cnpg_pg_stat_database_deadlocks[1h]) > 10, and and on(namespace, pod) cnpg_pg_replication_in_recovery == 0 for the archive rule. PR follows.

Source: cloudnative-pg/cloudnative-pg