loki-mixin: LokiCompactorHasNotSuccessfullyRunCompaction is defined twice with identical labels
Describe the bug
The loki_alerts group in production/loki-mixin/alerts.libsonnet defines LokiCompactorHasNotSuccessfullyRunCompaction twice, once for "not run in 3h since the last compaction" and once for "not run in 3h since startup". Both aggregate by (cluster, namespace) and carry only severity: critical, so they produce identical label sets. Two consequences:
promtool check rulesreports a duplicate rule on the compiled alerts, and with--lint-fatalit exits 3, which fails CI steps that lint rules with that flag.- Alertmanager routes and silences match on labels only, so the two conditions cannot be routed or silenced separately.
The expressions are mutually exclusive (the first needs a non-zero last-run timestamp, the second needs it to have stayed at zero for 3h), so they never fire together for the same cluster and namespace. The duplicate came in with #14239 and is present from v3.3.0 through main.
To Reproduce
- Check out main (or any release since v3.3.0).
- Run
promtool check rules --lint-fatal production/loki-mixin-compiled/alerts.yaml.
Checking production/loki-mixin-compiled/alerts.yaml
FAILED:
lint error 1 duplicate rule(s) found.
Metric: LokiCompactorHasNotSuccessfullyRunCompaction
Label(s):
severity: critical
Might cause inconsistency while recording expressionsExit code 3 (promtool 3.13.2).
Expected behavior
Every alert rule has a distinct label set, so promtool check rules --lint-fatal passes and the two conditions can be routed and silenced independently.
Proposed fix
Add a reason label to each variant: since-last-compaction and since-startup. This matches the Mimir mixin, whose CompactorNotRunningCompaction variants carry reason: in-last-<N>h, since-startup and consecutive-failures (compactor.libsonnet). Keeping the alert name unchanged means existing routes and silences keyed on alertname keep matching. Happy to open the PR.
Environment:
- Infrastructure: Kubernetes
- Deployment tool: jsonnet (loki-mixin, rules linted with promtool)
Source: grafana/loki