promtool test with `/dev/stdin` fails for multiple test cases
Author: TribuneXCreated Mar 18, 2024Updated Sep 27, 2026
What did you do?
I use promtool test with the following rules and tests:
rules
# prometheus-rules.yml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
spec:
groups:
- name: myalerts
rules:
- alert: ErrorInLog
expr: changes(log_messages_total{log_level="ERROR"}[1m]) > 0
annotations:
summary: 'New error(s) appeared in the log of container {{ $labels.container }}'
- alert: NoNewMessages
expr: sum(increase(messages_sent_total[3m])) == 0
annotations:
summary: 'My component did not send any messages for 3 minutes'tests
# test-alerts.yml
rule_files:
- /dev/stdin
tests:
- name: 'Should trigger if no messages were sent for more than 3 minutes'
interval: 1m
input_series:
- series: 'messages_sent_total'
values: '0 0 0 0'
alert_rule_test:
- eval_time: 4m
alertname: NoNewMessages
exp_alerts:
- exp_annotations:
summary: 'My component did not send any messages for 3 minutes'
- name: 'should trigger in case an error is detected'
interval: 1m
input_series:
- series: 'log_messages_total{container="mycontainer", log_level="ERROR"}'
values: '0 1 2'
alert_rule_test:
- eval_time: 2m
alertname: ErrorInLog
exp_alerts:
- exp_labels:
log_level: "ERROR"
container: "mycontainer"
exp_annotations:
summary: 'New error(s) appeared in the log of container mycontainer'It seems that there is a bug with the way promtool reads from /dev/stdin compared to executing the tests with a static file.
What did you expect to see?
The same tests with a static input file (alerts.yaml) pass:
yq .spec prometheus-rules.yml > alerts.yaml && promtool test rules test-alerts.yml && rm alerts.yaml
Unit Testing: test-alerts.yml
SUCCESSWhat did you see instead? Under which circumstances?
The tests fail for /dev/stdin:
yq .spec prometheus-rules.yml | promtool test rules test-alerts.yml
Unit Testing: test-alerts.yml
FAILED:
name: should trigger in case an error is detected,
alertname: ErrorInLog, time: 2m,
exp:[
0:
Labels:{alertname="ErrorInLog", container="mycontainer", log_level="ERROR"}
Annotations:{summary="New error(s) appeared in the log of container mycontainer"}
],
got:[]System information
macOS 14.4
Prometheus version
promtool, version 2.50.1 (branch: non-git, revision: non-git)
build user: [email protected]
build date: 20240226-11:31:35
go version: go1.22.0
platform: darwin/arm64
tags: netgo,builtinassets,stringlabelsPrometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response
Source: prometheus/prometheus