promtool tsdb create-blocks-from rules: loop terminates prematurely if first block has no evaluation points
What did you do?
Ran promtool tsdb create-blocks-from rules to backfill recording rules over a historical time range where --start falls near the end of a TSDB block boundary:
promtool tsdb create-blocks-from rules \
--start="2026-09-16T11:50:00Z" \
--end="2026-09-16T16:00:00Z" \
--url="http://localhost:9090" \
recording_rules.yml
The rule group in recording_rules.yml has an evaluation interval of 15m:
groups:
- name: test_group
interval: 15m
rules:
- record: job:test_rate:15m
expr: rate(test_metric[15m])
What did you expect to see?
promtool should backfill all evaluation points between 11:50:00 and 16:00:00. Since the first aligned 2-hour TSDB block [10:00:00, 12:00:00) contains no evaluation points (the first 15m evaluation point falls at 12:00:00), promtool should skip that initial block and proceed to generate blocks for [12:00:00, 14:00:00) and [14:00:00, 16:00:00).
What did you see instead? Under which circumstances?
promtool exited with status 0 without generating any blocks.
Looking at cmd/promtool/rules.go:114-117:
end := time.Unix(min(endOfBlock/int64(time.Second/time.Millisecond), end.Unix()), 0).UTC()
if end.Before(startWithAlignment) {
break
}
- Local variable
endshadows the function parameterend time.Time(the overall backfill end time). - In the first block
[10:00:00, 11:59:59],currStartis11:50:00andstartWithAlignmentis12:00:00. endis computed asmin(endOfBlock, userEnd), which evaluates to11:59:59.11:59:59.Before(12:00:00)evaluates totrue.- Because
breakis executed instead ofcontinue, the entirestartOfBlockloop terminates immediately, skipping all remaining blocks ([12:00:00, 14:00:00),[14:00:00, 16:00:00)). - The function returns
nilandpromtoolexits0, resulting in silent data omission.
The empty leading block should be skipped with continue, and break should only occur if startWithAlignment is after the overall requested end time:
blockEnd := time.Unix(min(endOfBlock/int64(time.Second/time.Millisecond), end.Unix()), 0).UTC()
if end.Before(startWithAlignment) {
break
}
if blockEnd.Before(startWithAlignment) {
continue
}
Environment
- System information: Linux / Windows (OS independent)
- Prometheus / promtool version: main (introduced in commit
bd217c58a7356bdc74f74b5a5645bfa85265942e, affects all releases since v2.31.0)
Source: prometheus/prometheus