#24332·loki

quantile_over_time returns max_over_time's value regardless of requested quantile

Author: mikkelkeCreated Sep 2, 2026Updated Sep 14, 2026
Labelsarea/logql

Loki version: 3.6.7 (all components — distributor, ingester, querier, query-frontend, compactor — consistently on this version, running since 2026-03-23)

Deployment mode: microservices (separate distributor/ingester/querier/query-frontend/compactor Deployments/StatefulSets, single shared config.yaml via -target=<component>)

Relevant config:

yaml
limits_config:
  split_queries_by_interval: 6h
query_range:
  align_queries_with_step: true
  cache_results: true
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        ttl: 24h

limits_config.shard_aggregations is not set (so the experimental sketch-based quantile sharding path, e.g. #20326, should not be in play).

Bug

quantile_over_time(q, ...) returns the exact same value as max_over_time(...) on the same stream/window, for every value of q I tried (0, 0.1, 0.5, 0.99) — instead of the correct rank-interpolated value.

Example, same log stream, same 1h window, instant queries via the HTTP API:

max(max_over_time({...} | unwrap file_size_bytes [1h]))          -> 24835635
max(quantile_over_time(0,   {...} | unwrap file_size_bytes [1h])) -> 24835635
max(quantile_over_time(0.5, {...} | unwrap file_size_bytes [1h])) -> 24835635

min_over_time(...) on the identical stream/window correctly returns a very different, much lower value (e.g. ~1.1M vs ~29M over a 24h window), confirming the underlying samples have real variance and the log/unwrap pipeline itself is fine — the bug is specific to quantile_over_time's own computation.

Ruled out:

  • Not a sort-direction bug: quantile_over_time(0, ...) (which should equal min_over_time) instead equals max_over_time, not min_over_time. A reversed sort would swap low/high quantiles, not collapse all of them to max.
  • Not query splitting/sharding: reproduces identically at both 1h and 24h windows; split_queries_by_interval is 6h, so the 1h window should never be split, yet still exhibits the bug.
  • Not stale/cached results: re-querying with the quantile argument varied (0 vs 0.5) and the query time shifted by 90s both still return the same value as max_over_time for that specific window — and the value correctly moves when the window itself moves (re-tested minutes later, all three functions shifted together to a new but still-identical value), so it's not a frozen cache entry either.
  • limits_config.shard_aggregations is unset, so the known immature sketch-quantile-sharding codepath (#20326) shouldn't be reachable.

Expected

quantile_over_time(0.5, ...) should return the median of the unwrapped values in the window, not the max.

Impact

This makes quantile_over_time silently useless for detecting "is this metric's distribution flat/pinned near its max" type checks (e.g. quantile_over_time(0.99,...)/max_over_time(...) > threshold) — the ratio always evaluates to 1.0 regardless of the actual distribution shape, since both sides compute the same value.

Happy to provide more detail/help reproduce against a minimal fixture if useful — this was found via a real alerting rule that turned out to be silently non-functional in its core statistic.