#1160·asynq

Inspector.GetQueueInfo is O(#groups) and blocks Redis for seconds when a queue has many aggregation groups

Author: tschellenbachCreated Jul 3, 2026Updated Sep 9, 2026

Inspector.GetQueueInfo runs two Lua scripts that iterate every aggregation group in the queue:

Since Lua scripts execute atomically, each script blocks the entire Redis instance for the duration. With fine-grained group keys (e.g. one group per user/entity), a queue can easily reach millions of groups. And because x/metrics.QueueMetricsCollector calls GetQueueInfo for every queue on every Prometheus scrape — and Inspector.Queues() includes queues no consumer is draining — this runs continuously in production. We hit this as a serious production incident: Redis CPU pinned by monitoring alone.

Still present on master.

Repro (1M groups)

go
package main

import (
	"fmt"
	"sync"
	"time"

	"github.com/hibiken/asynq"
)

func main() {
	// 1M tasks, each in its own group; no server running, so groups accumulate
	// (same shape as a slow or paused consumer).
	var wg sync.WaitGroup
	for w := 0; w < 16; w++ {
		wg.Add(1)
		go func(w int) {
			defer wg.Done()
			c := asynq.NewClient(asynq.RedisClientOpt{Addr: "127.0.0.1:6379"})
			defer c.Close()
			for i := w * 62500; i < (w+1)*62500; i++ {
				_, err := c.Enqueue(asynq.NewTask("noop", []byte(`{}`)),
					asynq.Queue("repro"), asynq.Group(fmt.Sprintf("g:%d", i)))
				if err != nil {
					panic(err)
				}
			}
		}(w)
	}
	wg.Wait()

	insp := asynq.NewInspector(asynq.RedisClientOpt{Addr: "127.0.0.1:6379"})
	t0 := time.Now()
	info, err := insp.GetQueueInfo("repro")
	if err != nil {
		panic(err)
	}
	fmt.Printf("GetQueueInfo: %s (groups=%d)\n", time.Since(t0), info.Groups)
}

Output on a laptop (asynq v0.25.1, Redis 7, localhost — a concurrent client measured PING latency during the call):

GetQueueInfo: 4.947s (groups=1000000 aggregating=1000000)
max PING latency from another client while GetQueueInfo ran: 2.546s

One call = ~5s of Redis CPU and two multi-second full-server stalls. Multiply by number of queues × scrape interval × processes running the metrics collector.

Suggested fixes

  • Bound the ZCARD loop the way memory usage is already sampled: SRANDMEMBER groups N + ZCARD on the sample, extrapolate via SCARD (exact Groups count stays O(1) via SCARD).
  • Or add an option to skip aggregating/memory stats in GetQueueInfo so the metrics collector can opt out.

Happy to send a PR if maintainers have a preference between the two.