#7623·questdb

Keep completed queries in query_activity() for a configurable TTL

Author: puzpuzpuzCreated Sep 14, 2026Updated Sep 14, 2026
LabelsEnhancementSQL

Problem

query_activity() shows only queries that are still running. QueryRegistry.unregister() removes the entry as soon as the query's cursor closes, and recycles it to the pool. After that, the query id, SQL text, and memory usage are gone.

That leaves a blind spot for monitoring:

  • Short or medium queries often finish between two polls of query_activity(), so a monitor never sees them, even when they used a lot of memory.
  • A client can't look up a query after it finishes to see how much memory it used or how it ended (finished, failed, or cancelled).
  • Related to #7622: even when a client knows its query id, a query that completes before the client polls leaves nothing to look up.

Proposal

Keep completed queries in query_activity() for a configurable TTL (for example, the last 5 minutes), alongside the running ones.

Design considerations

  • Final state. Retained rows need a terminal state that distinguishes finished, failed, and cancelled queries, plus an end timestamp (or duration). Today state has only idle, active, and cancelled, where cancelled means a cancel was requested for a query that's still running.
  • Peak memory, not last memory. QueryProgress calls unregister() after it closes the base cursor, so memory_used read at that point reflects memory after the cursor released its buffers, not what the query used while it ran. A useful history needs peak memory usage, and MemoryTracker doesn't track a peak today.
  • Bounded footprint. At a few thousand queries per second, a 5-minute window means hundreds of thousands of entries, each holding the full SQL text. The retention needs a maximum entry count as well as a TTL, and should keep to the zero-GC approach QueryRegistry already uses (pooled entries). Setting the TTL to 0 should turn retention off.
  • Registry semantics. cancel_query() on a retained, completed query should fail cleanly or do nothing, and the retained entries must not interfere with the entry lifecycle (ACTIVE -> RETIRED -> IDLE) that protects against a late cancel reaching a reused entry.
  • Visibility. Non-admin users should see only their own completed queries, the same rule query_activity() applies to running ones.

Existing alternative and its gaps

The _query_trace table (query.tracing.enabled) keeps a history of queries, but it doesn't cover this:

  • It's off by default.
  • It records only queries that complete successfully. Failed and cancelled queries aren't recorded.
  • It has no query_id or memory columns.
  • It stores rows in a table with a 1-day TTL, which suits auditing better than a short in-memory monitoring window.