Expose query id in HTTP API responses so clients can monitor queries via query_activity()

Problem

query_activity() lists running queries with their query_id, state, memory_used, and memory_limit, and cancel_query(query_id) cancels one. Both are keyed by the query id. The HTTP API (/exec, /exp) never returns that id to the client: it isn't in the JSON response body or in a response header.

A client that sends a query over HTTP therefore can't reliably find its own query in query_activity(). This blocks common use cases:

• monitoring the memory usage of a specific long-running query
• cancelling a specific query the client started, for example on a client-side timeout
• correlating a client request with the server's exe [id=...] / fin [id=...] log lines

Current workaround

Tag the SQL with a unique comment and search for it by text:

GET /exec?query=/* req-7f3a91 */ SELECT ...

SELECT query_id, memory_used, memory_limit
FROM query_activity()
WHERE query LIKE '%req-' || '7f3a91%';

This works because QueryRegistry stores the full SQL text, comments included. It's fragile, though. The lookup has to avoid matching its own text (hence the || split), text matching is ambiguous when several clients send the same SQL, and every monitoring client has to reimplement the trick.

Main challenge: the id must reach the client before the query finishes

To monitor memory, a client such as the Web Console needs the id while the query is still doing its heavy work. Just adding a header at the current send point doesn't achieve that.

The id doesn't exist until the query runs: QueryProgress registers a new id each time a cursor opens, so the same SQL text gets a new id on every run, even when HTTP reuses a cached plan. The server can't return the id in advance.

JsonQueryProcessorState.onResumeSetupFirstRecord() sends the HTTP 200 headers only after it fetches the first record. For queries that do most of their work before the first row (GROUP BY, ORDER BY, joins that build hash tables), an id sent in a response header at that point would reach the client after the memory-heavy phase is over. That's exactly the phase a memory monitor needs to watch.

Proposed design

What the code already allows

• QueryProgress.getCursor() registers the query before the base factory does any work.
• The expensive factories build their state on the first hasNext(), not in getCursor(): GroupByRecordCursorFactory and AsyncGroupByRecordCursor (buildMapConditionally()), SortedRecordCursor (buildChain()), and HashJoinLightRecordCursorFactory (buildMapOfSlaveRecords()). There's a gap between registration and the memory-heavy phase.
• HttpResponseSink.sendHeader() flushes headers on their own send (flushSingle()), so sending them earlier costs no extra syscall.
• The processor holds headers back only so that an error on the first record can still return 400. Errors after that point already arrive as a 200 with error/errorPos in the JSON body (querySuffixWithError()).

Flow (SELECT via /exec)

1. Compile. Compile errors still return 400, as today.
2. factory.getCursor() opens the cursor and registers the query. Errors here still return 400.
3. New: the processor sends 200 with an X-QuestDB-Query-Id: <id> header. Following the existing pattern, it moves the state machine to the next state before sending, so a resume after PeerIsSlowToReadException doesn't send the headers twice.
4. The processor fetches the first record (the heavy phase) and streams the result. The client already has the id and polls SELECT memory_used FROM query_activity() WHERE query_id = N on a second connection.
5. The client stops polling when the response body completes. With #7623, one last read would also return the final state and peak memory.

In the browser, fetch() resolves once headers arrive, so the Web Console can read response.headers.get('X-QuestDB-Query-Id') while the body is still pending.

Required changes

• Opt-in flag. Sending headers early moves errors on the first record from a 400 to a 200 with the error in the body. Clients that only check the status code or response.ok would miss them. A boolean request flag should enable the early headers, and the default behavior should stay as it is. The Web Console would set the flag.
• New error case: headers sent, no body yet. internalError() currently chooses between a 400 and an in-body error by checking bytesSent > 0, and querySuffixWithError() assumes the {"query":...,"columns":...,"dataset":[ prefix is already written. After headers-only, the processor needs to write a complete error object instead.
• Getting the id to the processor. Today the id lives in QueryProgress.sqlId, and only the MemoryTracker bound to the execution context carries it incidentally. QueryRegistry.register() should store it on the SqlExecutionContext explicitly, or the top-level factory should expose it.
• Audit getCursor() for heavy work. Any factory that does real work in getCursor() would delay the header. The factories listed above build lazily. Subquery materialization during function init() still needs checking.
• Scope. Start with SELECT. CTAS and INSERT AS SELECT register inside compiler.compile(), before the processor can send anything, and the processor can't flush headers from there because PeerIsSlowToReadException can't escape the compiler. /exp has the same shape as /exec and could follow.
• Visibility. Non-admin users should keep seeing only their own queries, as query_activity() enforces today.

Alternatives considered

• 103 Early Hints interim response. fetch() doesn't expose 1xx responses to JavaScript, so the Web Console couldn't read the id.
• Async submit, then fetch results (202 Accepted with the id). HTTP cursors are tied to the connection's non-blocking send loop, and holding them without a connection would be a much larger redesign.
• A query-started frame on QWP egress. A clean fit if the Web Console moves to the WebSocket protocol (QwpEgressUpgradeProcessor already has QUERY_REQUEST, RESULT_BATCH, and RESULT_END). But it's a wire format change that needs tests against the pinned client, and it doesn't help /exec users.