UI SQL autocomplete can issue unbounded getMapKeys scans without a date filter

What happened?

With field metadata enabled, a UI SQL autocomplete path can send an unbounded Map-key discovery query to ClickHouse when either dateRange or timestampValueExpression is missing.

The observed query was:

WITH sampledKeys AS
(
    SELECT getSubcolumn(ResourceAttributes, 'keys') AS keysArr
    FROM otel.otel_logs
    LIMIT 3000000
)
SELECT DISTINCT lowCardinalityKeys(arrayJoin(keysArr)) AS key
FROM sampledKeys
LIMIT 1000
FORMAT JSON

There is no timestamp predicate. On a large, date-partitioned Distributed table whose older parts are on an S3-backed cold tier, the inner LIMIT does not provide partition pruning. This query selected approximately 15,085 parts and 1.7 million marks. The coordinator timed out after about 300 seconds, while the remote queries continued for approximately 760-778 seconds before failing with broken-pipe errors.

During the incident hour, S3 GET operations increased from a normal baseline of roughly 10-200 per hour to 58,708.

Expected behavior: UI metadata discovery should not fall back to a raw-table Map-key scan unless it has a usable timestamp expression and bounded date range. If those inputs are unavailable, it should skip Map-key expansion, use a safe default lookback, use configured metadata MVs/text indexes, or require an explicit opt-in for an unbounded scan.

The team-level Disable Field Metadata setting prevents the query and is an effective workaround, but it also disables autocomplete and filter metadata.

This appears closely related to #1036. That issue contains the earlier form of the same no-date-filter lowCardinalityKeys query and was closed as not planned without a linked PR. #1201 added the sampledKeys pre-limit shown above, but it does not prevent an all-part scan. #2426 fixed the same class of missing metadataMVs/dateRange propagation for the MCP describe_source path; the UI path still appears able to reach the raw fallback without those inputs.

Steps to reproduce

1. Configure a log source backed by a large, date-partitioned ClickHouse table containing Map columns such as ResourceAttributes or LogAttributes.
2. Enable field metadata and use a UI SQL editor/autocomplete call site that supplies a tableConnection without both a date range and source timestamp expression (for example, a dashboard filter or raw SQL chart editor).
3. Open/focus the editor so that field autocomplete loads.
4. Inspect system.query_log for a query containing sampledKeys, getSubcolumn(..., 'keys'), and lowCardinalityKeys.
5. Observe that the generated query has no timestamp WHERE clause and selects parts across the full retention period.

The behavior is data-size dependent: it may appear harmless on small or filesystem-cached tables, but becomes expensive on large cold-tier tables.

How are you running HyperDX?

• HyperDX Docker image: hyperdx/hyperdx:2.33.0
• Bring-your-own ClickHouse: 26.7.3.19
• Three-replica ClickHouse cluster on Kubernetes
• Distributed OTel logs table with older parts on an S3-backed cold tier
• metadataMaterializedViews were not used by the observed fallback query

The relevant behavior is also present on the current main branch as of commit 808b345313485977b1a0c32629b418e6389adba2.

Where does it show up?

The HyperDX web UI. The ClickHouse query was issued through the browser-facing /clickhouse-proxy endpoint with user agent hyperdx 2.33.0. No /mcp request was present around the query start time.

The source path appears to be:

SQLInlineEditor / SQLEditor autocomplete
  -> useMultipleAllFields()
  -> metadata.getAllFields()
  -> metadata.getMapKeys()
  -> raw sampledKeys query

Relevant code:

• SQLInlineEditor passes optional dateRange and source?.timestampValueExpression to useMultipleAllFields: https://github.com/hyperdxio/hyperdx/blob/e73af38135f3003385e4b98509264dd420a06222/packages/app/src/components/SQLEditor/SQLInlineEditor.tsx#L96-L109
• useMultipleAllFields accepts both values as optional and calls getAllFields without guarding against a missing metadata scope: https://github.com/hyperdxio/hyperdx/blob/e73af38135f3003385e4b98509264dd420a06222/packages/app/src/hooks/useMetadata.tsx#L226-L264
• getAllFields forwards the optional values into getMapKeys: https://github.com/hyperdxio/hyperdx/blob/e73af38135f3003385e4b98509264dd420a06222/packages/common-utils/src/core/metadata.ts#L1422-L1486
• getMapKeys creates a time condition only when both values are present; otherwise the raw query is generated without WHERE: https://github.com/hyperdxio/hyperdx/blob/e73af38135f3003385e4b98509264dd420a06222/packages/common-utils/src/core/metadata.ts#L854-L873

The filters sidebar path itself passes both values, so the issue is not that all field-metadata calls are unbounded. It is specifically the fail-open behavior when a UI caller does not provide the complete metadata scope.

Logs

Sanitized system.query_log details for one occurrence:

event_time:           2026-08-31 01:37:08 UTC
interface:            HTTP
http_user_agent:      hyperdx 2.33.0
is_initial_query:     1
selected_parts:       15085
selected_marks:       1699997
coordinator_duration: 300376 ms
coordinator_result:   socket receive timeout from a replica

remote query 1:       760019 ms, broken pipe
remote query 2:       778333 ms, broken pipe

The application access log shows multiple /clickhouse-proxy requests in the same second as the ClickHouse query start and no /mcp request in the surrounding nine-minute window.

Suggested regression coverage:

1. A UI field-autocomplete query must not call raw getMapKeys until both a valid dateRange and timestampValueExpression are available.
2. getMapKeys should not silently generate a raw query without a time filter unless the caller explicitly opts into an unbounded scan.
3. Propagate React Query's AbortSignal through getAllFields to getMapKeys so abandoned metadata requests are cancelled.