Tail endpoint hangs silently when a line filter contains an invalid regexp

Describe the bug

A tail request whose query has an invalid regexp in a line filter (e.g. |~ "GET(") gets 101 Switching Protocols and then nothing: no entries, no error, no close frame — only server pings. The client hangs forever.

Server-side the error is logged, twice, every ~500 ms for as long as the client stays connected:

level=warn caller=grpc_logging.go:97 method=/logproto.Querier/Tail msg=gRPC err="rpc error: code = Code(400) desc = parse error : stage '|~ \"GET(\"' : error parsing regexp: missing closing ): `GET(`"
level=error caller=tail.go:230 component=tail-querier msg="Error receiving response from grpc tail client" err="rpc error: code = Code(400) desc = parse error : stage '|~ \"GET(\"' : error parsing regexp: missing closing ): `GET(`"

The same invalid query is rejected correctly everywhere else:

The last two rows are the same request differing only in start, which is what makes the silent case look unintended. Sweeping logcli --from on our node (max_chunk_age: 1h): 30/60/75/90 min → hang; 120/150 min/24 h → 1011.

To Reproduce

1. Start Loki, e.g. docker run --rm -p 3100:3100 grafana/loki:3.7.7 -config.file=/etc/loki/local-config.yaml. A fresh instance has no flushed chunks, so this reproduces deterministically; no ingested data needed.
2. websocat 'ws://localhost:3100/loki/api/v1/tail?limit=100&query=%7Bjob%3D~%22.%2B%22%7D%20%7C~%20%22GET(%22' (query: {job=~".+"} |~ "GET(")
3. Handshake succeeds, nothing is ever received, connection stays open. Loki logs the pair above every ~500 ms.
4. Contrast: curl -sG 'http://localhost:3100/loki/api/v1/query_range' --data-urlencode 'query={job=~".+"} |~ "GET("' -w '\nhttp=%{http_code}\n' → 400.

logcli query --follow --limit=100 '{job=~".+"} |~ "GET("' shows the same hang: it prints the ws://… banner and blocks with no output and no exit.

Expected behavior

Reject the query with HTTP 400 before upgrading the connection, as the tail endpoint already does for a bad regexp in a label matcher. At minimum, close the WebSocket with the error and stop retrying it.

Environment:

• Infrastructure: bare-metal single node, Loki in a rootless Podman container
• Deployment tool: none — single-binary target. First seen on grafana/loki:3.7.4, reproduced identically on a vanilla grafana/loki:3.7.7 with the steps above
• Client: logcli 3.6.12, plus a raw WebSocket client to inspect frames (20 s of a hung connection: HANDSHAKE: 101, then frame opcode counts: {'0x9': 20} — pings only, no close frame). tcpdump confirms a single 101 and no client-side retries.

Screenshots, Alloy config, or terminal output

Included inline above.

Our hypothesis (unverified, take or leave): ParseTailQuery validates label matchers but not pipeline stages, so the bad regexp survives the upgrade at pkg/querier/tail/http.go:49. The stage is then compiled in two places. Ingester-side (newTailer) it returns the 400 seen in the log, but on the querier that arrives at the tail stream's first Recv() (pkg/querier/tail/tail.go:230), indistinguishable from a dropped ingester — so Tailer.loop() reconnects and continues, never writing closeErrChan, which would explain both the 500 ms repetition and the silence. The querier also compiles it for the historical part of the tail, but pkg/storage/store.go:551 returns NoopEntryIterator before expr.Pipeline() when no chunk matches — which would explain why an old enough start produces the correct 1011 and a recent one does not.