query: a failed initial DNS resolution still lets the querier become ready with a partial endpoint list

Thanos, Prometheus and Golang version used:

Thanos main as of 2026-09-17, built with Go 1.26.5. The initial resolution in setupEndpointSet is unchanged since #8334 shipped in 0.40.0, so 0.40.0 through 0.42.4 behave the same. No Prometheus involved.

Object Storage Provider: none involved.

What happened:

A querier that discovers its endpoints through dns+ / dnssrv+ started while the nameserver was not answering (i/o timeout). The initial DNS resolution that #8334 added in setupEndpointSet is attempted once and its failure is only logged, so the first endpoint update ran with an empty endpoint list and the querier became ready 4 seconds after start. Until the next --store.sd-dns-interval (30s by default) /api/v1/stores was empty and queries returned 200 with no data and no warning. With several endpoints, only the ones that happened to resolve are attached.

In production this was a querier restarted after an OOM kill while SRV lookups were timing out. It came up ready with only part of its Prometheus shards behind it, and an automation reading those series treated the missing ones as gone.

What you expected to happen:

The initial resolution is retried within the --store.sd-dns-interval context that the code already creates for it, so the querier stays not ready until its endpoints resolve or that bound elapses, the way the sidecar retries reaching Prometheus at startup.

How to reproduce it (as minimally and precisely as possible):

Three Docker containers: a querier whose only endpoint is discovered through dns+, bound with --dns to a nameserver IP that does not exist yet, and a CoreDNS started at that IP 10 seconds later. thanos is a static Linux binary from CGO_ENABLED=0 go build ./cmd/thanos.

docker network create --subnet 172.28.0.0/24 dnsrepro
docker run -d --name upstream --network dnsrepro --ip 172.28.0.10 -v $PWD/thanos:/bin/thanos:ro alpine:3.20 \
  /bin/thanos query --grpc-address 0.0.0.0:10901 --http-address 0.0.0.0:10902
docker run -d --name querier --network dnsrepro --dns 172.28.0.53 -v $PWD/thanos:/bin/thanos:ro alpine:3.20 \
  /bin/thanos query --grpc-address 0.0.0.0:10901 --http-address 0.0.0.0:10902 --endpoint=dns+upstream.repro:10901
sleep 10
printf '.:53 {\n hosts {\n 172.28.0.10 upstream.repro\n }\n}\n' > Corefile
docker run -d --name nameserver --network dnsrepro --ip 172.28.0.53 -v $PWD/Corefile:/Corefile:ro coredns/coredns:1.12.2 -conf /Corefile
sleep 12
docker logs querier 2>&1 | grep -E 'initial DNS|probe status|adding new'
docker exec querier wget -qO- http://127.0.0.1:10902/api/v1/stores

Full logs to relevant components:

ts=2026-09-17T08:45:29.070998446Z caller=endpointset.go:377 level=info msg="performing initial DNS resolution for endpoints"
ts=2026-09-17T08:45:33.083498501Z caller=endpointset.go:391 level=error msg="initial DNS resolution failed" err="lookup IP addresses \"upstream.repro\": could not resolve \"upstream.repro\": all servers responded with errors to at least one search domain. Errs ;could not resolve upstream.repro.: no servers returned a viable answer. Errs ;resolution against server 127.0.0.11 for upstream.repro.: exchange: read udp 127.0.0.1:51855->127.0.0.11:53: i/o timeout"
ts=2026-09-17T08:45:33.083565216Z caller=endpointset.go:395 level=info msg="initial DNS resolution completed"
ts=2026-09-17T08:45:33.087586328Z caller=intrumentation.go:79 level=info msg="changing probe status" status=healthy
ts=2026-09-17T08:45:33.087663752Z caller=intrumentation.go:60 level=info msg="changing probe status" status=ready
{"status":"success","data":{}}

Anything else we need to know:

NXDOMAIN is not an error for the DNS provider (dnsSD.Resolve skips IsNotFound), so this only concerns lookup errors and timeouts. Rule builds its gRPC query endpoint set (grpc_config in --query.config) through the same setupEndpointSet. PR to follow.