vmagent: log peer IP address on `remotewrite` http errors

Author: ilyalabunCreated Sep 17, 2026Updated Sep 17, 2026
Labelsenhancement

Is your feature request related to a problem? Please describe

In https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11074 client-side load balancing was introduced in vmagent and vmalert. When vmagent is configured with e.g. -remoteWrite.url=https://dns+vminsert.svc.cluster.local:8480/insert/multitenant/prometheus or similar URL with srv+ prefix and request error happens, we are getting this message in logs:

couldn't send a block with size X bytes to "4:https://dns/+vminsert.svc.cluster.local:8480/insert/multitenant/prometheus": <error mesage>; re-sending the block in 1s.

(see https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmagent/remotewrite/client.go#L437).

The issue is that there's no information about which server instance returned the error. This information would be very useful during issues investigation. So the proposal is to log IP:port of the instance in error message.

Describe the solution you'd like

I couldn't find better way to implement this rather then wrapping http.Request into httptrace.ClientTrace, something like this https://github.com/ilyalabun/VictoriaMetrics/commit/3e0e8254daccdeee08c1d7298fa677f35740a30f .

We could also use http.Response.Request.URL.Host (it's set in https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/lib/httputil/transport_lb.go#L201 on original request's copy) but the issue is that it looks like we don't have http.Response instance on error.

At this point I'd like to ask the maintainers for advice if we need this feature in upstream and if yes, what's the best way to implement it? Thanks!

Describe alternatives you've considered

No response

Additional information

No response

Source: VictoriaMetrics/VictoriaMetrics