vmagent: log peer IP address on `remotewrite` http errors
Is your feature request related to a problem? Please describe
In https://github.com/VictoriaMetrics/VictoriaMetrics/pull/11074 client-side load balancing was introduced in vmagent and vmalert. When vmagent is configured with e.g. -remoteWrite.url=https://dns+vminsert.svc.cluster.local:8480/insert/multitenant/prometheus or similar URL with srv+ prefix and request error happens, we are getting this message in logs:
couldn't send a block with size X bytes to "4:https://dns/+vminsert.svc.cluster.local:8480/insert/multitenant/prometheus": <error mesage>; re-sending the block in 1s.The issue is that there's no information about which server instance returned the error. This information would be very useful during issues investigation. So the proposal is to log IP:port of the instance in error message.
Describe the solution you'd like
I couldn't find better way to implement this rather then wrapping http.Request into httptrace.ClientTrace, something like this https://github.com/ilyalabun/VictoriaMetrics/commit/3e0e8254daccdeee08c1d7298fa677f35740a30f .
We could also use http.Response.Request.URL.Host (it's set in https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/lib/httputil/transport_lb.go#L201 on original request's copy) but the issue is that it looks like we don't have http.Response instance on error.
At this point I'd like to ask the maintainers for advice if we need this feature in upstream and if yes, what's the best way to implement it? Thanks!
Describe alternatives you've considered
No response
Additional information
No response
Source: VictoriaMetrics/VictoriaMetrics