通过 SelfMetrics 记录 BatchingSink 中的批量传输延迟
The delivery to a remote sink is the part of logging that's least under anyone's control. Even when the application and the sink are both behaving perfectly, how long a batch takes to land depends on a long list of things that live outside the process: network latency and packet loss, DNS or TLS handshakes when connections are re-established, a proxy or load balancer in the path, the ingestion service itself being slow, throttling or rate limiting on the receiving side, a cross-region hop, etc. Locally it can be just as indirect: CPU throttling in a container, thread pool starvation, or a GC pause stretching out an otherwise quick send. From Serilog's side all of them look the same. Batches take longer to send. The queue in BatchingSink grows, and memory grows with it. In the end events are dropped, either at `QueueLimit`, or when `RetryTimeLimit` is over. Only this last step is visible today, through `SelfLog` or `ILoggingFailureListener`. The slow degradation before it is not visible at all. It would be very useful to see the timing of batch delivery in order to react in a timely manner when latency is growing. `SelfMetrics` already covers the pipeline, but not the delivery side of it. The same approach can be extended to `BatchingSink`. The main part would be a duration histogram around `_targetSink.EmitBatchAsync(...)` in `BatchingSink.LoopAsync()`, something like:
内容来源: serilog/serilog