feat: record token usage in the audit log
Feature request
Record the token usage (input, output, total) on each audit log entry.
An audit entry currently carries the request and response bodies and nothing else about the call. Token counts can only be recovered by parsing the copied response body, and for streaming requests that never works: the copy stops at MAX_BODY_LOG_SIZE (2048 bytes by default), so it holds the first SSE chunks while the usage chunk is the last thing on the wire.
That makes audit logs unusable for per-user cost and quota reporting, which is a main reason to enable them.
Wanted
prompt_tokens, completion_tokens and total_tokens as dedicated fields on the audit entry, filled for both streaming and non streaming responses, next to the model name.
Requested by @gaby on Discord
Source: open-webui/open-webui