Vault Agent 2.1.0-1: Mass log spam WARN renewer done
Hi, Since upgrading our Vault Agents to version 2.1.0-1 (installed via the official Hashicorp APT repository on Debian), we are experiencing a massive log spam issue across all our agents.
Every agent floods its logs at a rate of roughly 3 lines per second with the following warning: WARN renewer done (maybe the lease expired)
Context & Environment:
Vault Version: 2.1.0-1 (Agent and Server) OS: Debian Agent Configuration: Uses an auto_auth block and a cache block with use_auto_auth_token = true. No persistent cache path was initially configured (in-memory cache). Triggers: The issue consistently starts occurring after approximately 3 hours of continuous runtime following a service restart.
Additional Details: The underlying secrets (e.g., dynamic database credentials like MariaDB) have a default_ttl of 4 hours and a max_ttl of 28 hours and the approle have a token_ttl of 4h and token_max_ttl of 28h. The vault agent is restarted every day at 05h00. This behavior appeared immediately and uniformly across all agents following the recent upgrade, pointing strongly toward a regression or a changed behavior in the 2.1.0 agent's lease renewal / proxy handling mechanism rather than a configuration mismatch.
The snippet below clearly shows that the authentication token renewal triggers an abnormal failure of the dynamic secret renewer immediately afterwards, resulting in an infinite loop of warning logs :
Debug logs captured at the moment of failure (approx. 3 hours after start):
Sep 16 13:06:40 ztoq02 vault[1512425]: [INFO] agent.auth.handler: renewed auth token Sep 16 13:08:22 ztoq02 vault[1512425]: [DEBUG] agent.cache.leasecache: secret renewed: path=/v1/sys/leases/renew Sep 16 13:10:12 ztoq02 vault[1512425]: [INFO] agent.apiproxy: received request: method=PUT path=/v1/sys/leases/renew Sep 16 13:10:12 ztoq02 vault[1512425]: [DEBUG] agent.cache.leasecache: returning cached dynamic secret response: path=/v1/sys/leases/renew Sep 16 13:10:12 ztoq02 vault[1512425]: [WARN] agent: vault.read(mariadb/creds/mysqlp02_zendto_app): renewer done (maybe the lease expired)
Regards
Source: hashicorp/vault