Agent retry window counts from the start of a long poll, so the first error after an idle Next() or a running Wait() gives up without a retry

Component

agent

Describe the bug

The window is passed to backoff.WithMaxElapsedTime, and backoff.Retry measures it from the moment it was called. Next() and Wait() are server-side long polls (queue.Poll / queue.Wait block until work arrives or the workflow finishes), so for an agent that has been idle, or a workflow that has been running, for longer than the window, the very first transport error already exceeds it. The agent then logs

{"level":"error","error":"backoff: maximum elapsed time exceeded (last error: rpc error: code = Unavailable desc = closing transport due to: connection error: desc = \"error reading from server: EOF\", received prior goaway: ...)","message":"grpc error: next(): code: Unavailable"}
{"level":"error","error":"backoff: maximum elapsed time exceeded (last error: ...)","message":"runner error, retrying..."}

without a single retry.

• Next(): in multi-workflow mode the runner falls into its 5-second "runner error, retrying..." loop and eventually reconnects, so the configured window is simply never applied. In single-workflow mode the runner exits with "runner done with error".
• Wait(): runner.go treats a Wait() error as fatal and cancels the workflow context, so a server restart during a step that has been running for longer than WOODPECKER_RETRY_TIMEOUT aborts the pipeline instead of retrying the RPC.
• Whether the agent shuts down afterwards depends on which error happened to be the last one: only errNotConnected from the IsConnected() guard is translated into ErrConnectionLost, a transport-level Unavailable at the end of the window is not. Two agents hitting the same server restart took different paths here (one shut down and was restarted by Kubernetes, the other stayed in the runner retry loop).

Root cause. agent/rpc/client_grpc.go, retryOpts: backoff.WithMaxElapsedTime(c.connectionRetryTimeout); backoff.Retry (cenkalti/backoff v7) checks time.Since(startedAt)+next > MaxElapsedTime with startedAt taken before the first operation call. The retry window should start at the first failure. I have a patch with tests ready and will open a PR referencing this issue.

Steps to reproduce

1. Server and agent v3.18.0 (Kubernetes backend, one agent with WOODPECKER_MAX_WORKFLOWS=1, defaults otherwise).
2. Leave the agent idle for more than 2 minutes.
3. Restart the server pod.
4. Agent log shows the two lines above at the moment the server drops the connection.

For the Wait() case start a workflow with a step that sleeps for 5 minutes and restart the server after 3 minutes.

Expected behavior

After a server restart the agent keeps retrying Next() / Wait() for WOODPECKER_RETRY_TIMEOUT (default 2m) and reconnects, or shuts down with "connection to server lost" once that window has elapsed. A running workflow is not affected by a server restart that is shorter than the window.

System Info

{"source":"https://github.com/woodpecker-ci/woodpecker","version":"3.18.0"}

Helm chart 3.7.3, Kubernetes backend, k3s on arm64.

Additional context

Related: #4446 (agent stops taking jobs after a server restart), #6414 (introduced the retry window).

Validations

• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Checked that the bug isn't fixed in the next version already (agent/rpc/client_grpc.go is identical on main and v3.18.0).