FIFO expire resubmits a running task without restoring the backup row; Poll then drop-stale Error()s live work (3.17.0)

On 3.17.0 with the persistent queue (WithTaskStore), persistentQueue.Poll deletes the tasks backup row when work is assigned (TaskDelete, server/queue/persistent.go:69). That is by design for crash-restore (running work should not be re-queued on server restart).

fifo.resubmitExpiredPipelines (constant.TaskTimeout, hard-coded time.Minute — shared/constant/constant.go:41) later moves that same ID from running to pending without TaskInsert (fifo.go:421-431 — no store write; fifo holds no store reference at all). The next Poll of that ID hits TaskDelete → sql: no rows in result set → PR #6765's path (pull queue item: %s: not found in backup, dropping stale task, persistent.go:75) which then Queue.Errors the ID (persistent.go:76). The agent that still holds the containers is now desynced: it may run for many more minutes, then queue.Done logs queue: task not found / cannot ack workflow. The workflow is stored killed/Canceled with empty cancel_info.

PR #6765 assumed a missing backup row meant the workflow was already finished or canceled. That is false when the row is missing because of the first Poll plus an expire-resubmit.

Minimal repro

1. Woodpecker server 3.17.0, sqlite, WOODPECKER_LOG_LEVEL=warn is enough (drop-stale and queue: task expired are error-level).
2. One agent, a workflow that runs >2 minutes (e.g. sleep 600).
3. After assign, block agent→server gRPC for >1 minute while the container keeps running (iptables, or pause the agent process).
4. Server journal: queue: task expired / while waiting the queue returned an unexpected error, then later not found in backup, dropping stale task if another Poll occurs.
5. Workflow ends killed/Canceled, cancel_info null. Agent may still be executing.

Version matrix

• 3.17.0: affected (#6765 drop path in tree; TaskTimeout=1m since #4114).
• 3.18.0 / PR #7015: does not cover this. #7015's new terminal-workflow check sits only in the successful-delete else branch; the ErrRecordNotExist drop-stale path is untouched, and expire-while-agent-still-running has a non-terminal workflow at expire time — the row already gone from the first Poll never reaches the new check.
• Pre-#6765: expire-resubmit still loses the backup row; Poll would hand the ID to a new agent instead of drop-stale (the loop #6765 closed), still killing or duplicating the original run.

Suggested fix

resubmitExpiredPipelines must TaskInsert before requeue, or persistentQueue.Poll must not Queue.Error an ID whose workflow is non-terminal (log + skip). Do not treat ErrRecordNotExist on assign-delete as proof of cancel.

Related (strengthens the report): advertised agent keepalive env is a no-op

cmd/agent/core/flags.go:111-118 defines flags keepalive-time / keepalive-timeout (env WOODPECKER_KEEPALIVE_TIME / WOODPECKER_KEEPALIVE_TIMEOUT, default 20s). cmd/agent/core/agent.go:119-120 passes c.Duration("grpc-keepalive-time") / c.Duration("grpc-keepalive-timeout") into dial (agent/rpc/dial.go:81-83). Those flag names are not defined, so urfave returns zero; gRPC client keepalive Time stays disabled. Community advice to set WOODPECKER_KEEPALIVE_TIME=60s / TIMEOUT=150s (#4645) cannot work on 3.17.0. Current main still has the same split (checked 2026-08-24). Operators cannot tune around the expire/drop path via the documented keepalive envs.

Seen in production

Woodpecker 3.17.0, sqlite DELETE journal ~3 GiB. Agent linux/amd64 docker, 2 slots. Two parallel long cargo-test workflows expired together at T+160s; agent gRPC keepalive ping failed 11 min later; drop-stale + cannot-ack ~18 min after expire. 8 drop-stale events in 24h on this server; 7 cannot-ack workflows match this class (cancel_info null).