#6458·ubicloud

`deadline_start` carries over from `initialize_database_from_backup`, shrinking the extension budget for `wait_catch_up` and `wait_recovery_completion`

Author: furkansahinCreated Sep 16, 2026Updated Sep 16, 2026
Labelspostgresreliabilityfollow-up

Summary

register_deadline(..., allow_extension: 24 * 60 * 60) caps the extended deadline at deadline_start + 24h, and deadline_start is assigned with ||= (prog/base.rb:420). It is cleared only when the strand reaches its deadline_target label (model/strand.rb:215) — for Postgres servers, label wait.

initialize_database_from_backup sets deadline_start on its first observed disk growth (prog/postgres/postgres_server_nexus.rb:209). The provisioning path from there to the next extension point — initialize_database_from_backuprefresh_certificatesconfigure_metrics → … → configurewait_catch_up / wait_recovery_completion — never passes through wait, so the value survives.

Result: the 24 h extension budget at lines 587, 593 and 629 is not 24 h of catch-up or replay. It is 24 h minus however long the backup restore took.

Evidence

Probes run against the test DB with the real progs (RACK_ENV=test bundle exec rspec):

  • Ran initialize_database_from_backup in the InProgress branch with growing disk usage, then wait_recovery_completion on the same strand — deadline_start set by the former is reused by the latter.
  • With deadline_start rewound 25 h, wait_recovery_completion's extension produced a deadline_at 3600 s in the past — an already-expired deadline, paging on the first replay tick.
  • With deadline_start rewound 23 h, ticks still received the full 600 s. The cap only truncates inside the final 10 minutes, so the failure is a cliff at the 24 h mark rather than a gradual squeeze.

Why it matters

This bites exactly the population the extension exists for: databases large enough that the restore is slow. A restore that runs past 24 h means the subsequent replay or catch-up phase gets a deadline that has already expired, and the strand pages immediately despite being perfectly healthy.

Not a regression — before #6036 wait_recovery_completion did not extend at all — but it means the stated goal ("a long but healthy recovery does not page") is not met at the tail.

Proposed fix

Release the restore's budget when the restore finishes, alongside the existing cleanup in the Succeeded branch at prog/postgres/postgres_server_nexus.rb:202:

ruby
delete_from_stack("disk_usage", "initialize_database_from_backup_try_count", "deadline_start")

This fixes both consumers at once — standbys and read replicas reach wait_catch_up through the same path, so they inherit the same shortened budget today.

Worth considering instead: a reset_extension: keyword on register_deadline for callers that want a fresh window, if other progs turn out to share the pattern.

Acceptance criteria

  • After initialize_database_from_backup succeeds, wait_catch_up and wait_recovery_completion each get a full 24 h extension window regardless of how long the restore ran.
  • A spec that drives initialize_database_from_backup (InProgress, growing disk) and then the wait label on the same strand, asserting deadline_start is re-anchored rather than inherited.

Related, not in scope

register_deadline never clears deadline_notified. Once a Deadline page has fired, later extensions move deadline_at but the frame stays notified (model/strand.rb:224, :281), so the extension has no effect on paging. Probably worth its own issue if it's not already known.