[graphql] Checkpoint subscriptions can emit at or before their resume cursor
Problem
The staging checkpoint subscription can deliver a checkpoint at or before the requested after cursor. The existing checkpoint_subscription::test_subscription_resume_with_after_cursor test fails intermittently at checkpoint_subscription.rs:491.
This reproduces on unchanged base 87362d06c9684bf61e3ea9920d632e6a51b024cf, built in a separate, initially empty target directory. No changes from consensus replay PR #27909 are present in that build.
Reproduction
Linux ARM64 in Docker, Rust 1.96.1, PostgreSQL 16, two Cargo build jobs and two nextest test threads. The repository's CI profile and worker-stack setup were used.
CARGO_BUILD_JOBS=2 NEXTEST_TEST_THREADS=2 \
cargo nextest run --profile ci --features staging --test graphql_subscription \
-E 'test(checkpoint_subscription::test_subscription_resume_with_after_)'The cursor test failed twice and passed on attempt 3 of 4:
attempt 1: first checkpoint = 15, expected 16
attempt 2: first checkpoint = 11, expected 12
Summary: 2 tests passed (1 flaky)The test computes the expected value as resume_seq + 1, so these failures returned the cursor's own checkpoint instead of starting strictly after it. The same test failed all four CI attempts while validating #27909 (first checkpoint 12, expected 14), which prompted the unchanged-base control above.
Related evidence: main CI also marked the afterCheckpoint variant flaky (passed on attempt 2). That log does not establish whether its failure had the same cause.
Possible cause
This is a source-reading hypothesis, not an instrumented diagnosis: the reader retains watermark.cursor but drops watermark.checkpoint. The subscription infers coverage from that cursor, including empty pages, and can pin its live handoff behind the requested resume point.
I checked existing issues/PRs for subscription resumption, cursors, afterCheckpoint, and scan-to-live handoff, plus the affected file history, and found no matching open fix. #27876 was checked as a candidate; its merge commit 551e5d8e5b5a392f19aad02ea08ecab3df8ed06a is already included in the reproduced base.
Source: MystenLabs/sui