test(durable-messaging): investigate intermittent retry-limit callback result

Observed failure

During a native .NET 8 Release run of the DurableMessaging test project, InboxMissingHandlerTests.OrdinaryHandlerFailures_RetryUntilExactConfiguredAttemptLimit intermittently failed at the assertion following the first one-minute fake-clock advance (line 114):

Expected: RescheduleRequested
Actual:   InProgress

The full run executed 396 cases: 395 passed and this case failed. The same candidate passed all 396 cases on native .NET 10. A subsequent focused native .NET 8 run of the complete InboxMissingHandlerTests class passed both cases.

Scope

Observed while adopting inbox #11284 at f715c4c2163aea5cda1bde75eda907508d9c5555 into outbox #11285. All DurableMessaging production C# and tests are byte-identical to the previously validated outbox head bb68b10b1a55db19ee3b275f66b6cf50e25df8f6; the prerequisite change moves standard grain-bound manager lifecycle enrollment into its constructor. This report does not establish that enrollment caused the failure.

Phase boundary to investigate

RunPumpAsync starts/polls a durable callback, waits for any GrainTimerEvents.TickStop belonging to the grain when the callback reports InProgress, and polls exactly once more. The event predicate matches the grain but not the timer or callback execution. The inbox also has local-drain timers. Investigate whether an unrelated timer completion can satisfy that wait before the requested callback has produced its terminal result.

A deterministic test barrier should establish completion of the intended callback while preserving the exact retry-attempt, retry-time, journal-write, terminal dead-letter, and deduplication assertions. The observation alone does not justify changing runtime retry semantics or accepting InProgress as the expected terminal result.

Commands

dotnet test --project test\Orleans.DurableMessaging.Tests\Orleans.DurableMessaging.Tests.csproj --framework net8.0 --configuration Release --no-build --minimum-expected-tests 396 --max-parallel-test-modules 1 --report-trx

dotnet test --project test\Orleans.DurableMessaging.Tests\Orleans.DurableMessaging.Tests.csproj --framework net8.0 --configuration Release --no-build --filter-class "*InboxMissingHandlerTests" --minimum-expected-tests 2 --report-trx

SDK 10.0.401, Windows x64, native .NET 8 runtime 8.0.31 selected through a scoped DOTNET_ROOT_X64. The original failing full-suite TRX and passing focused TRX are retained in the outbox coordinator handoff. No runtime or inherited-test edits were made for this report.