#9914·dapr

Flaky tests: TestV1DirectMessagingEndpointsWithResiliency subtests failing intermittently

Author: sicoyleCreated May 8, 2026Updated Sep 14, 2026
Labelsstale

Description

The TestV1DirectMessagingEndpointsWithResiliency test in pkg/api/http is flaking with multiple subtest failures. Errors indicate context canceled rather than the expected resiliency behavior (retries, circuit breaker tripping, etc.), suggesting a timing/race condition in the test setup.

Where it showed up

Showing up on PR #9731 (unrelated changes):

Failing subtests

All under pkg/api/http TestV1DirectMessagingEndpointsWithResiliency:

  • Test_invoke_direct_message_retries_with_resiliency — got HTTP 500 instead of 200; only 1 call made instead of 2
  • Test_invoke_direct_message_retries_on_unsuccessful_statuscode_with_resiliency — got 500 instead of 450; 0 calls instead of 2; response body shows ERR_DIRECT_INVOKE: failed to invoke, id: failingApp, err: context canceled
  • Test_invoke_direct_message_fails_with_timeout — 0 calls instead of 2; superfluous response.WriteHeader call warning at directmessaging.go:255
  • Test_invoke_direct_messages_fails_after_exhausting_retries — 0 calls instead of 2; same superfluous response.WriteHeader warning
  • Test_invoke_direct_messages_can_trip_circuit_breaker — 1 call instead of 5; never received "circuit breaker is open" error; got context canceled instead

Sample failure output

=== FAIL: pkg/api/http TestV1DirectMessagingEndpointsWithResiliency/Test_invoke_direct_message_retries_on_unsuccessful_statuscode_with_resiliency
    directmessaging_test.go:973:
        Error: Not equal:
            expected: "failingKey2"
            actual  : "{\"errorCode\":\"ERR_DIRECT_INVOKE\",\"message\":\"failed to invoke, id: failingApp, err: context canceled\"}"

=== FAIL: pkg/api/http TestV1DirectMessagingEndpointsWithResiliency/Test_invoke_direct_messages_can_trip_circuit_breaker
    directmessaging_test.go:1015:
        Error: "{...err: context canceled\"}" does not contain "circuit breaker is open"
        Messages: Should have received a circuit breaker open error.

Impact

Blocking unrelated PRs from getting a green CI. The PR in question (#9731) does not touch direct messaging, HTTP API, or resiliency code.

Suspected cause

The repeated context canceled errors and superfluous response.WriteHeader call at directmessaging.go:255 suggest the request context is being canceled before the resiliency policy can execute its retries — likely a test-level timeout or context lifetime issue rather than a production bug.