Improve handling of `CANCELLED` errors in the dispatch layer
Author: tstirrat15Created Sep 8, 2026Updated Sep 9, 2026
CANCELLED at the server level should mean "the client cancelled the request and went away." Currently, with the way that our dispatching layer works, a dispatching gRPC client can return with a CANCELLED error at its layer and then that bubbles up to the caller, which obscures what's going on at the server level.
A couple things here:
- Wrap the dispatcher remote boundary to catch
CANCELLEDerrors and turn them intoUNAVAILABLEs if the caller's context isn't already cancelled. This happens ininternal/dispatch/remote/cluster.goat lines 206, 780, 731, and 744. Make sure the context check is done on the dispatcher's inbound context rather than the hedging context. - Make hedging cancels more visible with
context.WithCancelCauseincluster.goat lines 552 and 557. We use this pattern atinternal/graph/cursors.go:451, and we work around the lack of this pattern atinternal/graph/lookupsubjects.go:354.
Source: authzed/spicedb