feat(runtime): generalize view-synchronous ownership coordination

Summary

Extract a small, reusable ownership-coordination core from the work in #10969 while keeping the grain directory's execution engine, assignment policy, transport, and recovery behavior recognizable and service-specific.

The same coordination primitives should support both:

• Direct assignment of a finite resource set, such as Event Hub partitions assigned to silos.
• Hash-range ownership, as used by the grain directory.

This proposal captures the current design sketch and implementation plan. The abstractions should improve the directory even if it remains their only production consumer.

Motivation

The existing directory already has reusable concepts: canonical views, acquisition and release prerequisites, blocking transitions, fencing, and recovery. Its current transition type exposes methods for multiple directions, with runtime checks rejecting invalid combinations.

We can strengthen those APIs and isolate their invariants without creating a general-purpose service framework. The useful separation is:

The directory should retain its configured partitioning, system-target identities, RPC contracts, GrainAddress, membership recovery watermark, and activation-publication barrier.

Proposed design

1. Small view contracts

internal interface IClusterServiceView<TViewId>
{
    TViewId Id { get; }
    bool TryGetPredecessor(out TViewId predecessor);
}

internal interface IClusterServiceViewProvider<TViewId, TView> : IAsyncDisposable
    where TView : IClusterServiceView<TViewId>
{
    bool TryGetCurrentView([MaybeNullWhen(false)] out TView view);
    IAsyncEnumerable<TView> ViewUpdates { get; }
    ValueTask<TView> RefreshAsync(CancellationToken cancellationToken);
    ValueTask<TView> RefreshAtLeastAsync(
        TViewId minimumView,
        CancellationToken cancellationToken);
}

A view ID identifies one immutable, authoritative definition within a logical service. Components which compare IDs constrain them with IComparable<TViewId> and IEquatable<TViewId>.

The provider chooses the ID representation. Concrete views carry their own topology and other metadata. Classes and structs can implement the interface directly; shared consumers retain concrete generic representations where useful.

Explicit predecessor information matters: ordered IDs alone do not prove that no intervening ownership view was skipped. Adapters resolve gaps through authoritative history or their recovery protocol.

2. Typed gate hierarchy

TransitionGate<TViewId>
  OwnershipAcquisition<TViewId>
  OwnershipRelease<TViewId>
  ViewBarrier<TViewId>

The abstract base owns TargetView, Completion, common lifecycle Status, IsBlocking, and failure/shutdown handling. It exposes no public Complete or role-specific progression API.

Acquisition and release carry PreviousView; a barrier needs only its target view. The direction constructor parameter and combined direction/stage enum are replaced by the type hierarchy and role-specific phase enums.

Common lifecycle status is Pending, Completed, Failed, or Aborted. IsBlocking is true for pending and failed gates. Completion is the sole exception store: Fail(exception) faults it with the original exception. Faulted tasks therefore continue to constrain admission until terminal shutdown releases the gate.

Phase changes, completion validation, failure, and abort share the same synchronization boundary. Concrete Complete methods invoke a protected base mechanism which validates readiness while holding that lock. Generic role implementations permit named, sealed service subclasses; completion-validation overrides are sealed.

Use sealed directory-specific classes such as DirectoryAcquisition, DirectoryRelease, DirectoryBarrier, and DirectoryTransitions, rather than using type aliases.

3. Maps own associations; gates own lifecycle

• ResourceTransitionGateMap<TResourceId, TViewId> maps exact resource identities to gates.
• RangeTransitionGateMap<TViewId> stores RingRange-to-gate associations and selects overlapping gates.

Maps retain base handles and inspect only the common properties. Service drivers retain concrete handles and invoke the applicable progression methods.

A relevant gate blocks a request when its target view is at or before the required view and it remains blocking. Waiters re-query after awaiting a selected gate. Maps prune released gates, not all tasks whose IsCompleted is true.

Map queries are read-only and single-pass. Registration and completion/shutdown paths own collection maintenance. The gate does not call back into the map while holding its lock.

4. Use the existing 32-bit RingRange directly

Keep RingRange's established empty/full, wrapping, and exclusive-start/inclusive-end behavior, including its serializer identity and fields. The hash-range map needs no range type parameter or geometry interface.

A service with a 64-bit hash can project its high 32 bits onto the ring. Expanded-boundary facades must preserve the existing endpoint convention. More general geometries can be introduced when there is a concrete consumer.

5. Direct-resource changes use per-owner sets

For a local owner:

retained = oldOwned intersect newOwned
released = oldOwned minus newOwned
acquired = newOwned minus oldOwned

Enumerate the two differences directly:

foreach (var resource in previouslyOwned)
{
    if (!currentlyOwned.Contains(resource))
    {
        BeginRelease(resource);
    }
}

foreach (var resource in currentlyOwned)
{
    if (!previouslyOwned.Contains(resource))
    {
        BeginAcquisition(resource);
    }
}

Hash-based sets give expected O(|oldOwned| + |newOwned|) local comparison work. The provider supplies or maintains the owner-to-resource sets alongside the resource-to-owner map. Account separately for constructing those sets; building a full inverse map on each silo still costs a full view traversal there.

Build both representations together and publish them immutably. The sketch uses per-owner frozen sets; their construction cost is part of view production, not hidden in the local-delta claim.

An Event Hub resource identity includes namespace, hub, consumer group, and partition ID. Its adapter stops and drains the old receiver, checkpoints/releases ownership, establishes the destination's fencing, and installs a gated receiver. The pump rechecks current ownership, current gates, and receiver identity after waits.

A resource moving A -> B -> A through skipped views must not be treated as continuously owned by A. Preserve the history/recovery boundary. Checkpoint replay, receiver fencing, and downstream effect guarantees belong to the Event Hub adapter.

6. Keep directory planning local and ring-aware

Retain the current per-local-partition range differences. Unchanged local partitions start no transfer work. For changed ranges, query the existing ordered topology instead of constructing a global Cartesian product of old and new assignments.

Add output-sensitive owner visitation to ClusterServiceTopology, forwarded by DirectoryMembershipSnapshot. Reuse its sorted, collision-resolved boundaries and existing point search:

internal void VisitRangeOwners<TState>(
    RingRange query,
    Action<ClusterServicePartitionOwner, TState> visitor,
    TState state)
{
    ArgumentNullException.ThrowIfNull(visitor);
    var count = _ringBoundaries.Length;
    if (query.IsEmpty || count == 0)
    {
        return;
    }

    if (query.IsFull || count == 1)
    {
        for (var index = 0; index < count; index++)
        {
            visitor(GetOwner(index), state);
        }
        return;
    }

    var firstPoint = unchecked(query.Start + 1u);
    var indexOfOwner = SearchAlgorithms.RingRangeBinarySearch(
        count,
        this,
        static (topology, index) => topology.GetRangeCore(index),
        firstPoint);
    Debug.Assert(indexOfOwner >= 0);

    var queryLength = unchecked(query.End - query.Start);
    for (var visited = 0; visited < count; visited++)
    {
        var owner = GetOwner(indexOfOwner);
        visitor(owner, state);

        var distanceToBoundary = unchecked(owner.Range.End - query.Start);
        if (distanceToBoundary == 0 || distanceToBoundary >= queryLength)
        {
            return;
        }

        if (++indexOfOwner == count)
        {
            indexOfOwner = 0;
        }
    }
}

This sketch relies on the existing nonempty ring covering every hash point. The exclusive start requires searching for Start + 1, with unsigned wrap. A zero boundary distance in the walk denotes a full turn from the excluded start. The owner-count bound prevents reporting an owner twice when a wrapping query intersects it in two pieces.

For example, owners (0,100], (100,200], and (200,0] all intersect query (50,25], even though its first and last covered points have the same owner.

Cost is O(log P + K) for a partial query and O(P) for full-ring output. Across local changed ranges it is O(R log P + Ktotal). Traversal creates no intermediate owner collection or duplicate-tracking set.

Report each partner's full owner range once. Preserve one transfer/acknowledgement unit per source partner; the existing transfer operation owns intersection batching and acknowledges only after all required data is installed. Independently acknowledging intersection fragments could retire source state prematurely.

Directory maintainability and hot-path requirements

• Keep membership projection, ownership representations, RPCs, and recovery logic directory-native. Avoid a second translating topology facade and generic assignment/transfer-edge model.
• Preserve the direct WaitForRange fast path: a sufficiently recent partition view and no blocker return ValueTask.CompletedTask without an extra installation service or scheduler hop.
• Install gates synchronously before transfer work begins or the new local serving decision becomes visible.
• Source release waits at its predecessor view so it does not wait on its own newer outbound gate. Admission rechecks ownership after waits.
• Private directory acquisition/release/barrier entrypoints own registration through completion and pruning. Typed finish helpers centralize failure/shutdown policy and cleanup without exposing completion through the base handle.
• Preserve directory integrity-barrier behavior, range-operation diagnostics, and the fatal-error policy. A still-pending gate whose completion validation fails must fault and remain blocking.
• Measure the gate's per-instance synchronization object and monitor-based phase transitions separately from ordinary request lookup. Optimize synchronization only while preserving atomic readiness validation and the scheduler contract.

Implementation plan

1. Establish behavior baselines using existing controlled directory protocol, transition, membership-provider, and directory lifecycle tests. Preserve ring ownership, wire contracts, registration barriers, leases, and shutdown semantics.
2. Extract typed gates and the concrete hash-ring gate map. Use named sealed directory bindings, with no parallel ownership representation or new configuration framework.
3. Centralize directory-owned completion/pruning and keep lookup queries read-only. Cover failures immediately after registration, before/after awaits, completion validation, cancellation, and shutdown.
4. Implement ordered-ring owner queries as part of the production path. Compare with a full-scan oracle; cover empty/full/point/wrapped ranges, exact boundaries, collisions, and the same-first/last-owner case. Count search probes and visitor calls to establish logarithmic search plus output-sensitive traversal.
5. Compare steady-state lookup/register allocations, blocked requests, and membership churn against the current directory. Require no added steady-state per-request allocation or scheduler hop and no planning-complexity regression. Measure total gate allocation bytes/object counts before selecting synchronization optimizations.
6. Validate on supported target frameworks, currently .NET 8 and .NET 10, with expensive cluster/stress validation in CI. Keep the explicit-resource consumer as a focused reference until a production service adopts it.

Boundaries

This issue covers reusable coordination and a behavior-preserving directory integration. It leaves configuration-change/epoch rollout protocols and additional directory RPC metadata for separate design work.

Hash-range-aware entry storage is tracked separately in #11207: finding transfer partners efficiently does not itself accelerate scanning or extracting registrations. That work should be evaluated independently of the gate-library extraction.

Acceptance criteria

• The directory benefits from stronger APIs and isolated invariants even if it is the only production consumer.
• Incorrect direction-specific progression is unavailable through the concrete API; temporal ordering remains guarded at runtime.
• Failed gates retain the original exception and continue blocking; caller cancellation does not cancel shared progress.
• Resource deltas use local owned sets, with view-construction costs accounted for.
• Ordered-ring queries produce exactly the full-scan partner set, once per partner, with the required complexity and acknowledgement semantics.
• Directory routing, handoff, recovery, leases, registration publication, and existing RPC payloads preserve their intended behavior.
• The steady-state request path remains allocation- and scheduler-hop-neutral relative to the baseline; control-plane costs are documented with measurements.