feat(runtime): support extensible cluster service view sources

Summary

Follow-up to #10969: generalize the internal view-synchronous cluster-service machinery around an authoritative, ordered service-view source, while retaining membership-derived assignment as the default implementation.

#10969 has a deliberately bounded contract: fixed service configuration plus group membership determines assignment. Unchanged inputs produce the same assignment. This proposal extends the sources of authoritative views to support:

• Service-specific participation, such as only opted-in silos hosting a service.
• Explicit partition mappings published through a shared, strongly consistent register with monotonic revisions.
• Coordinated placement changes while cluster membership remains unchanged.

Implement this separately from #10969 so that extracting the existing protocol and extending its authority model remain independently reviewable.

Core model and guarantees

Service views govern placement; cluster membership supplies liveness information. The default provider derives both from cluster membership. A register-backed provider obtains placement from the register and observes membership to detect failed transfer partners and trigger proposals for replacement owners.

The common contract should establish:

1. Canonical identity: a service-view identity identifies exactly one participant set and assignment, including silo incarnations and relevant configuration.
2. Authoritative ordering: revisions are monotonically ordered within one configured authority. Per-silo counters and combinations of independently observed membership/register versions do not establish this order.
3. Atomic publication: participants, assignment, and relevant configuration are published/read as one authoritative snapshot. Concurrent mapping writers use conditional publication against the expected register revision.
4. Explicit continuity: ordinary handoff requires proof that the target view directly follows the installed view. Numeric previous + 1 is sufficient only when the source guarantees consecutive revisions. Otherwise use authoritative predecessor information; skipped or unproven continuity selects recovery.
5. Ownership activation: a new owner installs state and establishes the service's fencing conditions before serving the acquired partition. View publication identifies the owner; transfer, leases, or storage-side epochs establish the authority to act.
6. Explicit readiness: initialization and refresh expose unavailable/canceled/failed outcomes until an adequate view is available. A successful refresh satisfies its requested minimum revision.

For a register-backed mapping, replacement of a failed owner is installed through a newly published register revision. Until then, the affected partition remains unavailable. Newer liveness observations can stop unsafe work and trigger recovery decisions while preserving the mapping identified by the installed service view.

Illustrative C# abstractions

These are proposed internal shapes to guide design, rather than finalized public APIs or serialization layouts:

internal readonly record struct ClusterServiceViewVersion(long Value);

// Evolved shape of the existing identity; ordering is scoped to AuthorityId.
internal readonly record struct ClusterServiceViewId(
    string AuthorityId,
    ClusterServiceViewVersion Version,
    int ProtocolVersion,
    string ConfigurationFingerprint);

internal readonly record struct ClusterServicePartitionAssignment(
    string PartitionId,
    RingRange Range,
    SiloAddress Owner);

internal sealed record ClusterServiceView(
    ClusterServiceViewId Id,
    ClusterServiceViewId? PreviousViewId,
    MembershipVersion MembershipWatermark,
    ImmutableArray<SiloAddress> Participants,
    ImmutableArray<ClusterServicePartitionAssignment> Assignments);

internal interface IClusterServiceViewProvider : IAsyncDisposable
{
    ClusterServiceView? CurrentView { get; }

    IAsyncEnumerable<ClusterServiceView> ViewUpdates { get; }

    ValueTask<ClusterServiceView> RefreshAsync(
        ClusterServiceViewVersion? minimumVersion,
        CancellationToken cancellationToken);
}

Design points to settle:

• AuthorityId identifies the configured authority/epoch namespace for a service. Revisions from different authorities require an explicit transition/bootstrap policy rather than numerical comparison. Register recreation or restoration must preserve this distinction.
• PreviousViewId, when present, identifies the authoritative predecessor, not merely the previous view observed by this reader.
• MembershipWatermark records any minimum cluster-membership knowledge needed to interpret the view. Current liveness observations remain independently refreshable.
• PartitionId is scoped to the service and has explicit identity semantics across owner changes. Preserve existing directory partition addressing in the default adapter.
• ConfigurationFingerprint describes compatibility-relevant configuration; the view revision identifies the particular mapping. Ordinary mapping changes should support normal handoff when their configuration and lineage are compatible.
• A null minimum revision requests a refresh without a minimum-version constraint. A null current view represents initialization awaiting an authoritative snapshot.
• Retain ring-range assignments for the first increment. Discrete queue/shard topology can be introduced when a concrete consumer requires it.

Providers

Membership-derived provider: adapt ClusterServiceMembership to the provider contract. Preserve the current deterministic ring algorithm, empty directory startup baseline, and membership-version ordering. Existing directory behavior remains the default.

Service-specific participant groups: derive assignment from a commonly observed, versioned eligible-participant set. Dynamic opt-in/opt-out changes must enter that authoritative group view. Configuration and capability information used for eligibility must be shared inputs with defined consistency semantics.

Register-backed provider: read complete, immutable assignment snapshots from a consistent register and publish updates using compare-and-swap or an equivalent conditional-write primitive. Reads/subscriptions may skip revisions; the provider must expose enough lineage information to distinguish handoff from recovery. Start with one concrete reference implementation to exercise the contract, and choose the register backend during design.

Concrete changes required

• View identity and version comparisons: replace direct dependence on ClusterServiceViewId.MembershipVersion in transition ordering, readiness gates, refresh, and operation-result handling with source-neutral service-view ordering.
• Topology: separate ClusterServiceTopology's assignment representation and lookup logic from its construction using ClusterMembershipSnapshot. Support both projected and explicitly supplied mappings. Establish participant eligibility, partition identity, range-overlap, and coverage invariants at the assignment-source boundary.
• Transition coordinator: make BeginInbound, BeginOutbound, barriers, and blocking queries operate on service-view identity/version. Use the provider's continuity contract to choose handoff versus recovery. Preserve predecessor draining, state installation, fail-closed behavior, and fatal-error observation.
• Fencing: keep service-view revision, cluster-membership version, and provider fencing tokens conceptually distinct. Review ClusterServiceFence and its consumers so that installing a placement revision is not interpreted as proof that an external storage or delivery fence has been established.
• Directory adapter: adapt DirectoryMembershipService and DirectoryMembershipSnapshot to consume the default provider while retaining directory RPC aliases, payload field IDs, target addressing, and MembershipVersion on the existing wire. Define a separate compatibility/migration path before an alternate authority is used by the directory.
• Hosting and lifecycle: configure a provider per service, including its authority namespace and participation policy. Specify startup readiness, refresh cancellation, stream termination, disposal, and failure reporting.
• Serialization and diagnostics: preserve existing serialized-field meanings; evolve contracts additively or introduce distinct aliases where necessary. Include service identity, authority, view revision, predecessor, and transition state in diagnostics.

Acceptance criteria

• The membership-derived directory preserves its current assignment, addressing, startup, handoff/recovery, and wire behavior.
• A register mapping advances while cluster membership stays fixed, and both old and new owners follow the full transition protocol.
• Service opt-in/opt-out produces a canonical participant view and consistent assignment across observers.
• Concurrent register writers, repeated reads, skipped revisions, and authority-namespace changes have explicit, deterministic outcomes.
• Delayed old-view requests and snapshot responses preserve newer ownership/state; unavailable authority or failed transitions keep affected operations safely gated.
• Refresh never succeeds below its requested revision; cancellation and provider termination propagate to pending operations.
• Provider conformance scenarios exercise canonical assignment, ordering, continuity, readiness, and recovery using the production transition machinery and an independent ownership/state oracle.

Initial scope

Keep the abstractions internal and implement the membership-derived adapter plus a meaningful alternate view source. Service conversions such as reminders, streaming, or DurableJobs can follow independently, with their own state-transfer, storage-fencing, and external-effect contracts. A public extensible GrainService API should be shaped by experience with concrete consumers.