#21120·vitess

VTOrc: partition shard monitoring across instances with a consistent hash ring

Author: corbantekCreated Sep 15, 2026Updated Sep 16, 2026

Overview

Every VTOrc instance currently watches the entire topology — all keyspaces and shards. For large fleets this is a scaling ceiling: each instance processes topology events and health checks for every shard, and adding instances gives HA but not horizontal scale.

--clusters_to_watch allows static partitioning, but it requires enumerating keyspaces/keyranges and editing config whenever keyspaces are created or deleted — impractical for fleets with high keyspace churn.

Proposal

Add an optional consistent hash ring so a pool of VTOrc instances can split shard-monitoring responsibility deterministically, with no keyspace lists:

  • --vtorc-ring-size N (total instances in the ring) and --vtorc-ring-index i (this instance's 0-based position).
  • Each shard's primary owner is fnv32a(keyspace + "/" + shard) % ring_size.
  • The two ring-adjacent instances also watch each shard, giving 3-way HA coverage per shard (a primary crash/restart always leaves two instances already watching).
  • Fully backward compatible: default ring-size=1 watches everything (today's behavior). Ring sizes ≤3 are a deliberate no-op (every instance is primary and both neighbors for every shard), so partitioning takes effect at ring-size >= 4.

Even distribution (optional)

Raw hash % N can be uneven depending on keyspace naming. An optional --vtorc-ring-assignments-file maps virtual hash buckets to ring partitions for even load, without listing keyspace names (the file is just {"num_buckets": …, "bucket_assignments": […]}). Unset falls back to direct hash modulo, which is fine for small rings.

Backward compatibility

Opt-in and off by default; unset flags = current behavior. Composes with --clusters_to_watch (ring gating is applied on top of the existing filter).

Alternatives considered

  • Static --clusters_to_watch partitioning — operationally painful with ephemeral keyspaces.

An implementation PR is opened alongside this issue. Feedback on the design and flag naming is very welcome before it's reviewed in depth.