Distributor writes return 5xx during graceful ingester scale-down (no write extension)
Is your feature request related to a problem? Please describe.
The distributor builds the ingester write set with ring.WriteNoExtend. When an ingester is briefly non-ACTIVE, for example LEAVING during a graceful scale-down or a rollout, it is dropped from the replica set with no replacement. If there is no quorum slack (RF=2, or RF=3 when several ingesters change at once), ring.Get returns at least N live replicas required and the write fails with 5xx, even though the shutdown was graceful.
With ingester autoscaling this shows up as recurring write 5xx and constant retries from clients (Promtail, Fluentd, and so on) during scale-down and rollouts.
Describe the solution you'd like
Let the distributor use ring.Write, which replaces a LEAVING or JOINING ingester with the next healthy one and keeps write quorum while the ring changes, behind an opt-in flag that defaults to off. Mimir already exposes -distributor.extend-writes for the same purpose. Reads already include LEAVING instances and extend, so anything written to the extension target is still queried.
#23908 implements this.
Describe alternatives you've considered
Running with more quorum slack (higher replication factor) avoids the failures but costs more, and does not help when several ingesters change at once.
Source: grafana/loki