#1006·pingora

Zero-weight backend panics KetamaHashing and kills the load balancer's background update task

Author: sinhaparth5Created Sep 10, 2026Updated Sep 10, 2026

A backend with weight 0 reaching KetamaHashing permanently kills the load balancer's background update task (or aborts the process under panic = "abort").

Where

  • pingora-load-balancing/src/selection/consistent.rs, KetamaHashing::build_with_config, passes b.weight as u32 straight into pingora_ketama::Bucket::new.
  • pingora-ketama/src/lib.rs, Bucket::new: assert!(weight != 0, "weight must be at least one").

How it triggers

Backend::weight is a plain, unvalidated usize (pingora-load-balancing/src/lib.rs). Neither Backend::new_with_weight nor the ServiceDiscovery trait rejects 0. Some discovery backends use weight 0 as a draining/disabled marker, and a typo in a static config or DNS SRV record produces the same thing. Once one such backend is in the BTreeSet<Backend> passed to KetamaHashing::build_with_config, Bucket::new panics.

That call happens inside LoadBalancer::update() (pingora-load-balancing/src/lib.rs), driven by the background loop in pingora-load-balancing/src/background.rs (LoadBalancer::run). Nothing wraps it in catch_unwind or spawn_blocking. The newer LoadBalancerGroup rebuild path already runs build_selector inside spawn_blocking and checks JoinError::is_panic(), so this class of panic is handled there, just not on the plain LoadBalancer<S> path.

Impact

Once the panic fires, the spawned update task ends and is never restarted:

  • Backend membership changes (new/removed hosts) stop being applied.
  • Health check state freezes at whatever it was.
  • Under panic = "abort", the whole process goes down instead.

All from a single zero-weight entry in one discovery response, when the consumer uses consistent hashing (a common choice for cache or session-sticky routing).

Suggested fix

Clamp the weight to a minimum of 1 at the Backend to Bucket boundary in KetamaHashing::build_with_config, since Backend::weight is untrusted, unvalidated input and Bucket requires a positive weight by contract.