#8855·citus

Removed workers can roll back transactions belonging to the original coordinator

Author: serhatandicCreated Sep 18, 2026Updated Sep 18, 2026

Problem

Removing a worker resets its local group ID to 0 (the coordinator identity), but can leave its old node-directory entries in pg_dist_node. If the worker remains running, its transaction-recovery process can still contact the old cluster and mistake the original coordinator's prepared transactions for its own.

A prepared transaction contains durable changes that are still waiting for a final commit or rollback decision.

Example

  1. Coordinator C durably records that a transaction on worker A must commit, but A is still waiting for the final commit instruction.
  2. An unrelated worker B is removed. Its group ID becomes 0, but it retains its route to A.
  3. Recovery on B finds the prepared transaction on A and treats it as B's own. B has no matching commit-decision row in its local pg_dist_transaction table; that row belongs to C. B can therefore incorrectly tell A to roll back.

This can discard changes that the original coordinator already decided to commit, potentially leaving a distributed transaction only partially applied.

Expected behavior

Removing B must not let B interfere with transactions owned by C. A detached worker must stop using its old cluster routes, and recovery must not act on stale membership or a changing local identity. Detachment must preserve physical user tables.

CI context

Identified while investigating #8850. In the PG17 regression job, multi_colocated_shard_rebalance reported a missing prepared transaction during commit, followed by a missing shard table.

The mechanism above is a source-supported explanation, but the exact rollback actor in that CI run is not confirmed. Affected releases and production impact have not been established.