Patroni failing to elect a new leader with synchronous_mode_strict
What happened?
Patroni is failing to elect a new leader in a three node cluster with synchronous_mode_strict enabled. Since strict mode replicates transactions to at least 1 other node, then the cluster should recover if at least 2 nodes (out of the 3) are up and one of them is a synchronous replica.
How can we reproduce it (as minimally and precisely as possible)?
docker-compose up -d (from https://github.com/patroni/patroni/blob/master/docker-compose.yml#L7)
patronictl edit-config , add:
synchronous_commit: remote_apply synchronous_mode_strict: true synchronous_mode: true synchronous_node_count: 2patronictl list
- Cluster: demo (7488968659735597079) -+-----------+----+-----------+ | Member | Host | Role | State | TL | Lag in MB | +----------+------------+--------------+-----------+----+-----------+ | patroni1 | 172.18.0.8 | Sync Standby | streaming | 1 | 0 | | patroni2 | 172.18.0.7 | Sync Standby | streaming | 1 | 0 | | patroni3 | 172.18.0.6 | Leader | running | 1 | | +----------+------------+--------------+-----------+----+-----------+
- docker stop demo-patroni1
- docker stop demo-patroni2
- docker exec -ti demo-patroni3 bash postgres@patroni3:~$ patronictl list
- Cluster: demo (7488968659735597079) -----+----+-----------+ | Member | Host | Role | State | TL | Lag in MB | +----------+------------+--------+---------+----+-----------+ | patroni3 | 172.18.0.6 | Leader | running | 1 | | +----------+------------+--------+---------+----+-----------+
- docker stop demo-patroni3
- docker start demo-patroni1
- docker start demo-patroni2
- docker exec -ti demo-patroni1 bash postgres@patroni1:~$ patronictl list
- Cluster: demo (7488968659735597079) ------+----+-----------+ | Member | Host | Role | State | TL | Lag in MB | +----------+------------+---------+---------+----+-----------+ | patroni1 | 172.18.0.6 | Replica | running | 1 | 0 | | patroni2 | 172.18.0.7 | Replica | running | 1 | 0 | +----------+------------+---------+---------+----+-----------+
- Cluster fails to elect a new leader
What did you expect to happen?
Because synchronous_mode_strict is enabled then in a three node cluster, I expect Patroni to elect a new leader if at least 2 nodes are up and one of them is a synced replica.
Patroni/PostgreSQL/DCS version
- Patroni version: master
- PostgreSQL version: 16
- DCS (and its version): etcd 3.3.13
Patroni configuration file
Configuration file created by https://github.com/patroni/patroni/blob/master/docker-compose.yml#L7patronictl show-config
patronictl show-config
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:
parameters:
max_connections: 100
synchronous_commit: remote_apply
pg_hba:
- local all all trust
- host replication replicator all md5
- host all all all md5
use_pg_rewind: true
retry_timeout: 10
synchronous_mode: true
synchronous_mode_strict: true
synchronous_node_count: 2
ttl: 30Patroni log files
docker logs demo-patroni3
2025-04-03 06:39:15,565 INFO: Lock owner: patroni3; I am patroni3
2025-04-03 06:39:15,657 INFO: Updating synchronous privilege temporarily from ['patroni2'] to []
2025-04-03 06:39:15,701 WARNING: No standbys available!
2025-04-03 06:39:15,701 INFO: Assigning synchronous standby status to ['*']
server signaled
2025-04-03 06:39:15.703 UTC [32] LOG: received SIGHUP, reloading configuration files
2025-04-03 06:39:15.703 UTC [32] LOG: parameter "synchronous_standby_names" changed to "*"PostgreSQL log files
N\AHave you tried to use GitHub issue search?
- Yes
Anything else we need to know?
No response
Source: patroni/patroni