Auto-persist dynamically discovered peer IPs

Auto-persist dynamically discovered peer IPs

TL;DR

EasyTier's CLI daemon has no mechanism to automatically persist dynamically discovered peer IPs to disk. After a node restart (or a full-network power outage), the daemon has no record of the IPs it was just talking to — it can only reconnect via the static [[peer]] entries in its config. In residential / dynamic-IP environments, the very IPs that get written into [[peer]] are the ones most likely to change, so this leaves the network one restart away from split-brain.

I'd like to discuss the design before writing code. Several related issues (#1697, #1796, #1746, #2410, #1593) are adjacent but none addresses this gap.

Environment

• EasyTier 2.6.4 on Linux (NixOS).
• 3–6 nodes, fully-meshed overlay. Some have dynamic IPv4 (port-forwarded), some have dynamic IPv6 (prefix changes on reconnect).
• Bootstrap: 1–2 stable [[peer]] entries. Once the mesh is up, PEX handles the rest.
• Happy path already works (PEX converges in seconds, all peers go p2p).

Failure Mode

When a node's public IP changes (re-dial, ISP prefix rotation, VM reboot):

1. Other nodes notice via PEX and reconnect. ✅
2. The changed node restarts (crash, power outage, NixOS rebuild). ❌
3. Restarted node still has the old [[peer]] IPs in its config.
4. If the configured peers haven't changed → recovery via static entry. ✅
5. If the configured peers are the ones whose IPs changed → the restarted node has no live [[peer]] and no record of the IPs it was talking to 60 seconds ago. → split-brain.

What I Already Tried

--config-dir

Docs say:

Load all .toml files in the directory to start network instances, and store the received configurations in this directory.

I ran easytier-core --config-dir /tmp/et-test with a 3-peer mesh for 2 minutes. The initial .toml was loaded; zero bytes were written to the directory for the entire run. Reading easytier/src/rpc_service/instance_manage.rs (v2.6.4) confirms std::fs::write is only invoked from the run_network_instance RPC path (web/GUI control plane), and only with the current cfg snapshot — not the runtime PEX peer graph.

easytier-web --db et.db

From #1796, the official answer for config persistence. Per #2410, --config-dir is what the Windows GUI uses. Both persist user-issued configuration, not the runtime peer-graph. PEX-learned IPs never reach either store.

.machine_id file

strings easytier-core shows easytier already persists a stable per-node UUID across restarts (via easytier/src/common/machine_id.rs). This is exactly the right pattern for state-file persistence — it just isn't extended to cover peer IPs.

Related Issues / PRs

• #1006 / #1697 — wake-from-sleep reconnect; fixed by #1593 (clears alive_conn_urls, forces reconnect). In-memory fix — doesn't help if the changed node is the one that restarted and its static [[peer]] is stale.
• #1796 — config persistence in Docker; answered with easytier-web --db. User-config scope, not runtime state.
• #1746 — Docker --config-dir request; closed via #1540. Same scope.
• #2410 — Windows GUI service --config-dir bug. User-config scope.

None covers the "I lost my static peers because they changed; please remember what PEX showed me" scenario.

Possible Designs

I'd lean A + B: a flag that writes JSON/TOML adjacent to .machine_id, scoped to the existing machine_id + network_name digest so a fresh Nix generation can't load another node's cache. But I'd like your steer first.

Why An Issue, Not a PR

I'd like to know:

1. Whether you see this as a desired feature at all, or whether the official position is "use easytier-web + a stable bootstrap peer".
2. Which design (A–D, or something else) is most consistent with the project's direction.
3. Whether you'd accept a PR implementing the chosen design behind a flag.

If the answer to (1) is "no, we don't want this in core", I'll write the persistence layer as an out-of-tree sidecar (easytier-cli scrape + systemd timer) and document it as a community pattern. Either outcome helps — I'd just like to know before investing the time.

Repro (1 minute)

# Node A: stable peer, hardcoded in B's [[peer]]
# Node B: dynamic-IP, currently connected to A via PEX

# On Node B:
easytier-cli -n <inst> peer -o json   # shows Node A's current IP
# Restart Node B's easytier-core.
easytier-cli -n <inst> peer -o json   # can no longer reach A if A's IP changed

# Expected: Node B recovers via a persisted peer-IP cache.
# Actual: Node B is stuck until A's new IP is manually added to [[peer]].

Additional Context

• EasyTier version: 2.6.4 (nix store wrdcalsxgxvr1d3mb4q4p1k4imaq04id).
• Files inspected: easytier/src/common/config.rs, easytier/src/rpc_service/instance_manage.rs, easytier/src/rpc_service/config.rs, easytier/src/common/machine_id.rs.
• Happy to prototype whichever design is endorsed.