cmd/changelog: support applying changelogs to catch up with the source volume
What would you like to be added:
Add a command to apply metadata changelogs from a source volume to a destination volume, allowing the destination to catch up with the latest metadata changes after being initialized from a binary metadata backup.
Proposed usage:
juicefs changelog apply SRC-META-URL DST-META-URL --backup meta.binThe command should:
- Start from the
lastChangelogversion recorded in the baseline backup, or from a version specified by--from. - Apply changelog entries in order and keep following new entries by default.
- Support stopping after catching up with
--follow=false. - Handle the TiKV rewind window and deduplicate entries already included in the baseline backup.
- Validate that the source, destination, and backup belong to compatible volumes.
- Support dry-run validation and configurable error handling.
- Work consistently across Redis, SQL, and KV metadata engines.
The source and destination must share the same object storage because changelogs contain metadata changes only and do not copy file data.
Why is this needed:
A binary metadata backup only captures the source volume at a point in time. Changes made while the backup is being loaded leave the destination behind the source.
Applying subsequent changelog entries allows the destination to catch up without taking another full backup. This provides the incremental synchronization step needed for metadata migration, disaster recovery preparation, and maintaining a standby volume.
Related implementation: #7514
Source: juicedata/juicefs