Backing up in new timezone causes restic to store every tree again, even unmodified ones
Output of restic version
restic 0.19.1 compiled with go1.26.5 on darwin/arm64What backend/service did you use to store the repository?
Local filesystem
Problem description / Steps to reproduce
TLDR: Do TZ=UTC restic backup -v ... or, more generally, TZ=some_timezone_you_never_used_before_with_this_backup restic backup -v ... and observe the number of modified trees.
More careful reproduction follows.
$ # Setup
$ mkdir -p /tmp/tz/data && cd /tmp/tz
$ for i in $(seq 1 300); do mkdir -p data/d$i; echo x > data/d$i/f; done
$ export RESTIC_REPOSITORY=/tmp/tz/repo RESTIC_PASSWORD=t RESTIC_CACHE_DIR=/tmp/tz/cache
$ restic init -q
$ # Initial backup, 302 new tree blobs
$ TZ=America/Los_Angeles restic backup -v data
open repository
repository c2c7f592 opened (version 2, compression level auto)
created new cache in /tmp/tz/cache
no parent snapshot found, will read all files
load index files
[0:00] 0 index files loaded
start scan on [data]
start backup on [data]
scan finished in 0.031s: 300 files, 600 B
Files: 300 new, 0 changed, 0 unmodified
Dirs: 301 new, 0 changed, 0 unmodified
Data Blobs: 1 new
Tree Blobs: 302 new
Added to the repository: 204.840 KiB (100.903 KiB stored)
processed 300 files, 600 B in 0:00
snapshot d4a9b371 saved
$ # Incremental backup, nothing new
$ TZ=America/Los_Angeles restic backup -v data
open repository
repository c2c7f592 opened (version 2, compression level auto)
using parent snapshot d4a9b371
load index files
[0:00] 100.00% 1 / 1 index files loaded
start scan on [data]
start backup on [data]
scan finished in 0.018s: 300 files, 600 B
Files: 0 new, 0 changed, 300 unmodified
Dirs: 0 new, 0 changed, 301 unmodified
Data Blobs: 0 new
Tree Blobs: 0 new
Added to the repository: 0 B (0 B stored)
processed 300 files, 600 B in 0:00
snapshot 1467725a saved
$ # BUG: change in timezone causes 302 new tree blobs again
$ TZ=Europe/Berlin restic backup -v data
open repository
repository c2c7f592 opened (version 2, compression level auto)
using parent snapshot 1467725a
load index files
[0:00] 100.00% 1 / 1 index files loaded
start scan on [data]
start backup on [data]
scan finished in 0.023s: 300 files, 600 B
Files: 0 new, 0 changed, 300 unmodified
Dirs: 0 new, 301 changed, 0 unmodified
Data Blobs: 0 new
Tree Blobs: 302 new
Added to the repository: 204.838 KiB (101.054 KiB stored)
processed 300 files, 600 B in 0:00
snapshot 2e343cc5 savedExpected behavior
The last backup above should say "Tree Blobs: 0 new", like the second one.
Actual behavior
See above.
Aside: Perhaps curiously, if you TZ=America/Los_Angeles restic backup -v data once more after doing those steps, the tree blobs will be de-duplicated against the previous TZ=America/Los_Angeles backup.
Do you have any idea what may have caused this?
#1238 fixed a part of this: a60e75121 switched to .Equal() so a timezone change no longer causes unchanged files to be re-read. But node timestamps are still serialized with the local UTC offset, and a tree blob's ID is the hash of that JSON. So changing timezones still invalidates every tree blob in the repository, even though no file content changed.
AFAIAU, this timezone that restic stores is not actually used anywhere. restic restore converts all times to UTC and (most) filesystems store times as UTC. (apparently FAT is an exception, but it doesn't seem to store the timezone either, it just naively stores local time)
Potenital solution: Just have restic store the UTC time. This costs a one-time invalidation of every tree in everyone's backup. Perhaps it could be an opt-in config for now and become the default later.
Workaround: On Linux/macOS, at lease, use TZ=UTC restic backup ... (or TZ=America/Los_Angeles or any other single timezone). This might have side effects for schedules (#3722) and has side effects for e.g. restic forget --keep-daily, but solves this specific issue with tree blobs.
Did restic help you today? Did it make you happy in any way?
Yes. :)
As ever, thank you for maintaining it!
Source: restic/restic