#11469·seaweedfs

Volume server startup is much slower since 4.46

Author: bernardxCreated Sep 26, 2026Updated Sep 26, 2026

Describe the bug

After upgrading a volume server from 4.45 to 4.46, startup takes forever. The disks are read-bound and at the default log level nothing is printed, so the server looks hung.

The cause is Volume.recoverLastModifiedTs, added in #11167. It runs on every TTL volume at every load. For a volume that has been vacuumed (CompactionRevision > 0), findLastWriteAppendAtNs walks the .idx backwards and, for up to vacuumedLastWriteScanEntries (65536) live entries, does two random reads into the .dat: ReadNeedleHeader in findNeedleOffset, then readNeedleAppendAtNs. That is up to ~131k random reads per volume.

When the volume holds more live needles than the budget, the scan gives up and keeps the .dat mtime anyway. So on large volumes the whole scan is wasted work, and it is repeated on every restart because the result is not persisted.

Downgrade to 4.45 resolved this problem.

Logs (-v=2)

I0926 04:39:21.921801 volume_loading.go:320 loading memory index /data_temp/temp_329.idx to memory
I0926 04:39:26.132383 needle_map_memory.go:56 max file key: fcdd2ec count: 521217 deleted: 13961 for file: /data_temp/temp_329.idx
I0926 04:39:26.132443 disk_location.go:306 data file /data_temp/temp_329.dat, replication=000 v=3 size=8017109216 ttl=1M disk_id=2
I0926 04:42:33.896889 volume_loading.go:320 loading memory index /data_temp/temp_937.idx to memory

There is a ~3 minute gap with no output while the loading workers are inside recoverLastModifiedTs. temp_329 has 521k needles, far above the 65536 budget, so its scan could never succeed.

Expected

Loading TTL volumes should not add minutes of random I/O per restart, especially when the result falls back to the mtime.

Related: #11160, #11167