CLI daemon deletes database entries at startup: the initial scan is skipped when the file cache is partially warmed

Summary

On a headless CLI daemon, the startup mirror scan deleted 76 documents from CouchDB four seconds after the container restarted. Every one of those 76 files was present on local storage at that moment, and still is.

The cause is in NodeFileSystemAdapter.getFiles(): it decides whether an initial directory scan is needed by looking at fileCache.size. Because the daemon replicates from CouchDB before it scans, the cache is already partially populated by refreshFile(), so the scan is skipped and getFiles() returns a truncated listing. The reconciliation logic then sees the remaining documents as locally deleted and removes them from the database.

Version: CLI 1.0.18. The flawed method is unchanged on main (011a840).

Evidence

The same file, healthy at 07:14, deleted at 09:54 — the container restarted at 09:54:42:

2026-08-27T07:14:10.741Z [Daemon] STORAGE == DB :Inbox/veille-cyber.md
2026-08-27T09:54:42Z      (container restarted)
2026-08-27T09:54:46.415Z [Daemon] NEWER_WINS: Treating missing local file as deletion (Inbox/veille-cyber.md)
2026-08-27T09:54:46.415Z [Daemon] DELETE DATABASE: Inbox/veille-cyber.md
...
2026-08-27T09:54:46.700Z [Daemon] Synchronisation completed: 528 files processed (98 completed, 430 skipped, 0 failed)
2026-08-27T09:54:46.700Z [Daemon] Initialized, NOW TRACKING!
2026-08-27T09:54:46.700Z [Daemon] Mirror scan complete

0 failed — the run reported success.

The file was, and remains, readable inside the container:

$ docker exec livesync-cli ls -la /data/Inbox
-rw-r--r-- 1 node node 10428 Aug 27 04:15 veille-cyber.md

76 DELETE DATABASE lines in that single scan.

Root cause

1. The scan is skipped when the cache is non-empty — src/apps/cli/adapters/NodeFileSystemAdapter.ts:

async getFiles(): Promise<NodeFile[]> {
    if (this.fileCache.size === 0) {
        await this.scanDirectory();
    }
    return Array.from(this.fileCache.values());
}

fileCache has two writers: scanDirectory() (full, recursive) and refreshFile(p) (one entry). A single refreshFile() call is enough to make the cache non-empty and suppress the initial scan permanently.

2. The daemon warms the cache before scanning — src/apps/cli/commands/runCommand.ts:

// 1. Replicate CouchDB → local PouchDB so the mirror scan has content to work with.
const replResult = await core.services.replication.replicate(true);
// 2. Mirror scan to reconcile PouchDB ↔ local filesystem.
const scanOk = await performFullScan(core, log, errorManager, false, true);

Step 1 materialises documents to storage; each write goes through refreshFile(). By the time step 2 calls collectFilesOnStorage() → storageAccess.getFiles(), the cache holds only the handful of paths touched by replication.

3. Missing entries are read as user deletions — serviceFeatures/offlineScanner:

if (options.mode === FullScanModes.NEWER_WINS && state === "db-only" && doc) {
    const lastSeenMTime = getFileMTimeFromMap(fileMapKey);
    if (lastSeenMTime !== undefined) {
        const recency = compareMTime(lastSeenMTime, doc.mtime);
        if (recency === BASE_IS_NEW || recency === EVEN) {
            action = "delete-db";
        }
    }
}

Every document without a local counterpart falls into state === "db-only", and any of them already known to the mtime map is deleted.

That last condition explains why only a subset was hit: files written since the mtime map was last persisted are unknown to it and were spared. Two files in the same directory were treated differently — Dispositifs/Aide à l'innovation PME.md (older) deleted, Dispositifs/SME Packages.md (newer) kept.

Impact

• Silent data loss. The scan logs 0 failed and completes normally.
• The affected files cannot recover on their own. They are now storage present + db deleted → both-db-deleted. With extraOnLocal/extraOnRemote undefined (as the daemon calls it), shouldDeleteLocalWhenRemoteDeleted() is false and extraOnLocal !== APPEND_STORAGE_ONLY, so the pair resolves to skip on every subsequent scan. Re-writing the same path is refused as db-only-deleted; only a new path propagates.
• It repeats on every restart, and the blast radius grows as the mtime map learns more of the vault.
• Local files are not deleted, so nothing is lost on disk — but the vault silently stops converging.

Suggested fix

Track scan completion explicitly rather than inferring it:

private hasScannedFully = false;

async getFiles(): Promise<NodeFile[]> {
    if (!this.hasScannedFully) {
        await this.scanDirectory();
        this.hasScannedFully = true;
    }
    return Array.from(this.fileCache.values());
}

src/apps/webapp/adapters/FSAPIFileSystemAdapter.ts carries the same pattern.

I have a PR ready with this change plus a regression test.

A second suggestion, separate from the fix

Independently of this particular cache bug, a scan that finds no or very few local files while the database is populated currently proceeds to delete the database. Any future failure of the local enumeration — permissions, a mount not ready, an I/O error — would produce the same outcome through a different path.

A sanity check before mass deletion (abort, or require confirmation, when the local listing is empty or implausibly small relative to the database) would turn a silent data-loss event into a loud refusal. I'd be glad to open that separately if you think it's worth doing.