v4: duplicate "add" events for unchanged files (same inode/birthtime/size) — intermittent under directory activity
Describe the bug
Chokidar 4 occasionally re-emits the add event for a file it has already reported as added, with no actual filesystem change between the two events. The fs.Stats for both events are byte-identical (ino, birthtimeMs, size) — proving the file was never deleted or re-created on disk.
Gaps observed: 6–60 seconds. The race is intermittent and only fires when the watched directory is undergoing other modifications — duplicates don't appear when the directory is idle.
Versions
- Chokidar:
4.0.1(consumingesm/) - Node.js:
22.x - OS: Alpine Linux on Docker, AWS ECS Fargate (overlayfs)
Watcher config
chokidar.watch(mediaRoot, {
ignored: /(^|[/\])\../,
persistent: true,
ignoreInitial: true,
awaitWriteFinish: { stabilityThreshold: 2000, pollInterval: 100 },
}).on('add', (path, stats) => { /* ... */ });Workload
A producer writes a sequentially-numbered series of ~2 MB files into the watched directory at roughly one file per 6 seconds. After ~15 files in the directory, it also unlinks the oldest file each time a new one is created, holding the directory at a roughly steady file count.
Duplicates fire both before the steady-state phase (during the initial add-only ramp-up) and during the add+unlink steady state. The common factor is that the duplicate appears at the moment of some other directory activity, not on an idle directory. Adjacent unrelated writes/unlinks correlate with when the second add fires.
Evidence — two separate sessions, fresh worker processes
Session A (10 minutes, inode range ~534500):
| sequential index | gap | ino (prev = now) | birthtimeMs (prev = now) | size (prev = now) |
|---|---|---|---|---|
| 2 | 5.9s | 534500 | 1780060203959.96 | 2204488 |
| 3 | 30.4s | 534501 | 1780060209959.98 | 2115564 |
| 8 | 29.3s | 534506 | 1780060240380.09 | 2071572 |
| 13 | 23.8s | 534511 | 1780060269640.12 | 2095824 |
| 101 | 17.3s | 534513 | 1780060797800.77 | 2483292 |
Session B (after a clean worker process restart, fresh directory, inode range ~927700s):
| sequential index | gap | ino (prev = now) | birthtimeMs (prev = now) | size (prev = now) |
|---|---|---|---|---|
| 10 | 36.4s | 927723 | 1780063959720.82 | 1898048 |
| 16 | 60.1s | 927729 | 1780063996150.87 | 1656092 |
Same ino, birthtimeMs, and size in every prev/now pair confirms the file was never deleted or re-created. The clean restart between sessions rules out any stale state in the watcher instance — fresh FSWatcher, fresh directory, same bug.
Expected behavior
A file that has not been deleted and re-created should produce exactly one add event per watcher lifetime.
Suspected cause
Reading esm/handler.js, the most likely trigger is the _handleRead stream-end handler that diffs previous against current after each readdirp scan:
stream.once(STR_END, () => {
// ...
previous.getChildren().filter((item) => {
return item !== directory && !current.has(item);
}).forEach((item) => {
this.fsw._remove(directory, item);
});
});If a single readdirp enumeration returns an incomplete listing (which readdir/getdents is allowed to do when the directory is being concurrently modified — POSIX makes no guarantees), the handler will _remove the entries it didn't see. The removal is silent, because _remove short-circuits the unlink emission for paths still in _pendingWrites:
if (this.options.awaitWriteFinish && this._pendingWrites.has(relPath)) {
const event = this._pendingWrites.get(relPath).cancelWait();
if (event === EVENTS.ADD) return;
}The next directory-change event triggers another _handleRead, which now sees the file as not in previous and emits add again. This explains:
- the intermittence (depends on
readdirphappening to miss a file) - the byte-identical inode/birthtime/size signature (no actual file change between the two events)
- the gap variance (gap = time until the next directory-modification event triggers the re-scan)
- the correlation with other directory activity (no activity = no re-scan = no opportunity to re-emit)
Reproduction
I have not been able to isolate a deterministic standalone script repro. I tried a minimal Node 22 + chokidar 4.0.1 script in a Linux container, mirroring the watcher config and a similar workload pattern, at both gentle (1 file/sec, 5 min) and aggressive (4 files/sec + a competing readdir loop, 5 min) churn rates — zero duplicates in either run. The race appears to need filesystem-level timing characteristics that are sensitive to the underlying storage (we hit it on AWS ECS Fargate's overlayfs) and possibly to additional concurrent IO load. Happy to test any candidate patch against our production workload, which reproduces the bug reliably enough to detect duplicates within a few minutes of stream start.
Workaround
Dedup'ing in our add handler by comparing stats.ino + stats.birthtimeMs + stats.size against the previously-emitted add for the same path; skipping the work if all three match.
Source: paulmillr/chokidar