Are orphaned segment files recoverable after writer process crash?
We're running Tantivy as the index writer in a long-running service where writer processes occasionally get SIGKILL'd (OOM). After a crash, the index directory has segment component files on disk, but meta.json still shows "segments": [], "opstamp": 0 — the writer was instantiated and accepted documents, but commit() was never called (or never completed).
We sampled ~800 such directories from a workload. The on-disk state is consistent:
$ ls -la /data/indices/.../2026-09-08_20-47/
total 16
-rw------- 258 .managed.json
-rw-r--r-- 0 .tantivy-writer.lock
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.fast
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.fieldnorm
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.idx
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.pos
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.store
-rw-r--r-- 0 7a8c2a81201346f686c5126823dc31d3.term
-rw------- 4014 meta.json// meta.json (truncated)
{
"segments": [],
"opstamp": 0,
"schema": [ ... ]
}Of ~800 directories, ~799 have all six component files at 0 bytes (writer created, no data flushed to disk). One had a 123-byte .store (partial docstore spill, no other component files populated):
-rw-rw-r-- 0 43a2e3659e12...fast
-rw-rw-r-- 0 43a2e3659e12...fieldnorm
-rw-rw-r-- 0 43a2e3659e12...idx
-rw-rw-r-- 0 43a2e3659e12...pos
-rw-rw-r-- 123 43a2e3659e12...store ← partial spill
-rw-rw-r-- 0 43a2e3659e12...termStill segments: [], opstamp: 0.
Three questions:
Is this the expected on-disk state after a writer crash before
commit()? My read:Index::create_in_dir()writes the initialmeta.jsonand.managed.json;index.writer()acquires the lock and allocates a segment UUID with placeholder files; documents are buffered in memory untilcommit()callsprepare_commit()→finalize()which serializes the component files with footers. If the process dies beforecommit(), the component files stay at 0 bytes (or partially spilled) andmeta.jsonis never updated. Is that correct?On
garbage_collect_files: reading the code, it looks like GC runs insideprepare_commit()and diffs files on disk against files referenced by segments in theSegmentManager(loaded frommeta.json). Withsegments: [], the living set is empty, so GC would delete the orphaned component files on the next commit. But in our case, nothing ever re-opens these directories to run another commit — the process restarts with a new writer and new directory. Is there a recommended way to trigger GC against an existing directory, or is the intended cleanup justrm -rfof the whole directory?Is there any path to recovering data from the partially-spilled case (123-byte
.store, everything else 0)? My assumption is no — withoutcommit()having run, the component files have no footer, the segment isn't registered, and the in-memory state that would have completed the write is gone with the process. Just want to confirm there's no truncation-tolerant reader path I'm missing.
We're planning to sweep these directories at startup. Just want to confirm our understanding before doing so.
Source: quickwit-oss/tantivy