Ingest fails silently: persisted queue is not resumed after restart, and the source watcher stops firing
Environment
- LLM Wiki
0.6.11(Windows x64 portable), Windows 11 - Model
deepseek-v4-flash;sourceWatchConfig.autoIngest: true; defaultingestConcurrency - Project with ~680 wiki pages and ~66 files in
raw/sources/ - Observed across one long session with ~65 sequential ingests
First: this app does something genuinely useful, thanks for building it. I'm reporting two issues because they both fail silently — no error, no progress indicator — which made a whole evening of work much harder than it needed to be.
1. The persisted ingest queue is not resumed after restarting the app
Steps
- Get
ingest-queue.jsoninto a state with several"status": "pending"tasks. - Quit LLM Wiki completely, then start it again.
- The queue file is unchanged and still lists the pending tasks.
Expected: queued work resumes (or the UI says it was dropped).
Actual: nothing is ever processed. The app sits idle indefinitely. There is no error, no warning, and no indication that the queue is dead.
Workaround that worked: for each file, delete it from raw/sources/ and copy it back. That fires the watcher and the task gets picked up.
Suggestion: resume pending tasks on startup, or make it explicit in the UI that a restart drops the queue (plus a "retry queued" action).
2. The source watcher stops detecting changes after a while
Steps
- Ingest a few dozen files over a session.
- Copy a brand-new file (never ingested before) into
raw/sources/.
Expected: it gets queued.
Actual: nothing happens. Deleting the file and copying it back does not queue it either. There is no error and no indication anything is wrong.
Only fix found: stop and restart LLM Wiki.exe. Afterwards the watcher works again until it stops again.
Related observation (may be intended, but worth documenting): updating a file's timestamp without changing its content never triggers ingest. Only delete + re-copy does. I lost time assuming touch was a valid way to re-queue a file — a line in the docs would save others the same detour.
Suggestion: a watcher-alive indicator, and/or automatic recovery when the watcher stops firing.
Why these two matter together
Neither one produces an error message. From the outside, "the app is idle because it finished" and "the app is idle because the pipeline is dead" look identical. The only way I found to tell them apart was to compare ingest-cache.json against the files I had fed — which shouldn't be necessary.
Happy to share queue/cache state or a step-by-step repro. Thanks again.
Source: nashsu/llm_wiki