[Bug]: Creating an empty knowledge base and then uploading documents always fails with "Knowledge base not initialized (llamaindex)"

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• I believe this is a legitimate bug, not just a question or feature request.

Related: #1458 reports the identical ValueError and traceback, but attributes it to Unraid host file permissions. This report reproduces the same failure on plain Windows 11 + Docker Desktop where the container demonstrably does have write access to the KB directory, and traces it to a different cause. I believe #1458 is the same defect with a misidentified root cause — see "Additional Information" for the falsifying evidence. (As of writing, #1458 has no comments.)

Describe the bug

A knowledge base created without documents is marked ready and reported as successfully indexed, but no index is ever written to disk. Any later document upload to that KB then fails immediately in DocumentAdder.__init__ with:

ValueError: Knowledge base not initialized (llamaindex): <KB name>

The failure happens before parsing, chunking, or any embedding call — it is a pure precondition check on directory contents. Re-uploading fails identically; the KB list shows ready and gives no indication that index initialization was skipped. (The KB page's Re-index button does recover it — see "Workaround" — but nothing on the upload path itself does.)

This breaks the otherwise perfectly normal workflow "create an empty KB → upload files later". I reproduced it on LlamaIndex; from reading the code the bootstrap exemption is LightRAG-only (add_documents.py:193), so the other engines look similarly affected, but I have not tested them.

Steps to reproduce

1. Deploy DeepTutor via Docker (single container, official image), configure an embedding model in Settings → Catalog, and verify it with the embedding diagnostic (mine passes).
2. Knowledge Center → Create New Knowledge Base. Name it, choose LlamaIndex as the engine, and leave the file picker empty.
3. The UI reports success: "Knowledge base created (no documents yet)." The KB shows ready.
4. Upload any supported file (README.md in my case) to that KB.
5. The upload fails with the ValueError above.

Expected Behavior

Uploading documents into an existing, ready knowledge base should index them — either directly, or by first initializing the index that the empty-KB creation path never created.

If the KB genuinely cannot accept uploads yet, the UI should say so before accepting the files, and the KB should not be reported as ready / indexed.

Related Module

Knowledge Base Management

Configuration Used

• RAG engine: LlamaIndex (built-in default)
• Embedding: WeMM-Embedding-2B, 2048-dim, binding: custom, base_url: http://host.docker.internal:8100/v1/embeddings (self-hosted, OpenAI-compatible, on the Docker host; verified reachable from inside the container)
• LLM / task model: DeepSeek deepseek-flash
• Retrieval profile: hybrid (default), chunk_size 512, chunk_overlap 50
• Uploaded document: a single README.md (~10 KB, plain text)

Logs and screenshots

The original traceback, from the backend log:

File "/app/deeptutor/api/routers/knowledge.py", line 1063, in run_upload_processing_task
    adder = DocumentAdder(
            ^^^^^^^^^^^^^^
File "/app/deeptutor/knowledge/add_documents.py", line 197, in __init__
    raise ValueError(f"Knowledge base not initialized ({self.rag_provider}): {kb_name}")
ValueError: Knowledge base not initialized (llamaindex): 知识库

On-disk state of the KB after the failed upload — note the absence of any version-N/ directory (the flat layout where LlamaIndex storage lives in 1.6.x):

data/knowledge_bases/<KB>/
├── .progress.json
├── metadata.json          # no `file_hashes` key — nothing was ever indexed
└── raw/
    └── README.md

Probe output using DeepTutor's own helpers, run inside the container:

from deeptutor.services.rag.index_probe import inspect_kb_versions, has_ready_provider_index, provider_failure_summary
from deeptutor.services.rag.index_versioning import list_kb_versions, find_matching_version
from deeptutor.services.rag.embedding_signature import signature_from_embedding_config

kb = Path('/app/data/knowledge_bases/<KB>')
print('children             :', sorted(p.name for p in kb.iterdir()))
print('list_kb_versions     :', list_kb_versions(kb))
print('inspect_kb_versions  :', inspect_kb_versions(kb, 'llamaindex'))
print('has_ready_index      :', has_ready_provider_index(kb, 'llamaindex'))
print('failure_summary      :', repr(provider_failure_summary(kb, 'llamaindex')))
print('active signature     :', signature_from_embedding_config().hash())
print('find_matching_version:', find_matching_version(kb, signature_from_embedding_config()))

children             : ['.progress.json', 'metadata.json', 'raw']
list_kb_versions     : []
inspect_kb_versions  : []
has_ready_index      : False
failure_summary      : ''              # <-- empty; see note 1 below
active signature     : 3c3ef135a5c2b269
find_matching_version: None

Additional Information

• DeepTutor Version: v1.6.8 (image ghcr.io/hkuds/deeptutor:latest, digest sha256:1fe16f48e090da11c1c16e3e9c4b08b12801fa9f153249d36dc9d50ee60d0ede, built 2026-09-14)
• Operating System: Windows 11 Pro (10.0.22621), Docker Desktop
• Python Version: 3.11 (inside the container)
• Node.js Version: (as shipped in the image)
• Browser (if applicable): —
• Related Issues: #1458

Root cause

Two code paths have mutually incompatible preconditions.

1. Creating an empty KB never builds an index. In deeptutor/api/routers/knowledge.py, create_knowledge_base has an "empty KB" fast path (v1.6.8: lines 3095–3115) that only writes progress state and flips the status to ready:

# Fast path: no files uploaded — create an empty KB ready for web
# sources, GitHub sources, or later document uploads.
if not files:
    progress_tracker.update(ProgressStage.COMPLETED, "Knowledge base created (no documents yet).", ...)
    manager.update_kb_status(name=name, status="ready", progress={
        ...
        "index_changed": True,
        "index_action": "create",
    })
    return {...}

It never calls process_documents() / RAGService.initialize(), and never writes a version-N/ directory. But it records index_changed: True and index_action: "create", so the KB is presented to the user — and to kb_config.json — as already indexed:

"status": "ready",
"index_versions": [],
"last_indexed_action": "create",
"last_completed_at": "2026-09-15T14:48:13.346796"

status: ready with index_versions: [] is the tell.

2. The upload path treats an existing index as a hard precondition. In deeptutor/knowledge/add_documents.py, DocumentAdder.__init__:

has_provider_index = has_ready_provider_index(self.kb_dir, self.rag_provider)          # line 182
...
allows_lightrag_bootstrap = self.rag_provider == LIGHTRAG_PROVIDER and not list_kb_versions(  # line 193
    self.kb_dir
)
if not has_provider_index and not allows_lightrag_bootstrap:                            # line 196
    raise ValueError(f"Knowledge base not initialized ({self.rag_provider}): {kb_name}")  # line 197

The bootstrap exemption exists only for LightRAG. LlamaIndex gets no equivalent, so a brand-new LlamaIndex KB with no index cannot accept an incremental upload. (Other engines also write a synthetic meta.json during their own initialization — graphrag/storage.py:66, pageindex/storage.py:66 — but since the empty-KB fast path returns before any initialization runs, I would expect them to be affected too; I only reproduced and verified this on LlamaIndex.)

The upload endpoint (POST /knowledge-bases/{kb_name}/upload) schedules run_upload_processing_task unconditionally — it performs no "initialize the index first" step.

The frontend has a matching guard string, "This knowledge base is in legacy index format and needs reindex before upload.", but it only fires for the legacy rag_storage/ layout. There is no guard for "this KB has no index at all", so the upload is accepted and then fails server-side.

Evidence that this is not a permissions problem (contra #1458)

• The container runs the backend as deeptutor (supervisord user=deeptutor), and the KB directory is owned by deeptutor:deeptutor.
• The app successfully wrote raw/README.md and .progress.json into that very directory — the upload stages files fine. Write access is demonstrably present.
• Timestamps show the ordering that triggers the bug: metadata.json at 14:48:12, then raw/README.md at 14:48:21 — the KB was created empty, and the file arrived 9 seconds later. No Initialization failed line appears in the log, i.e. no initialization task ever ran.
• Independently ruled out: the embedding endpoint (called end-to-end from inside the container via get_embedding_client, returning 2048-dim L2-normalized vectors, single and batched), the embedding signature (3c3ef135a5c2b269 matches the active config exactly — no config drift), and the LLM (deepseek-flash returns 200).

Workaround

POST /api/knowledge-bases/<KB name>/reindex (the KB page's Re-index button) recovers the KB. Because the KB's status is error, the endpoint sets force_reindex = True and does not short-circuit; run_reindex_task reads the files already staged in raw/ and builds the missing version-1/. Verified: status → ready, index_versions gains one ready: true entry with doc_count: 9, has_ready_provider_index() → True, and a live RAGService.search() returns 5 relevant sources.

This is a recovery path, not a fix — every newly created empty KB will hit the same wall.

Two secondary issues found while investigating

1. The diagnostic is empty, which is why the error is so opaque. When no version entry exists at all, _inspect_llamaindex never reaches its "LlamaIndex storage directory does not exist." branch (that branch is only reached via an existing version entry's storage_path), so provider_failure_summary() returns ''. The user sees only Knowledge base not initialized (llamaindex): <name> with no hint that the real cause is "no index was ever created". Surfacing list_kb_versions(kb_dir) == [] as an explicit diagnostic would close this gap.

2. Re-index does not record file_hashes, so the obvious recovery invites duplicate content. metadata.json's file_hashes is written by DocumentAdder — the very class that is skipped in this failure mode. After recovering via re-index, re-uploading the same file to "verify the fix" is not detected as a duplicate; it is staged under a non-colliding name (README (1).md) and indexed again. Worth handling whichever way the main fix goes.

Suggested direction (maintainers' call)

Either make the empty-KB path produce a real (empty) index so has_ready_provider_index() is truthful, or give the upload path the same first-index bootstrap that LightRAG already has — and in the meantime stop recording index_action: "create" / status: ready for a KB that has no index.

I have not verified whether any of this is intentional design; the analysis above is read off the v1.6.8 sources in the published image. I did check main (ref 897fce5) and the empty-KB fast path and DocumentAdder precondition are unchanged there, and deeptutor/knowledge/add_documents.py was last touched 2026-09-03 (#1171, LightRAG-related), so this does not appear to be fixed on main.

Happy to test a patch or provide more instrumentation output.