LlamaIndex re-index can leak stale embedding progress after a timeout

Summary

On Windows, a LlamaIndex knowledge-base re-index can time out while its synchronous embedding worker continues running in the background. The worker can then emit progress into a later task through shared global embedding settings, causing misleading repeated progress and leaving no queryable index.

Environment

• DeepTutor v1.6.7 source checkout
• Windows 11
• RAG provider: llamaindex
• Embeddings: local Ollama, qwen3-embedding:0.6b (1024 dimensions)
• Re-index input: two PDFs, including one large textbook

Reproduction

1. Create a LlamaIndex knowledge base with one or more large PDFs.
2. Configure a local Ollama embedding endpoint.
3. Start Re-index and leave the page open.
4. If an embedding call stops completing, wait for the indexing stall timeout, then observe the task/progress stream or retry the operation.

Actual behavior

• The UI may repeatedly show an entire batch sequence, e.g. Embedding batches: 1/103 through 103/103, then restart from 1/103.
• The process-log count continually increases even though it is not a count of independent workers.
• A stalled task can eventually report a timeout, but the underlying executor worker keeps running because synchronous Python code cannot be cancelled.
• Progress is attached through the process-global Settings.embed_model callback, so a stale worker can write progress intended for a newer task.
• The KB can remain in a processing-looking state even after reaching 103/103, without a persisted docstore.json / queryable index.

Expected behavior

• One re-index task should own one progress stream.
• Once a task is marked failed/cancelled, its worker must no longer update user-visible progress.
• A retry should not overlap a still-live worker or receive its callbacks.
• The UI should not append duplicate process/progress lines indefinitely.
• A KB should only be reported ready after the complete persisted index is present and loadable.

Likely direction

A robust fix appears to be to create a per-index-operation embedding adapter/callback rather than mutating the global Settings.embed_model callback, detach that callback in finally, and prevent a new LlamaIndex indexing job from starting while a timed-out worker is still alive. The progress UI should de-duplicate interleaved raw process-log and structured progress events.

This is particularly visible on long local Ollama embedding jobs, where a retry can otherwise look like many parallel processes while producing no usable index.