syncIndexedCodebasesFromCloud() force-completes an actively-indexing codebase using partial row count

Author: murad-mocartCreated Aug 10, 2026Updated Aug 10, 2026

Problem

syncIndexedCodebasesFromCloud() (packages/mcp/src/handlers.ts) is called at the top of handleIndexCodebase, handleSearchCode, and (since #252 was fixed) handleGetIndexingStatus. Its "recovery" branch treats any collection found in Zilliz Cloud/Milvus that isn't in the local indexed snapshot list as something to adopt:

javascript
for (const cloudCodebase of cloudCodebases) {
    if (!localCodebases.has(cloudCodebase)) {
        const stats = await this.queryCollectionStats(cloudCodebase);
        if (stats) {
            this.snapshotManager.setCodebaseIndexed(cloudCodebase, {
                ...stats,
                status: 'completed'   // <-- hardcoded regardless of real state
            });
            ...

localCodebases is getIndexedCodebases() — codebases whose status is indexed. A codebase that is genuinely, actively indexing right now is not in that set (it's tracked separately as "indexing"), but it does already have partial rows in Milvus, since chunks are flushed to Milvus in EMBEDDING_BATCH_SIZE-sized batches (default 100) as indexing proceeds.

So: call index_codebase on a large codebase, then call get_indexing_status (or index_codebase again) while it's genuinely still running — the sync-from-cloud recovery logic sees "cloud has some rows, not in the indexed set" and immediately force-marks it status: 'completed' using whatever partial row count exists at that instant, silently truncating the real indexing run.

Impact

  • Indexing a codebase of thousands of files reports "✅ fully indexed" after only a few seconds, having actually processed only the first batch or two (always a clean multiple of EMBEDDING_BATCH_SIZE).
  • The truncation point is nondeterministic — it depends on exactly when the user happens to call get_indexing_status relative to the background task's batch-flush cadence. We saw 100, 200, and 400 files reported "complete" across repeated attempts on the same ~2,600-file codebase (real count verified independently via Context.getCodeFiles() with the actual merged ignore patterns).
  • This is silent — no error, no indexfailed status, just a confidently wrong "completed" result. Users have no signal that their index is incomplete short of manually verifying file counts.

Repro

  1. Point index_codebase at a codebase with >200 matching files (so it spans more than one EMBEDDING_BATCH_SIZE batch).
  2. Immediately call get_indexing_status on the same path a few seconds later, while it's still genuinely indexing.
  3. Observe: status flips to "✅ fully indexed... Status: completed" with a file/chunk count that's a clean multiple of 100, far short of the real file count.
  4. Compare against the real count via new Context({vectorDatabase}).getEffectiveIgnorePatterns(path, []) + getCodeFiles(...).

Suggested fix

Skip the recovery branch for any codebase whose local status is currently indexing:

javascript
for (const cloudCodebase of cloudCodebases) {
    if (this.snapshotManager.getCodebaseStatus(cloudCodebase) === 'indexing') {
        continue; // genuinely in progress — partial rows are expected, not a completed index
    }
    if (!localCodebases.has(cloudCodebase)) {
        ...

Happy to open a PR with this change if useful — verified locally that it stops the premature-completion behavior.

Related

Possibly introduced/exposed by the fix for #252 (which added the syncIndexedCodebasesFromCloud() call to handleGetIndexingStatus for consistency with the other handlers) — the call itself is correct, but the recovery logic underneath wasn't handling the "genuinely in-progress" case.

Source: zilliztech/claude-context