HTTP MCP daemon disposes active LLM contexts after five minutes, causing DisposedError and SIGSEGV

Summary

The persistent HTTP MCP server can run idle cleanup while a model-backed request is still active. This disposes native Llama contexts underneath in-flight embedding or reranking work.

The JS-visible result is DisposedError: Object is disposed. If the race reaches native code, the process crashes with EXC_BAD_ACCESS in llama_pooling_type while another thread runs llama_free.

Environment

• QMD 2.8.3 (facd35e)
• node-llama-cpp 3.20.0
• macOS arm64
• Metal backend
• Server: qmd mcp --http --port 8181
• Server supervised by launchd

Observed behavior

A structured lex + vec request using default reranking took 353,510 ms.

The following requests produced:

Reranker unavailable — skipping reranking (Object is disposed). Use --no-rerank to silence this warning.

HTTP handler error: DisposedError: Object is disposed
    at DisposeGuard.createPreventDisposalHandle
    at LlamaContext.js:446

The process then exited with SIGSEGV. The supervisor restarted it.

Equivalent one-shot CLI queries complete successfully.

Crash evidence

The macOS crash report records:

EXC_BAD_ACCESS
SIGSEGV
KERN_INVALID_ADDRESS at 0x0000000000000084

The main thread was resolving an embedding:

libllama.metal.b10361.dylib  llama_pooling_type
llama-addon.node             AddonContext::GetEmbedding

A libuv worker thread was simultaneously destroying the context:

llama_context::~llama_context
llama_free

This indicates a native context use-after-free rather than an out-of-memory or Metal initialization failure.

Root cause

Package-relative line numbers below refer to QMD 2.8.3.

The HTTP server creates a per-store LlamaCpp with a five-minute timeout and model disposal enabled:

// dist/index.js:82-88
const llm = new LlamaCpp({
  inactivityTimeoutMs: 5 * 60 * 1000,
  disposeModelsOnInactivity: true,
});

The idle timer calls canUnloadLLM() before disposal:

// dist/llm.js:535-547
if (typeof canUnloadLLM === "function" && !canUnloadLLM()) {
  this.touchActivity();
  return;
}

this.unloadIdleResources();

However, canUnloadLLM() only checks the module-level defaultSessionManager:

// dist/llm.js:1641-1645
export function canUnloadLLM() {
  if (!defaultSessionManager) return true;
  return defaultSessionManager.canUnload();
}

The HTTP server uses a per-store LLM and never initializes that default session manager. Therefore, canUnloadLLM() returns true even while the store's LLM is active.

Structured search also invokes the per-store LLM without a session that the timer can observe:

// dist/store.js:4730
await llm.embedBatch(textsToEmbed);

// dist/store.js:4854
await store.rerank(primaryQuery, chunksToRerank, undefined, intent);

After five minutes, unloadIdleResources() can dispose embedding contexts, reranking contexts, and models while those operations remain in flight.

The CLI path does not reproduce this because it registers the default LLM and wraps model-backed searches in withLLMSession().

Expected behavior

Idle cleanup must never dispose a context or model while an operation on the same LlamaCpp instance is active.

The HTTP server should either:

1. Track operations with an instance-local session manager or reference count.
2. Acquire the same lock for operations and idle disposal.
3. Wrap each request in a session bound to the store's LLM instance.

Periodic touchActivity() calls during reranking could reduce exposure, but they would not replace proper synchronization.

Why existing related fixes do not cover this

#40 / PR #41: activity pings during batch embedding

#40 identified an inactivity timer firing during a slow embedding batch. The merged fix, PR #41, calls touchActivity() after each successful embedding.

That reduces the chance of cleanup during a batch, but it does not establish ownership or synchronization:

• Cleanup can begin before the next activity ping.
• A single native operation can exceed the timeout.
• The timer does not wait for an active operation.
• The fix does not protect reranking.
• It does not connect the per-store LLM to canUnloadLLM().

PR #57 proposed broader disposal changes, including reranking pings and reduced automatic disposal, but it was not merged.

#935 reproduces DisposedError during embedding on QMD 2.8.3, confirming that PR #41 did not close the underlying lifetime race.

#682 / PR #874: concurrent context creation

#682 concerned two cold-start callers entering ensureRerankContexts() simultaneously. PR #874 fixed that race by adding rerankContextsCreatePromise, so concurrent callers share one context-creation operation.

This issue involves a different transition:

#682: create context ↔ create context
this issue: use context ↔ destroy context

PR #874 does not:

• Synchronize unloadIdleResources() with active operations.
• Change canUnloadLLM() or its global session manager.
• Register per-store LLM activity.
• Protect embedding contexts.
• Prevent cleanup after context creation completes.

The native crash occurred in AddonContext::GetEmbedding while another thread ran llama_free. A reranking context-creation mutex cannot protect that path.

#935: current embedding reproduction

#935 remains open and has no merged fix. It shows that the same disposal class still affects CPU-only batch embedding in 2.8.3.

A narrow fix that adds more touchActivity() calls could reduce #935's frequency without fixing this issue. The invariant must be that disposal cannot start while any operation uses the same LLM instance.

#938: current MCP crash reproduction

#938 remains open and has no merged fix. It reports the same MCP-versus-CLI behavior, but it does not yet identify or repair the mismatch between the per-store LLM and the global session manager.

A transport-specific fix, error catch, or launcher change would not prevent the native use-after-free. A fix for #938 covers this issue only if it synchronizes idle disposal with all active per-store embedding and reranking operations.

Required fix coverage

A complete fix must satisfy all of these conditions:

1. Track active work on each LlamaCpp instance rather than through a module-global manager.
2. Make unloadIdleResources() wait for that instance's active operations.
3. Block new operations while disposal is in progress.
4. Cover both embedding and reranking.
5. Test a per-store HTTP LLM with a shortened idle timeout and an operation held open across that timeout.

Activity pings and context-creation mutexes remain useful safeguards, but neither can enforce the required lifetime invariant.

Related issues and pull requests

• #40
• #41
• #57
• #682
• #874
• #935
• #938