Semantic search never stores anything with most non-OpenAI embedding models unless the operator sets the dimension variable by hand

Version: @agentmemory/agentmemory 0.9.29 · Node v22.23.2 · Linux x64 · EMBEDDING_PROVIDER=openai against a local OpenAI-compatible server

All line numbers are dist/index.mjs of the published 0.9.29 tarball.

Summary

With most embedding models other than OpenAI's own, semantic search never works unless the operator already knows to set one environment variable. Every memory is sent to the embedding server and paid for, then thrown away, and the store fills up with nothing searchable by meaning. Keyword search keeps returning results, so recall looks weak rather than absent, and nothing above a routine per-item warning tells the operator that every single embedding is being rejected. The fix is for agentmemory to ask the server what width it returns instead of guessing.

Mechanism

The width comes from a three-entry table

MODEL_DIMENSIONS (928) holds:

const MODEL_DIMENSIONS = {
	"text-embedding-3-small": 1536,
	"text-embedding-3-large": 3072,
	"text-embedding-ada-002": 1536
};

lookupModelDimensions also retries after a /, so openai/text-embedding-3-small resolves. Anything else does not, and resolveDimensions (939) ends with a fallback that gives every untabled model 1536:

return lookupModelDimensions(model) ?? 1536;

The server is never asked

embedBatch (1007-1015) sends only model and input:

body: JSON.stringify({
	model: this.model,
	input: texts
})

It sends no dimensions and does not read the width off the response. The stored width is whatever the server returns, ep.dimensions is whatever the table guessed, and nothing compares them.

The mismatch throws, and the throw is swallowed

Every provider is wrapped at construction time (createEmbeddingProvider, 1236-1247) in withDimensionGuard (1249), which throws on any width it did not expect:

const check = (v, where) => {
	if (v.length !== expected) throw new Error(`Embedding dimension mismatch in ${provider.name}.${where}: expected ${expected}, got ${v.length}`);
	return v;
};

The throw fires inside ep.embed(...) at 3414, one line before the explicit length test at 3415. The "vector-index add: dimension mismatch — skipping" branch in vectorIndexAddGuarded is therefore unreachable on any provider built by createEmbeddingProvider. Control goes to the catch at 3427 and the operator sees

vector-index add: embed failed — skipping   {provider, error: "Embedding dimension mismatch in <name>.embed: expected 1536, got 4096"}

The batch path (vectorIndexAddBatchGuarded, 3437) behaves the same way. embedBatch is wrapped too (1256-1260), so one bad width aborts the whole batch at 3445-3456 with {ok: 0, fail: items.length} and no per-item detail. rebuildIndex runs on this path. Its own dimension-mismatch branch at 3474 is unreachable through a guarded provider for the same reason. The error text names both numbers, and no code above the catch raises a 100% failure rate past a per-item warning.

The two explicit length tests, 3415 and 3474, are dead code behind the guard. The fix should remove them or say why they stay.

What the operator sees

• The warning describes one observation. No message says that this deployment will never index a vector, and no count of failures exists, so a 100% drop rate and a 0.01% drop rate produce the same log.
• On the rebuild path one mismatched width fails the whole batch (3445-3456) as a single embed failed — skipping batch line. A full re-index of tens of thousands of observations can complete, log Search index rebuilt: N entries, and have added zero vectors, because that count comes from indexRecords and counts BM25 insertions only.
• BM25 still works. Search returns plausible results, so the failure looks like weak recall. Hybrid ranking hides it further.
• No health surface reports it. collectHealth (15650) reports memory, CPU, workers and a KV probe. It does not report vector index size, so 0 vectors beside 22,000 BM25 records is not visible anywhere an operator would look. mem::diagnose's ALL_CATEGORIES (12140) has no index category either.
• The boot check does not run. The persisted-vector-index dimension check at 22857-22866 compares widths, but only when a vector index was loaded from disk. Here there is nothing on disk to check.
• The search path never fails. tripleStreamSearch gates the vector stream on this.vector.size > 0 (2246), so with an empty index the query is never embedded and the guard never fires at query time. The per-item add warning is the only signal this failure produces.
• The source comment on the guard points at the wrong one. MODEL_DIMENSIONS' doc comment (923-926) says "The dimension guard (index.ts) throws on mismatch, so a wrong value here breaks every embed call — keep entries accurate." Two things are called a dimension guard. The one in index.ts (vectorIndexAddGuarded) skips. The one that throws is withDimensionGuard in the provider layer, and the catch at 3427 turns each throw into a routine per-item warning.

Prior art on the tracker

The openai provider case has not been filed. The same shape has, twice, on the OpenRouter provider, and the fix that closed it is the mechanism reported here.

• withDimensionGuard came from #247 (closed): a Gemini 768-wide response corrupted a 1536-wide index, and PR #248 added the guard that throws at the first embed call, "well before the index gets corrupted". The guard now fires on every call and nothing above it counts the failures.
• #809 (open) reports the OpenRouter provider hard-coding 1536 and asks for OPENROUTER_EMBEDDING_DIMENSIONS. #1002 (closed) reports the same provider dropping every 4096-wide qwen3-embedding-8b vector and suggests either a model table "like the OpenAI provider's" or inferring the width from the first embedding response. PR #1136 (merged) closed #1002 by giving both providers the shared MODEL_DIMENSIONS table and resolveDimensions (918-945) with the env override. That code is in 0.9.29, so the OpenRouter provider (1098-1106) has this bug for any model outside the three OpenAI names, and #809 is still open.
• #1369 (open PR) sends OPENAI_EMBEDDING_DIMENSIONS to the server as dimensions when it is set, retries without it on a 400 or 422 that mentions "dimension", and truncates and renormalises client-side when the server ignores it. It still requires the operator to know the width in advance and does nothing when the variable is unset, so it does not change the default path reported here. It does interact with the fix below: with #1369 in place the startup probe should also send the override as dimensions, so the assertion then catches only a server that ignored or rejected it.
• #256 (closed, restore path bypasses the guard) and #456 (closed, AGENTMEMORY_DROP_STALE_INDEX has no effect) are the other two filings against this guard.

Reproduction

Point EMBEDDING_PROVIDER=openai at any OpenAI-compatible server whose model is not in the table and whose width is not 1536. A 4096-wide model is the case measured here:

OPENAI_BASE_URL=http://<host>/v1
OPENAI_MODEL=<any-name-not-in-the-table>
OPENAI_EMBEDDING_MODEL=<a-4096-wide-model>
# OPENAI_EMBEDDING_DIMENSIONS deliberately unset

Index any observations, then search. Expected: vector hits. Actual: vectorIndex.size stays 0, a vector-index add: embed failed — skipping line per observation carrying Embedding dimension mismatch in openai.embed: expected 1536, got 4096, and search answers from BM25 alone. On a rebuild the same condition surfaces once per batch as vector-index add batch: embed failed — skipping batch.

Severity

The exposed population is every operator using a model outside the three-entry table who has not set the override. It is larger than the population exposed to the MAX_STRING_LENGTH ceiling in #1372, since setting the override moves a user out of this bug and into that one.

Fix

The width has to come from the server before anything reads it. The change is one probe in the boot sequence plus the removal of what it makes dead.

1. Probe once at startup. After createEmbeddingProvider() (22718) and before indexPersistence.load() (22848), issue one embedBatch on a fixed string and set dimensions from the response length. dimensions is already a mutable field (990, 1000) and embed already delegates to embedBatch (1004-1006), so this is the only place it needs to be set. The probe has to run before the persisted-index check at 22857, which reads embeddingProvider.dimensions. Otherwise a correct 4096 index on disk fails that check against the guessed 1536 and is refused (22866) or, with AGENTMEMORY_DROP_STALE_INDEX=true, discarded (22865).
2. Delete MODEL_DIMENSIONS (928) and the ?? 1536 in resolveDimensions (939). Both are dead once the width is probed.
3. Treat OPENAI_EMBEDDING_DIMENSIONS as an assertion. If it is set and the probe disagrees, refuse to start and print both numbers, in the style of the existing 22866 message. A mismatch can then only mean a misconfiguration, and it is caught once at boot.
4. Retry a failed probe. Leave dimensions unset, log the failure once, retry with backoff, and skip vector adds until the probe succeeds. Those adds already behave this way when the server is down (embed failed — skipping, 3427). When the probe succeeds, run the 22857-22866 validation at that point against the loaded index, so a stale persisted index is still caught under the existing policy. Left in place it returns cosineSimilarity 0 (1574-1575) on every query.
5. Remove the two dead checks at 3415 and 3474. After this change withDimensionGuard (1249) can only fire if the server changes width mid-run, which is a real error, and throwing is the right response.

Related

#1372, the MAX_STRING_LENGTH ceiling. Setting OPENAI_EMBEDDING_DIMENSIONS to escape this bug lets the index fill and reach that one. With this fix in place, the attempt marker in that issue's rebuild gate is still needed, since a probe that never succeeds leaves the same empty vector index behind.

The health point above is the gap named in the comment on #1223: collectHealth reports a wrong number for memory and no number for the index.