OpenSearch vector storage does not use the model-isolation suffix, so switching embedding models wedges the index

Summary

OpenSearchVectorDBStorage is the only vector backend that does not use BaseVectorStorage._generate_collection_suffix(). Its index name is _sanitize_index_name(f"{workspace}_{namespace}") with nothing identifying the embedding model, so switching models reuses the same index. The dimension guard then refuses to start against it, and the operator's only way forward is to drop the index — losing the old corpus and any way back.

Milvus, Qdrant and PostgreSQL all append {safe_model_name}_{embedding_dim}d and get a fresh, isolated container per model.

Evidence this is an omission, not a decision

• The mechanism landed in the base class on 2025-11-19 — 13f2440bb "feat: enhance BaseVectorStorage for model isolation".
• OpenSearchVectorDBStorage was added on 2026-03-02 — b57d88d51 "Add OpenSearch as unified storage backend", over three months later.

The suffix helper was already available, already used by three backends, and simply was not wired up.

What adopting it involves

Not just a name change — the other three carry migration machinery with it, and OpenSearch's situation is harder in one specific way:

• Every existing OpenSearch index is legacy. For Milvus/Qdrant/PostgreSQL, "legacy" means a container created before 2025-11-19; here it means all of them. So adoption is a migration for every existing OpenSearch deployment, not a tail case. Either the data is carried over, or users restart against an empty index.

• Legacy detection and one-time migration, mirroring _find_legacy_collection / setup_collection (Qdrant) or _migrate_workspace_data (PostgreSQL).

• drop() must clear the legacy index's workspace data too, or the next startup re-migrates just-cleared data back — the resurrection Qdrant's drop() handles explicitly.

• Interaction with _claim_index_for_workspace. The legacy index must be proven to belong to this workspace before its rows are copied — index names are lossy (_sanitize_index_name, and OPENSEARCH_WORKSPACE collapsing distinct workspaces), so the un-suffixed name can belong to another deployment. A mismatch must skip the migration and warn, never raise: refusing to start is the wedge this issue removes.

  Correction. This originally said OpenSearch needs an equivalent of Qdrant's _legacy_collection_has_workspace_field three-state detection. It does not. That detection exists because Qdrant partitions one collection across workspaces via a payload field, which makes "is this legacy row mine?" genuinely ambiguous. OpenSearch indices are already per-workspace ({workspace}_{namespace}), so a legacy index belongs wholly to one workspace and the _meta ownership marker settles it outright.

Worth doing at the same time: record the model name and embedding_dim in the index _meta. It does not prevent collisions (an index whose model is unknown is exactly the case where the suffix is None too), but it gives future migrations a provenance to reason about instead of inferring from dimension alone.

The dimension guard is still required afterwards

_generate_collection_suffix() returns None when the embedding function declares no model_name (it is optional on EmbeddingFunc), and all three adopting backends then fall back to the un-suffixed name. On that fallback a model switch collides exactly as it does today, so the dimension check remains the only thing standing between a switch and an unusable index.

PR #3954 added that check to the read path's readiness probe (1004d0d71) for this reason. It is a backstop and survives this work; it is not a substitute for it, and this issue is not a substitute for it either.

Where should this land?

Given the migration surface, this may warrant a long-lived branch and discussion before implementation rather than a direct PR against main.

Generated with Claude Code