IndexIDMap2::reset leaves stale reverse-map entries
Environment
- Faiss:
faiss-cpu==1.15.0 - Build: CPU
- Interface: in-process Python API
- Index:
IndexIDMap2(IndexFlatL2)
Summary
IndexIDMap2.reset() clears the underlying index and id_map, but leaves stale entries in rev_map.
After resetting an index and adding a new vector with a different external ID, reconstructing an old removed ID incorrectly returns the newly added vector. check_consistency() then reports that rev_map and id_map have different sizes.
Minimal reproduction
import faiss
import numpy as np
index = faiss.IndexIDMap2(faiss.IndexFlatL2(2))
index.add_with_ids(
np.array(
[[1.0, 2.0], [3.0, 4.0]],
dtype="float32",
),
np.array([10, 20], dtype="int64"),
)
index.reset()
index.add_with_ids(
np.array([[9.0, 9.0]], dtype="float32"),
np.array([30], dtype="int64"),
)
print("ntotal:", index.ntotal)
print("reconstruct(10):", index.reconstruct(10))
index.check_consistency()Expected behavior
After reset(), all previous external IDs must be invalid.
Therefore:
index.reconstruct(10)should raise a RuntimeError, because ID 10 was removed.
The active ID should remain valid:
index.reconstruct(30)
# [9.0, 9.0]index.check_consistency() should complete successfully.
Actual behavior
ntotal: 1
reconstruct(10): [9.0, 9.0]reconstruct(10) incorrectly resolves the removed ID to the vector currently stored at internal position 0.
Then:
RuntimeError:
'rev_map.size() == this->id_map.size()' failedThe active ID 30 and ordinary flat search remain correct.
Calling the public repair method:
index.construct_rev_map()removes the stale entries. After that:
reconstruct(10)raisesRuntimeError;reconstruct(30)returns[9.0, 9.0];check_consistency()passes.
Root cause
The inherited IndexIDMapTemplate::reset() clears:
index->reset();
id_map.clear();
this->ntotal = 0;but does not clear the IndexIDMap2Template::rev_map.
When a new vector is added, add_with_ids() appends the new mapping:
rev_map[this->id_map[i]] = i;The old entries for IDs 10 and 20 remain in the map, while the new ID 30 is added at internal position 0.
IndexIDMap2::reconstruct() trusts rev_map, so both IDs 10 and 30 resolve to the same current vector.
check_consistency() detects the stale state because:
rev_map.size() != id_map.size()Relevant source:
[IndexIDMap.cpp](https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIDMap.cpp)[IndexIDMap.h](https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIDMap.h)[Index.h](https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/Index.h)
Impact
Applications may reconstruct deleted external IDs as unrelated new vectors. This can corrupt:
- ID-to-vector lookups;
- metadata joins;
- deletion and replacement workflows;
- consistency checks;
- lifecycle bookkeeping.
The failure is silent until check_consistency() is called.
Suggested fix
Override reset() in IndexIDMap2Template and clear the reverse map after resetting the base class:
void reset() override {
IndexIDMapTemplate<IndexT>::reset();
rev_map.clear();
}A regression test should verify that:
- old external IDs cannot be reconstructed after
reset(); - newly added IDs reconstruct correctly;
rev_mapandid_maphave matching contents;check_consistency()passes after reset and re-add;construct_rev_map()remains a valid repair operation.
Duplicate search
A bounded search of Faiss issues and pull requests using IndexIDMap2, rev_map, reset, construct_rev_map, and check_consistency found no exact report.
The closest result is [PR #3369](https://github.com/facebookresearch/faiss/pull/3369), which concerns deletion and reverse-map reconstruction optimization, but does not report stale reverse-map entries after reset().
Source: facebookresearch/faiss