IndexIVF helper APIs ignore quantizer_params and return inconsistent results
Summary
In FAISS v1.15.0, IndexIVF::search forwards
SearchParametersIVF.quantizer_params to the coarse quantizer.
However, search_and_reconstruct and search_and_return_codes call the
coarse quantizer without forwarding quantizer_params.
When an IVF index uses an HNSW coarse quantizer, these APIs can therefore
return different results even when they receive the same
SearchParametersIVF object.
Environment
- FAISS: v1.15.0
- CPU
- IVF index with an HNSW coarse quantizer
- Metric: L2
nprobe=1- One FAISS/OpenMP thread
- Nested parameter:
SearchParametersHNSW(efSearch=1)
Reproduction
Run the deterministic reproduction script:
python repro.pyThe script uses fixed float32 arithmetic-grid inputs, does not generate
random data, and passes the same SearchParametersIVF object to:
searchsearch_and_reconstructsearch_and_return_codes
The same input vectors, index, thread configuration, and search parameters are used for all three calls.
Actual behavior
The result is stable across 10 fresh processes:
searchreturns ID28search_and_reconstructreturns ID22search_and_return_codesreturns ID22
Expected behavior
All three APIs should honor the same SearchParametersIVF object,
including its nested quantizer_params.
They should use the same HNSW coarse-quantizer search configuration and return consistent IDs and distances for the same query.
Additional validation
- With the default HNSW setting
efSearch=16, all three paths return ID22 - With an
IndexFlatL2coarse quantizer, the results agree - An independent float64 brute-force oracle confirms that ID
22is the unique nearest neighbor - The divergence is reproduced in 10/10 independent processes
- The input hashes and output values are identical across all reproductions
Root cause
The normal IndexIVF::search path forwards the nested parameter:
quantizer->search(
sub_n,
sub_x,
cur_nprobe,
coarse_dis.get(),
idx.get(),
params ? params->quantizer_params : nullptr);Relevant source:
https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIVF.cpp#L336-L342
However, search_and_reconstruct uses:
quantizer->search(n, x, cur_nprobe, coarse_dis.get(), idx.get());Relevant source:
https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIVF.cpp#L1128-L1148
Similarly, search_and_return_codes uses:
quantizer->search(n, x, cur_nprobe, coarse_dis.get(), idx.get());Relevant source:
https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIVF.cpp#L1183-L1204
The public SearchParametersIVF structure contains the
quantizer_params field:
https://github.com/facebookresearch/faiss/blob/v1.15.0/faiss/IndexIVF.h#L68-L95
The official FAISS documentation also explicitly documents nested quantizer parameters for IVF-HNSW indexes:
https://github.com/facebookresearch/faiss/wiki/Setting-search-parameters-for-one-query
Suggested fix
Update both helper methods to forward the nested parameters:
quantizer->search(
n,
x,
cur_nprobe,
coarse_dis.get(),
idx.get(),
params ? params->quantizer_params : nullptr);Suggested regression test
Add a test using an IVF index with an HNSW coarse quantizer and a deterministic
nested SearchParametersHNSW object. Verify that:
searchsearch_and_reconstructsearch_and_return_codes
return identical IDs and distances when called with the same
SearchParametersIVF.
Impact
This issue makes the helper APIs silently ignore per-query HNSW search parameters. Applications that rely on per-query tuning may receive different neighbors, reconstructed vectors, or returned codes depending on which API they call.
Source: facebookresearch/faiss