Panorama merge_from corrupts merged vectors

Summary

faiss.IndexIVFFlatPanorama.merge_from() inherits the generic IndexIVF::merge_from() implementation. That implementation reads Panorama's special level-major, batch-padded storage as ordinary row-major IVFFlat codes, corrupting vectors transferred from the source index.

Environment

• FAISS: v1.15.0
• Package: faiss-cpu==1.15.0

Reproduction

import faiss
import numpy as np

d = 4
train = np.array([[0,0,0,0], [10,10,10,10]], "float32")
left_x = np.array([[1,1,1,1], [11,11,11,11]], "float32")
right_x = np.array([[2,2,2,2], [12,12,12,12]], "float32")

def make(x):
    index = faiss.index_factory(d, "IVF2,FlatPanorama2_2")
    index.cp.min_points_per_centroid = 1
    index.nprobe = 2
    index.train(train)
    index.add(x)
    return index

left, right = make(left_x), make(right_x)
left.merge_from(right, 2)

left.make_direct_map()
print([left.reconstruct(i).tolist() for i in range(4)])
print(left.search(right_x, 1)[0].ravel().tolist())

Run:

OMP_NUM_THREADS=1 python repro.py

Observed

[
  [1.0, 1.0, 1.0, 1.0],
  [11.0, 11.0, 11.0, 11.0],
  [2.0, 2.0, 0.0, 0.0],
  [12.0, 12.0, 0.0, 0.0]
]
[4.0, 4.0]

Expected

[
  [1.0, 1.0, 1.0, 1.0],
  [11.0, 11.0, 11.0, 11.0],
  [2.0, 2.0, 2.0, 2.0],
  [12.0, 12.0, 12.0, 12.0]
]
[0.0, 0.0]

The destination reports ntotal == 4, and the source index is emptied, but the vectors transferred from the source are corrupted. The same merge on vanilla IndexIVFFlat preserves all vectors and returns zero distances.

Root Cause

IndexIVF::merge_from() delegates to InvertedLists::merge_from(). The generic implementation obtains source codes through the bulk get_codes(list_no) API:

ScopedCodes codes(oivf, i);

This does not use the offset-aware get_single_code(list_no, offset) accessor.

ArrayInvertedListsPanorama overrides Panorama-specific write and single-code methods but inherits the ordinary ArrayInvertedLists::get_codes() implementation. Its source buffer is already stored in Panorama's level-major, batch-padded layout. The generic merge therefore interprets that buffer as flat row-major vectors and passes it to Panorama's add_entries(), which repacks the bytes incorrectly.

Relevant source paths:

• faiss/IndexIVF.cpp — IndexIVF::merge_from
• faiss/invlists/InvertedLists.cpp — generic inverted-list merge
• faiss/invlists/InvertedLists.h — ScopedCodes
• faiss/invlists/InvertedLists.cpp — ArrayInvertedListsPanorama::add_entries
• faiss/invlists/InvertedLists.cpp — Panorama single-code reconstruction

Impact

This is a data-integrity and search-correctness bug. The index metadata and vector count remain apparently valid, but merged vectors are permanently corrupted after the source lists are cleared.

Suggested Fix

IndexIVFFlatPanorama should override merge_from(), or Panorama inverted lists should implement a Panorama-aware merge that reads each source entry through get_single_code(list_no, offset) and converts it correctly before insertion.