hnswlib persists uninitialized heap memory into length.bin (malloc'd buffer written at full max_elements capacity)

What happened?

length.bin in a persisted HNSW segment can contain uninitialized heap memory from the writing process. The buffer behind it is allocated with malloc at full max_elements_ capacity and never zeroed, and the first persist writes the full capacity rather than the live element count, so every slot past cur_element_count is whatever was previously on the heap.

Observed on a Chroma 1.5.8 persistent collection (Python client, Rust backend). One segment's length.bin was 400 bytes (the DEFAULT_MAX_ELEMENTS = 100 case). Slots 1 to 10 held float32 vector norms near 1.0. Slots 11 to 100 (360 bytes) held recycled process memory: the printable runs in it matched embedding ids and document text from that same collection's chroma.sqlite3. Nothing about the mechanism keeps other process contents (for example a token that happened to be in a freed buffer) out of the sidecar.

Filed here because chroma-core/hnswlib has issues disabled. This is the fork pinned by Chroma's Cargo.lock at 6868102, and master still has the same code at the time of filing.

Where

In chroma-core/hnswlib hnswlib/hnswalg.h at commit 6868102:

• constructor, line 318: length_memory_ = (char *)malloc(max_elements_ * sizeof(float)); with no zero-fill
• initPersistentIndex(), lines 1028-1031: output_length.write(length_memory_, max_elements_ * sizeof(float)); writes the full allocation
• the same malloc pattern recurs at lines 1271 and 1456 in the load and resize paths

persistDirty() (lines 1087-1095) only writes the dirty slots, so the heap tail written by the first persist survives on disk until each slot is individually overwritten by a real element.

Reproduction shape

1. Create a persistent collection and add fewer than max_elements vectors (default 100; with hnsw:batch_size and hnsw:sync_threshold set to 2 this persists after 2 adds).
2. Read <segment>/length.bin. Its size is max_elements * 4. Unpack as little-endian float32: the first cur_element_count slots are norms, the rest are not.
3. Extract printable runs of 32+ bytes from the tail and search for them in the process's own data (chroma.sqlite3). They match.

Suggested fix

Any of: calloc instead of malloc for length_memory_ (all three sites), or memset after allocation, or write only cur_element_count * sizeof(float) in initPersistentIndex(). calloc is the smallest change and also covers the load and resize paths.

Versions

• chromadb 1.5.8 (Python, chromadb_rust_bindings), Linux x86_64
• hnswlib fork at 6868102bde454dc761136e1994490133a6a026bb, resolved from Chroma 1.5.8's Cargo.lock