#9424·lance

fix_schema renumbers a duplicated nested field without re-parenting its children, leaving the dataset unopenable

Author: wkaltCreated Sep 18, 2026Updated Sep 18, 2026

rust/lance/src/io/commit.rs::fix_schema runs on every commit whose fragments have more than one data file. When two files of one fragment claim the same field id, it maps the duplicate to a fresh id and applies the mapping to the schema with

rust
let field = manifest.schema.mut_field_by_id(*old_field_id).unwrap();
field.id = *new_field_id;

Only id changes. The field's children keep parent_id = old_field_id, so for a nested field the committed manifest can no longer be read:

LanceError(Schema): Field 'item' (id=8) references parent id 2, which must appear earlier in the protobuf field list

Reproducer: a table with chunks: list<struct<text: utf8, ordinal: int32>> (field ids 2..5) stored in one file with the other columns; commit an Update { update_mode: RewriteColumns } whose updated fragment keeps that file unchanged and adds a second file with fields = [2, 3, 4, 5]. The commit succeeds and writes __lateral id=7 parent=-1, item id=8 parent=2, text id=9 parent=3, ordinal id=10 parent=3; every later open of the dataset fails with the error above. A flat column in the same situation is renumbered consistently and reads back, so the fixup is only broken for nested fields.

The writer is at fault for committing duplicate coverage (a writer of ours did), but a fixup that leaves the manifest unopenable is worse than refusing the commit. Either fix_schema should renumber the subtree with Field::set_id semantics (children re-parented), or the duplicate-coverage case should be rejected at validation.