#9454·lance

bug: readers before v12.0.0 fail to open a dataset whose inline transaction they cannot decode

Author: wjones127Created Sep 21, 2026Updated Sep 22, 2026
Labelsbug

Opening a dataset eagerly decodes the manifest's inline transaction section purely to warm the session cache. Before #7740 the decode error propagated, so a transaction carrying an operation the reader does not know makes the whole dataset version unopenable — even though nothing about reading the data needs the transaction.

#7740 fixed this on main and first shipped in v12.0.0 (2026-09-02). It was never backported, so every maintained release line from release/v1.0 through release/v11.0 still has the failing form.

Why this is not hypothetical

Operations are added to the Transaction.operation oneof over time, and each addition breaks every reader released before it:

Operation First shipped Readers that fail
Clone, UpdateBases, UpdateMemWalState 1.x line earlier 1.x
DataOverlay (field 115) v9.0.0 (#7381) v1.0 – v8.x

So a v8 reader opening a table whose newest version was written by v9+ with a DataOverlay operation already fails today. #7740's own commit message says as much ("This already applies to recently added operation types ... read by older 1.x releases").

The inline transaction section has existed since v1.0 (#4774), so the exposure spans the whole 1.0–11.0 range.

Mechanism

Dataset::load_manifest (rust/lance/src/dataset.rs, release/v11.0 line 804):

rust
if let Some(transaction_offset) = manifest.transaction_section
    && manifest_size - transaction_offset <= last_block.len()
{
    ...
    let transaction: Transaction =
        lance_table::format::pb::Transaction::decode(message_data)?.try_into()?;
    // ^ only to populate metadata_cache

Transaction::try_from returns Error::internal("Transaction message did not contain an operation") when prost decodes an unknown oneof variant to None, and the ? fails the open.

Reproduction

  1. Write a table with a version whose operation is newer than the reader, e.g. a DataOverlay commit from v9+, small enough to inline (under MAX_INLINE_TRANSACTION_BYTES).
  2. Open it with a v8 (or any v1.0–v11.0) reader.
  3. Open fails. The same table opens fine on v12.0.0+.

Fix

Backport #7740 to release/v1.0release/v11.0. The change is small and self-contained: make the decode return Option, log a warning, and skip the cache warm-up. Paths that genuinely need the transaction (read_transaction, conflict resolution) still surface the error at their call sites.

Worth considering alongside it: a backport only helps readers that take the patch. For deployments pinned to an unpatched older version, the only lever is on the writer — not inlining a transaction whose operation postdates some compatibility floor, so transaction_section is absent and an old reader skips the block entirely. That is a bigger design question and does not need to block the backport.