#14007·skywalking

[NIDX-02 workstream] Replace the Property shard index

Author: hanahmilyCreated Aug 25, 2026Updated Sep 11, 2026
Labelsdatabasebydb file compatible change

[NIDX-02 workstream] Replace the Property shard index

Parent: #13990 Blocked by: #14002

Tracking parent — do not apply Backlog. Decompose this workstream into ordered TDD leaves only after every NIDX-01 leaf has merged and the actual native reader seam is available on main.

End-state boundary

banyand/property/db.newShard selects the native implementation for existing and new Property shards. The completed workstream owns compatible Property writes, exact/boolean/range queries, explicit sort, callbacks, snapshot publication/recovery, merge-time expiry, reference-safe GC, repair integration, backup, and same-file rollback.

The retained legacy Property constructor is an explicit rollback choice during the compatibility window, not a second active writer.

Cutover and writer ownership

The implementation choice is Property-database-wide during NIDX-02, not inferred independently for each shard.

  • The Property database establishes exclusive writer ownership before scanning its directory or opening any shard store.
  • While it owns the database, startup resolves exactly one implementation from an explicit Property-wide rollout selection; property/db.newShard uses that selection for every existing and newly created shard.
  • A process that cannot establish ownership fails before either implementation opens a shard writer.
  • The native index neither creates nor requires an index-local PID file, lock file, or writer-origin marker. Such runtime files are outside ICE/snapshot compatibility.
  • Selection is not inferred from ICE segments, snapshot manifests, CRC32 values, or other writer-origin heuristics; those files remain implementation-neutral during the same-file rollback window.
  • Rollback stops admission, drains durable callbacks, closes every active shard writer, changes the Property-wide selection, and then reopens the same files with the retained legacy constructor.
  • Simultaneous native and legacy writers for the Property database are prohibited.

Required behavior

  • Legacy-created shards open without rewrite and retain documents, repeated stored values, deletions, and explicit order.
  • Replace/upsert leaves one visible latest document; delete removes it from every supported query and scan.
  • Native output is ICE v3/snapshot v3 accepted by the pinned rollback binary for query, mutation, merge, and restart.
  • Publication and callbacks are durable; every crash cut selects a complete old or new generation.
  • Merge, expiry, and GC retain only data reachable from protected snapshots.
  • CRC32 fields remain present but are never calculated or validated.

Just-in-time decomposition gate

When #14002 closes:

  1. inspect the merged native reader and measured NIDX-01 run reports;
  2. identify the smallest live Property caller for each new writer/lifecycle capability;
  3. file leaf issues in dependency order so issue numbers ascend;
  4. give each leaf one pre-agreed seam, literal fixture results, one RED command, one e2e command, and focused package suites; and
  5. apply Backlog only to the first unblocked leaf.

Do not pre-file format-only writer, snapshot, merge, or GC tickets. Each leaf must activate a live Property behavior in its merge.

Completion criteria

All future leaves merge; the complete property/db.newShard role is native; the two-binary compatibility and crash matrices pass; and the workstream closes before NIDX-03 decomposition begins.

Lexical non-regression

Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive bluge tokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutral legacy oracle or compatibility writer labels plus an immutable revision or content hash rather than adding a retired module name.

This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.

Design

BDB-NIDX-SPEC-001 revision 0.2 — NIDX-02 behavior