Proposal: keys-only range reads for API 800 with C, Java, and CLI support
We'd like to add getRangeKeys so clients can scan a range without fetching its values. This follows #5789 and #7417, where people asked for the same capability and suggested filtering at the storage server. We'd like to target 8.0 / API 800 if the release timing allows it.
Today, applications fetch key/value pairs and discard the values. Even fdbcli getrangekeys does this. When values are much larger than keys, we spend bandwidth and allocate memory for data the caller never uses.
We propose adding:
- Native C++
getRangeKeys(...), with the same selectors, limits, reverse reads, and snapshot semantics asgetRange(...). - Two new C functions:
fdb_transaction_get_range_keys(...)to start the read, andfdb_future_get_range_keys(...)to extract anFDBKeyarray, count, andmoreflag. Neither function exists upstream today. - Java
ReadTransaction.getRangeKeys(...)overloads returningAsyncIterable<byte[]>.
We'd also route the existing CLI command through the new operation.
The storage server would return only keys. We'd reuse its existing range traversal and MVCC merge, and push the filtering into memory, Redwood, RocksDB, and sharded RocksDB where practical. Other engines could start with a full-read projection fallback. Disk blocks may still contain values, so we expect fewer copies and smaller replies, but we need measurements before claiming less physical I/O.
Compatibility
Applications would need updated headers/bindings and a library that exposes the new API. We propose these execution paths:
| Configuration | Execution path |
|---|---|
| Updated client, selected API 800+, compatible server | Use the keys-only storage-server operation for eligible reads |
| Updated client, selected API below 800 | Perform ordinary getRange and project keys inside the native client |
| Updated multiversion wrapper, external client missing the new functions | Use the same native projection fallback |
| Transactions with local mutations, or special-key reads | Initially use the existing range path and project keys |
We would preserve conflicts, pagination, cancellation, and result ownership. A value-only update would still conflict with a serializable key-range read.
The wire protocol needs a separate decision. API selection doesn't tell us whether a server supports the new RPC. We'd like to agree on a development-protocol bump or explicit capability negotiation before settling the 8.0 compatibility contract.
We have a local prototype covering the native client, C, Java, CLI, and the four engines. Focused integration and simulation checks pass. We'll validate the upstream port, run broader checks, and compare it with getRange followed by key projection across different value sizes.
Open questions
- Would you prefer a dedicated
getRangeKeysoperation with a typed key result, or a projection option ongetRange? - Is API 800 the right introduction point? Should the updated native library also support this operation when applications select an older API?
- Should 8.0 use a development-protocol bump, or negotiate support and fall back for existing prerelease binaries?
- What additional semantics or validation should we cover before upstreaming?
Source: apple/foundationdb