feat(python): add a shared read-through cache for URI-backed training assets

SDK

Python

Problem

When a training row contains a plain audio_uri such as s3://bucket/A.wav, LanceDB returns the URI string. The subsequent object download happens in the user's transform or collate code, outside the table reader. With several ranks and DataLoader workers, the same object can therefore be downloaded independently by multiple processes and again in later runs.

The existing table-data cache work (#3623 / #3656) addresses projected Lance table reads. It does not automatically cache arbitrary objects referenced by URI. Blob v2 is another modeling option, but applications with an existing URI column still need an explicit asset-read path.

Desired behavior

Offer an optional worker-safe read-through asset cache that can be composed with StreamingDataset transforms without making LanceDB responsible for audio decoding.

Acceptance criteria

• Key entries by URI plus a caller-provided checksum, ETag, generation, or object version; do not treat a bare mutable URI as immutable.
• Preserve the current row order while repeated references reuse cached bytes or a local file.
• Share completed entries safely across DataLoader workers, ranks on the same node, and later runs.
• Coalesce concurrent misses for the same object and bound total download concurrency.
• Use atomic writes and recover cleanly from interrupted or partial downloads.
• Configure cache directory, storage limit, and a documented eviction policy.
• Invalidate or miss when the object fingerprint changes.
• Report hit/miss counts, hit bytes, remote bytes, download latency, and current cache size.
• Keep fetch and decode metrics separate.
• Add multi-process tests for concurrent hits, changed-object invalidation, interruption recovery, and eviction.
• Add a complete-file pre-decode benchmark with the target eight-rank configuration.

Design boundary

This could be a reusable cache/resolver utility consumed by a transform rather than implicit URI interpretation inside StreamingDataset. The API should also document when storing audio as a Lance Blob v2 column is the better choice.