#9362·lance

Feature request: error handling for `push_uri` API for error handling

Author: dentinyCreated Sep 17, 2026Updated Sep 19, 2026
Labelsfeature

Hi team, I'm heavily doing dataset conversion in my daily work, for example, convert open dataset from huggingface to lance. One pain point that I have is

  • I prefer to use push_uri and Ingestion mode for blob download ingestion, which internally decides blob storage option based on size and handles streaming download
  • But more often than not, blob download fails because of all types of reasons; for example, HTTP response content-length reports incorrect number of bytes, the URL link doesn't work any more, DNS resolution failure, etc
  • In the current implementation, one single missing blob will fail the whole write process, which usually takes several hours long
  • To unblock myself, I have to do all the above work myself (download at application side and call push_blob), but I'm wondering if there's any chance lance could provide better support for error handling? For example, provides the ability to get users customize their own error handling logic, say, log and continue.