#12583·dask

da.to_zarr writes corrupt data when specifying chunks or shards

Author: joshua-gouldCreated Sep 4, 2026Updated Sep 8, 2026
Labelsneeds triage

If a user specified chunks or shards arguments to da.to_zarr that do not align with dask chunks, then zarr file is corrupted:

python
import dask.array as da
import h5py

a = da.arange(100).reshape((10, 10)).rechunk((2, 2))
da.to_zarr(a, "test1.zarr", overwrite=True)
print((a == da.from_zarr("test1.zarr")).all().compute())  # works

da.to_zarr(a, "test2.zarr", chunks=(4, 4), overwrite=True)
print((a == da.from_zarr("test2.zarr")).all().compute())  # corrupted

da.to_zarr(a, "test3.zarr", shards=(4, 4), overwrite=True)
print((a == da.from_zarr("test3.zarr")).all().compute())  # corrupted

a.to_hdf5("test1.h5", 'x')
with h5py.File("test1.h5", 'r') as f:
    print((a == f['x']).all().compute())  # works

# hdf5 seems to take care of misaligned chunks
a.to_hdf5("test2.h5", 'x', chunks=(4, 4))
with h5py.File("test2.h5", 'r') as f:
    print((a == f['x']).all().compute())  # works

Environment:

  • Dask version: latest from github
  • Python version: 3.13
  • Install method (conda, pip, source): source