`builtins.fetchTarball` fails with a large archive when url starts with `file://`
Describe the bug
Using builtins.fetchTarball with a url starting with file:// that references a file larger than 4 GiB results in nix crashing.
Note that I wasn't able to test this with the typical https:// url scheme, so it may have the same bug, but I suspect this bug only applies to file://.
Steps To Reproduce
- Create a tar archive larger than
4 GiBFor testing I used the following shell command to create the archive
$ dd if=/dev/urandom of=random.bin bs=64M count=64 iflag=fullblock && tar -cf random.tar random.bin- Use
builtins.fetchTarballto fetch the archive
$ nix eval --expr "builtins.fetchTarball {url=\"file://$PWD/random.tar\"; sha256=\"\";}"This hangs for a while and then fails. Here is the most relevant (and last) line of the output:
error: appending to git packfile index: failed to mmap. Could not write data: Cannot allocate memory (libgit2 error code = 2)In this case, I'd expect nix to complain about a hash mismatch, but it doesn't.
Expected behavior
Nix to both extract the tarball to the store, and not crash.
Metadata
$ nix-env --version
nix-env (Nix) 2.34.8Additional context
The bug didn't occur when I tried fetching an archive with a size of 2 GiB.
I have more than 8 GiB of memory available when nix crashes, so it shouldn't be running out of memory. The same goes for disk space.
Checklist
- checked latest Nix manual (source)
- checked open bug issues and pull requests for possible duplicates
- reviewed the contributing guide
Add :+1: to issues you find important.
Source: NixOS/nix