`du` counts hard-linked files once per path, so both size columns over-report
Bug report form
- I have done a basic search of the issue tracker to find any existing issues that are similar.
- I have checked that my version is at least the latest stable release available via my installation method.
Describe the bug
nushell's du sums every directory entry it walks. However, if there are multiple hard links, the file size's is reported once per link - so both apparent and physical columns in the output report more space than the tree actually contains.
This bit me earlier - compare nushell's du output for my cargo cache:
foobar in ~/.cache/cargo-target
✦ ❯ du * | to md| path | apparent | physical |
|---|---|---|
| /Users/victorhooi/.cache/cargo-target/CACHEDIR.TAG | 177 B | 4.0 kB |
| /Users/victorhooi/.cache/cargo-target/debug | 178.6 GB | 179.1 GB |
| /Users/victorhooi/.cache/cargo-target/doc | 52.7 MB | 57.2 MB |
| /Users/victorhooi/.cache/cargo-target/release | 779.0 MB | 788.7 MB |
| /Users/victorhooi/.cache/cargo-target/tmp | 64 B | 0 B |
vs GNU du (well, technically I think it's the BSD version):
foobar in ~/.cache/cargo-target took 5s
✦ ❯ ^du -chs *
4.0K .rustc_info.json
4.0K .rustdoc_fingerprint.json
4.0K CACHEDIR.TAG
64G debug
55M doc
752M release
0B tmp
65G totalI thought I was going crazy, so I used dua to confirm:
foobar in ~/.cache/cargo-target took 2s
✦ ❯ dua
0 B tmp
4.10 KB .rustc_info.json
4.10 KB .rustdoc_fingerprint.json
4.10 KB CACHEDIR.TAG
57.23 MB doc
788.77 MB release
68.86 GB debug
69.71 GB totalIn this case - rustc hard-links resued codegen units into deps/, which is what caused nushell's du to get tripped up.
AFAIK, Nushell's behaviour differs from every other du:
- POSIX: "A file that occurs multiple times shall be counted and written for only one entry, even if the occurrences are under different file operands." (link)
- GNU coreutils: Default behaviour is to count each hard-link once. You need to use "--count-links" to override the default. (link) (They also have a test specifically for hard links).
- BSD/macOS: Same as GNU coreutils - default is to count files with multiple hard links only once. You can use the
-lflag to override the default.
physical is also off: it is computed from allocated blocks and labelled as disk usage, but reports space that is not consumed.
Previously raised as https://github.com/nushell/nushell/issues/1317 (2020), but it was tagged as category:enhancement and delight and auto-closed by the bot. However, I'd argue this isn't a missing nicety - this is a column that actively reports the wrong number.
How to reproduce
foobar in /tmp via v3.14.7 via
✦ ❯ mkdir /tmp/du-links
foobar in /tmp via v3.14.7 via
✦ ❯ dd if=/dev/zero of=/tmp/du-links/a bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.065729 secs (1595301922 bytes/sec)
foobar in /tmp via v3.14.7 via
✦ ❯ ln /tmp/du-links/a /tmp/du-links/b
foobar in /tmp via v3.14.7 via
✦ ❯ ln /tmp/du-links/a /tmp/du-links/c
foobar in /tmp via v3.14.7 via
✦ ❯ du -sh /tmp/du-links
Error: nu::parser::unknown_flag
× The `du` command doesn't have flag `-s`.
╭─[repl_entry #6:1:5]
1 │ du -sh /tmp/du-links
· ┬
· ╰── unknown flag
╰────
help: Use `--help` to see available flags
foobar in /tmp via v3.14.7 via
✦ ❯ du /tmp/du-links
╭───┬───────────────┬──────────┬──────────╮
│ # │ path │ apparent │ physical │
├───┼───────────────┼──────────┼──────────┤
│ 0 │ /tmp/du-links │ 314.5 MB │ 314.5 MB │
╰───┴───────────────┴──────────┴──────────╯
foobar in /tmp via v3.14.7 via
✦ ❯ ^du /tmp/du-links
204800 /tmp/du-links
foobar in /tmp via v3.14.7 via
foobar in /tmp via v3.14.7 via
❯ ^du -chs /tmp/du-links
100M /tmp/du-links
100M totalExpected behavior
apparent and physical should each count a file once per invocation, regardless of how many hard links reach it - so 104.9 MB / 104.9 MB above.
Configuration
| key | value |
|---|---|
| version | 0.115.1 |
| os | macOS 25.6.0 (aarch64-darwin) |
| filesystem | APFS |
Source: nushell/nushell