Opening large models reads several times the file size from disk
Summary
Opening a large model reads far more data from disk than the file contains, because the file-backed streams always fill a fixed 256MB window on a cache miss, regardless of how many bytes the caller asked for.
node.FileStream._fill (source/node.js:69-83), browser.FileStream._fill
(source/browser.js:799-813), protobuf.StreamReader._fill (source/protobuf.js:631-645)
and python.StreamReader._fill (source/python.js:22753-22765) all do:
const length = Math.min(0x10000000, this._length - this._offset); // always 256MB
this._buffer = new Uint8Array(length);
this._read(this._buffer, this._offset);So a 14-byte signature peek reads 256MB from disk. For container formats whose entries are scattered across the file, the window is re-filled once per entry and thrashes.
Reproduction
A 1074MB .pt2 (torch.export.save, 16 nn.Linear(4096, 4096) layers). Instrumenting
node.FileStream._read to count bytes:
file 1074.1 MB -> total bytes read from disk during open: 3490.0 MB (3.25x)
2415.9 MB x 9 new zip.Entry @ zip.js:228 <- zip.Archive.open @ zip.js:131
268.4 MB x 1 pytorch.Reader.Pickle.open @ pytorch.js:1192 <- peeks 14 bytes
268.4 MB x 1 view.ModelFactoryService._openSignature @ view.js:7853
268.4 MB x 1 view.ModelFactoryService._filter @ view.js:7836
268.4 MB x 1 view.Context.peek @ view.js:6784Nine ZIP entries scattered over 1GB cause nine full 256MB window fills to read a few hundred bytes of local file headers.
Impact
This is in the shared stream layer, so it affects every format. Measured on models with an identical 16-node graph and ~1GB of weights, so the difference is purely data-path cost:
| 1GB model | open (before) | disk read | RSS |
|---|---|---|---|
.pt2 |
1.83s | 4.25x file | 609MB |
.npz |
2.80s | 2.25x file | 1415MB |
.pt (state_dict) |
0.64s | 2.25x file | 613MB |
.pt (torchscript) |
0.70s | 2.25x file | 617MB |
.safetensors |
0.19s | 1.25x file | 321MB |
.gguf |
0.19s | 1.25x file | 323MB |
.onnx |
2.07s | 2.56x file | 1210MB |
Only files larger than ~256MB are affected; below that the window already covers the remainder of the file.
Fix
Size the window to the request, with a small minimum. PR to follow.
| 1GB model | open (before -> after) | disk read | RSS |
|---|---|---|---|
.pt2 |
1.83s -> 0.13s | 4.25x -> 1.00x | 609 -> 76MB |
.npz |
2.80s -> 0.10s | 2.25x -> 0.00x | 1415 -> 71MB |
.pt (state_dict) |
0.64s -> 0.12s | 2.25x -> 1.00x | 613 -> 74MB |
.pt (torchscript) |
0.70s -> 0.14s | 2.25x -> 1.00x | 617 -> 80MB |
.safetensors |
0.19s -> 0.01s | 1.25x -> 1.00x | 321 -> 54MB |
.gguf |
0.19s -> 0.01s | 1.25x -> 1.00x | 323 -> 55MB |
.onnx |
2.07s -> 1.81s | 2.56x -> 1.25x | unchanged |
.onnx improves only modestly because most of its cost is elsewhere: protobuf.js:13 calls
data.peek() to materialize the whole buffer for files under 512MB, and
protobuf.BufferReader.read (source/protobuf.js:493-497) uses slice() rather than
subarray(), so every raw_data initializer is copied during parse. Happy to open that
separately if it is of interest.
Environment
Netron 9.2.2 (16910af), Node 22.14.0, Linux x64.
Source: lutzroeder/netron