#5708·burn

pytorch-reader: PytorchError has no source() chain, doubled hedges in Display, and a non-checkpoint file reports a raw opcode

Author: antimoraCreated Sep 16, 2026Updated Sep 17, 2026
Labelsenhancementstore

Summary

Three things about PytorchError that are cheap to change now and awkward after the first publish, since they are what every caller sees and matches on. None is a correctness problem; all came up while wrapping the crate's errors in another error type.

1. Error::source() is None for every variant

rust
impl std::error::Error for PytorchError {}
impl std::error::Error for PickleError {}

Io, Pickle, Zip, Tar and Serde all wrap a real error, but nothing reaches it through the standard chain. A caller behind anyhow / eyre / Box<dyn Error> gets the flattened Display string and loses the io::ErrorKind, the ZipError variant and the serde path. The crate already distinguishes "file disagrees with itself" from "OS could not read it" for Tensor::read (InvalidData / UnexpectedEof vs the rest); that distinction is lost as soon as the error is boxed.

Probe:

rust
let err = PytorchReader::new("/nonexistent/x.pt").unwrap_err();
err.source()   // None; the io::Error is inside `PytorchError::Io`

thiserror is already a dependency (used by nested::error::Error), so #[source] / #[from] on the two hand-written enums would do it and would also delete the two hand-written Display impls.

2. Display text stacks two hedges and a double period

PickleError::InvalidOpCode ends its message with a sentence, then PytorchError::Pickle appends another:

Pickle parsing error: Invalid pickle opcode: 0x5b. The file may be corrupted or use an unsupported pickle feature.. This may indicate an unsupported PyTorch file format or corrupted file.

The same wrapper adds "This may indicate an unsupported PyTorch file format or corrupted file." to every pickle error, including UnsupportedType which already says "This may indicate a full model save rather than a state_dict." Suggest the outer variants carry a short prefix only (pickle: {0}) and each inner message says what is actually known.

3. A file that is not a checkpoint reports a raw opcode

detect_format falls through to FileFormat::Pickle for anything that is not ZIP, TAR or the legacy magic, so a .safetensors, .json, .onnx or .gguf handed to PytorchReader::new fails with the message in 2: an opcode value (0x5b is [, the first byte of a JSON array or of a safetensors header) plus "may be corrupted". Nothing says "this is not a PyTorch checkpoint".

Two options, not exclusive:

  • Before treating a file as a plain pickle, check that its first byte is PROTO (0x80) or one of the protocol-0 opcodes a dict can start with ((, }, ], c, ...), and otherwise return InvalidFormat("not a PyTorch checkpoint: no ZIP, TAR, legacy or pickle header"). A safetensors file can be named in the message from its 8-byte length + { prefix, since that is the one people will mix up with .pt most.
  • Or leave detection as is and make InvalidOpCode at offset 0 say "the file does not start with a pickle" instead of "may be corrupted".

Not proposing

Keeping PytorchError #[non_exhaustive] is right; adding source() and rewording Display are non-breaking. Only the plain-pickle detection change alters behavior, and only for files that fail today anyway.