#3539·fission

storagesvc: stream-hash large /v1/archive uploads so HMAC verification memory is bounded

Author: sanketsudakeCreated Jun 24, 2026Updated Aug 20, 2026
Labelsenhancementarea-securityarea-storagesvcpriority/medium

Background

#3538 fixed large package uploads failing with 413 Request Entity Too Large by making the storagesvc /v1/archive HMAC body cap operator-tunable (storagesvc.maxArchiveSizeMib / STORAGE_MAX_ARCHIVE_SIZE_MIB, default 256 MiB).

That unblocks large packages, but only by letting operators trade memory for archive size. The underlying constraint is unchanged: the HMAC signature is computed over SHA256(body), so the verifier (pkg/auth/hmac/verifier.go) io.ReadAlls the entire request body into memory before it can check the signature, then re-injects it via bytes.NewReader. At a high cap, storagesvc transiently buffers the whole archive in RAM during verification — and the upload client (builder/fetcher) already buffers the same archive in memory twice (the multipart buffer in pkg/storagesvc/client/client.go UploadReader, plus the signer's io.ReadAll in pkg/auth/hmac/signer.go).

Problem

The MaxBodyBytes cap conflates two concerns:

  1. DoS protection — bound the memory an unauthenticated caller can force the verifier to allocate before the 401/413 (legitimate, must keep).
  2. Maximum supported archive size — an artificial ceiling that only exists because verification buffers in memory.

For the one bulk-data endpoint (/v1/archive), concern 2 should not require trading RAM 1:1 with archive size.

Proposed direction

Make HMAC verification of large bodies memory-bounded regardless of archive size, so the storagesvc cap can be raised (or effectively removed) without a proportional memory blowup:

  • In the verifier, stream the body through the SHA-256 hasher while spooling to a temp file once it exceeds a small in-memory threshold (e.g. reuse the multipart/io spill-to-disk pattern), then re-inject r.Body from the temp file with proper lifecycle/cleanup.
  • Keep a hard MaxBodyBytes as the DoS ceiling, but decouple it from "largest archive we can verify".
  • Mirror the same streaming on the client signer (pkg/auth/hmac/signer.go) and the storagesvc client multipart builder so the builder/fetcher don't hold the whole archive in memory either.
  • Verify the cold-path latency and disk-usage tradeoffs; ensure temp files are cleaned on every exit path (success, signature mismatch, read error, context cancel).

Acceptance criteria

  • Uploading an archive several × the in-memory threshold succeeds with storagesvc memory staying flat (not scaling with archive size) — demonstrated with a pprof/heap before-after.
  • HMAC semantics unchanged: body-bound signature still verified byte-for-byte; existing pkg/auth/hmac tests stay green.
  • Temp files are always cleaned up; no fd/disk leak under load or on rejection paths.
  • Client-side (signer + storagesvc client) memory for an upload is bounded too.

Notes

  • Touches the security-critical verifier — treat as its own change with adversarial review, not folded into a routine PR.
  • This is the deeper follow-up flagged in the #3538 PR.