Add opt-in request body passthrough to avoid buffering large uploads
Is your feature request related to a problem? Please describe.
The gateway buffers a request body three times on its way to the upstream. Two of those points are in the muxer, before routing. The third is in the reverse proxy, after routing. For a large upload, this multiplies memory use.
A verification run against a 1 GiB payload, combining the muxer's buffering with the reverse proxy's copy, measured a 3.04x total-allocation delta, close to three times the payload. That happens even though the body reaches the upstream unchanged. For an upstream that already accepts a streamed body, such as an object store or a media-ingest endpoint, that buffering adds no value. It only slows the upload and costs memory on the gateway.
Prior work already identified this problem: #2357, #7754, #3976 (closed without an implementation), and #8150 (which measured the memory multiplication but shipped no opt-in).
flowchart LR
subgraph Muxer[Muxer, before routing]
A1[Request arrives] --> A2{maxRequestBodySize set?}
A2 -->|Yes| A3["nopCloseRequestBodyErr<br/>greedy read, all frames"]
A2 -->|No| A4["copyRequest<br/>lazy read, skips chunked bodies already"]
end
subgraph Proxy[Reverse proxy, after routing]
B1["deepCopyBody<br/>always runs today"]
end
A3 --> B1
A4 --> B1
B1 --> C[Upstream]Describe the solution you'd like
Add a default-off, two-layer opt-in for request body passthrough.
- A gateway-wide flag,
http_server_options.enable_request_body_passthrough(boolean, default false). It gates the two muxer buffering points, since routing has not happened yet and there is no per-API spec to check. - A per-API flag,
server.requestBodyPassthrough.enabledin OAS, orenable_request_body_passthroughin a classic API definition. It gates the reverse-proxy copy.
A request streams end to end only when both flags are true. A request where only one flag is true falls back to today's buffered behavior, so partial adoption stays safe.
Describe alternatives you've considered
- A per-API-only flag, with no gateway-wide toggle. The muxer still buffers in that design, so memory only improves at the reverse proxy, about a third of the possible win. Rejected because the muxer holds the larger share of the multiplier.
- A labs config entry instead of a stable flag. Rejected: the design is simple enough, and test coverage already covers the two-layer default-off structure, so a labs entry adds no extra safety.
- A hook variable that toggles streaming at runtime, without a restart. Rejected: it needs a mutable global and a panic-safety wrapper. A config-driven flag matches the gateway's existing patterns, such as the
labsmap from #7754, without that complexity.
Additional context
Measured with runtime.MemStats.TotalAlloc, combining the muxer's buffering with the reverse proxy's copy on today's master: a 1 GiB payload produces a 3.04x allocation delta. With passthrough enabled, a 100 MB payload run through the same combined path adds under 1 KB, near zero rather than scaling with payload size. This mirrors the measurement technique in #8150.
max_request_body_size is enforced independently through http.MaxBytesReader, which does not buffer, so streaming does not weaken that limit. A middleware that already reads and re-wraps the body, such as validation or a transform, keeps working unchanged. Streaming is inert wherever a middleware already consumes the body.
Detailed analytics and GraphQL analytics both assume a re-readable body. They need the same early return already used for a websocket or gRPC request. That return prevents an empty or corrupted log entry on a streamed request.
AI assistance disclosure
I used an AI coding assistant (Claude) to draft this issue and the linked PR's code, tests, and description. I read and reviewed the diff and the reasoning behind it myself before opening either.
Source: TykTechnologies/tyk