Malformed chunked response detector creates false-positive ‘premature close’ errors when HTTP Agent involved
When sockets get reused for multiple requests to endpoints that support chunked transfer encoding, the code in node-fetch that's designed to detect malformed chunked responses gets confused and throws false-positive 'premature close' errors.
Sockets get reused in such a way when you use an http.Agent or an https.Agent in ‘keep alive’ mode.
Reproduction
Steps to reproduce the behavior:
- Add this logging to
node-fetch(diff link) - Create an HTTPS Agent with
keepAlive: trueandmaxSocketsof 5. - Make 6 requests in a row for an endpoint that supports chunked transfer encoding
In that repro, 6 requests are made with a limit of 5 sockets in keep-alive mode. One of those sockets gets reused for the 6th request. The presence of the 6th request as a pending listener for that reused socket confuses the logic in node-fetch that is designed to detect corrupt chunked responses.
Here's a repro project that you can clone that does all of the above: https://github.com/steveluscher/repro-node-fetch-premature-close-with-agent
Expected behavior
No premature close error should be thrown, since the chunked responses are well-formed.
Your Environment
| software | version |
|---|---|
| node-fetch | 2.6.8-2.6.12 |
| node | 18+ |
Additional context
This bug is the root of https://github.com/solana-labs/solana-web3.js/issues/1418.
cc/ @achingbrain.
Source: node-fetch/node-fetch