IMAP: large SEARCH response fails with CURLE_TOO_LARGE instead of returning results

I did this

Ran a plain IMAP SEARCH against a mailbox large enough that the untagged * SEARCH ... reply exceeds ~64 KB on its single line (a few tens of thousands of messages). For example, against a mailbox with 20000 messages:

curl -sS "imap://127.0.0.1/INBOX?ALL" -u user:secret

Reproduced against a real Dovecot server (2.3.21); the mailbox size just needs to push the SEARCH result past the receive-buffer limit.

I expected the following

curl returns the full * SEARCH <numbers> line.

What happened instead

The transfer fails and delivers zero bytes:

curl: (100) A value or data field grew larger than allowed    # CURLE_TOO_LARGE

The same request with a small result (few messages) works fine, so the failure is purely a function of the response line length.

Root cause

A SEARCH reply is a single untagged line listing every matching message number, so it can be arbitrarily long. In Curl_pp_readresp() (lib/pingpong.c) a response is accumulated in pp->recvbuf until the terminating newline. That buffer is capped at DYN_PINGPPONG_CMD (64 KB); once a single line crosses the cap, dyn_nappend() (lib/curlx/dynbuf.c) returns CURLE_TOO_LARGE and the whole transfer aborts. So any SEARCH whose result line exceeds ~64 KB fails outright.

This affects the standard SEARCH path (URL ?<criteria>) and custom -X SEARCH / -X "UID SEARCH ..." (which run through the LIST state) equally.

Note on the existing KNOWN_BUGS entry

docs/KNOWN_BUGS.md has an entry under "Email protocols" titled IMAP SEARCH ALL truncated response that attributes this to truncation code in pingpong.c "truncating it to 40 characters". That description looks stale; pingpong.c no longer truncates to a fixed length (it uses a dynamic buffer); the current behavior is the hard failure above.

I have a proposed fix (streaming over-long response lines to the body so memory stays bounded) that I can submit as a PR for discussion.

curl/libcurl version

curl 8.22.0-DEV (current master, 112a8b5adf), built --without-ssl. Believed to affect all versions since the pingpong receive buffer was capped.

operating system

Ubuntu 24.04, x86_64.