curl CPU usage waiting for input from stdin

(I intended to create this as a discussion and not an issue. I must have filled out the wrong form.)

While reviewing #22383 on my Windows machine I noticed that curl uses about 1% CPU (0.6-1.3) waiting for stdin when no data is available.

first window:
socat tcp-listen:1234,reuseaddr,fork -

second window:
curl -T . http://localhost:1234

This is probably due to a change last year to improve non-blocking STDIN performance (#17566) by unpausing more often when no data is available, so that the read function can be called more often, about 1000 times per second. To do that the special progress callback for stdin waits a millisecond (or non-Windows up to a millisecond) and then unpauses even if no data is available on stdin:

https://github.com/curl/curl/blob/68720b4837284335b2d63cb358f8f6ce65f5bc55/src/tool_cb_rea.c#L172-L187

According to #17566 the reasoning for that was the progress function was not called frequently enough (eg once per second) and therefore a longer wait (5ms, 2ms, etc) could delay processing of data.

@denandz I wonder if you see the same delay with a wait that's closer to the frequency of the progress callback (eg one second). This way even if the progress callback returns due to no data it should be called again immediately if there's no data available. I know this seems counter-intuitive but could you try this:

diff --git a/src/tool_cb_rea.c b/src/tool_cb_rea.c
index bf9bd15..0552633 100644
--- a/src/tool_cb_rea.c
+++ b/src/tool_cb_rea.c
@@ -170,9 +170,20 @@ int tool_readbusy_cb(void *clientp,
   if(config->readbusy) {
     if(ulprev == ulnow) {
 #ifndef _WIN32
-      waitfd(1, per->infd);
+      waitfd(1000, per->infd);
 #else
-      curlx_wait_ms(1); /* sleep */
+      {
+        fd_set bits;
+        int waitms = 1000;
+        struct timeval timeout;
+
+        timeout.tv_sec = waitms / 1000;
+        timeout.tv_usec = (int)((waitms % 1000) * 1000);
+
+        FD_ZERO(&bits);
+        FD_SET((unsigned)per->infd, &bits);
+        select(per->infd + 1, &bits, NULL, NULL, &timeout);
+      }
 #endif
     }
 

In Windows when infd is a socket for stdin this should work, which is why I have the Windows change as well but I'm curious specifically how this works for your use case with waitfd not Windows. If this works the code would need to be updated to account for configured timeout config->timeout_ms but disregard that for now.

edit: Well on second thought it is probably better to do like a 25ms wait for the fd in the read callback, this way the callback is still called more often. A 1 second wait in the progress callback is too long for other transfers. Still curious though.