hbbs rebuilds its HTTP client (full CA-store parse) for every client heartbeat when calling its own `/api/heartbeat`
Bug Description
We run rustdesk-server-pro (licensed) in Docker with host networking for about 85 devices, and noticed hbbs sitting at a quarter of a CPU core doing nothing but idle heartbeats. hbbr, nginx, everything else on the box is asleep. CPU grows linearly with the number of registered devices, roughly 0.3% of a core per device, which puts a 1-vCPU server at its limit somewhere around 300 devices. We saw this on 1.8.4 and it's unchanged on 1.8.6.
- All the CPU is in one tokio worker thread, almost entirely user-space. The sqlite threads are idle and there's basically no disk I/O, so it isn't db work.
- Every time a client sends its UDP heartbeat to :21116, hbbs opens a new TCP connection to
127.0.0.1:21114and POSTs/api/heartbeatto itself with{id, uuid, ver, modified_at}. New connection every time, no keep-alive, so a few hundred sockets sit in TIME-WAIT constantly. - The handler is not the problem. We replayed that exact POST on loopback, including for a real registered device with its real uuid, and it comes back in about half a millisecond with no measurable CPU.
- What is expensive is making the call. hbbs reads about 116 KB in ~30 read syscalls per heartbeat, and if you sample
/proc/<pid>/fdyou catch/etc/ssl/certs/ca-certificates.crt(182 KB) open about a third of the time. libssl/libcrypto are mapped into the process.
So it looks like the heartbeat path builds a brand-new HTTP client per call, which spins up a TLS context and parses the entire system CA bundle, in order to send a plaintext request to itself over loopback. Parsing ~140 certificates per heartbeat seems inline with the ~48 ms per heartbeat.
How to Reproduce
To reproduce, you just need a handful of online clients: watch top -H on hbbs (one hot thread), ss -tanp | grep 21114 (hbbs owning short-lived client sockets to itself), and loop ls -l /proc/$(pgrep hbbs)/fd for a few seconds and the CA bundle keeps showing up.
Expected Behavior
The fix seems straightforward, guessing at code, build that client once and reuse it, or skip TLS setup entirely since the target is plaintext 127.0.0.1. Reusing the connection would clean up the TIME-WAIT churn too. Our own loopback test showed ~100x improvement in response and CPU time. Another option, but more complicated, might be to skip the network stack all together with sockets (more complicated OS compat). Happy to test a build if that helps.
Operating system(s) on local (controlling) side and remote (controlled) side
Rocky Linux 10
RustDesk Version(s) on local (controlling) side and remote (controlled) side
1.8.4 -> 1.8.6
Screenshots
Additional Context
No response
Source: rustdesk/rustdesk