TCP stream source (server mode) never recovers when the sender disappears without closing the connection

Summary

A tcp stream source in server mode stays stuck forever if its sender goes away without a FIN/RST (sender host reboots, loses power, laptop sleeps, Wi-Fi/VPN drops). snapserver keeps waiting on the dead connection, never accepts the sender's new connection, and the stream stays idle until snapserver is restarted.

Related: #995 (TCP keepalive for client connections). This report is about stream sources, where the consequence is a permanently dead source rather than a lingering client entry.

Environment

• snapserver 0.31.0 (Debian trixie, armhf, Raspberry Pi), source: tcp://10.100.0.2:4955?name=VPS&mode=server
• Sender: a remote host over WireGuard, pushing 44100:16:2 PCM
• Checked the code in v0.35.0 and develop: the relevant paths are unchanged, so current versions should behave the same.

Steps to reproduce

1. Configure a server-mode TCP source, e.g. source = tcp://0.0.0.0:4953?name=TCP&mode=server.
2. From another host, stream PCM into it, e.g. socat -u /dev/zero TCP:<server>:4953 (silence is enough to keep it connected).
3. Make the sender vanish without the server seeing a close. The simplest way to simulate a reboot is on the sender host:
   复制

   iptables -I OUTPUT -p tcp -d <server> --dport 4953 -j DROP
   pkill socat            # its FIN/RST is dropped, so the server never sees it
   sleep 5
   iptables -D OUTPUT -p tcp -d <server> --dport 4953 -j DROP

4. Start the sender again: socat -u /dev/zero TCP:<server>:4953.

Expected

snapserver notices the old connection is dead (or prefers the new one) and starts reading from the new connection.

Actual

• The stream stays idle indefinitely (No data since ... ms in stream 'TCP', switching to idle is logged once, then nothing).
• ss -tn '( sport = :4953 )' on the server shows the original connection still ESTAB with an empty receive queue (owned by snapserver). The new connection is also ESTAB, with a filling Recv-Q, but it is never accepted. In our case a sender retrying every few seconds piled up ~200 such connections.
• Only restarting snapserver recovers.

Cause

• TcpStream::connect() (server mode) arms a single acceptor_->async_accept(...) and then only reads from stream_.
• A new accept is only armed after AsioStream::do_read() hits a read error or EOF, which calls disconnect() and then connect() 100 ms later.
• A peer that vanished without FIN/RST never produces a read error. The server only reads, so it never sends anything that would draw a RST from the rebooted peer, and there is no read timeout.
• SO_KEEPALIVE is not set on the accepted socket, so the kernel never probes the connection either.
• The idle timer (check_state()) only calls setState(kIdle); it doesn't close the stale socket.

Possible fixes (either would solve it)

1. Enable TCP keepalive on source sockets. Set it after accept in server mode and after connect in client mode. Ideally make it configurable through a URI parameter such as keepalive=<idle seconds>:
   cpp复制

   stream_->set_option(boost::asio::socket_base::keep_alive(true));
   #ifdef __linux__
   int fd = stream_->native_handle();
   int idle = 20, intvl = 5, cnt = 3;
   setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &idle, sizeof(idle));
   setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL, &intvl, sizeof(intvl));
   setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(cnt));
   #endif

   After the peer reboots, the first probe gets a RST, the read fails, and the existing disconnect() → connect() path accepts the new connection.
2. Let a new connection replace the current one in server mode. Keep the acceptor armed while connected. When a new connection arrives, close the old socket and switch to the new one. This also covers peers that reconnect before any keepalive would fire.

Option 2 matches how a single-producer source is usually meant to work: the newest sender wins. Option 1 is the smaller change and also helps client mode.

Workaround we use for now

An LD_PRELOAD shim that wraps socket() to set SO_KEEPALIVE/TCP_KEEPIDLE=20/TCP_KEEPINTVL=5/TCP_KEEPCNT=3 on every TCP socket. Accepted sockets inherit these from the listening socket. With it, the reproduction above recovers about 11 s after the network comes back, without restarting snapserver. I'm happy to test a patch.