[Discussion/Feature Request] High-throughput egress optimization & datagram batching (sendmmsg) for Pion WebRTC

Problem Statement

We are running a high-load live video streaming server built on Pion WebRTC (v4.2.18, pion/ice v4.4.0, pion/srtp v3.0.12). Under our baseline benchmark scenario:

• 600 ingested cameras (H.264, 25 FPS, 720p).
• 240 active WebRTC viewers via WHEP.
• Egress rate: ~50,000–60,000 RTP packets/sec (~65 MB/s).
• Packetization: Average ~8.1 RTP packets per frame (~7 packets for P-frames, ~41 packets for IDR keyframes).

When profiling with pprof, application logic and AES-GCM SRTP encryption (crypto/internal/fips140/aes/gcm.gcmAesEnc) are extremely fast and take less than 1% CPU.

However, Linux kernel syscall overhead (Syscall6 / sendtoInet4) dominates the entire WebRTC egress path, consuming ~21–25% of total server CPU:

pumpFrames (goroutine per peer) └─ TrackLocalStaticSample.WriteSample └─ TrackLocalStaticRTP.WriteRTP └─ interceptorToTrackLocalWriter.WriteRTP └─ nack.ResponderInterceptor.BindLocalStream.func1 └─ SessionSRTP.writeRTP └─ mux.Endpoint.Write └─ ice.Conn.Write └─ udpAddrPortReaderWriter.WriteToAddrPort └─ net.UDPConn.WriteToUDPAddrPort └─ syscall.sendtoInet4 (26.80s / ~21% CPU)

Because each video frame produces 7–41 RTP packets, every single frame synchronously triggers 7 to 41 consecutive sendto() system calls per peer connection.

---

Questions & Architecture Discussion

We would love to get advice from the Pion maintainers and community on the idiomatic way to optimize this:

1. Batching / sendmmsg Support: Are there existing plans or recommended patterns to support sendmmsg(2) in pion/transport, pion/ice, or pion/webrtc?

2. Recommended Hook / Extension Point: If we want to accumulate datagrams generated by a single WriteSample() (or a slice of *rtp.Packet) and flush them in a single batch directly to the socket:

   • What would be the cleanest architecture that preserves:
     • Interceptors (e.g. NACK responder caching unencrypted RTP)?
     • SRTP encryption (SessionSRTP)?
     • Per-peer isolation without cross-goroutine lock contention on shared ICEUDPMux?
   • Would a batch-aware interface on ice.Conn / udpMuxedConn (e.g. WriteBatch or BeginBatch/EndBatch) be open for upstream contribution?

3. Existing High-Load Best Practices: Are there any known configuration knobs or best practices within Pion (or custom SettingEngine options) for minimizing per-packet kernel transitions on high-egress servers?

Thank you for your incredible work on Pion!