agent: a non-tty exec can hang until a detached child exits (`notify_waiters()` loses the exit wake-up)
Version: 4.1.0, runtime-rs (containerd-shim-kata-v2 4.1.0, commit ddcb1ad8), Cloud Hypervisor.
Reproduce
for i in $(seq 20); do
s=$(date +%s.%N); nerdctl exec <ctr> sh -c 'setsid sleep 20 &'
echo "$(date +%s.%N) - $s" | bc
doneExpected: each exec returns about 0.5 s after sh exits. Actual, over 40 runs: 35 took the child's full 20 s, and 5 took 0.6 s. With the child's stdio redirected (setsid sleep 20 </dev/null >/dev/null 2>&1 &), 0 of 40 hung.
Cause
src/agent/src/rpc.rs:877-897:read_streamraces the read againstterm_exit_notifier.notified(), then drains for 500 ms and returns EOF.src/agent/src/signal.rs:98callsnotify_term_close(), which callsNotify::notify_waiters()(src/agent/rustjail/src/process.rs:197-199). That call wakes only futures registered at that instant and stores no permit.- If the process exits while no
ReadStreamcall is in flight, the wake-up is lost. Every later read then blocks on the pipe that the detached child holds. A short command exits before the first read, so it usually hits this.
We confirmed the order of events under gdb in a model of this code path that uses the same tokio version (1.52.2). At the notify_waiters() call, no read was in flight and the Notify state was EMPTY. The exec then waited out the child.
Possible fix: set an exit flag (an AtomicBool or a watch) before notify_waiters(). In read_stream, call enable() on the Notified future and check the flag before waiting. In the model, this took the hangs from 20 of 20 to 0 of 20, with no output lost.
Source: kata-containers/kata-containers