h264_amf hangs forever in ~avcodec_encode_session_t — third-party/build-deps was rolled back to a pre-FFmpeg-8.0 pin by merge 10fd290b

On AMD hardware, Apollo wedges during encoder probe. One core pins at 100% and the process never recovers. NVIDIA is unaffected. A user hit this on a Vega iGPU and we traced it with an instrumented build.

Where it stops

The log ends immediately before avcodec_send_frame(ctx, nullptr) in the encode session destructor, and the marker after that call never prints:

Info: Creating encoder [h264_amf]
Info: [PROBE] avcodec_open2 SUCCEEDED for [h264_amf]
Info: [PROBE] ~session: entering flush
<nothing further, one core at 100%>

That is src/video.cpp:328 on current master:

~avcodec_encode_session_t() {
  // Flush any remaining frames in the encoder
  if (avcodec_send_frame(avcodec_ctx.get(), nullptr) == 0) {
    packet_raw_avcodec pkt;
    while (avcodec_receive_packet(avcodec_ctx.get(), pkt.av_packet) == 0);
  }

The call never returns. It is not the receive loop — execution never reaches it.

Why

That drain arrived with Sunshine 02036920, "build(deps): Update to FFmpeg 8.0 branch (#4143)". That commit deliberately changed the code and the FFmpeg it runs against, moving third-party/build-deps from 94369e63 to a21ef2e3. The drain was written for FFmpeg 8.0.

Apollo took both halves correctly at the time. The pin then moved backwards:

10fd290b is "Merge remote-tracking branch 'sunshine/master'". Its parents pin cf5dffaf (db4c2fab) and c38829d6 (1a96d135). The merge result is a9a7f863, which is neither — it is a build-deps commit from 2025-07-12, predating the FFmpeg 8.0 bump by a month. It looks like a submodule conflict resolved to a stale value rather than to either side.

Master still pins a9a7f863 today. So Apollo currently ships a drain written for FFmpeg 8.0 running against a pre-8.0 FFmpeg. On AMF that call blocks forever. NVENC returns immediately, which is why this has gone unreported for so long.

What this explains

Only the mismatched combination fails, and only Apollo-derived builds have it.

Suggested fix

Bump third-party/build-deps forward again — c38829d6 is where the pin was before 10fd290b, so restoring that is the minimal change and puts the submodule back on the side the merge should have taken.

Deleting the drain also stops the hang, and is what we did in our fork as a stopgap. It is safe in isolation: the packets it receives go into a local that is discarded on the very next line, so it moves no data — its only effect is letting the encoder drain before a teardown that destroys everything anyway. But bumping the pin is the right fix here, since the stale submodule is the actual defect and it may affect more than this one call site.

Downstream report and our stopgap, for reference: vibesoftwarecoder/ApolloVibe#2 (fix confirmed by the reporter on the original hardware).