WebRTC play negotiates H.264 SDP but sends H.265 RTP when stream is HEVC (black video)

Summary

When an SRT publisher sends an H.265/HEVC stream, and a WebRTC player connects without an explicit vcodec query parameter, SrsRtcPlayerNegotiator::negotiate_play_capability selects H.264 for the SDP answer while the RTC bridge sends H.265 RTP packets. The browser receives an H.264 SDP answer but H.265 RTP — resulting in black video or negotiation failure.

Environment

• SRS 7.0.157 (branch v7.0-d0)
• Ingest: SRT (MPEG-TS with HEVC)
• Playback: WebRTC/WHEP
• Stream path: multi-level app (e.g. tenant-1/live/0/0/AZR-001)

Reproduction

1. Publish an H.265 stream over SRT:
   复制

   ffmpeg -re -i input.mp4 -c:v hevc_nvenc -f mpegts srt://server:10082?streamid=#!::r=tenant-1/live/0/0/AZR-001

2. Verify the stream is active and HEVC:
   bash复制

   curl http://server:1985/api/v1/streams/ | jq '.streams[] | {name, video: {codec: .video.codec}}'
   # → codec: "HEVC"

3. Open WebRTC playback without ?vcodec=h265 in a browser that supports H.265 (e.g. Safari, or Firefox with nvidia-vaapi-driver):
   复制

   POST http://server:1985/rtc/v1/play/
   {"sdp": "<offer with H264+H265>", "streamurl": "webrtc://server/tenant-1/live/0/0/AZR-001"}

4. Observe: the SDP answer advertises H.264, but the RTP stream carries H.265. Video is black.

Root Cause

1. Placeholder tracks are created H.264-first

SrsRtcSource::init_for_play_before_publishing() (in src/app/srs_app_rtc_source.cpp, lines 508–544) creates two placeholder video track descriptions before the publisher connects:

• H.264 track pushed first (line 513)
• H.265 track pushed second (line 531)

This is intentional — it lets clients choose either codec during SDP negotiation before publishing starts.

2. negotiate_play_capability picks the first track

When the player does not specify vcodec, negotiate_play_capability (in src/app/srs_app_rtc_conn.cpp, lines 3882–3892) falls back to the first track description:

SrsVideoCodecId prefer_codec = srs_video_codec_str2id(ruc->vcodec_);
if (prefer_codec == SrsVideoCodecIdReserved) {
    std::vector<SrsRtcTrackDescription *> source_track_descs = source->get_track_desc("video", "");
    if (!source_track_descs.empty()) {
        SrsRtcTrackDescription *first_track = source_track_descs.at(0);
        prefer_codec = srs_video_codec_str2id(first_track->media_->name_);
    } else {
        return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "no video track in source");
    }
}

get_track_desc("video", "") returns all video tracks in insertion order (lines 841–854), so at(0) is always the H.264 placeholder — even when the actual live stream is H.265.

3. SDP answer and RTP disagree

The negotiator then builds an H.264 SDP answer, but the RTC bridge (SrsRtcBridge) packetizes RTP using the actual source codec (H.265). The browser:

• Receives a=rtpmap:... H264/90000 in the SDP answer
• Receives H.265 NAL units in RTP with payload type KVideoPayloadTypeHevc
• Cannot decode → black video

This also produces the error no valid found h265 payload type when the browser's SDP offer includes H.265 but SRS has already committed to H.264 from the first track.

Expected Behavior

When the live source is already publishing, SRS should prefer the actual video codec from the live source's video sequence header (vsh_format()->vcodec_->id_) before falling back to track-description order. Track order should only be the fallback when no live codec is available (i.e. play before publish).

Proposed Fix

Fetch the live source and check its video sequence-header codec before falling back to the first track:

--- a/trunk/src/app/srs_app_rtc_conn.cpp
+++ b/trunk/src/app/srs_app_rtc_conn.cpp
@@ -3882,12 +3882,27 @@
         } else if (remote_media_desc.is_video()) {
             SrsVideoCodecId prefer_codec = srs_video_codec_str2id(ruc->vcodec_);
             if (prefer_codec == SrsVideoCodecIdReserved) {
-                // Get the source codec if not specified.
+                // Get the source codec if not specified. Prefer the codec the
+                // live source actually delivers (from its sequence header):
+                // the RTMP-to-RTC bridge packetizes RTP with that codec, while
+                // the default track descriptions list H264 first regardless —
+                // trusting the track order makes the SDP answer advertise a
+                // codec the RTP stream never uses, and browsers show black.
                 std::vector<SrsRtcTrackDescription *> source_track_descs = source->get_track_desc("video", "");
-                if (!source_track_descs.empty()) {
+                SrsSharedPtr<SrsLiveSource> live_source = _srs_sources->fetch(req);
+                if (live_source.get() && live_source->meta()
+                        && live_source->meta()->vsh_format()
+                        && live_source->meta()->vsh_format()->vcodec_) {
+                    prefer_codec = live_source->meta()->vsh_format()->vcodec_->id_;
+                }
+                if (prefer_codec == SrsVideoCodecIdReserved && !source_track_descs.empty()) {
                     SrsRtcTrackDescription *first_track = source_track_descs.at(0);
                     prefer_codec = srs_video_codec_str2id(first_track->media_->name_);
-                } else {
+                }
+                if (prefer_codec == SrsVideoCodecIdReserved) {
                     return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "no video track in source");
                 }
             }

Compatibility

• H.264 streams: vsh_format()->vcodec_->id_ returns SrsVideoCodecIdAVC → negotiates H.264 as before.
• H.265 streams: returns SrsVideoCodecIdHEVC → negotiates H.265 correctly.
• Play before publish (no live source): live_source.get() is NULL or vsh_format() is NULL → falls back to existing track-order logic → H.264 (unchanged behavior).
• Explicit ?vcodec=h264 or ?vcodec=h265: prefer_codec is already set → skips both paths (unchanged).

Verification

We have a custom Docker image (azr-srs-h265:7) running this patch in production:

• SRS API reports codec=HEVC for the stream.
• Firefox (with nvidia-vaapi-driver) receives the correct H.265 SDP answer and plays the video.
• Without the patch, the same browser receives an H.264 SDP answer and shows black video.
• H.264 streams (e.g. from real drones) continue to negotiate H.264 correctly — no regression.

Scope

This issue is scoped to negotiate_play_capability only. A separate HTTP multi-level app-path limitation in serve_http_impl (where SRS intentionally rejects paths with a different slash count than the mount template) is not part of this report — that is a documented design restriction, not a bug.