Claude: Backport #4742 to 7.0: answer the stream video codec. v7.0.162

Backports #4742 (`c52c0f9a9`) to the SRS 7.0 release branch. Fixes #4738.

Playing an H.265 stream over WebRTC without ?vcodec= answered H.264 and
delivered no video. SrsRtcSource::init_for_play_before_publishing creates
one placeholder track per codec, H.264 first, and negotiate_play_capability
fell back to the first one whenever the client asked for no codec. A
non-WebRTC publisher keeps those placeholder tracks, and the RTMP-to-RTC
bridge builds RTP on the track of the codec it detects in the sequence
header, so the player subscribed to the H.264 SSRC while every packet
arrived on the H.265 one and SrsRtcPlayStream::send_packet dropped all of
them.

The rule is now the same for H.264, H.265 and AV1. Before publishing the
codec of the stream is unknown, so the client decides and any codec it asks
for is allowed. While publishing the stream decides: asking for no codec
answers the codec of the stream, and asking for a different one is refused,
because SRS never transcodes for WebRTC and that client could only ever get
a black picture. SrsRtcRtpBuilder::initialize_video_track records the codec
it selected on the source as bridge_video_codec_, which publish_video_codec
returns, and on_unpublish clears it so the next player may choose again. For
a WebRTC publisher, whose negotiated tracks replace the placeholders, the
single remaining track description is the codec of the stream.

Refuse a player with no matching track instead of answering audio only. The
previous code left track_descs empty and built an answer without video, so
the client waited for a track that never arrived; it now returns an SDP
exchange error naming the codec. This also makes a VP9 request before
publishing an error rather than silent audio.

Add an AV1 placeholder track for play before publishing, so that a client
can negotiate the codec that only a WebRTC publisher can produce.

Inject the collaborators these paths reach: ISrsRtcSSRCGenerator is a new
interface over SrsRtcSSRCGenerator, and SrsRtcSource and
SrsRtcPlayerNegotiator now hold config, statistic, shared timer and
generator pointers instead of reaching for the globals. ISrsAppConfig gains
the five RTC getters the bridge uses. Without this the negotiation path
cannot be exercised in a unit test.

Cover the matrix in srs_utest_workflow_rtc_conn.cpp with MockRtcPlayScenario:
play before publishing, an H.264 and an H.265 stream from the bridge, an AV1
stream from a WebRTC publisher, and an H.264-only client asking for an H.265
stream. The bridge cases assert the answered SSRC is the one the bridge
sends with, which is what the reported failure got wrong.

The source and test changes apply unchanged from 8.0. The version bump and
changelog entry are the 7.0 ones, and the skills issue record in #4742 does
not exist on this branch, so it is not backported.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: winlinvip

internal/version/version.go

@@ -15,7 +15,7 @@ func VersionMinor() int {
 }
 
 func VersionRevision() int {
-	return 161
+	return 162
 }
 
 func Version() string {

trunk/doc/CHANGELOG.md

@@ -7,6 +7,7 @@ The changelog for SRS.
 <a name="v7-changes"></a>
 
 ## SRS 7.0 Changelog
+* v7.0, 2026-09-18, Merge [#4742](https://github.com/ossrs/srs/pull/4742): RTC: Answer the video codec of the stream for WebRTC play. v7.0.162 (#4742)
 * v7.0, 2026-09-17, Merge [#4741](https://github.com/ossrs/srs/pull/4741): Codec: Validate lengthSizeMinusOne to avoid abort on malformed SPS. v7.0.161 (#4741)
 * v7.0, 2026-08-20, Merge [#4729](https://github.com/ossrs/srs/pull/4729): SRT: Upgrade libsrt from 1.5.3 to 1.5.6 for CVE-2026-55868/55869. v7.0.160 (#4729)
 * v7.0, 2026-08-18, Merge [#4659](https://github.com/ossrs/srs/pull/4659): Proxy: Fix resource leak in ParseBody. v7.0.159 (#4659)

trunk/src/app/srs_app_config.hpp

@@ -544,6 +544,11 @@ class ISrsAppConfig : public ISrsConfig
     virtual int get_time_jitter(std::string vhost) = 0;
     virtual bool get_mix_correct(std::string vhost) = 0;
     virtual bool try_annexb_first(std::string vhost) = 0;
+    virtual bool get_rtc_keep_bframe(std::string vhost) = 0;
+    virtual bool get_rtc_keep_avc_nalu_sei(std::string vhost) = 0;
+    virtual bool get_rtc_server_merge_nalus() = 0;
+    virtual srs_utime_t get_rtc_pli_for_rtmp(std::string vhost) = 0;
+    virtual int get_rtc_opus_bitrate(std::string vhost) = 0;
     virtual bool get_vhost_is_edge(std::string vhost) = 0;
     virtual bool get_atc_auto(std::string vhost) = 0;
     virtual bool get_reduce_sequence_header(std::string vhost) = 0;

trunk/src/app/srs_app_rtc_conn.cpp

@@ -3187,12 +3187,14 @@ SrsRtcPlayerNegotiator::SrsRtcPlayerNegotiator()
 {
     config_ = _srs_config;
     rtc_sources_ = _srs_rtc_sources;
+    ssrc_generator_ = SrsRtcSSRCGenerator::instance();
 }
 
 SrsRtcPlayerNegotiator::~SrsRtcPlayerNegotiator()
 {
     config_ = NULL;
     rtc_sources_ = NULL;
+    ssrc_generator_ = NULL;
 }
 
 bool srs_sdp_has_h264_profile(const SrsMediaPayloadType &payload_type, const string &profile)
@@ -3877,8 +3879,23 @@ srs_error_t SrsRtcPlayerNegotiator::negotiate_play_capability(SrsRtcUserConfig *
             track_descs = source_audio_tracks;
         } else if (remote_media_desc.is_video()) {
             SrsVideoCodecId prefer_codec = srs_video_codec_str2id(ruc->vcodec_);
+
+            // While publishing, the codec of the stream decides, because SRS never transcodes for
+            // WebRTC. Answering any other codec delivers no media at all.
+            // @see https://github.com/ossrs/srs/issues/4738
+            SrsVideoCodecId stream_codec = source->publish_video_codec();
+            if (stream_codec != SrsVideoCodecIdReserved) {
+                if (prefer_codec == SrsVideoCodecIdReserved) {
+                    prefer_codec = stream_codec;
+                } else if (prefer_codec != stream_codec) {
+                    return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "stream is %s, not the required %s",
+                                         srs_video_codec_id2str(stream_codec).c_str(), ruc->vcodec_.c_str());
+                }
+            }
+
+            // Before publishing, the codec of the stream is unknown, so the client decides. If it
+            // requires no codec, use the first track, see SrsRtcSource::init_for_play_before_publishing.
             if (prefer_codec == SrsVideoCodecIdReserved) {
-                // Get the source codec if not specified.
                 std::vector<SrsRtcTrackDescription *> source_track_descs = source->get_track_desc("video", "");
                 if (!source_track_descs.empty()) {
                     SrsRtcTrackDescription *first_track = source_track_descs.at(0);
@@ -3959,6 +3976,14 @@ srs_error_t SrsRtcPlayerNegotiator::negotiate_play_capability(SrsRtcUserConfig *
 
                 track_descs = source->get_track_desc("video", "H264");
             }
+
+            // Refuse the player rather than answering audio only without any error, which leaves
+            // the client waiting for a video track that never arrives.
+            // @see https://github.com/ossrs/srs/issues/4738
+            if (track_descs.empty()) {
+                return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "no %s track in source",
+                                     srs_video_codec_id2str(prefer_codec).c_str());
+            }
         }
 
         for (int j = 0; j < (int)track_descs.size(); ++j) {
@@ -4007,7 +4032,7 @@ srs_error_t SrsRtcPlayerNegotiator::negotiate_play_capability(SrsRtcUserConfig *
             // Otherwise, generate a new SSRC for each player.
             // @see https://github.com/ossrs/srs/issues/3850
             if (!keep_original_ssrc) {
-                track->ssrc_ = SrsRtcSSRCGenerator::instance()->generate_ssrc();
+                track->ssrc_ = ssrc_generator_->generate_ssrc();
             }
 
             // TODO: FIXME: set audio_payload rtcp_fbs_,

trunk/src/app/srs_app_rtc_conn.hpp

@@ -1013,6 +1013,7 @@ class SrsRtcPlayerNegotiator : public ISrsRtcPlayerNegotiator
 SRS_DECLARE_PRIVATE: // clang-format on
     ISrsAppConfig *config_;
     ISrsRtcSourceManager *rtc_sources_;
+    ISrsRtcSSRCGenerator *ssrc_generator_;
 
 public:
     SrsRtcPlayerNegotiator();

trunk/src/app/srs_app_rtc_source.cpp

@@ -393,10 +393,15 @@ SrsRtcSource::SrsRtcSource()
 
     publish_stream_ = NULL;
     stream_desc_ = NULL;
+    bridge_video_codec_ = SrsVideoCodecIdReserved;
 
     req_ = NULL;
     rtc_bridge_ = NULL;
     circuit_breaker_ = _srs_circuit_breaker;
+    config_ = _srs_config;
+    stat_ = _srs_stat;
+    shared_timer_ = _srs_shared_timer;
+    ssrc_generator_ = SrsRtcSSRCGenerator::instance();
 
     pli_for_rtmp_ = pli_elapsed_ = 0;
     // Initialize stream_die_at_ to current time to prevent newly created sources
@@ -423,6 +428,11 @@ SrsRtcSource::~SrsRtcSource()
     srs_trace("free rtc source id=[%s]", cid.c_str());
 
     app_factory_ = NULL;
+    circuit_breaker_ = NULL;
+    config_ = NULL;
+    stat_ = NULL;
+    shared_timer_ = NULL;
+    ssrc_generator_ = NULL;
 }
 
 // CRITICAL: This method is called AFTER the source has been added to the source pool
@@ -498,14 +508,14 @@ void SrsRtcSource::init_for_play_before_publishing()
         audio_track_desc->type_ = "audio";
         audio_track_desc->id_ = "audio-" + rand.gen_str(8);
 
-        uint32_t audio_ssrc = SrsRtcSSRCGenerator::instance()->generate_ssrc();
+        uint32_t audio_ssrc = ssrc_generator_->generate_ssrc();
         audio_track_desc->ssrc_ = audio_ssrc;
         audio_track_desc->direction_ = "recvonly";
 
         audio_track_desc->media_ = new SrsAudioPayload(kAudioPayloadType, "opus", kAudioSamplerate, kAudioChannel);
     }
 
-    // video track descriptions - support both H.264 and H.265 for play before publishing
+    // video track descriptions - support H.264, H.265 and AV1 for play before publishing
     // This allows clients to choose their preferred codec during SDP negotiation
     if (true) {
         // H.264 track description
@@ -515,7 +525,7 @@ void SrsRtcSource::init_for_play_before_publishing()
         h264_track_desc->type_ = "video";
         h264_track_desc->id_ = "video-h264-" + rand.gen_str(8);
 
-        uint32_t h264_ssrc = SrsRtcSSRCGenerator::instance()->generate_ssrc();
+        uint32_t h264_ssrc = ssrc_generator_->generate_ssrc();
         h264_track_desc->ssrc_ = h264_ssrc;
         h264_track_desc->direction_ = "recvonly";
 
@@ -533,7 +543,7 @@ void SrsRtcSource::init_for_play_before_publishing()
         h265_track_desc->type_ = "video";
         h265_track_desc->id_ = "video-h265-" + rand.gen_str(8);
 
-        uint32_t h265_ssrc = SrsRtcSSRCGenerator::instance()->generate_ssrc();
+        uint32_t h265_ssrc = ssrc_generator_->generate_ssrc();
         h265_track_desc->ssrc_ = h265_ssrc;
         h265_track_desc->direction_ = "recvonly";
 
@@ -543,6 +553,22 @@ void SrsRtcSource::init_for_play_before_publishing()
         h265_payload->set_h265_param_desc("level-id=156;profile-id=1;tier-flag=0;tx-mode=SRST");
     }
 
+    if (true) {
+        // AV1 track description. Note that AV1 is published by WebRTC only, because the bridge
+        // from RTMP or SRT supports H.264 and H.265 only.
+        SrsRtcTrackDescription *av1_track_desc = new SrsRtcTrackDescription();
+        stream_desc->video_track_descs_.push_back(av1_track_desc);
+
+        av1_track_desc->type_ = "video";
+        av1_track_desc->id_ = "video-av1-" + rand.gen_str(8);
+
+        uint32_t av1_ssrc = ssrc_generator_->generate_ssrc();
+        av1_track_desc->ssrc_ = av1_ssrc;
+        av1_track_desc->direction_ = "recvonly";
+
+        av1_track_desc->media_ = new SrsVideoPayload(KVideoPayloadTypeAv1, "AV1", kVideoSamplerate);
+    }
+
     set_stream_desc(stream_desc.get());
 }
 
@@ -703,14 +729,13 @@ srs_error_t SrsRtcSource::on_publish()
         }
 
         // The PLI interval for RTC2RTMP.
-        pli_for_rtmp_ = _srs_config->get_rtc_pli_for_rtmp(req_->vhost_);
+        pli_for_rtmp_ = config_->get_rtc_pli_for_rtmp(req_->vhost_);
 
         // @see SrsRtcSource::on_timer()
-        _srs_shared_timer->timer100ms()->subscribe(this);
+        shared_timer_->timer100ms()->subscribe(this);
     }
 
-    SrsStatistic *stat = _srs_stat;
-    stat->on_stream_publish(req_, _source_id.c_str());
+    stat_->on_stream_publish(req_, _source_id.c_str());
 
     return err;
 }
@@ -729,6 +754,9 @@ void SrsRtcSource::on_unpublish()
     }
     _source_id = SrsContextId();
 
+    // The codec of the stream is unknown again, so the next player may choose any codec.
+    bridge_video_codec_ = SrsVideoCodecIdReserved;
+
     for (size_t i = 0; i < event_handlers_.size(); i++) {
         ISrsRtcSourceEventHandler *h = event_handlers_.at(i);
         h->on_unpublish();
@@ -737,14 +765,15 @@ void SrsRtcSource::on_unpublish()
     // free bridge resource
     if (rtc_bridge_) {
         // For SrsRtcSource::on_timer()
-        _srs_shared_timer->timer100ms()->unsubscribe(this);
+        if (shared_timer_) {
+            shared_timer_->timer100ms()->unsubscribe(this);
+        }
 
         rtc_bridge_->on_unpublish();
         srs_freep(rtc_bridge_);
     }
 
-    SrsStatistic *stat = _srs_stat;
-    stat->on_stream_close(req_);
+    stat_->on_stream_close(req_);
 
     // Destroy and cleanup source when no publishers and consumers.
     if (consumers_.empty()) {
@@ -857,6 +886,37 @@ std::vector<SrsRtcTrackDescription *> SrsRtcSource::get_track_desc(std::string t
     return track_descs;
 }
 
+void SrsRtcSource::set_bridge_video_codec(SrsVideoCodecId codec)
+{
+    bridge_video_codec_ = codec;
+}
+
+SrsVideoCodecId SrsRtcSource::publish_video_codec()
+{
+    // The bridge from RTMP or SRT keeps the tracks created for play before publishing, so the
+    // codec of the stream is the one it detected in the video sequence header.
+    if (bridge_video_codec_ != SrsVideoCodecIdReserved) {
+        return bridge_video_codec_;
+    }
+
+    if (!stream_desc_ || stream_desc_->video_track_descs_.empty()) {
+        return SrsVideoCodecIdReserved;
+    }
+
+    // A WebRTC publisher replaces the tracks created for play before publishing by the tracks it
+    // negotiated, which describe exactly one codec, the codec of the stream. While the tracks for
+    // play before publishing describe one track per supported codec, so the codec is still unknown.
+    SrsVideoCodecId codec = SrsVideoCodecId(stream_desc_->video_track_descs_.at(0)->media_->codec(true));
+    for (int i = 1; i < (int)stream_desc_->video_track_descs_.size(); i++) {
+        SrsRtcTrackDescription *track_desc = stream_desc_->video_track_descs_.at(i);
+        if (SrsVideoCodecId(track_desc->media_->codec(true)) != codec) {
+            return SrsVideoCodecIdReserved;
+        }
+    }
+
+    return codec;
+}
+
 srs_error_t SrsRtcSource::on_timer(srs_utime_t interval)
 {
     srs_error_t err = srs_success;
@@ -889,6 +949,7 @@ SrsRtcRtpBuilder::SrsRtcRtpBuilder(ISrsAppFactory *factory, ISrsRtpTarget *targe
 {
     rtp_target_ = target;
     source_ = source;
+    config_ = _srs_config;
 
     req_ = NULL;
     format_ = new SrsRtmpFormat();
@@ -920,6 +981,7 @@ SrsRtcRtpBuilder::~SrsRtcRtpBuilder()
     srs_freep(video_builder_);
 
     app_factory_ = NULL;
+    config_ = NULL;
 }
 
 srs_error_t SrsRtcRtpBuilder::initialize_audio_track(SrsAudioCodecId codec)
@@ -970,6 +1032,11 @@ srs_error_t SrsRtcRtpBuilder::initialize_video_track(SrsVideoCodecId codec)
         return srs_error_wrap(err, "initialize video builder");
     }
 
+    // Now the codec of the stream is known, so the player must be answered with this codec, on the
+    // SSRC of this track, otherwise every packet we build is dropped.
+    // @see https://github.com/ossrs/srs/issues/4738
+    source_->set_bridge_video_codec(codec);
+
     srs_trace("RTMP2RTC: Initialize video track with codec=%s, ssrc=%u, pt=%d",
               codec_name.c_str(), video_ssrc, video_payload_type);
 
@@ -987,11 +1054,11 @@ srs_error_t SrsRtcRtpBuilder::initialize(ISrsRequest *r)
     }
 
     // Setup the SPS/PPS parsing strategy.
-    format_->try_annexb_first_ = _srs_config->try_annexb_first(r->vhost_);
+    format_->try_annexb_first_ = config_->try_annexb_first(r->vhost_);
 
-    keep_bframe_ = _srs_config->get_rtc_keep_bframe(req_->vhost_);
-    keep_avc_nalu_sei_ = _srs_config->get_rtc_keep_avc_nalu_sei(req_->vhost_);
-    merge_nalus_ = _srs_config->get_rtc_server_merge_nalus();
+    keep_bframe_ = config_->get_rtc_keep_bframe(req_->vhost_);
+    keep_avc_nalu_sei_ = config_->get_rtc_keep_avc_nalu_sei(req_->vhost_);
+    merge_nalus_ = config_->get_rtc_server_merge_nalus();
     srs_trace("RTC bridge from RTMP, keep_bframe=%d, keep_avc_nalu_sei=%d, merge_nalus=%d",
               keep_bframe_, keep_avc_nalu_sei_, merge_nalus_);
 
@@ -1110,7 +1177,7 @@ srs_error_t SrsRtcRtpBuilder::init_codec(SrsAudioCodecId codec)
     codec_ = app_factory_->create_audio_transcoder();
 
     // Initialize the codec according to the codec in stream.
-    int bitrate = _srs_config->get_rtc_opus_bitrate(req_->vhost_); // The output bitrate in bps.
+    int bitrate = config_->get_rtc_opus_bitrate(req_->vhost_); // The output bitrate in bps.
     if ((err = codec_->initialize(codec, SrsAudioCodecIdOpus, kAudioChannel, kAudioSamplerate, bitrate)) != srs_success) {
         return srs_error_wrap(err, "init codec=%d", codec);
     }
@@ -3794,6 +3861,14 @@ SrsRtcSSRCGenerator::~SrsRtcSSRCGenerator()
 {
 }
 
+ISrsRtcSSRCGenerator::ISrsRtcSSRCGenerator()
+{
+}
+
+ISrsRtcSSRCGenerator::~ISrsRtcSSRCGenerator()
+{
+}
+
 SrsRtcSSRCGenerator *SrsRtcSSRCGenerator::instance()
 {
     if (!instance_) {