Skip to content

New BWE: Integrate SenderTransportCongestionController in Transport - #1995

Draft
ibc wants to merge 9 commits into
v3from
new-bwe-part-29
Draft

ibc wants to merge 9 commits into
v3from
new-bwe-part-29

Conversation

@ibc

@ibc ibc commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Details

  • Related to New Bandwidth Estimation stack #1895.
  • Integrate SenderTransportCongestionController in Transport under the MS_USE_BUILTIN_BWE macro condition as usual.
  • Do not advertise support for the "abs-send-time" RTP extension in audio, since mediasoup never uses received audio packets for receiver-side BWE (REMB) and never includes this extension in outgoing audio packets, as other REMB estimators do not use it for audio either (such as libwebrtc).
  • Advertise support for "transport-wide-cc-01" RTP extension also for sending in audio because we need it in the new BWE.
  • Remove BWE totally in DirectTransport because it doesn't make sense (obviously).
  • DirectTransport: Don't negotiate BWE related RTCP feedback nor RTP header extensions in produce() and consume().
  • PipeTransport: Don't negotiate BWE related RTCP feedback nor RTP header extensions in produce() (it was already done in consume()).
  • Many changes, improvements and fixes in other clases (mandatory changes).

Notes

Externally or non externally managed bitrate in Consumers

In OLD BWE, "externally managed bitrate" means that there is BWE, so the old TransportCongestionControlClient exists in the Transport and it redistributes the estimated bitrate across all Consumers every time it reports a new available bitrate. The Transport sets the flag on every Consumer at the very moment it creates that client, and the client is only created once a video Consumer that supports transport-cc or REMB is added.

In OLD BWE, "non externally managed bitrate" means that there is no BWE, so there is no TransportCongestionControlClient in the Transport.

In both cases, a Consumer (or its XxxProducerStreamManager) detects that a simulcast stream or a spatial/temporal layer is turned off or on, and both cases go through the same method, ProducerStreamManager::MayChangeLayers(), which is where the two behaviours part:

  • With "externally managed bitrate", it calls OnProducerStreamManagerNeedBitrateChange() so that the Transport redistributes the available downlink bitrate and the Consumer gets a chance to re-evaluate which simulcast stream or spatial/temporal layer to consume (otherwise it could get stuck on a stream/layer that is no longer active). It only does so when the spatial layer it works out has changed, since the temporal one follows from the bitrate it is given.
  • With "non externally managed bitrate", it calls UpdateTargetLayers() itself and applies the new stream/layer right there, without the Transport taking any part.

This PR removes the "non externally managed bitrate" concept from the Consumer and ProducerStreamManager classes, so ProducerStreamManager::MayChangeLayers() never applies its calculations anymore, it only asks the Transport for a new bitrate round. The logic becomes:

  • A PipeTransport and a DirectTransport never run BWE (IsCongestionControlSupported() returns false in both), so no BWE::SenderTransportCongestionController is ever built for them and nobody answers a bitrate round.
  • In a PipeTransport that is harmless, because PipeProducerStreamManager::RecalculateTargetLayers() always returns false, so such a Consumer never asks for a round in the first place and delivers all packets regardless. Note that a piped Consumer can also live on a non-pipe transport (consume({ pipe: true })), and there it is just traffic that cannot be moved, taken off the top of the available bitrate.
  • In a DirectTransport it is not harmless, and it is the open problem: a simulcast or SVC Consumer there uses a regular SimulcastProducerStreamManager or SvcProducerStreamManager, which do ask for rounds. With nobody to answer, their target layers stay at -1 and the Consumer forwards nothing. This needs the second half of the old flag back in some shape: not "who manages the bitrate" but "apply your own calculation when there is no one to ask".
  • In every other transport BWE always exists on the consuming side: Transport::MayCreateOrUpdateSenderTransportCongestionController() builds the BWE::SenderTransportCongestionController when the transport is connected and has at least one Consumer, and is called both from Transport::Connected() and when a Consumer is added. It is never built for a transport with nothing to send, since telling it that the network is available is what starts the RTP probing. Such a controller can work with transport-cc, with REMB, or with nothing but the loss reported in RTCP Receiver Reports. This was not true in the old BWE, where transport-cc or REMB was required for TransportCongestionControlClient to exist at all.
  • In those transports the receiving endpoint does not need to report anything for the stream/layer selection to keep working. When a Consumer asks for a round, Transport::OnConsumerNeedBitrateChange() runs DistributeAvailableOutgoingBitrate() there and then, with whatever target the controller holds, which with no feedback at all is still the initial one. What the endpoint reporting buys is the target moving: without transport-cc, REMB or Receiver Reports the target never leaves initialAvailableOutgoingBitrate.

TODO

  • A simulcast or SVC Consumer on a DirectTransport is now stuck: ProducerStreamManager::MayChangeLayers() only asks the Transport for a bitrate round and there is no BWE::SenderTransportCongestionController to answer it, so its target layers stay at -1 and it forwards nothing. The old externallyManagedBitrate flag meant two things, "there is no ceiling" and "apply your own calculation", and only the first one is covered. Note that a latched flag is not the right shape, since what matters is whether there is someone to answer right now, and Transport::IsCongestionControlSupported() is not the right predicate either, since it is true from a WebRtcTransport's constructor while the controller only appears later.
  • A PlainTransport whose receiving endpoint reports nothing (no transport-cc, no REMB, no RTCP Receiver Reports) keeps handing out initialAvailableOutgoingBitrate for ever, so its Consumers are capped at whatever fits in it. The old BWE had no ceiling at all there. Decide what such a transport should hand out.
  • ortc.ts and ortc.rs: Must revisit Dependency Descriptor RTP header extension for PipeTransport and DirectTransport.
  • In a pipe Consumer (and/or in its PipeProducerStreamManager) methods such as MayChangeLayers(), OnProducerStreamManagerNeedBitrateChange(), etc. should never be even invoked because it's a pipe Consumer so we want to deliver all RTP packets no matter what happens to the streams or layers of the Producer, and also because there is no SenderTransportCongestionController to deal with those events.
  • initialAvailableOutgoingBitrate option only exists in WebRtcTransportOptions. Why not in PlainTransport? NOTE: If we change this we also need to update the API doc in the website.
  • If we don't enable RTP probing when the peer doesn't support neither transport-cc nor REMB, then will things work fine when it comes to estimate downlink available bandwidth? Perhaps we should run probing anyway and, maybe, only check packet loss and so on?

ibc added 2 commits October 15, 2026 19:17
# Details

- Related to #1895.
- Integrate `SenderTransportCongestionController` in `Transport` under the `MS_USE_BUILTIN_BWE` macro condition as usual.
- Do not advertise support for the "abs-send-time" RTP extension in audio, since mediasoup never uses received audio packets for receiver-side BWE (REMB) and never includes this extension in outgoing audio packets, as other REMB estimators do not use it for audio either (such as libwebrtc).
- Advertise support for "transport-wide-cc-01" RTP extension also for sending in audio because we need it in the new BWE.
- Many changes in other clases (mandatory changes).
jmillan
jmillan previously approved these changes Oct 16, 2026
@jmillan
jmillan self-requested a review October 16, 2026 08:38
@ibc ibc mentioned this pull request Oct 16, 2026
39 of 42 tasks
@ibc
ibc dismissed jmillan’s stale review October 17, 2026 13:36

Infinite changes since the approval

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread worker/test/src/RTC/TestSvcProducerStreamManager.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants