Repository navigation
Add FlexFEC negotiation and subscriber FEC protection - #1071
cloudwebrtc wants to merge 3 commits into
Conversation
LiveKit.enableFlexFec turns on the FlexFEC field trials when libwebrtc is initialized, so flexfec-03 is negotiated. Codec preferences already keep unmatched codecs, so the repair codec is offered without further changes. RemoteTrackPublication.setFecProtection requests FlexFEC repair packets from the server for a subscribed video track through the new UpdateTrackSettings.fec field, which requires livekit/protocol#1987. FecProtection is a sealed class mirroring the protocol's modes: Disabled, Adaptive, and Fixed and Minimum, which carry a FecProtectionLevel (low, medium or high). The protection is only sent once requested, so other settings updates leave the server's setting unchanged. Unlike quality and dimensions it also applies while adaptive stream manages the track. FlexFecOnDeviceTest subscribes to a video track on a real server and checks the repairs received for each mode through the inbound-rtp FEC counters. It is skipped unless the lkUrl and lkToken instrumentation arguments are given. The protocol submodule points at livekit/protocol#1987 applied on top of the current protocol commit.
🦋 Changeset detectedLatest commit: 8a1a900 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
End-to-end test resultsRan Setup
Server config: rtc:
flexfec:
upstream_enabled: true
downstream_enabled: trueRun with: The server's own FlexFEC tests ( ResultsEach row is a 6 s window, after a 2 s settle following
An earlier run gave the same picture (LOW 17.1%, HIGH 25.1%, Minimum MEDIUM 18.6%, Adaptive and Disabled 0). What this shows:
Note on the ratios
libwebrtc versionWebRTC Android Not coveredNo loss was simulated, so recovery of lost packets and adaptive protection rising with loss were not exercised on device. The server's Upstream (publish) resultsRan Negotiation
libwebrtc only protects a single media stream with FlexFEC ( With Under lossUpstream loss was added with a userspace UDP proxy between the device and the server that drops only RTP. With 5% and 20% upstream loss the server's While doing this I found that the server's receiver reports to the publisher always carried
Run with: |
|
Diffuse output: AARJAR |
libwebrtc only protects a single media stream with FlexFEC, so it signals no FlexFEC SSRC for a sender with simulcast layers.
FlexFecPublishOnDeviceTest publishes a synthetic video track to a real server and checks the publisher negotiation: without simulcast the offer includes flexfec-03 and a FEC-FR SSRC and the server accepts flexfec-03, and with simulcast no FlexFEC SSRC is signaled. The track can be held for a while to watch the repairs arrive at the server under upstream loss. It is skipped unless the lkUrl and lkPublisherToken instrumentation arguments are given.
Note
Draft until livekit/protocol#1987 is merged. The
protocolsubmodule currently points atduan/fec-subscribe-presets-android(3a115c89), which is #1987 cherry-picked onto the protocol commitmainpins (30d320b7). Once #1987 lands, the submodule should be moved to the merged commit.Adds FlexFEC negotiation and per-track subscriber FEC protection.
FlexFEC negotiation
LiveKit.enableFlexFecturns on theWebRTC-FlexFEC-03-AdvertisedandWebRTC-FlexFEC-03field trials when libwebrtc is initialized, soflexfec-03is negotiated. It is process-wide and must be set before the firstRoomis created. Disabled by default.Subscriber FEC protection
RemoteTrackPublication.setFecProtectionasks the server to send FlexFEC repair packets for a subscribed video track throughUpdateTrackSettings.fec:FecProtectionis a sealed class mirroring the protocol modes:FecProtection.Disabled(default)FEC_MODE_DISABLEDFecProtection.AdaptiveFEC_MODE_ADAPTIVEFecProtection.Fixed(level)FEC_MODE_FIXED+ levelFecProtection.Minimum(level)FEC_MODE_MINIMUM+ levelFecProtectionLevelisLOW/MEDIUM/HIGH(about 15/25/35% repair overhead). Only the modes that use a level carry one, so invalid combinations cannot be expressed.fecis only sent aftersetFecProtectionhas been called. That follows theoptionalsemantics in #1987: omitting the field keeps the server's setting, so other settings updates do not reset it.Tests
RemoteTrackPublicationTest:fecis omitted until requested, it is kept on later updates, every mode and level maps to the right proto values, and it is sent under adaptive stream.FlexFecOnDeviceTest(androidTest): subscribes on a real server and checks the repairs received in each mode through the inbound-rtp FEC counters. It is skipped unless thelkUrlandlkTokeninstrumentation arguments are given. On-device results are in the comment below.