Skip to content

[Enhancement]Extend SimulcastEncoderAdapter to support adaptive streaming#75

Open
ipavlidakis wants to merge 3 commits intodevelopfrom
iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive
Open

[Enhancement]Extend SimulcastEncoderAdapter to support adaptive streaming#75
ipavlidakis wants to merge 3 commits intodevelopfrom
iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive

Conversation

@ipavlidakis
Copy link
Copy Markdown
Contributor

Summary

This fixes a runtime SimulcastEncoderAdapter edge case where SEA has multiple
encoder contexts, but only one spatial layer is actually active.

In that state, SEA was returning aggregated simulcast EncoderInfo instead of
the active encoder's runtime adaptation hints. The result is that a
single-active-layer call can behave differently from true single-layer
publishing, especially for quality scaling and other adaptation decisions.

Problem

SimulcastEncoderAdapter::GetEncoderInfo() only had a passthrough behavior
when stream_contexts_.size() == 1.

In practice, SEA can still have multiple stream contexts after init while
runtime bitrate allocation pauses all but one layer.

When that happened, SEA continued to report aggregated simulcast info and
explicitly disabled scaling_settings.

That means downstream components do not see the active encoder's real runtime
capabilities and hints, including:

  • quality scaling thresholds
  • min_pixels_per_frame
  • is_qp_trusted
  • resolution_bitrate_limits
  • min_qp
  • supports_native_handle
  • preferred_pixel_formats

Root Cause

The bug is in SimulcastEncoderAdapter::GetEncoderInfo().

The old logic treated "single stream context" as equivalent to "single active
layer", but those are not the same thing at runtime.

Once SEA enters multi-encoder mode, the number of stream contexts may stay
greater than one even if only one layer is currently unpaused.

Fix

In multi-encoder mode, SEA still aggregates the simulcast-specific fields as it
did before.

If exactly one stream is unpaused, SEA now forwards the active encoder's
runtime-sensitive fields:

  • scaling_settings
  • supports_native_handle
  • has_trusted_rate_controller
  • is_hardware_accelerated
  • is_qp_trusted
  • resolution_bitrate_limits
  • min_qp
  • preferred_pixel_formats

SEA still preserves its aggregated simulcast fields:

  • implementation_name
  • fps_allocation
  • requested_resolution_alignment
  • apply_alignment_to_all_simulcast_layers

If more layers become active again later in the call, SEA goes back to the
existing aggregated behavior without needing re-init.

Why This Is Safe

This is a narrow behavior change.

It does not change:

  • init-time encoder selection
  • SEA bypass mode
  • the pre-init GetEncoderInfo() path
  • bitrate allocation / pause-unpause logic
  • true multi-layer simulcast aggregation

It only changes the runtime case where SEA is effectively behaving like
single-layer publishing but was still reporting multi-layer aggregated info.

Tests

Added regression coverage in simulcast_encoder_adapter_unittest.cc for:

  • forwarding active encoder info when only one spatial layer is unpaused
  • restoring aggregated SEA behavior when a second layer becomes active again

The tests verify forwarding of:

  • scaling thresholds
  • min_pixels_per_frame
  • native-handle support
  • QP trust
  • resolution bitrate limits
  • min_qp

The tests also verify that fps_allocation still preserves SEA's spatial-slot
layout.

Validation

Validated by compiling the two edited translation units with the generated
build commands:

  • media/engine/simulcast_encoder_adapter.cc
  • media/engine/simulcast_encoder_adapter_unittest.cc

I did not complete:

  • a full rtc_media_unittests run
  • the iOS H.264 runtime repro validation in this PR

Reviewer Notes

Please pay special attention to:

  • whether forwarding supports_native_handle in the single-unpaused case is
    the right runtime behavior
  • whether there are any consumers of aggregated is_hardware_accelerated or
    has_trusted_rate_controller that should remain aggregated even in the
    single-unpaused state
  • whether we want a follow-up integration test around VideoStreamEncoder
    re-reading EncoderInfo after runtime pause/unpause transitions

@ipavlidakis ipavlidakis self-assigned this Mar 13, 2026
@ipavlidakis ipavlidakis added the enhancement New feature or request label Mar 13, 2026
@ipavlidakis ipavlidakis changed the base branch from main to develop March 13, 2026 11:47
@ipavlidakis ipavlidakis changed the title [Enhancement]Extend SimulcastEncoderAdapter to support adaptive strming [Enhancement]Extend SimulcastEncoderAdapter to support adaptive streaming Mar 30, 2026
ipavlidakis and others added 2 commits March 30, 2026 17:02
…encoder stall handling in RTCVideoEncoderH264 and VideoSendStreamImpl

- Implemented GetMuted method in FakeAudioSendStream to align with AudioSendStream interface.
- Introduced encoder stall detection and recovery logic in RTCVideoEncoderH264, including timestamps for successful encodes and forced resets.
- Enhanced VideoSendStreamImpl to track encoder timeouts and recovery events for better telemetry and operational insights.

# Conflicts:
#	media/engine/fake_webrtc_call.h
@ipavlidakis ipavlidakis force-pushed the iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive branch from b181ae4 to 5008872 Compare March 30, 2026 14:02
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 30, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 20f104d8-ea74-4e04-8275-fe2248907d75

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant