reducing audio packet rate while track is disabled

[fippo]

spawned from webrtc/samples#1009

http://w3c.github.io/webrtc-pc/#dom-rtcrtpsender-track talks about about reducing the frame rate for muted video.

Is there any reason why the same doesn't apply to audio? DTX reduces the packet rate drastically and while the audio is muted (by the user) there is a clear indication that we don't need to encode audio.

[alvestrand]

The "one black frame per second" language was added very recently; discussion in the group revealed that some people felt uneasy about some implementations just sending a black frame and then no more video data for a long time, while others felt that sending at full frame rate was just silly. This got added both to advise senders on what to do and to advise receivers on what they can expect.

I don't think audio codecs have the idea of "frame rate" (they use "sample rate", which is something quite different), so the same language can't be used. Suggestions for language that is reasonable for all audio codecs?

[fippo]

https://tools.ietf.org/html/rfc6716#section-2.1.9 actually uses frame.
Language is really tricky here since opus-DTX should be recommended by default as it can have a negative impact on voice quality - not if muted though.

I would very much like to avoid having to use replaceTrack just to reduce the packet rate (but I will if I have to)

[alvestrand]

We'd like to have people use track.enabled = false for mute, and that codecs do the Right Thing to reduce packet rate when the track is silent. @fluffy may have comments here too.

[alvestrand]

RFC 3389 about comfort noise:

The CN packet update rate is left implementation specific. For
example, the CN packet may be sent periodically or only when there is
a significant change in the background noise characteristics. The
CNG algorithm at the receiver uses the information in the CN payload
to update its noise generation model and then produce an appropriate
amount of comfort noise.

[aboba]

@fippo Can you respond?

[nils-ohlmeier]

I think the major difference between video and audio here are that video (usually) keeps state on the screen. So if you stop sending video out of the blue, the user would see a frozen picture. That why it makes more sense to replace that information with intended black.

But with audio there is not confusing last information rendered until replaced. Thus I would actually suggest to simply send nothing. Because in my opinion comfort noise is something which is not supported by all audio codecs and usually needs to be negotiated on top. Thus suggesting CN or DTX would make this features required to be supported.

[aboba]

CN is mandatory-to-implement in WebRTC, and we have RTCRtpEncodingParameters.dtx so there are multiple ways to control whether discontinuous transmission is enabled or not. This seems like a more straightforward way to control the bitrate than relying on side-effects of setting sender.track.enabled to false.

[fippo]

DTX has to do some magic to detect silence and this was recently quite broken in Chrome, delaying speech for 1+ second. Giving a clear indication that the track was muted might help the encoder here.

Arguably that is an implementation detail but so is reducing the frame rate for muted video.

[stefhak]

@fippo can you propose (perhaps in form of a PR) language?

I must admit I'm not convinced we need to do anything more here.

[aboba]

Currently the text in Section 5.2 says that the "RTCRtpSender MUST send silence". This would seem to imply that the DTX setting or CN negotiation state has no effect.