Cast Audio with Video when screen sharing using WebRTC in Electron | Drivers to pipe audio output as an audio input source that can be read by ElectronJS WebRTC and streamed to client along with video

[pavlobu]

Hi all 👋 The feature to cast audio output stream from computer along video stream -> to client viewing device was requested multiple times by users. It has a long way to go as well as virtual display drivers solution. So we need to know how to tweak os a system audio output as a virtual input device (virtual microphone).

Why do we need to make it like that? Because electron WebRTC has already a nice interface of working with microphones and other audio input devices. We just need to find ways how to make system audio output as its own separate "microphone" or virtual audio input device, so then system built in audio output can be easily shared with WebRTC stream. And it must work the same way on all operating systems.

Source code, project links everything you know or find will help us to implement this feature in future releases of Deskreen.

[pavlobu]

Seems soundflower for macos can do this! We can be thinking on embedding this package to Deskreen macos version.
https://github.com/mattingalls/Soundflower
Haven't tried that myself yet.

[wastee]

I notice WebRTC AudioProcessing in pulseaudio project.
Maybe pipewire will replace pulseaudio in long-term future. But now, most peopel on Linux using pulseaudio.

I'm a Archlinux user, but i don't know what it is and i'm not a developer.
I just think this infomation will be helpful.

[alectrocute]

Existential Audio’s BlackHole can do this for macOS and is a reliable driver. Easily bundled in an Electron app and installed via shell scripts.

[Reaverblade]

Discord music bots regularly capture and pipe audio over the internet from apps and desktop screens.

I wonder if you can look at some of the code for those and see how they implement this

[DominikNovosel]

Found another Electron app that seemingly implements desktop audio capture: daltonmenezes/electron-screen-recorder#2 (comment)
Hope this helps!

[DominikNovosel]

I also found this in the Electron docs: https://www.electronjs.org/docs/latest/api/desktop-capturer

[SeanRamey]

Has there been any progress on this? I'd like to use Deskreen to replace Discord video streaming on Linux, but without audio streaming, it's just as lacking as Discord on Linux is, so there's not much of a point. Other than this, it works great! Even online, all I have to do is port forward Deskreen in my router, allow that port through the firewall, and give them my public IP address instead of my local IP, and my friends can nicely watch my gameplay.

[cipitaua]

practically, audio is necessary for full functionality

[andzejsp]

Audio and reverse controls. Similar to weylus. Only weylus is super choppy, while deskreen is silky smooth. If these two are implemented here, then its pretty much RDP replacement. Tho i really wish we could have 60fps streaming, if the network allows it.

[marc1uk]

Piping audio to webrtc applications in linux is pretty standard and straightforward. There are several ways to do it (such as making a loopback and an echo cancellation source), but the simplest I belive is simply to make a remap source.

To do this with pulseaudio or pipewire-pulse the command would be:

pactl load-module module-remap-source source_name=Remap-Source master=<your_default_output>.monitor

You can find the name of your default output device with pactl info | grep 'Default Sink':

$ pactl info | grep 'Default Sink'
Default Sink: alsa_output.pci-0000_00_1f.3.analog-stereo

That should give you a new virtual "input device" which is listening to your default audio output and can be selected as the "microphone" by normal webrtc applications such as discord or zoom.

Having done this, however, I can't see any way to point deskreen at that virtual microphone.... or any kind of audio source...