The state of audio/sound libraries

[mooman219]

Quoting ryan.isaac.g:

Unfortunately the state of Rust audio is:

• The only pure-Rust solution is rodio / cpal
• cpal seems vaguely buggy in some scenarios
• rodio / cpal don't have a good maintainer-ship story
• cpal does not have working web support

It sounds like cpal needs some love.

[Ralith]

In particular, see RustAudio/cpal#272

[erlend-sh]

A good first step would be @tomaka agreeing to move rodio & cpal to RustAudio so it’s easier for the community to maintain, similar to what happened with winit: RustAudio/rodio#227

[Lokathor]

done

[mitchmindtree]

Just for reference, I submitted a post discussing the state of the RustAudio repos recently to the rustaudio discourse forum here. Specifically, here's the gist listing almost all crates within the organisation and my thoughts.

I'm very happy to have seen the RustAudio group take ownership over CPAL! I've been a maintainer at the repo for a while (merging PRs, have done a few major rewrites, originally added coreaudio support) and I've recently added ishitatsuyuki and now of course the rest of the rustaudio admins. The main feature work happening in CPAL right now is RustAudio/cpal#301 in order to address some API issues (see the related issue). The other major recent change is that CPAL now supports selecting between multiple Hosts at runtime RustAudio/cpal#289. This allowed for adding ASIO support on Windows and opens the doorway to adding support for non-native yet super-common hosts like PulseAudio and JACK. Otherwise a lot of UB has been fixed recently, though there are still some known cases to address.

I am a little less familiar with rodio, but it came up in some discussion last night - I'll post the relevant part of my thoughts on it here:

In general I feel that rodio seems to conflate a lot of roles that could be abstracted into separate crates. E.g. its approach to sinks, sources and mixers could probably better abstracted in a more generalised DSP graph crate. It's reverberation and spatial effects could also be abstracted into their own crates for more general use. The Sample type abstractions and conversions could be more generally supported by a lower-level crate like sample. It's for these reasons I've never taken a great deal of interest in rodio - it seems to me more like tomaka's all-in-one open source audio backend for games, whereas I personally am more interested in pro-audio, music playback, audio visual installations and performance, DAW development, etc and in turn am interested in more modular tools.

I realise this is of course a gamedev working group! That said, my hope is that we can find some way to consolidate it with the rest of the RustAudio ecosystem which is not only interested in stereo gamedev.

[kabergstrom]

My impression of rodio is that it uses the Rust type system in ways that makes it overly difficult to create a data-driven application, i.e. configure playback of audio in data instead of code.

I would absolutely welcome more unopinionated low-level crates that could be used to build applications that are not great fits for rodio currently.

[tomaka]

I would absolutely welcome more unopinionated low-level crates that could be used to build applications that are not great fits for rodio currently.

Can you explain how is rodio opinionated? It's basically a wrapper around cpal (which itself is designed to be unopinionated) with utilities to make it easy to use. The most opinionated thing in this stack is the operating system and the hardware, and we cannot change that.

[kabergstrom]

Can you explain how is rodio opinionated? It's basically a wrapper around cpal (which itself is designed to be unopinionated) with utilities to make it easy to use. The most opinionated thing in this stack is the operating system and the hardware, and we cannot change that.

I'll concede I haven't spent a ton of time with it, so I may have misunderstood things, but I'll try listing the biggest pain points I've discovered.

1. It's effectively impossible to compose mixing (source playback configuration) at runtime based on data since it requires chaining generic Rust types.
2. A Source that is Appended to a Sink (played on the device) is consumed, making it impossible to affect playback of an individual sound (you have to touch the Sink, but you don't know if your sound is playing or not). This goes against conventions of other industry standard audio libraries I have used, but I may be missing something here.
3. Mixing in a Sink doesn't seem to be possible? seems to be per Source. Usually you can attach a DSP chain to a channel (sort of like a Sink).

[tomaka]

A Source that is Appended to a Sink (played on the device) is consumed, making it impossible to affect playback of an individual sound (you have to touch the Sink, but you don't know if your sound is playing or not). This goes against conventions of other industry standard audio libraries I have used, but I may be missing something here.

It is indeed the Source that must be modified. A Source is simply a glorified iterator, and can be adjusted on the fly.

The primary way "things plug together" is Source and the play_raw function. The Sink struct has been added due to popular demand and just wraps around Source and play_raw.

The design of rodio is based on the fact that the operating system or hardware "just" wants data in its ring buffer. That data is the Source, and any modification you want to perform on the sound is done by adjusting the Source.

Mixing in a Sink doesn't seem to be possible? seems to be per Source. Usually you can attach a DSP chain to a channel (sort of like a Sink).

Sources can be merged together and a DSP applied to them.

I totally agree that lots of features are missing, but I don't think you can qualify the library of "opinionated".

[Lokathor]

SDL2 allows you to have an audio card mode where you just throw audio in a queue and then it'll just play the queue and if there's nothing in the queue it plays silence.

Does rodio / cpal directly do anything like that right now? Or would you have to write a QueueSource type or something?

[Ralith]

That's pretty much exactly what cpal is, minus a built-in queue.

[ethindp]

So, glad to see this working group (I didn't even know it existed, haha). Working on an OS in Rust following phil-opp's OS development tutorial -- and loving rust even more!
I know of a few more audio libraries that could be used for games; in particular, Alto comes to mind. Alto is just rust bindings for OpenAL, which perhaps is a good starting point (though CPAL definitely could become the "de facto audio library" for rust if enough work is put into it). I've used FMOD andBASS in the past, too; however, BASS has no Rust bindings and FMOD's rust dinbings are (last time I checked) horribly out of date. I've got a few more ideas in regards to other things the group to consider, so I'll be submitting issues regarding those. Good work so far though!

[Ralith]

Looks like cpal is making significant progress: the big API refactor to bring it in line with modern platform patterns just got merged!