Frame access and manipulation

[Be-ing]

This crate looks nice, but it seems to be missing a way to iterate over the frames of a buffer, only the samples within one channel at a time. Sometimes it is necessary to iterate over frames to do operations relating the values of different channels, for example panning. Another use case is zipping an iterator over frames with the RampingValueIterator that I wrote for smoothly interpolating parameter changes over the frames of a buffer.

It would be possible to create an iterator in this crate to do this, but I think it would be better to use the dasp::signal::Signal iterator trait which already exists. Curiously, dasp doesn't have structs for holding audio data. It would be nice to be able to use the dasp Signal trait on the buffers in this crate which would also make it seamless to use the variety of other tools in dasp on the buffers. Perhaps it would make sense to merge this crate into a module of dasp as well. As a first step, I think it would be good to start by replacing this crate's Sample trait with dasp::Sample.

What are your thoughts on these proposals? My overarching goal is to have a common set of audio buffer structs and iterators used throughout the Rust audio ecosystem. Currently everyone is writing their own solution, which makes it cumbersome to pass audio data between crates and can require unnecessary copying.

[udoprog]

Yeah. My goal is to author a set of traits and data structures that can be used across audio libraries in the ecosystem. I'm currently holding off on GATs landing, since that's needed to provide proper abstractions without incurring a runtime overhead. GATs been making a lot of progress, but is still probably going to take a while. I never intended to propose converging on a solution until that's in place.

I'd probably be happy to incorporate more Frame-esque APIs here, and I'd even be open to provide this crate as a namespace as long as the API is reasonable and is something that audio crates actually want to use.

I think it would be good to start by replacing this crate's Sample trait with dasp::Sample.

I'm not so sure. dasp::Sample has more things than you need in an audio abstraction and puts an overly high burden on what can and cannot be a sample. It has conversions and manipulation functions which are better suited for traits exclusive to dasp. In my view each crate provide their own Sample trait to provide the functions that they need (e.g. rubato) and I believe the audio abstraction should only require the minimum necessary it needs for it to work since that will make it easier to stabilize.

But I'm open to debate this! I never intended to do this until audio was in a somewhat complete state though.

[Be-ing]

Looks like GATs will like stabilize for Rust 1.61 or 1.62.

[udoprog]

I am definitely looking forward to it!

[Be-ing]

Welp, guess it's going to be a while longer before GATs are stabilized :/ rust-lang/rust#96709

[Be-ing]

The GAT stabilization PR has been merged! 🚀

[udoprog]

I'm working on getting everything into shape for the stable release of GATs in about 6 weeks in #4.

I'll be opening an issue to gather feedback about what kind of APIs people want to see added for an initial 0.2 release of these audio abstractions. The first on those will be frame iteration API as was initially proposed in this issue.

Thank you!

[udoprog]

We have added some preliminary support for frame-oriented iteration in git now. It would be great if you could take a look and see if it corresponds to what you're looking for. No mutable iteration (yet) but that should be simple enough to add.

Even better if you could point me to a project that we can try and port ;)

[udoprog]

What's been added so far are the following:

Implementations for:

• audio::buf::Interleaved
• audio::buf::Sequential
• audio::wrap::Interleaved
• audio::wrap::Sequential

/// A buffer which has a unifom channel size.
pub trait UniformBuf: Buf {
    /// The type the channel assumes when coerced into a reference.
    type Frame<'this>: Frame<Sample = Self::Sample>
    where
        Self: 'this;

    /// A borrowing iterator over the channel.
    type FramesIter<'this>: Iterator<Item = Self::Frame<'this, Sample = Self::Sample>>
    where
        Self: 'this;

    /// Get a single frame at the given offset.
    ///
    /// # Examples
    ///
    /// ```
    /// use audio::{Frame, UniformBuf};
    ///
    /// fn test<B>(buf: B)
    /// where
    ///     B: UniformBuf<Sample = u32>,
    /// {
    ///     let frame = buf.get_frame(0).unwrap();
    ///     assert_eq!(frame.get(1), Some(5));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [1, 5]);
    ///
    ///     let frame = buf.get_frame(2).unwrap();
    ///     assert_eq!(frame.get(1), Some(7));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [3, 7]);
    ///
    ///     assert!(buf.get_frame(4).is_none());
    /// }
    ///
    /// test(audio::sequential![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::sequential([1, 2, 3, 4, 5, 6, 7, 8], 2));
    ///
    /// test(audio::interleaved![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::interleaved([1, 5, 2, 6, 3, 7, 4, 8], 2));
    /// ```
    fn get_frame(&self, frame: usize) -> Option<Self::Frame<'_>>;

    /// Construct an iterator over all the frames in the audio buffer.
    ///
    /// # Examples
    ///
    /// ```
    /// use audio::{Frame, UniformBuf};
    ///
    /// fn test<B>(buf: B)
    /// where
    ///     B: UniformBuf<Sample = u32>,
    /// {
    ///     let mut it = buf.iter_frames();
    ///
    ///     let frame = it.next().unwrap();
    ///     assert_eq!(frame.get(1), Some(5));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [1, 5]);
    ///
    ///     assert!(it.next().is_some());
    ///
    ///     let frame = it.next().unwrap();
    ///     assert_eq!(frame.get(1), Some(7));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [3, 7]);
    ///
    ///     assert!(it.next().is_some());
    ///     assert!(it.next().is_none());
    /// }
    ///
    /// test(audio::sequential![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::interleaved([1, 5, 2, 6, 3, 7, 4, 8], 2));
    ///
    /// test(audio::interleaved![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::sequential([1, 2, 3, 4, 5, 6, 7, 8], 2));
    /// ```
    fn iter_frames(&self) -> Self::FramesIter<'_>;
}

/// The buffer of a single frame.
pub trait Frame {
    /// The sample of a channel.
    type Sample: Copy;

    /// The type the frame assumes when coerced into a reference.
    type Frame<'this>: Frame<Sample = Self::Sample>
    where
        Self: 'this;

    /// A borrowing iterator over the channel.
    type Iter<'this>: Iterator<Item = Self::Sample>
    where
        Self: 'this;

    /// Reborrow the current frame as a reference.
    fn as_frame(&self) -> Self::Frame<'_>;

    /// Get the length which indicates number of samples in the current frame.
    fn len(&self) -> usize;

    /// Test if the current frame is empty.
    fn is_empty(&self) -> bool {
        self.len() == 0
    }

    /// Get the sample at the given channel in the frame.
    fn get(&self, channel: usize) -> Option<Self::Sample>;

    /// Construct an iterator over the frame.
    fn iter(&self) -> Self::Iter<'_>;
}

Some of the documentation needs to be fixed up since in quite a few places I believe we refer to sample as a frame. The terminology also needs to be better documented overall (but that is on my TODO).

[Be-ing]

Rust 1.65 has been released with GATs on stable!! 🚀