Add transforms attribute for MixedCut

[desh2608]

Currently, for MixedCut, any transforms are always lazily applied on the underlying tracks. This is fine for transforms such as speed perturbation, but may not be ideal for other kinds of transforms. Consider the following examples:

1. We have a MixedCut consisting of 3 tracks: 2 MonoCuts and a PaddingCut, i.e., it represents 2 utterances from a speaker with some silence in between. Suppose we want to convolve it with an RIR. In the current method, the tracks will be individually convolved, and the silence portion will not contain any residue from the previous MonoCut. Instead, we should first mix the 3 tracks and then apply the RIR on the resulting audio.

2. We have a MixedCut comprising 2 MonoCuts with some overlap, and we want to perform loudness normalization. Currently, the tracks will be normalized individually and then mixed, and the resulting mixture may not have the target loudness.

To solve these problems, we introduce a transforms attribute for MixedCut. It is analogous to the transforms attribute in Recording, and contains the transforms that should be lazily applied at the time of audio loading, but after the underlying tracks have been mixed.

[pzelasko]

Your description here got me thinking -- does it really matter in which order we apply RIR and downmix? Casting aside the numerical differences, RIR reverb is a convolution, which is linear, so the sum of reverbed signals should be equal to the reverbed sum of signals, shouldn't it? Am I missing something here?

[pzelasko]

Ohh I missed the effect on track edges -- the signals are possibly not the same length and have different offsets, and before the PR we only pad them after (not before) the convolution. You mentioned this in the PR description.

[desh2608]

Yeah I was mainly concerned about edge effects in the case of reverb.

[pzelasko]

Cool contribution! One question: what happens if we mix another track after the reverb/loudnorm is applied? It looks to me like the behaviour wouldn't be correct (it would reverb/norm everything including the track mixed after).

[desh2608]

Cool contribution! One question: what happens if we mix another track after the reverb/loudnorm is applied? It looks to me like the behaviour wouldn't be correct (it would reverb/norm everything including the track mixed after).

Yeah, I think perhaps the right approach is to not allow mixing in new tracks once any of the mix_first transforms have been applied.

[pzelasko]

Yeah, I think perhaps the right approach is to not allow mixing in new tracks once any of the mix_first transforms have been applied.

Would it work if we added a MixedCut as a MixTrack when it has a transform defined on it? I think that technically mixing code only calls load_audio so it might work as expected..

EDIT there might be code all around the place though that doesn't expect MixedCut to be present in a track. Could you check if it's possible to make that work? It would be a pity to raise an exception here.

[desh2608]

Actually yeah we do allow MixedCut to be present as a MixTrack. In fact, the meeting simulation workflow explicitly uses this flexibility.

So you're right, if a new track is mixed into a MixedCut containing a transform, we can just create a nested MixedCut instead of adding to the tracks.

[pzelasko]

LGTM

[desh2608]

LGTM

Cool, thanks. Merging.