Loudness normalization with pyloudnorm

[desh2608]

Related to #966.

This PR adds a method normalize_loudness() for recordings. This takes an argument target which specifies the desired loudness (usually around -23 dB is a good value). The implementation uses pyloudnorm.

Also, we move ReverbWithImpulseResponse out of the torchaudio.py and into its own file, since it is not a torchaudio based transform.

[pzelasko]

If audio clipping is an issue here you can add a limiter as a post-processing step, e.g. https://github.com/pzelasko/cylimiter

[desh2608]

Sure, I can add it. But I don't have enough familiarity with limiters (or "loudness" for that matter) to know what to do exactly.

[pzelasko]

The defaults should "just work" with pretty much anything, it basically keeps track of the signal's loudness with a small lookahead and reduces the gain if it crosses some threshold. Think of it as soft clipping that doesn't introduce as much distortion as hard clipping.

[pzelasko]

.. but again I don't know if that's a real problem with this approach and worth the extra dependency, so it's your call :)

[desh2608]

So far I have only used it to make the LibriCSS distant mic audio louder (it is at around -52 dB originally), and it sounds okay even with the clipping. I suppose we can let it be for now and add the limiter later if someone needs it?

[pzelasko]

Looks good to me, I left a single comment, up to you :)

[lifeiteng]

@pzelasko should we make sampling_rate as a member?

class AudioTransform:
    def __call__(self, samples: np.ndarray, sampling_rate: int) -> np.ndarray:
        """
        Apply transform.

        To be implemented in derived classes.
        """
        raise NotImplementedError

[lifeiteng]

sampling_rate is missed

[lifeiteng]

@desh2608 @pzelasko

I used set.normalize_loudness as blow: https://github.com/lifeiteng/vall-e/blob/main/valle/bin/tokenizer.py#L173

if args.prefix == "aishell":
    # NOTE: the loudness of aishell audio files is around -33
    # The best way is datamodule --on-the-fly-feats --enable-audio-aug
    cut_set = cut_set.normalize_loudness(
        target=-20.0, affix_id=True
    )

cut_set = cut_set.resample(24000)

But model's accuracy drops a lot.

[desh2608]

@lifeiteng what metric is this curve? If it is validation accuracy, did you normalize loudness for both train and validation sets? I would think that for the training set, it may be better to "perturb" the loudness (within some range) rather than normalize it, similar to how we perturb volume.

[lifeiteng]

@desh2608 Training CrossEntropy Top10Accuracy.
yes, normalized loudness for both train and validation sets.
It's used in text-to-speech, yes "perturb" is better, but I want to make it simple at current stage.
I don't understand why normalization leads to a significant drop in accuracy. The bug(comments above) is not triggered.

[pzelasko]

@lifeiteng If it's for TTS, can you listen to the train and dev examples before and after normalization, as well to the model predictions? Maybe that could reveal if there's something funky going on.