Integrating new synthesizer

[fazlekarim]

Hi,

If I wanted to integrate a new synthesizer, (https://github.com/syang1993/gst-tacotron/), what would be the steps I would need to take?

[CorentinJ]

You're free to use that implementation or fatchord's Tacotron would work too, that is up to you.

First, you'll need to understand how Tacotron is modified to allow for voice conditioning. Refer to section 2 of SV2TTS. You can also check my code.

Then you need to ensure that you will use the same format of spectrogram and audio for the synthesizer and for the vocoder. Check the preprocessing scripts to see the what it done to the data. You will likely have to change the data loading routine of the vocoder so that it takes correct inputs.

The scripts that use the synthesizer are synthesizer_train.py and vocoder_preprocess.py. Ensure that your model correctly interfaces them. You have to provide an interface for inference as well.

And you will have to train all this, which is not a minor task! Good luck.