Tacotron 2

[DarkDefender]

Sorry if this is off-topic (deepvoice vs tacotron) but it seems like the tacotron 2 paper is now released.
The speech samples sounds better than ever (I think):
https://google.github.io/tacotron/publications/tacotron2/index.html

I must admit that I'm not too well versed in how much this differs from the original tacotron. But perhaps the changes made also could be used in your projects?

[r9y9]

Paper (https://arxiv.org/abs/1712.05884) seems submitted to ICASSP 2018. I have read that today. It is very nice! I plan to implement WaveNet vocoder when I finish multi-speaker work (#10). DeepVoice3 and Tacotron2 both uses WaveNet vocoder.

[DarkDefender]

Great! I wish you luck with the multi-speak and vocoder work! :D

[r9y9]

I started to implement WaveNet vocoder. It's still quite WIP, but I think I implemented all basic features. If you are interested, check out https://github.com/r9y9/wavenet_vocoder. Audio samples from a model trained on CMU Arctic (16kHz, ~1200 utterances) can be found at r9y9/wavenet_vocoder#1 (comment).

[r9y9]

It turned out to be easy to implement WaveNet vocoder. I think my implementation is already feature complete. The problem is that I don't have 32 GPUs :(

[DarkDefender]

Yeah, it is really a bummer that the vocoder requires that much compute power to be able to train in a reasonable amount of time. :C

Perhaps you could try the WORLD vocoder method they used here? http://www.dtic.upf.edu/~mblaauw/NPSS/
However the quality will probably not be as good as the wavenet vocoder...

[DarkDefender]

@r9y9 I had a listen to https://r9y9.github.io/wavenet_vocoder/ and I think that they sound really quite good!

The samples are much better (to me at least) than the tacotron samples as they do not seem to have the same harsh "sound compression artifact" noise. They instead sound like they have lower quality microphones or recored on a lower quality analog tape. (I guess most of it has to do with the 16kHz sampling freq)

So anyways, what changed? Did you buy 32 GPUs or did I missunderstand that it is not the wavenet vocoder itself that needs that much compute power? (IE it is tacotron + wavenet vocoder that requires that much)

[r9y9]

@DarkDefender Nothing changed:) I just trained WaveNets with my single GPU (GTX 1080Ti) . As I noted in the demo page, it took 22 hours to train for the single speaker version and 44 hours for the multi-speaker version. I used 1 ~ 7 hours of audio sampled at 16kHz. For larger and higher rate sampled data, it will take more time to train.

[rraallvv]

@r9y9 Wow! The output audio in the samples files is very impressive. If you don't mind I'd like to ask a couple of questions. Recently I was browsing some repos that do style transfer with deepvoice, in particular this one does a nice job, have your tried that kind of think? Also, do you know of an already trained network that I can run locally or online in a Jupyter notebook to generate speech from text?

Keep up the good work!

[r9y9]

Hi, @rraallvv. I currently have lots of other things to do and have not tried it yet, sadly. WN for TTS is still WIP at #21. There is no pre-trained models at the moment.

[rafaelvalle]

We're very close of issuing a pull request with a implementation of Tacotron 2 that is compatible with @r9y9's repo.

[rraallvv]

@r9y9 thanks for your quick reply.

[rafaelvalle]

@r9y9 probably beginning of next week we'll issue a PR with Taco 2. Here's the attention and predicted mel after 7k iterations.

[r9y9]

@rafaelvalle Great! I cannot wait next week:)

[neverjoe]

hi @rafaelvalle, i'am working on Taco 2 two, can u explain how to reprodurce your result which looks working?