Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrete token-based TTS implementation #5626

Merged
merged 84 commits into from
Apr 22, 2024
Merged

Discrete token-based TTS implementation #5626

merged 84 commits into from
Apr 22, 2024

Conversation

ftshijt
Copy link
Collaborator

@ftshijt ftshijt commented Jan 18, 2024

What?

The new task of discrete token-based TTS implementation (mostly focuses on acoustic modeling, by changing spectrogram to discrete token as output prediction)

TODO

  • Integrate monotonic attention implementation (verified but doing code cleaning)
  • Add more CI

@ftshijt ftshijt added Recipe New Features TTS Text-to-speech and removed README labels Jan 18, 2024
@ftshijt ftshijt added this to the v.202312 milestone Jan 18, 2024
@ftshijt ftshijt self-assigned this Jan 18, 2024
@mergify mergify bot added the Installation label Apr 14, 2024
@ftshijt
Copy link
Collaborator Author

ftshijt commented Apr 17, 2024

@sw005320 @simpleoier @kan-bayashi Hi guys, many thanks for your review! I've reflected on your comments and also added some unit tests for the functions added for tts2.

We will have three following PRs after this one (would want to keep them separated to avoid a very large PR)

  • tacotron2-based discrete tts2 model
  • integration test with tacotron2 (we would want to follow the existing TTS setup with tacotron2)
  • alignment-free fastspeech2 with VAE

It would be great if you can double-check the existing PR so that we can merge it in the early place~

@sw005320 sw005320 merged commit b9ba189 into master Apr 22, 2024
35 checks passed
@sw005320 sw005320 deleted the tts2 branch April 22, 2024 21:12
@sw005320
Copy link
Contributor

Thanks for your great efforts, @ftshijt!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants