Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TTS FastPitch Finetuning #2684

Merged
merged 22 commits into from
Aug 26, 2021
Merged

TTS FastPitch Finetuning #2684

merged 22 commits into from
Aug 26, 2021

Conversation

paarthneekhara
Copy link
Collaborator

No description provided.

blisc
blisc requested changes Aug 18, 2021
View reviewed changes
examples/tts/conf/fastpitch_align_44100.yaml Outdated Show resolved Hide resolved
examples/tts/conf/fastpitch_align_finetuning.yaml Outdated Show resolved Hide resolved
nemo/collections/asr/data/audio_to_text.py Outdated Show resolved Hide resolved
nemo/collections/tts/models/fastpitch.py Show resolved Hide resolved
nemo/collections/tts/models/fastpitch.py Outdated Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Outdated Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Outdated
"from nemo.collections.tts.models import HifiGanModel\n",
"from nemo.collections.tts.models import FastPitchModel\n",
"\n",
"hifigan_ckpt_path = \"/home/pneekhara/PreTrainedModels/HifiGan--val_loss=0.08-epoch=899.ckpt\"\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably also need to publish this model to NGC

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will need to do that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a placeholder for the FastPitch checkpoint as well, and add a comment that we plan on releasing .nemo files for this notebook soon.

tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
blisc
blisc reviewed Aug 19, 2021
View reviewed changes
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb
" plt.show()"
]
},
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would also be cool to show how people can fine tune their model on other data. Maybe even use their own voice after recording a few samples.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can move this to another PR

@blisc blisc mentioned this pull request Aug 19, 2021
Merged
@blisc blisc self-assigned this Aug 19, 2021
@paarthneekhara
Added speaker in FastPitch2 dataloader
1383553 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
changed fastpitch_align.yaml to have configuration for 441000 Hz
2d68e76 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
trying to fix g2p
fad6d3a 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
pytorch lightning version requirement
abff093 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
Cleanupfinetuning (#2)
0017e06 
* Added filelists and finetuning code

* image logging transoposed bug fix in FastPitch

* pytorch lightning version requirement

* added filelists for multispeaker hifigan

* hifigan 44100 Hz

* updated filelists, added another configuration of hifigan generator as per jason

* added notebooks and some scripts

* synthesize samples script update

* removed sample filelists

* reverted hifigan changes

* reverting hifigan.yaml update

Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
added finetuning notebook
f91a9e0 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
removed fastpitch2.py (redundant with fastpitch.py)
93c4379 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
restored old fastpitch_align.yaml, made just one finetuning yaml
715ad0c 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
reverting to old fastpitch align yaml
bd06780 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
reverted to old vocabs.py
b97b095 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
reverted to old vocabs.py
a751785 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
reverting requirements change
1b5291f 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
addressed pull request reviews -- updated notebook, speaker loading f…
68cb863 
…or fastpitch dataset, comments in configuration files

Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
notebook update
6d53329 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
removed redundant configuration file, updated notebook
d917be3 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
some more corrections for switching to single configuration file
322db0c 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
added url for hifigan dataset in notebook
bf78860 
Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
warning messages if optimizer configuration does not look correct whi…
8a6f8b0 
…le finetuning

Signed-off-by: Paarth Neekhara <[email protected]>
@paarthneekhara
style error fix
e9dfa32 
Signed-off-by: Paarth Neekhara <[email protected]>
@blisc blisc merged commit 0d7de7c into NVIDIA:main Aug 26, 2021
jfsantos pushed a commit to jfsantos/NeMo that referenced this pull request Nov 19, 2021
@paarthneekhara @blisc @jfsantos
TTS FastPitch Finetuning (NVIDIA#2684)
3e8aedd 
* Added speaker in FastPitch2 dataloader

Signed-off-by: Paarth Neekhara <[email protected]>

* changed fastpitch_align.yaml to have configuration for 441000 Hz

Signed-off-by: Paarth Neekhara <[email protected]>

* trying to fix g2p

Signed-off-by: Paarth Neekhara <[email protected]>

* pytorch lightning version requirement

Signed-off-by: Paarth Neekhara <[email protected]>

* Cleanupfinetuning (NVIDIA#2)

* Added filelists and finetuning code

* image logging transoposed bug fix in FastPitch

* pytorch lightning version requirement

* added filelists for multispeaker hifigan

* hifigan 44100 Hz

* updated filelists, added another configuration of hifigan generator as per jason

* added notebooks and some scripts

* synthesize samples script update

* removed sample filelists

* reverted hifigan changes

* reverting hifigan.yaml update

Signed-off-by: Paarth Neekhara <[email protected]>

* added finetuning notebook

Signed-off-by: Paarth Neekhara <[email protected]>

* removed fastpitch2.py (redundant with fastpitch.py)

Signed-off-by: Paarth Neekhara <[email protected]>

* restored old fastpitch_align.yaml, made just one finetuning yaml

Signed-off-by: Paarth Neekhara <[email protected]>

* reverting to old fastpitch align yaml

Signed-off-by: Paarth Neekhara <[email protected]>

* reverted to old vocabs.py

Signed-off-by: Paarth Neekhara <[email protected]>

* reverted to old vocabs.py

Signed-off-by: Paarth Neekhara <[email protected]>

* reverting requirements change

Signed-off-by: Paarth Neekhara <[email protected]>

* addressed pull request reviews -- updated notebook, speaker loading for fastpitch dataset, comments in configuration files

Signed-off-by: Paarth Neekhara <[email protected]>

* notebook update

Signed-off-by: Paarth Neekhara <[email protected]>

* removed redundant configuration file, updated notebook

Signed-off-by: Paarth Neekhara <[email protected]>

* some more corrections for switching to single configuration file

Signed-off-by: Paarth Neekhara <[email protected]>

* added url for hifigan dataset in notebook

Signed-off-by: Paarth Neekhara <[email protected]>

* warning messages if optimizer configuration does not look correct while finetuning

Signed-off-by: Paarth Neekhara <[email protected]>

* style error fix

Signed-off-by: Paarth Neekhara <[email protected]>

* dataloader fix after master merge

Signed-off-by: Paarth Neekhara <[email protected]>

Co-authored-by: Jason <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@blisc blisc blisc approved these changes

Assignees

@blisc blisc

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@paarthneekhara @blisc