Determining what learning rate to use

[JRMeyer]

>>> 20richardh
[May 20, 2019, 4:32am]

Hi,

Let's say I'm trying to use transfer learning to calibrate a model to a
specific person's voice. However, I only have a total of 20 people to
train my model off of, so I want to train more generally for them and
then calibrate my model to work specifically for this one person.

What would be a good learning rate for the general 20-person population
and the specific person I'm calibrating to?

I see that the default rate is 0.001 for DeepSpeech.py.

[This is an archived TTS discussion thread from discourse.mozilla.org/t/determining-what-learning-rate-to-use]

[JRMeyer]

>>> kdavis
[May 20, 2019, 8:24am]

I'd first try 0.0001 for everything.

[Archived Post]

[JRMeyer]

>>> 20richardh
[May 20, 2019, 4:27pm]

What if I did 0.001 for the general population? Do you happen to know if
I'd still get a fair loss on that?

I have 30,000 total files; 1 word each file; 455 different words

[Archived Post]

[JRMeyer]

>>> kdavis
[May 20, 2019, 5:51pm]

I'd try 0.0001 for everything.

[Archived Post]

[JRMeyer]

>>> 20richardh
[May 20, 2019, 8:52pm]

Ok cool. I'm going to go for that for a general model anyway, so I might
as well try that.

[Archived Post]

[JRMeyer]

>>> pete
[May 27, 2019, 8:35am]

Check validation loss vs. training loss. Closer they are the better. If
so, you have found good learning rate, which is not overfitting or
underfitting the model...

[Archived Post]