How do we scale tokens during inference time for silence prolongation

[JRMeyer]

>>> georroussos
[July 16, 2020, 2:51pm]

I am trying training using GST and it works very well. I find it makes
the speech much more expressive and it can learn a few different styles;
however, I would like to make the pauses in between sentences longer and
even though my dataset has the exact same duration of pause mid periods,
I haven't still figured out how to do it, as mentioned here
mozilla/TTS#167. My idea was that I could
try multiplying the tensors by a value, but nothing really worked.
Ideally, the style would stay the same, but the speech and pause would
change. My point is that I do not really know how to disentangle these.

[This is an archived TTS discussion thread from discourse.mozilla.org/t/how-do-we-scale-tokens-during-inference-time-for-silence-prolongation]

[JRMeyer]

>>> sanjaesc
[July 17, 2020, 9:47am]

If you want to condition on tokens, you will need to query your gst
layer for the specific token, which you can do as follows.

init the query: gst_embedding_dim = embedding dim of your gst layer
(like 256 or 512)

> query = torch.zeros(1, 1, self.gst_embedding_dim//2).to(device)

get your style tokens:

> slash _GST = torch.tanh(self.gst_layer.style_token_layer.style_tokens)

get the key for a specific token where TOKEN is some int value

> key = slash _GST slash [TOKEN slash ].unsqueeze(0).expand(1, -1, -1)

now you can query your gst attention layer for the specified key:

> gst_outputs_att = self.gst_layer.style_token_layer.attention(query,
> key)

this gst_outputs_att slash [tensor slash ] can now be modified by multiplying:

> gst_outputs = gst_outputs_att slash curl-run-all.sh discourse.mozilla.org html-to-markdown.sh ordered-posts ordered-posts~ TTS.cdx tts.commands tts-emails.txt TTS.pages tts-telegram.txt TTS.warc.gz 0.15 [ slash #example]

from here you compute it just as if you were to condition on a wav file.

Hope this helps.

Check this commit
mozilla/TTS@c5eaf12
for an example.

[Archived Post]