Stress on the vowel in Russian language.

[AngryBearr]

Self Checks

• I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find any relevant information that meets my needs. English 中文 日本語 Portuguese (Brazil)
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell us your story.

I love the model and your work overall, but there is an issue for the Russian language (I guess because of the lack of data model was trained). It's not an issue with the model itself, but some Russian language related problem.
There is a thing as a stress for the vowel. It can change the word in different ways if stress is on one vowel or another.

2. What is your suggested solution?

I would suggest to maybe train a model in some way with a special character before the vowel to add stress for it. In the long run someone can create a vocabulary and add words to it and have model read the vocabulary before the initialization and pronounce added words as they are in the vocabulary.
I know that this is just for one language and maybe it's too much work to be done, but maybe you can guide someone with more knowledge then me, to write the code for finetuning the LLAMA model to add that special character.
I can propose to look at some other model that was trained based on GPT-Sovits for Russian with addition of that stress symbol.
Here is a link to the repo: vosk-tts. Here is the model from HF. You put the model into the folder vosk-tts/gpt-sovitst/pretrained_models. You can also find mentioned dictionary on HF in the dictionary it contains the vocabulary with stress described in the 1 if stress is applied to a vowel and 0 if vowel should be read as is.

Hope that makes sense.

3. Additional context or comments

Maybe there is a way to add stress to vowels that I don't know. What I have tried so far to apply at least some stress to vowels in words that model interprets incorrectly:

1. Add another vowel (eg. if it letter o then add another o).
2. Add a grave accent symbol before a vowel (eg. `o)
3. Add a special char that we use in Russian literature when learning the language and how to read words it is called acute accent (eg. o´)
4. Use the capital letter if I need a stress for a vowel (eg. sOme wOrd)
5. Add a duplicate of letter and add a hyphen (eg. so-ome wo-ord)

4. Can you help us with this feature?

• I am interested in contributing to this feature.

[PoTaTo-Mika]

Thanks for your advice!

[cod3r0k]

Hi @AngryBearr , did it well for any language?

[cod3r0k]

Fishspeech is better than vosk tts isnt it? @AngryBearr

[AngryBearr]

Hi @AngryBearr , did it well for any language?

Hey. What do you mean? I have tested it only for russian. I know the model does well in other languages, but still I am interested in a particular one at the moment.
I have done a few more experiments and had no success, although I could sway the model to properly pronounce words with stress on the letter 'o'. I did it by changing the russian letter 'o' with the english letter 'o' 😄

Other then that I should have mention the great model by Den4ikAI called RuAccent which does the heavy lifting of putting the stress on the correct vowel in the russian words. He has made a lot of work for other russian models as they all need to have the stress predefined or they will put it in a random place in the word based on the model knowledge.

[AngryBearr]

Fishspeech is better than vosk tts isnt it? @AngryBearr

In some way it is better, but right now I stick to the Vosk Sovits implementation as it does a great job. I'd say it is better at voice clone, that is amazing at what Fish speech can do. And it adds some naturalness to the speech and I would use it if it didn't have the problems with the vowel stress.

[VitaliyAT]

Прямо в тексте используй буквы с ударением и будет ударение куда надо
Á á Ó ó É é ý и́ ы́ э́ ю́ я́

[AngryBearr]

Прямо в тексте используй буквы с ударением и будет ударение куда надо Á á Ó ó É é ý и́ ы́ э́ ю́ я́

If that would work, this discussion/request wouldn't be here in the first place, so the acute symbol does not work.