Word stress is incorrectly placed in Russian

[sulfur17]

I tried to process a small text of two sentences in Russian and three words were pronounced with the wrong stress. Maybe I can somehow put the stresses myself?

[sulfur17]

I left all the parameters except the language as default

[DrewThomasson]

What do you mean by "wrong stress"

I'm unfamiliar with this term

[ROBERT-MCDOWELL]

upgrade to version v25.3.21 and see if it's better. explain "stress" please.
also pronounciation is a model concern, so if you think you can improve the model you are free to train a new model fixing imperfections.

[sulfur17]

wrong stress is the emphasis of one of the syllables in a word. For example in word "example" stress on A: "exAmple" or "exa-a-ample". And wrong stress could be "E-e-example"

[ROBERT-MCDOWELL]

this is a model issue, not code issue. if you want to improve the model you must learn how to train a model then share a model that is more cleaner and accurate than the actual one. As it's not a code issue I transfer this issue to discussion.

[codeminatodor]

I fully confirm the problem, the program speaks russina like a complete idiot, confusing stress.
The londOn of the capitAl of gret britaIn.
This makes it completely useless to use in its current state in russian.

[artiomist]

https://github.com/artiomist/ebook2audiobook-ipa
this could be a solution. Please feel free to test. I think that original module is a bit better in intonation and ipa module is better in word stress

[artiomist]

thank you @ROBERT-MCDOWELL for you hard work!

the thing with word stress is the spelling is the same. "долгами" should be pronounced "долгAми" and not "дOлгами". It cannot be fixed by mapping only.

The "quick fix" I see is to create a list of "mispronounced" words in a separate russian_trascribed.py (e.g. "датасетом": "dɐtɐs`ɛtəm",), or even just a list of words (then processed by IPA_translator.py). If the language is Russian and the word is in this russian_trascribed.py, then the word is processed separately by xtts-ru-ipa model and replaced in the final.

But you are right, this solution is valid for one language only. The solution for multiple languages is SSML.

[artiomist]

SSML can be used for voice selection for dialogs for books in any language. In our case it would be possible to use IPA model for sentences with problem words.

[ROBERT-MCDOWELL]

SSML is sadly not recognized by XTTTS and others. only Bark but with their own params.
if by lowercasing words doesn't help it?

[artiomist]

It should be in lowercase. I uppercase novels to emphasize the stress. A great example of word stress in English is the difference between "con-tent" (meaning content or a state of happiness) and "con-TENT" (meaning to contain). Stressing the first syllable indicates a noun or adjective, while stressing the second syllable indicates a verb.

What would be the solution for correct pronunciation in English for the word "content"?

[ROBERT-MCDOWELL]

ok understood. "content" as happy is not really used anymore but on books or British special conversation. The nuance between both were indeed more flagrant, but today people don't need really to make any pronunciation difference as they know already which is the meaning through the phrase context.
So this word "stress" should be indeed a pre process for Russian (and maybe other languages) involving another type of AI that analyze the sense of the sentence and correct the word with the right pronunciation by adding uppercase where needed.

[ROBERT-MCDOWELL]

update to the last git or release version.
also uppercase are not advised unless for acronyms. you cannot convert an ebook straight to an audiobook. the audio rules are very sensitive and different as long as you use model X or Y.

[artiomist]

• Updated to 25.5.6 from 25.4.25. There is still a bug loading custom model. Have to reload it manually into docker image.
• I did 7 tests. 1) Internal XTTS model + Clone voice: none 2) clone voice: MorganFreeman 3) Clone Voice: Custom 13 sec sample. 4) IPA model + Clone voice: none 5) IPA model, clone voice: MorganFreeman 6) IPA model, Clone Voice: Custom 13 sec sample 7) NeuroDanu-RuXTTS model (https://huggingface.co/NeuroDonu/RuXTTS/tree/main/10march, Add Russian XTS model #662). Conclusion: The result is a bit inconsistent. Internal XTTS performs better. Clone voice custom introduced some errors by removing pauses. IPA model was fixing word stress but was worse in pauses and intonation. NeuroDanu-RuXTTS was skipping sentences (noting that as you say it has been integrated somewhere).
• removing competing vowel ([Feature request] In non-English models stress could be assigned incorrectly coqui-ai/TTS#3282) does not sound correctly.
• the solution could be SSML (https://cloud.google.com/text-to-speech/docs/ssml) tag. There are 2 usages. There are words that with different stress have different meaning and that should be embedded in the text itself. For words that are incorrectly pronounced instead of finetuning the model maybe easier solution would be adding the word to a dictionary, dictionary that is processed through IPA (which seem to work for word stress)

[artiomist]

https://github.com/omogr/omogre
This could simply put correct word stresses (accentuator) instead of full IPA (transcriptor) and using the model. For now it is adding "+" sign before the correct vowel. But I do not see the way to tell the model to use it

[ROBERT-MCDOWELL]

"There is still a bug loading custom model. Have to reload it manually into docker image." do you mean the builtin fine tuned models or the models you upload yourself?

[artiomist]

The custom model I upload. After drag and drop there is a loading animation. And it adds the custom model to the list too. In Docker image it adds a folder with model name. But the model files are not there. The model folder is empty.

[ROBERT-MCDOWELL]

did you try to rebuild the docker with the last git or v25.5.14?

[ROBERT-MCDOWELL]

git pull or v25.5.19. thanks to test again.

[artiomist]

tried v25.5.21
the same problem. After loading custom model in docker the model folder is empty
same thing in native mode

[ROBERT-MCDOWELL]

what are you trying to do? custom models are for XTTS only, it must be a zip without folder inside but 4 files: ref.wav, model.pth, config.json, vocab.json. also you don't need to check the ./models folder, it's all automatic on gradio in the finetuned list once the model is accepted or just refer to it in headless mode

[artiomist]

you were right. I had a folder in the zip. Now loading custom model works in native and gradio

I am trying to convert an epub in russian to audiobook. XTTS is the best for now. But it produces a lot of incorrect word stresses. That is why I have tried bark (which seems faster and better with word stress here https://huggingface.co/spaces/suno/bark, but e2a is broken and slow) and trying to use ipa custom model as workaround for word stresses.

[ROBERT-MCDOWELL]

no it's not eb2ab which is broken ans slow, it's BARK and your computer!. the only function that still not working is the cloned voice.