StyleTTS2 - Text to speech comparable to Eleven Labs quality.

[logikstate]

Has anybody seen this? Just had a play around with the collab notebooks and it really is very good quality and it also has voice cloning. I only ask because I can't find anybody talking about this anywhere at all. It may have slipped under everybodys radar?

https://github.com/yl4579/StyleTTS2

[cmp-nct]

Yep, I gave it a few hours of tests and looks.
It produces life-like speech, it's fast in synthesizing too, it's completely open source including the models.

Only downside: no international language support.
Lack of available GPU time and no datasets for them. But doable it seems.

[Green-Sky]

the phonemizer needs to be set to the correct language though.

[cmp-nct]

yl4579/StyleTTS2#41
International support is something the Author of it clearly would like to see but it's also a big effort, one that needs collaboration.

It appears there is already a Bert model that supports international languages, so that would be the smallest issue (if it performs as great as PL-Bert does) but then the entire model needs to be trained on 100 hours per language with many speakers and as flawless as possible text&audio.

StyleTTS-2 appears to be better than any other solution out there, including any paid one. If international support is added it would be a huge thing to happen.

I was thinking that in general audiobooks could be used for training, with whisper they can be translated and I'd assume a fine-tuned whisper is able to detect dirty spots (like background noise, music). It wouldn't be a copyright issue either as long as the model can't replicate the content, all the big companies probably do it that way as well.

[BarfingLemurs]

How is the speed, and quality comparison to another model "xtts"?

Do you think 4bit cpu performance in ggml could be decent?
Bark.cpp even works ok in 4bit, however, the quality of the base model is very random though.

[cmp-nct]

I did not meassure the actual time to synthesize but the total time from start to finish (with loading included) is a significantly lower compared to xtts v2 on my PC.
That also fits with model size, the multispeaker model is 3 times smaller than xtts_v2. However it's not as multi-lingual, once(if) that is solved maybe the model parameters also increase.

[logikstate]

Styletts2 repo has now matured little. It now has examples you can run locally as well as Collab notebooks

[briancullinan2]

Thank you! This is exactly what i was looking for, but dang do I wish this was all in 1 step.

[logikstate]

Things have moved on since last year... There's now Kokoro TTS which uses the same architecture as StyleTTS2... and now LLama also supports TTS directly via OuteTTS

[briancullinan2]

Thank you! I don't know how else I could have learned these names based on my search parameters. "I studied computers" CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. It is then shared with tech companies, bots, hackers, government agencies, and marketers. The security of this message is none, and it may be shared on Instagram at any time. If you are OK with this, please respond. There isn't really any security or privacy anywhere. If you disagree you may want to go camping and talk to people face-to-face like in old times.

…

On Sun, Mar 2, 2025, 03:55 Paul Carter ***@***.***> wrote: Things have moved on since last year... There's now Kokoro TTS which uses the same architecture as StyleTTS2... and now LLama also supports TTS directly via OuteTTS — Reply to this email directly, view it on GitHub <#4138 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A24QKTQTVW56KGWTHILPRWT2SLPQ5AVCNFSM6AAAAABYEZWD72VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMZWGUYTKNQ> . You are receiving this because you commented.Message ID: <ggml-org/llama. ***@***.***>