TTS

actually the hardest part of a locally hosted voice assistant isn't the llm. it's making the tts tolerable to actually talk to every day. the core issue is prosody: kokoro and piper are trained on read speech, but conversational responses have shorter breath groups and different stress patterns on function words. that's why numbers, addresses, and hedged phrases sound off even when everything else works. the fix is training data composition. conversational and read speech have different prosody distributions and models don't generalize across them. for self-hosted, coqui xtts-v2 [1] is worth trying if you want more natural english output than kokoro. btw i'm lily, cofounder of rime [2]. we're solving this for business voice agents at scale, not really the personal home assistant use case, but the underlying problem is the same. [1] https:

This is a spam website that was not created by the people who created Coqui TTS. This is the original: https://github.com/coqui-ai/TTS

Can anyone recommend an open source option that would allow training on a custom voice (my own, so I'd be able to record as many snippets as it needed to train on) to allow me to use it for TTS generation without sharing it off my machine? Edit: I'll wait to see if any recommendations get made here, if not I might give this one a go: https://github.com/coqui-ai/TTS

It's really easy for a technical person to do as well. I use Coqui TTS[0] as part of my home automation, I wrote a small python script that lets me upload a voice clip for it to clone after I got the idea from HeyWillow[1], and a small shim that lets me send the output to a Home Assistant media player instead of using their standard output device. I run the TTS container on a VM with a Tesla P4 (~£100 to buy) and get about 1x-2x (roughly the same time it'd take to say it, to process) using the large model. Just for a giggle, I uploaded a few 3s-5s second clip of myself speaking and cloned my voice, then executed a command to our living room media player to call my wife into the room; from another room, she was 100% convinced it was myself speaking words I'd never spoken. I tried playing with a variety of sentences for a few hours and over

For anyone who is interested, CoquiTTS (formerly, MozillaTTS) was great, but the project isn't maintained anymore (athough there's been some confusion about whether or not it's active. See: https://github.com/coqui-ai/TTS/issues/4022). Looks like there's an effort to keep an actively maintained fork here, though: https://github.com/idiap/coqui-ai-TTS

Listening to the piper demos [1] and comparing to coqui [2], I'd say coqui sounds better to me, but I'd love to hear others' opinions. Looks like Piper's latest commits were 3 months ago [3] while Coqui's were 8 months ago [4], so they both seem similar in recency. In terms of ease of use though, especially with this project, personally Piper seems way less overwhelming. [1] https://rhasspy.github.io/piper-samples/ [2] https://huggingface.co/spaces/coqui/xtts [3] https://github.com/rhasspy/piper [4] https://github.com/coqui-ai/TTS

I am super impressed by the quality of voice cloning offered by Eleven Labs and Play.ai. I feel like I see impressive OSS demos on social frequently, but last weekend I took a few popular ones for a spin and quality wasn't even close to the proprietary models. https://github.com/coqui-ai/tts https://github.com/serp-ai/bark-with-voice-clone https://github.com/metavoiceio/metavoice-src https://github.com/myshell-ai/OpenVoice https://github.com/collabora/WhisperSpeech https://github.com/neonbjb/tortoise-tts Has anyone else had success with these? Are there other projects I should look at?

The license is the MPL, which allows commercial use? https://github.com/coqui-ai/TTS/blob/dev/LICENSE.txt

When I did a similar thing (but with less LLM) I liked https://github.com/coqui-ai/TTS but back then I needed to cut out the conversion step from tensor to a list of numbers to make it work really nicely.

lol this marketing technique is getting very old. https://github.com/coqui-ai/TTS is already amazing and open source.

Recently I was baffled how far we've come with this. It doesn't work perfectly, but could be enough to fool someone. Just one pip install and a short voice sample away: https://github.com/coqui-ai/TTS

I've used coqui.ai's TTS models[0] and library[1] to great success. I was able to get cloned voice to be rendered in about 80% of the audio clip length, and I believe you can also stream the response. Do note the model license for XTTS, it is one they wrote themselves that has some restrictions. [0] https://huggingface.co/coqui/XTTS-v2 [1] https://github.com/coqui-ai/TTS

My only exposure to Coqui was their text to speech software. If I remember correctly the website was a commercialized service with TTS and probably some other related things. I hope the software work continues in the open. https://github.com/coqui-ai/TTS

With Coqui.ai shutting down it's studio and API it looks like they are reinvesting their efforts into their XTTS models and framework. A recent update to their GitHub also has a no-code gradio ui to facilitate fine-tuning and inferencing locally. https://github.com/coqui-ai/TTS/releases/tag/v0.21.3