F5-TTS

Cool. While I think this is indeed impressive and has a specific use case (e.g. in the embedded sector), I'm not totally convinced that the quality is good enough to replace bigger models. With fish-speech[1] and f5-tts[2] there are at least 2 open source models pushing the quality limits of offline text-to-speech. I tested F5-TTS with an old NVidia 1660 (6GB VRAM) and it worked ok-ish, so running it on a little more modern hardware will not cost you a fortune and produce MUCH higher quality with multi-language and zero-shot support. For Android there is SherpaTTS[3], which plays pretty well with most TTS Applications. 1: https://github.com/fishaudio/fish-speech 2: https://github.com/SWivid/F5-TTS 3: https://github.com/woheller69/ttsengine

You're very welcome. On f5 github repo is an active discussion (i'm involved too) on supporting other languages including german: https://github.com/SWivid/F5-TTS/issues/87#issuecomment-2418...

I tried to build something kind of like NotebookLM (personalized news podcasts) over the past months (https://www.tailoredpod.ai), but biggest issue is that the existing good TTS Apis are so expensive that a product such as NotebookLM is not really possible for a normal company that doesn't have internal access to Google's models. OpenAI has the cheapest / quality good enough TTS Api, but even then generating hours of audio for free is way too expensive. Open Source TTS models are slowly catching up, but they still need beefy hardware (e.g. https://github.com/SWivid/F5-TTS)

On related note a very good open source TTS model was released 2 days back: https://github.com/SWivid/F5-TTS Very good voice cloning capability. Runs under 10G vram nvidia gpu.

The trends around compute increasing is on the whole correct but by no means is it a universal rule. More optimized training procedures and model architectures are coming out all the time. In just the last week, we got F5-TTS which is trained on twice as much data as the previous leader in realistic TTS (Tortoise TTS) and is exponentially faster - taking only 3 weeks on H-100. We also got Meissonic, a text-to-image model that is exponentially easier to train than any existing model. IE you can train a Stable Diffusion like model from scratch on consumer hardware or in the cloud for abou $500. https://huggingface.co/MeissonFlow/Meissonic https://github.com/SWivid/F5-TTS The reason the trend is that compute costs are doubling is because this is an arms race and everyone in the corporate space is prioritizing bigger models over better archit

Looks cool! Combine this with this new TTS that released today that looks really good and an LLM and you'd have a pretty good all-local voice assistant! https://github.com/SWivid/F5-TTS