fish-speech

Cool. While I think this is indeed impressive and has a specific use case (e.g. in the embedded sector), I'm not totally convinced that the quality is good enough to replace bigger models. With fish-speech[1] and f5-tts[2] there are at least 2 open source models pushing the quality limits of offline text-to-speech. I tested F5-TTS with an old NVidia 1660 (6GB VRAM) and it worked ok-ish, so running it on a little more modern hardware will not cost you a fortune and produce MUCH higher quality with multi-language and zero-shot support. For Android there is SherpaTTS[3], which plays pretty well with most TTS Applications. 1: https://github.com/fishaudio/fish-speech 2: https://github.com/SWivid/F5-TTS 3: https://github.com/woheller69/ttsengine

There is a fork here https://github.com/idiap/coqui-ai-TTS 'coqui-tts' Though according to the TTS leaderboard, Fish Speech https://github.com/fishaudio/fish-speech and Kokoro are higher. https://huggingface.co/hexgrad/Kokoro-82M https://huggingface.co/fishaudio/fish-speech-1.5

Announcement on X: https://x.com/FishAudio/status/1853655232779313408 Demo: https://huggingface.co/spaces/fishaudio/fish-agent Repo: https://github.com/fishaudio/fish-speech Weights: https://huggingface.co/fishaudio/fish-agent-v0.1-3b

The team behind Fish Speech, including rcell, stardust, and lengyue, who brought you So-VITS-SVC, Bert-VITS2, and GPT-SoVITS, has made incredible improvements in this release. Fish Speech 1.3 now offers enhanced stability and emotion, and can clone anyone's voice with just a 15-second audio prompt! As strong advocates of the open-source community, we've open-sourced Fish Speech 1.2 SFT today and introduced an Auto Reranking system. And we'll be open-sourcing Fish Speech 1.3 soon! Playground: http://fish.audio GitHub: https://github.com/fishaudio/fish-speech Blog Post: https://blogs.fish.audio/announcing-fish-speech-1-3-30417c28...

While we are still figuring out ways to improve the agent's emotional response to OpenAI GPT-4 level, we have already made significant progress in aligning OpenAI's TTS performance. To begin this experiment, we collected 10 hours of OpenAI TTS data to perform supervised fine-tuning (SFT) on both the LLM and VITS models, which took approximately 30 minutes. After that, we used 15 seconds of audio as a prompt during inference. Demos Available: https://firefly-ai.notion.site/OpenAI-Examples-34975ae263a9496c84e89fb7b1ea25a4?pvs=4 As you can see, the model's emotion, rhythm, accent, and timbre match the OpenAI speakers, though there is some degradation in audio quality, which we are working on. To avoid any legal issues, we are unable to release the fine-tuned model, but I believe everyone can tune Fish Speech to this level within hours and fo