moonshine

Presumably it's not, but the TTS voice in the video sounds to me more like formant synthesis than diphone - it reminds me of my DECtalk. The project credits does mention espeak (which is formant based) as well as various other TTS projects, although it sounds like they are only using the pronunciation part of espeak, not the voice synthesis. https://github.com/moonshine-ai/moonshine#acknowledgements

If you look at this chart here it seems the tiny model has a WER of ~12%… not sure about the micro model: https://github.com/moonshine-ai/moonshine#when-should-you-ch...

I wanted to share our new speech to text model, and the library to use them effectively. We're a small startup (six people, sub-$100k monthly GPU budget) so I'm proud of the work the team has done to create streaming STT models with lower word-error rates than OpenAI's largest Whisper model. Admittedly Large v3 is a couple of years old, but we're near the top the HF OpenASR leaderboard, even up against Nvidia's Parakeet family. Anyway, I'd love to get feedback on the models and software, and hear about what people might build with it.

I use VAD to chunk audio. Whisper and Moonshine both works in a chunk, but for moonshine: > Moonshine's compute requirements scale with the length of input audio. This means that shorter input audio is processed faster, unlike existing Whisper models that process everything as 30-second chunks. To give you an idea of the benefits: Moonshine processes 10-second audio segments 5x faster than Whisper while maintaining the same (or better!) WER. Also for kyutai, we can input continuous audio in and get continuous text out. - https://github.com/moonshine-ai/moonshine - https://docs.hyprnote.com/owhisper/configuration/providers/k...

This reminded me of https://github.com/usefulsensors/moonshine

Only seems to cover half of what you're asking for... Starred this the other day and haven't gotten to trying it out : https://github.com/usefulsensors/moonshine

They supply their paper in the Git repo, here: https://github.com/usefulsensors/moonshine/blob/main/moonshi... The section "3.2. Training data collection & preprocessing" covers what you're inquiring about: "We train Moonshine on a combination of 90K hours from open ASR datasets and over 100K hours from own internally-prepared dataset, totalling around 200K hours. From open datasets, we use Common Voice 16.1 (Ardila et al., 2020), the AMI corpus (Carletta et al., 2005), Gi- gaSpeech (Chen et al., 2021), LibriSpeech (Panayotov et al., 2015), the English subset of multilingual Lib- riSpeech (Pratap et al., 2020), and People’s Speech (Galvez et al., 2021). We then augment this training corpus with data that we collect from openly-available sources on the web. We discuss preparation methods for our self-collected data in the following." It do

GitHub https://github.com/usefulsensors/moonshine

Git repository https://github.com/usefulsensors/moonshine