whisper

To be clear, it's not LLMs creating the captions. Whisper[0], one of the best of its kind currently, is a speech recognition model, not a large language model. It's trained on audio, not text, and it can run on your mobile phone. It's still AI, of course. But there is distinction between it and an LLM. [0] https://github.com/openai/whisper/blob/main/model-card.md

How about OpenAIs whisper[0]? My social science friends tell me it‘s been great for them. Not sure whether data privacy et all would be an issue of course, but I guess you can just run it locally :) [0]https://github.com/openai/whisper

I use a two-pass approach - first pass with ASR (OpenAI Whisper) and second pass with an LLM. I ask users to provide context upfront and use that as the "initial_prompt" parameter in Whisper: https://github.com/openai/whisper/discussions/963#discussion... Gemini might have similar capabilities for custom vocabulary, though I'm not certain about their specific implementation. The two-pass ASR+LLM approach could work with Gemini's output as well.

Very cool. Uses whiper small uder the hood. https://github.com/openai/whisper

It's even better for some languages other than English (e. g. Spanish), see: https://github.com/openai/whisper?tab=readme-ov-file#availab...

Whisper tends to fill silence with random garbage from its training set. [0] [1] [2] [0]: https://github.com/openai/whisper/discussions/679 [1]: https://github.com/openai/whisper/discussions/928 [2]: https://github.com/openai/whisper/discussions/2608

no, but whisper has a 39M model: https://github.com/openai/whisper

>Aside: Are there any models for understanding voice to text, fully offline, without training? OpenAI's whisper is a few years old and pretty solid. https://github.com/openai/whisper

I tried googling this and found questions from Telegram users why voice messages recognition sometimes produces this phrase and who is this person. Also I found this thread [1] claiming that the subtitles by DimaTorzok are coming from some Russian youtube videos on gaming like [2]. [1] https://github.com/openai/whisper/discussions/2372 [2] https://www.youtube.com/watch?v=FAqyUuahMlc&t=401s

> has a great package ecosystem So great there are 8 of them. 800% better than all the rest! > If you think Python is a bad language for AI integrations, try writing one in a compiled language. I'll take this challenge, all day, every day, so long as I and the hypothetical 'move fast and break things' have equal "must run in prod" and "must be understandable by some other human" qualifiers What type is `array`? Don't worry your pretty head about it, feed it whatever type you want and let Sentry's TypeError sort it out <https://github.com/openai/whisper/blob/v20250625/whisper/aud...> Oh, sorry, and you wanted to know what `pad_or_trim` returns? Well that's just, like, your opinion man

It's a very simple change in a vanilla python implementation. The encoder is a set of attention blocks, and the length of the attention can be changed without changing the calculation at all. Here(https://github.com/openai/whisper/blob/main/whisper/model.py...) is the relevant code in the whisper repo. You'd just need to change the for loop to an enumerate and subsample the context along its length at the point you want. I believe it would be: for i, block in enumerate(self.blocks): x = block(x) if i==4: x = x[,,::2]

It seems that the team that used to work on DeepSpeech then worked on coqui-ai STT https://github.com/coqui-ai/STT and now recommends using OpenAI Whisper (https://github.com/openai/whisper)

I was laid off recently and, instead of looking for another job right away, I decided to build something I always wanted: a transcription tool that respects user privacy. Existing transcription tools often require internet connectivity, send your private audio to cloud servers, or lock you into monthly subscriptions. I wanted something different—so I built WhisperBuddy, a privacy-first AI transcription app that runs entirely on your machine. WhisperBuddy uses OpenAI’s Whisper model (see: https://github.com/openai/whisper). While the model may not always deliver the absolute best accuracy, it provides solid performance and, most importantly, 100% privacy—no audio ever leaves your device. I optimized the app for local performance and usability, and I’m eager to hear feedback from the community. Thanks for checking it out!

According to the Common Voice 15 graph on OpenAI's github repository, Albanian is the single worst performance you could have had: https://github.com/openai/whisper But for what it's worth, I tried putting the YouTube video of Tom Scott presenting at the Royal Institute into the model, and even then the results were only "OK" rather than "good". When even a professional presenter and professional sound recording in a quiet environment has errors, the model is not really good enough to bother with.