whisper

I've been using Whisper with great results; GPU acceleration gets the job done, nothing leaves my box, and the medium model is overkill. https://openai.com/index/whisper/ https://github.com/openai/whisper

That's interesting - I have been playing with Whisper speech-to-text recently (both CLI, and also . Depending on the model & computer performance, it seems like it can stream (listen and transcribe to text) audio. Maybe you can pipe that to a text-to-speech service (remote or local), and configure that as an audio input to OBS. Are you live-streaming, or just recording video file to edit/publish later? If it's not live, that may relax some of the latency requirements. Related links: - https://github.com/openai/whisper (speech-to-text for terminal) - https://buzzcaptions.com/ (graphical interface to Whisper) Somewhat related, Descript mentions it does dubbing. I recently used this tool to edit a video, and I was impressed by how easy it was, and accurate too. It may not fit your realtime use-case, but might be worth exploring... https://de

To be clear, it's not LLMs creating the captions. Whisper[0], one of the best of its kind currently, is a speech recognition model, not a large language model. It's trained on audio, not text, and it can run on your mobile phone. It's still AI, of course. But there is distinction between it and an LLM. [0] https://github.com/openai/whisper/blob/main/model-card.md

How about OpenAIs whisper[0]? My social science friends tell me it‘s been great for them. Not sure whether data privacy et all would be an issue of course, but I guess you can just run it locally :) [0]https://github.com/openai/whisper

I use a two-pass approach - first pass with ASR (OpenAI Whisper) and second pass with an LLM. I ask users to provide context upfront and use that as the "initial_prompt" parameter in Whisper: https://github.com/openai/whisper/discussions/963#discussion... Gemini might have similar capabilities for custom vocabulary, though I'm not certain about their specific implementation. The two-pass ASR+LLM approach could work with Gemini's output as well.

Very cool. Uses whiper small uder the hood. https://github.com/openai/whisper

It's even better for some languages other than English (e. g. Spanish), see: https://github.com/openai/whisper?tab=readme-ov-file#availab...

Whisper tends to fill silence with random garbage from its training set. [0] [1] [2] [0]: https://github.com/openai/whisper/discussions/679 [1]: https://github.com/openai/whisper/discussions/928 [2]: https://github.com/openai/whisper/discussions/2608

no, but whisper has a 39M model: https://github.com/openai/whisper

>Aside: Are there any models for understanding voice to text, fully offline, without training? OpenAI's whisper is a few years old and pretty solid. https://github.com/openai/whisper

I tried googling this and found questions from Telegram users why voice messages recognition sometimes produces this phrase and who is this person. Also I found this thread [1] claiming that the subtitles by DimaTorzok are coming from some Russian youtube videos on gaming like [2]. [1] https://github.com/openai/whisper/discussions/2372 [2] https://www.youtube.com/watch?v=FAqyUuahMlc&t=401s

> has a great package ecosystem So great there are 8 of them. 800% better than all the rest! > If you think Python is a bad language for AI integrations, try writing one in a compiled language. I'll take this challenge, all day, every day, so long as I and the hypothetical 'move fast and break things' have equal "must run in prod" and "must be understandable by some other human" qualifiers What type is `array`? Don't worry your pretty head about it, feed it whatever type you want and let Sentry's TypeError sort it out <https://github.com/openai/whisper/blob/v20250625/whisper/aud...> Oh, sorry, and you wanted to know what `pad_or_trim` returns? Well that's just, like, your opinion man