whisper

> * https://github.com/openai/whisper I would be very concerned about any LLM model being used for "transcription", since they may injecting things that nobody said, as in this recent item: https://news.ycombinator.com/item?id=41968191

OpenAI whisper runs on iOS native on device, can't be just costs. https://github.com/openai/whisper/discussions/407

Has anyone had any luck with a free, offline, open-source, real-time speech-to-speech translation app on under-powered devices (i.e., older smart phones)? * https://github.com/ictnlp/StreamSpeech * https://github.com/k2-fsa/sherpa-onnx * https://github.com/openai/whisper I'm looking for a simple app that can listen for English, translate into Korean (and other languages), then perform speech synthesis on the translation. Basically, a Babelfish that doesn't stick in the ear. Although real-time would be great, a max 5-second delay is manageable. RTranslator is awkward (couldn't get it to perform speech-to-speech using a single phone). 3PO sprouts errors like dandelions and requires an online connection. Any suggestions?

Does it have to use Whisper? If so, can't you just run it on that server instead of the Mac? https://github.com/openai/whisper/discussions/1463 If it doesn't, there are a bunch of other speech recognition APIs. Most of them use older techs but might be good enough: https://www.gladia.io/blog/openai-whisper-vs-google-speech-t... Personally I found Otter.ai works really well for the transcription part, but they don't have an API: https://otter.ai You can also just upload them all to YouTube in a private playlist and it'll automatically transcribe them for you.

The whisper command at https://github.com/openai/whisper can generate .srt and .vtt subtitle files using --output_format.

https://github.com/openai/whisper vs https://github.com/ggerganov/whisper.cpp They are two inference engines for running the whisper ASR model, each with their own API AFAIK.

I’ve had good results with Whisper. You can use the OpenAI API and it is also open source: https://github.com/openai/whisper

Multi-language Whisper! https://github.com/openai/whisper

On could plug https://github.com/openai/whisper to transcribe recorded audio.

This is a brilliant and useful application of LLM technology, I'm impressed. One question- On the backend, is it downloading each video CC (closed-caption) transcript and feeding that into a tuned prompt? What happens for videos where this is missing? Asking because I've noticed CC is occasionally unavailable for some YouTube videos. If you cared to have a fallback, a potentially interesting experiment / solution for such cases is to download the video, extract the audio to a WAV file, then through the audio through Whisper [1] to generate the transcript. Using CPUa, it will still be incredibly intensive and slow, generally not much faster than real-time (e.g. a 5 minute clip will take on the order of ~5 minutes to complete transcription). However, with Whisper running on a fancy GPU it is insanely faster, between 100-200x faster, me