silero-vad

So I use a VAD onnx (Silero [1]) to automatically detect when someone is talking, and then it sends the audio into one of the voice recognition libraries. I originally tried to get away with just Whisper Tiny in the chess game [2], but it performs worse on the kinds of short phrases (knight E4, c takes d5, etc) used to dictate chess notation. Even with hotword-based phrasing and corrections, I found its accuracy on brief inputs noticeably poorer. So I switched over to Sherpa [3] trained on gigaspeech. It’s significantly more accurate, but it also comes with a correspondingly larger memory footprint. Ideally, I would have used just one engine, but I needed a fallback for iOS devices (especially older ones) which can easily OOM. [1] - https://github.com/snakers4/silero-vad [2] - https://shahkur.specr.net [3] - https://github.com/k2-fsa/sher

Hi, I built the client UI for this and... yea, I really wanted to get Firefox working :( We needed a way to measure voice-to-voice latency from the end-user's perspective, and found Silero voice activity detection (https://github.com/snakers4/silero-vad) to be the most reliable at detecting when the user has stopped speaking, so we can start the timer (and stop it again when audio is received from the bot.) Silero runs via onnx-runtime (with wasm). Whilst it sort-of-kinda works in Firefox, the VAD seems to misfire more than it should, causing the latency numbers to be somewhat absurd. I really want to get it working though! I'm still trying. The code for the UI VAD is here: https://github.com/pipecat-ai/web-client-ui/tree/main/src/va...

>How do you detect speech starting and stopping? https://github.com/snakers4/silero-vad

One of Silero-VAD developers here. Check out our VAD here - https://github.com/snakers4/silero-vad

Stellar quality. Highly portable. No strings attached. Supports 8 kHz and 16 kHz. Models < one megabyte in size. Supports 30, 60 and 100 ms chunks. Trained on 100+ languages, generalizes well. One chunk ~ 1ms on a single thread. ONNX up to 2-3x faster. https://github.com/snakers4/silero-vad