daanzu

kaldi-active-grammar

Python2020–2023

daanzu / kaldi-active-grammar

kaldi-active-grammar

Python Kaldi speech recognition with grammars that can be set active/inactive dynamically at decode-time

AI & Machine Learning Languages & Compilers

Explore on GitHub Read Hacker News discussions

7Distinct people

22Mentions

0Explicit recommenders

350GitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Python Kaldi speech recognition with grammars that can be set active/inactive dynamically at decode-time

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

dictation-toolbox/dragonfly · linked in 5 of the same discussions
alphacep/vosk-api · linked in 2 of the same discussions
dictation-toolbox/caster · linked in 2 of the same discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 8:03:49 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year22 dated mentions

Observed mentions: 2020-02-24 16:02:22 UTC to 2023-11-22 02:28:01 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Comments
2020	2020-01-01 00:00:00 UTC	2021-01-01 00:00:00 UTC	15	15
2021	2021-01-01 00:00:00 UTC	2022-01-01 00:00:00 UTC	0	0
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	6	6
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	1	1

What Hacker News says

15 sources

daanzu·Sep 7, 2020

A great demonstration of coding by voice. I want to point out that the Dragonfly library has backends other than the closed/commercial Dragon, and there has been significant progress expanding beyond it recently. I have been developing kaldi-active-grammar [0] for the past couple years as a fully open source, modifiable, cross-platform, and free alternative to Dragon when used with Dragonfly. While KaldiAG misses some of the user-friendly niceties of Dragon, these are least helpful when coding by voice. And KaldiAG has benefits such as lower latency that improves coding by voice tremendously. I created KaldiAG because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to

Read the rest

fit my needs than closed source ever could. [0] https://github.com/daanzu/kaldi-active-grammar

In Coding by Voice in Zig – Rickard Andersson

nshm·Aug 22, 2020

Try https://github.com/alphacep/vosk-api. It supports 10 languages, works on Android and RPi and also has big and more accurate server models. Other good ones are https://github.com/daanzu/kaldi-active-grammar and https://talonvoice.com/ There are toolkits for research like https://github.com/kaldi-asr/kaldi, https://github.com/espnet/espnet, wav2letter, Espresso, Nvidia/Nemo, https://github.com/didi/athena. You can try them too if you want to go deep. Some of them have interesting capabilities.

In Future of DeepSpeech / STT after recent changes at Mozilla

daanzu·Jul 1, 2020

Gathering, collecting, and publishing such a dataset would be great, and would certainly much improve the baseline speech recognition for people with disordered speech, but it can only help so much without personalizing to a specific individual. This is true for anybody, but more so for disordered speech. This is an area where I think "generic" solutions will inevitably struggle, even if they are somewhat specialized on "generic disarthritic" speech. However, this means that the gains to be had from personalized training are greater for disordered speech than for "average" speech. I develop kaldi-active-grammar [0], which specializes the Kaldi speech recognition engine for real-time command & control with many complex grammars. I am also working on making it easier to train personalized speech models, and to fine tune generic models with

Read the rest

training for an individual. I have posted basic numbers on some small experiments [1]. Such personalized training can be time consuming (depending on how far one wants to take it), but as my parent comment says, disabled people may need to rely more on ASR, which means they have that much more to gain by investing the time for training. Nevertheless, a Common Voice disordered speech dataset would be quite helpful, both for research, and for pre-training models that can still be personalized with further training. It is good to see (in my sibling comment) that it is being discussed. [0] https://github.com/daanzu/kaldi-active-grammar [1] https://github.com/daanzu/kaldi-active-grammar/blob/master/d...

In Mozilla Common Voice Dataset: More data, more languages

daanzu·Jun 27, 2020

I develop kaldi-active-grammar [0]. The Kaldi engine itself is state of the art and open source, but is focused on research rather than usability. My project has a simple interface and comes with a pretty good open source speech model. However, kaldi-active-grammar specializes in real time command and control, with advanced features that don't really apply to your use case. Vosk [1] is probably a simpler, better fit for you. It likewise uses Kaldi and can use my models, and offers some others of its own as well. Neither are particularly focused on transcription per se, but they are open. [0] https://github.com/daanzu/kaldi-active-grammar [1] https://github.com/alphacep/vosk-api

In Ask HN: Best open source voice recognition/transcription software?

daanzu·Jun 13, 2020

I agree with everything you said, but I would add that a critical component of voice command and control is strict grammars. There is so much structure and context in what we speak, and being able to limit what can be recognized to only what can be reasonably spoken (based on the current context) can allow massive increases in accuracy. (EDIT: ah, you edited to add a mention of this as well.) And one shameless plug deserves another! Vosk is a great project, but my kaldi-active-grammar [0] (mentioned in another comment here) also uses the same Kaldi engine, but extends it and is designed specifically for this use case. It supports defining many grammars, in any combination, and activating/deactivating them at will instantly per-utterance. I think it's probably a better fit as a backend for your project than vosk. My work focuses on the bac

Read the rest

kend technology, so it would be great to have more front ends using it to put it within users' reach (so to speak). [0] https://github.com/daanzu/kaldi-active-grammar

In Ask HN: Is there any work being done in speech-to-code with deep learning?

daanzu·Jun 13, 2020

Windows Speech Recognition is far from the best, so perhaps your trouble could be partly caused by how you had to speak in order to be understood, rather than the command style? I used to use WSR to code by voice, and it was far more laborious than my current setup. I develop kaldi-active-grammar [0]. The Kaldi engine is state of the art for command and control. Although I don't have the data and resources for training a model like Microsoft/Nuance/Google, being an open rather than closed system allows me to train models that are far more personalized than the large commercial/generic ones you are used to. For example, see the video of me using it [1], where I can speak in a relaxed manner without having to over enunciate and strain my voice. Gathering the data for such training does take some time, but the results can be huge [2]. Perfor

Read the rest

ming the actual training is currently complicated; I am working on making it portable and more turnkey, but it's not ready yet. However, I am running test training for some people. Contact me if you want me to use you as a guinea pig. [0] https://github.com/daanzu/kaldi-active-grammar [1] https://youtu.be/Qk1mGbIJx3s [2] https://github.com/daanzu/kaldi-active-grammar/blob/master/d...

In Ask HN: Is there any work being done in speech-to-code with deep learning?

daanzu·Jun 12, 2020

I wrote a simple little Python GUI app to record training audio. Given a text file containing prompts, it will choose a random selection and ordering of them, display them to be dictated by the user, and record the dictation audio and metadata to a .wav file and recorder.tsv file respectively. You can select a previous recording to play it back, delete it, and/or re-record it. It comes with a few selections of sentences designed to cover a broad diverse range of English (Arctic, TIMIT). Pretty simple and no-nonsense. https://github.com/daanzu/speech-training-recorder Originally intended for recording data for training speech recognition models [0], it should work just as well for recording to be used for speech synthesis. [0] https://github.com/daanzu/kaldi-active-grammar

In Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

daanzu·Mar 15, 2020

I develop Kaldi Active Grammar [1], which is mainly intended for use with strict command grammars. Compared to normal language models, these can provide much better accuracy, assuming you can describe (and speak) your command structure exactly. (This is probably more acceptable for a voice assistant for an audience that is more technical.) The grammar can be specified by a FST, or you can use KaldiAG through Dragonfly, which allows you to specify them (and their resultant actions) in Python. However, KaldiAG can also do simple plain dictation if you want. KaldiAG has an English model available, but other models could be trained. Although you can't just drop in and use a standard Kaldi model with KaldiAG, the modifications required are fairly minimal and don't require any training or modification of its acoustic model. All recognition is p

Read the rest

erformed locally and off line by default, but you can also selectively choose to do some recognition in the cloud, too. Kaldi generally performs at the state of art. As a hybrid engine, although training can be more complicated, it generally requires far less training data to achieve high accuracy, compared to "end to end" engines. [1] https://github.com/daanzu/kaldi-active-grammar

In Ask HN: Non-cloud voice recognition for home use?

lunixbochs·Mar 14, 2020

Hi, I'm the dev behind https://talonvoice.com I've been working with Facebook's wav2letter project and the results (speed on CPU, command accuracy) are extremely good in my experience. They also hold the "state of the art" for librispeech (a common benchmark) on wer_are_we [1]. Granted, that's with a 2GB model that doesn't run very well on CPU, but I think most of the fully "state of the art" models are computationally expensive and expected to run on GPU. Wav2letter has other models that are very fast on CPU and still extremely accurate. You can run their "Streaming ConvNets" model on CPU to transcribe multiple live audio streams in parallel, see their wav2letter@anywhere post for more info [2] I am getting very good accuracy on the in-progress model I am training for command recognition (3.7% word error rate on librispeech clean, about

Read the rest

8% WER on librispeech other, 20% WER on common voice, 3% WER on "speech commands"). I plan to release it alongside my other models here [5] once I'm done working on it. There's a simple WER comparison between some of the command engines here [3] Between this and wer_are_we [1] it should give you a general idea of what to expect when talking about Word Error Rate (WER). (Note the wav2letter-talonweb entry in [3] is a rather old model I trained, known to have worse accuracy, it's not even the same NN architecture). ---- As far as constraining the vocabulary, you can try train a kenlm language model for kaldi, deepspeech, and wav2letter by grabbing KenLM and piping normalized (probably lowercase it and remove everything but ascii and quotes) text into lmplz: cat corpus.txt | kenlm/build/bin/lmplz -o 4 > model.arpa And you can turn it into a compressed binary model for wav2letter like this: kenlm/build/bin/build_binary -a 22 -q 8 -b 8 trie model.arpa model.bin There are other options, like using a "strict command grammar", but I don't have enough context as to how you want to program this to guide you there. I also have tooling I wrote around wav2letter, such as wav2train [4] which builds wav2letter training and runtime data files for you. I'm generally happy to talk more and answer any questions. ---- [1] https://github.com/syhw/wer_are_we [2] https://ai.facebook.com/blog/online-speech-recognition-with-... [3] https://github.com/daanzu/kaldi-active-grammar/blob/master/d... [4] https://github.com/talonvoice/wav2train [5] https://talonvoice.com/research/

In Ask HN: Non-cloud voice recognition for home use?

daanzu·Feb 25, 2020

If you are willing to do some training, you can get tremendously improved results, in my experience. For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed Kaldi Active Grammar [1], and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. The personalized training is still pretty new and raw, and it needs a lot of setup to do the training itself currently. [1] https://github.com/daanzu/kaldi-active-grammar

In On Voice Coding

lunixbochs·Feb 25, 2020

I think to state "unmatched accuracy" in good faith we should actually come up with a common benchmark and measure against it. I believe there aren't really any clean benchmarks for command accuracy floating around (it would be ideal if we used a strict grammar to properly measure the command decoders), and a wav2letter model holds the state of the art for librispeech WER% as of 2019. I found your measurement here [1] which is against an unknown wav2letter acoustic+language model pair, as the web demo is at any given point in time running an arbitrary model based on having users test in-progress models, and it has never been running the model I am currently shipping with Talon. (As a small example, the unfinished wav2letter experiment I am training right now has a 3.17% WER on speech commands, and 6.86% WER on librispeech clean, both numb

Read the rest

ers without using a language model) [1] https://github.com/daanzu/kaldi-active-grammar/blob/master/d...

In On Voice Coding

daanzu·Feb 25, 2020

For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed my Kaldi Active Grammar [1], and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. Also, Dragon has many commands that cannot be disabled, and if your accuracy is already low, more commands available means more possible things to get wrong. Something like Dragonfly with KaldiAG could allow you to reduce the command set, and improve the practical accuracy. So, more personalized training/software may work better. [1] https://github.com/daanzu/kaldi-active-grammar

In On Voice Coding

daanzu·Feb 25, 2020

This comment fails to mention Dragonfly with my Kaldi Active Grammar backend [1], which is cross platform (Windows/Linux now and Mac functional and to be released soon), completely free with no private beta features (although I do accept donations), and 100% open source (unlike Talon). The speech recognition is local, with extremely low latency. See the video demonstration [2] on the project page. I think the underlying Kaldi engine delivers unmatched accuracy as a free non-commercial engine. I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could. Furthermore, the original article mentions C

Read the rest

aster (which is built on Dragonfly), but doesn't mention that KaldiAG works with it, and that work is underway to expand Caster's platform support. [1] https://github.com/daanzu/kaldi-active-grammar [2] https://youtu.be/Qk1mGbIJx3s

In On Voice Coding

LexiconCode·Feb 24, 2020

Greetings Everyone, I help maintain the Caster project. The key difference from other solutions out there as we seek to support a completely open source voice coding stack. Open source is only way to go long term if you're going to being using a tool for most of your life. Fortunately for some it acts as a bridge until their RSI symptoms becomes manageable or goes into remission. We are working towards cross-platform support Linux and Mac as well as adding support for Kaldi. Dragonfly is already cross-platform so just a few windows specific functions to be ported yet in Caster. Kaldi via daanzu's kaldi active grammar. https://github.com/daanzu/kaldi-active-grammar Talon may be free but is closed sourced.

In On Voice Coding

tbenst·Feb 24, 2020

The best alternative right now for voice coding is https://github.com/daanzu/kaldi-active-grammar. wav2letter is also under use by some, although requires more effort to setup

In On Voice Coding