SymSpell

Author of SymSpell here. Congrats on the launch of Lexiathan. Unfortunately, the comparison of Lexiathan vs. Symspell on your website regarding accuracy is misleading. 1. SymSpell has two parameters to control the maximum edit distance. Once you set both to 3, then also terms with an edit distance of 3 are accurately corrected: pronnouncaition -> pronunciation inndappendent -> independent unegspeccted -> unexpected soggtwaee -> software 2. SymSpell comes with dictionaries in several sizes. Once you load the 500_000 terms dictionary, then also the two remaining terms will be corrected: maggnificntally -> magnificently annnesteasialgist -> anesthesiologist https://github.com/wolfgarbe/SymSpell/blob/master/SymSpell.B... SymSpell accurately corrects all of your examples if used properly with the correct parameters and

This sounds very similar to: https://github.com/wolfgarbe/SymSpell Or maybe I am confused what you're saying. Are you trying to find more creative ways of saying the same phrase?

I integrated JSymSpell[0] into KeenWrite[1], which is based on SymSpell[2]. The algorithm provides numerous suggestions for "plannin'", with "planning" being the first hit[3]. Despite the English lexicon[4] not including all words for all localizations (US/UK) the results have been adequate for my use so far. What open-source solutions have you been using that aren't working as well as you'd like? [0]: https://github.com/rxp90/jsymspell [1]: https://github.com/DaveJarvis/keenwrite/ [2]: https://github.com/wolfgarbe/SymSpell [3]: https://i.ibb.co/Bs0W6w2/screenshot.png [4]: https://github.com/DaveJarvis/keenwrite/tree/master/src/main...

It was a bit fun comparing the Haskell version and the C# version: https://github.com/wolfgarbe/SymSpell/blob/master/SymSpell/S... https://github.com/cbeav/symspell/blob/master/src/SymSpell.h...

Something often overlooked in comparisons is "The many dimensions of beat". In some ways ORC probably already beats the best JVM while in others it may never. The best benchmark is, as always, one's own code. :-) I've found Nim quite efficient, though. The one time when I had trouble with Nim's GC perf was years ago implementing Wolfe Garbe's Symmetric Delete spell checking algorithm [0]. Even then, it worked with nim c --gc:boehm but blew up with others. So, there was nothing stopping me from doing it in Nim. This was my initial "in volatile RAM" solution before I went to a fully persistent on-disk memory mapped solution [1]. SymDel is an algorithm particularly demanding of prog.lang run-time basics as explained in that link. I've come to think of it as a good stress test. { Not the only one, of course :-) } I wonder how well J

I'm glad to see these built-in to Postgres, as these are the basics of fuzzy string matching. A quantum leap would be to integrate an implementation of the symmetric delete algorithm, such as https://github.com/wolfgarbe/SymSpell Soundex and Phonex can yield too many false negatives outside of phonetically English names. Levenshtein/Jaro-Winkler aren't indexable solutions themselves, so they require N^2 comparisons. SymSpell conceptually combines these two into an indexed string-distance solution. It has the usual index issue of being designed for many reads, few writes.

https://github.com/wolfgarbe/SymSpell lists 5 JS implementations (+ a Rust one that compiles to web assembly)

Really great series of articles about spellcheckers. I wish there was a similar project written in Ruby. There is https://github.com/omohokcoj/ruby-spellchecker but it serves a bit different purpose - to do safe autocorrections. Are you considering https://github.com/wolfgarbe/SymSpell algo to do suggestions? If I recall native hunspell suggestions are quite slow - the same algo must be even slower on python.

SymSpell is about 1 million times faster than Norvig's algorithm. https://github.com/wolfgarbe/SymSpell

I mean that if you visit [0] to see the translated version through google, JS throws errors: the sidebar won't render, search won't work and pretty much everything which needs JS doesn't respond. Using the in-place translate doesn't have that issue. About the search part, imagine that in busy days we'll have at most 2-3K active deals, therefore Postgres can do the job really fine with that amount of data. Elastic search was in the table but I think it would be overkill for my use case. Also by having 100% indexable queries performance impact is small. PG provides a lot of goodies to get the job done. You can try to play with it, just type an english word in the search bar eg "game" and then "geme" [0] https://translate.google.com/translate?sl=el&tl=en&u=https%3... [1] https://github.com/wolfgarbe/SymSpell

If you're fuzzily looking up members in a fixed corpus, you can also employ the trick from https://github.com/wolfgarbe/SymSpell, which is essentially just to, for each string in your corpus, enumerate all of the variants of that string with one letter removed, and put both the original string and each variant in a hashtable as a key, with the original string as the value. To do a lookup, you do the same enumeration (original query plus each variant with one letter dropped), and look all of them up in your hashtable. The values you get out of that are a list of all of the strings in your corpus at edit distances 0 or 1, and potentially some at edit distance 2; you can do a subsequent Levenshtein calculation on each to weed out the distance-2 strings, but you only have to do it on this massively reduced set rather than on your whole corpus

Here is the link to the SymSpell Github repository: https://github.com/wolfgarbe/SymSpell An here a benchmark between Norvig's spelling corrector, BK-tree and SymSpell: https://towardsdatascience.com/symspell-vs-bk-tree-100x-fast...

The best way to know is to look at the code itself. https://github.com/wolfgarbe/symspell/blob/master/symspell.c...