uFuzzy

Levenshtein distance is often a poor way to fuzzy match or rank. i suspect that in js, even the trie approach would incur significant GC/alloc thrashing or cost of building a huge trie index. i tried fuzzy matching using a cleverly-assembled regexp approach which works surprisingly well: https://github.com/leeoniya/uFuzzy

I have a soft spot for these JS fuzzy matchers, but there are so many that it’s worth talking about about the specific tradeoffs you chose / ideally offering an interactive comparison like μFuzzy does: https://github.com/leeoniya/uFuzzy

yes, Fuse is extremely slow (and FlexSearch is very fast after indexing is done), but i have no idea how that table presents FlexSearch as having the lowest memory footprint, maybe in its default config? in my testing of fuzzy/substring matching it takes quite a while to create its huge index: https://github.com/leeoniya/uFuzzy?tab=readme-ov-file#perfor...

this is 115 lines of TS, which is pretty lightweight, but some other impt table stakes might be missing that you should be aware of. see all the JS alternatives with their sizes and feature sets and perf: https://github.com/leeoniya/uFuzzy (scroll to bottom) i went down this path a few yrs ago and ended up picking ufuzzy: https://swyxkit.netlify.app/ufuzzy-search

> The conversion to a query is not simply a matter of pulling out the text strings and turning them into AND expressions, although that is part of it. heh, that's kinda what uFuzzy [0] does, but it can also compose a regexp that simulates a Damerau-Levenshtein distance [1] of 1 :) [0] https://github.com/leeoniya/uFuzzy [1] https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_di...

Thank you. We need more libs like that. I just researched the field yesterday and https://github.com/leeoniya/uFuzzy looked pretty good. But there is a gap in the market of such libs. Just few allow to send the whole html document, serialize and deserialize index to be used in browser, highlighting the matches is desired feature. Most importantly very few fuzzy search libs can get a simple substring match as a priority, which is understandable but not helpful. Imagine searching for “xample” and not having “example” among the results.

interesting. i made something much more stupid, but brutally effective: https://github.com/leeoniya/uFuzzy i guess if you wanted results ordered by contact frequency you can just keep the original haystack sorted by frequency, and would get the Richard match first. or keep the frequency in an adjacent lookup and sort/pick the result afterwards. PSA: don't use the ultra popular Fuse.js for fuzzy matching, it's super slow and has pretty terrible match quality. e.g. https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF...

this has now been addressed by 3 additional options that handle all variations of single substitution, single transposition or single deletion. (they're not exposed in the demo form controls yet, but can be enabled via url params): https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF... commit: https://github.com/leeoniya/uFuzzy/commit/63dc67b8bdb7577f85...

there are many ways to be fuzzy, so it's quite subjective. uFuzzy can be made tolerant to extra insertions in the matches between/around the specified needle chars, and can also handle out of order terms. those together cover a surprising amount of common cases. but it's not fuzzy in unlimited ways, such as letter omissions in the match (a superset of substitutions) like a spellcheck or levenshtein distance would be...but extreme tolerance often produces garbage results, too. actually, might be able to handle single-char-per-term omissions as well: https://github.com/leeoniya/uFuzzy/issues/2#issuecomment-126...

this is answered in other parts of the thread here [1], but there might be some hope for it still! https://github.com/leeoniya/uFuzzy/issues/2 however, it's probably infeasible to accomodate more than single-char-per-term substitution tolerance. thankfully, both your examples have 1-char substitutions :) [1] https://news.ycombinator.com/item?id=33042665

I think indexeddb is somewhat orthogonal to this library. The memory efficiency you might get would be that you don't need to hold the whole dataset in memory while running the filter step though at the moment it looks like it assumes you're working with an array in memory (https://github.com/leeoniya/uFuzzy/blob/main/src/uFuzzy.js#L...). That said I suspect there distance between this and something that could search against a stream of data is pretty short.

quality: https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF... perf: https://github.com/leeoniya/uFuzzy#benchmark

Hello HN! I became frustrated with the unpredictible/poor match quality and opaqueness of "relevance scores" in existing fuzzy and fulltext search libs, so I tried something different and this is the result. The main selling point is the result quality / ordering, with best-in-class memory overhead and excellent performance being bonuses. The API is pretty stable at this point, but looking for feedback before committing to 1.0. TL;DR The test corpus is a 4MB json file with 162k words/phrases, so give it a second for initial download. You can also drag/drop your own text/json corpus into the UI to try it against your own dataset. Live demo/compare with a few other libs (there are many more in the codebase, in various states of completion, WIP): https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF... In isolation for perf assessment