ann-benchmarks

(Blog author here). Thanks for the question. In this case the index for both DiskANN and pgvector HNSW is small enough to fit in memory on the machine (8GB RAM), so there's no need to touch the SSD. We plan to test on a config where the index size is larger than memory (we couldn't this time due to limitations in ANN benchmarks [0], the tool we use). To your question about RAM usage, we provide a graph of index size. When enabling PQ, our new index is 10x smaller than pgvector HNSW. We don't have numbers for HNSWPQ in FAISS yet. [0]: https://github.com/erikbern/ann-benchmarks/

Here you go :) https://github.com/erikbern/ann-benchmarks/pull/451

Try the original data source https://github.com/erikbern/ann-benchmarks

A more important question is why supabase went with one of the slowest [0] implementations instead of building a better plugin especially since you are a VC funded company that's hardly lacking in cash. Simply gluing together a few random open source extensions is not great for developer experience. [0] https://github.com/erikbern/ann-benchmarks

Check out https://github.com/erikbern/ann-benchmarks for some benchmarks on some of the different ANN libraries out there. I'd be interested in hearing other's experiences using these libraries in production.

They used to publish some benchmarks on their site, but seem to have removed them. You can find them on archive.org[1]. I guess it is understandable, since vector search performance is pretty unpredictable, and depends on a lot of factors. If their target market is people who want vector search without needing to read a bunch of papers first, benchmarks might be more confusing than they are helpful. edit: While I do think it's understandable, it's not great for transparency. Even if they don't want to open-source their index, I would admire it if they were willing to give ann-benchmarks[2] an API key to publish some independent results. Disclaimer: I work on vector search at a different company [1] https://web.archive.org/web/20210227105542/https://www.pinec... [2] https://github.com/erikbern/ann-benchmarks

This site keeps track of them: https://github.com/erikbern/ann-benchmarks#glove-100-angular ScaNN is currently SOTA: https://ai.googleblog.com/2020/07/announcing-scann-efficient...

great post and my favorite CS topic, LSH is particularly relevant to machine learning because of its use in indexing of embeddings, which are now omnipresent in ML (from word2vec to transformers to image and graph embeddings etc, etc.) it is now supported in ElasticSearch KNN index (they use HNSWLIB but you can call it a descendant of original LSH in a way) check out ANN benchmarks [0] for comparison of LSH performance to other state of the art methods like proximity graphs/HNSWLIB [1] and quantization/SCANN [2] As an introduction LSH (with MinHash) is also described in detail in the book "Mining Of Massive Datasets", ch.3, "Finding Similar items", highly recommended [3] if you want to play with LSH, python "annoy" library is the best place to start [4] [0] https://github.com/erikbern/ann-benchmarks [1] https://github.com/google-research/

Erik Bernhardsson who was one of original author on Annoy has a pretty developed test suite https://github.com/erikbern/ann-benchmarks.

This is great. This is already better that FLANN (by facebook), Annoy (by spotify). https://github.com/erikbern/ann-benchmarks#evaluated Still, We are very far away from creating reverse video search engine. Also, Most of these techniques are in-memory not on disk. I am really interested in making reverse video search engine.

how does it compare to state-of-the art? (https://github.com/erikbern/ann-benchmarks)

This is a great list; I'm bookmarking it! I would add nmslib to the list: https://github.com/nmslib/nmslib nsmlib is a great alternative to faiss that can be faster in some applications, as shown here: https://github.com/erikbern/ann-benchmarks