vectordb

It is faster! We needed a low latency, on premise solution that we can run on edge nodes with sane defaults that anyone in the team can whim in a sec. Also worth noting is that our use case is end to end retrieval of usually few hundred to few thousand chunks of text (for example in Kagi Assistant research mode) that need to be processed once at run time with minimal latency. Result is this. We periodically benchmark the performance of different embeddings to ensure best defaults: https://github.com/kagisearch/vectordb#embeddings-performanc...

https://github.com/kagisearch/vectordb/blob/453bb658bb710838... Looks like it uses one of these, depending on your settings: Fast model: google/universal-sentence-encoder/4 Multilingual model: universal-sentence-encoder-multilingual-large/3 Normal model (Alternative): BAAI/bge-small-en-v1.5 Best model: BAAI/bge-base-en-v1.5

The library is open source. Here's where you can see how they're creating the embeddings. https://github.com/kagisearch/vectordb/blob/main/vectordb/em...

The README.md example just has it in memory. Looking at the source, the Storage class uses a plain file and python's pickle module: https://github.com/kagisearch/vectordb/blob/main/vectordb/st...

If you are looking for lightweight, low- latency, fully local, end-to-end solution (chunking, embedding, storage and vector search), try vectordb [1] Just spent a day updating it with latest benchmarks for text embedding models. [1] https://github.com/kagisearch/vectordb

Any database can be used to extend the memory of LLMs. What a database does is store stuff and lets you search/retrieve stuff. Embeddings are differet form of data that are in many (but not all) cases superior to searching through text. You do not need a fancy cloud hosted service to use an embeddings database like you do not need one to use a regular databse (although you could). Check https://github.com/kagisearch/vectordb for a simple implementation of a vector search database that uses local, on-premise open source tools and lets you use an embeddings database in 3 lines of code.