sqlite-vec

complex beast: I am using PyTauri (https://github.com/pytauri/pytauri - Python bindings for Tauri). Python is used to import files, run the dimensionality reduction algorithms (t-SNE, UMAP etc.) and for most database logic (using SQLite3). The CLIP model (JINA CLIP v1) runs as ONNX model using the Rust crate ORT (https://ort.pyke.io/) - inference supports right now only CPU. The semantic search is powered by the SQLite3 extension https://github.com/asg017/sqlite-vec which just adds support for optimized brute-force based nearest neighbor vector search (no approximate vector index). So Digger Solo is not meant for millions of files, more like a few 100 thousands. Happy to answer questions!

PyTauri (https://github.com/pytauri/pytauri - Python bindings for Tauri with a Rust backend with JS frontend). Python is used to import files, run the dimensionality reduction algorithms (t-SNE, UMAP etc.) and for most database logic (using SQLite3). The CLIP model (JINA CLIP v1) runs as ONNX model using the Rust crate ORT (https://ort.pyke.io/) - inference supports right now only CPU. The semantic search is powered by the SQLite3 extension https://github.com/asg017/sqlite-vec which just adds support for optimized brute-force based nearest neighbor vector search (no approximate vector index). So Digger Solo is not meant for millions of files, more like a few 100 thousands. The installer contains all model files used (text, vision and audio encoders) that is why the download size is so big - I did not want to have additional downloads after installation. The Windows version had problems running the Python modules reliably for some customers (working on a fix). So I recommend trying the free version first and see if file importing works before purchasing a key.

arch be used to achieve this? Geoffrey's design is so simple it doesn't even need search - all it does is dump in context that's been stamped with a date, and there are so few tokens there's no need for FTS or vector search. If you wanted to build something more sophisticated you could absolutely use those. SQLite has surprisingly capable FTS built in and there are extensions like https://github.com/asg017/sqlite-vec for doing things with vectors.

ch as LangChain, LlamaIndex and txtai (disclaimer I'm the primary author of https://github.com/neuml/txtai) that handle generating embeddings locally or with APIs and storing them in databases such as Postgres.

edding float[8]) This feels like a promising new product area for rqlite -- super easy to use, network accessible Vector Search, all in a highly available package if you want that. I'm looking for any and all feedback, as this is a very new area for rqlite (SQLite extensions, and new types of search). [1] https://rqlite.io [2] https://www.philipotoole.com/rqlite-8-27-loadable-sqlite-ext... [3] https://www.philipotoole.com/rqlite-828-enhanced-docker-supp... [4] https://github.com/asg017/sqlite-vec