COComputerGuru·May 3, 2024
I guess this is an answer (not in answer, mind!) to the GitHub issue I opened against SQLite-vss a couple of months ago?
https://github.com/asg017/sqlite-vss/issues/124
In I’m writing a new vector search SQLite Extensionasg017
sqlite-vss
C++2023–2024
asg017 / sqlite-vss
sqlite-vss
A SQLite extension for efficient vector search, based on Faiss!
14Distinct people
17Mentions
0Explicit recommenders
2KGitHub stars
What to know before exploring
What it is
A SQLite extension for efficient vector search, based on Faiss!
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month17 dated mentions
Observed mentions: 2023-03-26 03:40:12 UTC to 2024-05-03 05:58:17 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Mar 2023 | 2023-03-01 00:00:00 UTC | 2023-04-01 00:00:00 UTC | 1 | 0 | 1 |
| Apr 2023 | 2023-04-01 00:00:00 UTC | 2023-05-01 00:00:00 UTC | 3 | 2 | 1 |
| May 2023 | 2023-05-01 00:00:00 UTC | 2023-06-01 00:00:00 UTC | 2 | 1 | 1 |
| Jun 2023 | 2023-06-01 00:00:00 UTC | 2023-07-01 00:00:00 UTC | 2 | 1 | 1 |
| Jul 2023 | 2023-07-01 00:00:00 UTC | 2023-08-01 00:00:00 UTC | 1 | 0 | 1 |
| Aug 2023 | 2023-08-01 00:00:00 UTC | 2023-09-01 00:00:00 UTC | 0 | 0 | 0 |
| Sep 2023 | 2023-09-01 00:00:00 UTC | 2023-10-01 00:00:00 UTC | 1 | 1 | 0 |
| Oct 2023 | 2023-10-01 00:00:00 UTC | 2023-11-01 00:00:00 UTC | 1 | 0 | 1 |
| Nov 2023 | 2023-11-01 00:00:00 UTC | 2023-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2023 | 2023-12-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2024 | 2024-01-01 00:00:00 UTC | 2024-02-01 00:00:00 UTC | 2 | 0 | 2 |
| Feb 2024 | 2024-02-01 00:00:00 UTC | 2024-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2024 | 2024-03-01 00:00:00 UTC | 2024-04-01 00:00:00 UTC | 1 | 0 | 1 |
| Apr 2024 | 2024-04-01 00:00:00 UTC | 2024-05-01 00:00:00 UTC | 2 | 0 | 2 |
| May 2024 | 2024-05-01 00:00:00 UTC | 2024-06-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
17 sourcesPerhaps sqlite-vss? It adds vector searches to sqlite.
https://github.com/asg017/sqlite-vss
In Embeddings are a good starting point for the AI curious app developerRe storing vectors in BLOB columns: ya, if it's not a lot of data and it's fast enough for you, then there's no problem doing it like that. I'd even just store then in JSON/npy files first and see how long you can get away with it. Once that gets too slow, then try SQLite/redis/valkey, and when that gets too slow, look into pgvector or other vector database solutions.
For SQLite specifically, very large BLOB columns might effect query performance, especially for large embeddings. For example, a 1536-dimension vector from OpenAI would take 1536 * 4 = 6144 bytes of space, if stored in a compact BLOB format. That's larger than SQLite default page size of 4096, so that extra data will overflow into overflow pages. Which again, isn't too big of a deal, but if the original table had small values before, then table scans can be slower.
One solu
Read the rest
tion is to move it to a separate table, ex on an original `users` table, you can make a new `CREATE TABLE users_embeddings(user_id, embedding)` table and just LEFT JOIN that when you need it. Or you can use new techniques like Matryoshka embeddings[0] or scalar/binary quantization[1] to reduce the size of individual vectors, at the cost of lower accuracy. Or you can bump the page size of your SQLite database with `PRAGMA page_size=8192`.
I also have a SQLite extension for vector search[2], but there's a number of usability/ergonomic issues with it. I'm making a new one that I hope to release soon, which will hopefully be a great middle ground between "store vectors in a .npy files" and "use pgvector".
Re "do embeddings ever expire": nope! As long as you have access to the same model, the same text input should give the same embedding output. It's not like LLMs that have temperatures/meta prompts/a million other dials that make outputs non-deterministic, most embedding models should be deterministic and should work forever.
[0] https://huggingface.co/blog/matryoshka
[1] https://huggingface.co/blog/embedding-quantization
[2] https://github.com/asg017/sqlite-vss
Has anyone used sqlite for storing embeddings? Are there any extensions or tips for making it easier?
I have a small command line python app that uses sqlite for a db. Postgres would be a huge overkill for the app
PS: is sqlite-vas good? https://github.com/asg017/sqlite-vss
In Pg_vectorize: Vector search and RAG on PostgresBy "edge", I was talking about mobile / IOT devices.
The closest I can see is the VSS extension[1] for Sqlite.
[1]: https://github.com/asg017/sqlite-vss
In Are we at peak vector database?Here is a SQLite extension that uses Faiss under the hood.
https://github.com/asg017/sqlite-vss
Not associated with the project, just love SQLite and find it very useful.
In Every database will become a vector database sooner or laterSQLite-Vss: A SQLite Extension for Vector Search
4 points · Open discussionI had a go at one of those a few months ago: https://datasette.io/plugins/datasette-faiss
Alex Garcia built a better one here as a SQLite Rust extension: https://github.com/asg017/sqlite-vss
In The Problem with LangChainSince the problem domain is that of anomaly detection from constructed request feature embeddings, I wonder if an ANN-search methodology using an embedded database (such as https://github.com/asg017/sqlite-vss or similar) was explored.
In Every request, every microsecond: scalable machine learning at CloudflareSQLite extension for efficient vector search, based on Faiss
1 points · Open discussionSQLite extension for efficient vector search, based on Faiss
4 points · Open discussionI think I remember there being a recently created faiss module for sqlite. It may have been this one https://github.com/asg017/sqlite-vss
In Ask HN: Vector Database on Top of SQLite?Author here, happy to answer any questions! My "Introducing sqlite-vss: A SQLite Extension for Vector Search" [0] blog post has more details about the motivations behind this extensions, along with a demo. Includes a semantic search engine hosted on Datasette, build with sqlite-vss and sentence-transformers, on fly.io. Simple and cheap!
Though my favorite part about this extension (and all my other SQLite extensions[1]): you can `pip install sqlite-vss` for Python, `npm install sqlite-vss` for Node.js, or use https://deno.land/x/sqlite_vss for Deno! The underlying SQLite extension has been compiled on different platforms and uploaded to pip/npm/deno.land/x to make distribution easier. There's also pre-compiled extensions on each Github release if you just want to use it with the sqlite3 CLI.
Currently working on getting Mac M1 arm builds,
Read the rest
hopefully by end of this week [2]
[0] https://observablehq.com/@asg017/introducing-sqlite-vss
[1] https://github.com/asg017/sqlite-ecosystem
[2] https://github.com/asg017/sqlite-vss/issues/13
SQLite Extension for Efficient Vector Search
7 points · Open discussionShowing the 15 most recent of 17 sources. Enable JavaScript to search and browse every indexed mention.