sql.js-httpvfs

Don't miss how this works. It's not a server-side application - this code runs entirely in your browser using SQLite compiled to WASM, but rather than fetching a full 22GB database it instead uses a clever hack that retrieves just "shards" of the SQLite database needed for the page you are viewing. I watched it in the browser network panel and saw it fetch: https://hackerbook.dosaygo.com/static-shards/shard_1636.sqlite.gz https://hackerbook.dosaygo.com/static-shards/shard_1635.sqlite.gz https://hackerbook.dosaygo.com/static-shards/shard_1634.sqlite.gz As I paginated to previous days. It's reminiscent of that brilliant SQLite.js VFS trick from a few years ago: https://github.com/phiresky/sql.js-httpvfs - only that one used HTTP range headers, this one uses sharded files instead. The interactive SQL query interface at https://hackerb

correct! it's a sqlite database which ends up being automatically gzip'd by the github cdn aka fastly - this ends up being ~4mb over the wire obviously this solution doesn't scale (versus something like https://github.com/phiresky/sql.js-httpvfs ) but sometimes it's fun to "just" send the sqlite database to the frontend and be done :-)

From "Show HN: TeaTime – distributed book library powered by SQLite, IPFS and GitHub" https://news.ycombinator.com/item?id=42264274 : >> phiresky/sql.js-httpvfs: https://github.com/phiresky/sql.js-httpvfs >> mmomtchev/sqlite-wasm-http: https://github.com/mmomtchev/sqlite-wasm-http >> This project is inspired from @phiresky/sql.js-httpvfs but uses the new official SQLite WASM distribution duckdb/duckdb-wasm: https://github.com/duckdb/duckdb-wasm

There are implementations for that: For example https://github.com/psanford/sqlite3vfshttp or https://github.com/phiresky/sql.js-httpvfs