duckdb-sqlite

You can use DuckDB WASM independently of Pyodide and can extend it with SQLite. Though it seems to be somewhat limited. I couldn't even check what version it has, since sqlite_version() was missing. Version in the repository [1] is 3.38.1, which is from quite a ways ago. At the moment DuckDB web shell can't load SQLite extension, since that hasn't been released for yesterday's 1.1.2. Earlier version does work using recently updated WASM edition. That can be extended with spatial including GDAL, vector search etc [2]. Making your own "SQL web shell" wasn't too hard, though docs weren't quite complete enough for me. [1] https://github.com/duckdb/sqlite_scanner/blob/main/src/sqlit... [2] https://github.com/duckdb/duckdb-wasm/releases/tag/v1.29.0

Nice! Might this pave the way for an ADBC_scanner extension since I imagine the adbc and nanoarrow dependencies will be built in!? Similar to postgres_scanner, sqlite_scanner, odbc_scanner https://github.com/duckdblabs/postgres_scanner https://github.com/duckdblabs/sqlite_scanner https://github.com/rupurt/odbc-scanner-duckdb-extension

It's not a dumb question at all. I'm pretty knowledgeable with DBs and still find it very difficult to understand how many of these front-end/pass-through engines work. Checkout Postgres Foreign Data Wrappers. That might be the most well known approach for accessing one database through another. The Supabase team wrote an interesting piece about this recently. https://supabase.com/blog/postgres-foreign-data-wrappers-rus... You might also want to try out duckdb's approach to reading other DBs (or DB files). They talk about how they can "import" a sqlite DB in the above 0.7.0 announcement, but also have some other examples in their duckdblabs github project. Check out their "...-scanner" repos: https://github.com/duckdblabs/postgres_scanner https://github.com/duckdblabs/sqlite_scanner

We have one of those! :-) And yes it is fast! https://github.com/duckdblabs/sqlite_scanner

DuckDB can actually read SQLite or Postgres directly! In the SQLite case, something like Litestream plus DuckDB could work really well! Also, with Pyarrow's help, DuckDB can already do this with Delta tables! https://github.com/duckdblabs/sqlite_scanner https://github.com/duckdblabs/postgresscanner

Excellent question! I'll jump in - I am a part of the DuckDB team though, so if other users have thoughts it would be great to get other perspectives as well. First things first - we really like quite a lot about the SQLite approach. DuckDB is similarly easy to install and is built without dependencies, just like SQLite. It also runs in the same process as your application just like SQLite does. SQLite is excellent as a transactional database - lots of very specific inserts, updates, and deletes (called OLTP workloads). DuckDB can also read directly out of SQLite files as well, so you can mix and match them! (https://github.com/duckdblabs/sqlitescanner) DuckDB is much faster than SQLite when doing analytical queries (OLAP) like when calculating summaries or trends over time, or joining large tables together. It can use all of your CPU cor