datasette-lite

I think there may be a space for super-large multi-GB files served from static storage being accessible from SQlite as well. Another one would be this full-text search of a 43GB SQLite database of Wikipedia's full text search: http://static.wiki/ . Hearing there's official support for this is awesome and I hope they also might add some nice stuff for those sticking with POSIX/Emscripten as well; maybe some optimizations to access patterns or other stuff like indexing or split DBs? There are some amazing things for SQLite in the browser especially if you're looking for ways to host queryable data for cheap. The example I have below costs $0.42 cents a month to host. For 28GB. Insane. I have a hacked up POC experimental version of the datasette-lite Python UI which runs in Emscripten to be able to look at multi-GB databases at https://githu

There's a web browser version of datasette called datasette-lite which runs on Python ported to WASM with Pyodide which can load SQLite databases. I grafted the enhanced lazyFile implementation from emscripten and then from this implementation to datasette-lite relatively recently as a curious test. Threw in a 18GB CSV from CA's unclaimed property records here https://www.sco.ca.gov/upd_download_property_records.html into a FTS5 Sqlite Database which came out to about 28GB after processing: POC, non-merging Log/Draft PR for the hack: https://github.com/simonw/datasette-lite/pull/49 You can run queries through to datasette-lite if you URL hack into it and just get to the query dialog, browsing is kind of a dud at the moment since datasette runs a count(*) which downloads everything. Example: https://datasette-lite-lab.mindflakes.com/index.

See also https://github.com/simonw/datasette-lite: Datasette running in your browser using WebAssembly and Pyodide

Close https://github.com/simonw/datasette-lite

Oh wow! I wonder how hard it would be to load that module into https://github.com/simonw/datasette-lite Hah, turns out I have an issue for that already, which I'd forgotten about: https://github.com/simonw/datasette-lite/issues/30

I have an open issue for that here: https://github.com/simonw/datasette-lite/issues/28 My initial hunch is that this will be really difficult - probably require a fork of something like https://github.com/coleifer/pysqlite3 then compiled for WebAssembly. I'm confident it's feasible, but I don't have the skills to figure it out myself.