PHphiresky·Aug 1, 2022
I did actually previously write a VFS, though it did something else entirely:
https://phiresky.github.io/blog/2021/hosting-sqlite-database...
You're right that a comparison to compression VFS is completely missing. I knew about their existence but I have to admit I didn't look too closely.
Note that the main "novelty" here is training a shared / partitioned compression dictionary specifically to take advantage of redundancy that wouldn't appear within only one row or even within a database page / block . The compression happens at the row level and can additionally use application knowledge - you couldn't do that in the VFS level. For example, you can have separate compression dictionaries per different columns and per groups of rows with some commonality.
I'll have to compare to a compression vfs (do you have a favorite?) and see if mayb
Read the rest
e these two methods can even be combined.
Edit: I see that https://github.com/mlin/sqlite_zstd_vfs does actually also train dictionaries. It's still at the database-page level so can't take application knowledge into account or compress only parts of the data, but that's still pretty smart then.