zbox

I am building an encrypted file system that runs inside application (https://github.com/zboxfs/zbox). It focus on security, privacy and reliability. The interesting part is it can utilize different storage, such as memory, OS file system, RDBMS and object store. I learned a lot and enjoyed working on this project.

Totally agree. To make file system crash-proof you really need an ACID compatible underlying storage layer or implement it inside the file system itself. In fact, file system and database are quite similar in case of atomicity and consistency. If an embedded ACID component is too heavy, utilizing a reliable database as storage, for example sqlite, might be a good idea. That’s exactly what I did in ZboxFS (https://github.com/zboxfs/zbox) which can use sqlite as a underlying storage to achieve ACID compatible behaviours.

I am using Rust building a privacy focused in-app file system ZboxFS(https://github.com/zboxfs/zbox). The fun part is not only joyfulness Rust bring to me, but also learned a lot about cryptography, file system, transaction control and many more interesting areas.

Database and filesystem share a lot similarity, but still have their own characteristics. To use database as a back-end for filesystem has some benefits, such as reliable persistence, ACID transaction and etc. It reduces complexity to achieve durable storage which filesystem should deal with. Many filesystems are able to use DB as back-end, I also made a filesystem ZboxFS (https://github.com/zboxfs/zbox) which can also use sqlite as underlying storage.

I think the best approach is never save unencrypted data on cloud. Always encrypted on client first. But by that way we lost dedup capability, so we have to do everything, such as encryption, dedup and compression on client side. I made an in-app file system dedicated for that purpose. https://github.com/zboxfs/zbox

Another approach I can think of is using rolling hash (https://en.wikipedia.org/wiki/Rolling_hash) to split file to different size of chunks, those chunks can be saved on different sectors which don’t need to be continuous. The file keeps a list of indexes for all those chunks, just like inode did for blocks. When insert some bytes in the beginning of file, thanks for the rolling hash, most likely only the first several chunks need to be re-chunk and re-hash, thus the change is localised and can be cheaply done, all the rest chunks still untouched. When this combined with append-only storage, things are even easier because it only needs to deal with bytes around that beginning, and the update chunk index list in file. This approach is already implemented in ZboxFS (https://github.com/zboxfs/zbox) to do content-based deduplication.

Totally agree. Log based file system makes things much easier, such as atomic write, data consistency and failure recovery. Along with COW semantics, transaction isolation also becomes easier. By its append-only writing nature, file versioning is trivial as well. Garbage collection is performance downside indeed, but it can be mitigated by COW switching and delayed bulk collection IMHO. Overall, the cost is worthwhile and that is what I am currently doing in ZboxFS: https://github.com/zboxfs/zbox.

There is an userspace implementations of filesystems ZboxFS: https://github.com/zboxfs/zbox.

Nice work!. I also made a similar file system Zbox: https://github.com/zboxfs/zbox. The difference is Zbox is a in-app file system focused on privacy, so FUSE is not supported intentionally. Although it already supports key-value store now, I am currently trying to extend its capability to cloud storage.

Thanks for your praise. To answer your questions: - Where exactly does "zero-knowledge" play a role here? Does this provide steganography? Or, what is meant? As I said in https://github.com/zboxfs/zbox/issues/4, this term was originally for marketing purpose. It means no data or metadata can be leaked to underlying storage, so for an outsider there no knowledge about its content. But it turns out many people think it is misleading to zero knowledge proof in cryptography, so I have already changed it "zero-details". - Why do they use libsodium (excellent choice, but C) instead of a native Rust crypto library? libsodium is a great library, it is fast, sophisticated, mature and has some advanced crypto features I like, such as Argon2 hash and AES nonce extension. I know there are some Rust crypto libraries are great, but overall it is still

Before an entire bikeshed discussion arises over the misused term "zero-knowledge", know that the author is sensitive to this issue and will likely correct it soon [1]. [1] https://github.com/zboxfs/zbox/issues/4