casync

I was wondering how this gets any common chunks at all with the removed file boundaries. Turns out that chunks don't have a set size, just min/max/avg values, so unaligned streams may end up synchronizing. https://github.com/systemd/casync/blob/master/src/cachunker.... If I understood that correctly, that's pretty cool. But looking at the code I'm having strong "nope" feelings. First, because of lines like "q += m, n -= m;". Second, because of int/enum/semantic abuse: `compression_type` may be _CA_COMPRESSION_TYPE_INVALID which I hope is -1, `>= 0` as a known compression type, or `-EAGAIN` as an error. (from https://github.com/systemd/casync/blob/99559cd1d8cea69b30022... ) I'd bet that just throwing afl at the decompressor will find issues :( (source - I had this feeling about systemd-resolved and threw afl at it, found issues) I do like

would casync [1] fit in this list? [1] https://github.com/systemd/casync/

Casync is a much more efficient protocol due to the rolling hash support: https://github.com/systemd/casync/

There are some programs that do this: https://github.com/systemd/casync . See the readme for similar tools.

I'm curious to hear what solution they are proposing for this, especially since there are at least two existing proposals in that field, in the form of OStree and casync: https://github.com/ostreedev/ostree https://github.com/systemd/casync/

zchunk, zsync, and casync spring to mind. zchunk and zsync require support for range queries but casync splits the chunks into multiple files. https://github.com/zchunk/zchunk http://zsync.moria.org.uk/ https://github.com/systemd/casync