squashfs-tools-ng

ll in a Windows DOS box, on an existing, German Windows 7 installation was an adventure on it's own and ended up breaking things in different ways than German text. Turns out, trying to write language agnostic command line applications on Windows is a PITA.

of the moment you might write something angry and threatening. But certainly not if you've had a few weeks time to calm down and think things over. Still publicly spewing vitriol at this point will certainly make me loose some of that respect again. > And you plagiarized part of his readme. https://github.com/plougher/squashfs-tools/blob/master/RELEA... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... https://github.com/AgentD/squashfs-tools-ng/blob/master/READ... Oh yes? Which part? > ... calling it spaghetti code (which isn't immediately verifiable) Here you go, have fun: https://github.com/plougher/squashfs-tools/blob/master/squas... However, I cannot blame anyone here, I totally get how those things happen and have witnessed it myself in action: You write a simple tool supporting a larger project. It's written by the seat of your pants without much planning, since it's not big and does one simple job. Then it gets used in production, eventually requirements change, other people pile on patches, but try to keep the diff small, so it's reviewable. It keeps growing and growing, getting more and more complex, but receives maybe a little less care than the actual project it supports. Nobody bothers to overhaul it or write documentation because, hey, it works, and any large changes might risk breaking things. Even if nobody is to blame for it, the end result is still the same: an undocumented mess that is hard to wrap your head around if you aren't the original author, who is the only one with the bigger picture. I tried for roughly a week to pull the code (there are some more files than this and some of the inter dependencies are nasty) apart into stacked utility libraries and a pure command line parsing front end, with the hopes to maybe get this upstream once it is done. I gave up and decided that at this point I understood enough about the format to start afresh and not touch what I believed to be unmaintained anyway.

tter name simply named it "*-ng". Other people seemed to have drawn similar conclusions[2][3]. We all later found out that the squashfs-tools git tree was simply moved to GitHub (without updating the kernel docs or "official" website to indicate this). But even there, activity fixing bugs only picked up after I already made 3 or 4 releases and announced them on the squashfs mailing list. At which point I finally got a response in the form of a few somewhat unfriendly words[5][6][7] (IMO to some extent understandable, but still...). But after that, squashfs-tools version 4.4 fixed a bunch of issues that I publicly complained about and this release got apparently quite busy copying features that I found useful (and set squashfs-tools-ng apart, e.g. converting tarballs directly). Given the distinct source trees and, lets call it "competitive nature", I'd say a merge is unlikely and I will continue maintaining my package. I wrote it to solve some specific problems I had and it IMO does that job quite well. It's nice if other people find it useful and I'm also proud to have ended up indirectly contributing to the official squashfs-tools development by providing new motivation :-). [1] https://sourceforge.net/p/squashfs/mailman/message/36298345/ [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931965 [3] https://news.ycombinator.com/item?id=20518019 [4] https://lkml.org/lkml/2019/8/1/1154 [5] https://github.com/AgentD/squashfs-tools-ng/issues/10 [6] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931965#29 [7] https://lkml.org/lkml/2019/8/1/565

is: https://github.com/AgentD/squashfs-tools-ng/blob/master/incl... This is usually my go-to-solution for working with low level on-disk or on-the-wire binary data structures that demand a specific endianness. In C I use "load/store" style functions that memcpy the data from a buffer into a struct instance and do the endian swapping (or reverse for the store). The copying is also necessary because the struct in the buffer may not have proper alignment. Technically, the giant macro of doom in the article takes care of all of this as well. But unlike the article, I would very much not recommend hacking up your own stuff if there are systems libraries readily available that take care of doing the same thing in an efficient manner. In C++ code, all of this can of course be neatly stowed away in a special class with overloaded operators that transparently takes care of everything and "decays" into a single integer and exactly the above code after compilation, but is IMO somewhat cleaner to read and adds much needed type safety.

" with Unix. [1] https://github.com/AgentD/squashfs-tools-ng/tree/master/lib/... [2] https://www.freebsd.org/cgi/man.cgi?query=tar&sektion=5 [3] https://mgorny.pl/articles/portability-of-tar-features.html [4] https://www.spinics.net/lists/git/msg363049.html

ashfs-tools-ng/blob/master/doc/... [3] https://github.com/AgentD/squashfs-tools-ng/blob/master/doc/...

~2GiB as XZ compressed SquashFS image). By playing around a bit, I also realized that the compressed meta data is "amazingly small", compared to the actual image file data and the resulting images are very close to the tar ball compressed with the same compressor settings. I can accept a claim of being a little smaller than SquashFS, but the claimed difference makes me very suspicious. From the README, I'm not quite sure: Does the Raspbian image comparison compare XZ compression against SquashFS with Zstd? I have cloned the git tree and installed dozens of libraries that this folly thingy needs, but I'm currently swamped in CMake errors (haven't touched CMake in 8+ years, so I'm a bit rusty there) and the build fails with some still missing headers. I hope to have more luck later today and produce a comparison on my end using my trusty Debian reference image which I will definitely add to my existing benchmarks. Also, is there any documentation on how the on-disk format for DwarFS and it's packing works which might explain the incredible size difference? [1] https://github.com/AgentD/squashfs-tools-ng [2] https://github.com/AgentD/squashfs-tools-ng/blob/master/doc/... [3] https://github.com/AgentD/squashfs-tools-ng/tree/master/doc