hachoir

tool.c: https://github.com/Cisco-Talos/clamav/blob/main/sigtool/sigt... clamav sigtool: https://www.google.com/search?q=clamav+sigtool https://blog.didierstevens.com/2017/07/14/clamav-sigtool-dec... : sigtool –-find-sigs "$name" | sigtool –-decode-sigs List of file signatures: https://en.wikipedia.org/wiki/List_of_file_signatures And then also clusterfuzz/oss-fuzz scans .txt source files with (sandboxed) Static and Dynamic Analysis tools, and `debsums`/`rpm -Va` verify that files on disk have the same (GPG signed) checksums as the package they are supposed to have been installed from, and a file-based HIDS builds a database of file hashes and compares what's on disk in a later scan with what was presumed good, and ~gdesktop LLM tools scan every file, and there are extended filesystem attributes for label-based MAC systems like SELinux, oh and NTFS ADS. A sufficient cryptographic hash function yields random bits with uniform probability. DRBG Deterministic Random Bit Generators need high entropy random bits in order to continuously re-seed the RNG random number generator. Is it safe to assume that hashing (1) every file on disk, or (2) any given file on disk at random, will yield random bits with uniform probability; and (3) why Argon2 instead of e.g. only two rounds of SHA256? https://github.com/google/osv.dev/blob/master/README.md#usin... : > We provide a Go based tool that will scan your dependencies, and check them against the OSV database for known vulnerabilities via the OSV API. ... With package metadata, not (a file hash, package) database that could be generated from OSV and the actual package files instead of their manifest of already-calculated checksums. Might as well be heating a pool on the roof with all of this waste heat from hashing binaries build from code of unknown static and dynamic quality. Add'l useful formats: > Currently it is able to scan various lockfiles, debian docker containers, SPDX and CycloneDB SBOMs, and git repositories Things like bittorrent magnet URIs, Named Data Networking, and IPFS are (file-hash based) "Content addressable storage": https://en.wikipedia.org/wiki/Content-addressable_storage

liked how Kaitai Struct allows easy definition of constraints (simple ones at least) into the file format specification so that you can say "this section of the file shall have a size not exceeding header.length * 2 bytes". Some alternative binary file format specification attempts for those interested in seeing alternatives, each with their own set of problems/pros/cons: 1. 010 Editor [4] 2. Synalysis [5] 3. hachoir [6] 4. DFDL [7] [1] https://github.com/kaitai-io/kaitai_struct/issues/158 [2] https://github.com/kaitai-io/kaitai_struct/issues/156 [3] https://raw.githubusercontent.com/davidhicks/kaitai_struct_f... [4] https://www.sweetscape.com/010editor/repository/templates/ [5] https://github.com/synalysis/Grammars [6] https://github.com/vstinner/hachoir/tree/main/hachoir/parser [7] https://github.com/DFDLSchemas/