meow_hash

See also: “Meow hash” by the legendary Casey Muratori, a fast non-cryptographic hash function: https://github.com/cmuratori/meow_hash > It’s the fastest hash function we know of, and we have benchmarked all the ones we could find. On modern Intel x64 CPUs, it hashes 16 bytes per cycle single-threaded. This means in cache it can hash at a rate of 64 gigabytes per second on a 4.2gHz machine. Out of cache, it hashes at whatever speed your main memory bus can provide to a single core, since that is usually the limiting factor on modern x64 CPUs. > It has also now been tuned to be the fastest hash on small inputs, too. Despite the fact that it is a full 128-bit hash, it still outperforms “fast” 64-bit hashes across all input sizes. — https://archive.is/CQOVm (originally https://mollyrocket.com/meowhash) Discussion on HN: https://news.ycombina

The AES instruction thing has already been done in MeowHash. Not sure how it compares. https://github.com/cmuratori/meow_hash

A collision was reported a couple months ago https://github.com/cmuratori/meow_hash/issues/80

for everyone who doesn't think highly of Casey Muratori (or at least the way he conducts himself online), the author of Meow Hash, he took the criticism quite graciously: https://twitter.com/cmuratori/status/1417546500083568641 https://github.com/cmuratori/meow_hash/issues/80 all of this is very interesting reading for someone like me who doesn't know very much about cryptography beyond the surface level!

This is a really great writeup. Amazing visualizations! Don't miss the issue report on the repo (basically, "here are two images that happen to collide???"): https://github.com/cmuratori/meow_hash/issues/80

Also in the super fast, but not designed to be cryptographically secure category: xxhash (https://github.com/Cyan4973/xxHash) with 32/64 bit output. The latest version, xxh3, supports up to 128 bit output. meow hash (https://github.com/cmuratori/meow_hash) The recently released Blake3 which is designed to be cryptographically secure is very fast also (https://github.com/BLAKE3-team/BLAKE3)

I helped design Meow Hash using AES-NI. It is not general purpose crypto strength, but ridiculously fast, targeting a theoretical performance of 16 bytes per cycle on some processors, too fast for memory to keep up. https://github.com/cmuratori/meow_hash

Algorithms using AES intrinsics are generally fastest at all key sizes now. For small keys, the AquaHash[0] algorithm is extremely fast, and several algorithms (e.g. MeowHash[1] and the aforementioned AquaHash) are essentially at the hardware limits for large keys. Algorithms like wyhash are useful for things like embedded CPUs without AES support, but if you are targeting x86/ARM then AES is often a better choice. [0] https://github.com/jandrewrogers/AquaHash [1] https://github.com/cmuratori/meow_hash

I just ran a simple collision test against it with real life, prod data. It resulted in 13 collisions on 116275 files in 512b mode. https://github.com/cmuratori/meow_hash/issues/7#issuecomment...

https://github.com/cmuratori/meow_hash/blob/master/meow_hash... What do AESRotate(foo,bar) and AESMerge(foo,bar) do? [edit] So it's a simple cyclic rotation preceded by A.Q0 = _mm512_aesdec_epi128(A.Q0, B.Q0) From Googling, _mm512_aesdec_epi128 looks like an Intel processor built-in.

Almost certainly faster than any of the ones you list for large data. I would expect the perf to be more comparable to the other aesni hashes (t1ha and falkhash), both which bench around 4 bytes/ cycle while meow claims 16 bytes/cycle. Meow does include implementation for smhasher, so benchmarking should be easy enough https://github.com/cmuratori/meow_hash/blob/master/meow_smha...