BLAKE3-specs

This is a difference between BLAKE3 and most other hash functions. In the usual arrangement ("Merkle–Damgård"), each block depends on the previous one, so the only way to verify some "slice" of the input is to re-hash the whole thing. But when you arrange the input into a tree shape (a "Merkle tree") instead, suddenly the right half of the tree does not depend on the left half until the very last step at the very top. If you give me the input to that last step, I can verify that it matches the root hash that I know, now I have the hashes ("chaining values") I'd need to verify either the left half or the right half without the other. Then I do our favorite trick in computer science, which is to recursively apply that same procedure all the way down, until I have an efficient "path" to whatever part of the tree I actually care about. For mo

https://github.com/Cyan4973/xxHash/issues/450 This person's testing has blake3 at 250 cycles for small hashes. It's not a thousand times slower. And for a lot of uses, 250 cycles is plenty fast. That's like one uncached memory access. That test does mention batching but I don't think it makes a particularly large difference, the paper shows 4 cycles per byte for 64 byte inputs and I don't think they're getting fancy: https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak...

Figure 4 from https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak... is a related benchmark. On that particular machine we saw good scaling up to 16 threads, but yeah somewhere in that neighborhood you start to run into memory bandwidth issues. Which is the problem you want I guess :)

We put a lot of effort into section 5.1.2 of https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak..., and the complicated part of BLAKE3 (incrementally building the Merkle tree) ends up being ~4 lines of code. Let me know what you think.

We go over some of our reasoning around that in section 7.5 of https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak.... An early BLAKE3 prototype actually didn't include the chunk counter (https://github.com/oconnor663/bao/blob/master/docs/spec_0.9....), so I'm definitely sympathetic to the use cases that wish it wasn't there. However, after publication we found out that something like a chunk counter is necessary for the security of the Bao streaming verification tool: https://github.com/oconnor663/bao/issues/41. It could be that there's a design that's the best of both worlds, but I'm not sure.

It's hard to give a short answer to that question :) - Yes, if you know your input is short, you can use a smaller state. The limit is roughly a BLAKE2s state plus (32 bytes times the log_2 of the number of KiB you need to hash). Section 5.4 of https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak... goes into this. - But it's hard to take advantage of this space optimization, because no libraries implement it in practice. - But the reason libraries don't implement it is that almost no one needs it. The max state size is just under 2 KiB, which is small enough even for https://github.com/oconnor663/blake3-6502. - But it would be super easy to implement if we just put the "CV stack" on the heap instead of allocating the whole thing as an array up front. - But the platforms that care about this don't have a heap. @caesarb mentioned r

It might be easier to think about it as a stack, rather than a tree. Each element of the stack represents a subtree -- a perfect binary tree. If you ever have two subtrees of height k, you merge them together into one subtree of height k+1. Your stack might already have another subtree of height k+1; if so, you repeat the process, until there's at most one subtree of each height. This process is isomorphic to binary addition. Worked example: let's start with a single leaf, i.e. a subtree of height 0. Then we "add" another leaf; since we now have a pair of two equally-sized leaves, we merge them into one subtree of height 1. Then we add a third leaf; now this one doesn't have a sibling to merge with, so we just keep it. Now our "stack" contains two subtrees: one of height 1, and one of height 0. Now the isomorphism: we start with the binar

> That's for 16KiB inputs. BLAKE3 needs 16 KiB of input to hit the numbers in that bar chart, but BLAKE2s doesn't. It'll maintain its advantage over SHA-256 all the way down to the empty string. You can see this in Figure 3 of https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak.... (BLAKE3 is also faster than SHA-256 all the way down to the empty string, but not by as large a margin as the 16 KiB measurements suggest.) On the other hand, these measurements were done on machines without SHA-256 hardware acceleration. If you have that (and Intel chips from the past year do), then SHA-256 does a lot better of course.

BLAKE3 is written in Rust wile BLAKE2s is written in C. That is probably a lot easier to push through. It also seems to be almost as fast according to the BLAKE3 specs PDF (page 11). https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak...

Its section 2.1 of the paper: https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak... Though, note, blake3 still provides enhanced resistance against the attacks against blake2 even in the case where you only have one block, due to the change in how the fundamental hashing primitive is used.

Speed comparison from the BLAKE3 authors: https://github.com/BLAKE3-team/BLAKE3/ I hadn't heard of BLAKE3 before (I thought BLAKE2b was the latest and greatest), judging by commits from the repositories of that user it's about half a year old now. Paper's abstract starts with: > BLAKE3 [is] an evolution of the BLAKE2 cryptographic hash that is both faster and also more consistently fast across different platforms and input sizes. BLAKE3 supports an unbounded degree of parallelism, using a tree structure that scales up to any number of SIMD lanes and CPU cores. On Intel Cascade Lake-SP, peak single-threaded throughput is 4× that of BLAKE2b, 8× that of SHA-512, and 12× that of SHA-256, and it can scale further using multiple threads. BLAKE3 is also efficient on smaller architectures [like] 32-bit ARM1176 In the introduction, the reason for

Take a look at Figure 5 (https://i.imgur.com/Izs23wf.png) in the spec (https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak...). That benchmark was done on a Raspberry Pi Zero, which is a 32-bit ARM1176.

The Performance section of the spec (https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak...) has a paragraph about short input performance. 64 bytes happens to be the BLAKE3 block size, and performance at that length or shorter is best in class. Look at the left edge of Figure 3 (https://i.imgur.com/smGHAKA.png).

From their BLAKE3 paper (https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blak...) they give an example of verified streaming in 6.4. Basically to verify a video file using serial hash functions you need to download the entire video file before you can perform the hashing. In BLAKE3 you can verify each chunk of the video as it is being streamed because the hash internally is just a Merkle Tree.