n2

In n2[1] I needed a fast tokenizer and had the same "garbage factory" problem, which is basically that there's a set of constant tokens (like json.Delim in this post) and then strings which cause allocations. I came up with what I think is a kind of neat solution, which is that the tokenizer is generic over some T and takes a function from byteslice to T and uses T in place of the strings. This way, when the caller has some more efficient representation available (like one that allocates less) it can provide one, but I can still unit test the tokenizer with the identity function for convenience. In a sense this is like fusing the tokenizer with the parser at build time, but the generic allows layering the tokenizer such that it doesn't know about the parser's representation. [1] https://github.com/evmar/n2

[ninja author] Ninja build targets in a semi-arbitrary order: the memory order of the pointers to the objects representing the build steps. This was chosen in part because it was easy but also because it was a little unpredictable (in the same sense as this nice shuffling idea). In practice I might expect the pointers to get allocated in the order they're encountered while parsing though, so perhaps similar to Make. My recollection is we experimented a bit with different prioritizations but didn't find any that reliably were better. It does seem like you could try prioritizing "longer" tasks. (There are also more complex models that take into account the graph; I have been mentally drafting a blog post about this area, there's some interesting research history...) To do so you'd need to keep around data from previous executions of tho