BUburntsushi·Jun 26, 2018
Indeed, I've actually written such a thing (although in Rust, not C++), I'm still looking for a clean concise version. :-)
Here's the public API of mine[1], which is pretty blah, because it takes a closure (per-thread initialization) that must return a closure (executed on each entry in the tree). The most interesting part of the implementation is here[2], along with a suspicious termination argument.
Outside of that, ucg (written in C++) also has a recursive directory iterator.[3]
I also know of two written in Go, one in sift[4] and another in the platinum searcher[5]. The ones in Go seem considerably simpler as they fall back to goroutines.
I haven't carefully benchmarked all of these, but if memory serves, they're all faster than the single threaded variants. In my experience, the hardest part about implementing this is coming up with
Read the rest
a solid termination argument because your consumers are also your producers. (I documented this a bit in `get_work` in my Rust code.)
[1] - https://docs.rs/ignore/0.4.2/ignore/struct.WalkParallel.html
[2] - https://github.com/BurntSushi/ripgrep/blob/b38b101c77003fb94...
[3] - https://github.com/gvansickle/ucg/blob/master/src/libext/Dir...
[4] - https://github.com/svent/sift/blob/2ca94717ef0bbf43068f217c8...
[5] - https://github.com/monochromegane/the_platinum_searcher/blob...