walkdir

> I believe that GNU find is slow because it is specifically written to allow arbitrary filesystem depth as opposed to "open file descriptor limit-limited depth". I haven't benchmarked find specifically, but I believe the most common Rust library for the purpose, walkdir[1], also allows arbitrary file system recursion depth, and is extremely fast. It was fairly close to some "naive" limited depth code I wrote in C for the same purpose. A lot of go-to C approaches seem to needlessly call stat on every file, so they're even slower. I'd be curious to see benchmarks of whether this actually makes a difference. [1] https://github.com/BurntSushi/walkdir

> It should be possible do it all in native Rust code as far as I am concerned. Yeah, it would be, but Rust's standard library doesn't provide any way to invoke syscalls. You either need to go through libc's syscall wrapper, or write Assembly. I used the former to get access to faster directory traversal: https://github.com/BurntSushi/walkdir/blob/ec33af7f9b25afefc... (The rest of that code in that module may be interesting to you, since the API for getdents is pretty interesting and very C-like. Buttoning it up behind a safe API that can never be misused took a bit of thinking.) Besides, Rust's standard library, on Linux at least, currently requires libc anyway. So you're already going to be linking to it.

In addition to what eatsfoobars said, you can also use attributes for conditional compilation depending on a number of things (pointer size, platform, etc.) e.g., https://github.com/BurntSushi/walkdir/blob/master/src/same_f...