perf-tools

For execsnoop, people running systems with DTrace can find the same: * https://github.com/jorgev/dtrace-scripts/blob/master/execsno... On macOS Monterey+ you'll probably have to install the Kernel Debug Kit per: * https://developer.apple.com/forums/thread/692444 The Linux variant was written Brendan Gregg (who previous did a lot of work on Solaris, where DTrace was created): * https://github.com/brendangregg/perf-tools/blob/master/execs... * https://github.com/iovisor/bcc/blob/master/tools/execsnoop.p...

I do this with Brendan Gregg's opensnoop, which is standard on macOS. https://www.brendangregg.com/blog/2014-07-25/opensnoop-for-l... https://github.com/brendangregg/perf-tools/blob/master/opens...

strace - to see what a program is doing / where exactly something is failing if the error message is bad ltrace - same as above, if strace is not enough perf trace - if you don't know the exact process to strace upfront https://github.com/brendangregg/perf-tools - to track disk accesses or exec calls env LD_DEBUG=all - debugging dynamic linker issues

The most challenging thing for us is running a new enough kernel to get these features. While upgrading to a newer kernel isn't particularly hard, small companies don't have a lot of engineering resources to run kernels that aren't maintained by their distro of choice (usually on the LTS release). The good thing is this is solved simply by waiting long enough. The bad thing is most developers can't just pick this up today without a bunch of extra effort. If you are looking for something you can use with old kernels you should definitely checkout Brendan's perf-tools repo[1]. It takes advantage of older kernel features and works with things as old as ubuntu 12.04. *Edit: Fixed Brendan's name [1]: https://github.com/brendangregg/perf-tools

Good to see people sharing extra commands! I had written some follow on commands at the end of the article, but trimmed them to make it shorter. They were: # CPU perf record -F 99 -a -g -- sleep 10; perf report -n --stdio # and flamegraphs execsnoop # from perf-tools turbostat # Memory cat /proc/meminfo slabtop # Disk df -h iosnoop # from perf-tools pidstat -d iotop # Networking netstat -s tcpdump tcpretrans # from perf-tools Where perf-tools is https://github.com/brendangregg/perf-tools.

Yes, it would have been nice to have DTrace in Linux a while ago. But the reality is a bit more complex than it might look. Many people I know haven't been aware that Linux has had in-built dynamic tracing capabilities for years, with ftrace, kprobes and later uprobes. These are much more difficult to use (which is why I wrote some front-ends: https://github.com/brendangregg/perf-tools). But if you really cared about performance, you could use them. So it's not that Linux has been completely missing out; It's been missing out on a some specific features (eg, in-kernel aggregations and variables), and a nice interface. As for attempts unabated to reinvent DTrace: FWIW, that's not really the genesis in the case of bcc/eBPF. Extended Berkeley Packet Filtering (eBPF) was developed to create virtual network infrastructures. Things like this: h

Those scripts and more for Linux (3.2 or newer) using perf_events/ftrace: https://github.com/brendangregg/perf-tools

Oh, certainly. I was only talking about OS X, which didn't build the DHCP client around a collection of shell scripts for portability. On OS X, you could see every time a process is invoked like this: sudo execsnoop -c bash On Linux, that requires work which fortunately Brendan Gregg already did: https://github.com/brendangregg/perf-tools/blob/master/execs...

I just added an experimental execsnoop to the repo as well: https://github.com/brendangregg/perf-tools .