TETeMPOraL·Nov 2, 2020
Seconding the post. The post focuses on strace & friends in the Linux/Mac world; for Windows, a powerful tool to look into is Event Tracing for Windows, which lets you trace events, syscalls and performance counters for everything that's running. I.e. a global trace of all (or at least most) processes.
Two main tools to look into are xperf, the cmdline tool for configuring and running trace sessions, and Windows Performance Analyzer, a GUI app for exploring recorded traces (additionally, there's Windows Performance Recorder, which seems to be an official GUI for xperf, but I haven't used it myself yet).
This is essentially a superpower that's even less known than strace is by Linux devs. I've only learned the basics quite recently, and already used it to debug two hairy issues in few hours, that have already taken a man-month worth of tim
Read the rest
e and would likely take another one. Being able to explore the recordings of the process tree evolution, see all file IO operations, their times and targets and relation to everything else happening in the system, all in both visual and tabular form - it gives you a perspective you otherwise can't get.
(An unfortunate side effect is that the tough bugs, like multiprocess race conditions or silent crashes, all seem to be directed my way nowadays.)
Unfortunately, xperf and WPA don't come pre-installed in most Windows distributions; the fastest way to get them is through UIforETW[0] - Google's UI for xperf that can automatically locate and install the correct version of Windows Performance Toolkit[1].
Bruce Dawson (of Google Chrome fame) wrote a lot about advanced usage of this tooling, see: https://randomascii.wordpress.com/2015/09/24/etw-central/.
--
[0] - https://github.com/google/UIforETW/releases - fetch the newest .zip package with binaries.
[1] - Alternative approach, if you need xperf on a computer with no Internet access (happened to me the other day with a customer's VM), you need to download the Windows SDK ISO (e.g. https://developer.microsoft.com/en-us/windows/downloads/wind...), mount it, and grab the Windows Performance Toolkit installer from the mounted image.