JVjvns·Oct 27, 2020
Two other profilers that also let you spy on a running Python program:
- py-spy: https://github.com/benfred/py-spy (written in Rust)
- pyflame: https://github.com/uber-archive/pyflame (C++, seems to be not maintained anymore)
The "no performance cost" thing is interesting: my experience writing a similar profiler is that there are a couple of things that can affect performance a little bit:
1. You have to make a lot of system calls to read the memory of the target process, and if you want to sample at a high rate then that does use some CPU. This can be an issue if you only have 1 CPU.
2. you have two choices when reading memory from a process: you can either race with the program and hope that you read its memory to get the function stack before it changes what function it's running (and you're likely to win the race, because C is faster
Read the rest
than Python), or you can pause the program briefly while taking a sample. py-spy has an option to choose which one you want to do: https://github.com/benfred/py-spy#how-can-i-avoid-pausing-th...
Definitely this method is a lot lower overhead than a tracing profiler that instruments every single function call, and in practice it works well.
One thing I think is nice about this kind of profiler is that reading memory from the target process sounds like a complicated thing, but it's not: you can see austin's code for reading memory here, and it's implemented for 3 platforms in just 130 lines of C: https://github.com/P403n1x87/austin/blob/877e2ff946ea5313e47...