py-spy

"All of this is made possible with the inclusion of frame pointers in all of Meta’s user space binaries, otherwise we couldn’t walk the stack to get all these addresses (or we’d have to do some other complicated/expensive thing which wouldn’t be as efficient)" This makes things so, so, so much easier. Otherwise, a lot of effort has to built into creating an unwinder in ebpf code, essentially porting .eh_frame cfa/ra/bp calculations. They claim to have event profilers for non-native languages (e.g. python). Does this mean that they use something similar to https://github.com/benfred/py-spy ? Otherwise, it's not obvious to me how they can read python state. Lastly, the github repo https://github.com/facebookincubator/strobelight is pretty barebones. Wonder when they'll update it

I've used Scalene at various times in the past few years, and always liked using it when I want to dig deeper compared to cProfile/profile. You might also want to look at: 1] https://github.com/joerick/pyinstrument 2] https://github.com/benfred/py-spy 3] https://github.com/P403n1x87/austin 4] https://github.com/bloomberg/memray 5] https://github.com/pyutils/line_profiler

Is there something about the actual profiler that differs from existing tools like pyinstrument [1] or py-spy [2]? I know pyinstrument has various output options and I wonder if it could potentially output something readable by the Firefox Profiler tool. [1] : https://github.com/joerick/pyinstrument [2] : https://github.com/benfred/py-spy

https://github.com/benfred/py-spy is also really nice, and has an actual oss license.

For CPU cycles, py-spy[0] is getting more and more used. For RAM, I would like to known too... [0] -- https://github.com/benfred/py-spy

Theres also Py Spy, a profiling tool that can generate flame charts containing a mix of python and C (or C++) calls. https://github.com/benfred/py-spy It's worked really well for my needs

Any advantages/disadvantages compared to py-spy [1]? [1]: https://github.com/benfred/py-spy

Similarly, py-spy is a sampling profiler for Python programs. It lets you visualize what your Python program is spending time on without restarting the program or modifying the code in any way. py-spy is extremely low overhead: it is written in Rust for speed and doesn't run in the same process as the profiled Python program. This means py-spy is safe to use against production Python code. I'm not sure if it exports results in a format Chrome can render but it does produce great interactive SVGs and is compatible with speedscope.app https://github.com/benfred/py-spy https://github.com/jlfwong/speedscope

It is pretty cool. py-spy has also been doing this for a few years https://github.com/benfred/py-spy

Sorry about leaving your program suspended - I believe that was because of a bug that should be fixed now https://github.com/benfred/py-spy/issues/390 =(

Thanks, I've gone through and added these HPC tools to the spreadsheet. FYI I'd consider py-spy for profiling Python+C extensions: https://github.com/benfred/py-spy#can-py-spy-profile-native-...

> you can either race with the program and hope that you read its memory to get the function stack before it changes what function it's running (and you're likely to win the race, because C is faster than Python), or you can pause the program briefly while taking a sample. Py-spy defaults to blocking because the results can be pretty wrong otherwise: https://github.com/benfred/py-spy/issues/56 . You can see this problem profiling a program like https://github.com/benfred/py-spy/blob/master/tests/scripts/... with or without the nonblocking flag in py-spy - the nonblocking version produces garbage output. Somewhat interestingly, this problem doesn't seem to occur with Ruby - and rbspy can get away without pausing the target program with only minor errors seen when profiling a similar function. I suspect this is because of differences betwee

Two other profilers that also let you spy on a running Python program: - py-spy: https://github.com/benfred/py-spy (written in Rust) - pyflame: https://github.com/uber-archive/pyflame (C++, seems to be not maintained anymore) The "no performance cost" thing is interesting: my experience writing a similar profiler is that there are a couple of things that can affect performance a little bit: 1. You have to make a lot of system calls to read the memory of the target process, and if you want to sample at a high rate then that does use some CPU. This can be an issue if you only have 1 CPU. 2. you have two choices when reading memory from a process: you can either race with the program and hope that you read its memory to get the function stack before it changes what function it's running (and you're likely to win the race, because C is faster

Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier. As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is super useful when your program is stuck and you don't know why. I haven't used pyinstrument yet but I do like the fact that it can do its flame graph in the console, sometimes I find saving down an svg file and opening up the browser a bit arduous. Excited to give it a try. [1] https://github.com/benfred/py-spy [2] https://jvns.ca/blog/2018/09/08/an-awesome-new-python-profil...

To extend on inspecting stack traces: I found that I often wanted to know the the content of (relevant) (local) variables of a stack trace. Whenever I got a stacktrace, I would often restart it in a debugger, or add print statements on variables. Until I extended the stacktrace to just include this information. The tricky part is that printing all local variables is too much. And also global variables might be involved. Or attributes (e.g. self.x or so). So now I'm parsing all variables and attributes from the source code line from the traceback (in a very simple way) and print only those. This covers about 95% of all cases - i.e. in 95% of the case, the stack trace contains all information I need to understand and fix the problem. I published that here: https://pypi.org/project/better_exchook/ https://github.com/albertz/py_better_exchook