pprof

I guess you didn't get very far in the README because near the top it tells you how to install it. It's a single command: go install github.com/google/pprof@latest

Every time I see people use flamegraphs it's the ancient Perl version. There's a much better version!!! Use the Go version of pprof: https://github.com/google/pprof Run it like `pprof -http : your_profile.out` and it will open a browser with a really nice interactive flamegraph (way better than the Perl version), plus a call graph, source line profiling, top functions, etc. etc. It's so much better. Don't use the Perl version. I should probably write a post showing how to do this. Another also-much-better alternative is Samply (https://github.com/mstange/samply) which uses the Firefox Profiler as a GUI. I don't like it quite as much as pprof but it's clearly still much better than what's in this article: https://share.firefox.dev/3j3PJoK

I strongly recommend not using this. Instead use pprof - it has a MUCH better interactive flamegraph, plus other nice performance visualisations (e.g. a call graph): https://github.com/google/pprof go install github.com/google/pprof@latest pprof -http=: prof.out I normally collect the profiles with gperftools (https://github.com/gperftools/gperftools) and then just LD_PRELOAD=/usr/lib/libtcmalloc_and_profiler.so CPUPROFILE=prof.out <your command> I've been meaning to try Samply though. Not sure if it works with pprof.

Damn I hate it when you write a whole project and someone comes along and says "this already exists" and you realise how much time you wasted (yeah even if some of it counts towards learning I'd still rather not needlessly repeat other people's work). Anyway, pprof has a fantastic interactive Flamegraph viewer that lets you narrow down to specific functions. It's really very good, I would use that. https://github.com/google/pprof Run `pprof -http=:` on a profile and you get a web interface with the Flamegraph, call graph, line based profiling etc. It's demonstrated in this video. https://youtu.be/v6skRrlXsjY They only show a very simple example and no zooming, but it works very well with huge flamegraphs.

How do flame graphs handle the case where most of the time is spent in some leaf function that is called from all over the program? In this case, each individual stack would not take much time but in aggregate, a lot of time is spent in the function at the top of all of the call stacks. This should not be that uncommon to have hotspots in things like copying routines, compression, encryption etc that are not associated with any particular stack. pprof from https://github.com/google/pprof can produce a DAG view of a profile where nodes are sized proportional to their cumulative time, e.g., https://slcjordan.github.io/images/pprof/graphviz.png and such a view would seem to cover the case above and subsume the usual use cases for a flame graph, would it not? Although I guess a flat text profile of functions sorted by time would also highligh

The pprof format is not tied to Go. From my understanding, it's used within Google across multiple languages. The format is defined in the pprof repository[0], and the visualization tool is source-language agnostic. I've seen libraries in numerous languages (e.g. Python, Java) to publish profiles in pprof format. This is an indicator the pprof format has become de-facto. Grafana Pyroscope[1] is a tool that's capable of parsing the pprof format, agnostic to the source programming language, and has instructions for Go, Java, Python, Ruby, node.js, Rust, and .NET. My understanding is that you're searching for a combination of the profiles, metrics, and tracing. Caddy supports all 3. [0] https://github.com/google/pprof/blob/main/doc/README.md [1] https://grafana.com/docs/pyroscope/latest/ metrics and tracing need to be manually enabled (for

Go also has pretty good out of the box profiling (pprof[0]) and third-party runtime debugging (delv[1]) that can be used both remotely and local. These tools also have decent editor integration and can be use hand in hand: https://blog.jetbrains.com/go/2019/04/03/profiling-go-applic... https://blog.jetbrains.com/go/2020/03/03/how-to-find-gorouti... [0] https://github.com/google/pprof [1] https://github.com/go-delve/delve

Go has pprof (https://github.com/google/pprof), which I've heard good things about --- and, the pprof data model was one of the influences I looked at when designing the Tokio console's wire format. But, I'm not sure if pprof has any similar UIs to the one we've implemented for the Tokio console; and I haven't actually used it all that much.

I use pprof to find and fix bugs in Go apps. I just recently used pprof to find and report a huge memory leak in coreDNS (when using it as an authoritative DNS server for very large zones). I was able to find the bug and get it fixed in less than 24 hours. https://github.com/miekg/dns/issues/786 https://github.com/google/pprof

pprof has a flamegraph and I'm fairly certain it isn't written in Perl. Yep it is JavaScript: https://github.com/google/pprof/blob/master/third_party/d3fl...

A nice writeup, thanks. There are a few variations on this workflow that I've found useful in practice; perhaps they'll be helpful to some folks: - Linux perf can profile unmodified Go programs. This is handy when your application doesn't expose the /debug/pprof endpoint. (http://brendangregg.com/FlameGraphs/cpuflamegraphs.html#perf has detailed instructions) - Recent versions of https://github.com/google/pprof include a flamegraph viewer in the web UI. This is handy when you want a line-level flamegraph instead of a function-level flamegraph.