Git-Heat-Map

Git-Heat-Map¹ does something similar to this. It's not open source but I found it easy re-implement the core algorithm² as a learning exercise in React. ¹- https://github.com/jmforsythe/Git-Heat-Map ²- https://www.win.tue.nl/~vanwijk/stm.pdf

Shameless plug for my project [0] using treemaps to visualise a git repo [0] https://github.com/jmforsythe/Git-Heat-Map

I was confused as my project [0] I called it a git heat map, not realising that a similar term was used for other completely different visualisations [0] https://github.com/jmforsythe/Git-Heat-Map

You could try the alternative version of the database-generator: https://github.com/jmforsythe/Git-Heat-Map/pull/6 - it shouldn't crash.

Computing diffs is what takes large amounts of time as the object database is used intensively along with limited efficiency of object caches. I couldn't resist and threw `gitoxide` at it, and it turned out to be more than 2x as fast (even though it uses way more CPU to do that, there is definitely room for improvement). The PR which adds the `db-gen` program: https://github.com/jmforsythe/Git-Heat-Map/pull/6

I experimented earlier to push the limits of SQLite inserts and wrote a blog post[0] about it. We can apply some of the learnings here. I reviewed the OP's code and did some benchmarks; SQLite is not the bottleneck here. The code first generates the commit info from the git log, prints that to stdin [1] and the python script reads from it one by one in a loop [2]. Each of the commit info is written to SQLite. So, with or without WAL, the time is almost the same. To confirm my hypothesis, I ran the project without insert calls. On my machine, for cpython, it took 160 seconds and without sqlite inserts 159 ish. I believe the git log will be fast anyway, so other ways to make it faster would be to read a bunch of commits at once and then do batch inserts. We can also make it run in parallel since each commit info is independent, and we don't