jmc

That's nice - I'd encourage you to play around with attaching e.g. JMC [1] to the process to better understand why things are as they are. I tried recreating your DataInputStream + BufferedInputStream (wrote the 1brc data to separate output files, read using your code - I had to guess at ResultObserver implementation though). On my machine it roughly in the same time frame as yours - ~1min. According to Flight Recorder: - ~49% of the time is spent in reading the strings (city names). Almost all of it in the DataInputStream.readUTF/readFully methods. - ~5% of the time is spent reading temperature (readShort) - ~41% of the time is spent doing hashmap look-ups for computeIfAbsent() - About 50GB of memory is allocated - %99.9 of it for Strings (and the wrapped byte[] array in them). This likely causes quite a bit of GC pressure. Hash

How does this compare with Flight Recorder? As of Java 11, Flight Recorder and its analysis tools (JDK Mission Control, formerly Java Mission Control) are free and open source: https://github.com/openjdk/jmc I have found Flight Recorder light weight enough to have it always enabled on production services. I didn't start using it until last year because it was a commercial feature on older Java releases. EDIT: another comment says that this is based on Flight Recorder. The value add proposition is around the interface rather than the instrumentation, then. I think there is an opportunity there. JMC is fairly heavyweight and doesn't have good visualization/exploration tools for everything.