jsonparser

Yeah that’s true, although there is a growing ecosystem of “allocation sensitive” libraries. We use this one a lot for protobuf: https://github.com/richardartoul/molecule jsonparser is great for JSON parsing: https://github.com/buger/jsonparser fastcache is a great “zero gc” cache: https://pkg.go.dev/github.com/VictoriaMetrics/fastcache Etc

jq performance is pretty terrible. Here I'm going to do something super simple like pull out a single field out of a large log file: $ wc -l big.log 979400 big.log $ du -hs big.log 570M big.log `count` is a small program that counts lines on stdin. like `sort|uniq -c |sort -n` jq takes 12 seconds: $ time cat big.log |jq -cr .method |~/bin/count 848000 GET 94800 POST 34000 HEAD 2400 OPTIONS 200 null real 0m12.381s user 0m12.427s sys 0m0.333s my tool takes .5 seconds $ time cat big.log |~/bin/jj method |~/bin/count 848000 GET 94800 POST 34000 HEAD 2400 OPTIONS 200 real 0m0.466s user 0m0.512s sys 0m0.198s `jj` is a little tool I wrote that uses https://github.com/buger/jsonparser

jq is just slow.. using github.com/buger/jsonparser to slice out some fields from a large json log is about 10x faster than using jq. The most common mistake I see is people composing pipelines and putting jq before grep. Or worse, using select() in jq. The grep should always come first. Even if you need to do something like .. | fgrep value | jq ... | fgrep value to pre-filter, and then filter again to rule out any false positives. I have some tooling built around jsonparser and fastjson that I need to split out and open source. jq just makes me sad.. very capable, but overkill for what most people use it for and very slow for the common case. source: wrote the C version of bro-cut that you like so much :)

Looks neat, but seems slow? $ wc -l conn.log 8505 conn.log $ cat conn.log |time ~/src/jtc-standard.json/jtc -a -w '<uid>l' -qq |shasum 0.44 real 0.43 user 0.00 sys 167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a - $ cat conn.log | time jq -r .uid|shasum 0.10 real 0.10 user 0.00 sys 167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a - $ cat conn.log | time json-cut uid |shasum 0.04 real 0.01 user 0.01 sys 167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a - json-cut is a crappy tool I wrote that use github.com/buger/jsonparser

The fastest Go JSON parser I know of is https://github.com/buger/jsonparser. I've used that one in production quite successfully. I don't see that in the comparative benchmarks for Gojay.

>requirements is higher than the time cost of using a less perfect language but richer ecosystem; but that could be totally wrong. Believe me, breakdowns will cost much more, especially if your device has no ssh, connected via ASI/some military obscure one-directional interface. I found that in many fields it's incredibly hard to approve third party libs usage, even some basic stuff like boost. >~1.5KLOC - sure, it's Go, but that's a bit more than 10 minutes Well, it is Go. This is go json parser: https://github.com/buger/jsonparser/blob/master/parser.go And this is angstrom: https://github.com/inhabitedtype/angstrom/blob/master/exampl... cron should be easier, it's either vars or cronlines, which are trivial. I prefer to write it in angstrom than to catch some unexpected panic in go because of type coercion or similar shit.

You should check https://github.com/buger/jsonparser and https://github.com/mailru/easyjson as well. They are even more faster.