RecordStream

It's interesting watching these types of tools get re-invented periodically: https://github.com/benbernard/RecordStream It shows the unix model of many small, composable tools is very powerful, but also shows that POSIX is missing some essential pieces that everyone keeps trying to add/reinvent.

I don't know about MillerCLI's portability, but RecordStream (https://github.com/benbernard/RecordStream) is my go to swiss army knife.

A plug of my tools: To visualize data coming in from a pipe, can pipe it to https://github.com/dkogan/feedgnuplot Very useful in conjunction with other tools to provide filtering and manipulation. For instance (the first one is mine): https://github.com/dkogan/vnlog https://www.gnu.org/software/datamash/ https://csvkit.readthedocs.io/ https://github.com/johnkerl/miller https://github.com/eBay/tsv-utils-dlang http://harelba.github.io/q/ https://github.com/BatchLabs/charlatan https://github.com/dinedal/textql https://github.com/BurntSushi/xsv https://github.com/dbohdan/sqawk https://stedolan.github.io/jq/ https://github.com/benbernard/RecordStream

There're lots of tools in this space that are similar in a very general way, but have widely different design choices. My tookit is https://github.com/dkogan/vnlog It's tsv-utils-like, but is strictly a wrapper around existing tools. So filtering and transformations are interpreted literally as awk (or perl) expressions. And the various cmdline options match the standard tool options because they ARE the standard tools. So you get a very friendly learning curve, but something like tsv-utils is probably faster and probably more powerful. And it looks like tsv-utils references fields by number instead of by name. Many of the others (mine included) use the field names, which makes a MAJOR usability improvement. Other tools in no particular order: https://csvkit.readthedocs.io/ https://github.com/johnkerl/miller https://github.com/eBay/tsv-ut

Also: - https://github.com/benbernard/RecordStream

Greetings fellow devs! I'm the CTO of Fieldbook, happy to talk about anything. We use a node backend stack, with mongodb, and a backbone based front end. We use socketio for realtime and rabbit for some back end messaging. I'm also one of the authors of RecordStream which has been discussed on here before (https://github.com/benbernard/RecordStream). I see Fieldbook as an extension of RecordStream but for the web instead of JSON records on the command line.

A similar toolkit, RecordStream, with support for many types of inputs beyond CSV: https://github.com/benbernard/RecordStream

You should check out RecordStream. It has a lot of convenient processing built around JSON, and a lot of ways to get data to/from JSON. https://github.com/benbernard/RecordStream/tree/master/doc https://github.com/benbernard/RecordStream/blob/master/doc/R...

recs has this functionality, and _way_ more; worth giving a shot: https://github.com/benbernard/RecordStream

Log every single action, decision, call, message, visit, and fault in detail. Log it with structure. Log it in JSON. Now go start playing with RecordStream: https://github.com/benbernard/RecordStream You can perform any post-facto analysis you can dream up - tomorrow or three years from now.

Also see also RecordStream: https://github.com/benbernard/RecordStream Someone mentioned jsawk in a sibling comment, too.

Very nice! I've wanted something like this for as long as I've been a RecordStream (https://github.com/benbernard/RecordStream) fanatic.

Have you seen RecordStream? It's some good cli tools based on streams of json which might be handy. https://github.com/benbernard/RecordStream I've used it a lot and it's a godsend for a most "record-y" manipulation.

RecordStream is a great solution for manipulating JSON log files: https://github.com/benbernard/RecordStream - makes it easy to perform relational-style operations, such as grouping, counting (recs-collate), and even joins (recs-join). It can also display and transform the logs in a variety of ways (recs-tocsv, recs-totable, recs-tognuplot for graphing). There are also tools to extract records from other data formats such as CSV or by querying a database. All in all, a fantastic tool for manipulating and querying complex data from the command line. I use it regularly.