sqawk

"Sqawk is an Awk-like program that uses SQL and can combine data from multiple files. It is powered by SQLite." https://github.com/dbohdan/sqawk

A plug of my tools: To visualize data coming in from a pipe, can pipe it to https://github.com/dkogan/feedgnuplot Very useful in conjunction with other tools to provide filtering and manipulation. For instance (the first one is mine): https://github.com/dkogan/vnlog https://www.gnu.org/software/datamash/ https://csvkit.readthedocs.io/ https://github.com/johnkerl/miller https://github.com/eBay/tsv-utils-dlang http://harelba.github.io/q/ https://github.com/BatchLabs/charlatan https://github.com/dinedal/textql https://github.com/BurntSushi/xsv https://github.com/dbohdan/sqawk https://stedolan.github.io/jq/ https://github.com/benbernard/RecordStream

There're lots of tools in this space that are similar in a very general way, but have widely different design choices. My tookit is https://github.com/dkogan/vnlog It's tsv-utils-like, but is strictly a wrapper around existing tools. So filtering and transformations are interpreted literally as awk (or perl) expressions. And the various cmdline options match the standard tool options because they ARE the standard tools. So you get a very friendly learning curve, but something like tsv-utils is probably faster and probably more powerful. And it looks like tsv-utils references fields by number instead of by name. Many of the others (mine included) use the field names, which makes a MAJOR usability improvement. Other tools in no particular order: https://csvkit.readthedocs.io/ https://github.com/johnkerl/miller https://github.com/eBay/tsv-ut

"Sqawk is an Awk-like program that uses SQL and can combine data from multiple files. It is powered by SQLite." https://github.com/dbohdan/sqawk

This is becoming a really crowded space. Some other similar tools that make slightly different design choices and that have variable envisioned use cases: - https://github.com/dkogan/vnlog - https://csvkit.readthedocs.io/ - https://github.com/johnkerl/miller - https://github.com/BurntSushi/xsv - https://github.com/eBay/tsv-utils-dlang - https://stedolan.github.io/jq/ - http://harelba.github.io/q/ - https://github.com/BatchLabs/charlatan - https://github.com/dinedal/textql - https://github.com/dbohdan/sqawk (disclaimer: vnlog is my tool)

Sqawk is an Awk-like program that uses SQL and can combine data from multiple files. It is powered by SQLite. https://github.com/dbohdan/sqawk

Shameless plug: Sqawk can do nearly the same without you defining a schema. $ ps aux | sqawk -output table \ 'select user, round(sum("%mem"), 2) as memtotal from a group by user order by memtotal desc' \ header=1 ┌────────┬────┐ │dbohdan │67.1│ ├────────┼────┤ │ root │3.5 │ ├────────┼────┤ │ avahi │0.0 │ ├────────┼────┤ │ daemon │0.0 │ ├────────┼────┤ │message+│0.0 │ ├────────┼────┤ │ nobody │0.0 │ ├────────┼────┤ │ ntp │0.0 │ ├────────┼────┤ │ rtkit │0.0 │ ├────────┼────┤ │ syslog │0.0 │ ├────────┼────┤ │ uuidd │0.0 │ ├────────┼────┤ │whoopsie│0.0 │ └────────┴────┘ Link: https://github.com/dbohdan/sqawk

Sqawk [1] is an Awk-like program that uses SQL and can combine data from multiple files. It is powered by SQLite. [1] https://github.com/dbohdan/sqawk

Similar idea, but uses sqlite to make an awk like utility with sql syntax: https://github.com/dbohdan/sqawk

https://github.com/dbohdan/sqawk It parses DSV data like Awk does, runs SQL queries against it and formats the output in one of several ways. An example I am particularly fond of is using this tool as a poor man's libxo (https://github.com/Juniper/libxo): $ ps | sqawk -output json,indent=1 'select PID,TTY,TIME,CMD from a' trim=left header=1 [{ "PID" : "3947", "TTY" : "pts/2", "TIME" : "00:00:07", "CMD" : "zsh" },{ "PID" : "15951", "TTY" : "pts/2", "TIME" : "00:00:00", "CMD" : "ps" },{ "PID" : "15952", "TTY" : "pts/2", "TIME" : "00:00:00", "CMD" : "tclkit-8.6.3-mk" }] I started a list of command line tools for querying, processing and converting structured text data: https://github.com/dbohdan/structured-text-tools.

I saved the post in a plain text file and fed it as input to Sqawk [1], which I develop, then modified the output somewhat. Namely, I removed the horizontal lines separating the rows (to fit the post in the 2000 characters HN allows) and manually centered the text in the header. [1] See https://github.com/dbohdan/sqawk#pretty-print-data-as-a-tabl....

There is a lot of tools of this kind [1] but nearly all of them work with CSV or other delimiter-separated data. What made recutils stand out for me is that it has tools to modify files in place, not just select data from them, and its file format is different. The rec format is interesting because it is a specification for what others have used before without one. It seems quite readable and does not suffer from the ",,,,,," problem of CSV. [1] Here's the list of ones I have run into so far: * https://github.com/harelba/q * https://github.com/tobimensch/termsql * https://github.com/samuel/squawk * https://github.com/tjunier/sqawk I've also written one: https://github.com/dbohdan/sqawk. It turned out I wasn't the only one to come up with the name.

I'm working on static site generator written in Tcl that is nearing its 1.0 release. It aims to be featureful but self-contained and have reasonable defaults to allow you to quickly start working on a new website. You can find it at https://github.com/tclssg/tclssg. I've also written an Awk-like tool in Tcl that speaks SQL and has table joins - https://github.com/dbohdan/sqawk.