Bash-Oneliner

So, grab yourself a Linux box (I suggest Debian), a large CSV file or JSON lines file you need to slice up, and an hour of time, and start trying out some bash one-liners on your data. Set some goals like "find the Yahoo email addresses in the data and sort by frequency" or "find error messages that look like X" or "find how many times Ben Franklin mentions his wife in his autobiography" Here's the thing. These tools have been used since the '70s to slice, dice and filter log files, CSVs, or other semi-structured data. They can be chained together with the pipe command. Sys admins were going through 100MB logs with these tools before CPUs hit the gigahertz These tools are blisteringly fast, and they are basically installed on every Linux machine. https://github.com/onceupon/Bash-Oneliner And for a different play-by-play example: https://a

A really nice list, thanks a lot for sharing! :) I'd add this when a filesystem gets almost full (but not overfilled, see below). This shows where most of the space goes: # du -axm / | sort -n | tail # takes a while on large filesystems, or ones with lots of files Then narrow down for each of the most filled directories: # du -axm /some/dir | sort -n | tail # subsequent searches are fast, now that metadata is cached. In case there is no space at all, sort will complain if the /tmp directory is on the same fs, then the only option is to search any suspect directories with du -sm $dir And about this one: https://github.com/onceupon/Bash-Oneliner#using-ctrl-keys A bit surprised that the Ctrl+b(ack one...) and Ctrl+f(orward one char) shortcuts are not included. As well as their Alt+b/f for a word back/forward too. Very convenient for go

From [0]: [For Bash] Ctrl + x + Ctrl + e : launch editor defined by $EDITOR to input your command. Useful for multi-line commands. I have tested this on windows with a MINGW64 bash, it works similarly to how `git commit` works; by creating a new temporary file and detecting* when you close the editor. [0] https://github.com/onceupon/Bash-Oneliner * Actually I have no idea how this works; does bash wait for the child process to stop? does it do some posix filesystem magic to detect when the file is "free"? I can't really see other ways