awk

Essential CSV shell tools: csvtk: https://bioinf.shenwei.me/csvtk/ gawk: https://www.gnu.org/software/gawk/manual/html_node/Comma-Sep... awk: https://github.com/onetrueawk/awk?tab=readme-ov-file#csv

> And performance of awk is crap. [...] I had replaced all the awk scripts in python and everything is a lot faster. My experience points exactly the other way: for data-processing tasks, especially streaming ones, even Gawk is a lot faster than Python (pre-3.11), and apparently I’m not the only one[1]. If you’re not satisfied with Gawk’s performance, though, try Nawk[2] or, even better, Mawk[3]. (And stick to POSIX to ensure your code works in all of them.) [1] https://brenocon.com/blog/2009/09/dont-mawk-awk-the-fastest-... [2] https://github.com/onetrueawk/awk [3] https://invisible-island.net/mawk/

I'm not sure what you're thinking of, but there at least 3 prominent and still-relevant awk implementations: nawk (One True Awk / AT&T awk / BWK awk): the original awk, still maintained; is used by macOS, FreeBSD, OpenBSD, and NetBSD https://github.com/onetrueawk/awk mawk (Mike's awk): the default awk on Debian https://invisible-island.net/mawk/ gawk (GNU awk): the default awk in lots of other places https://www.gnu.org/software/gawk/

both posix awk and the one true awk have printf: https://github.com/onetrueawk/awk/blob/2bab10b60b3f4d3bbefdc... (that said, I usually use awk's ability to print through pipelines given in the awk script itself)

Awk now supports a `--csv` flag for processing csv's. https://github.com/onetrueawk/awk/blob/master/README.md

Right, "the one true awk" corresponds to a book written in 1988, very explicity. https://github.com/onetrueawk/awk I was incorrect about 30 years. It's actually 35. You were the one that that said POSIX awk to begin with, I was using your terms. I understand what you meant, why don't you? As far as shitting on the GMU tools, I don't think I've seen someone do that in decades, especially just referencing the bottom of the man page like that. As far as speed, I don't know how you're using these tools, but if gawk is your bottleneck and not i/o, there's probably smarter ways to do things. This is not a productive conversation. You can live life however you want and if you're happy than I'm happy. We are definitely at an impasse. I'm off to bed.

Awk is awesome! Glad that they are looking to modernize the book. It wasn't really necessary, all the code examples in the original edition of the book still run just fine, although some are somewhat dated, like printing ASCII bar graphs. They also had examples of writing VMs, parsers and interpreters in the book, which run on modern implementations.[0] The language has some quirks. To declare temporary variables, it's common practice to add extra arguments to functions that won't be used. And traversal of associative arrays is implementation-dependent. I'm not sure what the situation is regarding locale and UTF-8 support. EDIT: Looks like Brian Kernighan added Unicode support last year.[1] [0] https://github.com/siraben/awk-vm/blob/master/vm.awk [1] https://github.com/onetrueawk/awk/commit/9ebe940cf3c652b0e37...

BWK’s commit and test files (mentioned in the video) https://github.com/onetrueawk/awk/commit/d3a19e6f2533d479841...

Mentioned in the video https://github.com/onetrueawk/awk/tree/unicode-support

I'm not kidding. Just look at the code: https://github.com/onetrueawk/awk/commit/d322b2b5fc16484affb... I am talking about that environment. You are not.

It needs the length for operations such as substring, or to apply length modifiers on regular expressions (such as \w{3,5}), which is a common thing in awk programs. In fact, the return value of the u8_rune as implemented in the branch we are discussing (https://github.com/onetrueawk/awk/compare/unicode-support) returns a length to be used as an offset later. This is not me saying, it's the author. There is a code comment there: > For most of Awk, utf-8 strings just "work", since they look like null-terminated sequences of 8-bit bytes. Functions like length(), index(), and substr() have to operate in units of utf-8 characters. The u8_* functions in run.c handle this. I know there might be different ways of doing it, but we're talking about a specific implementation. I was wrong to assume he is storing stuff in UTF-32. He could have, but

Looks like its not really done yet: https://github.com/onetrueawk/awk/compare/master...unicode-s...

>There's tons of incompatible dialects. I think that shows the problem with what you're saying. To my knowledge the major dialects in use are: - One True Awk (aka bwk) (https://github.com/onetrueawk/awk) - this one is bundled in all *BSD/macOS - Gawk (https://www.gnu.org/software/gawk/) - this one is bundled in most Linux - mawk (https://invisible-island.net/mawk/) - bundled in some Linux distros (?), known as the fastest byte-code compiled implementation. All three have very good compatibility, but Gawk is super-set over POSIX standard. I have some evidence here, since I regularly test [1] against these implementations and even some others, like GoAWK. [1] https://github.com/xonixx/makesure/actions/runs/1830978431

Happy to see this mentions https://github.com/onetrueawk/awk