Hacker News Reader

Comment by jandrese | Hacker News Reader

What a delightfully insane project. AWK is under-appreciated IMHO. Read the short manual and you'll discover a world of functionality in a highly orthogonal interface. Pattern Match->Manipulate. An incredibly useful workflow for many tasks.

View on HN

I wish there was like a "libawk" or something like it for programmatic usage. Its really a neat piece of work that is limited on being only a shell program.

At least, as far as I am aware, there is no programmatic interface

What would be a use case for this?

Text processing & extraction

Like a middle ground between regex and embedded Lua?

That's Perl.

I'm sure some element of perl is the middle ground of everything.

This is a somewhat different idea but I'd like to be able to feed structured data (in memory objects) to an (user defined) awk-like script without the need to serialize/parse everything. This could be used for packet filtering, conditional breakpoints, file searching, syscall filtering and intercepting, etc. Basically wherever you need a user defined filter more powerful than regex.

The embedder can decide what variables and functions to provide for each invocation of the script. You could get something like Wireshark filters out of the box:

    ip && port==1234 {print size, time}

Graphviz includes a tool "gvpr" which essentially implements this idea for processing graphs but of course, they end up reimplementing a large part of awk.

At this point I’be caved and just coerce through json via python and jam things through jq for most of my data inspection and sleuthing.

I don't know of a "libawk" C library, but my GoAWK interpreter can also be used as a library (Go) library. See reference docs: https://pkg.go.dev/github.com/benhoyt/goawk/interp

A project that's using it that way is Benthos, a "stream processor" written in Go. Awk is one of the "processors", the components that filter and transform data: https://www.benthos.dev/docs/components/processors/awk

I’ve been really impressed by your ability to share your project in entirely relevant threads recently. Hope that doesn’t sound snarky, because I actually mean it. Awesome project. Keep on.

Hehe, thanks. Yeah, I was on the fence about posting it the last time, thinking "Hmmm, am I promoting my own project too much?" But I've tried to only do it in very relevant contexts or when people ask and it would solve that question.

GoAWK's CSV mode is a great addition to AWK. Have you thought about adding a JSON mode, too, to get an AWK-flavored alternative to jq?

Edit: I see that you have: https://github.com/benhoyt/goawk/issues/152

Yeah, my comment there more or less sums up the thought I've put in so far. AWK doesn't really have nested data structures, but a lot of JSON / JSON lines is fairly flat, so it may work. It shouldn't be too hard ... though as I've noted there, there are a few tricky design issues to work though. Hope to come back to this at some point!

The implementation is so beautiful…thanks for sharing it. This will make me better, I promise.

Gawk added facilities for writing extensions, but I don't see how to use it as a library. Funny I never thought of it, considering how much I like awk, but using awk instead of Lua as an embedded extension language sounds appealing.

I love Lua but I can also see AWK working like a charm here.

Where would you use this? Most languages have good enough string processing to imitate a lot of what AWK does

You can write awk scripts directly if you start a file with a "awk -f" shebang or similar!

I am not totally sure if this is what you mean or not, but it's much nicer than embedding the script into a bash source file.

Perhaps:https://github.com/neacsum/awklib

and article on how to use:

https://www.codeproject.com/Articles/5264205/Embedding-AWK-i...

An elegant tool for forgetting inbetween all my organic needs for it.

Reply on news.ycombinator.com