Hacker News Reader

Comment by car | Hacker News Reader

AWK is a great tool, but I found it lacking when working with CSV files where the separator is found in quoted fields [0]. In this case, European number formatting was the problem, tripping up Sqlite's import.

I resorted to pandas, where the CSV import has parameters for the thousands and decimal separators.

Also see AWK HN post from 2021 [1].

[0] https://earthly.dev/blog/awk-csv/

[1] https://news.ycombinator.com/item?id=28707463

View on HN

This always worked for me

https://stackoverflow.com/questions/29642102/how-to-make-awk...

The frawk implementation (not quite POSIX, but extremely fast) and my GoAWK implementation both have an "-i csv" option that puts it in "input mode CSV". More here: https://benhoyt.com/writings/goawk-csv/

  $ cat quoted.csv
  "Smith, Bob",42
  $ awk -F, '{ print $1 }' quoted.csv
  "Smith        # you want to print the first field: Smith, Bob
  $ goawk -i csv '{ print $1 }' quoted.csv
  Smith, Bob    # that's better!

gawk's FPAT is sufficent for any CSV file without newlines in the fields. I've never run into a CSV with newlines in the fields in the wild.

Reply on news.ycombinator.com