Hacker News Reader

Comment by rusk | Hacker News Reader

> the ASCII standard has had unit- and record-delimiters baked into it from the beginning.

That's mad that I've never heard of that before.

I'm amazed this isn't used more often!

They don’t really get used IMO because if you’re storing ASCII or Unicode text, you have to be prepared for those characters to be in your data. And if you’re storing binary data, delimiters don’t really cut it in the first place.

also they cannot be found in keyboards, if you need to write CSV by hand

Can't be found, but you can enter them by hand by holding down alt while entering the ASCII code on the numeric keypad. I wonder if they'll survive posting a comment: 28∟ 29↔ 30▲ 31▼ Edit: looks like they do survive. Hmmm... Now I wonder about extended ASCII. └┐│└└╚╦╝

Also, you can leverage the ISO/IEC 2022 extension and represent them with Caret Notation. Note: recommend an escaping mechanism depending on data. REF: https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C0_(AS... An editor such as vim or pager like 'less' will display them. You can also 'cat -v' a file.

You can type it at a terminal the same way you can write other low-ascii codes. For example record separator, 30, is ctrl-^ - you can type it in the terminal by the usual literal control char way, ctrl-v[1].

So for example at a Python prompt:

  >>> ord('^^') # typed ctrl-v ctrl-^ here
  30

(the low unprintable ascii codes correspond to certain ctrl combinations because ctrl-x is just the ascii code of x with the high bits masked off (i.e. & 0x1f) )

[1] custimizable in your terminal settings, see stty lnext ("literal next")

I didn't mean for CSV. There's plenty of usecases for tabular data that aren't hand coded that this could be used for. Plenty of times I've been just wanting to STGDD and not having to mess around with a serialization library, or have to select delimeters, encapsulators, do escaping etc.

No it's fine. Just base64 encode the binary data. It's common and streams, or you can capture and convert the entire field at once if you need to seek around in the binary data.

https://en.wikipedia.org/wiki/Base64

That only works if the thing consuming your CSV can be configured to decode whatever encoding you picked. Uploading CSVs of data to various SQL servers is an example case where that is often not possible.

Reply on news.ycombinator.com