I always lament that since at least 1980s or so, it seems the vast majority of the control characters were never used for their intended purpose.
Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).
Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).
The only thing you'd need is editors to support some way of entering and displaying the RS, and CTRL+^ is a bit of a kludge as it ends up CTRL+SHIFT+6.
Of course, if a record itself can contain RS for subrecords, things become more complicated. I guess you could use `\^`.
Values were separated with char(254), subvalues were separated with char(253), and the third level were char(252) separated.
It was... unique, but worked. And to be fair, PICK originated in the 60's, so this method probably evolved in parallel to the ASCII table!
What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?
Of course, the fact that record separators aren't on keyboards is probably why CSVs use commas.
No, they'll be right exactly as often, 0% of the time.
But their mistake will show up less frequently, causing more problems when it does.
As soon as it's possible for some of your data to come from someone else's dataset, you're guaranteed to have to accommodate record separators within your data as well as within the metadata. You're better off using a system that plans for this inevitability than one that pretends it can't happen at all.
> But their mistake will show up less frequently, causing more problems when it does.
Enough people use CSVs (and have limited, small-scale use-cases) that I'd be willing to bet "less frequently" means never for at least 1% of people who use CSVs.
I don't know whether the chance of no problems is worth the increased difficulty of problems that do occur - considering that balance feels a bit silly because if you're aware there could be a problem in a context where you could choose between commas and unit separators, you could just add validation or escaping.
As soon as you have validation or escaping, having a record separator character loses its entire purpose. The existence of the character is predicated on the idea that you don't have to do that, and that idea is false.
That's why the character is never used. It's a conceptual mistake that was accidentally enshrined in a series of encoding standards that had enough free space to accommodate it.
I disagree with this - the data needs to be stored somehow, and while other characters (like comma) can be used, having a dedicated character can help - for example if the data might legitimately contain commas or newlines but not unit separators or record separators, then escaping isn't needed if you use unit/record separators (although validation is still necessary).
TSV is widely used, but lacks a way to escape the tab and new line characterss. RS-V is the same, but allows including tabs and new lines in records.
Not true. Validation is easier than escaping.
It was probably possible to type an RS with Ctrl+Shift+. and the others with similar combos.
Bash will insert the control character (rather than interpret it) if you prefix it with Ctrl-V.
It is still valid to disallow the ASCII control characters, one just has to make sure that it is done comprehensively, in all places users may input them. But that's not created by using ASCII control characters, that's a consequence of the "ban the control characters entirely" approach regardless of what the control characters are.
It's neat when you can get away with it, but I generally prefer to define a robust encoding scheme instead. A minimal one like "replace backslash with double-backslash, replace control characters with backslashed characters" and "replace backslash sequences with their control characters, including backslash-backslash as a single backslash" can be inserted almost anywhere in just a few lines of string replace (or stream processing if you need the speed). The only tricky bit is you need to make sure you get the order correct or you corrupt data, and while I've done this enough to have it almost memorized now I do recall feeling like the correct order is backwards from what I naturally wanted the first few times. But it is simple and robust if you get it right.
(A trivial evening project for some; not for all of us)
You use the ASCII escape character (0x1B), which is designed for exactly that purpose.
This comes up every time. Options:
1. You disallow it. And you might as well disallow all the control codes except the carriage return, line feed, and other “spacing” characters. Because what are they doing in the data proper? They are in-band signals.
2. You use the Escape character to escape them
3. Weirdest option: if you really want to nest in a limited way you can still use the group and file separator characters