Hacker News Reader

Comment by hnlmorg | Hacker News Reader

That's a POSIX shell thing rather than a Unix pipeline thing. Some non-POSIX shells don't have this problem while still passing data long Unix pipes

Source: I wrote a shell and it solves a great many of the space / quoting problems with POSIX shells.

View on HN

Same, I've built toy shells [0] that used xml, s-expressions and ascii delimited text [1]. The last was closest to what unix pipes should be like. Of course it breaks ALL posix tools, but it felt like a shell that finally works as you'd expect it to work.

[0] Really just input + exec + pipes.

[1] https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text

Here's one tool that it does not break. (-:

* http://jdebp.uk./Softwares/nosh/guide/commands/console-flat-...

As long as you re-serialise that data when piping to coreutils (etc) you shouldn't have an issue.

This is what my shell (https://github.com/lmorg/murex) does. It defaults to using JSON as a serialisation format (that's how arrays, maps, etc are stored, how spaces are escaped, etc) but it can re-serialise that data when piping into executables which aren't JSON aware.

Other serialisation formats are also supported such as YAML, CSV and S-Expressions too. I had also considered adding support for ASCII records but it's not a use case I personally run into (unlike JSON, YAML, CSV, etc) so haven't written a marshaller to support it yet.

Ascii records have worked better than json, xml, s-expressions (better as in provide the same bang for a lot less buck) in every case I've used them for.

I have no idea why I am the only person I know who uses them regularly.

ASCII records have their own limitations though:

- They can be as easily edited by hand like other serialisation formats which use printable characters as their deliminators

- It's not clearly defined how you'd use them for non-tabulated data (such as JSON, XML, S-Expressions)

- There isn't any standard for escaping control characters

- They're harder to differentiate between unserialised binary formats

- They can't be used as a primitive like JSON is to Javascript and S-Expressions is to Lisp.

And if we're both honest, reducing the serialisation overhead doesn't gain you anything when working in the command line. It's a hard enough sell getting websites to support BSON and at least there, there is a tangible benefit of scale.

Not that I'm dismissing ASCII records, they would have been better than the whitespace mess we currently have in POSIX shells. However I don't agree ASCII records are better that current JSON nor S-Expressions.

Reply on news.ycombinator.com