Hacker News Reader

Comment by tyingq | Hacker News Reader

It seems very comparable to "shar"...shell archives.

https://en.wikipedia.org/wiki/Shar

shar is specifically mentioned in the footer, with the drawback that it's really an arbitrary shell scipt. ptar is also mentioned, and seems rather nice. It's also way better documented[0] though with glaring holes e.g. how does it deal with non-UTF8 file content (it's not clear whether file size or delimiters is relevant, and why you'd have a closing delimiter if the filesize rules), it also specifies file names as UTF8 or ASCII, neither of which is sufficient to handle the full breadth of possible file names.

[0] https://github.com/jtvaughan/ptar/blob/master/FORMAT.md

There are file names that can’t be expressed as UTF8? I’m intrigued.

Posix file names can be any sequence of bytes other than '/' and 0

I guess that's true. I suspect the support for non-UTF8 names in modern tooling is very, very spotty, given how many config files and file formats that refer to other files use UTF-8 themselves. E.g. can you refer to one of these names in an nginx config? (just an example; I have no idea if its config is UTF-8 or not)

I have a collection of non-utf8 and other problematic files to test such tools:

https://github.com/benibela/nasty-files

You probably cannot clone the repo on Windows. It works well on Linux. But in KDE you could not delete it after checkout

Also, filenames with one or more newlines bork a lot of Linux software.

And Windows file names can include unpaired surrogates, which are not allowed in UTF8 (that’s why WTF8 exists).

Reply on news.ycombinator.com