Hacker News Reader

Comment by derefr | Hacker News Reader

Thought: what if Unicode decoding was "lossless" in the face of errors, such that the replacement characters represented the bitstring of non-decodable bytes? (E.g. 256 reserved-codepoints for each possible octet value, that render as e.g. "[FF]" in a box; and then another 255 for the set of 7-bit, 6-bit, 5-bit, etc. overhangs.)

View on HN

been there, done that; it is very useful.

  1> [(file-get-string "/bin/ls") 0..15]
  "\x7F;ELF\x02\x01\x01\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00"
  2> (file-put-string "foo" (file-get-string "/bin/ls"))
  t
  3> (sh "cmp /bin/ls foo")
  0
  4> (sh "sha256sum /bin/ls foo")
  a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0  /bin/ls
  a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0  foo
  0

There is no need to handle any fractional bytes if the original input is a sequence of bytes; so many whole bytes have to be recovered, not so many whole bytes plus three bits or whatever.

I was extremely curious what REPL this is, and it appears to be TXR: http://www.nongnu.org/txr/

(My conclusion was based on finding file-get-string and file-put-string in https://fossies.org/linux/misc/txr-176.tar.gz/txr-176/share/...)

I might be sleep deprived... but what exactly is this script doing?

You're making a copy of /bin/ls into foo, and sha256sum the copy and the original.

    $ head -c 15 /bin/ls
    $ cat > foo < /bin/ls
    $ cmp /bin/ls foo
    $ sha256sum /bin/ls foo

I don't get it.

> You're making a copy of /bin/ls into foo

By getting its contents as a character string formed by passing the binary through a UTF-8 decoder, and writing out that string via the UTF-8 encoder.

Reply on news.ycombinator.com