Hacker News Reader

Comment by smarx | Hacker News Reader

Thanks. We don't do anything special to help or hinder Lua's handling of Unicode. In general, Lua strings store bytes. This is handy when dealing with something like file uploads, but it's not always what you want for other text encodings. If you have any suggestions for what we should do (or what we should document), I'd love to hear them.

Interesting point about json.stringify. We should take a harder look at these kinds of cases. What would you expect the output to be? In general, JSON can't represent a lot of things that Lua tables can, so I don't expect json.parse(json.stringify(T)) to always return T. That said, anywhere we can do something more sensible, we should, so thanks for this feedback (and tell us more).

View on HN

Unicode: I wish I could say "the right way to handle Unicode in Lua is to X", but I haven't found X yet. slunicode at least allows you to do string matching on UTF8 strings: http://files.luaforge.net/releases/sln/

json.stringify: I suggest escaping strings by some character that cannot occur in identifiers or numbers, e.g., ":". Then the above would map to { "1" = 11, ":1" = 11}.

Another question: does your mail function send mail from your server? How do you avoid users sending spam?

I'm confused about your stringify suggestion... I think the primary use of json.stringify is to send that JSON to some API or back to a browser, but nobody else would be able to make sense of the ":1". Am I misunderstanding?

We don't run an SMTP server, so the email (port 25 traffic) comes from our servers but not from our SMTP server. Even still, we will probably have to limit port 25 use to avoid blacklisting. We're sorting that out.

OK, but who will make sense of encoding the array {2,3,5} by the object {"1"=2, "2"=3, "3"=5}? Shouldn't you be trying to represent arrays by arrays?

How about representing Lua tables by arrays with an object in position 0? Then { 11, ["11']=11} would map to [{"11"=11}, 11]. The extra layer of depth is annoying, but the semantics is closer to what you want and you avoid collisions between strings and array indices.

I'm not sure what you mean. json.stringify {2,3,5} == '[2, 3, 5]', which is correct, isn't it? (Strictly, JSON has to always have a dictionary on the outside, but it's fairly standard to allow this.)

We'll keep thinking about this, but I don't think it's sensible to try to encode {11, ["11"]=11} at all. That's neither an array nor a dictionary, and so it doesn't have a natural JSON representation. I'm inclined to make it an error, but right now we try to be generous in attempting to encode.

JSON does not have to have a dictionary at the root. You can have just an array.

Looks like you're right. From http://www.ietf.org/rfc/rfc4627.txt:

"A JSON text is a serialized object or array."

Now I don't know why I thought this wasn't allowed.

I had misunderstood the semantics of json.stringify - I thought tables always mapped onto dictionary objects, even if they were arrays. The semantics I suggested is an invertible way of handling that, though the inverse will be odd for JSON arrays whose 0th element is a dictionary object.

Flagging errors in cases with seems to be a good way to handle this kind of problematic input. It's surely better than having applications have nonsensical data if they are given peculiar input.

Reply on news.ycombinator.com