seagreen

I think the way to go is definitely by code point. We don't want to pull any more of the Unicode spec into Son than we have to.

Happily sorting is only used to order members within Son objects. Individual strings are never sorted, so there isn't a worry about sorted strings coming out messy.

Good catch. I kind of slacked on this since I knew it would be a solvable problem once everything else got worked out.

Right now the reference implementation sorts by comparing codepoints one-on-by one. When it reaches a codepoint that's unequal or nonexistant it orders the string with the lesser or nonexistant codepoint first.

So in your example, after serializing from JSON and deserializing to Son we get `{"op":0,"öp":1}`.

Two remaining questions:

1. What's the most unambiguous way to describe this process?

2. Right now the comparison is on unescaped strings. Should it be on escaped strings instead?

There's an issue open to discuss this here: https://github.com/seagreen/Son/issues/1

EDIT: I am curious what Unicode recommends for language-aware sorting, even though we're not going to use it. Is this the right place to look? http://unicode.org/reports/tr10/

EDIT2: RFC 7159 has language about equality that's relevant to comparisons. I'm confident now we're on the right track: https://tools.ietf.org/html/rfc7159#section-8.3

jq is great. You can actually see how it's used to test the reference implementation of Son here (https://github.com/seagreen/Son/blob/master/implementation/t...) using `jq --compact-output --sort-keys .`

Unfortunately, jq doesn't provide flags to control scientific/non-scientific notation or which characters are escaped, meaning if you want very tight control over the JSON generated it's not a full option.

(Consider that the motivating example in the Son README isn't all you might want to use Son for. For instance some people need consistent hashing of serialized JSON documents).

Thanks for brining Canonical JSON up. There's an issue to investigate it here: https://github.com/seagreen/Son/issues/7

It doesn't appear that Canonical JSON does anything about redundant escape sequences in JSON. I'm still looking into it to be sure. This is a big part of the motivation behind the Son spec, and represents about half the EBNF: https://github.com/seagreen/Son/blob/master/son.ebnf#L19

For anyone else who knows of more JSON subsets, if you report them here: https://housejeffries.com/page/7 it would be really appreciated. I definitely don't want to do duplicate work.

> I wouldn't bother turning it into a formal specification.

Too late!

> But you can always put JSON through a pretty-printer which puts values into canonical form before diffing.

Son is a starting point for building such a pretty printer. It takes care of messy details like eliminating the redundancies in string and number encoding, so all you have to do to specify the pretty-printer format is say where you want your newlines and how much to indent by.

I wanted to name it something like "mson" for "minimal-JSON", but that's already taken.

EDIT: Not meaning anything is a plus! But I see your point about being un-googleable.

Author here. I originally started this project because I wanted a consistent way to serialize JSON so that the serialized bytes would hash the same way every time.

As I worked on it though I realized it might be of general interested to people. Thus the example in the README of piping JSON through multiple tools without generating trivial changes that mess up diffs.

Most of the decisions I made were clear: no insignificant whitespace, object keys must be ordered, etc.

There are two things I'm still not sure about:

+ Son doesn't provide escape sequences for any Unicode character that JSON allows to be written unescaped. This includes U+007f (ASCII "delete"). Will that cause a problem for many programs? All the other ASCII control characters are required to be escaped by JSON, U+007f is the only one left out.

+ Son doesn't allow trailing zeros in fractions. This means you can't serialize `1.0`, you have to serialize it as `1`.

I was confident in the decision to take out scientific notation (it would be cool if JSON parsers actually treated numbers as being in scientific notation and tracked significant digits, but they don't so I feel like that ship has sailed). Trailing zeros are different though because some JSON generators do use them to distinguish integers from fractions. The problem is that many parsers don't care about them, so you end up in a situation where parsers are tossing out information about documents, meaning they can't serialize them faithfully again which is the whole point of Son.

I think he's talking about how it can be hard to tell what's already been evaluated and what hasn't.

As a huge Haskell proponent: this is a totally legitimate question, sorry you're being downvoted.

Writing extremely performant Haskell is a very specialized skill. Happily, Haskell is still extremely fast even without fine optimizations.

It depends on what you want to do: if you're writing a moon lander and don't know anything about GHC internals you may be overreaching yourself=) But for most things like web apps etc. knowing the basics is enough.

My understanding is that Docker only stays reproducible if after every change you kill the container and start a new one. Otherwise a particular change may only be working because of a side-effect of a change you introduced earlier and then deleted.

This isn't a huge issue, but still it's nice in declarative systems like Stack and NixOS not to have to worry about that kind of thing.

Absolutely. In fact, one of big things holding back Haskell has been the immaturity of its libraries (because for a long time they were built by hobbyist and academics, with little industry support, in a language where lots of things were new and old architectures didn't work well). Happily that's now mostly behind us: https://github.com/Gabriel439/post-rfc/blob/master/sotu.md (editor support being the main exception)

Absolutely! Thanks for standing up for rigor here.

(The actual argument for functional programming is its adoption by elite programmers like Standard Chartered's Strats team, Facebook's anti-spam group, and Jane Street as a whole.)

Consider this situation: three different developers are working on the same application. They should all have the exact same dependencies installed, right? Therefore they should be working of of a freeze file of some kind.

Why use an entirely ad-hoc freeze file when you can start from a known-working snapshot (that some of them might already have installed on their machines!) and modify it from there. I find this the perfect option in this kind of situation, and so object to saying that stack is just for non-experts.

Well, the browser already did replace Unix for most people=( Presumably we'd be trying to design something good and not awful though, so lets not stop there. But it does give us a starting point:

+ one click installation of applications

+ applications are always up to date

+ applications are sandboxed

Then, since we want users to be able to tinker:

+ the source for all software installed through this mechanism is available

+ optional binary caches NixOS style so it's not slow

And while we're talking about NixOS:

+ immutable package directory so you can rollback at will

And then on to data storage, which is my favorite topic:

+ defaults to tags instead of a hierarchy for organization

+ everything is schema'd, so a file saying it's a contact entry or whatever actually has to be that thing

+ data is immutable by default, so undo and versioning can be built-in (you could still turn this off for certain tasks like video-editing where immutability isn't feasible)

I have more thoughts on data storage here if anyone's interested: https://juniorschematics.com/

It works great for me (also on Firefox).

Python is a nightmare for static analysis. Basing anything contract-related on it is the opposite of reassuring.

I think there are some really cool ideas in Etherium. Proof-of-stake is great. Provably terminating functions are great! These ideas deserve to be paired with a higher level language worthy of them.

Nope, they're still using Solidity.

http://www.stephendiehl.com/posts/smart_contracts.html :

> "Solidity, while being an interesting proof of concept, is dangerously under-contained and very difficult to analyze statically."

Solidity docs are here: http://solidity.readthedocs.io/en/develop/solidity-in-depth....

No referential transparency, no purity, no algebraic data types . . . it's scary stuff for writing contracts.

It sounds like the core team has realized what a hole they're in and are taking it seriously, which is great. But I think a lot of the Etherium-related businesses still have no idea what a sandy foundation they're building on. It's going to be ugly.

I actually do use markdown for the git-repo based prototype. My personal website itself actually uses that prototype. You can see an example "page" here: https://github.com/housejeffries-pages/2

Thanks for the nice words about Son. I need to get it completed (the spec still needs some touch ups) and get prototype implementations written, once that's done I think it will be pretty useful.

Thanks for the tip about StrongLink, that's definitely up my alley. Already I'm finding it useful, for instance in the docs it mentions ni URLs (https://tools.ietf.org/html/rfc6920), which is something I'd never heard of before.

If you have any more comments or criticisms of my projects feel free to email me, none of them are quite to the stage I can start promoting them yet, but getting feedback is still super important.

If you read down to the "Problems" section of the page, you'll see that I agree with you. I don't consider my prototype (that uses git) an acceptable solution at scale.

The actual solution would need to be as easy to set up and use as gmail. But if you're going to do the work of setting up hosted personal servers for wiki pages, you might as well do it for all content. And that's actually the plan! I'm working on it here if you're interested: https://juniorschematics.com/

If anyone's interested I've got thoughts on how to improve wikis here: https://housejeffries.com/page/4

My main issue with current wikis is that if you make a contribution and it's deleted, it's gone from the web. I think that if you're making heavy contributions to a page they should start on a personal wiki and then be pulled into the main wiki "pull request" style.

That way if some over-aggressive editor deletes your work it stays on the internet, just on your personal wiki instead of the main site.

Have you read the specs for both? You should read the JSON spec. It will take you ten minutes. The YAML spec (http://yaml.org/spec/1.2/spec.html) may take you a little longer. . .

Thanks for the info. Glad this stuff is being taken seriously by the core part of the community.

Language choice matters. If asked to write a pacemaker in PHP I would refuse.

A flagship Solidity product has already been pwned for $50 million dollars, leading to a hard fork of Etherium. This is not fake "~deep concern~", this is a real problem.

Please consider trying to get off of Solidity at some point.

It doesn't matter how much auditing is done, if you're building on a shaky foundation mistakes will eventually slip through. Solidity is definitely a shaky foundation.

I really admire projects like yours for their daring, but that needs to be backed up with hard engineering work. I would hate to see smart contracts go down the path of the "Internet of things" -- finally succeeding only to cause more harm than good due to compounding security vulnerabilities.

What language are the contracts written in? Do you have a specification of that language?

EDIT: Nevermind, I see in an another comment that you're using Solidity. Thanks for the answer!

Now I have a new question. What do you think about the claim that, "Solidity, while being an interesting proof of concept, is dangerously under-contained and very difficult to analyze statically." (http://www.stephendiehl.com/posts/smart_contracts.html)

I have a lot of accurately placed anger.

Linux is great for experts, and I really respect it for that. It has been very empowering to people like us.

The web is a peasantizing TV replacement and a trash fire. Linux is great on its own merits, not because it enabled this "web" abomination.

(Talking about webapps here, not web sites. Your personal static site is awesome. The fact that many people's experience with computers is remotely delivered JS storing data places they don't control is not.)

> Then Linux happened, then scripting languages like Perl, Python and PHP proved themselves capable of getting the job done, and the modern web came about. This would never, ever have happened without those tools.

So Linux was so good . . . that it enabled a world where normal people don't use it, but instead use JS apps that store information on other people's servers? Because that's what I hear when you say Linux is a success because it enabled the web.

  Maybe it's because I don't have the "depth"
  of some of the HN users, but Go feels great to me.

I doubt I have any more depth than you as a programmer, but I've gone one step farther than you down the language safety progression, so maybe my thoughts from here will be interesting.

When I moved from Python to Go not only did my code become more safe, I actually became a better programmer. There are a lot of silly things that you can do in Python that you can justify saying "just this once!" or "it's only temporary!" that are much harder to do with Go. I started writing a lot less complicated functions that did different things with different types, and moved to writing simpler, single purpose functions whenever possible. This was great and I learned a lot.

Then I moved from Go to Haskell, and the same thing happened again.

Haskell makes you specify whether a type can be null or not. It has referential transparency, so you can't mutate variables as a side effect of a function. It has sum types, eg:

  data User = Unauthenticated | Normal UserId | Admin UserId

.. that make it really easy to enforce invariants and are also super convenient to use because of pattern matching. I write a lot less "stringly typed" code now than I did when using Python or Go.

Now there are downsides -- Go has a great standard library, and it's tooling is fantastic (even down to little things like gofmt). I'm not saying you'd have a better experience with Haskell, just that there are real reasons why someone wouldn't want to go back after trying a more featureful language than Go.

Have you considered replacing the example entirely? A toy program that looks good but is terribly nonperformant is exactly what I'd put up if I wanted to make a Haskell-style language look bad.