Hacker News Reader

Comment by dig1 | Hacker News Reader

> is that for a long time it was the only viable configuration format

Actually, this is not the case. We had INI format for simple stuff and XML (protected with entire schema) for complex things many years ago, which worked. Yet, we wanted something readable (like INI), but able to express complex types (XML).

I don't think Toml is a viable replacement - for me, it has an INI-level of simplicity with even worst hacks to have nested structures. But, give it time and you'll have another YAML.

But yes, YAML is confusing for edge cases (true/false, quoting), but I'm going to find a powerful replacement that is not XML. Maybe EDN can jump in, but for anything more complex, I'd rather have a Lisp interpreter and structures around than any of mentioned and upcoming formats.

Thinking that JSON is a suitable replacement, imagine writing Ansible or k8s stuff in it; that would be fun ;)

View on HN

> Thinking that JSON is a suitable replacement, imagine writing Ansible or k8s stuff in it; that would be fun ;)

Writing in YAML doesn't feel much better. YMMV but I've been on teams using Pulumi for k8s and the developer experience has been significantly better. I can automate, type check, lint, click through to definitions the same way I do with other typescript.

Pulumi is a young product with many rough edges but it's already been a game changer for me.

JSON has no comments. That disqualifies it immediately.

    {
        “/*These are the ports we use”: “*/”,
        “ports”: [22, 80, 443]
    }

Now, write a json-schema for that.

Can be done by either additionalProperties: true and not specifying the property, or "patternProperties": { "^/\*": { "type": "string" }}.

I'd rather have real comments than this epicycle.

"additionalProperties": true

“Comment”: ”This is a comment that is optional and ignored by code”

"Comment": "but you can only have one per object"

"Comment1": "This is the first comment"

"Comment2": "This is the next comment"

"Comments": ["Here is one", "Here is another!"]

"Comments": { "Name": "This is the name of the customer", "Age": "This is the age (0 to 1000) of the customer" }

"Values": [ {"Name":"Age", "Value":"20", "Comment":"This is the age of the customer"}, {"Name":"Name", "Value":"Mr. Foo Bar", "Comment":"This is the name of the customer"} ]

The only limit is your imagination.

Many JSON dialects do in practice.

I checked the sample Kubernetes project in Java [0] and it looks hideous. YAML looks much better in comparison.

[0] https://www.pulumi.com/docs/get-started/kubernetes/review-pr...

What, you mean you don't want to write code like

  Deployment("nginx",DeploymentArgs.builder().spec(DeploymentSpecArgs.builder().selector(LabelSelectorArgs.builder().matchLabels(labels).build()).replicas(1).template(PodTemplateSpecArgs.builder().metadata(ObjectMetaArgs.builder().labels(labels).build()).spec(PodSpecArgs.builder().containers(ContainerArgs.builder().name("nginx").image("nginx").ports(ContainerPortArgs.builder().containerPort(80).build()).build()).build()).build()).build()).build());

I actually laughed out loud when I looked at the example and just saw a bunch of carefully-indented “.build()”s

Don't disagree there. Java support is less than a year old and the Java API is.. not great. TypeScript is the more popular and mature choice.

It's very simple actually.

... just fucking don't, generate config in your configuration management tool of choice and then serialize it to YAML. You get all of the advantages (nice to read)and none of the disadvantages (need editor or it is PITA)

I would love to swap Pulumi for Ansible, but it's cloud-only for the most part.

> YAML is confusing for edge cases (true/false, quoting)

I'm not sure I would consider booleans an 'edge case'

Booleans aren't an edge case, on is an edge case way to spell false.

specifically according to https://yaml.org/type/bool.html:

     y|Y|yes|Yes|YES|n|N|no|No|NO
    |true|True|TRUE|false|False|FALSE
    |on|On|ON|off|Off|OFF

Not all parsers support all of these options, but true|false is always supported as YAML is a superset of JSON which makes these mandatory.

That is actually only the case for yaml 1.1; the 1.2 spec (circa 2009!) eliminated this forest of boolean encodings, leaving only `true` and `false`: https://yaml.org/spec/1.2.0/#id2602744

I can't imagine thinking this is a good idea if you've done any programming whatsoever. What happened here?

If you aim to be "human-friendly" (and that is, as I understand, the raison d'etre for YAML), there is a subtle semantic difference between "true" and "on" (and "false" and "off") and as a human it may be nice to express that semantic difference.

As for that semantic difference, if we expect the light source to have one of exactly two states (that is, "not a dimmable light"), we probably want to express that as "lightsource: on" rather than "lightsource: true".

And that is where the friction between "humanfriendly" and "computer-friendly" starts being problematic. Computer-to-computer protocols should be painfully strict and non-ambiguous, human-to-computer should be as adapted as they can to humans, erring on "expressive" rather than "strict".

I am also not sure if I am happy or sad that the set of configuration languages in the original article didn't include Flabbergast[1], which was heavily inspired by what may be simultaneously the best and worst configuration language I have seen, BCL (a language that I once was very relieved to never have to see again, and nine months later missed so INCREDIBLY much, because all the other ones are sadly more horrible).

[1] https://www.flabbergast.org/

Even leaving aside the word <on>, it's problematic that, syntactically, <true> and <foobar> look the same but are treated differently.

Its something of a tradeoff space that is always there you either have things that look the same be implicit (eg is 1 and 10000000 the same type? How about 1 and 1. ? How about 1 and -1?, how about 1 and moo? 1e2 and 1f2?

You can have things be explicit and verbose (have explicit unambiguous identifiers on every value), or you can have things magic and then surprising in something like this.

Rails also made a similar decision to basically auto convert what would be the string true to the Boolean value true in many cases. It's not obviously right or wrong, it's just a choice that has tradeoffs.

I think my own sensibility is that it's actually reasonable for something like yaml to allow you to choose to omit quotes and if you do then it's automatic typing: 1e2 is a number and 1z2 is a string. If you want it to be a string you can put quotes on every value if you want, exactly like you would in json, choosing unquoted for things that are strings is choosing the automatic typing case.

But these other values like y and n being autotyped as bool are suspect autotype behavior to me: even once you've accepted unquoted is autotyped you are more likely to be caught off guard by this than by 0 vs true vs 0z.

I'm nose deep in an Oracle to Postgres conversion at the moment and from that experience, among others, I can absolutely assure you that although booleans are definitely simple they are also most definitely a very sharp edge case.

But who doesn't love the OracleDB tri-state boolean?

Booleans, or NULL specifically?

yes (in oracle - at least the early 21st century version I'm using you have to roll your own boolean-ish - traditionally char(1).

The "edge case" here is "I'm using dynamically typed language and some parsers implemented alternative wording for true/false"

It was a part of the YAML spec for a long time, so it has nothing to do with dynamic typing or parser extensions.

It does. If I define struct to contain boolean, Go yaml parser will always parse it to boolean

If I define it as any other type, it will parse it as that type. So any conflicts like in the article just don't happen, or at worst, produce error in parsing.

Static typing in this case acts essentially as schema

Go isn’t the only language using YAML

XML did everything and it's perfectly readable if it's formatted and structured well. This zoo of different markup/object/whatever languages we have to deal with now is largely a mistake.

This is confusing because XML is the archetype of what I would consider unreadable. If I got a prompt in a programming language design workshop to intentionally design something unreadable for humans, I would start thinking about XML and see where it leads me. Can you name any language any more unreadable than XML to help me?

https://www.mssqltips.com/tipimages2/2899_img5.jpg

I look at this and I see a visually-well-organized hierarchy of objects with properties and children. It's a sideways tree diagram. I honestly don't get what's supposed to be unappealing about it.

> Can you name any language any more unreadable than XML to help me?

Most of the other markup languages depending on the usecase. The "redundant" tags everywhere like </Address> in XML are very helpful if you're in the weeds of a big document that's taller than your viewport - you still know exactly what you're looking at and where in the hierarchy you are, as opposed to JSON or YAML which I find much easier to get lost in.

Anyone can pick up a (properly-named) XML document and tell what it means at a glance, but JSON is easily a mystery unless you know the implicit schema beforehand.

Maybe there's some minor friction to get past looking at XML because it's a little visually busy before you learn how to parse it out? Is that all there is to it?

I understand where you’re coming from because it’s true XML is going to perform the functions required without all the problems implicit with YAML’s “readable” syntax…

But I have worked with some very complex XML and I feel you’ve chosen a simpler to read example.

I have seen very complicated XML from Microsoft’s own configurations that is so large, long, and complicated that it is very hard to follow

XML is a mistake. 80% of every XML document is just redundant noise because the IBM lawyer who invented the SGML syntax had never heard of S-expression.

The "redundant" noise makes it more human-readable in my opinion. If you're in the weeds of a big XML document that's larger than your viewport, being able to see that you're at the end of a </Address> helps you keep your place, as opposed to something like JSON which I find much easier to get lost in. Even if it's not that big I appreciate the extra rails on the XML making it easier to keep my place.

I consider this a fallacy. The condition you are given is not a common occurence. In the event they do occur, the solution can be found in your editor or viewer. It does not make sense to inject spurious junk into what is designed as a data interchange format, in which includes network information interchange.

There's probably a name for this kind of fallacy that exaggerate the effect of a problem and the resulting contortions that were engineered around it.

Writing documents with S-exprs will get very tedious once you have to deal with stuff like whitespace and such that will inevitably show up when you're dealing with text documents. SGML is specifically optimized for those use cases, which is why it is so much more awkward to use when all you want is serializing some random structured data.

All you need is some sort of Python’s raw string or triple quote or HEREDOC. What’s so hard about that?

Try writing a document that's mostly text, but with plenty of markup interspersed in the middle, in this manner.

We are talking about a configuration language here, which is never about markups. Also, most of the so-called whitespace problem in Ansible or k8 configs comes from embedding config that's not YAML. In these cases you need a path or URI for the processor to resolve before applying, not inverting the entire configuration language to cater to 1 uncommon use case.

The point of my original comment was precisely that SGML was optimized for text documents. I agree that adopting it for configs was a mistake, but the complaint that "IBM lawyer who invented the SGML syntax had never heard of S-expression" doesn't make any sense in that context.

SGML is way shorter than XML.

But as complex as YAML to understand

XML is too much of a pain to write by hand.

XML is like violence. If it’s not solving your problem, you need to use more

XML is pretty tolerable with a slightly more sophisticated editor that supports completion. The things that make YAML awful never get any better.

Throw in copilot to anticipate all the boilerplate and repetition for you and it's downright pleasant.

It’s not that bad really. About the same as HTML, yet more difficult to mess up.

Has comments, unlike JSON.

More difficult to mess up than YAML.

If JSON and YAML were first on the scene; XML would be invented as an answer.

Hard to mess up. Robust schema language. Very flexible. Easy to process.

Yeah, it’s verbose, but that’s the trade-off between usability and robustness.

Now get off my lawn!

What XML replaced was a combination of custom designed (often binary) formats and HTTP query-string like syntaxes. It is quite verbose compared to either of them, and explodes both your bandwidth and serialize/deserialize times, but it is arguably easier for your new co-worker to guess that the address line 2 goes in <address2>...</address2> rather than prefix tagged with 13, or worse yet at offset 42-61.

I got used to XML, tho I never could quite understand XSLT and the desire to program in it. I got used to json, but yaml I just can't bring myself to parse. YAML is 90% stuff you can't guess and just 10% data. And why so many?

> Thinking that JSON is a suitable replacement, imagine writing Ansible or k8s stuff in it; that would be fun ;)

k8s via helm is often templated via go template strings; which works by creating an unreadable and unhighightable mess, introducing lots of its own bugs.

INI has no spec, and there are many variations all slightly incompatible with each other, kinda like Markdown. YAML is really the only sane configuration language that lets you denote nested structures while keeping them look nested.

Basically, all of the problem identified in the article can be dealt with 1 rule - always quote your strings. I agree with the author we should have reduced, safe and minimalist subset of yaml, which is basically YAML 1.2, released in 2009.

P.S. Please just stop using PyYAML.

There's HOCON which is pretty good if you can run on a JVM. It's a superset of JSON designed for readability and human-friendliness when writing config files. It doesn't change the type system and doesn't have yamls weird edge cases, but is still a lot easier to write than JSON. There's also a relatively tight spec.

https://github.com/lightbend/config/blob/main/HOCON.md

We use an extended version of it for our app and the resulting config is pretty clean. You can see an example here:

https://hydraulic.software/blog/8-packaging-electron-apps.ht...

The only downside is that the reference implementation is hardly maintained anymore.

I second that. And if people need to deal with YAML in Python, they should be using ruamel.yaml, which is a far superior library on just about every level: https://pypi.org/project/ruamel.yaml/

Except that the author of ruamel.yaml:

refuses to use git [0]

refuses to take community submissions (except through Stack Overflow? Seems like a misuse of SO) [1]

and refuses to implement .dumps() [2].

He is difficult to work with, and any time I need to debug code that intimately deals with ruamel.yaml types, I wince.

[0] https://github.com/pycontribs/ruyaml/issues/1

[1] https://yaml.readthedocs.io/en/latest/contributing.html

[2] https://stackoverflow.com/a/63179923/15170511

Why would I care that the maintainer prefers Mercurial? The document you linked to doesn't say anything about them not taking community submissions: I think you need to scroll down further in the page. And it certainly doesn't say that you have to use SO to submit changes.

Or if you want to go really minimalist there's also https://github.com/crdoconnor/strictyaml

> Please just stop using PyYAML.

Why?

From the article:

  Although yaml 1.2 is more than 10 years old by now, you would be mistaken to think that it is widely supported: the latest version libyaml at the time of writing (which is used among others by PyYAML) implements yaml 1.1 and parses 22:22 as 1342.

Jsonnet and Cue are the places to be looking.

Cue is one of the most exciting developments that's impacted me professionally in recent years. I can't advocate for it enough.

Cue is interesting, but why is it only available as a command-line tool rather than a library? I'd want to integrate such a configuration language in my programs, so I could use its evaluation and validation capabilities rather than writing a custom parser/validator.

If you're writing in Go, you can use it as a library. This is poorly documented, unfortunately.

Also their website is terrible. One of those projects that assumes you've already decided (or been forced) to use it and have cleared out a week of your schedule to learn how to use it.

I learned cue from it during one weekend with plenty of time to play with kids, using it in production since 2020, it's been absolutely great, zero problems, very terse configs, intuitive formalism.

I took another look and eventually found the bit of the website that they should put front and centre in the Tutorial page. Still difficult to navigate (why doesn't the page tree show up on the left?) but it is at least well written and to the point.

The "learn more" button on the front page should link to that, perhaps with a single paragraph giving motivation.

And the main page breaks the fundamental rule of programming languages/formats. Put examples on the front page!

I assumed they hadn't done that because the examples would be too complex or maybe the concepts were too difficult to demonstrate with small examples but having gone through the tutorial that isn't there case at all.

Are you talking about https://cuelang.org/docs/tutorials/tour/intro/ ? Even that is a bit light on detail for real usage, while https://github.com/cue-lang/cue/blob/v0.4.3/doc/tutorial/kub... is kinda rambling. I don't think that there's "One true way" to introduce these concepts; How you teach cue to a config-generation novice is very different from someone who's used to using an IDE to generate kubernetes YAML.

Spec [0] is very good + practice with small files.

[0] https://cuelang.org/docs/references/spec

I find specs nearly unreadable when trying to first digest a language; While invaluable for advanced usage and implementation, I can't read a BNF-Style Spec and make heads or tails of what's going on unless I also have an annotated example next to it.

I agree. The website is heavy on theory and very light on practical usage.

No. XML is broken for structured data that isn't HTML because it's not really clear (there aren't even "best practices" afaict) what should be a text node, what should be an attribute node, and what should be a subtag node.

It's not like other formats don't face similar questions. E.g. if you have a list of key-value pairs to serialize to JSON, do you translate those keys to JSON properties in an object, or do you translate each pair as an object in an array?

The motivation for doing that in JSON comes with best practices: you use the key-value objects thing if and only if you have an ad-hoc list of items that you'd really really like to shove into a well-typed schema.

Use S-Expressions?

My distaste for yaml led me to attempt just that. The first piece that was missing for me was a python parser that could produce reasonable error messages and could be transformed into the desired internal representation in python. So I wrote one [0]. It was supposed to be a single file that was less than 100 lines and could be copied and pasted into any project that I needed. Turns out that the issue was a bit more complex.

The issue is that there is sufficient complexity in finding a portable representation for configuration formats that it just kicks the can down the road. On the other hand it means that as soon as you decide what format you are going to support you can quickly implement it. There is more or less a intersectional grammar that works across most if not all lisps, and that is the plist `(:k v :k2 (:k3 v2))`. So I settled on that for my own use.

After all that work I have not dealt with the fact that numbers and chars do not have a portable representation across lisp dialects, which is a key complaint in other threads here. Limited support for let binding constants also seems like a feature that would allow for just enough expressivity to make the format useful without opening up the terror that is `&` and `*` in yaml (cool and useful as it may be).

In summary s-expressions are: 1. missing good parsers in a number of language ecosystems 2. not standard across lisp dialects 3. need additional semantics for binding, multiple expressions, etc. 4. still better than yaml and json

0. https://github.com/tgbugs/sxpyr

I know what s-expressions are, vaguely. Vaguely in terms of "I couldn't write a grammar for them off the top of my head.", that is, not "what are they".

Is there a single agreed-upon defined grammar that everyone can use? Preferably one simple enough that like JSON's it is at least capable of being used as a graphic on the home page for the format? https://www.json.org/json-en.html

This is an honest question, because there may well be and I don't know it.

However, I will put this marker down in advance: If multiple people jump up to say "oh, yes, of course, it's right here", and their answers are not 100% compatible with each other, then the answer is no.

The other marker I'll put down is "just use common lisp", I want verification that it really is 100% standardized, no question what any construct means, ever, and I still bet we get people who would rather see Scheme or Clojure, and I bet there's some sort of difference.

Neither of these objections is fatal to the idea. JSON is technically not just "javascript objects", so if someone carved out a defined format from s-expressions, then held it up as a standard, that would be as valid as what Crockford did. But at least as of right now, I'm not aware of anyone having done that standardization work. Replies welcome.

Dune [0] is using sexp [1].

[0] https://dune.build

[1] https://dev.realworldocaml.org/data-serialization.html

I don't see a grammar I could use for an external standard there. I could be missing it. And I assume there must be one implied, since it appears to be a read-write format. However such things have a nasty habit of turning out upon further inspection to not be as generic as supposed and have local idioms buried in them, often in surprising places. Even JSON, simple as it is, had that problem, and it is after all a subset in an attempt to squeeze out those localisms.

Is it compatible with Common Lisp, such that all such s-expressions will be compatible with it in some natural way?

Grammar is here [0].

ps. there is also jq like cli [1].

[0] https://github.com/janestreet/sexplib/blob/master/src/parser...

[1] https://github.com/janestreet/sexp

  SEXPR := YOUR-CUSTOM-LITERALS
         | "(" SEXPR* ")"

Ah, no types, no NIL. Every user has to specify how their usage expresses strings, integers, booleans, etc.

Maybe start from one of the already existing standards https://en.wikipedia.org/wiki/S-expression#Standardization ?

Along with the many other issues that would have for just straight-up not being a usable grammar, the word "CUSTOM" appearing in a nominal cross language standard is a non-starter.

Oh come on. Did you think anybody would write an RFC in the comments? Add arbitrary list of whitespace tokens everywhere and pick whatever favorite grammar you have for literals. Json strings and numbers since you seem to like them.

Of course not. But you are at least semi-seriously trying to propose something. I promise you it won't be that easy to get any sort of agreement.

This is a classic "oh come on how hard can it be" that, once you get into it, it turns out the answer is very. Arbitrarily so. Agreement is hard. Details are hard. A standard with no details is not terribly different from "just give us some text".

Well, I would have suggested Racket. Common Lisp is a bit ugly, both conceptually and syntax-wise.

With each tool using its own macro-based DSL for managing them. That would be fun

S-expressions do not necessarily imply Lisp nor an executable environment, it's a (family of) syntax(es) to build a document format. Like EDN.

So even worse as you can't use any prior knowledge of lisp

JSON is good because it doesn't let you do stupid things like writing programming languages in it.

This is exactly what i'm doing with this YAML to SQL converter: https://gist.github.com/revskill10/57ecd8efb72f361b93e6d9d9f...

Why???

- No parentheses is needed

- No forced order of select, order, having,... Want from before select ? Fine.

- Composability

- Extensibility.

- Security: Prevent SQL injection when sent from browser.

Is INI a format in the sense that it has a formal grammar and a bunch of compatible parsers and serializers, or is it more of a convention?

Reply on news.ycombinator.com