YAML: Probably not so great after all (2019)
arp242.net
arp242.net
Turns out the country code for Norway, "no", is interpreted as boolean false when unquoted in yaml.
Suggesting that you were using a function as a map and "prototype" as the key seems even more made up.
If you use a template you have to put "" for strings.
That sounds like there are no human-writable formats.
> If your human-readable format requires special cases for certain keywords but treats others as strings, it's not human-readable.
That I can 100% support; sometimes needing to quote strings is a recipe for disaster.
I'm not sure why it sounds that way. There are formats where types are explicit, either by stating them or by their delimiters or by their contents.
https://github.com/adrienverge/yamllint
It can be configured to require strict string quoting.
https://yamllint.readthedocs.io/en/stable/rules.html#module-...
And to explicitly restrict bools to only `true` and `false`.
https://yamllint.readthedocs.io/en/stable/rules.html#module-...
Trying to get to almost jsonschema level without the complexity of specification and reuse with the actual code
Sorry, but YAML is a disaster and YAML Lint and JSON Schema are weak attempts to improve it.
I'd be happy if YAML evolved to a true schema support, use of brackets or some other tip so white space wouldn't be meaningful, and silly things removed like three hundred ways to define Boolean values.
Edit: I suppose you mean that the config format should be "user friendly". I guess it should, but I have personally had far fewer problems with YAML than with TOML.
I wrote it to read true/false, but then there were some test cases that failed. Turns out, the schema also allows yes/no and 1/0 as synonyms.
Though: now that I think about it, most of the problems happen at the block level, not the line level. So maybe significant newlines are fine but not indentation.
SML and (I think) Ocaml don't, and I had assumed F# was like Ocaml, since its syntax is largely the same. In the tiny bit of F# I tried writing once I don't remember having had to pay any attention to whitespace.
Significant whitespace is one of the things I didn't enjoy about Haskell and Idris. Life's too short to indent code by hand.
let function1() =
for i in 1 .. 10 do
printf "%d " i
printfn ""
function1()
A colleague spent a few hours tracking down a python bug caused by a bad merge that messed up the indent without it outright causing the code to crash, just produce wrong output. That's IMO too expensive a bug type to permit just to save a few keystrokes. But YMMV as they say.
Re: the bad merge, that can happen the other way to: this was the cause of Apple's infamous goto fail bug. I think block structured languages should do what go and some others do: braces on conditionals and loops are mandatory, parens are optional.
That’s a 1000x advantage in my book. Not mention exceptions are often avoided with a professional editor with whitespace highlighting and indentation guides.
You're not even trying to have a reasonable discussion about this, are you.
At least this guy has a datapoint. You just have a hunch that you can write code faster if it's formatted slightly differently. I don't think it's true -- I don't think typing semicolons and curly braces slows down my coding enough to warrant omitting them. Most of my time coding is spent thinking and reading, probably less than 15% of my time is spent typing. So if I only type semicolons and commas 5% of that time, that means I'm saving 0.75% of my time by not typing them, which is about 14 hours a year if I work 40 hour weeks.
I've definitely debugged a yaml error or two due solely to formatting in the past year, which probably cost me at least 4 hours of productivity. If my entire codebase was formatted solely by indents and returns, I imagine that number would balloon significantly, which is why I opt to use semicolons and other formatting characters.
Also, they're not "reading impediments," at least not universally. Being able to use my code-editor to jump to the end of a code block (by matching a { or }) has been immensely useful in my experience.
If you need 4 hours to fix a yaml file, then what can I say? Perhaps you need better tools, or reduce tech debt.
Not to my knowledge, and I have a great interest in UIs and usability, so a link would be very interesting. It would also help resolve a lot of whitespace vs. brackets debates if you can find it, thanks.
> But there are more uncharitable interpretations
That was nasty.
You're unlikely to find many peer-reviewed studies on this, the way you won't find many on proving other things learned centuries past.
- http://impact-information.com/impactinfo/readability02.pdf (about prose but mentions bullet points with space)
- https://www.researchgate.net/publication/320407173_A_COMPARI...
- https://alistapart.com/article/whitespace/
- https://softwareengineering.stackexchange.com/questions/2985...
- https://ieeexplore.ieee.org/document/8091188
- https://dl.acm.org/doi/10.1109/ASE.2019.00047
A quicker method however: Do you find regular expressions more readable than string methods? If so, you're highly unique. If not, you already agree.
That paper's worth nothing. It's literally schoolwork.
and one I looked at is https://alistapart.com/article/whitespace/ Whitespace is used there as a feature of graphics layout mainly (though text too, as margins, space between lines etc). From it
“Whitespace,” or “negative space” is the space between elements in a composition
As for the question about regexps, they aren't punctuation that punctuates anything else, they're a mini, dense string language. They reveal nothing about python spaces vs braces, and nothing about punctuation in an informal or formal language.
To answer your question about regexps, which is clearer depends on what's required. A common emacs regex is to remove left-spaces from data:
^[ \t]+ -> (empty string here)
To swap 2 fields separated by a space (assuming the 2 fields don't contain spaces, common in data scrubbing):
\(.\) \(.\) -> \2 \1
In these cases regexps are cleaner than methods, but that I find it so neither supports nor denies agreement with you.
This is silly. Good night.
Yes, recognizing a bug, investigating it, sourcing it to a yaml file, finding the offending line, fixing it and testing it, and getting the fix merged up probably took me a minimum of 2 hours, and I had to do it at least twice.
You just handwave away every argument that is brought against you, without actually thinking about what you're saying or if it's even true. You're the worst type of engineer.
Apple's infamous "goto fail" was in part caused by a mismatch between how the computer understand the block level and how the programmer did.
Is it possible to write a formatter that would fix an issue like that for yaml or python for example?
In short, there's no issue to fix.
Just not by a regex, which so many formatting tools rely upon.
if (test)
do this;
if (test) do that;
if (test)
do this;
if (test)
do that;
In python if I had a similar mis-indented construct how could you possibly know if it is supposed to be a nested if statement or not?
I'm not picking on python I'm curious. My preferred language is clojure. I feel like the python whitespace is probably like lisp parens. It doesn't matter after the first week of learning.
But when I'm typing clojure in emacs I don't care about formatting my code at all. Emacs sees the parens and indents correctly for me whenever I hit enter. I could probably write all my code on one line, run it through a code formatter and get a reasonable result.
On the rare occasion I'm fixing some python when I start a new line I sometimes need to think about indenting and tab cycle until I hit the right depth.
int foo() {
a();
b();
c();
if (test())
doSomethingIfTest();
doSomethingUnconditionally();} def foo():
a()
b()
c()
if test():
doSomethingIfTest()
doSomethingUnconditionally()
Your issue can be fixed with a linter, mine can't.
Look at this python code: https://github.com/django/django/blob/master/django/core/mai...
In my opinion, it's rather difficult to tell where one if/else block ends, and the next begins. At a glance, I couldn't even tell you how many if/else statements there are. In my opinion, braces give me a frame of reference for my eyes to lock into, even in my periphery, and give me a far better understanding of the code structure.
Structured data with only vague, and often misleading, indication of its structure is awful to work with. Sure, it's "human readable", but if your data is of any real complexity you're almost certainly better off just machine-generating it.
My personal belief is that XML was "good-enough" having been engineered for many use-cases and easy to comprehend for what it provides.
It's a tragedy that XML was abused so vigorously in the early naughts. People got so sick of it because they were compelled to edit XML manually in a text-editor with shitty to none schema support. It didn't help that the worst monstrosities of that era (soap and ws-* crap) heavily invested in turgid, badly designed XML.
I think that if there had been some more effort on tooling and good-practices, XML could have remained popular and we would not have had jason, yaml, and whatever awful thing is next.
One of the selling points of json is file size, xml started dying when people realized that more than half their bandwidth was going to xml structure rather than actual data
"Yet Another Markup Language (YAML) 1.0"
But also, I'm not wrong per the current version of the spec: https://yaml.org/spec/1.2/spec.html
"YAML Ain’t Markup Language (YAML™) Version 1.2"
One of the things I most dislike about yaml is that they persuaded JSON to remove comments from the spec for some psuedo compatability nonsense. Without comments, json is less useful for config files and working/documented examples.
I have written a pre-parser to permit comments in JSON and for nodejs apps I use javascript as config. Naturally where security is not a concern.
Yaml always seemed like a mess to me.
Liking Toml, decent compromise.
And linux style: name space value, hash and semi for comments. Unless I really need a heirachy.
JSON5 allows comments and a few other niceties. I'd trust JSON5 parsers more than my own hand-rolled one
The human unreadability if JSON is greatly exaggerated. While you can get horrible looking JSON, you can also then pretty print it into something much better. If we could just all agree to allow comments into a spec it’d be fine.
I also detest this, but it's not really about YAML - it's ridiculous that so many of our tools no longer allow any validation / linting of files before attempting to use them; some that do require the "full service" running to provide sufficient context; even those that don't don't always give good (i.e. consistent, structured) errors. This problem would remain if the format was JSON, TOML, or even XML when working without schemas for every namespace.
The worst is the tools that layer Jinja on top of the YAML (Salt/Ansible/Puppet) which is basically impossible to validate statically. At least with GitLab, Docker Compose, and Kubernetes I have some hope - but integration into other tools is awful.
> Don’t get me wrong, it’s not like YAML is absolutely terrible – it’s probably better than using JSON – but it’s not exactly great either. There are some drawbacks and surprises that are not at all obvious at first, and there are a number of better alternatives such as TOML and other more specialized formats.
> One good alternative might be to just use commandline flags.
> If you must use YAML then I recommend you use StrictYAML, which removes some (though not all) of the more hairy parts.
(I do agree YAML is in retrospect a mistake. The reasons why remind me in some ways of the problems with Markdown).
(Oddly, I can't seem to find an up to date TOML parser in ruby that supports the 1.0 spec...)
YAML is a complex and unintuitive mess that allows doing every thing in a million ways. I’m surprised it ever got so much traction. TOML is a breath of fresh air next to it.
{"3":{"14159":"pi"}}
That's false. https://metacpan.org/pod/JSON::XS#JSON-and-YAML
"Who approved this?!"
I can't imagine the cluster fuck of ideas in a person's head to lead to thinking that this was an ok design for a configuration file. The person who designed this and I just don't live on the same planet.
And here I thought the reason I didn't use YAML was because the syntax looked stupid.
A more conservative implementation wouldn't have that particular vulnerability.
!!python/object/apply:os.system
args: ['ls /']
Authors of some libraries did not consider the security implications of the object serialisation part of the spec. I assign the blame to them for not looking beyond their own limited horizon, metaphorically speaking. When they made their libraries, the reference implementation was already available and it was secure by default.
That does make it closer to an unfortunate (but entirely avoidable) accident rather than what you believe, that evil people maliciously and intentionally added a backdoor. You can keep believing it, but that does not make it any more true.
Debugging nearly invisible indentation problems can be such a pain.
however, this biggest issue why YAML is my least favorite config format. unquoted strings with colons is another top one.
When I first saw YAML, it was nice but it felt a bit too complicated.
All I really want is Indented JSON. New lines instead of commas. That’s it.
Json is fast to parse. See simdjson doing it at > 2.5GB/s. One can’t do this with YAML where no could mean many things.
And god I hate k8s for their bajillion yaml configs. Thank god for jsonnet that can dump to yaml. Jsonnet is truly nice and makes working with json like configs a pleasure.
Serious question: why doesn’t this language space have a universally adopted candidate like SQL?
JSON is pretty common, and liveable with a few conservative extensions like comment support. But the extensions haven't gained a lot of traction yet.
But everyone thought "that's too complex" and so reinvented something simpler. Then kept adding features, and more features, and now YAML is more complicated than XML.
This process will repeat ad infinitum.
It is so ironic to see these fads come and go, and then one wonders why we are so cinic regarding newcomers.
One can use the XML Configuration Provider just as well.
We're on a mix of 2.2 and 3.1 where the default still is json or have i got this wrong?
I just thought that XmlConfigurationProvider wasn't in 1.x, as I avoided that crash train with multiple reboots how projects and .NET Standard was supposed to look like.
So I stand corrected regarding when it was introduced.
TOML is mentioned but I do not see wide adoption, e.g. no up-to-date Java or similar implementation. (https://github.com/toml-lang/toml/wiki#implementations)
Additionally I think TOML is more verbose compared to YAML but avoids the indentation ugliness.
Python gives you all the power of scripting including validation and json gives you the easily readable format for both humans and machines. Great for knowing exactly what the settings evaluated as.
And when you don't need the power of full expressive python script, you can just json.dump() a python dictionary and be done with it.
I've also used sqlite as a config file format and that is very polite and easily read from anything I use.
There are tons of IDEs supporting XML (syntax highlight & collapsing brackets, etc...). XML schema is also great to ensure the config is conforming. I keep thinking we are trying to reinvent a worse wheel here.
yaml.load is now a wrapper around yaml.safe_load that negates the risks he highlights
If yaml is not working for you have to search a serializer format that fit in your project.
There is not silver bullet.
If I need a serializer format, it's not human-readable/writable, and I might as well have something like XML or s-exp that are easier for many tools to work with, more easily composable, easier to generate automatically, etc.
> There is not silver bullet.
This phrase needs to be retired. It's true, but invoked far too often to excuse shit bullets.
I beg to differ.