Why JSON Isn’t A Good Configuration Language (2018)
lucidchart.com
lucidchart.com
JSON is fantastic for organizing and passing data around, storing it, even for configuration files where the user doesn't need to edit the raw configuration. If you are like me and write a lot of tools that run in the background, lack a UI and where the only thing a user will be doing is editing a configuration file and running it, you're asking for trouble with JSON. Curly brackets and nesting and lack of basic formatting are bad for UX, and while it's a fantastic compromise for machine and human readability, it focuses more on machine readability and the human readability is just enough for you to look at it. It is not easy for a human to parse with their eyes.
I go with TOML for my configuration files most of the time. I am not a fan of YAML or XML. I don't think those offer any real benefit over JSON, and both have more downsides than JSON. TOML is a breeze to read and understand just looking at it, and is analogous to JSON as far as machine readability to the point that it's trivial to convert to it when you need to.
I do use JSON a lot, even for configuration files, but only when I'm storing a configuration in a flat file that the user should never have to see.
I'm okay with YAML but TBH, and I know this makes me totally uncool, for config that only nests to a single level I quite like INI files. I sometimes cheat slightly and take account of the ordering of entries in a section but generally speaking it's easy to read, easy to write, and very easy to parse.
My solution? Write a DSL with Typescript, and "compile" your configs down to JSON.
Define the structure of your configs as TS types. Write functions with free variables to work as templates, or define snippets as composable fragments. Writing the end result to a JSON file results in transparency (what you see is what you get) and compatibility.
the composition model strikes a good balance between data extensibility / language expressiveness / ease of use.
the generated json leads to easy-to-understand and portable data, and if you write jsonschemas from jsonnet, tools like json-schema-to-typescript [1] make it easy to import a consistent interface, and almost every language has a reasonably up-to-date validation library.
TS's types are easier to use at write-time, but json schema includes a lot of batteries that save a lot of time when you need them, but become annoying in TS, like patternProperties in dictionaries, length constraints in keys. Of course, there are situations like key1 XOR key2 where you will need custom logic one way or another, but given json schema's evolution over the years, I think it's pretty solid. An added benefit is that if you stay in json schemas, you're almost guaranteed a validator exists in $other_language, it _almost_ feels like a first-class construct everywhere.
After thinking for a while, I think that json schema might be the best approach to it (possibly generated from jsonnet if complicated enough).
https://www.blogger.com/blog/post/edit/9049876254685009339/1...
The author of the post is already signalling descent into config hell, they call JSON a language, which it TECHNICALLY is, but it shows their strong desire for more power and features and a descent into config hell.
Having a proper type system removes so many possibilities of inevitable accidental configuration mistakes. One of our other types of deployment projects has a yaml configuration. One time a section had the wrong tab indenting, took a little while to debug through why a table wasn't being provisioned.
So instead of having 20 lines of code for every alert policy like at https://www.pulumi.com/registry/packages/gcp/api-docs/monito... you create a function that generates all the standardised alerts from concise strongly typed config such as
cloudSqlAlerts({
instanceId: 'sql-main',
highCpu: {
percent: 70,
duration: _5min
},
highMemory: {
percent: 80,
duration: _5min
},
highDisk: {
percent: 80,
duration: _10min
},
deadlockDetection: true
},
projectNotificationChannels)The ”DSL” is the typings, and the rest is nothing more than a simple TS program which calls JSON.stringify() with the type-checked config object and writes the resulting string to disk.
The type system also has great second order benefits like allowing us to build an language server protocol implementation for CUE that has rich diagnostics, auto-complete, jump to definition, rename symbol features. Something that cannot be done _as well_ effectively in untyped languages.
I'm still scratching the surface. CUE does more than add types to config and I would encourage you to dig into it if you have spare time. By virtue of providing a great type system, it also manages to reduce boilerplate as yet another second order benefit. Boilerplate reduction is something where tools like Jsonnet attack as a primary goal but over enough time you're back at having seas of untyped config and indirection that are hard to navigate or contribute to and you're back at square one.
Also, imports and dependency management for config will be big
No.
No.
No no no.
Definitely no.
I wish for a standardized configuration format so that I don't have to learn a new format for each software. At least JSON, TOML (and INI) and YAML are widely used enough so there is little chances that you don't know them.
> If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.
When does a key-value configuration format not meet your needs? Using a scripting language is still about setting key/values.
No, I claim that writing your own is *never* an appropriate choice.
So "think long and hard and then say no".
I'm personally of the opinion that using a scripting language is the worst option. Going that route is giving up on most forms of static analysis, since you can't inspect configuration values without executing untrusted code.
> I'm personally of the opinion that using a scripting language is the worst option. Going that route is giving up on most forms of static analysis, since you can't inspect configuration values without executing untrusted code.
Sounds like the original case of the "inner-platform effect". A "custom configuration file" format for a "rules engine" is a scripting language - you will almost certainly make it accidentally Turing complete even if you were trying not to - and it's one that's even less susceptible to analysis than a standard scripting language.
The least-bad solution is an "inner" DSL in your language, IME, with lots of support code to make it as nice as possible (admittedly my experience is mainly in languages that make this easy) - that way you get tooling for free and you can leverage existing static analysis tools. Yes, your users will find ways to shoot themselves in the foot, but they were always going to.
I see what you mean, but if a project is defining its own configuration language, it can impose as many restrictions as it wants on what expressions are allowed. There are some great examples of non-Turing complete configuration DSLs out in the world, though the most successful examples seem to be used by application frameworks (protocol buffers, GraphQL schema language, Thrift IDL, Avro IDL, Smithy) and infrastructure-as-code projects (Terraform HCL, Azure Bicep).
At a previous job, there were a couple popular "inner DSL" projects, but one was written from the beginning to eventually evaluate to a JSON document, and the other was rewritten to do so (rather than be interpreted and acted upon iteratively) because TypeScript and Ruby scripts could just embed too much arbitrary complexity to be reasonably analyzed in toto by humans once projects reached a certain size.
Sure, but it's very easy to accidentally be Turing complete. You added loops? Boom. You added some kind of alias / extract-common-value feature? Boom. Etc.. Of course you can forcibly restrict the implementation (I wrote a Y combinator in one of them and found that the interpreter would detect recursion at runtime and refuse to process it) but at that point you're making your language inconsistent which is even worse.
> protocol buffers, GraphQL schema language, Thrift IDL, Avro IDL, Smithy
Those aren't "rules engine"s though. Once you get to the point of having expressions in your language, you slide down the slippery slope pretty quickly, IME.
Yeah, I don't know why I picked rules engines as an example of things that are hard to configure with JSON. My day job is working on an infrastructure-as-code DSL, and I previously worked on an RPC framework IDL. Both of those projects are replacing an alternative in a traditional serialization language (JSON and XML, respectively) where users spent a lot of their time fighting against JSON or XML gotchas.
The IaC language supports expressions but not named expressions, so we've been able to avoid Turing completeness so far.
{"rules": [
"x > 1",
"x % 2 = 1"
]}
rules = [
x + 1,
x % 2 == 1
]That may be the best approach for a given project! But if you're using configuration to define an API contract, cloud infrastructure, or a query, you might be better off using a purpose-built DSL like protocol buffers, HCL, or SQL, respectively. That approach can let you define a greater level of expressivity than is allowed in a serialization language like JSON or XML without letting config authors write scripts of arbitrary computational complexity (like they would be able to with config in a scripting language).
So I choose a standardized configuration format: JSON
Now I store my config in JSON, and wrote a JSON parser
Unfortunately my JSON parser threw an exception when parsing floating point numbers, and my app did not start anymore.
I also wrote an XML parser. In hindsight, it would have been much better to store my config as XML, since there are no floating point numbers. Parsing floats is incredible complicated.
I have found some success in my org using JSON with JSON Schema [1]. Combine with a json schema aware IDE like VS Code it solves the documentation problem.
> If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.
I have been surprised by how silly that conclusion is, even though it's from 2018.
JSON is so bad it is on the standard library of all major languages, natively supported by php, python, nodejs, javascript, ruby and many others, unlike YAML, TOML, FADFH or whatever solution someone comes up to a problem that is essentially solved.
If you want perfection then by all means reinvent the wheel. But otherwise, don't be an edgy teen, just go with what the entire industry tells you that works.
This is like claiming that HTTP is a good way for people to type messages to each other, because it's in the standard library for all major languages. Existing in a standard language doesn't mean it was meant to be typed directly by humans. The lack of comments was an intentional design choice [1] to make sure its intent, a data exchange format, was understood.
TOML and YAML are each supported by every language you listed.
Ah yes, because writing a requirements.txt is complicated, writing a pyproject.toml is complicated, or distributing your app with pyinstaller is complicated.
The Python ecosystem has many options that solve dependency management, this is what makes the standard evolve (like the PEP about pyproject.toml which is now a standard). This is healthy, and installing Python deps, even native has never been a problem since the wheels package have been standardized.
Can we stop with those obsolete claims that Python packaging is a mess? The package format barely changed in the last 20 years, and every solutions rely on setuptools and/or pip.
Writing it isn't complicated. Getting deterministic behaviour out of it is. (Yes, there's freeze which is better than nothing, but that doesn't help if you're actually developing a project; you can freeze your transitive dependencies, but sooner or later you'll want to upgrade something that depends on something you froze, and so then you have to unfreeze everything and hope that nothing else breaks).
> The Python ecosystem has many options that solve dependency management, this is what makes the standard evolve (like the PEP about pyproject.toml which is now a standard).
The Python ecosystem has many options because they all suck. Every few years someone writes a new one that claims to fix the problems, but they still haven't managed to catch up with where Maven was 20 years ago. (Indeed I'd argue they've actually gone backwards in some ways, e.g. including pip in newer versions of Python).
> Can we stop with those obsolete claims that Python packaging is a mess? The package format barely changed in the last 20 years, and every solutions rely on setuptools and/or pip.
The claims are not obsolete, and the fact that things have barely changed in 20 years is a big part of the problem.
As you said, there is `pip freeze`, but also the Pipfile.lock for pipenv and the poetry.lock for poetry.
> but sooner or later you'll want to upgrade something that depends on something you froze, and so then you have to unfreeze everything and hope that nothing else breaks
Same for Rust with its Cargo.lock, same for JS with it's package-json.lock or yarn.lock, same for everything that lets you freeze your dependencies.
EDIT: That's also what test suites and CI/CD are for.
> The Python ecosystem has many options because they all suck.
They let me write software, manage dependencies and virtualenvs, there is even pdm which supports the recent PEP for __pypackages__ (equivalent of node_modules) instead of a virtualenv.
Can you clarify how every single one of them suck?
> they still haven't managed to catch up with where Maven was 20 years ago
What does Maven do to prevent the problem of upgrading frozen dependencies?
> I'd argue they've actually gone backwards in some ways, e.g. including pip in newer versions of Python
Care to clarify?
You don't need to freeze your dependencies in the first place, because dependency resolution is deterministic. (In the case where you want to upgrade a transitive dependency without upgrading your direct dependency, you specificy it explicitly; if you want the latest available version, or the latest available version with the same minor number or whatever, you can run a command to populate your project file with that, but it's an explicit, visible operation).
> EDIT: That's also what test suites and CI/CD are for.
That's an admission that the dependency management isn't doing its job.
> Can you clarify how every single one of them suck?
My point is that the huge proliferation of options is not a sign that they're good, but rather the opposite. Ecosystems that have decent dependency management don't feel the need to write a new system every couple of years. Maybe PDM is finally the one that doesn't suck, but at some point after six or seven tries that all suck (and yet were always widely trumpeted as "python packaging is fixed now") my confidence is pretty low.
> > I'd argue they've actually gone backwards in some ways, e.g. including pip in newer versions of Python
> Care to clarify?
Sure. I want a single entry point that I can install on my system (via my package manager) and run to build any Python project, when realistically I'm going to be working with a bunch of projects that each have separate sets of dependencies and each require separate Python versions. Putting pip in the standard library means a) half my python installs have pip and half don't b) I have several different versions of pip installed at any given time. And so it entrenches the complexity where I have to have a separate tool like virtualenv for managing which Python version I'm using for each project as well as pip. (Which wouldn't be so bad if accidentally running pip for one project while your shell was in the virtualenv for another didn't fuck up that virtualenv, potentially permanently...).
In Rust you have the amazing serde with serde_json but at this point, you can use toml which is also based on serde. I consider serde as being standard.
In C you have the lib jason which is very good.
In Elixir, I use the compile-time configuration (config/config.exs) with environment variables for production, but that's because in the end, my Elixir system is in a Docker container, running on Kubernetes, with a ConfigMap defining the environment variables, so in the end, it's YAML (or JSON).
The same industry was—substantially still is!—telling us to use XML everywhere. Let's not.
Pretty sure that "all" isn't correct here. Or we have different lists of major languages.
{"this_article": "// would be better",
"this_article": "// if it werent",
"this_article": "// immediately",
"this_article": "// right off the bat",
"this_article": "// wrong",
"this_article": "meh"}
//=> {"this_article": "meh"}
I actually agree, JSON is kind of a pain in the ass to work with. It's shocking to think of a pre-'jq' world, given how saturated we all are with JSON, but we lived like that for a long time. (We also evidently didn't even have Firebug until 2006?[1] Holy shit!)
But is it a problem? I dunno. Is it worth doing something else? Ooof, scary proposition.
I think one of the missing questions we don't ask, don't know is: what tools help us fix our json fast. What tools can see, oh there's a comma missing between these two lines, and just fix it for us.
> * The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.*
This means that different tools can—and, if experience is any guide, will—handle your example inconsistently. This can be especially troubling when JSON processing tools are used internally by some other system like a database, queue or server; most of the time these tools parse and serialize JSON without changing its meaning, but they can change or lose information when people rely on patterns like the one that you used.
I would honestly be surprised if there isn't at least one realistically used tool that interprets your example differently, like producing {"this_article": "// would be better"} instead.
[1]: https://www.ecma-international.org/publications-and-standard...
Edit: I believe a number of these non-determinism did get formally resolved in EcmaScript itself, which is yes, not the same, but is an active, alive, & notable specification with some bearing.
λ> decode f :: Maybe Value
Just (Object (fromList [("this_article",String "// would be better")]))
The handful of other tools I spot-checked all keep the last pair, so it seems like almost but not quite standard behavior.
I was curious about other programs and, searching around, found a discussion[1] about standardizing the behavior to "either reject objects with duplicate keys or keep the last value"; however, from a quick scan, that wasn't added to any of the standards documents I found. Apart from the standard I linked earlier there are a couple of JSON RFCs[2][3], but they basically say "different systems do different things and if you depend on the behavior, your JSON won't be universally interoperable"
> When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates.
> JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences.
If anything, the fact that it is almost but not quite standard behavior probably makes it worse—you can rely on it for quite a while until it unexpectedly breaks and the figuring out why it broke will be a real headache.
[1]: https://esdiscuss.org/topic/json-duplicate-keys
> In the case where there are duplicate name Strings within an object, lexically preceding values for the same key shall be overwritten.
So in JavaScript, it indeed will always work. But yes, other languages & libraries are under no obligation to follow this behavior. As you clearly show, the JSON spec itself highlights it's displeasure with & the unpredictability of duplicate keys, while also noting the two prevailing behaviors: accept last, or fail.
[1] https://262.ecma-international.org/12.0/#sec-internalizejson...
More details if you ever wanted them, here: https://seriot.ch/projects/parsing_json.html
I dont think I implied at all that kson was canonical nor absolute. I think I agreed thay it was unclear .
* https://bind9.readthedocs.io/en/latest/reference.html#config...
* https://wiki.debian.org/Bind9#File_.2Fetc.2Fbind.2Fnamed.con...
* https://www.zytrax.com/books/dns/ch7/#overview
I've been experimenting with NSD (and Unbound) recently, and their configuration is JSON-like. There seem to be some 'arbitrary' attributes that are treated as "top-level":
> At the top level only server:, key:, pattern:, zone:, tls-auth:, and remote-control: are allowed. These are followed by their attributes or a new top-level keyword. The zone: attribute is followed by zone options. The server: attribute is followed by global options for the NSD server. […]
* https://nsd.docs.nlnetlabs.nl/en/latest/manpages/nsd.conf.ht...
It seems to be good/best practice to indent the sub-attributes of the 'top-level' attributes to know where the 'stanzas' for each top-level start and end. The white space has no significance.
While I like the functionality of NSD/Unbound, I lean toward liking the use of braces (curly brackets; {}) à la BIND to explicitly denote stanzas and statements, even if one also uses indentation for human consumption.
* supports comments
* flexible with good signal to noise (no need for so many brackets and commas)
* gets rid of a lot of YAML's complexity [2]
* explicitly typed for less ambiguities [3]
The main page also gives comparisons against HJSON, HOCON and TOML [4]
[1]: https://hitchdev.com/strictyaml/
[2]: https://hitchdev.com/strictyaml/features-removed/
[3]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/
Every time we had issues, it was because we were trying to spin our OWN version of configuration file. It breaks backward compatibility and versioning. Code collaboration was horrible and devs were getting frustrated.
Want comments in JSON? Spend 20 minutes refactoring a single BUILD rule and now humans can write JSON5 that's transformed into JSON
Want more flexibility? Have humans write cue or dhall or jsonnet
Wish you also had a copy of a subset of the same config data in YAML? Easy
Want to write a compile-time check in a programming language of your choice that a certain setting is never missing? Easy
Want all of this to work with reproducible builds across a variety of computers and distributed build farms? Easy
We don't have to let legacy build systems limit our imagination.
* It's in the standard library for most languages so you can write tools that read and write it that have zero dependencies. This is great for portability and for use in offline environments.
* You can use the same JSON files as input for tools that expect JSON (eg Terraform .tfvars.json) and tools that that expect YAML (anything kubernetes related)
* You can easily see at a glance exactly what whitespace characters are in multiline strings like PEM certificate strings. This is useful to avoid LF vs CRLF line ending issues, and ensuring start and end whitespace is exactly what you want to make sure input strings are compatible with whatever is consuming the configuration.
The downside is that it's a little bit ugly compared to YAML, but editor auto-formatting and key-sorting plugins make that a minor issue.
Regarding point 3, are you talking about using \n in single-line strings (the ones you can use in JSON)
I think that the main reason this debate exists is we are using a good data-format (which JSON is) to configure things, which it's not a use case JSON is designed for.
Everything is much more efficient with binary format like protobufs or flatbuffers. And if the format has a schema, the schema is the documentation, an d you do not need to have comment in the config file itself.
The main benefit of a flat file format for configs is that it can be added to source control and edited by any plaintext editor. If I kept config as protobuf, I would need special tools to handle it, which could be annoying (e.g. if I'm remoted into a remote system). It would also make it more difficult to see diffs between different config versions stored in git.
Edit: one thing I think all json python libs suck at is error messages. A syntax error in a json tree shouldn’t require me to rip nodes out to a/b test a parse error. Sure it can’t correctly break down AST but it could at least try to provide something more effective than Exception(f”{line} {col} glhf!”)
But that’s more from the IaaC world.
I guess .net application configuration might still be heavy on the JSON side.
I just feel like this isn’t as much of a problem anymore. There are so many ways to utilize alternatives.
JSON is only a data _format_, not a language. I agree it's awful for maintaining configs.
YAML has issues with the more complex parts of its spec, but it's great for the stuff JSON gets used for.
LuaJIT. Or at least Lua.
Thank you for coming to my TED talk.