TOML: Tom's Obvious Minimal Language
toml.io
toml.io
tbl = {
hello = "world",
}
now if you see "config in yaml" you know nothing, zero, nada, about the format because all versions are so different and everyone implemented the version at the time and didn't bother to mention version. not to mention you can use a dozen syntaxes for yaml/toml and each application may not understand them all.
all this is so silly. we will stick with json and informal-ini forever one way or another.
Many commonly-used standards today weren't created by a bunch of wise men in a room thinking how to bestow their wisdom upon the rest of us, they often originated in real-world applications, were refined over a period of time based pn real-world experience, and then became a standard.
TOML is the same. I hope it will become an RFC some day. We just need to fix a few outstanding issues first.
Most commonly used TOML parsers support 1.0; adding 1.1 support should be pretty easy as the changes aren't that large (I did it in the parser I maintain, and it's 10 lines of code or so that had to be changed, most of them quite trivial).
Without that your config file is a boobytrap, with it everything is fine and your parsing libraries can even be backwards compatible.
(and mentioning version could be a requirement in a future version)
TOML went through some substantial changes in the past with 0.4, 0.5, and 1.0. As I mentioned in my other comment[1], it's not ideal, but it is what it is.
I wouldn't be surprised if 1.1 would be the last version. Maybe there will be a 1.2 to clarify some things, but I wouldn't expect any further major changes.
Great. Very obvious.
In fairness it's probably not too bad as long as everyone actually migrates to the newest version eventually... But that isn't guaranteed - look at YAML. Or even JSONC. VSCode has a hard-coded lists of which `.json` files are actually JSONC. Gross.
https://github.com/avakar/pytoml/issues/15#issuecomment-2177...
Datetimes were clearly a mistake to include too. It took the M out of TOML.
I agree, especially in regards to the comments, because sometimes the data itself isn't enough and additional human-readable context can be really useful!
In that regard, JSON5 is a wonderful idea, even if sadly it isn't widespread: https://json5.org/
It also supports the trailing commas and overall just feels like what JSON should be, to make it better without overcomplicating it.
But if you did want to fix JSON, the yes, trailing commas and comments are the absolute minimum bar, but single quote is actually probably the the third absolute must-fix.
The reason is just that so much JavaScript tooling is now configured to autoformat code (including the JSON bits) to swap " to ', thanks in large part to Prettier (which also should almost never be used, sigh, but that's a topic for another HN bikeshed...)
hexadecimal identifiers, yeah, nah
I'm curious what you have against Prettier.
https://nigeltao.github.io/blog/2021/json-with-commas-commen... https://github.com/tailscale/hujson
I'm unconvinced.
YAML is a proper superset of JSON.
Until you realize you can't actually store real integers because every number in js is a float...
It’s not ideal, but 2^64 integers is also finite.
That's what standards are for, isn't it?
In my experience, violating type constraints causes problems in downstream systems (usually with parsing or trying to operate on invalid values).
Number, as defined by the JSON Schema spec. A 32-bit signed integer. It has a minimum value of -2,147,483,648 and a maximum value of 2,147,483,647
BigInt is defined by various (MSFT, MySQL, etc): -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807
Most systems use a JSON String for large numbers, out of necessity, not JSON Number.
It's like API's that mess up the semantics of PUT/GET so implementing idempotency is extra annoying.
One of JSON's biggest benefits is that you don't need to know the shape of the data when you parse. JSON's syntax tells you the type of all of its fields. Unfortunately, that stops being true with numbers as soon as double precision float isn't appropriate. If you use more digits in a JSON number, you can't decode your JSON without knowing what precision you need to decode your data.
Even javascript has this problem if you need BigInts, since there's no obvious or easy way to decode a bigint from JSON without losing precision. In the wild, I've seen bigints awkwardly embedded in a JSON string. Gross.
Putting responsibility for knowing the number precision into the language you're using to decode JSON misses the point. Everywhere else, JSON tells you the type of your data as you decode, without needing a schema. Requiring a schema for numbers is a bad design.
- Separate int / float types
- A binary blob type
- Dates
- Maps with non string keys.
Even javascript supports all this stuff now at a language level; it’s just JSON that hasn’t caught up.
(in practice for config it usually is but enforcing it is horribly patchy)
It’s in the name, but be careful not to get confused with JSON being JavaScript.
Not following a set standard is undefined behaviour, leaving it up to the implementation is a large problem in other areas of computer science. Such as C compilers.
I wonder how many times this gets violated though, and how many times this “I dunno… you decide” approach causes problems.
Heck, some of them will even choose different endian-ness and sometimes it will matter.
I still remember the first time I dealt with a Java developer who was trying to send us a 64-bit ID and trying to explain to him that JavaScript only has 52-bit integers and how his eyes widened in such earnest disbelief that anybody would ever accept something so ridiculous. (The top bits were not discardable, they redundantly differentiated between environments that the objects lived in... so all of our dev testing had been fine because the top bits were zero for the dev server in Europe but then you put us on this cluster in your Canadian datacenter and now the top bits are not all zero. Something like a shard of the database or so.) We have bigints now but JSON.parse() can't ever ever support 'em! "Please, it's an ID, why are you even sending it as a number anyway, just make it a string." But they had other customers who they didn't want to break. It was an early powerful argument for UUIDs, hah!
Edit: Omg that story. Eep. I guess if someone provided too-large numbers in a JSON format, you could use a custom parser to accept them as strings or bigints. Still, that must have not been a fun time.
So either you don't target javascript (which would be a bit silly in the case of JSON), or you go the other way and forbid integers, even in languages that do support them. Which is also kind of silly.
Ultimately the real issue is that javascript doesn't have integers and if you're interacting with it, you need to be aware of that, JSON or not.
You could declare your own "int32" type[2] for example, and use that. Then validate the input JSON against the schema before parsing it further.
[1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-sch...
[2]: https://json-schema.org/draft/2020-12/json-schema-core.html#...
The baseline is anything written in C and C++, which don't have bignum or decimal types and so more or less always parse JSON numbers to either int64 or double, at best.
JSON !== JS
{name: "value"}
It's a slippery slope ... pretty soon it's hard to write a JSON parser, and there are more bugs
The trailing comma one is trivial, I'll grant that
The 123_456 syntax comes close to that, but is much less impactful.
None of those will make the language hard to parse.
print("{")
for elem in list:
print('"key": "value",')
print("}")
print("{")
contents = ""
for elem in list:
contents += '"key": "value",'
print(contents[:-1])
print("}")In such a hypothetical format, eliminating nulls entirely should Also be considered, the difference between a missing key (undefined) and a null value is significant in JS, but other (particularly statically-typed) languages struggle with differentiating those two cases, and this leads to `serialize(deserialize(a))` representing a different document than `a`.
edit: just found https://github.com/toml-lang/toml/discussions/915#discussion..., super happy to be able to rest this out early
[1]: https://json5.org/
Hence the "O" in "TOML" ("Obvious"). And this is the use case for TOML, simple user facing configuration that they are very likely to just get right.
JSON is fine for more intricate data structures or very complex configuration, but if you just need them to enter a few numbers and booleans it is overkill.
Apple's XML plist format seems like a mistake, though maybe the newer JSON format is OK.
> JSON is basically perfect if it allowed trailing commas and comments
Apparently Apple actually supports JSON5, an extended JSON based on ES5 that allows trailing commas, comments, and (my favorite!) unquoted key names, among other convenient features.
Here's the toml.io example in perfectly legal YAML but using only YAML's nicer, JSON-like, compatibility grammar:
title: "TOML Example"
owner: {
name: "Tom Preston-Werner",
dob: 1979-05-27 07:32:00 -08:00
}
database: {
enabled: true,
ports: [8000, 8001, 8002],
data: [
["delta", "phi"],
[3.14]
],
temp_targets: { cpu: 79.5, case: 72.0 }
}
servers: {
alpha: { ip: "10.0.0.1", role: "frontend" },
beta: { ip: "10.0.0.2", role: "backend" },
}
Multi-line strings in YAML are also very similar to TOML and you can ignore all different character mixups and stick to '|' (w/ newlines) and '"' (no newlines).
# same as " hello world! "
str1: "
hello
world!
"
# same as "apples\noranges\n"
str2: |
apples
oranges
# integers
int1: +99
int2: 42
int3: 0
int4: -17
# hexadecimal with prefix `0x`
hex1: 0xDEADBEEF
hex2: 0xdeadbeef
# octal with prefix `0o`
oct1: 0o01234567
oct2: 0o755
# fractional
float1: +1.0
float2: 3.1415
float3: -0.01
# exponent
float4: 5e+22
float5: 1e06
float6: -2E-2
# both
float7: 6.626e-34
# infinity
infinite1: .inf # positive infinity
infinite2: +.inf # positive infinity
infinite3: -.inf # negative infinity
# not a number
not1: .nanJSON6 proposal [1] supports all of this, except (4). Unfortunately it also supports more and make the spec a bit too complex for my taste (JSON has a compact spec; any extension should honor this). Same issue with JSON5 proposal [2].
EKON [3] is certainly the best candidate. However, it didn't get any traction.
[1] https://github.com/d3x0r/JSON6 [2] https://json5.org/ [3] https://github.com/ekon-org/ekon
That being said, it would be nice to use railroad diagrams to describe syntax in our spec.
It doesn't help that there's an insane amount of bike-shedding going on in this space. The people who have forked JSON5 to create JSON6 for example have created an endless amount of confusion over which standard to use, harming both of them. I think they might want to think really hard whether the inclusion of octal literals was really worth the hostile fork.
https://stackoverflow.com/questions/3503102/what-are-top-lev...
(Exploits also depended on a form of cross-site request forgery that (a) has been well-understood and avoided for fifteen years now (and with a perfect solution available for five years via the SameSite cookie attribute), so if you’re affected you very probably messed up in other exploitable ways too, and (b) is often even protected by default now: Chromium switched the default to SameSite=Lax in early 2020, so the sensitive cookie would need to be explicitly set with SameSite=None in order to be vulnerable at all. Safari and Firefox haven’t yet shipped this behaviour, though they all agree they want to, since it does break some older sites.)
CORS only stops you from fetching and reading resources, not from evaluating JavaScript with <script src="…">. (Fun related topic: JSONP.) This vulnerability depended upon the fact that a JSON array literal is a valid JavaScript program that just does nothing (unlike a non-empty object literal, since its opening { will be treated as a block, and so `{"key":` triggers a syntax error). Thus, you could use <script src="/path/to/json/array"></script> and it’d run just fine, doing nothing—except that back then you could intercept array and object creation. That was the crux of the vulnerability, and that was fixed in ECMAScript 5.
As for sane default browser policy: the only policy that has changed here is SameSite=None → SameSite=Lax on cookies, and that has actually still only shipped in Chromium-family browsers.
Yaml is one of the worst config languages.
Comments are preserved when you read a file, change some of its contents, and write it back out again.
Why do you require conversion in this use case?
You're 90% way there to YAML.
And YAML 1.2 cleaned up most of the annoying YAML edgecases too
Jsonnet is quite mature, with fairly wide language adoption, and has the benefit of supporting expressions, including conditionals, arithmetic, as well as being able to define reusable blocks inside function definitions or external files.
It's not suitable as a serialization format, but great for config. It's popular in some circles, but I'm sad that it has not reached wider adoption.
I should rewrite the project again. But be as simple as possible.
That was my first C project ever during my University. :P
The fixes for these are both fairly well known (always quote strings and keys, use safe_load or an equivalent API), but it's very easy to make a mistake.
TOML, by contrast, is much simpler - strings are strings are strings, they must always be quoted but other than that behave pretty much as expected. Similarly, there is no mechanism by which a TOML file can instruct the parser to execute arbitrary code as part of deserialisation, which makes it (a) a lot simpler, and (b) a lot safer.
This was fixed in YAML 1.2, but the problem is that almost nobody uses (and there is patchy support for) it.
In particular, the plethora of boolean values... here let me just grab the regex from the spec [1]:
y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF
Then there's the language-specific types or, as YAML calls them, "local tags". The stuff preceded by `!` like
!!map {
? !<tag:yaml.org,2002:str> "foo"
: !<!bar> "baz"
}
I'll admit there are some other, er, "complications" that I actually like. In particular, anchors (`&FOO`) and aliases/references (`*FOO`) [2] make it possible to describe arbitrary graphs, and I find it very useful to factorizing blocks. Also, "folding" [3] text, which allows you to spread value/text across multiple lines for readability, while respecting the surrounding indentation, is neat.
[1] https://yaml.org/type/bool.html
Language tags can get weird but they're ultimately just additional types implemented by your yaml parser you should basically always turn off/not turn on unless you know you want it. You can serialize classes in your language without it.
Tags are really good for readability if you use them thoughtfully
value: !!binary
base64string
# ordered map
value: !!omap
a: 123
b: 455
I don't mind TOML but I don't find it either intuitive or obvious. The syntax for how nested things are flattened I just find really hard to read and write. It's fine though, not that big a deal.
[servers]
[servers.alpha]
is error prone. An sub heading shouldn't need to know about its parent, much less to have to copy it entirely.
To the very best of my knowledge, Properties are merely Map<String,String> (see: https://docs.oracle.com/en/java/javase/11/docs/api/java.base... ) and any subsequent interpretation happens in the application (e.g. Spring treating `foo[1]=bar` as a list-ish assignment but the actual key returned from `getProperty` is the string literal as written `foo[1]`)
Python 3.11.3 (main, May 4 2023, 05:53:32) [GCC 10.2.1 20210110]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.
In [1]: !cat /tmp/x.toml
[servers.alpha]
x = 10
In [2]: import tomllib
...:
...: with open("/tmp/x.toml", "rb") as f:
...: t = tomllib.load(f)
...:
...: t
Out[2]: {'servers': {'alpha': {'x': 10}}} --allowed-downtime-0-start=20230523170000Z
--allowed-downtime-0-end=20230523170500Z
--allowed-downtime-1-start=...
--allowed-downtime-start='20230523170000Z,...'
Genuine question, as I actually like the plain old flagfiles/Java-properties-ish style.
allowed-downtime[0][0] = 20230523170000Z
allowed-downtime[0][1] = 20230523170500Z
allowed-downtime[1][0] = ...
allowed-downtime[0].start = ...aaa.bbb.enabled = true
aaa.ccc.enabled = true
As a human, I prefer to edit by hand an unordered list of key-value pairs (even if the keys are long), than a json object. I often find myself using gron/ungron to edit json data by hand in a comfortable manner. The fact that several implementations of gron exist suggests that I'm not alone in this preference. Editing json requires a global understanding of the structure. But on a list of keys/values there is no order, and each line stands for itself and is understandable in isolation.
YAML/JSON/CSV/XML/etc are not config formats at all. They are data serialization formats. If you don't know the difference, you probably didn't get a CS degree, or read or understand the specs.
Config formats should be tailored to the application, because the entire point of a config file is to make it easy for a human to configure one specific application. That's why there are 100 different config file formats for old UNIX programs. For each one, it's vastly easier to use that one config format for that one application, than it would be to force some arbitrary data serialization format to represent what the human really wanted the program to do.
TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes. The user can express what they want faster, easier, clearer, and the application doesn't have to struggle with alternate implementations of the spec for some poorly defined data serialization format.
> TOML is designed to map unambiguously to a hash table.
Sounds like a data serialization format to me. Besides the obvious differences in design and stylistic choices, it only really differs from JSON or YAML in that the top-level object of the data structure must be an object. If this type of constraint is what makes it a config file format, I am going to change the world with my revolutionary new config file format "JSON, but the first byte must be {".
As another evidence for the blurriness of this line: Cargo uses TOML for its config file (Cargo.toml) which matches the usage that you describe. But then it also uses TOML for the serialization of its internal locking data structure in Cargo.lock, which is a file that humans are specifically not told to edit directly because it just contains serialized data. It just so happens to use TOML, probably because they already had a parser for that available at the time.
> TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes.
Disadvantages of rolling your own format:
- You now have to write your own parser, which costs at least a bit of time and is going to be more prone to bugs than a widely-used deserialization library like Serde. I see that you dunk on data serialization formats being "poorly defined", but if the popular formats are poorly defined, then certainly the average tool developer is not going to do a better job when they actually want to implement the tool and not the parsing for a config file format.
- Users have to learn the format, and have to mentally switch between formats for each application. I just hate how every homegrown config format uses slightly different syntax, esp. which character starts a comment. With `config.yaml` or `config.toml`, I don't have to guess.
- If the config file is YAML or TOML, I immediately have some basic syntax highlighting in my editor.
It's a somewhat ambiguous concept. But generally speaking, config formats lack features that would be useful for a general purpose, and include features that are useful for a specific purpose. For example, it may lack a data type, but it may have some special characters that denote some string is a regular expression. The main difference being how easy it is for a human to deal with it.
The main purpose of a config format is for a human to tell a program how to operate. It is typically distinct from the actual input data used by an application to create output data. It's like the difference between **argv and stdin.
> You now have to write your own parser, which costs at least a bit of time and is going to be more prone to bugs than a widely-used deserialization library like Serde
Actually I argue the opposite. Parsers aren't compatible between implementations which leads to bugs. Specs aren't well understood by either users or programmers which leads to bugs. A home grown implementation doesn't need to be touched after it's first written, unless you're adding or changing the features of your configuration format.
> Users have to learn the format, and have to mentally switch between formats for each application. I just hate how every homegrown config format uses slightly different syntax, esp. which character starts a comment. With `config.yaml` or `config.toml`, I don't have to guess.
Most users never learn those formats properly, leading to false conclusions like the "Norway problem", which isn't a problem with the spec at all, it's a problem of users never reading the spec and attempting to write it (when YAML was never intended to be human-writeable, only human-readable; read the spec!).
Compare this to a home grown config file which is designed to be easy to write, adds functionality to make advanced behavior easier (without adding an entire Turing-complete DSL), and doesn't overload an existing format with broken and confusing changes (see Ansible's and other bastardized YAML hybrids).
"Simplified config file formats" are often just shitty generic DSLs that still lack functionality to actually make the user's life easier. Roll a custom format and hard things become easier [for the user].
I prefer KDL. It's like XML but with almost zero syntax. Probably not always the best idea because it's so niche but it's just really simple while having all the basics like comments and multiline strings.
YAML on the other hand is just as good at any nesting depth. I mean it's pretty awful at all depths, but the act of nesting doesn't make it any more awful.
I had something like
rules:
- name: Only allow X
- condition: ip=123
I hate yaml.
JSON support is often in a language’s standard library, or close, and its ambiguity around what numbers you can actually use can be seen as a worse-is-better approach.
(There are similar tradeoffs with not specifying a max length of a string. It’s awkward, nit-picky, and error-prone to pick a maximum, but there are practical reasons you don’t want to allow unlimited lengths in a UI.)
I'd describe it as JSON-like with much more flexible syntax and the ability to reference itself.
TOML fills that role wonderfully. And it isn’t full of complexity that turns Yaml into a mess.
I’m curious what parts of TOML do people find confusing? It’s such a tiny markup language.
That's not something I come across a lot - all my software (that I use) has a visual interface, and so raw editing of text/config files is not something we do.
So context really comes into play here. I can see how this would be very useful for some programs, and near useless for others. And more specifically useful for those who consume user-edited configuration files.
There's a whole spectrum of config formats including JSON, JSON5, strict YAML, YAML. It's quite messy.
It is made to be beautiful and easy for a human to read, but the trade-off are a lot of implicit rules you need to be aware of.
Which is 14 years old.
EDIT: this sounded adversarial which was not my intention. Plenty of libraries do not support 1.2 which sucks, I just meant that it's something solved in theory a long time ago.
> The most tragic aspect of this bug, however, is that it is intended behavior according to the YAML 1.2 specification. The real fix requires explicitly disregarding the spec - which is why most YAML parsers have it.
From https://yaml.org/spec/1.2.2/ext/changes/
> Changes in version 1.2 (revision 1.2.0) (2009-07-21)
> [...]
> Only true and false strings are parsed as booleans (including True and TRUE); y, yes, on, and their negative counterparts are parsed as strings. Underlines _ cannot be used within numerical values.
# config.toml
[[podcasts]]
title="Podcast 1"
export_label="Label 1"
[[podcasts]]
title="Podcast 2"
export_label="Label 2"
Of course, my use case is pretty simple and these are simple string values, but I found it pretty nice.
There was also an earlier entry where it was "Tom's Own Markup Language" (2013-02-24):
Option-style algebraic types are another, and more or less strictly superior if working in a language with first class support for such, because they force you to explicitly handle the null/None/Empty/whatever case if you’re doing something with it that could fail (basically anything besides assignment or passing it along).
You can add advanced semantics on load, the file does nothing by itself after all.
Personally I'm happy it doesn't have explicit null and somewhat surprised it supports NaN (which I thought it didn't)
To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound.
I disagree. There is a big difference between configuration language formats and programming languages.
> To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound.
I would agree, but in that case you now often need a secondary key to "turn something off" which is often not done.
You're right! But in my opinion the difference is that a configuration file should have a default state that could be written to disk and loaded with no change in behaviour.
eg you can set `feature = "on"` or `feature = "off"` and if you don't set either it's the same as writing `feature = "off"`.
Having a secret third thing of `feature = null` should be outlawed and I'm glad TOML doesn't encourage it
It's also particularly odd in lists where I have seen `["foo", "bar", {}]` show up in the real world with `{}` being used as a replacement value for null.
In [1]: example = {"foo": None}
In [2]: print(example["foo"])
None
In [3]: print(example["bar"])
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[3], line 1
----> 1 print(example["bar"])
KeyError: 'bar'
In [4]: print(example.get("bar", "sensible default"))
sensible default
I acknowledge people have done silly things regarding missing keys already - but I think they should be discouraged from continuing to do so, not enabled.
In Rust it's possible to deserialize to an Option.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
In general: I agree. But configuration formats (including TOML) also need to map well to programming languages, so you can't seem them as entirely separate. One argument against Null is that it doesn't elegantly map to a data structure in all programming languages.
TOML has plenty of constructs that don't map well to many languages. Plenty of language implementations do not know how to work with the datetime type, some TOML implementations barf on objects in lists. In some TOML implementations you also end up with nulls showing up when certain empty table constructs are used.
Right, and IMO datetime was a mistake to include in TOML.
- unset: ignore values of this property
- initial: use the default value
These would be useful e.g. with overrides.
I agree that nulls in a config files are often not a good idea because it is not clear how they are going to be interpreted.
PS: Css also has inherit that could mean "use the local default" while initial could mean use system default.
They are about as useful in a configuration language, where the cascading logic also applies.
If you are referring to Tony Hoare, the “billion dollar mistake” was not the existence of nulls per se, but that all references were nullable, with no support in the type system to distinguish between nullable and non-nullable references.
https://github.com/toml-lang/toml/issues/921
https://github.com/toml-lang/toml/issues/803
https://github.com/toml-lang/toml/issues/802
https://github.com/toml-lang/toml/issues/146
https://github.com/toml-lang/toml/issues/30
Obviously there is a bit of a demand, and it certainly has valid use cases, but also has downsides.
And that ignores the issue of nulls in arrays which are sometimes needed. For instance I have been exposed to parameterized SQL queries in TOML where you need to use the empty object to represent NULL, a particularly absurd incarnation of this.
TOML > YAML > JSON > SQLite
I'm lazy, so I wish I could just manage them all in Sublime Text.
TOML is great for simple config files that don't need anything beyond key/value or arrays.
YAML is great for config files with more complicated data structure requirements.
JSON is great for self-describing serialized interchange.
SQLite is in a whole other class of technology. I suppose you can store config information in a table if you're already using SQLite in your app. If you're exposing it the way you would with a normal config file, the non-text format introduces friction.
> TOML is a bad file format. It looks good at first glance, and for really really trivial things it is probably good. But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read.
> I personally abandoned TOML and as such, I'm not planning on adding any new features
> If you're still reading, I'd like to put forth CSON as a format of choice.
https://github.com/avakar/pytoml/issues/15#issuecomment-2177...
You mean the author of this specific python implementation, or did Tom Preston-Werner ever disavow it?
It sucks for describing data, its structure is too primitive, has wrong primitives, doesn't allow extension and has no mitigations for that.
Essentially, it's used because it's simple to parse and there are existing parsers. But the problems it creates are very complicated.
So, here's one example of a popular use-case: pyproject.toml. I mean, PyPA aren't the brightest minds around, so, they've created this joke. The stated goal of pyproject.toml was unification of dependency / requirements specification. But, what they ended up doing is this:
[tool.joke]
tool.joke.requires=<requirements>
[tool.clown_fiesta]
tool.clown_fiesta.requires=<requirements>
And so, PyPA has created an piece of junk software that today has to be used by hundreds of thousands. Because we live in the world of worse is better.
1. Build requirements - requirements needed to build your package from an sdist
2. Runtime requirements - requirements your library needs at runtime
3. Extra requirements - optional runtime requirements
For example my library pyspnego https://github.com/jborean93/pyspnego/blob/main/pyproject.to... has all 3:
1. Cython for Win32 needed to build the sdist and setuptools as the general build system - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f...
2. cryptography as a runtime dependency - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f...
3. optional extras kerberos and yaml - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f...
Granted the toml format and what is used in pyproject.toml has its warts but I'm curious what your joke and clown_fiesta examples are actually from as from where I am standing each section currently serve different purposes.
It's common that your build requirements and your runtime requirements have a big overlap that has to iterate together. Here's a realistic example: NumPy comes with various build helpers that you need at build time, if you are making a native module. So, you need to depend on a particular version of NumPy both during the build and at runtime.
In other words, you would want to be able to reference the same package + version in two places.
You could say that this is a problem with "specific" package, where the developers didn't care to separate two kinds of requirements... and in some sense you are right... but: there is a value to having build helpers and the rest of the runtime code in the same package. It's simpler. It prevents those using your package from accidentally messing up.
Finally, it's NumPy we are talking about, it's not a "specific" package... it's like 50% of the entire Python ecosystem.
---
Now, to what I mean in my example.
This is taken verbatim from "real Python" site for the lack of a better example:
# pyproject.toml
[tool.poetry]
name = "rp-poetry"
version = "0.1.0"
description = ""
authors = ["Philipp <philipp@realpython.com>"]
[tool.poetry.dependencies]
python = "^3.9"
[tool.poetry.dev-dependencies]
pytest = "^5.2"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
[project]
dependencies = [blah]
[project.optional-dependencies]
groupname = [blah]
> PyPA aren't the brightest minds around, so, they've created this joke
You have some interesting and valid criticisms of the format. Why not just say them without all the harsh insulting language? Is this how you'd want someone to speak about you and/or your work?
Also, I select what to say to say precisely what I want. Junk is something that is likely of poor quality, old and broken and therefore discarded, useless. This perfectly describes what TOML, JSON, YAML and the rest in the sea of junk formats are.
PyPA are awful at doing their only job: defining rules for Python packaging. In the post above I gave one illustration of the poor quality of their work. Something that could've been easily prevented had they applied themselves to critically assessing the outcomes of their work. And, if need be, I can find a dozen of examples of the same, and probably more. And they don't work in isolation, even in Python world the problem of packaging Python code is not unique to them. They can be compared oranges-to-oranges with other similar efforts. And, yes, they aren't the brightest by a wide margin. And they should be told about it, because they self-selected to handle the problem plenty of people who didn't nominate them for that role depend on in their daily lives. It's a moral duty of anyone who understands how bad they are to repeatedly remind them about it.
You don't need to be an asshole to get your message across.
Regardless of whether I am an asshole or not, the impact of my actions is negligible compared to the damage PyPA has done and keeps doing to the very thing they are meant to support. They should be called what they are, without hesitation and euphemisms. This is the right thing to do.
> They should be called what they are, without hesitation and euphemisms.
You disagree with decisions made by the people at PyPA. I don't think you presented any reason to believe that these people were dumb? Or that they had a low IQ? Or anything else to that extent?
You disagree with decisions, and then concluded that this _must_ make them dumb. Is it possible that they are simply making different trade-offs from you? Or that there's some (possibly mutual) misunderstanding of the problem? As another commenter has pointed out, some of your claims about the state of PyPA tooling were spurious, yet nobody would say that "you're not the brightest" because of it!
You seem like a passionate person, and I genuinely hope that you come to the conclusion that empathy (especially in technical contexts) is a better route than hurling insults. I hope that you can direct your passion to making the world a better place, rather than putting people down. The world could certainly use the help.