You want to see a real abuse of YAML? Take a look at SaltStack's Jinja rendered YAML. I just grabbed a random forumla as an example: https://github.com/saltstack-formulas/openssh-formula/blob/m...
I mean, sure, you can convert one into the other syntax-wise, both form trees, but what do you gain?
It's easy for someone who never {wanted to, learned to} program to accept yet another configuration format (with all options specified up front) and then accidentally learn (bastardized) programming concepts. It'd take much more effort to turn them into a 'real' programmer, learning a language from scratch, practicing that, and only then introducing some tool in the form of a library.
-Define the desired state of your system in a declarative way. This requires a DSL, and some tools such as Ansible and Kubernets use YAML as that DSL.
-The configuration engine (Kubernetes, Ansible, Chef, Salt) will converge the system to the desired state in an idempotent way using the DSL.
-Many systems have mostly the same config and many systems also run in environments that are mostly similar but with some differences. It doesn't follow the DRY principle to create separate but mostly identical desired state configurations for every system/environment, so we need programming primitives and templating tools to accomplish it. It's no less valid than a web app templating HTML for instance.
It's easy to conflate "mixing programming with configuration" with "using programming to solve configuration generation."
Meanwhile, YAML is not a good DSL framework, it's not even a programming language, it's a markup language. Any logic defined in it will be inherently bolted on and likely based on text templating and full of stringly-typed confusion. A classic CM tool that does get this right is Chef, where normal Ruby code is used to build the declarative intended state. You can extremely easily implement any logic you want to create an intended state (even retrieve it from other components) and then emit a desired, declarative resource state (and --why-run will then show what steps will be taken to converge tit). You can do these things right, just don't try to turn a markup language into a turing-complete DSL.
You compare this to templating HTML - but nobody would ever seriously consider declaring such templating logic within HTML itself with some sort of Jinja2 templating engine duct-taped to the side. Instead, we use whatever is in control of serving HTML to render the final markup, with the bulk of the logic implemented in a normal programming language, while a templating engine takes care of rendering variables and loops.
Comparing Ansible's YAML and Kubernetes' YAML is also wrong. A better comparison would be comparing Ansible to Helm, since they both take the templated-YAML approach. Kubernetes by itself doesn't even let you just use YAML files as intent and idempotently apply them (some objects can be created from YAML using apply, but then not updated, and instead have to be recreated, and that's the logic that's pushed onto external tools like Helm, Kubecfg, ...).
And even instead of using a real DSL that's built into a particular tool, there's the second approach that I currently follow and recommend: use whatever programming language you're comfortable with to emit a desired state in whatever interchange format your management tools needs, and let those tools handle the domain-specific task of converging the state.
If you're saying that, you should clarify for anyone not familiar with various config mgmt tools that in order to provide you with ruby, it compiles ruby in 1 pass, then evaluates the result in ruby again after enriching its environment with the results of the first pass.
This creates surprises when code apparently disappears (because it's been evaluated). In addition, to accomplish some of its other goals, chef performs an up-to 18-layer collapsing of various variables to make them available via the node object, which means that depending on which evaluation phase has completed and what else may have enriched the various layers, the state of the node object can be very hard to understand.
I say this as a big fan of chef - it's quite a lovely product. But on the other side, salt and ansible work to provide fewer footguns in the default case where the goal is to provide structured data to a function that will act on it. It allows you to footgun yourself differently, but your excitement about yaml vs. a programming language is understandable, but impractical since putting programming languages into the position of managing the messy reality of running a program on an operating system requires a lot of any programming language - it's where the rubber meets the road and the road is neither smooth nor straight.
And back in the day sysadmins wrote programs like tcp_wrappers, postfix, sendmail, etc. It was definitely common and encouraged if not expected.
The idea that system admins should be able to get away with never professionally stepping up to doing programming is a late-90s/early-00s aberration that was generated by the need to scale up during the dot com boom before good automation practices existed. And it is not a good thing and it stunts intellectual growth. Devops is in some sense back-to-the-future by tearing down an artificial wall that never should have been made in the first place.
Sure, lots of systems people program and love programming, but the job isn't to write write tcp_wrappers, postfix, sendmail etc. it's to do what's needed to make sure that such things run, which is a different kind of programming.
My take is:
The complex admin work will be going to the cloud providers.
The rest (besides some niche stuff) will get so simple a programmer can do it on the side.
Admins that don't learn to program are left behind. Only those who have some cushy full time employment, were they can't be let go, will remain.
While I personally am a Software Engineer + Systems Engineer combination perfect for modern Devops... I do not agree with you one bit.
There will always be a place for non-programming systems administrators -- especially as we get more and more systems over time.
Will those administrators be able to /scale/ to managing 100s or 1000s of system without learning some sort of programming language or configuration management system? Probably not.
However, there are plenty of businesses out there that only have a handful of servers and services which can totally (and more cheaply) be managed manually.
Automation has an up-front cost that doesn't always make sense for the business to spend.
I wish we started analyzing this as the hard problem it is rather than getting angry that people didn't always make the right decision by correctly anticipating how their software would end up being used.
Why couldn't they use a regular programming language to create the markup? I mean this is done with HTML and JS all the time.
jinja is in wide use in the python community.
Sure, you can probably build your own concept of functions on top of Jinja, either by embedding them or building your own paradigm on top of Jinja's template inheritance, but it's a hack either way.
In the case of thread OP, that template is pretty messy, yes. In ansible, I would just create a custom module or action plugin in python to spit out whatever it is I need. I don't know how flexible salt is in this regard, but ansible is quite flexible.
For simple cases, jinja works great if you're just doing a find/replace type operation in yaml.
Please don't give markup a bad name because of YAML, or the clusterfuck linked by GP.
I just don't understand why someone would create such a "clusterfuck linked by my GP"
If you re-invent a programming language in markup, you basically get a new custom programming language that "probably" can do the same as another non-custom programming language, but without the benefit of many people already know them.
Why not integrate something like Lua or Dyon instead of YAML or XML?
And if I'm writing code, it should look like code you'd actually want to read and edit. Not something with hacked-on delimiters {%- all over the place %}.
My company's scripts have lots of "if fail, try two more times, and hope it succeeds" logic as a result. -_-