Hacker News Reader

Comment by BiteCode_dev | Hacker News Reader

Wisdom for ansible would be to ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters.

Instead, we get a terrible dsl baked in a bad markup language that is hard to write, painful to debug and impossible to maintain.

That you have to come up with concepts like an inventory, fact and magic variables should have been your first clue that they were reinventing badly what any language ecosystems provides for free.

And the worst is, ansible is probably one if the best deployment systems at our disposal.

View on HN

That's a hot take. In my experience Ansible is trivial to maintain and, due to the lack of better templating like Salt, ensures most playbooks look relatively similar as it's meant to be a simple, straight forward task runner. Onboarding folks into Ansible is a breeze as everything matches the standard (roles, tasks, group_vars, etc). It's dumb, reliable, and does exactly what it says on the tin.

My biggest complaint with Ansible is the execution aspect. The fact that mitogen hasn't become the standard strategy plugin and instead Ansible still opens an SSH connection per task is nuts.

My experience is that Ansible becomes exponentially more difficult with the difficulty of your workflows, where programmatic tends to be logarithmic. For low complexity tasks, Ansible is simpler and for high complexity tasks it gets more complicated.

Just as a random example, for complex group structures you have to know/remember that although Ansible allows nesting groups, groups are actually a flat structure. If you have prod.webservers and uat.webservers, the 'webservers' group will include members of both.

That's not hard to work around, but it can easily lead to situations where you end up deploying to prod even though you only intended to deploy to UAT.

Variables are kind of a mess too. Assuming the docs are authoritative, there are 22 different places that variables can come from and which one is used will depend on the precedence. At some point, a 'switch' or series of 'if-elif' statements start to sound much simpler.

RE: variables, I'm a heathen and set `hash_behaviour = merge`. This forces nested dict values to be merged instead of overwritten (lists cannot be merged, unlike Salt...). For whatever reason Ansible encourages users to not use this [1], which makes variable usage far more difficult than it should be IMO and leads to situations like you describe where folks have various top level variables for overrides (ssh:, ssh_local:, ssh_global:, etc to combine values).

[1] https://docs.ansible.com/ansible/latest/reference_appendices...

Oh wow, I've been working with ansible for a few years now, how have I not seen this before? I've even searched for it, but I didn't know what it was called, so...

Do you know if this behavior can be enabled per-role, or even better, per-task?

For context, I’ve written probably a million lines of Ansible and even maintain my own fork to add features like loopable blocks. I love it enough to curse its name.

Large Ansible code bases are painful to maintain. Roles don’t encapsulate well enough and the setting that makes their variables private is unusable if you need to “exports”/“outputs” and I guarantee you do, flow control isn’t powerful enough and you either supplement by dropping down to script/shell or by wring whole programs in Jinja, variables are so spooky action at a distance that you need an external tool to graph where they come from, vault variables aren’t greppable so good luck there, you can’t write a type checker because it’s all strings and any large codebase will dynamically generate variable names due to lack of expressiveness, you will desperately want a way to tell Ansible to eagerly evaluate some variables instead of keeping them as strings and find out how ugly the workaround is without your own patches, you eventually desperately wish you had the ability to evaluate filters and lookups on the target and might eventually, like me, write a plug-in for that. Your options for complex data transformations is writing 50 line filter pipelines with item.0 item.1 or dropping to python and just writing your own filter, you will also eventually desperately wish you could define new modules via “subroutines” by chaining a few trivial module invocations together and then find out that you must drop to an action plugins which expose all the underbelly of Ansible. You will pull your hair out getting Jinja to strip the whitespace from strings to get Ansible to recognize the results as an array since it’s all strings, you will find out that Ansible has its own string type that doesn’t play nice with str and that is somehow exposed to you in bog standard Ansible.

If Ansible was a Python library I would scream for joy at how much unnecessary complexity and necessary bullshit trivia I could drop from my brain.

> and instead Ansible still opens an SSH connection per task is nuts

Mitogen is not worth the bugs compared to using SSH ControlPersist.

Control persist still creates multiple SSH sessions though?

I have a SSH hook that sends a chat message whenever someone SSHs into a server, and Ansible without mitogen floods these, even with ControlPersist.

If you have it working, can you share your ansible.cfg?

Well yeah, because ControlPersist is just “make session creation fast” by multiplexing them over a single SSH tunnel. So you still have “ssh internal session creation” overhead but the tcp and ssh handshake is the slow part. The speed gap between pipelining with ControlPersist and Mitogen is small enough that it’s, to me, not worth the bugs and limitations of Mitogen.

I do get the annoyance of your script though.

Yeah after using CloudFormation, Terraform, Pulumi, AWS CDK and k8s + YAML, I'm convinced using a real programing language idiomatically is the correct way of doing IaC.

Many languages totally support declarative idioms out of the box (like you mention), there's no need to reinvent anything here.

In particular I'm a big fan of AWS CDK, although it only works for AWS. (Yes, I know about CDKTF, but it's buggy alpha software).

> ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters.

Fully wholeheartedly agree but, as I stated in previous ocasions in this site, after the nightmares I've seen people, in many different places, doing with a high level tool like Ansible (and also Terraform), I apply heuristics about the probable outcome of every DevOps using a proper Turing-complete language as a general trend and I just want to leave CS and buy a hut in the middle of nowhere. Perhaps something like that must come after a shift in the mentality of the whole business where people in CS in general, assumes one needs to be a good coder, no matter if you do DevOps of software and therefore, learn about software design patterns, reusability and best coding practices.

The issue is that Ansible right now today is just Python but if you tied chopsticks to your fingers when typing. It’s not declarative, it’s sequentially evaluated with all the flow control and concurrency bugs to go with it.

You have to already be a good coder to write Ansible but you have all your limbs tied behind your back because you get none of the features of Python to help you reign in complexity and validate correctness.

The killer feature of Ansible vs. Python code is, noting I have a ton of both here for infrastructure related tasks (as well as shell, why not), when you have it running for three years on some box somewhere and something breaks, you go look at it for the first time in three years and it makes sense. My Python infrastructure tools OTOH, while they are great, if something goes wrong with them I have to spend half a day reading the sourcecode and remembering what their particular architcture / conventions are.

it's like the dumbness of Ansible is really the main feature it has.

I agree with you but I think that dumb + size => complicated which is where I live. When you have a playbook that is thousands of tasks long the dumbness leads to exactly the problem you have with Python.

I agree with that! my ansibles are for really straight stuff deployed to a few boxes. Where I work, oddly enough at Red Hat :), the very large playbooks are definitely difficult to work with. It's hard to say though if a custom Python app of similar size is really any better though, unless your developers are Michelangelo.

I've been very slowly rebuilding all our Ansible roles in PyInfra for that exact reason.

Also been going down this path. pyinfra is overlooked I think.

It's not perfect, I still feel like I'm dropping down to just hitting `server.shell` a bunch , but at least as a programmer first, it feels more comfortable vs the mountains of yaml (and doc searching) for Ansible.

On the whole it's a less frustrating experience. I still have a vague feeling that in my case - where each server(s) is generally customised per client and I've never felt comfortable just pointing ansible (or pyinfra) and a configured server and re-running it - that perhaps I would really just be better off with a bunch of shell scripts. I don't generally tear-down and throw-up multiple servers a day, week or even month.

rrconf sort of fills this but works off a bit more infrastructure than I want to manage with its different git repos per action.

[1] https://github.com/rrconf/rrconf

I understand where you are coming from. It's interesting to see the Puppet/Chef-debate played out over and over again.

You generally want an overarching single source of truth for IT systems. One that defined which parts a system contains and how it is set up. This starts to get important when you want maintainable testing environments, and more so as the number of system integration points grow, but it generally makes tracking changes and having a controlled deployment process so much easier.

So it is natural to start with a declaration of how small pieces fit together. This is Puppet/Ansible where the language is just a list of objects, where each object is implemented in "real" programming language. But the list soon needs logic, if not for other reasons than because testing environments have subtly different requirements, and the list grows into a semi-language.

That's not enough and people new to the system soon feels constrained, why not ditch that domain specific thing and implement the whole thing in the "real" language? This is Chef/Salt. But now all you have is another object, which needs an overarching list of objects, and soon we have another simple description for that.

And then we repeat this all over again.

(There are specific issues for some parts of this, which is not part of the above story. Ansible didn't have to choose yaml for their domain specific language, they should have seen that coming. But that's not for this story. We instead get to see this repeated again in the container space with templates and Helm charts. Also Jenkins pipelines which slowly mutates to Groovy and back.)

Pulumi could have filled this nicely but only supports a few cloud providers.

https://github.com/Fizzadar/pyinfra

Wisdom for ansible would be to ditch ansible and any similar tool.

Reply on news.ycombinator.com