Hacker News Reader

Comment by mnutt | Hacker News Reader

It’s good to know which command line tools are idempotent, but I think once you start running automated scripts repeatedly you’re better off with something like Chef where you declaratively define the end state and let chef figure out how to converge upon that state. Taking the “create a directory” example, you would say “I want to end up in a state where this directory is created and has these permissions” and chef will determine if the directory already exists.

There are of some downsides to that approach which stem from the fact that you can’t possibly declare every possible thing. For instance, if you were to run the above declaration and then later remove the declaration, chef wouldn’t know to delete the directory; at that point chef wouldn’t know anything about the directory at all. So ideally you can build infrastructure from scratch each time (Docker, etc) rather than having to converge an existing machine.

View on HN

Absolutely this. When the intent is to get the system into a desired state, reach for Puppet/Salt/Ansible.

If you are doing sysop^Wdevops work it is the single most important thing you can learn, once you have a good grasp of your shell and your editor.

The difference between a giant git archive of shell scripts that various people have modified over the years and state changes described in a configuation management language is the difference between fixing things in the small and reasoning about integrated systems. It's something that needs to be experienced to be appreciated.

Reasoning about system state as an integrated whole is just as relevant when you are shipping applications as containers, if not more so. It's not uncommon to start using something like Kubernetes without first being able to describe global state and the result is just as messy as before, if not worse. Something like Helm is impossible to understand unless you have complete control over your configuration.

Is it not a bit of a large jump from "shell script" to "chef"? There is some inbetween.

I understand a shell script is not suitable for orchestrating a datacenter, but sometimes it is suitable for installing a couple of things.

Ansible is good for small or large installs.

100% Agree. I originally got into Puppet after spending several months with a client that wanted me to write bash scripts to push out a major middleware deployment, switchover, and uninstall to 10,000+ stores. There was no standard for the hardware in the stores, and their configuration was all over the place because 100's of teams also used (non-idempotent) shell scripts to push changes.

I learned a lot about catching all the different bad things that can happen, adding prerequisite checks, rolling back changes automatically, etc. In the end, a tool like Puppet/Chef/Ansible would have taken a fraction of the time to develop scripts for.

Containers have definitely lessened the need for mutable configuration management, but they are still useful in the 90% of environments that are still working in a past decade.

Chef does have an action on their primitives, so it is possible in the directory example to set the action to :delete instead of only implying :create

You’re right, it’s totally functional, the (minor) challenge I see with it is that it breaks the illusion that you’re describing the state of the system, and feels more like describing a set of changes you want to make.

Chef isn't magic, it just uses a lot of cookbooks which are just running things to check what the state is.

Usually those cookbooks will want to have things in a particular way, which might not be the way you want.

Chef/Ansible/Puppet all have the problem of having many layers of overhead for the same thing.

This makes them slow, hard to debug, hard to read and even write (in my opinion).

Sure, bash is hell, but the go from bash to something like Rust, instead of Chef/Ansible.

Sure, again, this might sound rather unconventional, but you get safety, speed and convenience of a full fledged programming language, single binary output, etc. And you need to test cookbooks/playbooks too, so why spend enormous effort on cobbling together scripts and high-level abstractions instead of writing just what you need with the assumptions you really have - instead of what a general cookbook/playbook has to have (which is a vast difference).

Writing a shell script in rust would be obnoxiously difficult for no real benefit.

Yeah, and tools like Chef are 99% blocked on disk operations anyways, so I have no idea why GP thinks Rust would help

Rust would help due to the type system and lack of library dependencies (it does static linking AFAIK), I suppose.

I feel like my comment went in one eyeball and out the other.

Chef and friends are blocked on disk I/O. How does a typesystem and/or thinner abstraction layer to disk I/O speed up the underlying expensive operation: blocking disk I/O?

I - on the other hand, feel like your comment somehow forgot that the op said “safety, speed and convenience”, so you attacked the least important point made in the first place.

It’s much easier to see the failure conditions in a Rust program rather than in bash. Also rust seems like easier to maintain, too.

How does having a borrow checker and snazzy memory safety benefit you when nearly all the operations you're performing are disk I/O?

How is a 100 lines of rust easier to maintain than 10 lines of shell script?

It's exactly what the other commnet said: "Writing a shell script in rust would be obnoxiously difficult for no real benefit."

Well, for once Rust's type system can enforce the usage of objects (e.g. temporary file must be used or deleted).

>> How is a 100 lines of rust easier to maintain than 10 lines of shell script?

In 10 lines of _proper_ bash you won't be able to even check if the arguments supplied to the script even exist, let alone parse something more than simply subscripting argv[].

Rust would be a step up from writing bash scripts of yore—imagine, being able to tell the difference between a variable being empty vs unset—but really any non-stringly-typed language will get you that, and the GC-based ones are unlikely to have memory errors and are generally fast enough for this purpose.

I don’t think there would be anything wrong with a rust implementation of infrastructure management, but I also don’t think it’s the silver bullet to solve what plagues the space.

CM systems spend a lot of time on bootstrapping, creating their workspace, downloading the scripts, parsing them, etc.

Then they simply execute other programs to then parse their output. And/or fiddle with files (parse them, alter them, write them).

Sure fundamentally the syscalls and apt/dnf/yum will be the slow parts, but I found that development of CM scripts/plays/recipes are usually bottlenecked on the turnaround time of the CM system's own workflow. And execution time is a significant part. (The bootstrap, the transfer of whatever files, and so on.)

Rust would help with writing things that are relatively well error handled at compile time, and gives you single binaries.

Why do we care about speed here? Surely reliability and readability are paramount features...

Development speed is important. You can use the most reliable thing in the world if it takes a day to test/build/deploy.

And simple imperative script is a lot more readable than a custom DSL with who knows what ruby hooks.

Currently the CM system that makes sense on the long run is a git repo for Terraform. (Because everything else runs in containers anyway, and to set up the immutable images you don't need "state management". And where you need it you need active state management such as k8s and its specific operators.)

It's about functionality and maintainability, not performance. If you have complicated scripts then they start to resemble actual programs, at which point you might be better off using a real programming language to code.

I wouldn't pick Rust for this but there are lots of examples of using Node/JS, or C# or Python. It's a much nicer environment and benefits from full IDE support.

For a good example, look at what Pulumi is doing for modern infrastructure: https://www.pulumi.com/

They didn't claim Chef was magic, and yeah it's just a bunch of cookbooks, but that doesn't change their point that it's declarative and useful for describing an end-state. Chef is hard and slow if you rely solely on others' work without understanding what it's doing, which is functionally equivalent to magic. But if you know how to tune things, keep your cookbooks down to only required ones, and a few other tricks you'll learn from usage, it can be fast as anything. I've seen one particular setup meant to keep a server up to date and configured to have HAProxy serve over 100+ endpoints run in under a minute. It's possible, just a different approach.

I loved Chef after we got familiar with it. It was nice, cute, had solutions for a lot of problems. (foodcritic, kitchen, rubocop, etc.)

But it simply is a solution in search of a problem in today's container orchestrator/platform world.

HAproxy is amazing, but a k8s Ingress service [0] has a nice API, so I don't have to run Chef to add a new vhost. Yet I can persist the config in a YAML. (Or json, or whatever.)

[0] which can of course be backed by HAproxy down below, but traefik has dynamic config; though haproxy2 will do that too

https://www.haproxy.com/blog/haproxy-2-0-and-beyond/

Rust doesn't get you anything resembling what you want here.

Chef/Ansible/Salt could be described as rudimentary type systems for operating system states, that enable you to convert between different types/states.

It's conceivable you could build such a system in Rust, but Rust itself has no primitives that would make it easy. It certainly isn't a good starting point when you want to set up a database server.

All Config Management systems execute other programs, parse their output, check/parse files underneath their fancy skin.

I've used both Chef and Ansible. Their maintenance cost is pretty high unfortunately, and they are not that flexible, nor resilient to worth it.

No wonder the container/k8s boom is so big. Immutable infrastructure (docker images) with really powerful operational features is what can justify high maintenance efforts. Whereas fancy but brittle (due to the inherent problem of state discovery) CM systems are at best only useful for the initial platform setup.

Rust is just useful because you can quickly and relatively safely produce single binary programs. (Many people use go for this, but it's easier to skip error handling in go which result in runtime problems, which is very inconvenient in a provisioning/setup system.)

Nix/guix might be a better approach. Ansible and chef describe part of the desired state of a system and require a very large definition to cover the whole system. nix aims to track the whole system state and allow you to supply changes which again combine to another whole system state

For tracking changes:http://sysconfcollect.sourceforge.net/

Yes. Though I'd probably want to simply use something "trusted", either Ubuntu LTS or CentOS/RHEL, keep the installed packages to a minimum, use a local repository mirror proxy, track package changes there, etc.

And the image build should be just a simple imperative install these packages, use this config, run this command on invocation.

Nix is rather amazing with its powerful CLI stuff it provides (S3 compatible dependency store, fetching via SSH, closures, etc).

My only problem with NixOS is that it's very much like Gentoo. It has infinite composability built-in, but it means you have to rebuild everything. In Debian/Ubuntu land you usually can simply enable/disable install/uninstall specific feature related packages. (For example postfix and postfix-mysql packages.)

Yeah, I find out-of-the-box cookbooks about as helpful as out-of-the-box docker images: useful as a starting place but to leverage chef you really need to write your own. Chef resources/providers are nicely composable and the primitives are pretty easy to rewrite if you want, my main point is that chef gets you to think in terms of convergence instead of brittle imperative changes.

You could rewrite your bash scripts in rust but while it would probably be safer/more correct than bash, the whole paradigm of writing imperative scripts to manipulate existing machines is flawed, whichever language you write it in. And executor performance is almost never the bottleneck in my experience.

My comment tried to argue that Chef's convergence is nothing more than a well hidden imperative thing.

I agree that manipulating existing machines is folly, but that's why people use containers, and immutable images. And then you can prepare the images anyway you like, but since they are very deterministic (modulo the external repositories/packages/curl-bash-piped-scripts), you usually use something simple (eg bash), because there's no need for that state management.

And actual state management should be left for the platform (k8s, swarm, or some cluster manager).

I found the chef workflow slow, not just the executor itself. (Uploading cookbooks, fiddling with dependencies, testing them, debugging them, etc.)

Ansible was even worse in my experience. Much slower execution, harder to debug, opaque python blobs, extremely confusing and fragile "YAML programming" coupled with hostile variables files. (All the things that are tenfold more intuitive in Chef.)

Fair enough, most of my experience is with a sort of homegrown chef-solo. It has served us pretty well for almost ten years but these days we're in the process of moving to kubernetes + immutable infrastructure.

Many of the comments in this thread seem to be sucked into a false dichotomy. There are better options than _either_ Rust or Bash for writing complicated shell scripts. Perl/Python are more suited to the task than either. The challenge of deciding what degree of complexity warrants the investment of more complicated language is a design decision left to the developer to figure out. Any thought given to designing more resilient software is a good thing, but I can't help thinking that expecting any real degree of safety and security from bash is expecting too much. I don't work in devOps mind you, but whenever I have to write scripts that do anything more dangerous than creating files I leave bash on the shelf.

My experience is that going from bash to something without a compiler has an overhead that rarely pays off. mypy or TS can probably do the job well, but at that point Rust gives you a better deployment story.

Don't get me wrong, I'm absolutely an advocate for statically typed languages. Most of the work I'm doing with bash scripts involves command invocation, piping, and filesystem I/O. Which are things that I've never seen compiled, statically typed languages do particularly better than Perl/Python. I've never done this kind of programming in Rust, but my experience with Rust suggests to me that it's not any easier or more robust to do this work in Rust than it is in Java or C++. Mind you, you might have a completely different experience depending on the task at hand.

Reply on news.ycombinator.com