Hacker News Reader

> It's the same as using Puppet or Ansible to modify a running VM, to "fix" the VM in place. This is a laborious and error-prone process which requires a great deal of investment in the Configuration Management - essentially writing programs just to tell the system where each file should be.

This doesn't track with my experience. Tools like terraform can be used in such a way that, instead of running an apply to update a resource in place, a change to the machine configuration would result in a deletion/new instance being spun up. Some operations are mutations, like updating the name of a security group for instance. But updating a host's configuration should be done by updating user data and creating a new instance, though it's true terraform offers the ability to perform the update either way.

> If you design a system architecture today, you cannot build the cloud infrastructure, get it to a working state, and then make an immutable snapshot of how the system works, and restore it later if the system changes or becomes unstable

You can. Here's what you'd need:

- a snapshot of the code (eg: git commit SHA)

- a snapshot of the DB schema that correlates to the code (eg: database using linear migrations checked into the code)

- a snapshot of the DB state when upgrades or downgrades take place

- immutable (remove & replace) infrastructure checked into the same git SHA

This requires writing infrastructure, DB schema, and some features a certain way. The problem isn't a technical limitation, it's that you'd have to know a good amount in at least three different domains to set this up, much less run and operate this system. It raises the bar for knowledge well beyond treating each of these systems as isolated systems. That's why most teams don't do it. It's far easier and more realistic for most teams to say" let's fail forward".

Immutable infra is usually provided by some stateful tool that maps state changes to API calls

That's not true. Immutable infrastructure state should not be dependent on previous state because there is no state. It's remove and replace only.

If there's a piece of infrastructure that cannot be regarded as immutable and this is mimicking immutability behind an API (like a CDN) then it's not immutable. Understanding this and segregating that infrastructure away from totally immutable components (like servers, functions, load balancers, etc) is key.

The CDN mutability probably can be flushed out by some processes like etags but the most important problems are solved, just hard.

What tools are you using to accomplish this? Can’t be terraform

As I said, you need to know how that infrastructure is built. Terraform is just fine. If you call to spin up a VM with a disk that's been imaged to a standard, that is immutable. Functions can also be made immutable by programming in a way that takes into account potential shared state.

Terraform works by tracking mutable state though which was my point

Ah, state is relative in this context. A VM lacks state when you determine the starting state. A function lacks state depending on how you code it and the contracts you have. A CDN is always stateful. Sometimes the Terraform state overlaps with an entities internal state but most of the time not. "Immutable infrastructure" state refers to internal state, not Terraform state.

Ah I see what you mean and I agree it’s possible to achieve immutable infrastructure in that sense.

Oh look, another post proclaiming mutability as the source of all nasty bugs and demanding an immutable solution. You know, you'd think that if immutability really was a golden elixir of correct coding, that Scala would have taken over the business world by now instead of shriveling on the vine.

Oh look another comment pointing out that as long as companies can throw bodies at problems, creating drudgery in the name of profitability, there's no problem to be seen. The fact that good tools exist but are not mainstream is proof positive surely.

I would say that's not being able to rollback to a last-known-good snapshot-of-the-world a feature, not a bug.

A major premise of cloud computing is that everything is disposable.

Consequently, this is meant to encourage consumers to express as much of their system's configuration with code and automation as possible.

For example: the change to the S3 bucket policy that you mentioned would, theoretically, be done by changing the policy doc in Terraform and reapplying the Terraform configuration.

Modification to those policies would, theoretically, only be allowed by anyone with the "AdministratorAccess" role, which would only be provided to the IAM user running Terraform (via temporary STS tokens) or admins through some sort of a "oh-shit" process.

> A major premise of cloud computing is that everything is disposable.

That norm is itself a way of coping with the hopeless state of state management, a lesson learned from running imperative/convergent configuration management systems on VMs running mutable operating systems a generation before 'cloud native' was a thing.

That whole 'best practice' is a workaround for shaky foundations.

It seems to me every time someone promises to make it easy to do something complex, they've just created a more efficient way to sweep problems under the carpet.

I'm a little confused by this comment. It feels like you are complaining about the problems that are solved by configuration management and infrastructure as code, yet you seem to reject it as too complicated.

All the problems you raise are solved by infrastructure as code.

No they are not.

Infrastructure as Code is literally is nothing more than keeping a config file in Git and running a script to apply the config to a REST API (or whatever). Terraform stands in as an overcomplicated script. Infrastructure as Code does not determine if what is being managed is immutable or not.

Configuration Management is a program designed to attempt to change mutable state so that it once again resembles what you want.

Immutable Infrastructure is immutable. You cannot change the state. Hence configuration management is pointless and unnecessary.