Clutch, the Open-Source Platform for Infrastructure Tooling | Hacker News Reader

Clutch, the Open-Source Platform for Infrastructure Tooling

eng.lyft.com

116 pointsjbredeche28 comments

With this and Backstage it looks like there is a real need for a single pane of glass across an organization's infrastructure. From both a GUI perspective and an API perspective.

It looks like it would aid in more self-service and keep users away from cloud console's.

The workflow engine looks the most powerful since it would allow for a way to avoid fat fingering common manual changes.

Sad that it doesn't seem to offer any deployment visibility options. I would love to connect it to our CI/CD system to see what changes are currently being rolled out.

Yeah definitely. A major pain point I've experienced recently is trying to figure out how to stand up various pieces of infrastructure needed to deploy my app at my company.

There's the k8s manifest, Cloudbuild config, various operators for databases, a CloudFormation repo for AWS resources, another one for Datadog config etc. It all feels very disjointed.

Does Helm help with any of this? I apologize if not, I just hear it used when these topics come up.

https://helm.sh/

You can deploy aws services (via cloudformation yaml) as a K8s manifest

You need none of that, and a single thick server on Debian will cover all your needs.

Monolithic architectures are more performant generally. That's the reason why we fly in giant aluminium tubes rather than by dangling off a flock of bird-sized drones.

Ah, efficiency vs. aesthetics is truly a perennial battle.

Checkout https://www.shuttleops.io/

It helps abstracting deployment away.

Disclaimer: I work @ ShuttleOps

I disagree. Working in IT infrastructure / DevOps for years theres nothing worse than maintaining aging infrastructure that was too tighly coupled or "too unified".

One job, one tool is still valid today.

> Sad that it doesn't seem to offer any deployment visibility options. I would love to connect it to our CI/CD system to see what changes are currently being rolled out.

Deployment visibility is what my startup - https://sleuth.io - is trying to focus on, with light workflow sprinkled in. Other options I've found are usually tracking as a side feature of a tool like Sentry, Rollbar, or GitHub, or a completely custom app like what Slack built in https://slack.engineering/deploys-at-slack-cd0d28c61701

Do the problem spaces of Clutch and Terraform overlap ?

I'm not entirely sure what problem does Clutch solve that is not solvable using Terraform

Looks like it has more "self service" capabilities. But not sure. But i think most of the problems are better solved by running an IaC tool like terraform in CI with a pull-request workflow.

terraform forces you to write custom providers instead of letting you glue tools together. cloudformation has the same issue with custom resources, though the abstraction is better since you can write custom resources in whatever language you want.

neither of them solve for CI or pull request workflows themselves, so people try to bridge the gap with Jenkins and now the frankenstein monster comes to life -- now you've got golang, groovy, yaml, none of which your product devs generally have any familiarity with, let alone the idiosyncrasies of terraform state management, cloudformation stacks stuck in ROLLBACK_COMPLETE, yaml whitespace sensitivity, Jenkins scripting sandbox woes, troubleshooting from logs without being able to step through a debugger in an IDE, errors that only happen in eu-west-3, etc, etc...

Clutch looks great!

I have to say, this looks potentially awesome.

I previously worked at Stripe, where we had something vaguely similar; a UI for managing deploys, infra changes, etc. All a thin wrapper around a CLI we could also use, plus some niceties like embedded monitoring. I'm going to really miss that tool the next time I build my own web service.

It really helped product engineers make changes to production with speed, confidence, and safety.

Before, doing so was a combination of yaml config changes, a smattering of in-house and third-party CLI's, and sometimes complex combinations thereof. You could do it, and we did all the time, but it was less discoverable, slower, and more prone to error.

I'll be curious to see if Clutch is something that proves easy for other orgs to adopt to their needs.

Is this something a small team could use? It seems to require significant customization.

Trying it out today, and the existing features work. It could use some polish, And a lot more modules. But so far my experience here is similar to my experience with envoy. The maintainers are engaged, helpful and very nice. Questions are answered and easy issues fixed as we speak.

The real promising part is how this is made to be extended. It seems like a self-service ops API built around protobufs which is a good starting point.

> There is a popular continuous delivery platform that addresses many of the same overarching issues as Clutch (e.g., lowering MTTR, user-friendly UI). However, it requires significant investment in running many microservices and migrating applications to a structure different from our own.

Any idea which one?

Edit: I assume Backastage as it’s the first listed on the linked comparison page.

I assume they are referring to Spinnaker which they refer to as "architecturally complex, comprised of more than 10 microservices".

There is a comparison page which names a few: https://clutch.sh/docs/about/comparison

I've been trying to do something similar with a less complex approach using incremental automation scripts and creating an operational meta database, then do some light integrations (a Grafana datasource that can use the ops metabase to use information in templated dashboards)

I think this is a new model of job security?

It generally looks like:

- Go to company with major growth

- Tackle a problem they face

- Leverage company resources to build a open source solution

- Sell management on all the benefits of open source

- (Optional) Wait for stock to fully vest

- Create consulting services company for the tool you built

That... sounds good? Like, good for everybody involved?

This plan won't work if management doesn't agree that you're solving a valuable problem, and also won't work if other companies don't view the problem you're solving as valuable.

> Leverage company resources to build a open source solution

This part requires permission from the company in some jurisdictions

Can anyone explain what is infrastructure tooling in laypeople (or layprogrammer)'s term?

If a software company is like an automotive company the infrastructure tooling is like the factory.

Comparison isn’t perfect but it’s usually good enough.

I think a more accurate comparison would be the assembly line of the factory that allows push-button-combination-of-parts-to-build-a-motorcar. Infrastructuree tooling allows push-button-combination-of-application-component-stack-to-deploy-the-functioning-software-application.

In simple terms with no dubious metaphor, infrastructure tooling is the set of tools provided to engineers that allows them to deploy and run their software on the infrastructure that is unique to their company.

Although lots of companies use common tools (let's say, randomly, Kubernetes), companies' infrastructures are usually a pretty unique combination of software and rules. The "infrastructure tooling" abstracts away those software and rules and allows developers to deploy "things" simply using the provided tools.

For example, imagining a company using Kubernetes, an infrastructure tool could be a web UI that allows you to deploy a specific application by simply clicking buttons. That tool would in fact deploy to Kubernetes, yet as the user that detail doesn't matter to you.

You know how you need to setup k8s, logging, DB, and a handful of other services before you can deploy your app. Well this tried to tie together these services and hopefully make it easier.

Reply on news.ycombinator.com