Hacker News Reader

Comment by acdha | Hacker News Reader

That’s a choice, like using Kubernetes or being a micro service fundamentalist. A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data, and before that you had the J2EE types building elaborate multi server systems for what could have been simple apps.

This repeats constantly because the underlying problem is a broken social environment where people aren’t aligned with what their users really need and that’s rewarded.

View on HN

> A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data

So said Gary Bernhardt of WAT fame in 2015 https://x.com/garybernhardt/status/600783770925420546

> Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000.

Which, in turn, have inspired https://yourdatafitsinram.net/

I remember that, too. There were also just a lot of places where it’s like … your data is already in a SQL database, why not get better at reporting queries until you’re running so many of them that you don’t have an easy path to handle them? It was especially tragicomic in places where they were basically hoping not to need to hire analysts with technical skills, which is possibly the most MBA way ever to very expensively “save” money.

I think it is because cloud computing demands far more programming the machine than deploying to compute instances does.

Does it, though, or are you just more familiar with traditional servers? There are a lot more things to manage in a traditional environment and things like tools are more primitive so you’ll have to invest a lot more time getting anywhere near equivalent reliability or security. That’s especially important if you have compliance requirements where you’ll be on the hook for things provided or greatly simplified by the cloud platform.

>> There are a lot more things to manage in a traditional environment

Can you be specific because that does not sound right to me.

I know both pretty well and seems to me there is vastly more things that need to be tuned and managed and configured in cloud.

How about you give some specifics of things that need constant fiddling in the cloud but are free on bare metal?

Well its a good point, it's my feeling at the moment.

I think it would be interesting to do a systematic study of the time required for each.

In a cloud environment, I turn a few knobs and I can command an absurd amount of compute power, have it autoconfigured with all kinds of service discovery and whatnot. Things that would have taken weeks to months to build in a physical datacenter. Some people take that spare time to train as architecture astronauts I guess, and overcomplicate everything with all kinds of fiddly layers and microservices. I get it, it's fun to play with new toys. Doesn't mean the toys are bad, they're just not necessarily ready for production on day 1.

My experience on multiple cloud projects over many years is endless fiddling and problem solving and spaghetti.

That's because it's easy to get started with one of the hundreds of services available in the cloud and before you know it your cloud project is using 50 of them that depend on each other.

Here’s a simple one: if I need to say something accepts inbound 443 from 1.2.3.4, I create an AWS security group rule and will never need to touch that for at least a decade, probably longer. If I’m doing the same thing on premise, the starting cost is learning about something like Cisco hardware which costs a lot more to buy and I am now including in my maintenance work. Over a year, I will spend no time on my AWS rule but I will have to patch the physical switches and deal with primitive configuration tools for a complex networked device. If my organization cares about security, I will have to certify a ton of configuration and practice unrelated to packet filtering whereas I can point auditors at the cloud service’s documentation for everything on their side of the support agreement.

What about running code? If I deploy, say, Python Lambdas behind an API Gateway or CloudFront, they’ll run for years without any need to touch them. I don’t need to patch servers, schedule downtime, provision new servers for high load or turn them off when load is low, care that the EL7 generation servers need to be upgraded to EL8, nobody is thinking about load balancer or HTTPS certificate rotation, etc. What you’re paying for, and giving up customization for, is having someone else handle all of that.

Reply on news.ycombinator.com