Hacker News Reader

Comment by nostrebored | Hacker News Reader

This is a strangely limited view. Cloud providers have done the work of building fault-tolerant distributed systems for many of the _primitives_ with large blast radius on failure.

For example, you'd be hard pressed to find a team building AWS services who is not using SQS and S3 extensively.

Everyone is capable of rolling their own version of SQS. Spin up an API, write a message to an in memory queue, read the message. The hard part is making this system immediately interpretable and getting "put a message in, get a message out" while making the complexities opaque to the consumer.

There's nothing about rolling your own version that will make you better able to plan this out -- many of these lessons are things you only pick up at scale. If you want your time to be spent learning these, that's great. I want my time to be spent building features my customers want and robust systems.

View on HN

I see where you’re coming from — no doubt, services like SQS and S3 make it easier to build reliable, distributed systems without reinventing the wheel. But for me, the decision to shift to European cloud providers wasn’t about wanting to build my own primitives or take on unnecessary complexity. It was about mitigating regulatory risk and protecting revenue.

When you rely heavily on U.S. hyperscalers in Europe, you’re exposed to potential disruptions — what if data transfer agreements break down or new rulings force major changes? The value of cloud spend, in my view, isn’t just in engineering convenience, but in how it helps sustain the business and unlock growth. That’s why I prioritized compliance and risk reduction — even if that means stepping a little outside the comfort of the big providers’ managed services.

> For example, you'd be hard pressed to find a team building AWS services who is not using SQS and S3 extensively

I design and develop products that rely on queuing systems and object storage; if its SQS or S3 is an implementation detail (although S3 is also a de-facto standard). Some of those products may rely on millions of very small objects/messages; some of them may rely on fixed-size multi-MB blocks. Knowing the workload, you can often optimize it in a non-trivial way, instead if just using what the provider has.

> The hard part is making this system immediately interpretable and getting "put a message in, get a message out" while making the complexities opaque to the consumer.

Not really, no. As you said, is already a solved problem. Aws obviously has different scale requirements than my reality, but by having ownership I also have only a fraction of the problems.

> There's nothing about rolling your own version that will make you better able to plan this out -- many of these lessons are things you only pick up at scale.

I cannot agree with you on this. As an example, can you tell me which isolation level is guaranteed on an Autora instance? And what if it is a multi-zone cluster? (If you can, kudos!); next question is, are developers aware of this?

If you have done any cursory solution design, you will know the importance of mastering the above questions on development workflow.

Reply on news.ycombinator.com