Comment by gjulianm | Hacker News Reader

Comment by gjulianm | Hacker News Reader

Parent Full thread

When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy machines. At my job we build probes for network monitoring, and we're able to ingest quite a lot of traffic (up to 14Mpps per link, for a fully saturated 10G link in our hardest tests), ingest a good volume logs or metrics from remote systems, do some extra processing and serve a web application in just a single instance. We use C/C++ for the traffic capture and processing, Postgres/Elastic for storage (depending on the balance performance/flexibility), some AWK for quick file processing (you'd be surprised how fast can it be) and then Python for plumbing code.

On the other hand, I've seen technical blogposts and whitepapers on how to optimize a "big data ingestion system" that requires clustering and tweaking and whatnot for a fraction of what we're able to do with a single machine. People should try harder to scale up before starting a cluster and scaling out.

I was in a discussion on HN just yesterday where the assertion was made that startups should default to the cloud because the upfront cost of allocating real hardware is too high. I’ve got a $20 Netcup server that can probably handle the traffic of 90% of any projects I’ve ever worked on (including my stint at big tech cos). It’s amazing how cheap and fast modern computing is when compared to the standard cloud offerings.

... but netcup is just another cloud offering? What are you getting at here? I mean, it always makes sense to shop around and look at pricings before choosing one, but my unresearched guess is that there are so many providers nowadays that the pricings across all of them are within cents / dollars for the same service (based on RAM, storage, etc)

If they don't want to pay upfront for hardware, they can also "default to cloud" while just using cloud-based compute instances.

> When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy machines.

Beefy machines? You can get decent throughput (https://news.ycombinator.com/item?id=29659986) on the cheapest VM your cloud provider ... uh ... provides.

Beefy machines don't give you redundancy though. Beefy machines arguably make redundancy more expensive.

If you run on a single beefy VM, you need at least two of those running for redundancy, meaning you're 100% overprovisioned (m6a.32xlarge x2 = $11/hr).

If instead you run on 4 smaller instances split across multiple regions (or AZs or whatever), you only need to overprovision by 25% if you want to handle a single failure (m6a.8xlarge x5 = $7/hr).

Obviously if you're doing offline or non-latency sensitive processing, that doesn't matter.

Conversely, since no one is actually doing true resilience anyway (just look at how many services failed when one AZ in us-east-1 went down), why not just start a new machine with the same code and data (either through active replication or shared hard disks), take the 1h downtime until that's up and running, and be golden?

You can even deduct it from your customer's monthly bill.

But if you have 20 of them then the frequency of problems goes up.

In the context of this conversation, where people are bringing up situations with 100’s of servers, 4 servers still counts as vertical.

For AWS, where you can lose an AZ (and lately that feels like a certainty), there are arguments to be made that 3 or 6 servers are the least you should ever run, even if they sell a machine that could handle the whole workload. You should go down 2 steps and spread out. But don’t allocate 40+ 2xlarge machines to do the work. That is also dumb.

What is also screwing up the math here is that AWS is doing something untoward with pricing. With dedicated hardware, for the last six decades, you have a bimodal distribution of cost per instruction. The fastest hardware costs more per cycle, but if your problem doesn’t fit on one (or three) machines you have to pay the price to stay competitive. At the other end the opportunity costs of continuing to produce the crappiest machines available, or the overhead of slicing up a shared box, start to dominate. If you just need a little bit of work done, you’re going to pay almost as much as your neighbor who does three times as much. Half as much at best.

Depends on what kind of redundancy you want. You can achieve quite a decent uptime for a single instance with local redundancy, process health watchdogs to restart things, crash isolation… Yes, some people will need a cluster with no downtime on failure events, but the clustering itself adds complexity and can even increase downtime if it causes problems. For example, a single machine can’t have load balancer issues, or replication errors.

Also, you’re assuming an active-active redundancy scheme, which might not be always necessary, and not counting the cost of the elements required to support the cluster.

I don't really think that these ideas mutually exclusive. The example given here ( https://news.ycombinator.com/item?id=29660117 ) discusses 4000 nodes running extremely inefficiently. Depending on how inefficient it could be, you could replace it with multiple beefier servers placed redundantly in different zones. (And if it's _really_ inefficient you might be able to replace it with an order of magnitude fewer servers that aren't beefier at all.)

I think really the point being made is to efficiently use your resources. Of course some of the most expensive resources you have are programmers so inefficient hardware solutions may end up cheaper overall, but I've personally seen solutions that are architecturally so complex that replacing them with something simpler would be a big win across the board.

I get the feeling that for non-interactive workloads you want to go as big as you can get. There are many problems you can just crunch through using a computer engineer’s cleverness instead of overdoing it with your own. These are usually throughput problems and the solution domain is distinct.

When humans are involved you don’t want a single failure to be felt by everyone at once. And they get grumpy when we don’t pretend their button click is the most important thing we will handle today.

Reply on news.ycombinator.com