Hacker News Reader

Comment by MaxBarraclough | Hacker News Reader

If we want securely-isolated low-horsepower machines, why don't we just fill server-racks with large numbers of low-horsepower physical computers, rather than small numbers of big-iron computers using software isolation in the form of VMs?

Wouldn't that win half the battle, at least on server-side? It wouldn't apply to workstations, of course. Malicious/untrusted processes running there would still need to be isolated (malicious JavaScript in the browser, say).

Also, am I missing something, or are these concerns really not applicable to gaming applications (which the article emphasises)? That's a highly trusted codebase, so securely containing it isn't worth a 15% performance hit, no?

View on HN

The cloud is only able to be cost effective because they are able to utilize those fleets of big-iron computers to provide nearly infinite VMs and resources to anyone who signs up for their services. Relegating each customer to their own physical computer would raise costs so high as to essentially eliminate the purpose of the cloud's existence, and most businesses would be better off going back to their own data centers at that point.

> Relegating each customer to their own physical computer would raise costs so high as to essentially eliminate the purpose of the cloud's existence

plenty of places rent bare metal machines, and the price doesn't seem quite that world-ending. see packet.net for instance.

and it only gets better if on the bare metal machine you get to turn off all of the performance-draining fixes.

It's actually much cheaper than cloud. It doesn't have the scaling flexibility, but that is partially compensated for by running faster, lower latency, etc. Lots of people use Cloudfront etc but still host on their own bare metal.

Isn't it still 'cloud'? It's still on a server somewhere that I don't think about, which is really all 'cloud' tends to mean.

I don't care whether Amazon use virtualisation to get me my 'instance'. They're always careful to stick with 'instance', and never to commit to 'VM'. Wise move, now that they're offering physical ('dedicated') instances [0]. Time will tell whether they take an interest in low-horsepower dedicated instances.

[0] https://aws.amazon.com/ec2/pricing/dedicated-instances/

There is a non-smooth transition from a server you own/lease and sits in specific rack in a known data centre with well understood routing and one provisioned as a dedicated instance in AWS/Azure etc that you manage with an API.

> Relegating each customer to their own physical computer would raise costs

I'm not convinced the cost difference would be a showstopper. More little computers rather than few big computers. Same total horsepower. More motherboards, sure.

More boards, more rack space, more hardware management overhead, more and longer wiring with associated losses from resistance, bigger buildings... it's not really simple.

> More boards, more rack space

Not if they can be made smaller and packed more densely.

> more hardware management overhead

I'm not sure this is a significant issue.

> more and longer wiring

I don't think this follows. They could be clustered. A cluster could be made to look a lot like a conventional big-iron server.

We're talking about replacing a single large machine running, say, 16 VMs (each with an IP), with 16 small machines (each with an IP). Put an Ethernet switch at the last-mile and it's virtually the same, no? Different number of MAC addresses, sure.

For certain kind of workloads renting bare metal is waay cheaper than VMs. What the small time bare metal providers cant compete with is all the different services sold along with the VMs. If it was just price per VM I suspect bare metal renting would be a lot more cost efficient in a lot of scenarios.

The other area they can't complete is on variable or disaster recovery scenarios, the sort where you'll reserve CPU instances, but only spin them up for your heavy load parts of the day or when you have an unexpected migration. And when you're not using them, AWS puts them on their spot market.

They're probably a bad match for when you have a single batch of work to do, like when the New York Times converted their archives, or every time Netflix has to reencode their video library.

More machines means more power to cool them. Electricity bills are a big chunk of the cost of running a datacenter.

I was just discussing this with a friend. Obviously there's a certain degree of multi-tenancy sharing that's most cost effective. For electrical power use and removing heat there's a floor where, if not being used, the best you could do would be to turn a system off, which has its own issues. Go too far and you won't be able to also run a spot market in CPU instances like AWS does.

> For electrical power use and removing heat there's a floor where, if not being used, the best you could do would be to turn a system off, which has its own issues.

What issues? If your cloud architecture can't save power when demand is low, how can that be a positive?

> Go too far and you won't be able to also run a spot market in CPU instances like AWS does.

As I understand it, a spot market isn't ever an architectural goal, it's a way to exploit the practical realities of unused resources/capacity. If that necessity goes away, that'd be a win.

I see at least 3 possibilities:

1. The spot market is about filling temporarily-vacant VM slots on host physical machines

2. Additional physical machines must be kept online so that VMs can be quickly spun-up, and the spot market is about capitalising on this necessity (this is really a variation on Option 1)

3. The spot market is about deriving at least some revenue from the unutilised fraction of an always-on fleet, encouraging customers to run heavy jobs during times of low load on your cloud

I doubt Amazon's fleet is always-on -- I imagine they have fairly significant capacity in reserve to keep up with growth and spikes -- so I doubt it's Option 3. (This is a perfectly obvious question but I really can't find a definitive answer. Maybe my Google-fu is weak today.)

I think Amazon’s hands are mostly tied when it comes to powering off machines. Many VMs are run indefinitely, so whatever machine they’re on has to stay powered on. They may have live migrate (I think they do, but no official word), but they can’t be too aggressive about that or customers will notice.

I stumbled across a 2012 article on this [0] that says they'd 'retire' instances. Wouldn't force the VMs over to another host, they'd just terminate them. They'd generally give forewarning.

I wonder about live migration. I imagine there must be some brief interruption even there.

> Many VMs are run indefinitely, so whatever machine they’re on has to stay powered on

Amazon must be adequately motivated to find a way to power-down unused capacity. My guess is they have some kind of system to maximise this.

[0] https://www.theregister.co.uk/2012/12/20/aws_ec2_servers_ret...

Yeah, I’ve experienced several instance retirements. Most with a fair bit of warning, one a few minutes after the instance stopped responding. I haven’t gotten a retirement warning in several years, so it seems they're doing something to mitigate hardware failures, although my infrastructure is small enough it could just be luck.

Plus Xen has live migrate now, so I assume AWS’s frankenxen would have it as well.

There is indeed a brief interrupt, which is why I said they can’t be too aggressive with it.

Anyways, I’m sure Amazon does what it can to power down machines, I just think they’re stuck leaving much the data center running because they can’t kick off running VMs.

> More machines means more power to cool them

That doesn't sound right. A large number of weak computers, should be easier to cool. A cluster of smartphones can get by with passive cooling, but my laptop cannot.

Apart from cooling, I'm not an expert on this but I don't know that performance-per-watt has to improve as overall power increases. Going with my idea you'll need to power more motherboards, but they'll be less powerful motherboards. (The total number of CPU cores being powered remains the same. Same goes for RAM modules.)

What's expensive, and some time ago became a significant limiting factor in densely packing machine rooms, is removing heat from the room. As long as you can supply cool air to the racks they can take care of removing heat from motherboards and other components.

If cooling systems for machine rooms haven't caught up with modern systems that dissipate more heat, the direction you're thinking has merit. But I doubt there are serious limiting factors here, besides cost.

> If cooling systems for machine rooms haven't caught up with modern systems that dissipate more heat, the direction you're thinking has merit. But I doubt there are serious limiting factors here, besides cost.

I think we agree, but I wasn't really thinking of power-savings as the real motivation here, I was thinking of security.

It seems likely we're going to see a steady steam of Meltdown/Spectre-like security concerns. We can have our cake and eat it -- isolation and all the CPU cleverness there is -- if the isolation between instances is done physically rather than with VMs.

> A cluster of smartphones can get by with passive cooling

Is that true? My phone gets quite hot if I run a taxing load on it for too long. My, perhaps poor, intuition tells me that phones can rely on passive cooling because of the burst nature of phone use. If you’re utilizing your smartphone cluster, I can imagine your fleet overheating.

> If you’re utilizing your smartphone cluster, I can imagine your fleet overheating.

iPhones survive long sessions of 3D gaming even when wrapped in protective plastic shells. If it was an issue, use a bigger passive heatsink than the one they cram into the smartphone form-factor. You could never do that with a server-grade CPU.

With a high-power CPU, the heat emissions are more concentrated than if we use many low-power CPUs.

My iPhone X definitely gets uncomfortably hot after less than 30 minutes of playing PUBG, and I don't even use a case. To illustrate my point, there are "gaming cases" with fans on them, so it's enough of an issue for companies to market solutions at it.

Ok, let's assume the iPhone's cooling system -- i.e. the metal back of the phone -- isn't adequate for long-running heavy loads. I don't think there's a deeper point behind this. Fit a proper passive heatsink and you're all set.

The Raspberry Pi would have been a better example.

If you go with older CPUs you're gonna have higher loads to execute the same work. Which means more heat.

You'd be right, but I didn't say older CPUs. Take modern server CPUs and reduce the number of cores. May as well go all the way down to single-core.

There's clearly a market for weak, cheap, machines -- Amazon didn't originally offer a 'Nano' instance-type, but they do now.

Reply on news.ycombinator.com