Intel Brings a Big Fork to a Server CPU Knife Fight
nextplatform.com
nextplatform.com
Cloud providers have structured their compute offerings around dollars per core. This creates the incentive to deploy the densest possible core count processors, and zero incentive to make those cores faster.
It can't have escaped people's notice how slow cloud VCPUs are. My 6 year old crappy laptop CPU has the same single core perf as a cloud instance that costs hundreds of dollars per month. And the frustration is not just cost, when you actually need good performance for something that doesn't scale so well horizontally, it's hard to get it.
Hence having a lot of low power cores which can be quickly scheduled and have ample bandwidth to the rest of the system is what makes them feel snappy on the cloud.
Another fact is, currently processors are really way more powerful and have more capacity demanded by the tasks. I have a new N100 NUC system. It has 32GB single channel memory, a paltry 500MB/sec SATA and another paltry 800MB/sec NVMe disk, and it's way snappier than my old desktop, which is more than enough for me (It's running Debian & KDE).
So having tons of E cores on a bog standard server won't hurt people who are trying to serve CRUD apps, but they may need a couple of P-core filled servers for DB, etc.
It's not very different from what Sun did with Niagara processors which moped the floor (and moped again just for the giggles) with Intel in web serving and small file processing tasks (500% difference, c'mon now).
The only downside is, we're going deeper on specialization cycle. Now, processors are the new accelerators, which are task specialized, and your server may not be just repurposed for something else on the other rack.
P.S.: I'm not a big Oracle fan, but the ARM servers they offer at OCI are way more snappier than the Premium Intel server I have at DigitalOcean. Result may change if I test their "long-straight performance", but ARM one works like local machine response wise.
Not really. They can and do price it otherwise e.g. with different instance types.
If we had to blame something then it's the move away from the traditional software licensing model. In the past, self-install software e.g. databases, OSes, etc would charge per core / CPU and that would be a huge chunk of the expenses added up. Now that we're free from this things are a lot different.
Having said that, the desktop side is still aiming for high-perf cores for e.g. gaming. It's not lost - just harder and harder to do.
> It can't have escaped people's notice how slow cloud VCPUs are
It's not that slow. It comes down to choosing the right instances and staying current. EPYC 4th gen single core on e.g. Amazon i.e. c7a is not the same as a 6th year old laptop CPU? It's also not hundreds of dollars per month (reserved / spot instances are the way to go).
Also this is nothing new. I remember the days of companies running their own VMWare data centers and those server Xeons were a lot slower. There are just heat and density concerns even if you had to do it yourself.
And it definitely costs hundreds of dollars per month. Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316.
It's never a value add to use the t* types. That's poor misinformation or at least outdated information that people keep spreading.
The advantage of c7a or Graviton 3 though is you get real cores. Most of the t* get you vCPUs, and 1/2 of it are just threads. It can be up to 2x faster in that sense.
> Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316.
Does your laptop have 8 real cores not threads that can 24/7 keep its peak speeds? Most overheat and throttle... are we comparing the same thing?
??? USD?
c7a.2xlarge (8 vCPU) spot instance pricing is $0.1032 = ~$80/month (us-east-1)
Even on just a 3 year compute savings plan and we're on $144.71/month. You're not required to commit to a particular instance type - just the spending, so all good?
This is excluding storage. Either way there's no reason to use EC2 On-Demand without a savings plan or spot. At least commit some of it a year and it's a big saving. Otherwise don't use AWS.
Side note: EPYC 5th gen has been announced and we'll likely see way more improvements on top. Pricing will definitely go down.
Also: this is just a rant on AWS. Hetzner, OVH or even Oracle cloud is way cheaper, like maybe 1/2 to 1/10 of this $316... it's not a cloud issue - just AWS.
They have is a page that lists this for every single instance type which I can't be bothered to search for right now.
As for spot instances, they're a great if you can use them, but you can't hold their price as being representative. Now using savings plan prices as representative is defensible, but it won't bring the cost below the "hundreds per month" order of magnitude.
"One of the major differences between M7a instances and the previous generations of instances, such as M6a instances, is their vCPU to physical processor core mapping. Every vCPU on a M7a instance is a physical CPU core. This means there is no Simultaneous Multi-Threading (SMT). By contrast, every vCPU on prior generations such as M6a instances is a thread of a CPU core."
You made me go check and you're right! Thanks for the correction.
Those design decisions then bubble up to server space, not the other way around.
I don't think you realize that that's not a niche, whimsical requirement that you can only explain with conspiracy theories.
For decades, all hosting providers have been paying a premium to have more computational power with a smaller size and power footprint.
This is not a AWS whim. All cloud providers want this, and have been paying a premium for this. If anything, AWS had the budget to roll out their own CPUs.
How long ago did AMD launched its Bulldozer line?
It goes like this:
Let's say you run a company that has a critical database that cost $10M in software licensing, $100M in software development, and is central to your $1B business. Now go host it on a cloud server that has margins of $10 per core per month. If that cloud vendor can cut your performance by 10% to save $1 without you switching providers, they will do that, because across millions of cores and millions of customers, that's huge money... for them. Sure, you've lost $111 million worth of performance on $1.11B of your stuff, but that's your problem. The cloud vendor's executives are whistling a happy tune while flipping through a yacht prospectus. They don't care about your lost efficiency, they care about their single digit dollar margins.
You'll see this any time some huge company uses a platform, input material, or service that is very low cost relative to the value of the service or product that it goes into.
As another random example: the quality of the cotton cloth that goes into name-brand T-shirts. You can buy a $100 shirt that has 10c of material in it because some guy worked very hard to make it as thin as physically possible to cut that down from 11c. He doesn't care if your nipples show through.
Conversely, the companies with the best products in the world typically have good vertical integration. E.g.: Apple.
PS: All major cloud providers use processor "C-states" to save a few dollars per server per month on their power and cooling costs. Turning that off is the magic turbo button on database servers. I do this for kit that I have direct access to. You can't do this in the cloud, and never will be allowed to, no matter how many millions of dollars this costs you.
PPS: All cloud SLAs refund the cost of the service you purchased, not the value of the service you were unable to sell due to the outage.
Space is practically finite, if more of the same performance can be crammed in, that's still a win! Your laptop and servers are solving for entirely different workloads and constraints.
The egregious pricing has always been there, but the proportional nature makes sense. It's less perverse incentive and more... natural characteristic
Was that the benefit? I thought it's reliability, scalability, etc.
I remember having a FAST shared server for CI. "Someone" was in charge of it. Every time it went down everyone lost 1/2 a day trying to ask "someone" to reboot it.
No 1 got rid of any policy. Cloud meant any team could just take out their company card and sign up. Same as how SaaS got popular.
The problem is, what is critical? That server was non-production and there were 100s to 1000s of things more critical to deal with. It was low priority.
Maybe you know but to many up the chain in many orgs - they don't know. I've seen way worse scenarios - things on someone's laptop with ports opened that were production critical.
Cloud isn't the cure, you're right, but it forced a lot of change.
> Cloud isn't the cure, you're right, but it forced a lot of change.
That's the thing - there are plenty of organizations that operate solely in the cloud but do not give their teams the authority/ability to spend without prior approval. And there are plenty of organizations that have no cloud presence that allow their teams the discretion to spend money when spending money is the right answer.
For a lot of companies, the move to the cloud was a super large benefit. I'm just arguing that the real benefit to those organizations was a side effect and that many organizations probably still don't realize this.
My point was exactly that internal use cases still has a reliability/scalability factor.
> it is making more and more sense to self-host either on prem or in a data center
I'd say it's the other way around. Automation improvements, infrastructure as code and less knowledge has meant Cloud now makes sense. Back in the days it was a lot easier to find people skilled in self-hosting and the common language was something like Redhat and not AWS. Self-hosting didn't meant getting internal networking and stacks enough to run kubernetes. You'd have a single server or 2 and that was it. It's a totally different game.
> I have never seen a big three providers cloud costs per $ for a given machine ever reduce
It has in a lot of ways, e.g. similar prices for faster CPUs (newer generation) and e.g. with AWS getting 2 real cores vs 1 (core) + 1 (thread).
The problem is server hardware is cheaper. Electricity, rent (data center) and wages is not. This growing cost eats into self-hosting a lot more than the hardware savings. The prices not going up is actually already a cost saving. Most data centers want to up your contract by at least inflation % per year if not more.
Electricity of a Macbook M1 Pro running my build locally is cheaper than self-hosting or Amazon, as far as I can tell so while I get your reasoning - I'm talking about Dev compute.
Also MRKL and other tools (even yes for Infra as code) are making it to my mind more viable and easy than the old days. Good times are ahead.
I wonder why IBM doesn’t do more with POWER10 in their cloud, because each core can go up to, IIRC, 8 threads.
Same for Oracle, BTW, but they are no longer investing on SPARC, just milking until it dries up completely.
we can't notice of coure due to how software is these days. your average desktop CPU might have 8Cores / 16 threads.
if you rent a dedicated machine n skip the 'cloud sharing' your average web app -- won't be able to exhaust a single machine with 128GB Ram to 2TB Ram and 64 core CPU. Now we're talking about 288 Cores with the Intel Efficient CPU's.
For a big part of the 2010s CPUs were kind of stagnant since Intel didn't really have much competition and was happy resting on their laurels and dropping slow incremental upgrades (like a 2012 CPU and a 2016 CPU were basically the same), and AMD were floundering and playing catch up, but when they did catch up and really blow past Intel, it really lit a fire under Intel's ass and now there's real competition for x86 CPUs again.
I think it's a pretty favorable description of cloud providers that they will sell you 2015 performance for 2024 dollars.
Having said that, making software more parallel is still a good investment.
I think your take is too conspiratorial. There are plenty of good justifications for this sort of approach. One of the selling points that stand out is that the bulk of the threads/processes don't have high computational needs and act more as event handlers, and typically they stay idle while waiting for IO, which represents the bulk of their workloads.
Why would you waste your budget on high-performance cores if most of what you do is sitting idle while waiting for data?
There's a reason why things like node.js are a success.
> and typically they stay idle while waiting for IO
That's just not how you size servers. If most of your cores are idle you will add more work to the server, or you will have bough one with fewer cores to begin with.
At some point youre just actually pushing 10 Gbit/s through a process that is using a few 10-15% per core.
I don't think you understood the issue.
It's not that cores will be idle. It's that the bulk of their workloads is not performance-oriented. They are there to context-switch while handling IO-bound tasks. You'd be wasting money if you pay a premium on high-performance cores if you're using them in IO-bound workloads. You waste money buying the processor, and you waste money powering performance-oriented cores to run workloads that do not benefit from this performance. For the same budget, you are far better off having a larger number of efficient cores than a smaller number of high-performance cores. Because you are running more efficient cores, you can cram more of them in the same package without ramping up cooling needs.
Think about it for a second: where is the performance bottleneck of your server if the bulk of its load consists of IO?
You should turn your conspiracy theory upside down and ask yourself how come a cloud provider is interested in running more cores in smaller spaces and in spending less energy cooling them.
Truth is, you don't have the right to need fast cores any more. Those times are over, when you could write poor code and buy a more expensive CPU. Now, if you want your software to scale, it's up to you to make it scale horizontally. Either this, or suck it up!
I believe that they did the right move by investing a lot in fabs. It takes many years to get them running and profitable, but then it's steady income.
Now, on their processors business unit, it could be that they never manage to become the #1 again, but they could as well play the AMD strategy now, by selling cheaper hardware, and even eventually become a very small player in this market.
It's also curious the other efforts, like their GPU with Battlemage, can they really catch up with decades and have a competitive product? It feels like building all the software needed to properly run games in a GPU like the Radeon or GeForce is a time consuming effort.
All things considered I think their GPU efforts are very strong. The Hardware is competitive and the pace of improvement on the driver end was impressive, even if the initial showing was weak. They didn't drop Alchemist, they worked on it non till it became a viable option in its price class, even if a weak one. For a first showing, that is promising.
I feel like I've been hearing similar things for the past decade...
Web workloads: I just care about threads per core. Without hyper threading I can -- regain all the perf lost to heart bleed. -- Get near ARM like power performance.
The servers with this chip will be able to put ram in the PCIe slots. You will be able to build for some interesting workloads (redid/valkey memory as cache/disk/table)
Candidly with cloud costs being what they are, running your own hardware is having more appeal by the day.
Whereas cache uses a lot of silicon area (expensive), and memory bandwidth/data transfer costs power. Speculative memory reads trade lots of memory bus capacity and power for performance.
Therefore, for maximum compute per watt and compute per dollar, you probably want excess cores and to make sure you fully utilize any cache and memory bandwidth. You don't do much speculative read-ahead and you accept lower per-core performance to get more throughput per dollar.