Hacker News Reader

Comment by alberth | Hacker News Reader

I thought TDP was a selling point of ARM for servers.

Cavium has a higher TDP (180W) than either of the Intel Xeon's listed in the article (150W, 165W).

Is this more about "performance per watt" then?

EDIT: Also makes me wonder just how much more energy efficient and/or performant Cavium would be if they moved away from TSMC 16nm fab over to Samsung 10nm fab (like Qualcomm). There's a sizeable difference between 16nm vs 10nm (assuming the fabs are using the same metrics). Qualcomm using Samsung 10nm fab is only 120W.

View on HN

Yes, it's about performance per watt, and also performance per dollar. The Intel chips lose badly by that measure. Also, TDP isn't necessarily the right figure of merit here. It's defined and measured differently by different vendors, so it's not always directly comparable. The more important and more directly comparable measure is actual system-wide consumption when running a defined workload.

> The Intel chips lose badly by that measure.

That's true, but then the Intel chips shown in that chart are Xeon Gold/Platinum parts that heretofore have had essentially no competition in the market and thus command huge premiums. Intel is making crazy bank on its server parts, basically.

So in the presence of real competition (which this looks likely to be, though to be fair it's not actually in the market yet and is likely priced at a point intended to grab eyeballs), it's likely that Xeon parts will drop in price. You won't be looking at that delta for long if Intel starts losing market share.

So... if they were merely cost-equivalent, is this part worth it? Because the pure silicon metrics here (performance per watt and per die area), as mentioned upthread, are sort of a wash. And maturity/stability issues (c.f. the bit in the linked article where they point out that the Cavium board draws 300W at idle!) kill products in the datacenter space all the time. Consumers are OK with occasional glitches, but no one is willing to put anything on their rack that has any kind of failure risk at all.

> no one is willing to put anything on their rack that has any kind of failure risk at all.

Are you sure about that? The people who have hundreds of thousands to millions of machines also have systems designed to handle a certain failure rate, and have historically been quite happy to run on non-premium hardware. The costs do add up. Even if it's just a handful of companies, in terms of market percentage that's far from no one.

It's a standard thing in the enterprise space, yes. No one buys anything that seems flaky. It's true that big datacenter operators (Amazon, Google, Netflix et. al.) have a lot more freedom to handle risk.

But regardless, things like the 300W idle draw are going to kill a sale even there. That's almost a dollar a day, so there goes the price delta with Intel over the life of the system already.

The idle draw is exactly the sort of thing one would expect to be fixed well before volume production. I've been at companies that made hardware (in fact at least one of my co-workers from one of those jobs is at Cavium right now) so I honestly think that's a bit of a red herring. Comparing pre-production hardware with production is always a bit unfair; let's not make it more so.

Wait wait wait. If it's preproduction hardware then the prices in the article are meaningless because you can't buy it. If it's unfair to compare performance metrics on preproduction hardware than surely it's equally unfair to compare prices (I mean, they may not even know the yields yet). You don't get to claim "Intel loses badly" and then hide behind "well, they can't have expected to have fixed everything yet!".

For comparison, those Xeon parts are Skylake-SP dies that first shipped 11 months ago. You can buy them on Amazon and rent them on AWS. They're a mature product in a well-established channel and their prices are stable and matched to the market.

The bottom line is that Cavium pulled out all the stops, shipped a board that works and performs... very adequately, with a few glitches. And frankly I don't know that glitchy acceptability is going to sell a whole lot of servers. So they slapped a price tag on it that draws eyeballs that the performance alone won't. If they can do that it's probably good news for them, but it's still a long road from here to revenue.

Yes, the prices on both sides are kind of meaningless, but one can make reasonable guesses about how they're likely to change. There is no plausible scenario in which Intel drops their prices far enough for this one to be close. They've never done anything like that before, even when it might have made sense. One can also make reasonable guesses about what kinds of flaws are likely to be fixed before the hardware goes into full production. Power management not kicking in when it should is a pretty obvious one. Yes, there's some guesswork, but there's a big difference between educated guesses vs. "Cavium's problems are carved in stone and Intel's prices are infinitely flexible" wishful thinking. Don't put all of your money into Intel stock.

> There is no plausible scenario in which Intel drops their prices far enough for this one to be close.

"No plausible scenario." Just put that one on a T shirt and come back in a year. The tech industry is littered with the graves of hardware products that got close to being great but ultimately didn't work out. And you're comparing it to a year-old mature product literally stocked at Amazon.

You do realize that Intel themselves were the upstarts once, right?

And unlike the literally dozens of semiconductor failures that littered the valley of the 1970's, they succeeded. This doesn't help your point.

I'm not saying Cavium cannot succeed, I'm saying delivering a chip to a reviewer that doesn't actually beat a Xeon probably isn't enough, no matter what price tag you invent.

Let me tip you off about something people with actual business experience know: if you don't get your stuff in front of reviewers, flawed as it may be, you fail. Hardware has long lead times. To get design wins, you have to sell on promise. Even some of Intel's own foundational products were objectively worse than contemporaries (e.g. even the IBM PC designers knew that the 68K was a better chip than the 8088) but got better over time. They got better because they had the wins due to selling on promise, and therefore have the funds. Any Intel competitor who waits until their product is better than Intel's on every metric, beyond any doubt, before they show anything to anyone, will run out of funding and quietly die.

As I said, one has to make educated guesses about what kinds of problems are likely to be fixed before full launch. Dogmatic statements based on experience limited to irrelevant domains aren't useful to anyone, and plain old pro-Intel FUD even less so.

I know many people that have evaluated ARM for a variety of server applications and ARM's advantages, in terms of tradeoffs, are not what many people assume. Where ARM shows well is applications that spend a significant amount of time idle in power-restricted environments (battery-powered analytics clusters being a common example) and running software that is weakly optimized for efficiency (some web app stacks look like this). In practice, this is a niche set of requirements for servers.

The challenge that ARM has long had in servers is that highly optimized and efficient Intel codes have competitive performance per watt and performance per dollar. Cavium is quite open that their cores are optimized for low ILP/IPC software, which is essentially arbitraging software that is poorly designed for x86 silicon. As data and analysis scales have increased, more standard server software is specifically designed to be hyper-efficient on Intel, not just HPC codes like the old days. This makes it increasingly difficult for ARM to compete with the operation throughput per watt/dollar that is possible with Intel and well optimized software.

There is the additional issue with platforms like ThunderX2 that even though it could be competitive in theory, it would require software to be specifically engineered around the peculiarities of the microarchitecture. Software that assumes Intel microarchitectures and optimizes toward that, either explicitly or implicitly, will inherently be suboptimal on ThunderX2. This isn't something that can be fixed in a compiler, it is intrinsic to the architecture of the software at a higher level e.g. C++ codes embed many assumptions about CPU cache topology and properties.

For cache-tuning, there is polyhedral optimization, available in LLVM via Polly and in GCC via Graphite. Of course they don't fix everything, but they are great at automating loop nesting decisions with the goal of matching data locality to cache size(s), and, potentially, even SSD/HDD storage (think compiling the software so that it makes good use of the 1GB ram/core you can give it, and not be awfully inefficient with it's swapping.

Unfortunately, the polyhedral optimization was still broken in gcc last I tried. Perhaps it's fixed in gcc 8.

My tests on a recent (6 weeks, I'm too lazy to re-build) git snapshot from master worked reasonably well, but I didn't quite reach the full testing stage, as it was doing something (I was trying to benchmark CockroachDB on Goldmont (server atom), and the -mnative flags are just a couple weeks old on the mailinglist, and a month younger in git master.) and I didn't have time to check back. But it appeared to work when I activated the command line flags.

I worked in HPC for over a decade. Most of the codes are not hyper-efficient on Intel - particularly not current-generation Intels that need efficient use of very wide vectors to reach peak performance.

Many codes are memory-bound, and the TX2 has excellent bandwidth. This shows up in real-world simulation codes such as OpenFOAM.

Very interesting as I used to do OpenFOAM parallel runs years ago on x86. Do you have any links to OpenFOAM benchmarks on ARM64?

The conclusion of the linked article has a reference to a relevant paper

But, as usual, you don't know what the profile of the calculations were, in particular because you don't know the mode it's operating in/data it's operating on. ("OpenFOAM" is actually many different programs.)

That said, the indications are that the performance is decent for HPC. What I haven't seen is a comparison with Ryzen (or POWER9). There's also a lack of data even on the SIMD hardware in ThunderX2 and POWER9, at least that I've been able to find.

Indeed -- memory- or communication-bound. Somewhere there's a Dell "roofline" plot indicating IPC in some codes running in an unspecified way. See also John McCalpin's writings.

Things doing 3-D FFTs (common in materials science) tend to spend a lot of time in an MPI collective at sufficient scale. I spent half an hour profiling and then changing an MPI parameter for 30% improvement in one case at not very large scale.

Small microservers with cellphone class cores in them were something a few companies pursued but they never really took off. And there was one company with Atom based servers in the same space, it wasn't just all ARM A9s.

This one is about trading off minimum latency for more throughput with lots of more medium sized cores in a way that's probably better suited to most server workloads.

> assuming the fabs are using the same metrics

They're not. Nanometers are almost meaningless now.

Reply on news.ycombinator.com