Comment by otterley | Hacker News Reader

Comment by otterley | Hacker News Reader

(I work for AWS. Opinions are my own and not necessarily those of my employer.)

I've been doing some initial M6g tests in my lab, and while I'm not able to disclose benchmarks, I can say that my real-world experience so far reflects what's been claimed elsewhere.

Graviton2 is going to be a game changer. It's not like the usual experience with ARM where you have to trade off performance for price, and decide whether migrating is worth the recompilation effort. In my lab, performance of the workloads I've tried so far is uniformly better than on the equivalent M5 configuration running on the Intel processor. You're not sacrificing anything by running on Graviton2.

If your workloads are based on scripting languages, Java, or Go, or you can recompile your C/C++ code, you're going to want to use these instances if you can. The pricing is going to make it irresistible. Basically, unless you're running COTS (commercial off-the-shelf software), it's a no-brainer.

I feel like this is where hyperthreading is finally starting to bite Intel in the rear. Cloud providers have been selling "VCPUs" that aren't actual cores. I best most customers don't even know what they're buying. Even if ARM cores are slower (and they don't really have to be), they're still going to be faster than hyperthreads.

I don't think so. Most apps get 0.5-0.6 instructions completed per clock cycle, and Intel cores can put out multiple instructions per cycle.

Most customers don't measure, and don't optimize cache usage for the actual tradeoffs (cache, tlbs) to matter.

Most, yes. But things like databases, video/data compression, compute / deep learning workloads, etc, _are_ negatively affected by the fact that cores aren't really cores. Basically anything that's actually using the CPU to an appreciable extent will be affected by that. Add to that the hyperthreading-specific CVEs as well.

I'm confident that those working on applications where CPU performance is an important issue, as well as reliability of said performance, are actually not running their critical applications on virtualized infrastructure. The cloud is good for "good enough" solutions where architectural design leveraging horizontal scaling does the job well. In those applications, how a CPU is used or what their performance is is something that falls in the wrong side of the domain of microoptimization. In the Cloud world, the only thing that's relevant is if an application infrastructure needs to scale, and what's the financial impact of that in the operational cost.

Your confidence is misplaced. Netflix is a well known user of cloud services for encoding, and just about everyone runs at least part of their deep learning workloads on AWS or Google Cloud. Not to mention databases.

Encoding is an embarrassingly parallel problem which is trivially scalable by launching new instances. That usecase fits precisely the scenario where CPU raw performance is not an important issue, and the cloud is already good enough.

That's fine for small counts. When you're taking a 20-50% efficiency hit for this, that's a good sized bill difference.

Performance is not tied with price, only with instance count and the amount of computational resources assigned to each virtual instance.

I am extremely excited about this development. Less so getting everything that assumes x86 in the stack supporting a “parameterized” platform.

(Hi Michael!)

Hi Rick! I know it's only a part of the toolchain needed to support the migration, but multi-arch Docker image support for Amazon ECR is definitely on our immediate roadmap.

heh, I've had a very fun experience with spotinst.com — my a1 spot instance went down and that service couldn't restore it because it did not label the AMI as arm64. Reported that to them, got a couple acknowledgements but haven't heard back in a while, so presumably this is still not fixed.

It is surprising, because I was under impression that Java has so many optimizations for x86 and ARM was so new that it's almost impossible to beat x86 without very significant investments. It's nice to hear that I was wrong.

It's not very "new". People have been working on porting, improving and optimizing software for the Arm server ecosystem for more than a decade now; really performant and widely available hardware may be new on the scene, but it would be silly to wait for that before starting work on the software side of things...

ARM has been so dominant in the mobile market that there has been a lot of effort around ARM optimisations, both in compilers and interpreted languages. Way more so than alternative architectures like MIPS etc.

In a previous job, we had a cross compiler that cross/decompiled a static x86 Linux binary to Java bytecode (with syscalls emulated in Java). Testing this on an ARM processor the Java version of a scientific benchmark we were using was significantly faster (1.6X) than the best gcc could do with the original C. This wasn't anything fancy our cross-compiler had done, but the JVM itself. This isn't a comparison to x86 hardware, just an indication of the effort put into the ARM based JVM.

x86's age and complexity give it a significant disadvantage. Both the cache coherency model and the instruction set incur a lot of overhead to do at speed.

That's very interesting. Could you elaborate?

A lot of the most commonly run software out there doesn't use the large, complex instructions offered on x86, so a bunch of pristine silicon goes to waste. Use the space taken by AVX512 etc to make more, simpler cores, and you get more performance for the same price, or less cost for the same performance. Simpler cores are easier to clock higher with less voltage, and less likely to have defects that would pull down yields.

The big vector units aren't the problem though. They're a consequence of the big complicated schedulers that most x86 cores are designed with. As long as the core has to be huge anyway, you might as well spend some space on more powerful math units.

It's possible to design an x86 chip with much more priority on throughput per square centimeter, with many more simple cores working together, but I have no idea how it would work out.

There's a lot of logic and complexity on decoding, fusing, etc.

FWIW, 32 bit x86 (i386) and ARM arrived in the same year (1985.)

Java doesn't have that many x86 specific optimisations actually. It does some auto-vectorisation, it supports things like AES-NI and other specialised hw instructions, but, those are easy to port to ARM.

The vast bulk of the effort in modern JVM compiler optimisations is more at the program structural level: removing allocations, merging methods together so they can be optimised as a whole, removing abstraction, and so on. All that stuff is CPU independent.

I too have been playing with m6g, and while I’m allowed to disclose benchmarks I haven’t bothered to run any; the pedantry that unleashes is enough to drive me to drink.

“Lies, damned lies, and benchmarks.” I can say that the qualitative experience is superb; everything “just works” as you’d expect it to, and performance is stellar.

Does the chip still manage to have superior power efficiency versus x86 even at these performance levels?

To my uninformed understanding, power efficiency wasn’t really the design goal. If it’s AWS’s power bill, do we care as much (presuming sustainable energy)?

Power and heat dissipation are a huge part of TCO for datacenters. I'm sure Amazon picked an appropriate point on the performance per watt curve.

Given that your cost is going to be correlated to AWS’s cost of goods sold, I’d argue of course we care.

I care what AWS charges me; I don’t have the energy to care what their underlying cost structure and its constituent parts looks like.

Indeed; we shouldn't care about AWS power efficiency but we also shouldn't assume that it's bad just because we can't see it.

As I said in my comment, their underlying cost structure has a direct relationship to the cost you incur (at least in the long run with semi efficient markets). There’s a reason why commodity markets tend to revert towards variable cost.

> There’s a reason why commodity markets tend to revert towards variable cost.

Is that some theory of economical model ?

Or is that a way of saying the final price of a product varies greatly because the commodity part with the product is only a small percentage of the TCO / BOM ?

>If it’s AWS’s power bill, do we care as much (presuming sustainable energy)?

I care because it's an interesting question regarding future trends in technology. More broadly, people care about a lot more than just what has direct short term applications to their jobs.

Fair. Trouble is, we’re never going to get Graviton2 devices of our own outside of an AWS datacenter or device; the power profile is intellectually interesting, but not likely to ever enter the public sphere.

I'm not sure why Amazon won't try to recoup some of the investment by designing and marketing other boards / devices with this chip, or by just selling it in quantities to some manufacturers (think telecom, home entertainment, industrial equipment, etc).

Based upon my experience, I’d say it’s going to be profitable in its own right just by powering EC2 instances. The same argument could be said to apply to Apple’s ARM chips.

If something is possible and economical then others will copy it in time.

> the power profile is intellectually interesting, but not likely to ever enter the public sphere.

It does have an impact on operational cost, and cost is a very important factor.

I can think of exactly 0 circumstances where Amazon wouldn't pass on its power bill to its customers.

> presuming sustainable energy

Pretty big presumption.

Just a small reminder this is comparing a x86 on Intel 14nm+++++ and another one on TSMC 7nm. i.e the power efficiency would likely have absolutely nothing to do with the ISA in use.

From the comments section of the article:

> Because there are so many power sensitive applications where ARMs are used, much has been invested in power minimization and management and they do very well. It’s easy to get remarkably better power consumption with an ARM part. But, in this particular case, our focus was more on server-side price/performance and, with that focus, our power consumption isn’t really materially better the alternatives.

Hmm, so did Qualcomm miss out on a huge sales opportunity by bowing out? Or did they luck out not having to compete against vertical integration?

Any possibility of Graviton or ARM based computers hitting the main stream computing devices like Laptops aswell?

Microsoft and Qualcomm are selling ARM laptops.

Those are probably using a smartphone processors, the die-size alone of Graviton2 makes it desktop/server class.

Qualcomm 8cx cpu is based on smartphone one but optimized for laptop use.

The Neoverse N1 cores used in the Gravitron2 are very close architecturally to the Kryo 495 Gold cores in the 8cx. Of course, the 8cx only has 4 of them, while this has 64.

Would these chips be reasonable for something like an i3en class?

Will it be avail available on Fargate?

Reply on news.ycombinator.com