Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian | Hacker News Reader

Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian

anandtech.com

119 pointstambourine_man40 comments

The best part of POWER8 is the ability to flash and verify the CPU firmware using a completely open toolchain. All modern Intel and AMD processors have secret sauce signed firmware (that can even access and use the network). You can't ever be sure what is running on them internally. POWER8 is a secure architecture even in the case of an NSA targeted hardware intercept during shipping. You can just reflash with known/verified good firmware when you get it.

How does this work at the lowest level? How does one know that the flashing itself is not being subverted or altered by the microcode?

There has to be a level below all of the turtles, right?

You can probably hide evil in a few gates, in the right place. A little additional logic that perturbs an MMU mapping or TLB load, or fail to register an exception if a bit of hardware has seen a recent pattern of behavior. Heck, with a few thousand gates you could have a full microprocessor embedded on the sly.

I don't know at what point this stuff becomes noticeable; do you subvert the silicon toolchain? Do you buy off the people who make the masks? Are there feedback loops in the silicon production chain where changes would make it back to a designer, who would say "Hmmm, that's funny..."?

In the early 80s it looked possible for a chip designer to stick in Something Bad without much notice. With billion transistor chips, it seems a lot easier.

Is there a VHDL equivalent of the Underhanded C contest? :-)

How about a single capacitor that isn't even connected to anything? https://news.ycombinator.com/item?id=11768980

I don't know how POWER8 sets it up, but it's not particularly hard. You either use an external flash chip, or you set up internal flash so that it can be directly accessed by an external programmer. Then the microcode can't interfere with flashing.

This doesn't save you from someone that can custom-fabricate alterations to the CPU, but honestly nothing will save you from that.

An external programmer would be connected to some black box though, right? I mean, how do you know the interface the programmer is writing through isn't compromised inside the CPU?

I'm not trying to be pedantic; I'm simply unsure that the original comment of "The best part of POWER8 is the ability to flash and verify the CPU firmware using a completely open toolchain." and "[For Intel and others] you can't ever be sure what is running on them internally."

How does one know that with POWER8 the same isn't true? Just because you think you can flash it "directly" doesn't seem good enough to warrant that statement.

(Disclaimer: I have connections to both processors; my opinion on which is "better" or "more open" or "less likely to be compromised" is not stated here, and should not be inferred from this discussion. I'm merely trying to point out that I don't think the original comment's assertion is backed up by the reasoning it presented)

> I mean, how do you know the interface the programmer is writing through isn't compromised inside the CPU?

Inside which part? I suppose if you're worried about compromises inside the casing, you need to take off the heat spreader and look for yourself.

Anything on-die falls under the custom fabrications exception.

Forget custom fabrication; how does one know the real, official fabrication isn't compromised? I mean, the original comment was comparing the openness of POWER8 to the closeness of other processors, using only the ability to flash it openly as evidence, but if one is worried about other processors delivering compromised processors, why isn't one also worried about POWER8 delivering a compromised processor?

I guess I don't understand how that's an argument to support "one can't ever be sure what is running on [the others] internally", unless you manufactured the processor yourself from the open design.

Got a link to the source code of the POWER8 CPU firmware?

https://github.com/open-power/skiboot (runtime firmware)

https://github.com/open-power/hostboot (initialisation firmware)

https://github.com/open-power/occ (on-chip controller firmware)

https://github.com/openbmc (BMC - currently in development!)

[Disclaimer: IBMer, opinions my own.]

> Once all threads are active, the IBM POWER8 core is able to outperform the Intel CPU by 41% (geomean average).

Nice! I see good multi-threaded performance. Can use regular memory in it as well. Also someone mentioned already I see but I understand POWER8 doesn't come with something equivalent to Intel Management (ME) -- the friendly backdoor inside your server <insert Intel Inside joke here>

Yes, that is why for the Talos workstation Power8 was chosen: https://www.raptorengineering.com/TALOS/prerelease.php https://www.raptorengineering.com/TALOS/prerelease_specs.php

"POWER is the only open, owner-controllable architecture that is competitive in performance."

If you are interested in using anything other than Intel ME/AMD PSP "backdoored" desktop, please register your interest, they will only build Talos if enough people want to order it.

I would be interested, but even their minimal configuration is overkill for me. If there's a 50-70W quadcore option, sign me up.

For anyone not familiar with ME, it has been around for a long time. It used to live on network cards and then migrated to the motherboard.

http://www.slideshare.net/codeblue_jp/igor-skochinsky-enpub

It has a built-in Java VM !

And a web server, controllable by HTTP and SOAP sitting on top

It has a BMC and On-Chip Controller (OCC), but the firmware for both is open source.

Got a link for those?

There will be a talk about the (Open)BMC firmware: https://lcccna2016.sched.org/event/7JWX/openbmc-a-new-secure...

Edit: remove dup links

Talk at linux.conf.au 2016 on the broader view of the OpenPOWER firmware stack: https://www.youtube.com/watch?v=a4XGvssR-ag

I think most of them have things like IPMI already anyway and admins often uses them.

$4,900 for a barebones system is still too high for a platform meant to compete with a PC tin bucket server, where 1U barebones start at $800, and a decent 32GB / 4TB server can be had for $1,800.

Next on the list, if I don't want to run Linux (and I don't) what other freeware choices do I have? As far as I know only beta versions of AIX were gratis, everything else had a hefty licensing fee. Those kinds of games don't fly in this day and age, hence anything that is not intel and freeware and open source has been relegated to dwindling sales, and close to non-existent market uptake. itanium, POWER, UltraSPARC - examples aplenty.

When will AIX, the volume manager, and the compilers become freeware / open source, or at the very least freeware?

Do you have any specific preference for what operating system you'd want to run? Linux seems to be what most people are going to run on it, though the devices do also run AIX (according to Wikipedia)

Have you looked into the various BSD's to see if they support it (yet?) I used them a couple years ago on some old Apple PPC hardware I had clunking around as they ended up running circles around Linux in hardware support and stability

(Disclaimer: PPC Yellowdog Linux on PlayStation 3 used to be my primary development system.)

I'd prefer SmartOS, but that only works on intel, as PPC support was removed from the illumos kernel a long, long time ago. Nevertheless, reverting those changesets and getting it running again would be cool.

Next on the list, then, would be AIX, an operating system developed on POWER for POWER.

I also read that FreeBSD has some POWER support, but unlike AIX, it's nowhere near production quality material yet. FreeBSD would have been my next choice.

I believe FreeBSD also run on POWER8.

IBM is pushing hyperledger / blockchain on Power8 pretty hard.

Given how much memory bandwidth is a bottleneck on updating the internal Merkle trees, Power8 might be good for tx throughput

IBM are also providing hardware and engineering to get Cloud Foundry running on POWER8.

I work for Pivotal, we donate the majority of engineering on CF, but IBM is the next biggest contributor. They've started assigning engineers to work alongside multiple teams on porting to POWER8 and have started setting up test clusters as well.

Just this week I was in a remote tri-pairing session with an IBM engineer working on getting our buildpacks pipelines[0] running on POWER8.

I personally can't wait to see how it performs.

[0] https://github.com/cloudfoundry/buildpacks-ci

But what is performance per buck?

I'm still waiting for cheap(ish) POWER8 development boards. Any news on that?

Depends how you define cheapish. Raptor are building a dev board, estimated price is $3700: https://www.raptorengineering.com/TALOS/prerelease.php

Somewhere in the $500-1000 range would be cheap enough that I could buy one and expense it.

Their estimated price for the board without the CPU is $2700, indicating that they’re expecting a minimum price for $1000 for the CPU alone.

IBM POWER CPUs are really expensive because their yields must be very small. However there are other sources of POWER CPUs. POWER CPUs aren't intrinsically expensive.

Freescale are one (eg. the e5500 - I can't keep up with whether that exact model was POWER7, but last I checked Freescale did a POWER7 CPU). These are much cheaper. But the problem with them is they are embedded CPUs (book E?), so they omit some instructions required to run server versions of Linux. Also IIRC they are all big endian, or at least had some problem running in LE mode.

Anyway to get to my point: unless there are affordable development boards for POWER, no one is going to spend much time porting software and fixing bugs for POWER.

Does POWER have an advantage in price-performance ratio over Intel? When I say price, it's the TCO, including energy costs.

My take on this is that if you need peak single thread performance and are willing to pay whatever that costs you should consider POWER. If you are at all sensitive about price, or if you can expend development effort parallelizing your work across lower performance cores, you should go with x86. They are different markets for different people.

I have access to POWER8 hardware through work and it really does scream. Forget benchmarks, it's noticeable at the command line. Which is nice, but I'm not paying for it.

> Forget benchmarks, it's noticeable at the command line.

What do you mean exactly?

Well I compile libguestfs (http://libguestfs.org/) on POWER8 fairly regularly and the compilations just fly past. I haven't benchmarked it, and I'm not allowed to benchmark it because of the NDAs we sign, but it's obviously faster than the Intel machines we have. Because of the single-threaded parts of the build (eg. ./configure), these builds benefit much more than just having lots of cores - Amdahl's law and all that.

Like the article says, it depends on your workload. With this machine and if you’re expecting idle time then no. If they’re under constant load, and given the right type of load, then yes.

I would love to see the same benchmark with --march=native. I doubt AVX would be used without this flag on the Intel platform, and this might be also hindering the Power8.

FWIW RHEL 7.3 (ppc64le) will be compiled for POWER8 only. As I discovered the other day, binaries will randomly segfault if you try to run them on POWER7.

Assuming they are similar price, POWER8 is roughly 40% better, double the Idle power, 20% slower Single Thread Performance.

So... why would I want to buy that? I believe at 14nm, Google would likely make the switch as their scale is different. ( Any % of improvement is a lot in Google Scale )

Summary: it's all about the bandwidth.

Reply on news.ycombinator.com