Hacker News Reader

Comment by tmd83 | Hacker News Reader

The security implications of this as commented on the article seems pretty bad. Wonder if any microcode fix can work or it would need a silicon level fix. The second I presume would be very bad?

-- Keeping my earlier comment but it seems there's a microcode fix in testing currently. http://forum.hwbot.org/showpost.php?p=480922&postcount=30

View on HN

AMD already has a fix, it requires a BIOS update for the motherboard. They have not given details, but some experts think the fix is merely an increase in the power delivered to that part of the chip. Their conclusion stems from teh fact that the FMA3 bug disappeared when the chip was over-clocked because the over-clocking required increasing the voltage to the chip. So the BIOS update probably adjust the voltages up a bit.

http://www.fudzilla.com/news/processors/43166-amd-confirms-r...

A power issue would also plausibly explain why disabling SMT avoids the problem, since disabling SMT powers off a bunch of stuff (a whole thread context does need a bit of power), and, in a multi-threaded scenario, tends to reduce core usage thereby reducing average (! so likely not relevant) power draw as well.

And (probably more importantly) reduces overall IPC by around 30% generally. The whole point of SMT these days is to have continuous work for the ALUs despite recent cache misses.

Since they are pushing how efficiently the chip are regulating power to various parts of the chip to keep power draw to a minimum it sounds like a likely cause.

Assuming most of the testing budget is spent on the most common and likely instruction sequence patterns is probably not unrealistic. Still a bit odd that such a short sequence of repeats of the same instruction failed.

Which got me thinking.

At more than 2000 opcodes for x86-64 and legacy opcodes, testing all 3-sequences with one input is more than 8G sequences, to cover that and (guesstimated average) 64 bits of input per opcode is a staggering 10^29 number of instructions, or 2T CPU*year at 4Gops/s. Not easy to test all that, even if it would be off by a couple of magnitudes in the right direction!

Might be that valid 2-sequences is the most that can hope to be even close to exhaustively tested while at the same time covering some significant fraction of the operand space ?

The fact that modern CPU's work at all given their complexity in so many different areas looks pretty darn close to magic, even from a pretty close distance.

At least we try, with SMT solvers and such. Surprise surprise, math works!

If it's not fixable in microcode, then yeah, I'd say it's pretty bad. Makes it useless as a cloud machine, if anyone can cause a DOS and take out every other VM on a host.

For a personal machine, it's probably not terrible if you are using linux or something else where you can compile everything yourself. But running binaries built by someone else would be a crapshoot. I wonder how many games (of the Windows, AAA variety) use these multiply instructions?

There's also not a lot of detail in the article—like if the data has to be specific or just the combination of instructions is enough to crash it.

Cloud doesn't have to mean virtualised and shared. There are plenty of operations that run on dedicated hardware.

For many cloud vendors, including EC2, "dedicated hardware" just means yours will be the only virtual machine on that computer. But they're still running your image under a hypervisor.

Yes but if you're worried about attacks from co-tenants, that's OK (the cloud provider can always access your hardware anyway)

It would still be a pain for cloud providers if the tenant can unwillingly DoS the hypervisor. Sure they have watchdogs in place, but it gets more complicated if the DoS can be non-malicious.

I suspect Amazon (for instance) has far more shared machines than dedicated machines. If something isn't viable for shared machines then a huge section of the market is ripped away. Still seems catastrophic to me.

> Flops is only affected when the SMT is enabled, so disabling the SMT can be used as a temporary work-around (until the actual fix arrives).

Does this mean disabling SMT will also fix the bug, or is that specific to this app?

I've heard that some things improve performance on Ryzen with SMT off, but I've heard that's because OS level task schedulers need to be optimized better. But I still wonder if Ryzen's SMT implementation is on par with Intel's first implementations.

It's happened to AMD before, and it was pretty bad for their brand.

http://www.anandtech.com/show/2477/2

Intel had serious problems as well, let's not forget: the Pentium fp bug and the F00F bug (though those are more than a decade old at this point)

Haswell and early Broadwell had a bug in its transactional memory extensions so serious that the microcode "fix" simply permanently disabled the instructions.

CPU errata like this aren't uncommon.

While those are the most remembered bugs, almost every CPU have some kinda of CPU errata that can lock up the system in really weird and specific ways. For example, my Intel Ivy Bridge CPU running Linux:

$ dmesg | grep -i microcode

[ 0.000000] microcode: microcode updated early to revision 0x1c, date = 2015-02-26

[ 1.415548] microcode: sig=0x306a9, pf=0x10, revision=0x1c

[ 1.415665] microcode: Microcode Update Driver: v2.2.

The fact that most users does not hit those bugs is because modern OS already patches those microcode before execution (like the case above).

Of course, if AMD can't fixes those bugs without performance regressions (remember the infamous TLB Bug from earlier Phenoms?) it can be pretty bad. However I don't think the majority of users needs to be too cautious about it.

Some of which are quite serious: http://yuhongbao.blogspot.ca/2015/06/why-your-core-2-process...

Don't forget the (other) f00f bug in the Intel Quark![1]

[1] - https://en.wikipedia.org/wiki/Intel_Quark#Segfault_bug

Yeah, that was kind of unfortunate as it made the Intel Quark incompatible with all normal Linux distros. I don't think it even had any kind of microcode update facility, being effectively a slightly updated 486.

Even with a runtime microcode update facility, no BIOS update equivalent would still make it pretty fatal for packaging except as an entirely distinct build target, because otherwise you'd need to do something kinky like always first booting a custom kernel+initrd target built to involve no locks, load the microcode update, then kexec into the real kernel.

I don't know if this bug affects Phenom II chips, but I've been running a Phenom II for 8 straight years in my primary desktop with nary a hiccup. Rock solid reliable. Guess I haven't been running the "correct conditions" for a crash?

You could also have a BIOS update with the workaround already baked in, or a chip that has the later silicon without the bug...

(I also had a Phenom II for many years without any mysterious crashing problems, even though the chip was the correct revision and there was no BIOS erratum workaround enabled.)

Windows 7 and later distribute and activate microcode updates through Windows Update.

It seems like it's not a microcode update fix, but an MSR killbit flip from the BIOS? [1]

[1] - http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf page 66

Yes, which reduces performance if I remember correctly. It was quite famous. They eventually released a fixed stepping, and Phenom II is not affected.

True, my mistake.

(While it doesn't apply for this specific example, it's still the reason why one generally will see microcode-fixed errata go away rather quickly, independently of BIOS updates)

Phenom II is not affected.

Reply on news.ycombinator.com