AMD's New Threadripper Chips Have a Hidden Fuse That Blows When Overclocking
extremetech.com
extremetech.com
https://www.servethehome.com/amd-psb-vendor-locks-epyc-cpus-...
You can see that economy on ebay. Those parts extend the life of existing boxes by years.
I'm typing this on a machine powered by parts from 'split' servers.
Refurb server parts market is more than alive, especially when a 5 year old server CPU is almost as good as a new one, but for $100 instead of $5000.
this used to be even bigger in the Xeon days. Being able to drop in a $50 upgrade to x99 and get the 2-3 best chip in socket owns actually. X99 isn't bad if you've got a 18-core chip running at 3.8 ghz all-core (with all-core turbo enhancement), and you can address 256GB of memory in a consumer board with dirt-cheap 32GB RDIMM sticks (they are basically under $20 now) since many X99 boards also support RDIMM (including on consumer cpus).
Like let's not cheer literal e-waste here. Secondary users keep hardware out of landfills, a lot of those customers have existing systems they want to upgrade or want to build using whitebox brands like supermicro/asrock rack, the EU absolutely needs to tamp down on this practice.
The most silly part is, how does locking it to the brand do anything anyway? If you locked the CPU to the board then sure, that makes sense, but the threat model here is an employee is stealing your CPUs and swapping them out with something cheaper, and this makes them... find another lenovo or dell branded CPU to match it with? and with PSP allowing secure manipulation of non-volatile storage, there is zero reason to make this a permanent fuse, why shouldn't you allow this to also be unlocked when it's time for disposal? like if you want to prevent the "evil maid" attack you would want it to be both more tightly locked to the board and also unlockable so it can be serviced.
(even if you don't want to trust non-volatile storage, you can also make "pseudo-non-volatile" out of plain old e-fuses. even number of fuses blown = unlocked, odd = locked. being able to lock and unlock say 128 times is plenty, in practice.)
And ironically the e-waste this produces (brand-locked chips) actually makes the cost of these evil-maid swapping attacks even cheaper, because nobody wants them on the secondary market, the cheapest platform-locked naples chips are probably already absolutely worthless because nobody wants naples to begin with, let alone brand-locked naples.
it so very obviously is narrowly targeted at making sure the secondary market is a confusing mess that buyers want to avoid. Creating e-waste to sell more chips. Again, the EU really needs to step in, it is noxious, they should even be forced to release a firmware update unlocking existing chips (AMD is the root of trust and can do this).
Sadly Intel has also really cracked down on xeon secondary sales as well - starting with broadwell, xeons can't be used in consumer Z97 boards, and they went to the same thing on X299, as well as cracking down on unlocked multipliers on certain skus (1660v3 has it, among others) and all-core turbo enhancement with broadwell-EP. It would be nice if you could drop an epyc into your threadripper board, too, that's basically the modern equivalent of ye olde xeon ebay upgrade. On the other hand, there’s no surprises either - socket compatible cpus are socket compatible, your board may only work with consumer chips but it does work with all of them.
And frankly people are also underplaying the fact that AMD does this on consumer socket as well - consumers would benefit from having a bunch of cheap zen2/zen3 chips from old OEM systems in a year or two here, that would push prices down even farther.
There is also a general danger that if the systems don’t have any resale value anymore that they will be scrapped entirely instead of resold as either parts or a complete server. If the value of the server is nothing, nobody will bother re-using it. You’ve literally turned working parts into e-waste, nobody wants them anymore and it stops being economically viable to bother for the handful of people who do. Everyone benefits from reduced friction in a market.
(I get your point though, I agree, it's very important for kids, especially in a world of increasingly walled gardens. A ton of the poorer world also makes do with whatever - X79 and X99 poverty builds are a real thing even today, because they're still a viable platform and china has cranked out cheap off-brand boards with resoldered chipsets. good workstation/server boards are <$200, and you can get a decent cpu for $50 or so, and memory for $20 a 32GB stick, it still is a relatively high-value thing for the capabilities it offers, if you need more than a normal consumer system.)
I was an early adopter and got bitten by it via a Lenovo.
https://www.youtube.com/watch?v=bFNJVaO9E-o&t=215
not only is this removing a cheap source of upgrades for consumers and sending it to e-waste instead, but it's going to pollute the consumer secondhand market in the same way, once AMD starts getting more traction in OEM sales.
just like with epyc, you will need to wade through piles of "HP ryzen" and "Dell ryzen" and find the one that didn't list it as locked. Oops, still locked anyway, that seller just didn't list it properly.
people are gonna wake up one day and realize the secondhand market is completely fuckered, OEMs sell a lot more PCs to businesses than consumers, there is a disproportionate amount of volume that's going to be locked when things start getting parted out. Again, "fortunately" AMD didn't have much traction with OEMs during the AM4 years but AM5 is going to be a real mess.
If they did such a thing with car components, it would be shut down in no time. No different for computer parts.
Also a CPU that can be rendered permanently inoperable due to a software issue could be considered not fit for purpose and thus a defective product. Especially if it was intentionally designed this way - such that the CPU can self-cripple itself.
Maybe a legal firm would want to take up a class action against AMD for this potential product defect? Viruses and malware should not be able to physically destroy a CPU. There should be physical hardware protections against such destructive behaviour occurring due to a software induced logic error.
The PSP is not truly secure, someone should write a PSP eFuse bricker tool (Ryzenkill?) that can be run as a proof of concept to show the CPUs are defective, and then we can have a recall or free replacement of the CPU by AMD. I'll look into it myself - I need to ensure it's legal for me to write it from the country that I'm residing in right now. I personally have extensive experience hacking older AMD processors and GPUs through JTAG.
I'm sure it's still legal to provide a list of addresses and data writes that you need to perform to permanently brick the CPU?. I don't think that is classed as a "hacking tool". I can get t-shirts printed with it on, together with the words "AMD Ryzen CPU Self-Destruct Sequence". I don't think selling those would be illegal?
https://consolemods.org/wiki/Xbox_360:Disabling_the_eFuse_Bu... (see Method 2)
I fondly remember the Core 2 Duo days when I overclocked my CPU with more than 50% using the stock cooler. It's nice to 50% performance for free, particularly when you are poor.
I even remember overclocking an old laptop with K6-II (those days laptop CPUs were socketed) by around 15%.
Downclocking is literally throttling. But there's like another 10% performance that can be unlocked if you'd go full custom watercooling.
It's actually less than 10%. I'd say closer to around 5% - as a custom watercooling enthusiast. 2-3% gains going from a great dual tower air cooler to an All-in-One closed loop (but extra noise from the pump) and another 2-3% gains for a great CPU waterblock and thicker, denser radiators of custom watercooling (and least noise of all, assuming you choose an appropriate pump speed).
5% performance gains, and 500% price increase <:o)
But also, whether a chip is actually stable at settings X or Y has become a lot harder to determine due to complexity: multicore architectures, frequency scaling and power management in general make it hard to predict how a chip will behave under real conditions. It has become unrealistic for a user to test all cores under all possible combinations of workloads. Consequently, and from anecdotal experience at least, many overclocked systems seem a lot less stable than their owners claim.
There are many good software for stress testing CPUs and GPUs. Just run AIda64 or OCCP stress tests for a long time and you are good to go if no problems arise. If there are problems, you redo the settings and run the tests again.
The undervolting makes the chip run cooler, which in turn means it can stay at higher frequencies longer.
If you have a higher binned CPU then you can reduce that voltage and still be stable at a given clock, the chip will then be able to boost higher because there will be more thermal headroom.
Also, on AMD there is an extremely widespread culture of overclocking the fabric which again, can result in degradation over time (progressively lower clocks at a given voltage, or instability at a fixed clock). People very much still believe in the fairy-tale of “24/7 safe” overclocking - obviously the manufacturer has every incentive to certify the chip as fast as it’ll go, they set the limit at the 24/7 safe point already, but the gains are so potent that people tell themselves whatever they need to hear.
https://www.amd.com/en/legal/claims/gaming-details.html
AMD and intel have been very very clear for a very long time that this is not covered by warranty (see AMD GD-106/GD-112) but people have low key continued to do it and warranty the CPUs when they fail, because there wasn’t a way for AMD and intel to tell. And to be fair - AMD and intel have often muddled the waters by using these numbers in their benchmarks etc. But there remains a general sense of “if you didn’t turn on XMP you didn’t do the build right” and “XMP can’t hurt anything as long as you don’t increase the voltages” despite this already being done automatically by the board.
XMP absolutely does cause damage and I killed a 9900k with no other overclocking simply by enabling XMP on a fast kit. But consumers still don’t get the memo, they think it is only a problem if you instantly blow up the chip like the 7000 series at launch. XMP is widely viewed as safe and even something you're supposed to be doing.
Electromigration happens very quickly at 7nm and 5nm nodes, and people are not used to it, and it’s low key also a thing on 14nm and 10nm class too. But it is already something that CPUs have to be designed around and designed to detect and monitor and tolerate as part of the boost algorithm (that stability margin is also headroom the boost algorithm wants to exploit), they devote an enormous amount of effort to electromigration already. And the mismatch between that public understanding of the problem and the reality is causing a lot of warranty claims. You can see from enterprise fleets that CPUs don’t normally fail, yet every enthusiast knows someone who had a chip die “even without overclocking! I just enabled XMP!”. Of the failed CPUs getting returned for warranty, probably a plurality are memory overclocking failures. And this is only going to continue to get worse over time - 2.5D and 3D packaging are both progressively even more sensitive due to thermals, and have tighter requirements for voltage stability since they need to talk to the other dies at very low voltages (especially across heterogeneous dies with different metal stacks/totally different processes etc).
https://semiengineering.com/3d-ic-reliability-degrades-with-...
https://semiengineering.com/on-chip-power-distribution-model...
From AMD's language, and common sense, the thoughts on this impacting warranty are misguided. I suspect this is a normal feature designed for debugging.
If I am AMD, and I e.g. receive 20,000 defective / failed chips returned from a bad run, or am sampling failures to just improve reliability, I want to know how those devices failed.
If, what's failing, are overclocked chips, I probably do want to lock down chips, void the warranty, or similar. If what's failing is a distribution of chips representing real-world use, I want to address the defect. In either case, if doing an RCA to understand the defect, I want to know how the chip was used. Reporting of maximum temperature, voltage, overclocking, etc. (all the data HDD keep in SMART) is super-helpful for that sort of analysis.
Also: Voiding warranty for overclockers would have been dumb. I've never overclocked, but for anyone here running a business: Vocal thought leaders are important to your brand. Overclocking may be out-of-spec, but if you want the word on the street to be that your chips are awesome, they had darned well be singing your praises. I've never overclocked, but I've cost both Google and AMD millions of dollars of business:
* I used to do business with Google before they broke my personal account, wiped out data for a startup I was working with, and broke / discontinued several services I rely on. I'm well enough known that me going around saying "Don't build a business on Google" (and ditto for many other folks on HN) means that GCP is starting a 100 meter dash with Azure and AWS about a hundred extra meters back.
* I bought an AMD GPU for machine learning since I liked open-source. It didn't work, and AMD discontinued support a few months after I bought it. AMD wouldn't provide warranty support for lack of suitability to advertised purpose. I learned my lesson, and tools I'm building (which will likely have broad-based adoption) don't support AMD.
There are customers you want to treat like a restaurant treats a food critic. It's not about them being profitable directly, but they're super-important.
There's even worse possibilities, I think some hardware has ~1KB of executable OTP which can be programmed in that manner - so it may just be possible to implement some kind of backdoor that resides in the physical CPU. Maybe something only a state-level attacker could do.
For example, it could prevent CPU side-channel mitigations from ever working, somehow, e.g. by repeatedly writing to a private internal register to stealthily keep the CPU vulnerable. And few would ever know about it, because it's hidden in an on-chip security processor.
Personally I utterly hate all AMD chips which have a Platform Security Processor. Locking down your own property to prevent you from accessing it should be illegal.
There is a full Trustronic Trusted Execution Environment running in there, on both GPU and CPUs, I believe. For example the TEE firmware blob for a radeon GPU on Linux is "/lib/firmware/amdgpu/psp_13_0_7_sos.bin", with "sos" meaning Secure Operating System. And some of these firmware files are encrypted, so you can't reverse engineer them.
The moment some ARM SoC company such as Rockchip comes up with a chip that's within an order of magnitude in performance, then my AMD chip (EPYC) is going in the bin. After being smashed to pieces with a hammer, live on YouTube, with an explaination why. That might get AMD marketing to pay attention.
Update: Ampere Altra CPUs have seperate power pins for blowing eFuses, you can find it in the public datasheet here[1], on page 55, the supply pins are EFUSE_MFG_VDDQ1P8 and EFUSE_PCP_VDDQ1P8. It says tie to GND if unneeded.
Also they have a public datasheet for the chip. I wonder if we can buy unfused CPUs without secure boot enabled? Or is that the default state of the chip?
Could someone design a simple and cheap open-source motherboard for one of these processors, it's only a matter of time before the chips start turning up on Ebay? We will need more documentation from Ampere than just the datasheet, of course.
1. https://uawartifacts.blob.core.windows.net/upload-files/Altr...
And I fear that even then, lobbyists and/or intelligence agencies will neuter any response.
Shoot, now I'm just making myself depressed.
It was AMD with their PSP that pushed me into FPGAs and creating open source hardware, now that everything is getting locked down.
With the smaller process sizes there's less voltage tolerance before degradation occurs. The X3D chips from AMD pack things supposedly pack the cache so tightly that it's easy to get into trouble
A lot of chips burned out with something relatively mild like 1.3V on the SOC!
That protection is fairly limited. There aren't enough sensors for everything that's heat sensitive. Reducing voltages/clock speeds also may not be enough to save all of those.
Very long way to say: not that common... but decently.
edit: As Paul mentions in a peer comment, this is largely due to the memory controller being moved onto the CPU.
I had a 9900K fail due to XMP, no other overclocking.
>Overclocking and/or undervolting AMD processors and memory, including without limitation, altering clock frequencies / multipliers or memory timing / voltage, to operate outside of AMD’s published specifications will void any applicable AMD product warranty, even when enabled via AMD hardware and/or software.
From: https://www.amd.com/en/technologies/expo
The moral of this story is overclocking is not for the naive nor the ignorant, and you must be willing to write off any hardware subjected to overclocking.
Point being that these e-fuses don't blow up randomly by themselves.
https://support.xilinx.com/s/question/0D52E00006hpKdKSAU/zyn...
I understand burning eFuses is necessary during production, but having them blowable when the device is operating normally could be compared to building disposable devices. One logic or software error and the device can be bricked permanently. That is a ridiculous state of affairs. What about single event upsets or some EMI induced glitch causing eFuse corruption?
The power for burning the important fuses should be off at all times during normal operation. And we can have a test pad to supply the VPP during production.
In the case of the AMD processors, a separate bank of fuses can be used to store diagnostic data for warranty returns, if necessary. If those get corrupted somehow, the device won't malfunction. But the important ones should not be possible to program after manufacturing, no matter what happens to the logical state of the hardware, period.
There are so many other dominating factors that it is almost pointless to worry about per core frequency.
Memory bandwidth is probably a bigger constraint for many practical applications. Faster core clocks doesn't do anything to resolve that.
* on
* idling
* at room temperature
Current passing through conductors wears them down. More current or heat means faster.
What's more, there aren't enough sensors (or room to put them) for every heat-sensitive component.
for the price of this new Threadripper ($9,999) I wouldn't take the risk of overclocking it..
Yes? The main point of the Threadripper (especially non-PRO) is that you have a many-core CPU that can still run at high frequencies. Othervise EPYC (many-core) or Ryzen (high speeds) would be enough.
Also shit move from AMD because even if you overclock with their software, the warranty still becomes void
If I'm a home freelancer staking my livelihood on this workstation, I'm not taking the risk.