Comment by gjsman-1000 | Hacker News Reader

Comment by gjsman-1000 | Hacker News Reader

Parent Full thread

I don't know how much power efficiency would be saved - my concern is more that it is completely logical that running any part at higher temperatures causes increased risk of failure, whether it be a computer part or a mechanical part. How much? I don't know - I just don't blame JEDEC for recognizing this is not a clear and obvious win.

Imagine if the failure rate was raised by as little as 1%. RAM Failure is not uncommon compared to other parts - I've had it happen before and render a system unable to boot, that's why we have Memtest86 and not CPUtest86 or SSDtest86. A 1% increase in failure over 5 years would have effects just as unbelievable as the power saving that increasing the temperature would be. How many smartphones would be junked? How many PCs would be thrown out for not working by people who are average Joes who can't diagnose them, and the extra waste that generates from both disposing the old PC and purchasing a new one? Perhaps the new PC is more efficient, but which is better, greater emissions or more eWaste in the ground due to the new PC being likely more efficient than the old one?

The point is that it is not a clear win. With further research it might be, and I might be all for it. I'm only nitpicking the description of it as being a "compromise" as though it were obvious.

[@picture: I'm at my posting limit for the day because HN is, well... I'll leave their censorship policies for another day. I would agree with you if the RAM with the 90C limit were strictly ECC RAM because that is most often used in data centers and not consumer parts. Maybe we have non-ECC/85 RAM and ECC/90 RAM options...]

Most datacenter hardware is fine at 95 degrees F (inlet temp). Approved configurations are usually available to 105 degrees F or slightly higher. Some devices can run as high as 130F.

In the operating range, you’re not going to have any measurable change in operations or failure rate - if you do the parts are defective. All of the stories you hear about this and that are conjecture.

Interestingly, computer chips can often be run at lower voltage and wattage for a given frequency if they are kept at a colder temperature. As a home user I can significantly reduce power draw for a CPU/GPU by improving the cooling solution and lowering voltages.

The reasons this doesn't work for datacenters is two-fold, I think: First, they won't see efficiency improvements just by keeping their CPUs and GPUs (or RAM) cooler because the power levels/tables for the chips are baked-in, and operators aren't going to the trouble of tweaking voltages themselves. Second, even if they did tweak voltages, the cost of sustaining lower temperatures with better cooling likely won't outweigh the savings resulting from lower power draw for the chips.

Still, this raises the question of whether designing hardware for higher operating temperatures is always the right move. At some point there's going to be a cost in performance and/or efficiency that outweighs the savings from allowing higher temperatures. Ideally these tradeoffs should be balanced as a whole.

I think you missed the biggest reason this doesn't do much for servers - they already run at low frequencies & voltages.

For example take the Epyc 7742, at 225W it sounds super power hungry. But the 64-core chip only boosts to 3.4ghz max (2.25ghz base). That's less than the base clock of almost any of the Ryzen consumer CPUs. And if you look at the lower frequency end of https://images.anandtech.com/doci/16214/PerCore-1-5950X.png there's not a whole heck of a lot of efficiency gains likely to be had below that ~3-3.4ghz mark. They're already basically sipping power at something like 3w per CPU core or less. 225w / 64c = 3.5w/c, but the IO uncore isn't exactly cheap to run and iirc sits more like in the 50-70w range. So subtract that out and you're at more like 2.5-2.7w/c. I don't think throwing cooling at this is really going to get you much of a gain.

This usually isn't true at scale though. The chip manufacturers do a ton of validation and qualification to set the operating parameters (voltage, etc).

You can undervolt (or overclock) one specific chip, individuals have been doing this at home for basically ever, but there's (almost) always a system-specific validation process you then do to make sure the system is stable for a specific workload with the new parameters.

And these parameters differ between batches of chips, or even between chips within a batch.

It's also significantly harder to drastically reduce the temperature of the chips inside of a single server, given the machine density of a typical data center.

> It's also significantly harder to drastically reduce the temperature of the chips inside of a single server, given the machine density of a typical data center.

It’d be fun, however, if we could dynamically adjust that according to workload. If workload is light, you could consolidate load into fewer sockets/memory sticks and power down everything in that socket.

It’s been a while since I was doing hardware in a DC, but even dynamic frequency changes in servers were noticeable in workloads.

We’d see serious latency spikes with high volume increases. And high volume would correlate to high I/O on shared storage exacerbating the problem. You’d get a bunch of transactional failures from the spontaneous latency hit.

Same was true with older virtualization/partitioning of hardware. You’d even see apps back up into memory and swap themselves to death.

Now, the apps were awful: inefficient and intolerant. But that’s the reality for the vast majority of business software. Churned out by devs with little understanding and tight timelines for features.

Reliability being an NFR kills me.

It’s an interesting thought. I think in the datacenter today it makes more sense to evacuate and shutdown servers vs. tune down the devices.

That's for sure. I 100% agree that increased temperature will statistically increase failure rate. I'm just thinking that, the most common mechanisms of thermal failure in electronics are caused by repeated thermal cycling which cause fatigue and stress failures at interconnects (solder bumps, silicon bonding, etc). Data centers are designed to be operated in a relatively very constant temperature environment, so I would suspect that the failure rate may not be raised significantly.

Well now you're just being hyperbolic. As you say, this is an engineering problem so solutions are far from their ideals states in either direction. However, 1% increase in ram failure rates ruining the world? That doesn't sound right. Errors are encountered in RAM all the time and guess what, they're corrected often by the hardware before even bothering the system. I'm sure we could deal with a 1% increase...

Reply on news.ycombinator.com