Hacker News Reader

Comment by throw0101c | Hacker News Reader

Meta: Oxide has talked about the designs of their cooling [1][2][3], so I'm curious to know if they ever start offering GPUs how they'd handle that.

Folks seem to be moving toward liquid cooling[4] either to the rack/chassis[5] or even to the chip[6].

[1] https://oxide.computer/blog/how-oxide-cuts-data-center-power...

[2] https://www.youtube.com/watch?v=4vVXClXVuzE

[3] https://www.youtube.com/shorts/hTJYY_Y1H9Q

[4] https://blogs.nvidia.com/blog/blackwell-platform-water-effic...

[5] https://datacentremagazine.com/data-centres/top-10-liquid-co...

[6] https://zutacore.com

View on HN

Oxide's blogs talk about cooling _a lot_, apparently their stack runs very cool b/c they've reorganized the whole thing around efficiency, and written all the firmware to support that goal.

Skeptical of that. There's only so much you can do against the physics of moving electrons around at high speeds... "Bigger Fans" and "compute density" doesn't change that

I believe the fans are actually smaller. The rack is definitely quieter than other racks, but he says in the rear rack tour that it's quite hot. Check out these videos of it

[0] - https://www.youtube.com/watch?v=dHbgjB0RQ1s [1] - https://www.youtube.com/watch?v=lJmw9OICH-4

The fans are 80mm, traditionally 1U is roughly half that size.

Commodity hardware doesn't quite tend to operate at a compute vs efficiency Pareto frontier — there's a lot of wasted energy that we've been able to optimize with our vertical integration. (I work at Oxide.)

I don't have any particular knowledge about oxide's cooling, but think about how bloated and inefficient literally every part of the compute stack is from metal to seeing these words on a screen. If you imagine fixing every part of it to be efficient top to bottom, I think you'll agree that we're not even in the same galaxy as the physical limitations of moving electrons around at high speeds.

But the majority of heat is going to come from the CPU and this is a product to run arbitrary customer workloads.

If the customers leave these things idle, then oxide is going to shine. But a busy rack is going to be dominated by CPU heat.

Their rack scale from-scratch redesign includes fans big enough that they've reportedly managed to cool CPU hardware that was actually designed for water-cooling, with no expectation for air cooling (though admittedly, they say they only achieved this just barely, and with a LOT of noise). That seems like something that's going to be objectively verifiable as a step up in efficiency.

According to Oxide Computer, they found that going from 20mm to 80mm fans dropped their chassis power usage (efficiency is to the cube of the radius): a rack full of 1U servers had 25% of its power going to the fans, and they were able to get down to 1.2%:

* https://www.youtube.com/watch?v=hTJYY_Y1H9Q

From their weblog post:

> Compared to a popular rackmount server vendor, Oxide is able to fill our specialized racks with 32 AMD Milan sleds and highly-available network switches using less than 15kW per rack, doubling the compute density in a typical data center. With just 16 of the alternative 1U servers and equivalent network switches, over 16kW of power is required per rack, leading to only 1,024 CPU cores vs Oxide’s 2,048.

* https://oxide.computer/blog/how-oxide-cuts-data-center-power...

This is not any different than the "blade" form factor that was popular in the 90s. Shared power and cooling that was not constrained by the height of a 1U rack chassis, with larger fans. Hell, even Supermicro has blade-style chassis with 80mm fans. This is not novel.

It's just plain old engineering, optimized to sell whole racks not individual servers or <=8U units, sprinkled with opinions about low-level firmware etc, with a bespoke OS and management stack.

> It's just plain old engineering, optimized to sell whole racks not individual servers or <=8U units, sprinkled with opinions about low-level firmware etc, with a bespoke OS and management stack.

Yes, "just".

Yes, we're primarily an engineering company, not a research organization.

It's also about what we don't have. We don't have a UEFI, for example, which means we don't have UEFI vulnerabilities.

Yeah and you're doing good work there. It just kinda annoys me when people go from "oh that's a cool company" into idolatry. 1U servers were always a poor form factor for modern day hot chips & drives. Breaking that mold has been done over and over and isn't something that should be treated as new.

Scaling from the 8U (that blades could already do in the 90s) to full rack as the unit of "slide unit in to connect" DC power and networking is way cooler than using 80mm fans.

Re UEFI: I feel like that part is less about UEFI itself and more about how you have very minimal third party firmware.

I'm pretty excited about openSIL and such in general. If only AMD could execute well in the world of software.

I can't speak to others' views, but having worked with large-scale bare-metal deployments at Meta, I personally admired Oxide for its clear product vision and rigorous first-principles approach (Rust is a real game-changer!), and applied to work here for that reason.

An F1 car is also just plain old engineering, optimized to get around the track quickly, sprinkled with opinions and with a niche bespoke drivetrain. Nothing to see here.

20mm fans aren’t used in server cooling applications. You must be thinking of 40mm fans.

Going from 40mm fans to 80mm fans will not take energy usage from 25% to 1-2%. They must have taken an extreme example to compare against. What they’re doing is cool, but this is a marketing exaggeration targeted at people who aren’t familiar with the space.

Oxide also isn’t the only vendor using form factors other than 1U or focusing on high density configurations. Using DC power distribution is also an increasingly common technique.

To be honest, a lot of this feels like Apple-esque marketing where they show incredible performance improvements, but the baseline used is something arbitrary.

Our claim is not that just switching fans drops from 25% to 1-2%. We are claiming that the rack has very low energy usage, and we like to talk about the fans as one part of that reason because it's very visceral and easy to understand.

I think 1U was poorly optimized for scale, and thus bigger chassis in a rack could use bigger heatsinks and fans at lower speeds instead of small screamers.

maybe you're not familiar with just how stupidly written most code is.

you're right that there are efficiency limits, but not once in my career have I ever seen anyone even attempt to write their code so that it is efficient to run, outside of gaming.

Anything can be improved. Or almost anything. The question is by how much, 5%, 10% or 300%. In this case, I am not really I understand the problem oxide is set to solve so I can't really comment precisely but to me it sounds that if we generally say that data center equipment is suffering from power budget and cooling issues then I don't see this as a problem that software can solve.

> I don't see this as a problem that software can solve.

Neither do we, which is why we are building both software and hardware, together. That's the only way to truly tackle thorny issues like these.

Oxide is doing some great things, but there’s only so much you can do with firmware tweaks. A CPU running any load at all is going to completely eclipse the power usage of everything else in the system.

Incremental improvements from things like more efficient fans and reducing the number of power conversions is great, but the power drawn by the CPUs or GPUs is on another level.

If after that you're not satiated with data center cooling talk, Jane Street's Signals and Threads just did an episode about their cooling infrastructure a few days ago:

https://signalsandthreads.com/the-thermodynamics-of-trading/

Reply on news.ycombinator.com