Hacker News Reader

Comment by smuhakg | Hacker News Reader

> The only real systemic risk is the risk of for some reason not needing any data centers.

How is this not a systemic risk? Asserting that:

> There's still a need for data centers to host non-AI servers.

Isn't true because a NVL72 rack consumes 23x the power of a traditional server rack which is why there's new construction. You could retrofit such a datacenter with normal servers but then you're taking a loss on the GPUs which is the majority of the cost.

https://epoch.ai/data-insights/ai-datacenter-cost-breakdown

View on HN

It's not a systemic risk because outside of society collapsing we are going to use lots of data centers regardless of what happens with AI.

At this point the question isn't really if we're going to use the GPUs they're building out. The question is how much profit they are going to make. I've seen people show margins as high as 80% on inference. That's a lot of headroom before we see data centers being shut down.

It is a big assumption that society is going to need more data centers no matter what. That requires a gain in efficiency by using more compute power. Most gains in efficiency in the past have been from using fewer resources. Spending more or producing more doesn't make your market larger. Before AI, the future of data centers was low power, efficient cooling, cheaper costs. How many low power ARM boards you could stuff in a rack rather than how many GPUs.

Wonder if we'll see ultra-dense CPU compute (1000s of cores in 1U) to take advantage of this power density whether the AI bubble pops or not.

As of writing, the most dense server CPU is the AMD EPYC 9965, which has 192 cores. It is possible to build a 2x CPU 1U server, so that is 384 cores per 1U. I am unsure what is the limiting factor: heat or space on the die (probably lots is just L1/2/3 cache). When you look at the interior of a vanilla 1U server, I don't think there is any remaining space to add more sockets (CPUs). Final note: SuperMicro also offers an Ampere ARM chip with 192 cores.

Reply on news.ycombinator.com