Hacker News Reader

Comment by jacquesm | Hacker News Reader

> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.

They'd be wrong. Especially about Europe. Many companies are in a bind about this: they can't upload their confidential files to a service provider but they do want AI. So right now they end up with a choice: do nothing, breach their contracts, use a service provider that pinky promises not to share their data (which we have already seen they will violate) and run local models.

The latter is the reason you can't buy powerful GPUs for reasonable prices because a local model (GLM 5.3, DS 4, etc) + some hardware is all you need to solve the problem in a way that avoids the legal issues.

View on HN

You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center buildout, not because some mid tier European insurance co needs to serve deepseek

Yes, well, you're wrong about that. They're buying GPUs by the pallet here.

Mid tier insurance co is working with local inference providers and some of these now have thousands of GPUs up and running. Don't drink the HN cool aid that the EU is a backwater where nothing happens too much or you'll end up surprised one day.

The one thing where Europe really does lag is in training, that would be fair. But inference providers are popping out of the ground like mushrooms and there are some billion euro+ projects underway. We -unfortunately- also have some scammers that are bilking (or attempting to bilk) investors pretending they have more than they do, this is annoying but I would expect that other countries have similar players as well.

Wow, whole pallets!?

There are roughly 70 GW of datacenters under construction in the US right now, and a GW is roughly 500k GPUs, so 35 million GPUs.

The 'by the pallet' buys are for small local projects, not data center size projects (of which we have more than a couple as well), as you probably already knew. This isn't some silly dick measuring contest.

The point was: they're buying them from the retail channels in relatively high volume because there is a massive shortage of GPUs in general, but that does not seem to stop parties with smaller projects who do not qualify for buying directly from NV (assuming NV even has stock to sell).

5090's now sell for the same amount (or more) that you could buy an RTX 6000 for a few months ago, and RTX 6000's are ~19K and up.

I found a stash somewhere, bought two, then two more and when I was ready to buy the remainder they were all gone. Serves me right for being too careful.

No, the point is that US demand is what's driving up GPU prices, as GP asserted. Europe's purchases are a rounding error. And even then, most of the data centers under construction in the EU are from American companies.

A rounding error is taken to be sub 1%. Anyway, I shouldn't engage, good luck.

The US has never cancelled as many data center projects as now.

Not nearly as many as you’d think from news headlines: https://newsletter.semianalysis.com/p/datacenters-2027-the-y...

Your link doesn't speak of cancellations only about a certain GW worth of data centres being built but that are not operational.

There’s a link in the first paragraph that addresses what you’re looking for

this is a fun comparison, so I ran the numbers - in the US the cancelations alone are as big as Europe's entire planned buildout (~11 GW), or almost 3x of what is currently under construction (~3.8 GW)

> US frontier labs are spending nearly a trillion dollars this year

They are not, not even close. I wonder if you're aware how money is being "spent" between Nvidia and US AI corporations!

You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center build

Its not just the US ... Where do you think all those 5090's end up? Its just crypto mining all over again. Most people do not realize that China was the biggest crypto mining operation because of its cheap electricity and people willingness to take big risks.

The same happens now also, with people buying up 5090's (and lower class cards) in bulk, to strap into AI rigs. With Nvidia discontinuing the 32GB 5090s, we already see pressure on the lower end cards that had a better price / performance for a while now.

China will do China things *lol* ... People there are just more willing to take bigger gambles (and the cheap elec helps a ton). But that is also a factor why a lot of consumer level hardware gets under pressure. Its not just the supply chain to the enterprise market that is eating a lot for AI.

That does not matter so much anymore, when Huawei starts selling those NPUs for a fraction of the NVIDIA price. It's all downhill from there on for OpenAI, Anthropic and NVIDIA.

It's all downhill for OpenAI and Anthropic investors, because they will never recoup what was invested. NVDA is safe(r) - Huawei first needs to catch it, and as AMD learned, it's a fast moving target.

Indeed NVIDIA has an actual moat. The other two, not so much.

The other choice is AWS Bedrock, if it happens to have the model you're interested in. They do have GLM-5.3. They don't have DS-4 or 4.1 yet, and updates are intermittent at best, but they're already on the trusted provider list for anywhere large enough to be interesting.

That may not be acceptable for many companies outside of the USA.

Depends on the parameters of "acceptable", but they do offer inference in non-US zones. If the company is already on AWS for meaningful workloads with sensitive data (which is not uncommon), they've already been through the hard part.

That being said it's not quite as simple as would be ideal, you do need to check whether there's an additional EULA on your specific model choice.

Acceptable as in 'does not break the law'.

For data residency reasons? You can meet data residency requirements on AWS, I'm pretty sure. Mind sharing which condition blocked it in your experience? Mostly curious because in the US there isn't that much interest in on-prem deployments because it's easier to scale up and upgrade using the cloud and it's easier to use AWS because you don't have to run a new compliance process for each part of the stack.

You could until the Trump admin decided to (attempt to) overrule the deal that was in place (the DPF).

The perceived lack of independence of the FTC is a major driver behind the exodus on this side of the Atlantic and it isn't just the smaller players that are moving. Airbus for instance.

Now the datacenters supposedly operating under at-arms-length management are no longer sufficient for compliance.

There are calls to repeal the whole thing with respect to the US.

https://noyb.eu/en/us-supreme-court-just-blew-eu-us-data-tra...

Yes, biased source and title, but the contents are pretty good.

I see. So there's a risk that the EC or CJEU make a determination against the existing system because of the non-independence ruling of the FTC. So the movement away to regional providers is because no one wants to be caught surprised if the EC/CJEU make such a determination. Interesting. It must be early stage because from what I can see the most spend is still on the big two, but this does bode well for Mistral and EU-based open model hosts. Lots of room in the 20%.

It's early stage mostly due to inertia but there is a groundswell of movement. Airbus really turned some heads.

Yeah, about that... My kid works for a very small company. Chatted the other day about in-house inference vs ZDR providers and all. His team is trying to get access to GLM on AWS Bedrock. It is a second week they are trying to schedule a simple meeting to discuss what they need. Random silence, and back and forth from a new person each time. 2 WEEKS to just schedule a meeting... And as far as I know, no luck so far.

Update: they got back to us! Scheduled meeting early next week :facepalm:

You do realize there are hundreds of datacenters in Europe and the US that rent GPUs to European companies, right? You don't have to buy a GPU or even figure out how to use one. Pay for SageMaker.

Europe isn't in a bind. They can, and have been, running open weights on inference providers around the world. And they pay a premium for US models and accept its risk for the same reason they remain US allies. The cost is worth it.

Yes, I do. Not sure what in my comment prompted you to write all this.

Reply on news.ycombinator.com