Hacker News Reader

Comment by hintymad | Hacker News Reader

Most of the loss comes from hefty cost of inference, right? OAI runs on Azure, so everything is expensive at their scale: the data, the compute, and GPU instances, and storage, and etc.

I'd venture to guess that they will start building their own data centers with their own inference infra to cut the cost by potentially 75% -- i.e., the gross markup of a public cloud service. Given their cost structure, building their own infra seems cheap.

View on HN

Given their scale and direct investments from Microsoft, what makes you think this is any cheaper? They’ll be getting stuff at or near cost from azure, and Azure already exists and with huge scale to amortize all of the investments across more than just OpenAI, including volume discounts (and preference) from nvidia.

I assumed that Microsoft gave them discount and credits as investment, but the cost by itself is like any other big customer can get.

Of course the cost isn’t near zero, but is the pricing “at Microsoft’s cost”? If it is, then them building a datacenter wouldn’t save anything, plus would have enormous expenses related to, well, building, maintaining, and continuously upgrading data centers.

Good point. I updated the assumption accordingly. I assume the cost of using Azure cloud after discount but before the credit will be on par with Azure's big customers. And given the scale of OAI, I suspect that the only way to be profitable is to have their own infra.

The cost of vertically integrated inference isn’t zero either.

Wouldn't it be foolish for microsoft to give them a discount?

Like you could discount say a 10B investment down to 6B or w/e the actual costs are but now your share of OpenAI is based on 6B. Seems better to me to give OpenAI a "10B" investment that internally only costs 6B so your share of OpenAI is based on 10B.

Plus OpenAI probably likes the 10B investment more as it raises their market cap.

As part of Microsoft's last investment in 2023 OpenAI agreed to exclusively use Azure for all their computing needs. I'm not sure what the time limit on that agreement is, but at least for now OpenAI cannot build their own datacenters even if they wanted to.

Microsoft restricted their computing needs provider, but they could have allowed them to build their own datacenters.

If they only have enough for a year or two of runway at current costs how the heck would they have enough capital to build an AI data center?

I cannot believe they don't already get steep discounts via custom contracts with providers.

Someone has to pay for the hardware and electricity.

Building datacenters will take a significant amount of time. if they don’t have locations secured then even more so.

Are there enough gpus available at the scale that they need them?

Reply on news.ycombinator.com