Hacker News Reader

Comment by nelaggy | Hacker News Reader

so if the moat isn't the model and the moat isn't the inference layer, i guess the moat is the hardware now? what does this mean for frontier labs?

View on HN

hardware has always been the moat in the AI space

Will always be true

Yeah a lot of people bought into the idea of CUDA being a moat but we’re well past the point where it’s pretty trivial to jump to another equivalent stack on the software side.

Those are the two main interfaces: the inference hardware and the endpoints. Endpoints are pretty much standardized to the point that you can switch them out in seconds. CUDA still has an edge but every release like this chips away at that. Even so, TPUs are unobtanium so while I appreciate Google's effort here I also recognize that it is not something that will be useful to me.

In that sense AMD is still better positioned to give NVidia a run for their money but given the family connection and AMD's underdog status I am not very hopeful that we'll see meaningful competition in inference hardware any time soon.

NVidia is a much harder nut to crack than OpenAI or Anthropic. They are easily a decade ahead of viable competition. Check out the GB300 if you don't believe that.

A decade is a long time and the highest end isn't the only market segment.

High bandwidth flash will be here soon, I bet someone makes an economical competitor thats not quite as fast but much cheaper.

While I have the same sentiment, I am also curious why a competitor can't accelerate their developments with AI. In the end, hardware development is to a large extent also software development. Is it the patents that NVIDIA has that are preventing this? Or maybe some trade secrets? AFAIK the NVIDIA hardware is manufactured entirely by TSMC.

Google does provide TPUs to big players like Anthropic and other large enterprise customers for on prem deploymens. That's their target just like those the GB300 is aimed at.

Even if it's allegedly possible, the devil has always been in the details. It always sucks to find out that there's some quirk of your model architecture that breaks an implicit assumption that underpins the export process.

Products can also be a moat.

Reply on news.ycombinator.com