Hacker News Reader

Comment by setgree | Hacker News Reader

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.

[0]https://pmarchive.com/guide_to_startups_part4.html: "In a great market—a market with lots of real potential customers—the market pulls product out of the startup... The product doesn’t need to be great; it just has to basically work."

View on HN

You should listen to the podcast Acquired, specifically Nvidia and then Jensen Huang. They basically lucked into AI. Some researcher was using Nvidia gaming cards, and reached out to them about questions on CUDA. That email eventually turned them into a trillion dollar question.

What year are you talking about? When I was in grad school, around 2007, Nvidia was aggressively marketing GPUs for high performance computing. They would go to campuses, talk to professors, etc.

Yes, the whole Deep Learning thing was luck, but as with most lucky things, they ensured they were positioned to capitalize on it.

Probably cerca 2014 as that's when AlexNet was released, demonstrating that neural networks could beat traditional ML models at image recognition tasks. I recall the researchers used Cuda to optimize their training setup.

AlexNet kicked off a new wave of research around neural networks by demonstrating they could be scaled well and trained on GPUs.

Yeah - definitely by 2014 they were well entrenched within academia with their CUDA offerings. By that point it wasn't "luck".

to their credit, there was a lot of work behind "luck". Jensen showed up in person in 2017 in NEURIPS and he and likely a lot of his top brass basically sat down and read the entire conference proceedings/abstracts; there was likely a lot of work behind the scenes to behind the ML research pivot.

And 2017 was _late_ in their pivot. They'd been active for much, much longer. Last winter break I sat down to watch every GTC keynote, going back to 2009[1]. Even then, he's talking about expanding to non-graphics workloads. Google's GPU paper[2] just slotted naturally into their existing narrative and were happy to support it. "fortune favors the prepared" as they say.

[1]: https://www.youtube.com/watch?v=fYuH2Kl_b98 [2]: https://scholar.google.com/citations?view_op=view_citation&h...

They definitely were sponsoring ML conferences before 2017, but symbolically, having Jensen/the CEO actually show up at a dedicated session and demonstrate detailed technical knowledge of the conference proceedings in my mind was a turning point. Albeit I remember his words to the crowd of grad students and post docs at the time: "Only Nvidia would announce their flagship card...to an audience who is completely broke!!"

Yeah, The NVIDIA Way goes into a lot of detail on how and why the pivot from graphics to AI happened. This is a prime example of “you make your own luck.” Jensen engineered an organization that was primed to recognize and pounce on the next big thing, and it ended up being AI. But they saw it coming WAY in advance (like 2011/2012, not 2017) because they were explicitly on the lookout.

> an organization that was primed to recognize and pounce on the next big thing

e.g.: previous crypto hype-cycle

https://www.pcgamer.com/nvidia-cmp-graphics-card-availabilit...

Crypto was a stupid fad, but Nvidia certainly made a lot of money, so being a vendor to a fad is not stupid.

Which makes it a Exempli gratia of "an organization that was primed to recognize and pounce on the next big thing." In addition, they also recognized early on some of the weaknesses of crypto-mining as an industry and limited their exposure while making a pivot to the next thing.

They didn't make their GPUs for crypto. Crypto companies just bought a lot of their GPUs.

If they had a choice, they would have sold every GPU to gamers instead because a crypto boom was always followed by a crash which flooded the market with used GPUs and crashed Nvidia's stock price.

> They didn't make their GPUs for crypto.

Sure they did and still do. They did a good job of managing the bubble but participate in the market nontheless. Read the above article: Nvidia has promised that its new lineup of cryptocurrency mining GPUs, called CMP for short, won't impact the supply of GeForce graphics cards for PC gamers. . . Since these cards are destined for mining rigs they will also lack video outputs. That may mean the resale value is diminished, which has been one reason why miners prefer gaming graphics cards.

Or take it from the horse's mouth [0]:

  NVIDIA Cmp Hx
  Dedicated GPU for Professional Mining

0. https://www.nvidia.com/en-us/cmp/

They made a dedicated chip because miners were buying up all the GPUs from gamers.

They also nerfed GPU hash rates through software to prevent miners from buying them.

AlexNet was 2012 and they explicitly called out the use of NVidia GPUs

In 2006. The next 20 years of cuda support weren't luck, as anyone trying to use AMD will know.

I might have believe this story, if not at the same time Intel had made an expensive bet on producing not-quite-gaming cards, later looked at the same trillion dollar question.. and then almost decided that this did not bring enough luck to keep spending.

Was this the case in the past?

My vibes were that Apple wound down the “actual work” side of their operations (including machines like Xserve), because Ives couldn’t handle the unsexiness and unpredictability of business requirements in hardware.

He was self-indulgent and only wanted to work on things that “vibed” with him, rather than what the customers needed. It’s easy to be creative when you get to do what you want to do, it’s hard when you have hard constraints.

Apple has not in the past three decades really courted the capital E Enterprise market. They'll definitely sell to Enterprise customers and have Enterprise sales teams for big customers. But they're not and never have been Dell or HP.

Enterprise sales sucks. There's infinite amounts of politicking and glad handing and buyers will get all sorts of sweet brib..."sales dinners" then go with the cheapest option. Margins on hardware sucks and the only money is in support contracts. Apple instead invests in consumer sales/support primarily and all the other channels are side businesses.

Stuff like the Xserve existed mostly for Apple internal purposes and ended up being sold externally to goose the scale enough to make them not a huge loss. At one point a large percentage of the offices on Bubb road were packed with Xserves running portions of the iTunes Music Store and the Apple online store. More offices were packed with Xserves doing media ingest and encoding for iTMS. Just about every building had racks of them as build and file servers.

I think Jobs was quite sceptical about courting enterprises. Personally this is one of the reasons I choose Apple over Microsoft.

Maybe a bit of hindsight bias / the outside view here, but I feel like they're completely asleep if they didn't anticipate strong demand for this specific use case.

I think a reasonable story could have been told that goes like this: local models aren’t as good as frontier models with a $20/month subscription, and the hardware costs a lot. So only a few enthusiasts will buy Apple machines for this purpose.

This story turned out to be false but I think smart, reasonable people a couple years ago could have believed it with conviction. It doesn’t really seem like “completely asleep” to me.

They were investing in ANE and Metal before everyone in consumer. Hardly asleep. They just underestimated the market size, as pretty much everyone did.

I don’t understand how that’s possible. They should have had a better idea of what was happening in the memory markets than pretty much any other entity.

Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU <--> CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras.

They have been taking NPU's seriously on their phones, tablets and laptops since the M1.

Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's = 2TB RAM. Perfect for a large Mixture-of-Experts model.

It would be very strange if they didn't notice their product line had landed in a new sweet spot.

FWIW, the reported reason for OpenAI buying Macs has nothing to do with the memory by the sounds of it. Every single outlet I can find reporting on this seems to repeat that the intended use case is for agentic workloads and generating training data for reinforcement learning. They don't appear to be doing inference nor any sort of training AFAICT.

For OpenAI, their data center archipelago is their own "local" and "personalized" AI.

I am curious about the corporate disconnect from the frontline to the generals.

While the company I am in is embracing AI the disconnect and delay between what is available and possible versus what is approved and permitted is a three month window. The State employees I speak to are just now getting around to writing their usage policies for internal AI usage.

Same here. For every little use of AI we have to fill out a 3 page proposal to get a PoC for 3 weeks in copilot studio (the worst AI builder around but due to MS lobby the others are banned). Then we have to find a business sponsor to decide it's useful, find 2 financial controllers to underwrite its token cost (even if very little), go through rounds of lifecycle and governance approvals. Then we can move to preprod but nooo we're not there yet. Now comes the security review and DPIA. That's another hugely complicated process that takes months. Eventually when both are done we can go the the deployment team and go through their process which I haven't even seen yet because most of our projects got cut off beforehand.

All the while the top of the company is of course praising AI and saying we should do everything with it right away.

It's a joke really. We had to go through all this rigmarole just for an agent that does some preliminaries on a service support ticket (making sure all data was filled in correctly and contacting the requester of not) before sending it to a human for final review. It couldn't action anything, not a single thing. Just assist with the prerequisites.

And yet they call our company 'innovative'. We're certainly innovative at inventing bureaucracy.

Tim Cook has been touted as the greatest supply chain logistics person on the planet and revolutionizing Apple's product delivery, securing exclusive contracts years in advance, etc., etc.

But "oops, we missed that people are interested in AI work on our machines" seems like a really fucking big myopia. But then again, Tim's off to retire on a bed made of cash this week, so...

It's also fun to see how many people here believed this was all some clear deliberate strategy in the first place rather than an accident.

They didn't "accidentally" add tensor units to the GPU cores in the M5 generation.

However, I don't think they expected the level of Enterprise interest they saw.

No ‘staff focused on developer relations’ is entirely unsurprising based on what I see from the outside.

That raw statement is completely and totally false.

“Not as fully staffed as some people might hope” or “Developer Relations isn’t as responsive as I’d like” are both at least not obviously false.

> "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy"

This is clearly a mis-statement, they have a whole annual conference for developers. Maybe they mean specifically AI devs.

Reply on news.ycombinator.com