Even if he was wrong about those statements, exploring other types of models isn't necessarily a bad thing. Especially if we can find ones that perform well with less demanding hardware requirements.
But even in computing, when there is a promising way forward that the basic reaearch has been done on, its hard to justify side quests.
...And because they have monopoly money. Basically, research labs are usually vanity spending for corporations with too much money. Xerox parc was funded by a monopoly. Bell labs was funded by a monopoly. Google funds its moonshots with its monopoly.
All display technology past CRTs has been a mistake. The vectrex in the 1980s had better refresh rates, color fidelity, beautiful phosphor glow, etc - and we abandoned it for a horrible pixel based future we are now in. We should return to vectors and lasers.
How does that work when the research team moved elsewhere? Re-hire talent? That takes time - the 1 thing you don't have in a fast-moving field like AI. Post sums it up nicely:
"Do the math—how much is it worth to halve training and inference cost given that the industry has investments approaching a trillion dollars?"
How much is these researchers' salaries, compared to what MSFT spends on compute? Chances are it's peanuts in comparison.
Companies will try to ruthlessly rid themselves of labor as long as those members of labor have agency and rights.
If someone disagrees they can convince me by explaining why a company wouldn’t choose to spend X on a machine they wholly own vs X on an employee they don’t.
There’s some room for argument that employees can do more than machines can but in the current AI zeitgeist arguing that we have machines with equivalent to human intelligence I don’t see how you wouldn’t see this coming if you believed the zeitgeist.
In a way this suggests that MS doesn't aim to be a leader in AI <the core tech>. Only in its applications + related services (read: profit to be had there).
Nothing wrong with that btw. But understandable that fundamental researchers then pack up & leave.
That’s bolstering my argument. Companies are willing to spend a shit ton on machines. They are not only unwilling to spend on labor but are actively seeking ways to remove labor spend, even though this new technology supposedly makes every unit of labor significantly more productive and thus profitable.
This is a cultural choice not an economic one.
If I see a company laying off people while admitting they’ve maximized a niche and automation lets them increase profitably, or I see them go on a hiring spree because labor is now netting them 10x-100x-1000x-whatever productivity increase they claim, then I will believe them about the value add of AI.
Currently I see companies using AI’s productivity increase as a reason to cut labor and achieve roughly the same productivity.
They aren’t competing, they just want to cut out workers.
Ah, well, good thing we've almost entirely defunded academic science.
Seems weird when it's become apparent that LLMs aren't really special sauce outside of just having the compute.
10% chance to be market leader doesn’t sound as good as 60% second place to them for example
Once you get close to the top, the risks don't look worth it.
It's his call for sure, but rather funny as an argument.
It is because even if it would pan out, it would mean temporarily losing AI leadership, reducing stock price, losing the ability to hire talent, and by virtue of that losing the opportunity permanently.
He was against LLMs, reinforcement learning, etc. Claiming that those are a dead-end. He was wrong. His own "JEPA" ideas have basically gone nowhere. Meta was right in moving on without him.
Let me know when LLM's get those. Otherwise I trust it as much as a human under anarchic rule. Probably less because I know exactly who controls it.
Zuckerberg wanted to placate his shareholders with a shiny new thing now.
would've stopped Meta from delivering a polished product like Muse sooner.
Muse is not polished, unless usability and programming standards have fallen so far in the LLM vibe coding era that a pre-alpha state is now considered polished.
Muse is imperfect, as is all software, but calling it pre-alpha is a bit much.