Genuinely think google + Meta might be the only companies to make a profitable AI product at this rate.
Consider: if you had built a hyperintelligent stock-trading AI, would you:
- Charge $0.70/M tokens for other people to ask the bot what stocks to buy.
- Get rich following the bot's trade advice and never tell anyone about your bot.
The fun thing about open markets is the market will punish you for this with second order effects and it's in your best interest to get some whales on the bandwagon.
Pretty much every technology in history is available at several orders of magnitude lower cost than the first few years of it being invented.
AKA eventually everything runs on a potato.
Not today's raspberry pi, but the equivalent of it absolutely. I don't work in it, but have a number of friends who are involved in LLM development - the "breakthrough" optimizations are obvious work in hindsight. We are still in the very early innings of optimizing both hardware and software techniques. The recent market imbalance has blurred the idea that hardware simultaneously gets both cheaper and much better, but no one serious expects that to stop on a longer timeline.
Going beyond the "my watch has more computation power than NASA at the moon landing" - speech-to-text used to be unreliable tech that needed datacenters backing it as recently as the early iphone, and now runs in near-realtime on smartphones that cost about the same. A ~$50 processor running in my closet does realtime person/vehicle/package detection on my front door camera reliably.
https://huggingface.co/microsoft/bitnet-b1.58-2B-4T#evaluati...
Which LLM use cases can be solved with compact decision trees is an awesome open problem.
If we're talking web servers, sure. Video games? For the most part. Numerical high performance computing? ehhhhhhhhh i'm not sure history bears that out
Any numerical high performance computing from 1995 will run on a 100$ android phone.
https://en.wikipedia.org/wiki/Cray_T3E
And if this was the fastest, the average ones were significantly smaller.
What you're missing is that the things you mentioned sometimes took months to run in the fastest computers thirty years ago, but you are mentally expecting them to work quicker now.
Anyway, interesting talk, thanks. Have a good one.