AMD acquired this company, and I hope they start releasing consumer hardware based on their tech.
Your move Intel and NVIDIA, give us what we want and stop hording GPUs in data centers.
Those on-device weights are horribly outdated, underperform for their size, and cannot be updated to use a different model. I don't think Nvidia is afraid of Taalas.
To push beyond the envelope of what a mobile GPU can do, you'd have to build gigantic coprocessors for 30-100B models, which would probably cost more than the phone/laptop itself to integrate on-die. Nevermind the fact that SRAM is several times more expensive than DRAM gig-for-gig.
There was a time when people were running microcomputers, but they were just toys; the future was mainframes and minis to do real work.
The way tech goes is that the "toys" become real. They become where the action is. The bigger stuff doesn't go away - there are still mainframes, even today - but the action moved on to microcomputers, and then to phones.
There will still be "big" models, but don't dismiss models on a chip because they are "toys".
Note well: I do not claim that models on a chip will certainly follow that trajectory. All I'm saying is that, given the trends in tech over the last several decades, don't dismiss them just because they are toys.
Crypto miners already charged down this route for mining, and now there are thousands of ASIC mining rigs that are worth nothing, when GPU rigs are resold at a profit. ASICs are a dead-end, and I think that's why Nvidia couldn't give a rats ass what Taalas builds.
They make new car models each year. Plenty of folks drive old ones. Few do annual upgrades.
We should all know by now that it depends on who you are. For conversational use, toolcalling and structured response, it was a few years ago. For coding it will probably be a few months from now. And for advanced 3D/computer use/biology/engineering applications, we're probably looking at a longer time horizon.
We have GPUs and NPUs capable of accelerating models this size on-device, most of the time. Replacing that flexible logic block with a static ASIC is a downgrade, especially since users don't ever have 1000+ tok/s or batched request requirements. The on-device acceleration is perfectly fine for >8B model inference.
I get that people are starry-eyed, and that HN is supposed to be the place for curious inquiry. But creating a coprocessor that uses a redundant, static memory system that gives users no noticeable advantages is much more expensive and stupid than investing in a CUDA-style GPGPU architecture.