Hacker News Reader

I would self-host if it didn't cost so much for the hardware. But right now that's cost prohibitive for most individuals and small or medium businesses.

View on HN

Maybe that's the reason why AI firms tried to monopolize ram, chips like AMD AI MAX 395 with AMD not being beholden or useful to the AI firms coupled with 128gb+ ram would probably have been capable of making home-hosted open models competitive enough that the AI firms valuations would've sunk like stone before their IPO's.

The question is if the MoE advances and compression (TurboQuant) makes common computers fast enough to run local models that are "Good Enough".

Funny how that happened, isn’t it?

How what happened? How everyone stopped having cutting edge graphics cards in all their devices? Most people have never been in a position to be able to do local inference on cutting edge models.

Reply on news.ycombinator.com