Hacker News Reader

Can anybody ELI5 why aren't there multi gpu builds to run LLMs locally?

It feels like one should be able to build a good machine for 3/4k if not less with 6 16GB mid level gaming GPUs.

Reddit's LocalLLama has a lot of these. 3090s are pretty popular for these purposes. But they're not trivial to build and run at home. Among other issues are that you're drawing >1kW for just the GPUs if you have four of them at 100% usage.

6 * 16 is still nowhere near 512gb of vram. On top of that that monster that you create requires hyper specific server grade hardware, will be huge, loud and pull down enough power to trip a circuit breaker. i'm sure most people would rather pay a 30 percent premium to get twice the ram and have a power sipping device that you can hold in the palm of your hand.

Reply on news.ycombinator.com