https://www.reddit.com/r/LocalLLaMA/
Search for terms like hardware build, running large models, multiple GPU’s, etc. Many people there have multiple, consumer GPU’s. There’s probably something about running multiple A100’s.
HuggingFace might have tutorials, too.
Warning: If it’s A100’s, most people say to just rent them from the cloud as needed cuz they’re highly costly upfront. If they’re normally idle, then it’s not as cost effective to own them. Some were using services like vast.ai to get rentals cheaper.
You don't want an A100 unless you've already got datacenter provisioning at your house and an empty 1U rack. I genuinely cannot stress this enough - these are datacenter cards for a reason. The best bang-for-your buck will be consumer-grade cards like the 3060 and 3090, as well as the bigger devkits like the Jetson Orin.
If you need a guide, the A100 class cards are certainly not what you want. Running data center level clusters in the home is no walk in the park. You wouldn’t even be able to power a decent sized cluster without getting an electrician involved for proper power. Networking is expensive. Even second-hand it all adds up fast. Don’t underestimate how much it would cost.
Getting some 3090s in a case with a big gaming PSU is the way to go for home use. If you really want A100 level, rent it from a cloud provider. Trying to build your own cluster would be like lighting money on fire for a rapidly depreciating asset.
Or rephrasing for somebody that never tried to experiment with models: which one do you need to have something like ChatGPT-4o (text only) at home?
Edit: outside the HPC community specifically, I mean