It is just a Asrock motherboard and 1000w corsair power supply sitting on the bottom shelf of a "mining rack" open air case. The GPUs are attached with janky pci risers (the motherboard has four pci slots). I went with 5060s so I could stay with "short" 2-fan cards. The boot drive is a 500GB NVME with an encrypted 2tb NVME for storage, both on the mobo. I added four cheap fans and an oldschool (with a nob) fan controller to circulate air away from the GPUs. RAM options are maxed out at 4x16GB DDR4 sticks. Expansion options include adding a fourth 5060 and any number of SATA storage drives for content/models.

It, of course, runs on linux. I went with stock ubuntu so I could attach a screen and/or remote in to a full GUI. Having a full GUI also makes the rig more relatable to people who's eye glaze over when they see a terminal interface. "It's just a funny-looking desktop computer."

At 48GB of VRAM it can run basically any LLM I would want to download. Speeds are fine for a single user (Tok/s numbers change based on model/mode). But if I were using it for something like ComyfUI I would ditch the multiple 5060s and go with a 5070ti or 5080 as ComfyUI does better with a single powerful GPU.