Cooling is usually what kills you with server gear at home. I don't have a cold aisle, or a case that flows that much air. Can I water cool it enough?
No video outputs would probably be my biggest annoyance. It should be fine at headlessly rendering games for streaming though, as well as AI.
If there's a will, there's usually a way.
So, sort of specialized? but not really.
> If there's a will, there's usually a way.
That's the spirit!
Yes they use pcie electrically, but its not the same slot shape and there is usually carrier boards
You wouldn’t repurpose these for home gaming, it would still be for llms
You can just mod them to run on consumer fans that are designed to spin much slower but have higher diameter
This is already happening.
You can order the SXM carrier boards on eBay/Alibaba today.
> You wouldn’t repurpose these for home gaming, it would still be for llms
When small businesses, home-labbers and prosumers have cheap access to potent GPUs, there will be a Cambrian explosion of grassroots creativity and innovation in areas that are "beneath" AI labs expansive ambitions; I want to see the OBS-equivalent of LLMs. Hell, I'd love to see what some broke artist can do with OBS, an H100, and an open video model.
It can certainly be parted out with proper motivation but it’s going to take a whole new ecosystem to support these in your typical home lab. Even then 20-30kw slices are electric car level energy consumers.
> Good luck powering a 400kw rack in your basement
Wut? Open-weight models already exist and don't need a cottage industry.
I'd gamble on 1 million curious people with 1 SoTA GPU each over one AIaaS lab or company with 1M GPUs when it comes to developing delightful things and experiences. We still don't know what small, specialized models can do because AI labs only ship products that could justify the trillions invested so far. With a single Nvidia A100, I have trained small models that beat the brakes off "SoTA" models in niche areas (estimated TAM is <10M people, world wide) - the "Bitter Truth" doesn't apply to the long tail.
This whole subthread is just sad. "Oh no, it's not the same form factor. It is useless".
Where's the "Hacker" in "Hacker News" there.
It's a shame that we need to defer to "China" as a shorthand for "actual engineers" just because our own engineers lack imagination and drive that would validate being given that title in the first place.
There are a lot of reasons other than skill why China is the way it is. Culture and locality are big ones. I don't know a whole lot but I disagree that any "actual engineer" could easily come up with a sustainable business for this stuff in the US -- for example, harvesting and processing used parts for their components is a hugely laborious process, and China has a huge culture of recycling and reworking that simply does not exist in the US (and would largely be socially unacceptable to introduce!). Take for example the pre-modded 4090s, or the BGA desktop motherboards I mentioned, or any number of other products that are based not merely on sheer engineering. It's not exactly engineering on its own that causes these things to happen, but more the fact that nearly anybody can just throw together an operation like that and have tons of workers available to make it sustainable.
Please don't misunderstand me as attacking you. I was attacking that other guy further up the thread with the empty GitHub account and the equally empty ambitions, trying to drag all of us crabs back into this bucket we find ourselves in.
Oh, plenty of people do not choose not to -- there are tons of blog posts and YouTube videos out there of people doing stuff exactly like this! It's just the making it sustainable part that's very difficult outside of somewhere like China, because -- unless you have special VC privilege -- you can't really offer reasonable prices to the masses, which makes your purchase volume go down, which further drives up your prices even more, until it converges on some insane number.