Hacker News Reader

Comment by Renaud | Hacker News Reader

My guess is that AI workloads are going to be a significant factor of demand for high-end cards. It used to be gaming and mining and now we have AI art generation that requires high-end cards to be productive.

Now mere mortals can play around with Stable Diffusion and other art-generation technologies, this is going to increase demand for these types of cards.

View on HN

Folk are already using lower end cards with Stable Diffusion, even the 4GB models.

My 3080 is well capable of generating 512&768 images with a high step count in under a minute, and 500 odd steps (just for fun) in around 2 minutes (or even less with different samplers)

The interesting bit is when we get more VRAM (24-32GB) and can actually train the models with off the shelf hardware without breaking the bank.

And soon, it won't just be the people who are "playing around" with AI models on their home machine. It will be games and commercial products using ML at the client level that will have system requirements around GPUs with a lot of memory for the better results. A lot of this can be done at the cloud level, but if you're clocking heavy hours on cloud GPU, that subscription model is going to be very expensive and will introduce latency if a lot of data needs to be exchanged. Since you can't (by the license agreement on NVidia's drivers) just throw a bunch of consumer-grade hardware in a datacenter to do ML loads like you can for mining, the price shoots up if you want to do the heavy lifting on a central server.

Reply on news.ycombinator.com