Hacker News Reader

Comment by yieldcrv | Hacker News Reader

the real question is whether ChatGPT needs that kind of hardware for a single user. a client side version.

Yes it does.

When stable diffusion came out, my 8gb gpu could handle 1 image at 512x512. Now i can generate 1000x800 with batch size 4. I wonder if Chatgpt has the same unused optimisation potential

Stable Diffusion is a lot smaller than GPT3. It has a bit less than a billion parameters while GPT3 has 175 billion parameters.

I would bet that we can do better than GPT3 with less computing resources but that would need a different model architecture.

> I wonder if Chatgpt has the same unused optimisation potential

Doubtful. As an example, OpenAI released Triton, a programming language explicitly for optimizing models. For the scale they're at they'd be crazy not to throw an engineer or two at optimizing the models they're going to send to production.

Make a Google of "hardware overhang"

Our brains outperform GPT-3 with only 30w of power. The potential for software optimization may be many orders of magnitude.

Wonder where is that 30w comes from. I found some said chess player used 6000 calories per day and that does not seem to tie in the number.

The brain mostly uses the same amount of calories all the time. It's the large muscles that can influence energy use.

what packages are you using?

I think I have DiffusionBee installed but I haven't updated it for months

Reply on news.ycombinator.com