IMHO it's only a very small part of why GPU power consumption is going up. The main reason is the completely unnecessary chase for the performance crown.
From personal testing: my GPU manages to get 95% of its peak performance while being power limited to 80%. So the in order to squeeze the last 5% of performance out of the device, 20% more power is pushed through it. It stays above 99% peak performance while being power limited to ~87%.
But even just looking at the raw numbers paints a different picture. About 12 years ago, a high-end GPU (e.g. GTX 480) had a power draw of 250W at a theoretical peak FP32 performance of 1,345 GFLOPS. This year's RTX 4070 has a theoretical peak performance of 29.15 TFOPS at 200W, so we went from 5.38 GFLOPS/W to 145.75 GFLOPS/W in 12 years - a 27x improvement in efficiency and a ~22x improvement in raw performance.
Now let's compare that to the numbers from a decade ago: a GTX 580 from 2010 had a power rating of 244W at 49.41 GTexel/s. A Geforce2 Ultra from 2000 used about 10W at 2.0 GTexel/s. So we went from 0.2 GTexel/s/W to - you've guessed it - 0.2 GTexel/s/W, so same efficiency with a ~27x increase in performance over a decade, though the efficiency is only a guess, since neither GFLOPS nor official power draw figures are readily available for 2000-era hardware.
Fast forward a few years so we can get reliable power draw numbers and comparable performance in GFLOPS, we have the high end GeForce 8800 GTX at 155W for 345.6 GFLOPS in 2006. Ten years later, the comparable model would have been the GTX 1080 from 2016 with 180W at 8.873 TFLOPS. So 2.2 GFLOPS/W versus 49.3 GFLOPS/W or a 22x increase in efficiency and a ~26x increase in performance over the course of a decade.
So during the past 23 years, power efficiency steadily improved, while raw performance increase also showed no signs of slow down in the GPU space. This is given the same generous time frames, to account for the occasional generational leap.
GPU workloads will eventually run into the same scaling limits. That is we will be unable to speed up each execution unit any further, or the primary work we give the GPU will not be able to be split into more threads and accomplish useful work.
So maybe GPUs still have some room until they run into the same problem as CPUs.
Absolutely. The 2006 "A View from Berkeley" is still a great paper [1]. And we still have a long ways to go on this recommendation:
> To maximize application efficiency, programming models should support a wide range of data types and successful models of parallelism
We are still stuck in the winner-take-all mindset when it comes to software development.
[1] https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...
Not quite. NVidia's entry level price/performance has not improved much since 2016. What's skyrocketing is the price of the top of the line models.
High end is now becoming a case of throwing us much money at the problem as possible.
The only real way forward that isn't a temporary workaround seems finding a new type of semiconductor that has lower overall resistance than silicon. Whoever figures out how to dope graphene and produce wafers without defects will probably make trillions.
The energy is a mix of leakage current and active current. Leakage current can be thought of as resistance - it's how much current flows through a transistor that's off. This can be better based on the material, but gets harder with smaller transistors. (Thinking about quantum tunneling as a resistance is good to get intuition, but not good enough to help solve the problem. A material with a lower bulk resistivity will not help here.)
Active current is based on capacitance. Each FET has a little capacitor that needs to be charged and discharged every time the logic is switched - that adds up. Lowering the capacitance of each FET would reduce the energy required to switch it, but generally comes with bad tradeoffs. High-k dielectrics increase the capacitance, all other things being equal. But all other things are not equal, and they are used to create better performing FETs with lower power leakage.
Those are the same thing. Or at least close enough as makes no practical difference. Only an extremely tiny fraction of the power used but a CPU is becoming anything other than heat.
Intuitively, you might think of it like this: to charge a capacitor (or transistor) up to a certain voltage, you need a fixed number of electrons. That number of electrons will always pass through the resistor and generate heat based on their energy. Even if you change the resistor value, its still the same number of electrons, and the same amount of energy.
What does change with resistance, though, is the time over which the power is dissipated. In practice, you have to make sure the resistors are small enough such that you can achieve your desired clock speed.
There are actual resistive losses too, but they're mainly related to power delivery.
[1] https://en.wikipedia.org/wiki/RC_circuit#Time-domain_conside...
To add a bit of troll physics here (but I'm told computers using this sort of principle actually exist), why not then channel those electrons to a boost converter that pipes them back into VCC, recycling most of the current? Theoretically a 99% power usage improvement, minus what the converter loses to heat, and that can be as low as 10%.
Re: channeling electrons - what you've described doesn't quite make sense. Fundamentally, if you're taking an electron at ground or 0V potential, and changing it's potential to VCC, it requires energy that comes from somewhere. The battery (or power supply) is doing exactly that. As the electrons flow back to the ground, the battery "recharges" them up to VCC potential.
What you can do, though, is put circuits in series between supply and ground. That way the electrons flow through the "top" circuit, do their thing, then flow through the "bottom" circuit. There's no free lunch though, as the voltage across each circuit will be reduced. Nonetheless, this is a common technique for low power analog circuits, and one I've used in the past. It's just not practical or worth it in digital circuits like a CPU.
Modern processors are very careful about this, and actively turn off the supply voltage to large parts of the die to prevent extra leakage current. The funny-but-appropriate name for this is "dark silicon" https://en.wikipedia.org/wiki/Dark_silicon
Damn TIL, I never would've expected that. But I guess it makes sense to use a few of the older, larger transistors that don't leak as much to power off a section of the smaller leaky ones while they're not performing any operations.
I understand leakage will go up if we increase voltages to support higher switching speeds, but aren't there still a lot of losses that happen with logic transitions and reduce when the states are stable, even if voltages are held constant?
I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced? I.e. much of the waste is resistive losses while charging and discharging those gates.
Not really. It makes more sense to think about it as filling and emptying capacitors. You are charging the gate capacitance up to the supply voltage, then dumping that charge to discharge the gate to 0 again. The energy of each capacitance that gets charged and dumped is CV^2/2, which happens for each logic transition.
> I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced?
If there was no resistance when distributing charge, it would help a bit, but not enough to change the clock frequency by more than 20%, assuming that the fantasy superconducting-fet had normal leakage and gate capacitance.
I guess I am entertaining the idea of an idealized Maxwell-demon CMOS circuit, if we could bounce the charge between gates with very little work to just pump the charge back and forth.
If you had a lossless bidirectional voltage converter circuit for each gate capacitance, then you could charge the capacitor from the supply and discharge it back into the supply, removing any switching losses.
As the sibling comment says, you would need voltage converters running both ways to avoid this waste.
Asynchronous clockless designs might also drastically cut the power budget but those have failed to find adoption for some reason.
Clockless designs mean that you compute readiness information on the fly instead of having it precomputed at design time. This additional run-time computation is not free, and tooling for clocked designs is good enough that the extra slack they need is often cheaper.
Disclaimer: I'm not a material scientist, so this is probably only partly correct.