OTOH you have a living proof that an amazingly huge neural network can work on 20W of power, so expecting multiple orders of magnitude in power consumption reduction is not unreasonable.
Should be able to go more efficient as the brain has other constraints such as working at 36.7 degrees C etc.