But normalizing for power reduces the GPU advantage even more! Haswell for instance achieves about 5.3-5.8 GFLOP/W, compared to 24-28 GFLOP/W of Maxwell. That's less than a 5x theoretical computational gain.
Anyway, Cortex-A15 is capable of 8 flops per cycle per core which puts it pretty good in theoretical efficiency for its likely power draw at current clocks.
And I never managed to get close to 8 ins per cycle on A15, but, for example, an FFT implementation on VC4 is pretty close to a theoretical performance limit. And a fully loaded 4-core A15 will draw far above 500mW anyway.