The FFT is rarely the only thing you're doing, so at the very least you get to keep the data local if it was already on gpu.