Memory bandwidth keeps the cores from being bottlenecked, but saying it is the reason GPUs are fast is simplistic.