With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation
I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.