That’s not a big deal on a multi-core 1ghz processor. Much bigger deal on a single core 200mhz microcontroller that doesn’t have branch prediction or any of those fun optimizations.
I know you're pointing at the general class of hardware performance enhancements including branch prediction, hardware prefetching, superscaler execution, pipelining, etc. But out of all of them, if you get a speedup for any advanced branch prediction beyond just always assuming branches are taken then I'd be very worried your cryptographic algorithm is broken.