Seems like speculative execution is just fundamentally insecure. With SPECTRE/MELTDOWN mitigations, doesn't CPU performance drop below the same CPU performance with no branch prediction at all? Should we move back to CISC? Or maybe VLIW?
Memory is slow. Insanely slow (compared to the CPU). You can process stupid fast if your entire working set can fit in a 2KB L1 cache, but the second you touch memory you're hosed. You can't hide memory latency without out-of-order execution and/or SMT. You fundamentally need to be parallel to hide latency. CPUs do it with out-of-order and speculative execution. GPUs do it by being stupidly parallel and running something like 32-64 way SMT (huge simplification). Many high-performance CPUs do all of these things.
Instruction level parallelism is simply not optional with the DRAM latency we have.
But I was just really trying to point that in-order cpus are still around, they did not disappear with in-order atom.
I dont think its inevitable, might be caused by greed. You could have small separate sections of cache (or additional tag) dedicated to per thread speculation, but no designer is willing to sacrifice real estate to something that will be thrown away the instantly.
No. Processors with no branch prediction would be many times slower (up to dozens of times slower on workloads that don’t fit in cache)