"Leaves branch prediction and speculative execution up to the software" means that in practice you need to either use the Intel compilers or hand-optimise your software to get the benefits, while simply recompiling your legacy software with GCC or LLVM ends up being disappointingly slow.
Intel could have solved this problem by contributing to GCC and LLVM instead of keeping their optimizations proprietary. Then they would also be more easily auditable as well.
There were no magic secret optimizations to release. It just straight up did not work. They had to add back dynamic branch prediction, and even then the load store latency was such trash that they had to put ginormous L3 caches on it to get even close to reasonable performance.
Yeah, compilers today are no better. We found the limits of statically scheduled parallelism pretty fast. On code that uses static scheduling, a modern OoO processor can easily duplicate what IA64 was capable of (and a pipelined loop using AVX will utterly smoke it), while being far better at all the stuff IA64 failed at.
I remember going to a corporate presentation by Intel about the revolutionary CPU architecture. They had to turn the machine off because the cooling fan was making so much noise we couldn't hear the speaker.
And this at precisely the wrong moment in history, as mobile devices, with their stringent power consumption constraints, were becoming popular.