Most of those CPUs are designed by CS or EE students taking a computer architecture class, so... in a sense... one can argue that defending against Spectre-like attacks is actually super simple: just use a simple CPU design.
To actually become vulnerable to Spectre, you need a very complex CPU design, so in the same way, it can be argue, that making a CPU vulnerable to Spectre is actually hard, since it takes a lot of work to create such a CPU design.
Now, if what you want is a CPU that's both fast and secure, then I'm sorry to tell you that such thing cannot exist. Those two goals are at tension. You can either get a F1 or a tank, but no vehicle that offers the same amount of protection as a tank is going to be able to compete against a F1 car, and vice-versa.