> If just visiting a webpage with some JS will let them do ACE on even 0.1% of visitors

Spectre is not an arbitrary code execution kind of attack.