These vulnerabilities can jump process address space boundaries. It's a lot harder but can be done, look at the original Spectre paper: https://spectreattack.com/spectre.pdf