The CPU would "just" need to look at all executable pages to find binaries of common AES implementations, and when it finds one it could wait for the key to be loaded and then exfiltrate it. It could also detect (although at greater effort, possibly much greater) when you're using it to compile AES and inject spying code into the output, even if the target is a different architecture.

If the attacker has access to your computer, you've already lost. If the attacker built your computer, it was never even a fight.