I can’t be bothered to check the exact assembly to see if timing attacks are possible.
But crypto really is just a small thing. I’m not going to design a nuclear reactor but I’ll definitely do stuff that depends on the electricity from one. Similarly one can still do their own authentication; its not like even Apple forgot to rate limit…