But, they do. Any electromagnetic wave can refract.

https://en.wikipedia.org/wiki/X-ray_optics#Compound_refracti...

3Blue1Brown has a very nice series on optics: https://www.youtube.com/watch?v=Cz4Q4QOuoo8