Depends how deep you want to go. But I'd personally suggest going through past courses from someone in the field. For example:

https://igl.ethz.ch/teaching/shape-modeling/sm2022/

is a course that Olga Sorkine-Hornung taught in the Spring. Seems to have some decent course titles, but apparently the course slides are only available at ETH itself... (anyway, find some other course and dig in!)

The nitty gritty math of convex shape overlap tests is super interesting but unlikely to help if you're interested in Graphics programming (for the most part). Theres a few books on game development from the late 2000's like Real Time Collision Detection that cover them, for example.

If you're interested in Graphics programming it's probably the wrong path to go down, and I don't have any resources I can provide there sorry. I'm not a graphics programmer at heart, unless it needs to be done!