For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics.
It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).