What do I need to learn from the practical side of things as well as from the theoretical side?
Where do I learn them? Which books or courses?
The Ian Goodfellow book on Deep Learning is good:
https://www.deeplearningbook.org/
The Stanford CS231n course videos are a good resource:
https://www.youtube.com/playlist?list=PLC1qU-LWwrF64f4QKQT-V...
Depending on your math background you might need to bone up on some stuff. A certain amount of Calculus knowledge helps, and likewise for Linear Algebra. It probably wouldn't hurt to have some Probability theory as well.
That being said, if all you care about is learning so that you can build on your own, and you can pay for the computing resources yourself (don't underestimate that part!), it should be totally possible to go your own way, provided you have a decent background in math.
I'd start with some of the basics like the Hastie et al. ESL and the Goodfellow DL books, if you haven't already worked through them, and then look at the papers at the latest conferences. Read them, look up the papers they cite and the papers that cite them, basically build up your Google-Scholar-Fu.
Depending on how motivated you are, I reckon it should be possible to learn how to build something cool (not necessarily academically publishable) quite a bit faster than the typical 4-6 years to get a PhD. But don't forget that in DL a lot of the edge comes from having access to gigantic data sets and the infrastructure to train 1bn+ parameter models on massive GPU clusters.
Practical Deep Learning for Coders is also a good place to start: https://course.fast.ai/