It is a lot to learn at once, but no more than web dev where you are learning html and databases and HTTP and god knows what else at once. And like you said once you grok it, it's pretty easy.
Graphics programming terrified me for the longest time because people acted like it was dark magic and hard. Figuring this stuff out slowly but surely it shocks me how little there is to it and I wish someone would have told me sooner.
I've gotten rendering a cube down to an art (gotta start somewhere). You're talking ~100 lines of JS and double that of C++ but no more than that in any decent language. And that's a cube (or multiple) and a basic camera.
A library like regl shows how simple this stuff can get without lots of boilerplate.
https://github.com/regl-project/regl/blob/gh-pages/example/b...
Boom a triangle in 40 lines, but the underlying process of vertices, buffers, shaders, etc is apparent. Cool stuff.
Those asking about Vulkan - Vulkan only increases the boilerplate by design.