If you want to learn the minutiae of game programming, I recommend getting a book that goes over all the relevant maths, in a practical way. I have "Mathematics and Physics for Programmers", and it's pretty good. (Do not be fooled by the title, it's a very hands-on game maths book.)
Learning about OpenGL, the graphics pipeline, complex transformations and all that is fascinating, but I'd put it off for now (most game math books, e.g. the one above, do cover it in the later chapters). Back in the day, I started by making really simple 2D games and worked my way up from there. 2D game development is very similar to 3D, just easier - in regards to both programming and game design.
I can recommend SDL 2 to anyone who wants to make 2D games. It now supports hardware rendering and Android/iOS, and generally doesn't leave much to be desired (like SDL 1, image loading, font loading, audio, joystick input etc. are all in there).
The second thing I'd do is participate in a game jam, once you've had time to mess around a bit:
http://www.ludumdare.com/compo/ http://globalgamejam.org/
If you need more information there are communities out there. Here's a short list:
http://gamedev.stackexchange.com/ http://forums.tigsource.com/index.php#2 http://www.gamedev.net/page/index.html http://forum.unity3d.com/forum.php
Using OpenGL 2.0 with some simple "Draw yourself" objects for sprite rendering works nicely--it'll be tricky enough to teach you something, but not hardcore enough to sidetrack you from making a game.
SDL is C, has lots of language bindings, is well-tested, and has lots of extensions (of vastly differing quality).