Do you really need photons and quantum stuff though? "Just" learn about how dielectric materials work + how EM waves work from Griffiths and then everything will make sense.
I'm still confused. How is it an inaccurate simplification to state that the speed of light in glass is lower than in a vacuum?