No, it's not a transverse wave in the sense that you're describing, like a wave in water. It's a wave in the E&M fields, both of which are 'vector fields'. Mathematically, we describe both electric and magnetic fields as a vector (a direction + length, like a little arrow) at each point in space. Light waves are oscillations in the lengths+directions of these vectors. There's no medium moving transversely, but the vector field points in the direction and with the magnitude drawn in those squiggly line diagrams.
See Maxwell's equations, which describe how those vector fields change over time. They do not describe how any sort of medium moves over time.