http://en.wikipedia.org/wiki/Gramophone_record#Stereo_sound

While the stylus moves horizontally when reproducing a monophonic disk recording, on stereo records the stylus moves vertically as well as horizontally. In fact, prior to the full development of the 45/45 system, the first stereo cutting heads were made by bolting together one lateral cut head and one vertical cut head sharing a common stylus holder. Feeding the driving coils with suitably phased material achieved the 45/45 groove.