Matplotlib is terrible (once you get to the point that you need to do non trivial things with it), that's why you use the excellent plotting library for python called "Bokeh"
(I think matplotlib has an openGL backend that slightly improves but doesn't fix the issues with matplotlibs handling of large datasets)