I once used Sympy to parse formulas written in infix notation and convert them to S-expressions. But that's just less than 1% of what Sympy can do. I'd like to find more uses for it.
I'm a weensy bit biased; I probably took part in conversations around the plotting interface as it was taking shape, and got familiar with that before learning matplotlib in earnest.
What happened is that in 2006 I hired Alex Clemesha to implement a "Mathematica-like API on top of matplotlib for Sage", and he spent a year doing so. Then Robert Bradshaw (lead dev of Cython) and Carl Witty wrote some really sophisticated just in time compiler code to make it more efficient to evaluate symbolic expression at many floating point values, which helps in plotting (especially for 3d plots). Over many years after that other people filled in little missing pieces of functionality and exposed more of Matplotlib's functionality, and also just added a bunch of new functionality (e.g., for drawing gridlines, 3d implicit plots, etc.) that wasn't available in the Python ecosystem.
And that's Sage's plotting functionality...