The major issue I've had with wp is the speed. I don't know if it's a php problem or a wp problem, but running wp on shit-tier commodity hardware results in a complete inability for your site to be {hn'd/slashdotted/dugg/whatever} due to the ~500ms page generation times (that only increase with any plugins at all). The caching plugins mostly resolve this, but still not well enough to be accessible during a wave of pageviews.

It seems like unless you're willing to pay for managed wp hosting (or fancy hardware), you can't expect more than 10 pageviews/s to be handled. This makes me sad.