It's a joy to set up once you know what you're doing
Which is (in my own long and active AWS experience) another way of saying "You need to make a significant investment in time, effort, and tooling to get to a point where it is fast and simple to bring up a reasonably complex environment. You are re-learning domain knowledge you likely already have, just so that you can use Amazon's product. This in-itself isn't a bad thing, depending on what you get back for it, but the return value is "I now know what I am doing with AWS".
performance is more than adequate
I can get more than adequate pretty much everywhere. I am paying a serious premium, so I really want to be in the amazingly fast bracket, not the adequate.
Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise.
Yeah, they are, both directly as well as in terms of value for money. There are many ways to get increased performance at lower costs with reduced complexity (complexity also cotst real money). I change hosting provider roughly every 12 months to take advantage of investments in new generation hardware and good deals, and everytime I do so, I check out new features, updated pricing, and do the sums to work out value for money (I offer high performance, managed, HA hosting to selected clients)