I think the invite only beta hurt it more. I got in from a random redditor, but no one else I knew was in it, so I played with it for 5 minutes and then forgot about it.
I don't think the closed beta-ness of wave really hurt it. If wave had actually worked well, then maybe there'd have been a point where it was worth opening up to many users - but wave never got there, so it's a moot point.
Google+ falls down for mainstream users, though, which is where the "no one here" perception comes from.
If memory serves, Google Wave was the same way: great for the people who used it, but the value wasn't perceived by the majority.