Gravatar support is, for example, a feature I'd like to get from my OpenID provider. After all, my image belongs to my identity.
Next, with a single identity comes also privacy concerns. Automatically generated disposable E-Mail addresses for each new sign-up would be nice to have, in my opinion.
Privacy concerns, however, are also related to the most common business models of these providers: It seems, nearly all of them want to display or sent me advertising. Or they want to be able to improve their advertising displays.
Can't they just ask for, say, US$29 or US$39 a year?
Then, I'd also need some sort of easy Identity transfer. If I switch providers, for example, the new one should be able to ask the old one for all the data and notify the consumers about the switch. Otherwise, the transaction costs of a switch would be rather high, exposing me to monopoly problems.
Browser support would also be nice to have, so there's a check that I'm being redirected to my true ID provider, not just something that looks like it.
Maybe, this stuff is too sophisticated for a sufficiently large number of people. But maybe, it works with the right sales pitch.