Sounds like a really intriguing idea but why? What exactly do you want that up voting to do for you?

It could be a way to ensure that emails from contacts with higher votes are prioritized. Is that it?