That's a feature IMO -- HN could even make them even smaller.
It should take conscious effort to vote, and the small target size mentally reinforces this model; that it's not supposed to be the most common action.
EDIT: See, clearly the vote targets simply aren't small enough! </s>