>> should the average man on the street really decide? <<
As opposed to who? We had only land owning white males voting early in the Republic. They were educated and well acquainted with the issues. They voted their interests, which was those of land owning white males.
Limit the election to whatever group you prefer, and they will vote for their own interests.