Because then every minority can vote whatever they want. A few of your neighbours can vote to kick your family out of your house. A few rival supporters can vote to kick your team out of your favourite league. Some students can vote to kick your kids out of college because they do not like them. If we let anyone to invent polls and votes that affect the majority of people then nonsense happens.