Who, precisely, should decide who shouldn't get the same rights as the rest of us?

Maybe we should have some sort of system to decide this on a case-by-case basis.