What do you mean we tolerate? If you mean criminal behaviour we don't really tolerate that. If you mean kids or disabled I think I've heard it justified that we have a contract to their parents that we still respect their rights.
For disabled I guess you could say it's a contract with the rest of society? Because we don't like the idea of treating other humans below a certain threshold. Or.. reasons I guess haha