The problem is that you are indeed part of a tiny minority - but that isn't a problem in practice!

Properly sorting stuff is hard. You can't just look at whether something is vaguely plastic-ish or paper-ish, you actually have to inspect every single item for the recycling label. There are dozens of tiny gotchas to stumble over.

Luckily we don't actually need to do all that. There are automated plants which can take in a mixed domestic waste stream, and separate out things like metal and the various forms of plastic. This means you no longer have to worry about people sorting it incorrectly, and you no longer have to worry about people who can't be bothered to sort at all. It's a win-win, really.