How many people do you encounter? In which parts of the country? How much do you know about their political views? How do you know whether someone is a Nazi if they don't announce it?
You could have met plenty and not known it. Or your sample size of acquaintances doesn't include people who frequent 4chan.