That's his point; using your social circle as a litmus for believability is just going to exclude everyone who isn't like you.

It's a great way to arrive at discriminatory conclusions, and it's a great way to cause harm to people who aren't like you.