I think you can interpret "like you" in a few different ways. Someone can be "like you" if they share common values, even if they have completely different background, interests, personalities, etc.
I don't want to speak for him, but I believe Sam's point is that those are the people that you forge the strongest bond with.