The distinction is that people who disagree with us are not out to do us harm.
We can disagree and work together.
Trust is a nuanced subject, which is the very fact the "Us vs. Them" attitude ignores. If we clump everyone who disagrees with "us" into one group, we turn a prudent critical attitude into blatant distrust.
It is important for us to be critical of ideas, but not of people.