This is a classic logical fallacy I see too often. It's like mentally creating a Venn diagram of "People who said a" and "People who said b" and then just assuming that the two circles basically entirely overlap - without any basis.

I feel like it's somehow a variation on a straw man argument, but I can't work out if that's a good enough analog. Would be interested to know if anyone knows of a more formal name for it.