I doubt there can be refutation in the sense that "its conclusions are obviously wrong".... the catch is how you interpret/act on those conclusions.
Yes, stereotypes become stereotypes because they are, or at least recently were, "largely true" (in the statistical sense - i.e. not always but "too often to ignore"). E.g., in my country, a lot of the petty criminals are gypsies - it would be foolish to reject that truth. However, the stereotype "gypsy = criminal" is still incredibly harmful (despite being grounded in truth), both at the individual level (producing harm to innocents) and at the group level (doesn't solve the problem, only contributes to making it worse).
The bigger problem is that people are ill-equipped to handle these thoughts in the brain simultaneously: that a stereotype can be largely true, and yet it's harmful/a bad idea to act instinctively based on the information it provides. The correct responses to a stereotype are complex/should be based on reason - those based on instincts/reflexes are very often wrong. But it's so hard to resist the instincts, that often it's simpler to just dismiss the stereotype as "false".