Those "fringe" racists make up a large amount of whites in the USA. Likely some of your friends, bosses or partners hold those same beliefs, would act in the same way as they would, and feel just as attacked as they do.