Ding! You hit the nail on the head! To be succinct, if you hear monkey and think black people, then you are a racist regardless of any good intentions you may or may not have.
It reminds me of the ok hand emoji kerfuffle; where it was briefly racist/taboo to use it.
I think it's total madness to let people I disagree with influence my fantasy world games. Why give them that power? Maybe I don't get it because I'm not American and this is a cultural issue specific to the US (and the majority of people here, I assume, are American).