White people who want the ability to say the n-word often use a similar argument. ("It's only problematic [when/if]...") .
And I'll take the bait. We avoid the N-word because it is a pejorative, intended to diminish its subject. But no one uses "Master" as a pejorative the same way they might use the infamous N-word. The word has a much longer history, wider scope of meaning and wasn't invented as a pejorative (rather it is a compliment!). It is very possible to use "Master" in a sense that it isn't possible to use the "N-word".
Would you provide me with an objective definition of "problematic", and if I find other common words that meet that definition some point in the past, in any country between any group, will you stop using them?