Many racists were probably raised that way by society, too, and undoing that is probably both hard and unfair - but necessary and a reasonable prerequisite for being a part of polite society.
"Don't be an asshole" is what? "Being an asshole" means different things to different people. THAT is the core of the issue.
For instance: I've got coworkers, who if they screw something up, I can just straight up say - that was wrong, do it this way next time, then we won't have an issue. For other coworkers that direct interaction would be the absolute end of the world. And instead you have to tap dance around until you eventually get to a state where you're fairly certain they know not to do what they did again.
Would it make me an asshole for not knowing one person can handle direct feedback, and others can't? Is it "being an asshole" for telling someone factual information without sugar coating? I don't think so, plenty of people out in the real world do, however.
And, quite frankly, most of the people who want direct feedback consider people who tapdance assholes because so much effort is wasted on getting to the point.
A racist may live in fear of the criminality of (insert race here) without seeing the bigger picture of poverty that causes it. A racist may have a majority of bad experiences with (insert race here) without realising their own prejudice was to blame.
Racism isn't "being mean", racism is potentially diverse set of prejudices and overly broad opinions that are founded on faulty logic or incomplete data. Racism can be propped up by an entire web of social beliefs and a lifetime of personal experience. Racists don't think they're "being mean", they think they hold sane, justifiable opinions against other "mean" or even "inferior" groups of people. Indeed, it may "not even occur to them they are being offensive".
The long and sordid history of racism around the globe can attest to the fact that racism is not something "easy to nip in the bud".
As you say, you can stop someone overtly declaring their racism, but that doesn't mean they won't still covertly (or even unconsciously) continue with racist behaviors. Which is why it isn't so "clear", because even a simple situation with coworkers, you can be facing an uphill battle.
Anyway, I would wager we both broadly agree and this is now just a semantic argument.
It doesn't make you an asshole (intent) not to know that, but it does make your interactions less effective to not work "with the grain" of that dynamic.
At our company everyone takes the Birkman and we train everyone on how to leverage the knowledge it provides on your personality style and how that intersects with others' style. Whenever you're having trouble in an interpersonal interaction with someone, you can think about / looks at the profile and tweak your approach to improve things. It had been incredibly effective for making it easier, more effective, and more pleasant to work with others.