I believe we would be better off as a society if we either decided it was ok, or not ok and not situational.

In a perfect world we'd just all decide to be better people and not racist without external pressure, but that seems like a pipe dream.

There are plenty of spaces where you can be openly anti-black and anti-semitic without consequence. That you can't do so in, say, a black church or a synagogue doesn't mean white Christians don't have the same rights as black people and Jews. Try to be anti-white or anti-Christian on any "free speech" platform or many places in the American deep south and you'll definitely face consequences.