While I agree that the saintly thing to is respond without bringing up your persecutors' race, I am pretty sympathetic to people who have been the victims of overt racism seeing some things in terms of race. They've had their faces rubbed in it.
And I'm sorry to come down hard on you, but it's pretty arrogant to tell someone who is still suffering the consequences of being harassed because of his race not to take notice of the race of the people harassing him.