You can call the Roman Empire many things but to call them racist is very anachronistic
It almost always targeted at the tribal Anglo, Celtic or Germanic peoples. And in these circumstances it was really an insult at their style of government rather than their ethnic identity.