Banning visas from a list of failed states was not racist. You can say the rhetoric surrounding it was, but not the policy itself.
Rhetoric in many cases informs policy and vice versa. You can't just say they exist in a vacuum when the reason why the visa ban occured was on racial pretenses.