Discrimination based on caste is different than discrimination based on race, though. There is probably race based discrimination in India too, and it might intersect with caste, but conflating both probably doesnt really make sense.

Also, I dont think people really ignore people of color being racist within their countries. Most of the time when people bring up arguments stating that the people being racist are white people, its in the context of western countries with a colonial/slave trade past where systematic racism doesnt target white people.