I would get scowled at using that term when I first moved out of SA. It is referring to a specific group of people: http://en.wikipedia.org/wiki/Cape_Coloureds