(Colours from memory, I think that's right, they were certainly different anyway.)
Point is you can certainly learn not to treat contractors like employees, have badged access, etc., without having such hostility attached to it.
Edit: no! Red actually was a 'badge of shame', that was 'I forgot my badge today and had to get a spare from reception'. Anyway, it's beside the point exactly what was what. Different badges and access/treatment don't have to bleed into social treatment, they don't even have to be that visible.