It depends on whether they realize the error of their ways and have improved. If not, they shouldn't be around women.
I mean, I don't want to stigmatize people and I want to believe that people can change if confronted with the error of their ways, but I've seen too many instances where it just didn't work like that. If they are still behaving inappropriately in the workplace, they should not be in the workplace.