Of course, if you're an hourly employee and don't (as you mostly shouldn't) have non compete clauses of some sort it's mostly fine. However, as a salaried employer who generally can't also work for a competitor, it's pretty scummy unless it's something unrelated and you're putting in your hours--which it would be hard to do. I'm sure there are "life hacking" people who think this is fine but it's really not.

And if you're anyone other than a low level drone of course people know who you work for and you're publicly visible.