I view it much like having a conversation in the hallway or by the watercooler. It would be creepy and weird, and unexpected, to have microphones and cameras all over the office so the boss can record your conversations and eavesdrop whenever they like.
It also screams micro-management if the boss needs to drop in, even if you are just using it "for the purpose of doing [your] job."
I tend to feel that if you trust your employees so little, then you should just fire them. Conversely if my employer trusted me so little, I should probably quit.
Where I would probably make an exception would be if one of the employees filed an HR complaint or something, then I think looking at the existing record would be fine. But, it should be limited in scope to that incident.