I mostly like the informal dress, but have been told more than once (by executives) that it makes me appear like someone who cant be trusted with authority. When I dress up, many of my ilk feel uncomfortable around me.
Wear what you like, but I have learned time and again that whatever clothing choice you make, everyone is always judging it and making decisions based on that judgement.
I'd consider that a cloistered environment, when you are judged by others on physical appearance. It really sucks, but I also worked hard to make sure I have my own authority. Would you still trust Richard Stallman, wearing a soggy full-size panda-pyjama suit and carrying a small stuffed pink elephant while singing "bye bye blackbird" quietly to himself? Probably!
I take your point that where I work might be considered a cloistered environment, but the comment about people judging your appearance is a world wide phenomena, not something that lives in one workplace.
For example:
1. Recently HN had an article about a younger guy who added a fake beard to his freelancer profile, and noticed he received more gigs as a result.
2. Attractive people get more opportunities in life http://www.businessinsider.com/beautiful-people-get-more-job...
The average callback rate was 30% across all of the CVs sent out. For attractive women, it was 54%, and for attractive men, 47%. Unattractive women had by far the worst results, with a 7% callback rate. Unattractive men had a 26% rate.
I could go on, but this is not a me thing, its a we (people) thing.