I hate wearing "fitted" clothes and shoes...god how I hate shoes...I have spent the last several years of my professional career wearing flip flops or sandals around the office.
I have never worn pants suit, in the winter I opt for a hoodie, tshirt and jeans - and generally substitute the jeans for a skirt in the summer.
As a result, I am biased towards people who wear suits (especially with ties!) - The question in the back of my mind is "what are they trying to hide?". (It should be noted that these observations are from corporate land where I no longer reside.)
I have met people who believed that simply because they were in a suit they should be listened to...even when it was clear they had no knowledge about the situation at hand - this tends to apply to project managers and the like.
When I met somebody in t-shirt and jeans I tended to get along with them, they tended to be knowledgeable and could stand their ground.
I have since moved out of corporate land, and into an environment where tshirt and jeans is the norm - and these people "get shit done" - their career development and self esteem comes from the fact that they are the best at what they do...not because "I look good this morning"
You shouldn't get respect from the clothes you wear...you should demand respect from the skills you have.