well, that's fair. I'd say I don't really care about my appearance. So?
I look at the young guys I work with now getting all excited about vendor gear, smile, and reminisce about being the same age, but to me shirts are worthless (and as noted a stigma in some situations).
(or replace appearance with fashion)
Unless of course that person is someone of authority over you - then you are screwd.