I'm the same way, but it has struck me that this is a kind of inverse brand fixation. If I were indifferent to brands, I wouldn't mind that my clothes happen to advertise them, but I do. I think on some level I view highly visible brands as crass, which is to say that a lack of branding is itself a class or status signifier.