But they do care! If you wear a suit they'll think you're not technical like them. A big part of Google culture originally (when it was unusual) was wearing hoodies and jeans rather than chinos and shirts. If you didn't fit in with that you made yourself look like an outsider.
And the unfair bit is that it's not written down anywhere. You just have to implicitly know it. If you don't have family or friends in the industry you won't know it, they won't tell you, and so you won't fit in.
I've seen tech people scoff at people in suits, and look down at them. Have you never seen that?