Yes, there is! The fallacious belief in a company "culture" that is signified by what candidates wear isn't benign, but it isn't the worst implication of "culture fit". Intolerance of normal work/life balance, of different mentalities about software development, of comfort levels with intoxication, of different management styles, or just of meaningful diversity --- those "culture fit" pathologies are much worse. And if the company you're thinking about working for filters based on candidates wearing suits, it's more likely than not that some of the more malignant pathologies are also there.