> I've never worn one
I live in the southeastern US. It's quite hot here, for quite a lot of the year.
Ever wonder why seersucker is Hollywood shorthand for "small-town Southern lawyer"? It's not just Atticus Finch; that's what people wore before air conditioning was widespread. Seersucker is breathable, cool, and naturally wrinkled in such a way that it doesn't really show. Linen wrinkles more obviously but is far more comfortable in a hot climate than even the best cottons (like seersucker).
I can wear a linen suit and shirt in New Orleans in July and be no less comfortable than I would be in a T-shirt and shorts. I might ditch the coat, but that's it.
If you don't like how they look, well, that's a matter of taste, and de gustibus non est disputandum. But they don't have to be heavy wool.