I grew up being taught that cotton was best and for decades wore it almost exclusively. But something I have realized over the years is that it is also much, much heavier than polyester or nylon blends, especially when it's wet.
If your shirt can't dry overnight then you're forced to buy more shirts just to handle the washing/drying overlap, which is wasteful. Not to mention only owning cotton makes doing anything outside in the rain require more calculus around how many days you will need to dry the clothes that got wet, so now you need to own overlap clothes for still-wearably-clean clothes too. Even worse, if you're in a cold environment then you likely can't keep wearing your wet clothes as they dry, because wet cotton loses insulation, making you feel much colder, so there's more incentive to change even before the end of the day.
Wool is a better option than cotton if you want "natural" fiber that is still wearable when wet, but that comes with the moral guilt and ecological impact of animal husbandry, so I'm not sure if it's better or worse than plastic, which at least is only made from animals and plants that died millions of years ago.
Personally I have chosen to keep cotton for underwear and tank tops for comfort, and also because I anyway own 7 of those so there is enough overlap for them to dry over the weekend when they get wet and I'm in a cold and/or humid climate. But for pants and shorts where I only own two of each (and only bring one of each when traveling) then polyester/nylon that can dry overnight in most any climate is more practical. Socks and long sleeves I've decided to go with wool despite the animal cruelty because it's the most practical. I only need a couple pairs of socks and a single long sleeve to get by in a warmer climate, and I have a polyester coat for colder climates.
It still feels a bit high impact, but as you say, most stuff lasts many years so I don't feel too bad about it when I see people buying more stuff than I have in my entire wardrobe on a single casual shopping trip.