> I imagine for shorter men, it's the inverse but equally bad.
Not really, as a quite short guy, many shops will offer me to have the clothes fitted, and if not it's pretty trivial to fit them myself. Maybe on the most extreme end of short it's more of an issue, but in general I suspect shortening pants and shirts is signficantly easier than lengthening them.