Sport coats evolved from riding and shooting jackets. I suspect some mobility has been lost over the years, but neither of those sports requiring raising your arms very high.
Then our most popular outerwear shirt, the T-shirt, used to be underwear with a strong military association, and came into common use the same way much of the rest of our modern fashion did: youth culture, which AFAIK is also how we got the blazer (dad's orphaned suit jackets) and two-button and especially two-roll-three suit jacket styles (kids wearing hand-me-down 3 and 4 button suits and orphaned jackets "casually" with fewer buttons done up).
Yes, but also the collar is buttoned down because those shirts don't have bones and are not starched.