Thanks for the reference to pants. In US pants covers jeans, chinos, slacks. Shorts are well, shorts. Underwear is our boxers, briefs, tighty-whiteys, etc.
Shorts are mostly just shorts, but we do have jorts and chorts depending on how hipster your shorts wearing cohorts are.