Whatever you want to say about the ethics and experience of wearing some kind of pants in public, that custom has been part of humanity for at least hundreds of years, if not many thousands of years. (I don't know the entire history of humans wearing pants)
There is no human on Earth who grew up and didn't experience some kind of pants as a normal thing. We didn't all wake up one morning and suddenly get forced to wear pants. Pants-wearing is all we've ever experienced. Whereas unless you're from some parts of Asia (I believe), wearing masks has never been an ordinary part of your life experience. And even there, it was always a courtesy and not a requirement. Being forced to wear a mask to buy food is a new thing, and certainly a restriction on human freedom.
And as a further disagreement with your analogy, pants play no role in restricting many useful things like human communication and childhood development, whereas masks do negatively impact many aspects of life. I live in an immigrant neighborhood and I can't understand many shopkeepers garbled English through masks. And what about people who are hard of hearing and need to read lips? And how are kids expected to learn normal communication skills if they can't see body language like smiles and frowns and learn to interact with other humans? The cost of masks, unlike the costs of pants, is considerable.