Swiss taxes vary by location but are generally not higher than the US taxes. So it's not about paying more, it's about wasting less.
But more importantly, Switzerland isn't a NATO member.
(..and yes, only men are conscripted. Switzerland is a great country, but there continue to be problems with sexism, even today)
https://en.wikipedia.org/wiki/Military_budget_of_the_United_...