Even if that wasn't the case, the argument can simply be "why contribute even more to the military war machine than I already have to," which isn't at all an unjust argument.
This argument also presumes there's no barrier to simply changing residency of country, when in fact, it can be very difficult to get a visa, let alone a job, for a militarily neutral country like Switzerland.
One can usually choose not to work in a military-related occupation and probably not face any major repercussions. However, if one chooses not to pay taxes, one may soon find oneself without the ability to earn an income of any sort, for starters.
Everyone has limits to what they feel they can do in protest.
I don't particularly think it's all that great that childless homeowners pay school taxes on their property, but I accept the reasoning for it.