That is a myth. These defenses you talk about were built partly with cheap material and therefore significantly less effective. This caused a major post-war corruption scandal.

Nazi Germany did not invade Switzerland, because Switzerland was actually more useful as a neutral state for Nazi Germany to exchange gold for commodities, for spying and Switzerland produced militiary equipment for Nazi Germany anyway.