I think there were a number of factors also, including economic and others, but I don't know that hair and eyes were ultimately very big ones - since those didn't seem to help surrounding areas like Austria and the Netherlands. The map of the territories at the end of WWII just amazes me with Switzerland standing out almost like an island:
https://en.wikipedia.org/wiki/File:Second_world_war_europe_1...
There was also a report in the mid 80s penned by, among others, former General George S. Patton (who was active during WWII)... "In short, Switzerland is an armed bunker." (PDF page 8, https://docs.google.com/viewer?embedded=true&url=http://... )