The incredible success of a small group of Hungarians from one region and time has sometimes been attributed to the unique social situation in Hungary then. Part of which was linguistic. In the case of the Hungarian Jews, they were usually Hungarian speaking at home, and often from families of German or Yiddish-speaking background further back. Usually educated in German. Some were educated in Hebrew at religious school. In the case of Hebrew, Hungarian and German, that's three radically different languages, about as unalike as three languages can be. Maybe that has an effect on the mind?
Though as I recall, von Neumann's family was non-observing. So he was fluent in Ancient Greek by the age of 5, instead. So I'm not that strongly wedded to the hypothesis. Maybe they really were from Mars, as Leo Szilard once quipped.