> soy is one of the most common allergens
This is, of course, a concern if you're allergic to soy. However, if you avoid everything that anyone's allergic to, you're not going to be eating much. Even avoiding the EU's 14 notifiable allergens, say, would leave you able to eat very little.
> but most normal men would think twice about consuming it if they were told what we currently know about phytoestrogens and their effect.
Does "what we currently know", here, refer to what conspiracy theorists currently know? General consensus is no known effect (from phytoestrogens; some other substances in soy do have known health effects, particularly in terms of drug interactions).
Interestingly, people rarely have the same concern about _cow's milk_, which of course actually does contain mammalian estrogen (there's some evidence that this may be carcinogenic, but I don't think it's particularly strong). One assumes the milk lobby have a bigger advertising budget...