Cherrypicking a single summary statistic gives a far too rosy impression of past sexual/family culture. Ultimately if you go back far enough the original purpose of sexual mores related to marriage were primarily about clarifying and maintaining patriarchal land inheritance and designating women and children as chattel of the male head of household.
Are you possibly suggesting that abuse never gets inflicted on prostitutes, and that reducing them to human chattel, as their bodies are bought and sold, has improved their lot?
The old pattern might have had less infidelity (though I wouldn’t be surprised if available data about past prevalence of infidelity is inaccurate and unreliable), but it also had many other severe social problems, some of which have improved substantially between 1970 and today.
A culture (and the laws, mores, economic relationships, etc. which enforce it) isn’t a simple system that you can poke in a particular way and get a clear and simple result out.
I won't judge a body for being a brony, or a never-nude, or a benign cultist, or a thousand other things. I don't expect normalcy from anyone.
That said, there are unqualified evils in the world. Things and acts that slowly crush and rend human lives, making the world a darker place, more filled with despair, jealousy, and hatred. Slavery, including debt slavery, is one. The sex trade, the topic at hand, is another.
To not speak out against evils of that scale and consequence is to side with them.
Changing topics back to your quibble about infidelity: it's also evil, if on a more personal scale. Where there are vows of that weight and import, abandoning them unilaterally is ruinous to yourself, your family, those who know you, and many that you will never know or meet. It may not be a matter for the law, but it's not a silly little quirk to smile and wink at.