"People worried about womens' rights in the early 20th century should have realized that Harvard was a private university. If I held a party at my house and didn't want to invite you because you were a woman, you'd get laughed out of court if you tried to sue me over it. People should be allowed to organize under whatever principles they want to organize under. Disallowing Harvard to use whatever criteria they want to use to admit students, unless of course they breach free speech rights, is ultimately decreasing the amount of intellectual diversity in the world. Women that don't like Harvard can apply to any other college. There's no shortage of them to choose from. But if you tell Harvard what they can and can't do, you run the risk of destroying the individuality of Harvard."

"People worried about reproductive rights should realize that Hobby Lobby is a private corporation."

"People worried about gay rights should realize that Chick-fil-A is a private corporation."

The choices of private organizations and individuals have huge effects, and it's reasonable to criticize them if we disagree with their actions. But on the other hand, I agree that it's important to give private organizations freedom. This is a difficult trade-off; we need to consider both sides of it, and whatever principle we decide on, we should apply consistently. So unless you'd also defend historical gender segregation at private universities, or defend Hobby Lobby and Chick-fil-A, you should find a different defense of Harvard's actions.