Unlike small, homogeneous groups with intimate hierarchies, modern society is huge, the data available on your life is enormous, and the ability of someone to hurt you with that data (be it through impersonation or accusation) or indeed you to hurt someone else is virtually unlimited. There's no discussion of the ways in which the potential consequences of the violation of privacy make us more protective of it ("The Lives of Others" is a great film on the subject).
Maybe at some point we can achieve a society where the power structures and human relationships are such that we can again be as open again with strangers as we once were with our immediate families/communities, and in so doing relax about the need for strict rights of privacy. But right now the stakes are too high.