You're free to know me, but not where I live. It's not hard to see the difference in types of personal information.
Don't get me wrong I would absolutely prefer a world where everyone is guaranteed privacy and governments would need a courts permission to even look up your phone number. It just will never exist and the only thing truly private is our thoughts.