I’m sure all the people in your contact list who don’t want their information shared would be happy that you say no. Frankly, I’m always somewhat amazed that people do share that information. To me, it doesn’t feel like that’s mine to share.
Appreciate the comment! We do this to see if there is an existing user for a connection (like any chat app). We don't store the contact book on a server nor process anything other than that.