Of course that would be impossible to differentiate, but adding it to the privacy policy is what Google requests. Adding it and still having it refused can absolutely be blamed on Google.
What I don't get is why does sending contacts need to be added to the privacy policy? The point of the policy is to state what you will and won't do in a legally binding way. If my privacy policy says "the app may access contacts locally, but that information never leaves the device", that means users can sue me if I break that promise and I could even be criminally prosecuted. What more assurance does Google need?