In this case, there is a "false positive" where Google thinks something bad is happening when it isn't. They see that contacts are being touched, but they don't know what is being done with that data and assume the worst (as they should).
It would be nice if the accuracy could be dialed-up by inspecting code or other manual processes, but this is labor intensive and therefore expensive for society as a whole, regardless of who pays the price (devs, Google, or users).
Question for Voys: If there was a service where for $10k/yr you had a white-glove app-approval service from Google/Android/Play where they do actually inspect your code, would you pay for it? Maybe you would, and maybe there's a good market there. I don't know.