Misleading title but scary nonetheless. It's not my complete phonebook... not even close. But it does have numbers of friends who aren't in my phone (which is useful and possibly OK considering they added me as friends).
Nevertheless i never explicitly opted into this which is bad.