Doesn't the phone have to be local to the AP in order to get the MAC to send to Google for the location?

Google doesn't broadcast it to the whole internet either: they just use its position for clients which happen to be near your AP.