Yeah that's a limitation of Android at present, but check out what's coming in Android Q: https://developer.android.com/preview/privacy/device-locatio...