It only does the popup after you try to activate them, yes? Why not allow the user to turn them on with a button and not use them until then? "Hey this app is a lot better with push notifications, tap this to turn them on. A dialog will appear, tap 'Allow' to get notices etc. etc."

You said elsewhere that ~40% of users assume your app will spam them, but you don't know it's not ~40% of users simply don't want push notifications even from "benevolent" applications.