All-or-nothing is not a sufficient replacement for this use case. Given access to contacts, it's the app's choice to not give the user a "never (for this contact)" option. This is a blatantly pricacy-disrespecting feature with no off switch.
When announcing that a workaround works for you, you derail the conversation with disingenuity unless you're entertaining the notion that your workaround is not actually equivalent to a solution.