Why can't the phone number be unique? I have a unique phone number and trust them as my email unless I transfer myself out of my country.
I might verify the number, and then stuff happens in the real world while I haven’t logged into your service, and someone new registers with my number.
The number is assigned to my account, but the new account can prove they own the number. If you’ve made the number unique, then you need special handling here.