Go to "account recovery" option and remove the phone number listed there.