Because Google can unilaterally reject valid credentials if it doesn't feel it's certain it's you.
Although I can see why this would be impossible if you have a phone number set that you no longer own. I'm not sure what exactly Google would ask for if you never had a phone number set.