But I'll bet you know 20 passwords, or more. That fundamental skill is not lost. You're just not getting any phone number practice anymore. You're getting password practice all the time, I'll bet.
EG, 123-444, 123-323, 123-789, 555-121. So, you really needed to remember the prefix and the last 4. There were exceptions, but most of your circle lived within this finite space of telephone numbers.