> They keep saying this is 2-factor authentication, but without the first factor (something you know e.g. a password)

I always wonder about this even when there is a password. For most services, a phone compromise would be a total compromise. I rarely get prompted for a password on my phone (nor do I enjoy typing one when I'm trying to do something quickly.)