I wish this would be the default. I expose my homelab port 22 directly to the internet. I'm _pretty_ sure I always always always disable password auth but I do worry about it because most distros have an unsafe default.
(A lot of this risk is mitigated by not having login passwords but I definitely have one node where I have a login password, it's an old laptop so I thought I might want to physically log in for local debugging).
I guess the ideal solution here is to run a prober service that attempts logins and alerts if it gets any responses that smell password auth is possible. But no way I have time to set that up.