It does make passwords weaker. But 26^8 is still 200B. People aren't making 100B login attempts to break your case-insensitive alphabet-only eight character password.
You really don't want to rely on a breach being just the password hashes and nothing else. In the case where the adversary was able to do literally nothing other than exfil the hashes, a stronger password will help you. But is this actually a common threat model?
And if you have SMS-2FA enabled, an adversary with your password needs to sim-swap you anyway, which is doable but scales very badly.
Easier just to phish people.