What good is a phone messenger app if I can't send someone a message on their phone number?
Signal just works, I use it as my day to day sms app, and it shows me who of my friends also have signal so I text them on messenger if they do, and WhatsApp if they don't.
So the real reason is Matrix just isn't even trying to compete with signal or WhatsApp, it's competing with MSN Messenger, which everyone stopped using over 15 years ago.