I really wish I understood comments like these as what you describe is just not my experience at all in over a year.
My partner, entire social circle, etc and I all communicate via matrix, via a mix of clients ranging from Element to Weechat to Fluffychat across a number of different servers for both personal and business.
I have helped migrate hundreds of people to matrix and in the last year or so I have not had to hand hold people anymore as UX has improved dramatically. They just go sign up and drop in just like they would with Signal or WhatsApp or any other messenger.
I also admin a number of rooms with hundreds of people and everything just works.
Personally Matrix and IRC (often also via Matrix) are all I use for messaging.
I login with a new device, have an existing device verify me when prompted, then type messages in boxes to people and people see them, with end to end encryption that works. For conversations that require high trust I additionally do key verification.
I do encounter minor bugs from time to time like stuck messages, or occasional slowdowns of the giant primary matrix.org server, but honestly I have found it dramatically more reliable than SMS for me, but with actual E2EE.
We really need the internet to be open and decentralized and that happens with technical people going all in on efforts like Matrix and helping report any bugs found and get then fixed.
Element -was- for sure as buggy as you describe in my experience for many in my circle a couple years ago but even then we were happy to accept occasional glitches than continue to build our social graph in walled garden ecosystems that don't care about censorship resistance, security, or privacy.
It really depends on what you value.