The centralized model of singal is a problem, as is reliance on phone numbers to sign in.
Not to mention them publishing the source code for the server is only useful if you implicitly trust that is the code they publish is what they are running
Setting up XMPP and PGP sounds like a pain (almost everything to do with PGP is a pain). Even if you set up such a system, good luck convincing your friends and family to use it with you. I've had some success with Signal adoption but I suspect that XMPP would be a non-starter.
Much easier to tell friends / family "invite to Signal."
I'm now actively trying to get contacts off XMPP because XMPP is too unreliable for me.
And that's before we talk about the sad state of XMPP apps on the desktop. You cannot win the messaging market when you have only one good app on a single OS.
I mean I get it, but those who trade security for convinence deserve neither
If you only need to support one platform maybe it's doable, otherwise just not worth it.
If you are writing some proprietary app that will never interface with anything externally, it's much easier, quicker and cheaper to roll your own protocol.