I trust my homeserver, since I host it myself. I do not need end to end encryption. I understand this is an issue of the client, not the protocol. But there aren't many clients. Prebuilt element for android from fDroid will connect to matrix.org, vector.im and other network hosts and I don't want that at all, although this tracking can't easily be disabled. It feels very centralised.
I run my own homeserver and want to chat with my friend that runs his own server. It is unacceptable to me that clients connect to anything other than those two homeservers (apart from CRL and OCSP, which should also be disabled by default, as I consider those protocols great spyware). Not to mention the fact that homeserver software itself is known to make connections to the servers of the developers by default, without ever talking to someone on matrix.org. This is also unacceptable. My homeserver should only connect to other homeservers.
3PID is a failed attempt, as it centralises identities and works by utilising a single point that gathers a lot of personat information at one place. I don't want it.
XMPP does not have those issues. Set up prosody and use dino or Conversations and they won't make any connections to non-essential servers. Furthermore, the end to end encryption is way easier to use in XMPP (OMEMO), and it's easy to turn it off if you don't need it.