True for Signal. XMPP is using JIDs which are functionally comparable to email addresses, and similarly anonymous (or not).
Is there a way to run your own Signal server?
The Signal client is open source, the server mostly so. You could create your own semi-Signal community, but it would not be able to talk to official-Signal users.
XMPP is designed for federation by default, and you can run your own server, akin to email. Though some server operators decide to disable or cripple the federation feature (Facebook, Gmail).
Would I need to trust Signal more than I would need to trust Jabber?
This is a complicated matter. You need to trust the server operators to handle metadata in both scenarios, but you can run your own server with XMPP. You need to trust the client developer, which is Signal in one case, or the developer of your choice (or yourself) in the other. You need to trust the device manufacturer/Google in both cases.