1. Open source. 2. Has verifiable builds (just being open source isn't enough). 3. Uses end to end encryption.
You need all three of those to get what you want. Also you need to spend a ton of time building Signal from source and auditing it, which I doubt you're going to bother with but at least in theory it is possible.
They can't, provided you verify the (reproducible) builds you download from the app store and check your verification codes.
Besides, it would be rather difficult for Signal to target just you and hand-deliver a custom build to you – Apple/Google would have to in on this, too. And if you use a Play Store alternative like the Aurora Store with a random Google account, this wouldn't work, either.
> Why should I trust someone to route my messages?
Unless you want to put the cables in the ground yourself and run the entire internet's infrastructure on your own, you'll always have to trust someone.