We don't have any mechanism other than trusting Whisper Systems that their client binaries matches the sources.
The truly paranoid could be their own client application from source.
And I'm not interested in the client side, but in the server side of the security. I assume I can rebuild the client using the sources and compare their client with my build.
What I can't assume is that their servers aren't, for example, infested by some state security malware which takes over the control. I want the protocols that work even when the server is subverted.
There are that are working on eliminating the need for central server for message delivery and key server roles like
https://whispersystems.org/blog/asynchronous-security/
"At registration time, the TextSecure client preemptively generates 100 signed key exchange messages and sends them to the server. We call these “prekeys.” A client that wishes to send a secure message to a user for the first time can now: - Connect to the server and request the destination’s next “prekey.”" Etc.
How is this not giving the server chance to insert its own keys and not the ones user generated?
What the QR code thing is actually doing? From what I understand, the TextSecure/Signal in one way or another send your contacts to the server and the server stores "prekeys" and delivers them to the clients. Where is any analysis that says that all this stuff doesn't mean that we have to trust that the server is not evil?
If the QR code means "yes it's the person you want to communicate to" but the contacts are on the server and the prekeys are delivered from the server, am I any better?
I need to read somewhere that the server is not weak point and why it is not. Any link?
Because the server can't sign the key exchange messages - the private key is generated on the client and never leaves it.