Though I understand your points, Signal is a protocol, you can implement and build things around it the way you want. Sure you can say that WhisperSystems should enforce a secured architecture around it, but it is not their job, and I doubt Whatsapp, Messenger and Skype would be using Signal if they were forcing that.
And I would definitely not say Telegram is the best tool here [1][2]. If you want something really secure, I would recommend the Signal app [3].
In the end, it is always who you decide to put your trust in.
[1] https://security.stackexchange.com/questions/49782/is-telegr... [2] https://eprint.iacr.org/2015/1177.pdf [3] https://signal.org/