Here's what Telegram has to say to that: https://telegra.ph/Why-Isnt-Telegram-End-to-End-Encrypted-by...

Seems like an okay compromise to me, but I'm no security expert.