That’s fair, but almost every messaging app does this (WhatsApp, Signal, iMessage, etc.). The question is: do you trust Telegram more or less than the alternatives? Personally, I see it as the lesser evil compared to some big tech options.
iMessage sort of does, but that's only because it's more a part of the OS, which already has your contacts anyway.