In order for Signal to reach "the masses", it needs to become popular with a large number of people who don't really understand encryption or care about it, but are using Signal anyway because they happen to communicate with people who do care about encryption. That's a good thing!
Unfortunately, it's a really hard sell to tell someone, "Hey, download this new messaging app so you can talk to me. Oh, and by the way, when you get a new phone, you'll lose your entire chat history with me".