I don't know the latest details about Android/iOS app signing, but presumably reproducible builds + sufficiently strong signing would make it secure enough for most users. For those who are truly paranoid, then can build it themselves (subject to their own device OS's requirements, which are hardly a unique problem to Signal).
In short, Signal's security should be as good as any mobile app can be, and can be even better if you're willing to put in legwork.