> I have yet to see an explanation for why Signal can use phone numbers from a client-side contact list, but not email addresses from a client-side contact list
OS functionality? There no "Grant access to Gmail contacts" (= email addresses) on Android/iOS, so that client-side list would have to be manually maintained, while (practically) everyone already has a contact list containing phone numbers of their friends.
That said I don't see why a user would have to have a stored social network at all, why can't it simply be opt-out?