Speech synthesis is substantially more of a solved problem than you think. Look at Vocaloids, for example. Here you have entirely computer-generated "voices" singing pop songs, and achieving a surprising degree of commercial success. Or look at something like the Tupac "hologram" that sang one of Tupac's unreleased songs at Coachella. Granted, that was taking samples of Tupac and combining them, but it was combining them seamlessly to perform a new work. That's pretty close to pure synthesis.