Even professional native voice actors get a reading wrong in audiobooks sometimes so it's not surprising to me that LLMs are not quite reliable enough either at this point