An audio-to-audio model is definitely a step forward. And I do think that's where things are going to go, generally speaking.

For context relating to real-time voice AI: once you're down below ~800ms things are fast enough to feel naturally responsive for most people and use cases.

The GPT-4o announcement page says they average ~320ms time to first token from an audio prompt. Which is definitely next level and is really, really exciting. You can't get to 800ms with any pipeline that includes GPT-4 Turbo today, so this is a big deal.

It's possible to do ~500ms time to first token by pipelining today's fastest transcription, inference, and tts models. (For example, Deepgram transcription, Groq Llama-3, Deepgram Aura voices.)