I’m part of the engineering team and I wanted to share a little bit about the engineering bit of it. We’ve used LLMs extensively across the product. Our general approach has been to use GPT-4 for the higher order stuff and then using 3.5 for the actual execution of the individual actions.
We’re using vocode for audio, and playwright + webrtc for the video part.