It is the latency, yeah. And it doesn't have to be seconds, even 200ms is enough to turn a fluid call into a very awkward "wait now talk". "wait, now talk" dance. Similar energy as driving and going from stop light, to stop light. Phone calls are way better at this.