Incidentally, an unexpected benefit of the server-relay FaceTime implementation is the ability for Apple to allow 32-person FaceTime calls which was a feature just announced on Monday.
They can’t stitch and re-encode the video streams if they can’t decode it server side.
I believe Apple explicitly said it was still end-to-end encrypted.
There might be a way for it to be end-to-end encrypted, but it's not a simple problem.
Most messaging based solutions keep the same individual key exchange for group sessions also instead of figuring out some key ring or exchanging a key between all individuals.
This simplifies things and allows you to have only one key exchange algo.
Video and any other high bandwidth application would require you to always have a shared key on a group level because otherwise you are doing a lot of extra work and a lot of exra upload.