Our tech implements server-mediated video chat for up to 6 participants. It requires some processing power on the server, but it is designed with a "scale-out" architecture so you should be able to get it running cheaply at scale without major rewrites.
Custom server or public? ( fms/wowza/etc. ) How many streams per server can it handle? Why bother with a server doing relay if its only 6 people?
It's currently iOS only, but it should work to copy the iPhone code's structure to implement clients for all relevant platforms -- Android, WinPhone, Blackberry, web, and/or desktop.
It is my understanding that developing video chat for android is a major clusterfuck. Not only does each O/S version create crazy complications, the differences and nuances required just across phone models is a nightmare. For instance, on a Evo you need to flip the image. On a newer droid the f/b camera is reversed from the API. Its enough that developing video chat for android is a nightmare in disguise
Performance will be critical for slower devices, so you'll need to implement it carefully. The video codec is h.264 at 144x144. (We experimented with up to 320x320 and found that with current connection quality on phones, especially on urban wifi and 3G, sticking with a low-resolution default tends to produce the best user experience.)
Don't things look like shit in 144x144 for non-group calls? Do you scale larger when its just a 1 to 1 call?
The audio codec is Opus at 16 KHz.
Are you muxing connections together on the server? Do you lose individual volume control per feed?
I think the most important question is related to server technology. If you can only get a small amount of users on each server, its not going to be possible to be profitable moving forward.