It doesn't scale well beyond a handful of people. You need *N bandwidth to send and receive, and without a thing called 'simulcast' (creating multiple, different quality stream simultaneously) which doesn't have good browser support the quality is defined by the lowest common denominator. A central server solves many, many issues that result in better quality.

Jitsi itself barely works on Firefox and not at all on mobile devices (without their app).