Large scale conferencing may be challenging but it's also pretty expensive and an attractive service to take a slice of. We currently use a service from our telco and have run up to about 40 people for our training sessions. So it's definitely doable.

You can't have more than a few talking at once or it becomes very unwieldy. Is that what makes it hard? Having to multiplex 40 input channels together? Would it be easier if most of the lines were muted? Because that is a workable compromise.

As for SIP support, it potentially opens up twilio service to some neat applications were one end of the phone connection could be hosting in a web app via flash/silverlight.