Take current 100/20 Mbps speeds. An average Zoom call uses 0.6 Mbps upstream, while a super-HD 1080p one uses 3.8 Mbps up. (And virtually nobody videoconferences from home in 1080p anyways, who wants coworkers to see your skin blemishes in maximum detail?!)
So a connection or 20 Mbps upload supports 33 users in theory, or 5 at super-HD. Even allowing half that in practice... seems fine to me.
The 100 Mbps download is necessary when you've got someone watching an 8K VR video on their headset, and a few 1080p movie streams going as well, which is more reasonable in a family setting.
But most people just don't have any use for upload speeds anywhere near as fast as download. Or at least certainly not until we start doing 8K 3D video calls in front of massive 3D displays, which isn't anytime soon...
[1] https://support.zoom.us/hc/en-us/articles/201362023-System-r...