The reason most people aren't as bothered by this situation as they "should be" is... things were worse under the old Taxi system for everyone we should have sympathy for.
Uber didn't make everything perfect, but they improved things dramatically.
So sure, let's keep improving things. But if the "anti-Uber" fanatics take down Uber, they won't get an improvement. They'll get the old system back, which was worse for drivers and passengers alike.