It seems like the mayor and police chief should be voted out as well. If they cannot maintain control they should go as well.
TL;DR When a cop is fired there are several safety nets that can reinstate them. If I remember right, the union reinstates ~45% of fired officers. Neither the Chief nor the mayor have the LEGAL power to even fire bad actors in the police department. They literally cannot do anything about the problem within the law.
This does not mean that other things should not also be done.
I beleive getting rid of / modifing the police unions would be beneficial. Fixing the whole qualified immunity issue would also be good. But I also think police need representatives to protect them from false accusations or from being railroaded. Maybe that is from a union or maybe its from something else.