Not firing quickly means you fire slow. Which means the person who should be let go continues to stick around and potentially cause problems for the rest of the people. This can be just as sad, if not more so, for the other employees who are impacted by what the person who should be fired continues to do.

Firing quickly doesn't mean not to offer regular feedback, or not letting people improve, or not setting expectations well, or not being transparent. It means that it's better to rip the bandaid off instead of picking at it weeks or month ignoring the employee with the hope that things will turn around.