The other factor is Prigozhin is popular with the rank and file in the russian army, and so has a chance to effectively control the army if he decapitated the leadership. Putin's underlings would probably fall in behind him if they believed he did.
Putin, for all his personal power, is only one guy, he depends on the loyalty of the people around him, and this being russia, I imagine they're very much looking out for themselves, rather than some higher calling, so they can be convinced of what's good for them in the current situation.