You have it backwards. The assumption is that there is no privileged reference frame. And that assumption (along with a few others) does imply an invariant speed. That's the whole point here.

Let's assume there exists one or more privileged ref system(s), and deal the two cases one by one. If there was one privileged reference system, and in our deduction process it's not involved,then the original deduction still holds. therefore the special ref system is an redudunt assumption. If there were many such ref systems, we would have to carefully get them involved in any possible deduction process, which is impossible. QED