Proof of work is currently controlled by 3 companies in terms of hashpower and 1 in terms of hardware - Bitmain.

So, the absolute worst case for PoS is already pretty much the case for PoW.