> The stakers (and their adjoints) can collude and decide to reorg the chain for a different outcome. This can be done indefinitely at no additional cost.

No, they can't. As a part of the attack they will have to produce a two different signatures for the same decision (block N has hash X). This is where slashing comes in - anyone can submit those signatures to the chain to slash signers.