You're mistaking the technical implementation of the Fediverse with the social contract that maintains its existence. The Fediverse is built on the idea of consent, where interacting with you happens because you consent to that interaction happening, and you can withdraw that consent at any time by e.g. deleting your original post. (That's also why the "quote tweet" feature from Twitter is absent on the Fediverse.)
When people and instances don't follow that contract, they simply get defederated.