Yes, you can always defederate from other instances if you want. In fact, a lot of fediverse instances do just that to protect their hugboxes from the rest of us who actually value our ability to communicate with each other. And there are certainly a number of instances that defederate from others on technical grounds rather than social ones.