> If the work you do after the defer has other side effects

Defer is by definition the last work you do in a function, there won't be more work except by the caller who will get the error returned to them.

If you are structuring a function that writes a file, and then does something with it, defer isn't appropriate, since you should close it before you do any more work.