>the biggest challenge is that the logic for mutating that object is all over the codebase

Just use immutable data structures and be done with it. In departing from old OOP views and becoming more functional programming and data oriented programming friendly, C# introduced Records, which are immutable. Probably Java and Python have similar constructs. Javascript allowed the use of immutable data since long time ago.

If you insist of using fat models, you will still mutate the data all over the place by doing calls, but you just obfuscate it.