Well, I can imagine the F16 g-limiting system messing with flight controls if the g-sensor is botched.

In my eyes the problem isn't so much that the systems are overly invasive, it's more that the failure modes of automation aren't well understood or signalled and cannot always be remedied.