Thanks for the reference. I agree that the solution to getting I action is to model the disturbance, and augment the state. The system certainly becomes uncontrollable in the augmented state space.

I've run into a conceptual problem when using this approach with [infinite-horizon] LQR to try to recover what looks like a PID controller, which IIRC is due to the control u not necessarily converging to 0. This is expected, to counteract a constant disturbance, but it means that the sum doesn't converge.

I only skimmed the reference, but I did not find a discussion of this issue.