Or... if most knowledge work happens on a computer, and you just want the AI to replace knowledge workers, then maybe how people use computers is the only thing the AI needs to know.

Probably true for AGI but the overloaded term "AI" could be applied to learn from how people use software to _carefully_ suggest improvements.

Usage stats could help you improve a user interface, teach users new skills (eg: "you always do this, here's a shortcut to save you time"), perhaps more personalized operating system interfaces and defaults.

You probably don't need a fancy pseudo "AI" system for any of these, but that's the current gold rush so someone's gonna do it

That's pretty absurd considering the average american has a screen in front of their face 8-10 hours a day. You're basically saying what people do for 8-10 hours a day, every day, for their entire lives isnt useful information to AI.