I tend to think it's going to turn out to be something along the lines of a perpetual motion machine. In the sense it seems very possible on paper but the subtle facts of reality prevent it.

People will make things that look a lot like self-improving machines but on closer inspection that won't truly be the case.

I suspect this violates the Labor Theory of Value in the same way that perpetual motion violates the laws of physics.