You can't do better than perfect. But if the default is sufficiently bad, you can seem to violate conservation by putting more effort in.

Similarly to when a machine has an efficiency of 1%, optimizing it to 10% does not violate thermodynamics.

There is a limit to what you can learn from limited knowledge. Every bit of information you learn can only divide the space of possible theories by half. However, current LLMs are ludicrously far from that limit.

There must be. Since there is a finite amount of real world reward signal captured by these models, there can only be a finite amount of grounded knowledge that can come out. Usually without reasoning, the output knowledge "yield" will be far lower than input.

The interesting piece is if we can get LLMs to reason well, that finite reward capture can take LLMs much farther than possibly humans could do with the same signal.

Oof I really don’t think there must be such a law. The basic physics of the universe and the mathematical structure of a human engineered LLM are just very different things.

The "compressed" knowledge is fixed. But reasoning can create new knowledge, that's what humans do all the time.

Nope! That surely doesn't apply to humans, why would it necessarily apply to learning machines?

My Pareto law senses are tingling.