Hacker News Reader

Comment by sobellian | Hacker News Reader

In some amount of time (centuries? decades? years?) it will be trivial for any shmuck to train their own automated theorem prover. If people think they are competing against monolithic data centers they are wrong. The brain consumes ~20W and it carries out theorem proving mostly as a side effect. We should not be surprised if the core algorithm requires much less than this.

View on HN

The algorithm is cheap and simple. A human brain requires decades of experience and information to train it to be generally useful. An AI model requires a similar equivalent scale of information and time to train it to be generally useful, and that isn't cheap to gather or implement.

Sure, once trained, every usage technically lowers the cost per usage. Once trained...

AI methods are improving all the time. People think that modern chess engines are more powerful due to hardware advances but search refinements are enough - SF on a Pentium II would destroy even Kasparov and Deep Blue. In a like manner the guys at the frontier labs discovered an indirect method for attacking many problems and we can expect it to receive considerable refinement. It will not permanently require expensive training.

There was a rather interesting article (sadly LLM-written and not very well either, because the underlying content would be actually good) about this era being "AI's 2 FPS moment".

The comparison was made with the graphics engine team at Microsoft who were writing the first version of DirectX at the time when then-current hardware could just about do 2 frames per second. The sentiment apparently was along the line "why would you bother?" By the time the release grade stack shipped, hardware had caught up and 30 FPS was commonly doable.

We should expect to see similar step-change improvements over time, as the algorithms are refined and hardware further adapted to the new bleeding-edge constraints. In a way we already are, because apart from the frontier models ballooning like crazy, the "regular" class models are becoming more capable in smaller sizes.

Training a human takes much less money and data than training an LLM. Even if I spent my entire life doing it, I would not be able to read the entire corpus of any LLM, even counting just the English subset. LLMs need so much more because they start out as blank slates, while we don't.

yes, but LLMs can hold all this data and once they are trained they blow any human out of the water. It takes a few years for an LLM to ingest the entire internet. No human will ever be able to do that.

Reply on news.ycombinator.com