> they've trained on < 1M tokens of math texts, never mind the 70T tokens that GPT5 appears to be trained on.
Somewhat apples and oranges given billions of years of evolution behind that human. GPT-5 started off as a blank slate.
Somewhat apples and oranges given billions of years of evolution behind that human. GPT-5 started off as a blank slate.
"How could a telescope see saturn, human eyes have billions of years of evolution behind them, and we only made telescopes a few hundred years ago, so they should be much weaker than eyes"
"How can StockFish play chess better than a human, the human brain has had billions of years of evolution"
Evolution is random, slow, and does not mean we arrive at even a local optima.
Also consider that during training LLMs spend much less time on processing, say, TAOCP (Knuth), or SICP (Abelson, Sussman, and Sussman), or Probability Theory (Jaynes) than on the entirety that is r/Frugal.
20 thick books turn a smart teenager into a graduate with a MSc. That's what, 10 million tokens?
When we read difficult, important texts, we reflect on them, make exercises, discuss them, etc. We don't know how to make an LLM do that in a way that improves it. Yet.
But it's indeed apples and oranges. There's no good way to estimate the information encoded by the GPT architecture compared to human DNA. We just have to be empirical and look at what the thing can do.