Hacker News Reader

Comment by lelanthran | Hacker News Reader

> It has no baggage, it's built from the ground up to be a modern 21st century human. We're not, we train decades to become ones.

The whole point of machine learning is to do all that "decades" of training in a few hours.

In terms of learning material, it takes LLM a few orders of magnitude more material to learn than it does a human. After all, a human child trained with very few books (by age 7 a child would have seen only a few books) still outperforms CHatGPT trained with the same material, because ChatGPT trained on 4 books is basically useless.

View on HN

They train on years of real-time streaming video. Feedback loops all over the place. Higher order structures and rules all get learned way, way before they even are able to touch a book. I cannot phantom the amount of data and analyses that passes a typical human child in a day.

So, sure, they can learn fast after all that. But so can GPT, which also one-shots a lot.

> They train on years of real-time streaming video.

...

> I cannot phantom the amount of data and analyses that passes a typical human child in a day.

You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color).

A 6 month old baby who has only ever seen close family (parents, siblings, pets) is quite able to discern the same features on strangers.

Nothing we have in AI right now can take 3 example models of humans, and detect hair, nose and mouth in a brand new never before seen model of a human.

Not really the same thing as training on millions of different images and videos. The number of distinct and different objects in a 7 yo child's training set is rarely the same as "every single youtube video ever uploaded".

Going back to the original point: ChatGPT needs millions of books to learn to regurgitate information as well as a semi-intelligent young adult, while typical toddlers are quite able to regurgitate what they've just seen.

Humans and animals get by with millions of times less training data than AI models need.

>You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color).

Then it sounds like he is slightly overestimating the dataset (since some of the frame is monochrome) and you are massively underestimating it.

Faces may not be the best example, because there's a lot of evidence babies are 'hardcoded' to be interested in and pay attention to faces. This implies there's some knowledge of them built in.

However, if you substitute abstract shapes (blocks, balls, etc.), your point absolutely stands up.

In some ways, though, isn't it more interesting to have an intelligence that learns in a completely different way from humans, as opposed to one that learns through essentially the same method?

Reply on news.ycombinator.com