Hacker News Reader

Comment by throwaway920102 | Hacker News Reader

throwaway920102

I agree. The idea behind predicting the next token is that you regress to the mean. If humans are mostly idiots, then the mean is idiots. If you filter your data to only include genius humans, then you can train and get an average answer that a genius would write online to a given prompt. But how would you train to generate an order of magnitude of intelligence or insight above a genius if the entire corpus of training data is at most genius-level?

You'd have to hope for some sort of emergent intelligence or knowledge-breadth integration based intelligence I suppose. But I already get annoyed by ChatGPT 4o for having Average Redditor Tier ideas and responses.

View on HN

You need a reward system or game. There are lots of professions and areas/topics of debate where one can just make up convincing nonsense. In a competitive area or one with quality feedback there is no need for the blabbering of fuzzy humans.

One of my 1000 business plans I will never execute is to build an automated chemistry lab where one can order different ingredients to be subjected to different processes, treatments and measurements. The researcher/customer would have no influence on the input or output and no humans in the building. I'm clueless about the scale to have a positive cost benefit analysis but if it can be good enough to throw things at the wall and see what sticks it seems an AI could be useful to pick the best ways to blow up the lab.

I think of it differently. The model becomes a simulator, an actor of a sort. If it has enough of good data to build good patterns or reaches a certain intelligence threshold, the content generated by less than average intelligence becomes an acting material to understand those flawed ideas and how to emulate them, which still requires to build and improve empathy. It can be challenging to exactly predict what a "stupid" person would say next, so you need to develop understanding to human psyche in order to solve for that well.

And then with a prompt you can tweak, "a reddit tier response", "an expert response", etc.

It kind of works already, but the more intelligent it gets the better it should get at acting different roles.

Moreover, you only get an answer for a question which has already been posed and answered, hence is represented by a pattern which the large language model can arrive at --- it's not possible to get an answer which wasn't already present in the training data.

We might develop a selective corpus with say, the contents of the "Great Books" of western civilization and add the results of scientific history.

But the underlying problem in this is politics: everyone has a different idea of what is appropriate for that corpus (and consequently the resulting AI). Ergo the various brou-ha-ha's about "safety" etc. Indeed one assumption of these discussions may be correct: NN AIS may be as malleable, hard-headed or gullible as any human intelligence [and I don't know whether that is good or bad]. So many questions arise: "Should we let it read Karl Marx?", "What about St. Augustine?", etc.

Presumably we're modeling an intelligence akin to ourselves. We each occupy a single mind but the difference between minds can be great. The most familiar approach is therefore to develop an AI that is as much like us as possible.

We could also model many single minds with different corpuses and let them communicate, discuss et al as humans do. Maybe they would let us interact too.

FWIW I think you should be happy that any "intelligence" shown so far is of "Average Redditor" value. What would you do if you scattered some holy water on a pentagram in your upstairs living room, hurled out a diabolical incantation calling forth spirits and something akin to Satan himself appeared? That's (kind of) where we are with GPT.

Reply on news.ycombinator.com