recitedropper

Repo seems legit, and some of the ideas are pretty novel. As always though, we'll have to see how it scales. A lot of interesting architectures have failed the GPT3+ scale test.

As a sidenote--does anyone really think human-like intelligence on silica is a good idea? Assuming it comes with consciousness, which I think is fair to presume, brain-like AI seems to me like a technology that shouldn't be made.

This isn't a doomer position; that human-like AI would bring about the apocalypse. It is one of empathy: At this point in time, our species isn't mature enough to have the ability to spin up conscious beings so readily. I mean look how we treat each other--we can't even treat beings we know to be conscious with kindness and compassion. Mix our immaturity with a newfound ability to create digital life and it'll be the greatest ethical disaster of all time.

It feels like researchers in the space think there is glory to be found in figuring out human-like intelligence on silicon. That glory has even attracted big names outside the space (see John Carmack), under the presumption that the technology is a huge lever for good and likely to bring eternal fame.

I honestly think it is a safer bet that, given how we aren't ready for such technology, the person / team who would go on to actually crack brain-like AI would be remembered closer to Hitler than to Einstein.

[flagged]

I appreciate the clarification, although I was linking this as clearly ConceptARC has a number of tasks where GPT-4 fails badly compared to humans. On "Extend To Boundary" category GPT-4 scores 0.2 and humans score 0.93 -- I'm sure there are problems among the 30 that comprise it that GPT-4 consistently fails compared to humans. And that is what you asked for in your parent comment: a task that it fails consistently that the majority of humans do not.

Claiming GPT-4 doesn't "fail anything" here is a little pedantic, if you are saying saying well hey it doesn't get a 0 score on anything. The conclusion of the paper is literally GPT-4 fails "to robustly form abstractions and reason about basic core concepts in contexts not previously seen in its training data".

And yes, I know the whole "generalization is always a data problem" / "humans come with millions of years of training data" take, but if you follow ARC, it's pretty obvious this type of abstraction forming is the clearest spot where models consistently fail and humans excel. Which to me implies less of a data issue and more of a architectural difference that has yet to be overcome.