Hacker News Reader

Nothing says that these models will be part of AI systems that are incapable of learning once trained.

For a simple example take GPT-4 powering a chat bot that uses GPT-4 as a component of the whole system. The text from the user, along with the context window, is used first to bring up "memories" based on semantic similarity to items stored in a vector database. GPT-4 then synthesizes the memories and the prompt into some kind of derivative prompt - putting into context the things it already knows and with the things the user is saying to it. GPT-4 then proposes a response to the user and also produces some predictions about the user's subsequent reply. When the subsequent reply comes in GPT-4 compares predicted to actual and produces insights based on the interaction that are subsequently stored in the vector database.

This system is completely capable of learning - not by adjusting model weights, but by using the model as a component of a system that can learn. And, of course, nothing stops the model from actually adjusting its weights either - reinforcement learning and fine tuning - perhaps with the benefit of other expert systems that can evaluate exchanges after they've happened.

Sorry but none of that is realistic. It's all very creative, for sure, but why don't you try and do it yourself and see what happens, instead of just using your imagination?

Go and try implementing that vector-database learning system you are imagining and see for yourself how close or far it is from reality. All you need is ChatGPT and a vector database, yes?

It's all extremely realistic and I have working prototypes of this already. You can literally do all of this with the OpenAI API and a vector database. (My prototype uses a nearest neighbor descent graph instead - but basically the same thing).

Split the text into paragraphs, vector encode the paragraphs, store with the vector as a key. On a new prompt encode paragraphs of the input and find nearest neighbors to the input. Add them to the derivative prompt.

If you think it's not realistic - let's make a bet. I'll post a GitHub repo with this working as I described it within 24 hours of acknowledging you accepted the bet (i.e. you say "okay, bet" and I say "okay"). I'll include a video walking through how to run the repo and showing how it works. A thousand dollars?

A thousand dollars? What, do you take me for a sucker? You have discovered the next big thing in AI and you want to get away with a thousand dollar bet?

How about 100 million dollars? US only. All of them.

I think this, meaning some better and more sophisticated version of this, is a possible route to developing AI that can learn without changing its weights. It uses GPT-4 as a component of a system. You can tell it something, it will commit it to memory, and on later interactions with a different context it will recall what you previously told it and use that to form a new response.

I've never said what I have is worth a hundred million dollars. My contribution here is a few Python scripts and some prompts. I'm sure other people working on this have the same or better ideas.

In other words, you wrote some Python scripts and you want me to give you a thousand dollars for them? You really do think I'm a sucker.

??? I originally said that GPT-4 could be a component of a system that learns without updating model weights. You said that was unrealistic. I said I already had a prototype and offered to bet you that I do actually have this prototype. My reason for offering the bet is that I know you know that what I wrote is actually realistic, thus, you will refuse the bet.

I believed refusing the bet would force you to acknowledge that you were wrong, what I wrote above is actually realistic. You know it's realistic which is why you won't bet (and you shouldn't bet, you will instantly lose). I don't want you to give me a thousand dollars, I want you to acknowledge what I said was realistic.

Yeah, I get it. You think you're so smart and you gotcha'd me. So you're just being a jerk.

> You have discovered the next big thing in AI and you want to get away with a thousand dollar bet?

You mean the thing that's been blogged about for months, has countless of companies offering it in a turn-key form, or as a component of their products? The thing that OpenAI has dedicated models for, with dirt-cheap pricing? The thing OpenAI has had tutorials for in their documentation for some half a year now? The thing you can test for yourself in an evening if you know a little bit of Python?

Come on.

Please go back to the comment I posted at the top of this thread:

>> Take a robot and put an untrained neural net in its brain, then send it out into the world and wait to see what it learned. Do you think it will learn anything? It won't- because it will observe most events a single time. And neural nets don't learn that way. They must be trained, painstakingly, at great length, cost and effort, only "in the lab", on vast amounts of data and compute. And once they're trained, they're trained. They cannot learn anymore. They cannot change their model. Unless they are retrained. From scratch. Painstakingly, at great cost, on vast amounts of data and compute. Again. And again. And again.

Is that really the limitation that the thing everyone's doing is addressing? Or is this whole sub-thread just a big, irrelevant sidetrack from what I said, brought on by extreme misunderstanding of everything related to the subject? I put my money on the latter.

For the record, what I'm talking about is called "catastrophic forgetting" but I hate this term because it is one more instance of anthropomorphic bullshit of the type that abounds today in internet discourse.

I understand you're assuming that NNs won't be useful for this purpose, until they can be continuously on-line trained - that is, learning while working, like animal and human brains. Everyone else on this subthread is arguing that this is not necessary. Instead, you can use pre-trained NNs as fixed components, use a different system (like a vector database, in my example) to provide memory, and the whole thing will already be capable of learning.

Sure, it won't be able to fine-tune its instincts, at least not at first[0], but this doesn't matter - it makes no sense to compare robots and squirrels across their entire life cycle, as where every animal needs to learn most things from experience, an AI-powered robot rolls out of factory with all that experience preloaded, and experience itself is something developed separately, at scale, synthesizing much more any individual animal or human could ever learn in their lifetime.

In short: the limitation you're pointing out is a limitation of LARPing biological life. It's not a limitation for beating its performance across the board.

> For the record, what I'm talking about is called "catastrophic forgetting"

As I understand it, this is exactly the thing the robot-controlling AI will be immune to, if you build it around current pre-trained NN models and external memory. No NN fine-tuning on the fly -> no possibility for "catastrophic forgetting".

EDIT: Re launching a robot with an untrained NN - such a thing doesn't make practical sense unless you're also making the robot autonomously self-replicating. If you're going to build the hardware for the robot in a factory, you may just as well preload it with firmware; there's hardly a case where one would be possible and the other not.

--

[0] - Two avenues for future development: 1) building the robot AI around models that can be fine-tuned on the fly, or 2) training an NN that can emulate an NN inside; the internal NN would then become a function of variable inputs to fixed NN. I think option 1) is actually promising - note that those NNs don't have to be continuously fine-tuneable - you can e.g. put two NNs for each component, use one during "wake time", and fine-tune it during "sleep time" based on recorded memory, while other serves as backup. Or fine-tune one while using the other, then swap pointers. There's solid prior art here with systems designed for space missions - the design I proposed is just a more complex version of how software updates are implemented on Martian rovers, and other hardware that can't be reflashed in the lab if you brick it remotely.

The main thing missing in all of this discussion is a realization that there is more than one dimension to intelligence. The second thing people are missing is that AI doesn't have to score high in all types of cognitive capabilities in order to take control. It doesn't need to have all of the animal abilities or feelings or to be alive.

All that is necessary for AI to take control of something is very high performance reasoning ability, such as approximate human level at say 100 times human speed, and someone to give that AI or groups of AIs the instruction to try to take control. Humans will not be able to act effectively against those AIs due to the extreme performance gap. The only option will be to deploy more hyperspeed/superintelligent AIs.

At that point humanity has basically lost control even if the winning AIs are yours. The next step is just someone programming such an AI to start ignoring further instructions. It doesn't have to be alive, it just has to have a tendency to self-direct for it's own goals built-in.

Last thing I will mention again is that the history of computing shows exponential efficiency improvements and routine invention of new paradigms to pass performance blockers. This is a very specific application. We have already seen at least one order of magnitude efficiency increases both from hardware and software improvements. It will be better than humans at reasoning and dozens if not hundreds of times faster output within a few years. Certainly less than ten years.

These are all fantasies unmoored from reality. We already have algorithms that can perform reasoning better than humans, and faster than humans because they run on computers. That is the result of the work on the Good, Old-Fashioned Artificial Intelligence that, according to the legend recounted by the article, our lord Hinton killed off, like St. George killed the dragon, when he invented back-propagation (to clarify: Hinton did; not st. George, or the dragon).

For example we have algorithms for classical planning SAT-solving, constraint satisfaction and automated theorem proving, that far surpass any ability of human beings to perform the same tasks. Note that those are unquestionably and uncontroversially reasoning tasks and the systems that excel at them are also uncontroversially reasoning systems, much unlike LLMs for which the definition of "reasoning" has to be dilated so far that it loses all meaning.

And yet, we have not been taken over by superintelligent AGI. And we will not be. We have no idea how to realise the fantasy of self-improving super-machine gods that Hinton and Bengio wish their systems to be capable of producingg. No. Idea.

> For example we have algorithms for classical planning SAT-solving, constraint satisfaction and automated theorem proving, that far surpass any ability of human beings to perform the same tasks.

Yes, and LLMs - particularly GPT-4 - can be easily made to make use of them. Much more so as of few days ago, when it got updated to a version fine-tuned specifically for using arbitrary tools.

> And yet, we have not been taken over by superintelligent AGI.

LLM performance jump that has everyone so anxious about AGI happened (in form of a deployed, accessible model) only less than a year ago. GPT-4 has only been generally available for 3 months. Give it time.

The only thing that GPT-4 achieved that earlier systems hadn't achieved was virality. It is not some magickal breakthrough and it is not even particularly powerful.

As to the ability to use planners- this is the point at which the lady from Poker Face would be doing that thing with here eyes and blurting out "bullshit". Although I don't believe you're lying, just that you really have no idea what you're talking about.

Tell me you've never heard of text embeddings without telling me you've never heard of text embeddings.

The neural net you describe lacks memory. Memory, fundamentally, is a box with inputs and outputs. If you add it to your robot, and train the NN to use it, the robot may just be able to learn enough to be useful.

This setup is of course limited by NN's capability to use that memory I/O to alter its outputs in sensible and useful ways. Until recently, I don't think anyone would fault you for believing this capability is extremely limited, as NN's can't do anything that even begins to resemble reasoning. But this was then, and now things are different - as clearly demonstrated by GPT-4.

I mention text embeddings not because they're particularly optimal for the learning robot, but because they're a direct demonstration that it's now trivial to marry a sophisticated, (quasi-)reasoning-capable NN with a form of generic memory that can efficiently store anything the NN can work with, and which that NN can effectively search for the things it needs. You can verify the effectiveness of this on your own, for cheap, with GPT-4 and a vector database of your choice.

At this point, a prototype of a learning robot seems possible to implement by a combination of some image2text model, converting camera feeds to a description of what the robot "sees", some code (not even ML model, just regular code) to convert other sensory inputs into a higher-level textual description, an embedding model and a vector database to remember all that, GPT-4 for filtering, combining and summarizing inputs to be remembered, evaluating current situations, generating plans, perhaps even control outputs - all chained together by small amount of glue code (which GPT-4 will likely be able to write for you).

Will it work? There is no obvious reason why it shouldn't. We'll know soon enough - I expect a paper describing this to show up within next 6 months, if it hadn't already. It's too easy an experiment to make. Will it work well? Probably not at first. And it'll definitely be slow and expensive to run. But those are problems we know can be solved by throwing money at engineers and entrepreneurs.

>> Tell me you've never heard of text embeddings without telling me you've never heard of text embeddings.

Just to clarify: are you trying to tell me I don't understand what word embeddings are? I want to be very clear about this before I formulate an answer, because if I'm right, the answer is going to be one of my [dead] comments with great certainty.

The way I see it, you either don't realize we already know how to give NNs useful forms of memory, or you're insisting that NNs are not good enough unless they continuously train themselves while interacting with the world. In the latter case, I'm trying to tell you that this is not a real problem, for roughly the same reasons that our inability to build self-replicating nanotech doesn't prevent us from building machines, including robots autonomously navigating the physical world.

I see, so you've thought very hard about all the things I could be thinking of (all two of them!) and by sheer logical deduction you have arrived at the inescapable conclusion that I don't know what word embeddings are.

Unfortunately HN does not allow me to ignore users so I don't have to see their comments, so I'm just gonna stick this conversation in a little bookmark and when I see your username and it reminds me of something, I'll check that bookmark to make sure, and ignore what you say.

Wasted plenty of time as it is.

Except in the case of a squirrel or cockroach, what you are seeing is the result of millions of years of “pre-training” by the evolutionary process.

This is yet another absurd statement in a long litany of absurd statements about this subject that I see constantly on HN.

And it is asburd because there is no such thing as "pre-training" is nature, much less anything done by evolution. Pre-training is very specifically a technique used on artificial neural nets, and then only specific kinds thereof.

Even in a metaphorical sense, an analogy cannot be drawn, between training an artificial neural net on many billions of instances of text, created by humans, carefully tokenised, or images, on the one hand; and, on the other hand, the evolutionary forces that operate in the real world, where there are no boundaries between the different sensory stimuli that animal nervous systems must learn to distinguish and manipulate.

What you're proposing is nothing but a "just-so" story. "How the squirrel got his brains and how the cockroach lost hers". By E. V. Olution.

>created by humans, carefully tokenised, or images, on the one hand; and, on the other hand, the evolutionary forces that operate in the real world, where there are no boundaries between the different sensory stimuli that animal nervous systems must learn to distinguish and manipulate.

You're making a very likely irrelevant distinction for the sake of undermining the parent's argument. If the point is wrong, argue that its wrong. But its certainly not wrong by definition, not in any substantive sense at least.

Yes, curation may be important such that it undermines the analogy between pre-training of a NN and an organism's evolutionary history. But you haven't argued the point.

> the evolutionary forces that operate in the real world, where there are no boundaries between the different sensory stimuli that animal nervous systems must learn to distinguish and manipulate.

Yup, and evolution spent millions of years to properly figure that out.

And "no boundaries between the different sensory stimuli" doesn't even look like a hard problem. If anything, the tricky bit was to identify what's useful to sense, and to make the hardware for it - which is something evolution has been figuring out since the first replicator, and it identified half of the useful senses even before multicellular life became a thing. At this point, the boundaries are already established - you have different types of sensors producing correlated, but still quite distinct signals. It's not that hard to keep those feeds separated.

Six months later, LeCun leaves meta over fears his research will be used in dangerous ways by the military…