Comment by barfbagginus | Hacker News Reader

Comment by barfbagginus | Hacker News Reader

Parent Full thread

Can I please convert you into someone who summarily barks at people for making the LeCunn Fallacy rather than making the LeCunn Fallacy yourself?

And can you stop talking about AGI when it's not relevant to a conversation? Let's call that the AGI fallacy - the argument that a given development is worthless - despite actual technical improvements - because it's not AGI or supposedly can't lead to AGI.

It's a problem.

Every single paper on transformers has some low information comment to the effect of, "yeah, but this won't give us AGI because of the curse of LeCunn". The people making these comments never care about the actual improvement, and are never looking for improvements themselves. It becomes tiring to people, like yours truly :3, who do care about the work.

Let's look at the structure of the fallacy. You're sidestepping the "without a major redesign" in his quote. That turns his statement from a statement of impossibility into a much weaker statement saying that auto regressive models currently have a weakness. A weakness which could possibly be fixed by redesign, which LeCunn admits.

In fact this paper is a major redesign. It solves a parallelism problem, rather than the hallucination problem. But it still proves that major redesigns do sometimes solve major problems in the model.

There could easily arise a regressive model that allows progressive online updating from an external world model - that's all it takes to break LeCunn's curse. There's no reason to think the curse can't be broken by redesign.

This thing will still hallucinate, not matter what new bells and whistles have been attached to it, meaning it will never be used for anything important and critical in the real world.

You have no proof that every modification of the architecture will continue to have hallucinations. How could you prove that? Even LeCunn admits that the right modification could solve the issue.

You're trying to make this point in a circular way - saying it's impossible just because you say it's impossible - for some reason other than trying to get to the bottom of the truth. You want to believe that there's some kind of guarantee that no offspring of the auto regressive architecture can never get rid of hallucinations.

I'm saying they're simply no such guarantee.

Plus, humans bullshit all the time, even well paid and highly trained humans like Doctors and Lawyers. They will bullshit while charging you 400 an hour. Then they'll gaslight you if you try to correct their bullshit.

AI will bullshit sometimes, but you can generally call it on the bullshit and correct it.

For the tasks that it helps me with, I could work with a human. But the human I could afford would be a junior programmer. Not only do they bullshit more than a well prompted AI, but I also have to pay them 30 an hour, and they can't properly write specs or analyze requirements. GPT 4 can analyze requirements. Much better then a junior, and I'm many ways better than me. For pennies.

I do use it in the real world, to maintain and develope software for the industrial design company I own. It would be foolish if I didn't. I've been able to modernize all our legacy codes and build features that used to stump me.

Maybe the fact is that I'm an incompetent programmer, and that's why I find it so helpful.

If that's the case so be it! It's still a significant help that is accessible to me. That matters!

Here's a system that uses an llm to generate equivalence proofs for refactoring operations.

https://news.ycombinator.com/item?id=40634775

In this system, the llm can hallucinate to its hearts content - the hallucinations are then fed into a proof engine and if they are a valid proof then it wasn't a hallucination, and the computation succeeds. If it fails it just tries again. So hallucinations cannot actually leave the system, and all we get are valid refactorings with working proofs of validity.

Binding the LLM to a formal logic and proof engine is one way to stop them hallucinating and make them useful for the real world.

But you would have to actually care about Proof and Truth to concede any point here. If you're only protecting the worldview where AI can never do things that humans can do, then you're going to have to retreat into some form of denial. But if you are interested in actual ways forward to useful AI, then results like this should give you some hope!

Good luck and good day either way!

> Binding the LLM to a formal logic and proof engine is one way to stop them hallucinating and make them useful for the real world.

Checking the output does not mean the model does not hallucinate and thus does not help for all other cases in which there is no "formal logic and proof engine".

What if I consider the model to be the llm plus whatever extra components it has that allows it to not hallucinate? In that case then the model doesn't hallucinate, because the model is the llm plus the bolt-ons.

Remember llm truthers claim that no bolt-ons can ever fully mitigate an llm's hallucinations. And yet in this case it does. But saying that it doesn't matter because other llms will still hallucinate is moving the goal post, or at least discounting the utility of this incremental progress. I think it's unfair to do this because there are many many domains where things can indeed be reduced to a formal logic amenable to approve engine.

If they don't care about the actual output of a hybrid system that doesn't hallucinate, because it's math and not speech, then do they care about solving the issue at all, or providing human utility? I get the feeling that they only want to be right, not the benefit anyone.

This shows that in cases where we can build good enough verifiers, hallucinations in a component of the system do not have to poison the entire system.

Our own brains work this way - we have sections of our brains that hallucinate, and sections of the brain that verify. When the sections of the brain that verify our sleep, we end up hallucinating dreams. When the sections of the brain that verify are sick, we end up hallucinating while awake.

I agree with you that the current system does not solve the problem for natural language. However it gives an example of a non hallucinating hybrid llm system.

So the problem is reduced from having to make llms not hallucinate at all, to designing some other systems, potentially not an llm at all, that can reduce the number of hallucinations to a useful amount.

Reply on news.ycombinator.com