Hacker News Reader

Comment by sph | Hacker News Reader

On a philosophical level, I wonder if ChatGPT gives us mostly nonsense because most human output is bullshit.

To paraphrase a famous quip, what if this is proof that 99% of everything humans have created is bullshit? The problem perhaps has never been machine learning but identifying the process that enables we humans to somehow go through life and create objective societal and scientific progress out of a massive pile of nonsense.

I might be completely wrong here, but the next ChatGPT will assume this comment of mine to be absolute truth and happily ingest it wholesale.

View on HN

A key problem is that the transformer operates over very short fragments of text. Long range correlations need to be maintained to ensure 100% accuracy in reproduction/inference of relationships.

I realize this is a philosophical take. It also suggests a technical solution: model longer range correlations. GPTs seem so capable because previous models were not as good at this, but the context lengths modeled with transformers are still extremely short relative to those that we work with.

In result, GPTs are very good at translation and reformatting over short ranges, but struggle to deal with long form texts---a typical technique makes summaries of summaries to build out a model of a longer text. That's hardly ideal, and it's not how our minds work.

That's a good observation, though I'm not sure it's just a matter of longer range correlations, i.e. fixable by merely adding more computational power. The truth problem is fundamentally different than what those neural networks are trying to solve. We got the learning part almost right, we have made no progress towards understanding.

I might be completely off the mark, my knowledge of AI is very limited. But earlier today, while reading on Lisp's history and it's demise during the AI Winter, I very much enjoyed a Wikipedia article describing how symbolic AI, the main paradigm of AI research from the 60s to the 2000s, solves a completely different problem than the currently hyped neural networks—which are very good at learning, but not much more. We might need a complementary, unifying model of both approaches to reach AGI.

https://en.m.wikipedia.org/wiki/Symbolic_artificial_intellig...

The average of two true statements can easily become false, so models like ChatGPT will spout nonsense even if it was only trained on 100% truthful statements.

Can you give an example as to what you mean please?

One classic example from linguistics is the two (true) sentences

"Politicians lie."

"Cast iron sinks."

Combine them syntactically and you get "Politicians lie in cast iron sinks", perhaps not completely false but certainly less useful than the component parts.

EDIT: It seems this is actually from Douglas Hofstadter

It gives us nonsense because it's goal is to generate something that looks like it fits in with training data. You can have training data that is all completely factual and still have a generator spit out nonsense that looks completely correct without fact-checking.

So, exactly like people then?

Sure, a lot of human output is incoherent, but current models give nonsense for a different reason - they lack many mechanisms typical for human intelligence.

Reply on news.ycombinator.com