Comment by bjornsing | Hacker News Reader

Comment by bjornsing | Hacker News Reader

I’ve been using Claude a lot lately, and I must say I very much disagree.

For example, the other day I was chatting with it about the health risks associated with my high consumption of grown salmon. It then generated a small program to simulate the accumulation of PCB in my body. I could review the program, ask questions about the assumptions, etc. It all seemed very reasonable. A toxicokinetic analysis it called it.

It then struck me how immensely valuable this is to a curious and inquisitive mind. This is essentially my gold standard of intelligence: take a complex question and break it down in a logical way, explaining every step of the reasoning process to me, and be willing to revise the analysis if I point out errors / weaknesses.

Now try that with your doctor. ;)

Can it make mistakes? Sure, but so can your doctor. The main difference is that here the responsibility is clearly on you. If you do not feel comfortable reviewing the reasoning then you shouldn’t trust it.

"Trust but verify"

With an LLM it should be "Don't trust, verify" - but it isn't that hard to verify LLM claims, just ask it for original sources.

Compare to ye olde scientific calculators (90s), they were allowed in tests because even though they could solve equations, they couldn't show the work. And showing the work was 90% of the score. At best you could use one to verify your solution.

But then tech progressed and now calculators can solve equations step by step -> banned from tests at school.

> it isn't that hard to verify LLM claims, just ask it for original sources

Have you tried actually doing this? Most of the time it makes up urls that don't exist or contradict the answer it just gave.

Google it.

There's a bunch of scientific papers talking about that.

Bet that plenty of those researchers have written python programs they've uploaded to GitHub and you just got one of their programs regurgitated

It's not intelligence mate, it's just copying an existing program.

I don’t mind to be honest. I don’t expect more intelligence than that from my doctor either. I want them to identify the relevant science and regurgitate / apply it.

>It's not intelligence mate, it's just copying an existing program.

Isn't the 'intelligence' part, the bit that gets a previously-constructed 'thing', and makes it work in 'situation'. Pretty sure that's how humans work, too.

that's a wild assumption that has no place here

And how would you define intelligence then?

Reply on news.ycombinator.com