Hacker News Reader

Comment by Retric | Hacker News Reader

It’s interesting just how bad of a summery that is compared to the human created abstract included in the text.

“It is now clear that the production of unintentional conchoidal sharp-edged flakes can result from tool-assisted foraging in nonhominin primates. Comparisons with Plio-Pleistocene lithic assemblages, dating from 3.3 to 1.56 million years ago, show that flakes produced by macaques fall within the technological range of artifacts made by early hominins. In the absence of behavioral observations, the assemblage produced by monkeys would likely be identified as anthropogenic in origin and interpreted as evidence of intentional tool production.”

View on HN

That’s not plain English though, and I’m not seeing why that makes the plain English chatgpt attempt poor?

It’s both wrong and misleading. Think about this sentence literally: “These stones are almost identical to the ones early humans made, which suggests that the macaques unintentionally produced them while searching for food.”

First the similarities between these tools doesn’t suggest they were unintentionally created by the macaques. Next foraging and searching for food suggest different things, the macaques were breaking apart nuts with stones not looking for nuts the way “searching for food” implies. etc

Anyway, the “almost identical” bit is ignoring what’s actually being described. The stones created by macaques are quite distinct from what our ancestors made. So the similarities can help refine our understanding of those artifacts and possibly what may have kick started this kind of tool creation/use, but the differences make it clear which archaeological examples are of actual tool creation. At the same time current methods don’t account for the similarities and may conceivably misidentify some sites.

Only the most extreme pendant would argue that "searching for food" is an inappropriate simplification of "foraging." ChatGPT also simplifies the term of art "lithic assemblage" which, devoid of context, I don't think most would understand.

And, regardless of what the rest of the article may say the original abstract does NOT make it clear that the "stones created by macaques are quite distinct from what our ancestors made"--that information is not present in the abstract and therefore ChatGPT has no chance to preserve that subtlety. (Notwithstanding the original abstract uses the phrase "almost indistinguishable" which a reasonable reader cannot interpret as "quite distinct".)

The issue here is context. “lithic assemblage associated with a primate foraging behavior undertaken by long-tailed macaques” could be reasonably simplified to “stone assemblage associated with long-tailed macaques [searching for food]” or other foraging synonyms. However when you drop the word associated and say “macaques unintentionally produced them while” you are now describing a more direct relationship.

Similarly the abstract clarifies the phrase “almost indistinguishable” with “likely to be confused with” etc which makes “almost identical” inappropriate.

Anyway, this kind of mistake shows up frequently because of how ChatGPT abstracts words and phrases. The approach has real upsides, but it falls on its face when subtle differences actually matter.

ChatGPT made some mistakes in simplifying the language, but I think a lot of humans might make the same mistakes because the article is just badly written.

The "unintentional" part is of course a ChatGPT hallucination that implies something that isn't there in the original.

In my opinion the article is great meanwhile ChatGPT made 4 significant mistakes in a single sentence. “almost identical” “early humans” “, which suggests” and “searching for food”

I don’t think that says as much about the algorithm as way people are doing evaluations. So yes people might mistakenly use “early human” when describing what was around 3.3 to 1.6 million years ago but that doesn’t somehow make it correct to do so.

> I think a lot of humans might make the same mistakes

Indeed, it seems ChatGPT is not that much of an improvement over the current state of affairs. At least not in the realm of science journalism.

We don't need automatic mistake-makers, eh?

Reply on news.ycombinator.com