Definitely not exactly a human. OTOH Low hanging fruit is low.
The Eliza effect is off the scale.
What I'm trying to say is that the underlying method is not a valid reason to discredit one thinking process over another.
Anthropomorphism and Anthropodenial are both variants of Anthropocentrism, and share the same limitations. Have you considered other axes of thought?
I can readily admit that lots of humans will naively anthropomorphize horrendously, but I think that:
- The eliza effect is not what people think it is
- What is actually going on is obscured by all the anthropomorphizing
- But this is yet no grounds to throw out the underlying phenomenon, especially when a) it can be useful and/or b) it causes people to get hurt.
Are you familiar with the physical church turing thesis?
Similar to the eliza effect, people still take the original reading of Clever Hans: "he couldn't really do maths, he's just taking social cues from his handler"
But what's the actual difference between Eliza, Clever Hans and RLHF? They're doing the similar things, right?
Now look at how we valued that in the 20th vs 21st century:
How much does an ALU even cost anymore? even a really good one? (it's almost never separate anymore, usually on the same silicon as the rest of the cpu/microcontroller)
Meanwhile... what's the TCO to deploy a sentiment classifier? Especially a really good one?