This would form a filter not unlike, yet distinct from, our understanding of personal experience.
you could make the exact same argument against humans, we just learn to make sounds that elicit favourable responses. Besides, they have plenty "skin in the game", about the same as you or I.
If you want to understand where a systems limitations are you need to understand not just what it does but how it does it, I feel like we need to start teaching classes on Behaviorism again.
Are you sure you aren’t just defining reasoning as something only a human can do?
You know how we insert ourselves into the process of coming up with a delicious recipe? That first person perspective might be also necessary for reasoning. No computer knows the taste of mint, it must be given parameters about it. So if a computer comes up with a recipe with mint, we know it wasn’t via tasting anything ever.
A calculator doesn’t reason. A facsimile of something we have no idea about its role in consciousness has the same outlook as the calculator.
Jet planes are so far removed mechanically from a bird that the idea they fly is not even remotely worth considering.
But since we knew properties of wings were major comments to flight dating back to beyond the myths of Pegasus or Icarus, we rightly connected the similarities in the flight case.
Yet while we have studied neurons and know the brain is apart of consciousness, we don’t know their role in consciousness like the wing’s for flight.
If you got a bunch if daisy chained brains and that started doing what LLMs do, I’d change my tune—because the physical substrates are now similar enough. Focusing on neurons, and their facsimilized abstractions, may be like thinking flight depending upon the local cellular structure of a wing, rather than the overall capability to generate lift, or any other false correlation.
Just because an LLM and a brain get to the same answer, doesn’t mean they got there the same way.
Bailey? Reason.
How reasonable are the outputs of ANNs considering the inputs? This is a valid question and it has a useful response.
From ImageNet to LLMs we are finding these tools to give some scale of a reasonable response.
Recommended reading: Philosophical Investigations by Wittgenstein.
If not, then why shouldn’t differently constructed but algorithmically similar systems be able to produce similar phenomena?
See C-sections versus natural birth. Formula versus mother's milk. Etc.
What non-sentience based property do you think something should have to be considered reasoning. Do you consider sentience and reasoning to be one and the same? If not then you should be able to indicate what distinguishes one from the other.
I doubt anyone here is arguing that chatGPT is sentient, yet plenty accept that it can reason to some extent.
No, but I think they share some similarities. You can be sentient without doing any reasoning, just through experience, there's probably a lot of simple life forms in that category. Where they overlap I think, is in that they require a degree of reflection. Reasoning I'd say is the capacity to distinguish between truth and falsehoods, to have mental content of the object you're reasoning about and as a consequence have a notion of understanding and an interior or subjective view.
The distinction I'd make is that calculation or memorization is not reasoning at all. My TI-83 or Stockfish can calculate math or chess but they have no notion of math or chess, they're basically Chinese rooms, they just perform mechanical operations. They can appear as if they reason, even a chess engine purely looking up results in a table base and with very simplistic brute force can play very strong chess but it doesn't know anything about chess. And with the LLMs you need to be careful because the "large" part does a lot of work. They often can sound like they reason but when they have to explain their reasoning they'll start to make up obvious falsehoods or contradictions. A good benchmark if something can reason is probably if it can.. reason about its reasoning coherently.
I do think the very new chain-of-thought models are more of a step into that direction, the further you get away from relying on data the more likely you're building something that reasons but we're probably very early into systems like that.
Real reasoning is being able to manipulate symbolic expressions in a consistent manner while preserving some invariants.
Personal experience as logic is how you end up with the Holocaust.