> and have codified a huge body of knowledge which helps systems with producing outputs from input strings
That is specifically what the programmers have not done.
That is specifically what the programmers have not done.
Glad you agree.
None of the free parameters of these statistical models were selected by programmers.
It shares one property of traditional programming: garbage in garbage out.
The fruits of their labour indirectly is a model. It's proving valuable. You want to ascribe the value to .. the bit nobody claims to understand? I'm sticking to the labour theory of value here, either the robot makers imbued value, or we have to redefine the value.
Or better they shift the distribution by finetuning the model on curated data.
But no, no one change the parameters manually, it's just too complex, an artificial brain.
No. They did not change the logic. They cannot change the logic. They wouldn't have a clue how to do that. No one does. The neural network is practically impossible for us to understand and manipulate directly at that level. Way too complicated. They trained it to be "less racist" by RLHF until it was sufficiently woke.
The logic is exactly what is "learned" automatically by the machine and is beyond our current abilities to understand at a low level. The neural net is too vast.
> you think they just "asked it to be less racist"
Yes that's basically what they did. It is incredible technology.
Maybe we need a similar discipline to understand models and how each neuron impacts the overall behavior of the model. Neuronics?
If you want to define "metaprogramming" in that way, then you'd be making all our brain functioning also that.
I don't think that's your intent.
Also, I'm not sure if I was clear before, but "[n]one of the free parameters of these statistical models were selected by programmers" for me also implies "humans don't adjust the weights".
> It shares one property of traditional programming: garbage in garbage out.
Can you name any system or entity that doesn't? I was going to write "Humans spent millennia thinking men and women had different numbers of ribs, because they took in garbage and didn't check", but when looking up citations to make sure this wasn't just an urban legend it turns out lots of people still think that.
Other modern legends: people today who believe "everyone thought the world was flat until Columbus", when Columbus' main contribution was overconfidence in bad maths making him think the combined width of the Pacific, the Atlantic, and the continent of North America… could be crossed in a fairly short single sailing trip.
> The fruits of their labour indirectly is a model. It's proving valuable. You want to ascribe the value to .. the bit nobody claims to understand? I'm sticking to the labour theory of value here, either the robot makers imbued value, or we have to redefine the value.
Labour theory of value is fine if that's what you want to do. That said, "the makers imbued value" isn't incompatible with "and the thing they made has learned something" — if they were logically incompatible, the same argument could be made with old-fashioned organic human children to deny them likewise. I'm sure you don't want to do that either, as I remember you from a previous time this topic came up :)
The same problem also exists with books: you read a book, did you learn, or was the value all in the author and editor? Both, I say.
One can certainly say "LLMs regurgitate human text" (specifically LLMs rather than all AI as some, e.g. AlphaZero, learn without any human contact), in the same way one can dismiss a human as "only book-smart, not street-smart". I wouldn't do either (of a human it is an insult, of an LLM it is unclear) but one could.
I know they understand the world and the meaning. But, process of handling the language itself, is not statistical at all?
I am sure if the machine is not alive, but all live things has statistical inference part of their learning.
The whole point of neural networks is that we don't know shit and we don't know shit about how to distill the mechanics of some very real, very important processes so we figured out a way to make the machine figure this out on its own.