Is this surprising though? To be honest, I don’t think we have any such thing as “AI experts” currently. LeCun, Hinton, etc. were early pioneers in neural networks and Hutter did a lot of the first work with reinforcement learning, but none of these people seem to agree on much now with regard to AGI, and I wouldn’t even say they have any particularly deep insight into LLMs beyond that of a research scientist at OpenAI (perhaps less even, given the time spent on social priorities now).
We’re reaching a point where it’s likely that further development just isn’t going to be well-understood by either the general public or the people creating the tech, and this will lead to a lot of “voodoo” explanations of how things work. Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process.
> Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process.
This sounds wrong. It sounds as if you're arguing that empiricism and the usual scientific method ends up in mysticism. If it does, maybe the things we're creating should command that respect from us, as much as all other natural phenomena that we don't understand.
The traditional idea that many CS experts popularized that computers are just applied maths and you could infer anything on a computer with pure thought is wrong IMHO.
I think we’re on the same page... I’m arguing that the current approach to creating better LLMs is more akin to an art involving educated guesses and tinkering than it is a hypothesis-driven process with randomized controls (e.g., traditional engineering disciplines or applied physics / chemistry).
(Not to say those who believe in horoscopes and astrology know how it works though...)
While I don't think non-technical people think its "magic", you can hardly fault them considering 1) the "main stream media" hype around it and how it's going to be so transformative on jobs if nothing else, and 2) some people have been promised this kind of stuff since star trek or earlier, or their previous experience was having Siri say "sorry I don't understand what you said" back to them before setting the timer manually. Just having a coherent responses from a parser that's pretty hard to "trip up" is, well, unsurprisingly magical.
I mean, I just suggested a pretty complicated model relationship to chat3.5 and asked for a DB schema and it gave be back quite a coherent answer, even I thought it was close enough to magic. Siri would have thrown a fit.