I’ve been looking into this. Nothing definitive yet, but my hunch is that current LLM’s struggle because they lack curiosity. They will answer your question, however vague, with the first most obvious answer they can think of.
This is great, if you’re a junior team member. Super talented, very eager. But a more senior engineer approaches the problem differently. They ask more questions than provide answers. They’ll happily spend the first 25min of a 30min meeting asking heaps upon oodles of really dumb sounding questions.
Then the last 5min, that’s the magic. They now have a solution perfectly tailored to the problem at hand, with all sorts of edge cases either explored or eliminated through questioning. The questions they kept asking weren’t dumb after all, they were pruning a decision/option tree in their head of all possible solutions until they landed on the most optimal solution for a set of known constraints. With further options to dig and improve.
I think you can make an LLM do this (i’m trying), but it’s very very slow still.
If I had to make a bet, I suspect Google and Meta will royally lose this generation: Apple will dominate personal access to LLMs, Microsoft will dominate business access to LLMs -- and honestly Palantir for the first time terrifies me. The current generation of LLMs are not about automation as such, but helping individuals accomplish self-directed goals (as possible in language and code).
This is because curiosity requires free energy, hence it is very expensive when you're limited on very expensive compute.
This is what tree of thought is attempting to simulate in some ways. Build a set of multiple questions around the original question and then build on and prune that list based on a 'show your work' set of steps, and then keep iterating.
Humans naturally solve the halting problem when thinking about things... we work on something long enough without a break and we'll pass out. Maybe when we wake, eat, and go to work we'll stop working on the same problem. But an LLM never sleeps. In theory with TOT and no time limit, you could find out your AutoGPT spent 10 million in computing resources contemplating navel lint. So, there are a number of unsolved problems there.
What really becomes concerning is if Nvidia achieves its goals of speeding up training/inference by 1 million times in the next few years, and if the amount of compute we produce increases by a few million times. You and me simply can't use hundreds of minds thinking for years straight and machines could.
I don't have much opinion on this though, i just know i see that description of GPT4 a lot. I use GPT4 daily, but i think i don't really challenge it most days, as i've never trusted it enough to do so hah.