LLMs can help tremendously with this part, though. Exploration/brainstorming, discovery, data analysis, etc can be made significantly easier with LLMs in most knowledge based fields.
I've seen much time wasted on strange ideas originated by an LLM. It's a lot easier for a human to organically have good ideas than to refute LLM-generated ideas that are a dead end.
Some companies are releasing code that nobody on the owning team has ever read or reviewed, or has any inkling exists.
I'm not sure you've quite thought this through. "Holding it wrong" is very, very often the case, especially with technology.
The chatbots kept steering people toward uncreative ideas and overshadowing novel insights that might come from genuine domain expertise. AI generated meeting summaries became a force for miscommunication as they de-emphasized or even outright misrepresented what was really important about the conversation. And people became much worse at proving ideas out before making significant resource commitments thanks to effects that we now describe with terms like AI psychosis.
All AI is doing here is rapidly identifying the people who have Peter-principled themselves.
> It's a lot easier for a human to organically have good ideas than to refute LLM-generated ideas that are a dead end.
What works better is when you can use an LLM to obtain a negative opinion on your own ideas that you think are good. LLMs aren't great at original thinking, but they're pretty good at finding the flaws in poorly thought-through human ideas.
Sigh. Where are the actual smart people here who think deeply? This surface level stuff doesn’t cut it
All of it relies on learning and information processing capabilities that LLMs simply do not have. The idea that any of it will automatically emerge from the current architectures simply by adding more parameters is about as sensible as thinking that if you keep counting long enough eventually you’ll get to complex numbers. We’re going to need to invent new technology to fill these gaps, and there is no way of knowing how long those developments will take.
The god machine can prompt itself. Just go "Write me a prompt to <whatever>. Ask me as many clarifying questions as needed.". It is not a differentiator.
Deep domain knowledge and pattern matching that allows you to quickly verify outputs might be a differentiator. Maybe. But you might end up the bottleneck if you verify everything.