(In Soviet Russia, improv actors inject prompts into you.)
LLMs are gullible, naive and will do anything to please you, because that's how they are trained.
They're dogs who speak very good English and memoized the Internet.
LLMs are gullible, naive and will do anything to please you, because that's how they are trained.
They're dogs who speak very good English and memoized the Internet.
I regularly have hard problems that I can digest into chunks that LLMs routinely help me categorize, solve, and document.
I feel like these comparisons are just not in touch with the actual amount of help that LLMs provide in a day to day context with programming related tasks.
Their suitability for a particular task depends largely on your tolerance for that category of error. If you're generating fiction, it's irrelevant, if you're generating a newsletter that might be skimmed at a high level by disinterested people (or other LLMs) it's a low risk, if you're writing UI code it can be bad but will probably be corrected if it's bad enough, if you're writing critical code an insidious error can be disastrous.
This specifically is a solvable problem with grammar constraints on the token output. You can give LLM no other option than syntactically correct C code, chess moves or whatever with the right grammar restriction.
Its one of the (IMO) biggest revolutions in the LLM space that precisely no one uses.
Otherwise I’d be a couple hours into reading up on what the hell HSL is in color theory.
It streamlines work that would otherwise require me to have extensive knowledge in an alternate domain.
These tools are so much more useful if viewed as “coding buddies” who may get it wrong as much as you do, but they’re also skilled in several domains that you’re not. But bouncing ideas back and forth is crazy valuable.
The sycophancy bias is part of OpenAI's alignment, its not necessarily an inherit property of LLMs. Even the "default" personalities of some llama models are far less agreeable in a chat format.
Of course, if you just give it raw text, its going to complete the text, whatever it is.