telling a model not to guess. How could it define what a guess is?
YOU'RE HUMAN?!?!
Please explain to me, how an LLM can easily distinguish between information it knows and a guess.
Perhaps you meant: "within the available documentation for the technology that you're working on and its current version, after having also double checked the outstanding issues via the github repo. If the technology has widespread use then forums might also contain relevant information where users might share further information about known issues."
But that's a teensy bit more than "never fucking guess", which is the lazy and stupid version of it which is what gets people in trouble. Do you not realise how this alternate version explicitly specifies the search space where "never fucking guess" specifies pretty much jack shit?
"never fucking guess" is akin to "don't make mistakes" in how its completely useless as an instruction.
Sure:
- explicitly stated in the context window or logically follows from what is -> evidenced or mechanically follows, thus not a guess
- not explicitly stated in the context window and doesn't mechanically follow from what is either -> not evidenced nor mechanically follows, thus a guess
Through the power of indeed being human, one can then cut a reasonable threshold for such guesses and degrade "never fucking guess" to "be very certain in the actions you take". This threshold-setting and gap identification then is thus a matter of alignment. You ask the model that - to the extent it has a sense of it - not take risky leaps. It's basically a reminder or a theme signal.
And there you go. In spirit, yes, this is indeed similar to "make no mistakes" [0], but I do believe it to have a bit more tangible grounding. The models will absolutely have a latent space (if not many) for representing uncertainty and likelihood, as well as for metacognitive considerations relative to uncertainties and likelihoods, since these concepts are very, very extensively represented via all kinds of literary devices throughout human literature. To be able to respond to things not working out as expected, one has to be able to depict hedges. To do so efficiently, one must be able to keep those efforts calibrated and measured. So LLMs should absolutely have a sense of their own certainty of actions, especially if they can depict them in their context window, which they totally can.
> YOU'RE HUMAN?!?!
Yes? Are you? Ever heard the phrase "playing it safe"? Surely you have a concept of its meaning? What kind of question even is this, when the rest of your comment is about arguing what an LLM can do, not a human?
[0] Have you ever remarked something supply chain worries related in a system / initial prompt, then saw your agent being indefinitely paranoid about artifact signatures and hashes? Same idea. Float the idea of mistakes, it will invite the agent to run linters and maybe spin up critic subagents. Float the idea of only ever taking low-stakes hedges, it will frequently sound how confident it is in any given action, and may prompt for your approval more often. These are memes, yes, but they're not baseless. Not sure why you seemingly think they are.
Exactly, and that's the level of conversation people should be having, instead of giving it "don't fucking guess" and calling it a day.
Its just too ambiguous a construct to lead to reliable output and its an operator error.