Do you have a share link to the session?
Why is every confident claim about a frontier model being brutally wrong about something presented without any details or evidence? It’s so trivial to share even details like what language was used, but even that had to be obscured for some reason.
How about the opposite? Share a normal session (by normal, I mean something longer than, say, 4 turns, not absolutely trivial) where LLM doesn’t do a single silly thing. Including in thinking portions.
Providing even a minimal set of details is important because “an LLM” could mean anything from the Google search header to Fable or Astra.
It’s like someone making a claim that they tried a diet and didn’t lose weight, therefore all diets are bad and shouldn’t be trusted. The details of the diet, how long they tried it, how strictly they followed it, and the rest of their situation are pertinent.
There is an audience who demands content like that: People who want do hate diets and believe they don’t work will eat that commentary up because it levels up the belief they want to hold. Everyone with critical thinking skills or experience with dieting knows that one person claiming diet failure means nothing.
We know diets are a set of tools and they’re not perfect, but they are used successfully by people who can understand their limitations. Now replace diet with LLM and my statement is the same
> How about the opposite? Share a normal session (by normal, I mean something longer than, say, 4 turns, not absolutely trivial) where LLM doesn’t do a single silly thing. Including in thinking portions.
How about Terence Tao sharing his ChatGPT logs having ChatGPT work with him through understanding the Jacobson conjecture?
https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed...