It's time to bind "Please be concise in your answer and only mention important details. Use a single paragraph and avoid lists. Keep me in the discussion, I'll ask for details later." to F1.
I really like not being complimented on literally everything with a wall of text anymore.
1. Rules that get prefixed in front of your prompt as part of the real prompt ChatGPT gets. Like what they do with the system prompt.
And
2. Some content makes your prompt too big for the context windows where the rules get cut off.
Then, it might help to measure the tokens in the overall prompt, have a max number, and warn if it goes over it. I had a custom, chat app that used their API's with this feature built in.
Another possibility is, when this is detected, it asks you if you want to use one with a larger, context window. Those cost more. So, it would be presented as an option. My app let me select any of their models to do that manually.
Also, if that works, why doesn't copilot/cursor write lots of excessive code mixed with lots of prose only to distill it later?
The “thinking” models are really verbose output models that summarise the thinking at the end. These tend to outperform non-thinking models, but at a higher cost.
Anthropic lets you see some/all of the thinking so you can see how the model arrived at the answer.