For work I used ChatGPT to write a very specific programming problem and it wasn't a solution you'd ever copy/paste...the lack of context to the surrounding system means that's impractical 99% of the time outside of toy problems.
But it was still super useful for getting my brain working and suggesting a really good basic structure it would have taken me 3-4 failure cycles to get to.
The same exists in DALL-E/GPT for corporate art generation, writing 'essays', or stories or w/e. It's almost never producing the end product (besides toy problems).
So, if it can't do that in the first place, why evaluate it as if that's what it's supposed to be?
When I talk about coaching I meant that I told ChatGPT to answer only with questions. In a way, ChatGPT is good at having socratic conversations. Sometimes it gets out of character but it depends mostly on your answers or questions.
> The model is able to reference up to approximately 3000 words (or 4000 tokens) from the current conversation - any information beyond that is not stored. (https://help.openai.com/en/articles/6787051-does-chatgpt-rem...)
So if you overflow that it'll loose context.
Personally, I’m one of these people, and I’ve struggled with how much others either don’t work this way, don’t want to or don’t know how. I think it’s powerful and that finding good collaborators at this way of thinking is a factor for general success.
Interesting to think about how AI can provide a general base level for this kind of thinking.
I've already forgotten where I saw it (probably here), but it'll be interesting to see how quickly we get into a period like iPhone circa 2010 with the mantra "there's an AI for that".
ChatGPT gave me answers I would have never gotten regarding extremely technical concepts that the wikipedia articles would never surface, and would need hours of research papers or formal academic work or maybe just working in the field and being in those communities.
I think it is ironic that Google is sitting on a better conversation AI that is just too busy finessing its own engineers trying to escape.