Indeed, this is an assisted brain storming tool, not automated problem solving where you're supposed to trust the result wholesale.

For work I used ChatGPT to write a very specific programming problem and it wasn't a solution you'd ever copy/paste...the lack of context to the surrounding system means that's impractical 99% of the time outside of toy problems.

But it was still super useful for getting my brain working and suggesting a really good basic structure it would have taken me 3-4 failure cycles to get to.

The same exists in DALL-E/GPT for corporate art generation, writing 'essays', or stories or w/e. It's almost never producing the end product (besides toy problems).

So, if it can't do that in the first place, why evaluate it as if that's what it's supposed to be?