Hacker News Reader

Comment by jcims | Hacker News Reader

And that you can accurately determine difficulty and/or confidence.

Accurately or the way humans do it?

Seems like passing the input with a question of "where should this input be handled?" into the LLM would be a good first step.

Why would the LLM be any good at determining where the input should be handled?

There's a long history of "this looks hard, I won't implement/fix it" in GitHub issues the LLM can train on.

And then the LLM can probably know whether an issue can be fixed or not. That’s far from generalizing the problem though.

LLMs have shown some limited ability to generalize.

(Opinion) I think internally, they record how closely 2 words are in meaning based on the training data.

If everyone uses similar language to describe different problems, then the LLM should be able to at least act like it’s generalizing

GPT4 uses a "mixture of experts" system which is already sort of kind of like doing this.

The first tasks of gpt was to evaluate the emotion of a sentence.

Like determine if a comment/review is good or bad.

can probably easily be interpreted from the text?

Did we just discover a new halting problem?

It doesn't seem like this should be the case. Humans can estimate the complexity of a task without performing the task. Why couldn't an advanced LLM?

Isn't a large part of software development's difficulty that people can't actually do that?

>Humans can estimate the complexity of a task without performing the task.

I would argue this ability drops precipitously as the complexity grows. That's part of the reason cost and schedule estimates are notoriously unreliable on big projects.

Reply on news.ycombinator.com