Copilots are like TV doctors. They know a lot about their subject, but

techtarget.com

1 point·torstenvolk··4 comments

LLM reasoning is not logical

Thank you for pointing out the obvious.

The results are probablistic --- not deterministic.

This is a fundamental design characteristic. And it severely limits their utility. Using a LLM for an application where the results are essential is likewise illogical.

Giving an LLM authority or control over anything important (like say your bank account or stock portfolio) is a recipe for calamity.

Unfortunately, some people in positions of authority either don't understand this or they don't care.

https://gizmodo.com/trump-medicare-advantage-plan-artificial...

Yup, good point. They don't understand or they don't care or possibly both. The frustrating part is that this is still a hugely impactful technology, even without showing us a clear path toward AGI. But if we keep talking about AGI being the 'next stop' on this journey, we'll burn a lot of excitement and trust in LLMs...

we'll burn a lot of excitement and trust in LLMs.

Maybe trust in LLMs needs to be burnt.

Just for fun, try asking an LLM if it's output is trustworthy.

I'd be curious to hear everyone's opinion on this analogy. The difference in 'reasoning' between LLMs and human developers is critical, as LLMs use probabilistic calculations, while humane use deductive logic. That is why I do not think that copilots will replace human developers anytime soon.

Reply on news.ycombinator.com