Hacker News Reader

Comment by kortilla | Hacker News Reader

The rust produced by LLMs is quite bad. It’s overly verbose (misses combinators) and often subtly wrong (swallows errors on result types when it shouldn’t). A single errant collect or clone call can destroy your performance and LLMs sprinkle them for no reason.

Unless you are experienced in rust, you have zero ability to catch the kind of mistakes LLMs make producing rust code.

View on HN

I just had an LLM (something 3.7) propose a 100 line implementation and after I stared at things for a while, I reduced it to one. I’m sure I’m in the minority of not just accepting the 100 line addition.

Could you elaborate? What were those lines doing?

Lots of the lines were safely dereferencing things in JS where you could just do ?., putting try/catch blocks around that (each time!) and calling into things in a way that a single line would have sufficed. I almost took a screenshot for the lols.

Be honest, it was one line that should have been broken into 3 for readability. This vicious calumny must stop, only 97% of LLM output is rubbish and not the 99% you claim.

> Unless you are experienced in rust, you have zero ability to catch the kind of mistakes LLMs make producing rust code.

I'd say it's on par with other languages then... LLM are roughly 95% correct on code generation but that's not nearly enough for using them.

And spending time finding which 5% is looking good but actually wrong is a frustrating experience.

Those programs are making different kind of mistakes than humans and I find them much harder to spot.

Is this deficiency likely to persist long-term as LLMs grow more powerful?

I think one issue is less extant Rust code to train on. As more Rust is written and published, LLMs should in theory get better.

I heard most code today is written by LLMs. I hope that's not true, but if it is, the future improvements will be slow (barring something like, actually teaching them to read the docs).

Reply on news.ycombinator.com