Hacker News Reader

Comment by ilitirit | Hacker News Reader

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was:

   WHERE Column1 <> Column2

The suggested improvement:

   WHERE Column1 <> Column2

When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead:

   WHERE Column1 > Column2

I'm trying to understand how this helps me become more productive...

View on HN

> I'm trying to understand how this helps me become more productive...

LLMs writing code have strange performance characteristics. Sometimes they produce working output many times faster than the fastest of senior developers, other times they can't understand a bug even when you point it out to them.

Try giving your LLM this prompt:

Please write a python script that loads a file named input.png, converts it to greyscale, plots a chart of the intensity of the pixels of the first row, and saves the chart as output.png . The chart should be a line chart with a blue line, and the lowest and highest points on the chart should be marked with a red cross. Thanks!

So LLMs can sometimes produce working code very fast. Other times, as you've identified, they produce completely broken results.

Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.

GPT4 produces dubious Rust code all the time for me, especially when I point out mistakes. It gets into a cycle of doubling down on mistakes and degrading into more and more bizarre solutions and gets into a loop of repeating itself.

It gets the rough outline of a solution OK but very easily gets lost if the problem requires 2 or 3 levels of what I guess I'd call nested analysis. Or rather, it's fine for creating a surface solution, but it gets lost in the weeds quite easily for things like type system or borrow checker problems.

Because it doesn't reason -- it (re)produces patterns.

ChatGPT is ALWAYS doing stuff like this to me. Suggest code that doesn't compile, suggest SIMD instructions that don't exist, reiterate on incorrect answers with the exact same answer, etc. Granted I use ChatGPT 3.5.

> suggest SIMD instructions that don't exist

well that's one way to make code faster, just require the underlying hardware to compute things for you

Ok I don't use it so much for coding and I always use GPT4 (I got a subscription out because it's so useful).

But every time it gave me a very sensible solution.. Usually not something that worked right out of the box but never something totally stupid.

Reply on news.ycombinator.com