Hacker News Reader

Comment by jdlyga | Hacker News Reader

From my experience, Github Copilot is useful for its seamless autocompletes. But Github Copilot chat, a new feature, noticeably lags behind GPT-4. I get far better performance copying and pasting code into ChatGPT (GPT-4) than using Github Copilot. Both definitely help productivity, especially when generating code using an unfamiliar framework or library. It's great for generating skeleton code for small programs using new libraries, but it in its current form can only get you around 80% of the way there. You will spend hours fighting with the prompt to try and get the remaining 20% complete when it's much easier to just finish it yourself.

View on HN

And for those who say that it doesn't speed up development, then maybe they're right, because the main benefit isn't at speed. It reduce the devs mental / cognitive load and pass it to the ai instead, we just need to check and review the result.

If it doesn't monumentally increase the speed, it helps devs to stay productive longer

Reviewing code is famously harder than writing it. I don’t see how it’s a gain to have to review more code than to write.

It's shorter, you're working directly with the code base you're already thinking about, and it's plugged into it rather than having to start a new mental task when you're looking at a pull request.

When I write code, I usually get a complete understanding of the problem and constraints. I have a picture in my mind of all the edge cases and branching that can happen, which most of the time allows me to write correct code without bugs.

When I review code, I am completely unable to get this mental picture, and I usually miss a lot of issues that I would otherwise have avoided.

Which is why I don't see the point of Copilot.

When you review pull requests you're unable to get the mental picture. But it's different with copilot because it fits in your mental picture, you can compartmentalize what you're working on, and the copilot output can come in the form of small pieces that are easily understood and slot in the prepared places where you need it.

I agree 100% that reviewing human code is a lot of work but not with copilot IMO because of the immediacy of it all.

For me to understand what you mean here, I'm going to need examples... for the stuff I work on, the things you are saying here are verifiably impossible.

There is no possible way to 1. get a complete picture 2. understand all the constraints

because we pull data from the real world, and the real world is constantly changing, and we still don't even have a single theory of everything in physics.

In some cases where I often use it I find it easy. Usually I give an ORM class and ask it to parse to mysql / postgres create table. Tedious tasks and easy to review.

Another time I asked to generate a code to listen to SMTP server using a library, and again it's easier to review because I was not familiar with the library. From that I can make my own adjustment.

This has been exactly my experience. It is great at autocomplete and boilerplate stuff. It's very hit and miss for StackOverflow type of stuff. And it's usually a complete miss when you ask it something where it needs to understand even basic things of what you're actually trying to do. It can also get very confused and circular when it hits those edges as well. I've found myself fighting with it, and so part of my use of it has been learning when to let it go.

I find ChatGPT 4 is the best place to ask a question, then often use it's answer to find more in docs or stack overflow. It's basically a better search engine for certain coding tasks. I wasn't overly impressed with ChatGPT 3.5 as it gave me wrong answers enough of the time that it cancelled out the amount of time it saved me.

Weird, because Copilot chat /is/ GPT-4 as far as I know. Or maybe they run it on 3.5?

It runs both. Had seen this on the discussions on Github.

https://github.com/community/community/discussions/58059#dis...

That original note about it using both 4 and 3.5 was from this comment on November 2nd: https://github.com/microsoft/vscode-copilot-release/issues/6...

GPT-4 Turbo - the much cheaper version of 4 - was announced on November 6th. So there's a good change they've switched to that at some point in the last 6 months.

They mix it to save on compute.

That's been my experience as well. Doesn't help me that much in Django where I am pretty familiar. It helps me loads in the frontend, where I am less familiar.

Reply on news.ycombinator.com