Hacker News Reader

Comment by armchairhacker | Hacker News Reader

Copilot is a game-changer and very underrated IMO. GPT4 is smart but not really used in production yet. Copilot is reportedly generating 50% of new code and I can't imagine going without it.

View on HN

Where do you get that 50% number? Do you mean 50% of all new code in the industry? That seems beyond extremely unlikely.

(I read it as 50% of their code)

The number is 40%, and it's 40% of code written by Copilot users. It's also just for Python:

> In files where it’s enabled, nearly 40% of code is being written by GitHub Copilot in popular coding languages, like Python—and we expect that to increase.

https://github.blog/2022-06-21-github-copilot-is-generally-a...

I wonder if this properly counts cases where copilot writes a bunch of code and then I delete it all and rewrite it manually.

From what I remember they check in at a few intervals after the suggestion is made and use string matching to check how much of the Copilot-written code remains.

It's all about the denominator!

There was some discussion by the copilot team that x% of new code in enabled IDEs was generated by copilot.

It varies, but here's one post with x=46 from last month. So, very close to half.

https://github.blog/2023-02-14-github-copilot-for-business-i...

Measuring output by LOC is not a very useful metric. The sort of code that’s most suited to ai is closer to data than code.

I would really love to see that. So far, all I've seen is cookie cutter code to reduce a bit of typing time. Everything else was more or less hot garbage that just stood in the way of typing. Maybe in a few iterations or years. So far, personally, I haven't seen anything useful. Not saying there isn't anything, just that I haven't seen any use and code offered by it stank. Is there a demo of someone using it to showcase this game-changing power?

It surprises me to hear this. Have you used it as I described by writing a descriptive comment first then waiting to see its response?

I only noticed it getting good at this after I was somewhat far along on a project, so I assume it requires an overall knowledge of what you're trying to do first.

Copilot only writes boilerplate, it can't really handle anything non-trivial. But I write a lot of boilerplate, even using abstraction and in decent programming languages. A surprising amount of code is just boilerplate, even just keywords and punctuation; and there's a lot of small, similar code snippets that you could abstract, but it would actually produce more code and/or make your code harder to understand, so it isn't worth the effort.

Plus, tests and documentation (Copilot doubles as a good sentence/"idea" completer when writing).

For my side projects, copilot easily generates 80% of the code. It snoops around the local filesystem and picks up my naming schemes and style to help recommend better. It makes me so much more productive.

For work projects, I tried it on some throwaway work because we're still not allowed to use it for IP reasons, but it is very good at finding small utility functions to help with DRY, and can help with step by step work, but can't generate helpful code quite as easily since some of our API and codebase just doesn't follow its own norms or conventions, and it seems to me that copilot makes a lot of guesses based on its detected conventions.

> It snoops around the local filesystem and picks up my naming schemes and style to help recommend better.

Are you sure about this? It doesn't seem to work on my machine. I think it will infer things that might be in other modules, but only based on the name. I'm basing this on the fact it assumes my code has an API shape that's popular but that I don't write (eg free functions vs methods).

It looks at your recently-viewed files in your IDE. I don't think it looks at anything outside your open workspace but maybe...

Reply on news.ycombinator.com