Comment by monkeynotes | Hacker News Reader

Comment by monkeynotes | Hacker News Reader

I see posts like this comparing past attempts, and in this article even 1990s Homestead, and I immediately think this person does not get it.

Additionally all the statements that follow along the lines of "GPT today has a hard time with xyz" are all problematic too. Today is not tomorrow, and what it can do tomorrow is what we should be talking about.

The fact is in just 3 months we've gone from fairly basic code reviews to super helpful code reviews with code examples and intelligent code contextual comments.

Comparing GPT to Homestead or any number of early attempts to translate mockup to HTML+JS is unhelpful. This is wildly different technology. Talking about what GPT can do today and landing at "it's not good enough" is also missing the point, we are nowhere near the limits of this tech and it's evolving so fast that these assessments are not insightful in the slightest.

Talking about GPT as being merely a probability evaluator is also demonstrating the authors limited insight. WE ARE ALL probability evaluators. Humans work on prior experience, employ heuristics to make decisions based on that. GPT, indeed all AI, is more or less based on that paradigm. The difference is an AI can hold the whole of its learning in a perfectly memorized model. It doesn't forget what's in that model and can draw on billions of data points to make its probabilistic determinations. We pull on bias, false memories, misunderstandings, and so on. Now, that's not to say AIs won't also be afflicted by these comprehension errors, but that just doubles down on AI doing what humans do which is astounding either way.

Jobs are going to be lost, it's a fact. Even if it's just that junior who was learning code reviews and supporting basic development, that job will be around for maybe a year or two at the rate we are moving.

> This is wildly different technology. Talking about what GPT can do today and landing at "it's not good enough" is also missing the point, we are nowhere near the limits of this tech and it's evolving so fast that these assessments are not insightful in the slightest.

Just as often with AI's detractors who claim that it "merely" predicts the next word, I have to wonder if those making these grandiose claims about the near future of AI have actually used GPT-4 and its predecessors in their work. There has to be some sort of logical fallacy regarding "well [insert literally any limitation here] won't exist in GPT(n+1) at this rate."

> Jobs are going to be lost, it's a fact.

What percentage of a programmers' time have compilers automated? It's gotta be multiple orders of magnitude of efficiency gained. I could see a GPT-4 based GitHub Copilot being something like a 3-5x improvement on development in the best use cases (e.g. CRUD development built on public frameworks.) That's a colossal impact, but it's hard to even put on the scale to other ways programmers have automated their work in the past. Every time programming has automated giant swaths of programmer time, the end result has been that programmers can generate far more economic value even faster. This end result has consistently (and paradoxically) led to far more work and jobs for developers than before.

> Even if it's just that junior who was learning code reviews and supporting basic development, that job will be around for maybe a year or two at the rate we are moving.

At least in my admittedly niche area of work, juniors just aren't all that helpful. The time they take away from seniors or other members of the team makes them at net negative for at least a few months. It's often a year or more before we reach break even. Junior positions are investments in potential future independent contributors at a time where filling IC roles is still extremely difficult.

> I have to wonder if those making these grandiose claims about the near future of AI have actually used GPT-4 and its predecessors in their work

I have used GPT-3 a fair bit, and used GPT-4 today. The differences are impressive, scary impressive so far. I am basing my `GPT(n+1)` observation on the rapid rate the tech is filling in holes that previous detractors pointed out.

The differences in code reviews is amazing. In GPT-3 it basically explained the method and said it looked good, "add some comments" was its only suggestion. GPT-4 on the other hand gave a code snippet to explain why JS "await" was unnecessary, suggested better variable names, and suggested moving some imports to a more simple file structure to reduce verbose import paths.

I think that's a beyond expectations improvement, it's so impressive it beats out the humans who reviewed the code and we implemented one of the suggestions.

If GPT-4 had ingested the entire code-base code reviews would easily be automated, humans would still do their passes but as a co-worker GPT would be invaluable. It would find swaths of missed efficiencies, fix slow SQL queries, look for testing gaps, and perhaps even assist in refactoring.

Personally I think we are underestimating the potential, I feel like we are where the world was in 1990 looking at the emerging internet. So many people underestimated the impact of the internet, many technology writers said it was hyperbole, couldn't replace bricks and mortar stores, email was never going to be for everyone, and so on. All of it wrong, profoundly missing the potential.

This stuff is beating out expectations at every release and I think that is reason enough to take it seriously and be much more considered when downplaying the hype.

As for junior devs, we don't hire IC1 for the same reasons you outlined. However, small studios do because they are cheap and serve a purpose where it's just WordPress maintenance and HTML POCs. Those jobs will disappear.

> If GPT-4 had ingested the entire code-base code reviews would easily be automated, humans would still do their passes but as a co-worker GPT would be invaluable. It would find swaths of missed efficiencies, fix slow SQL queries, look for testing gaps, and perhaps even assist in refactoring.

This is where my 3-5x efficiency improvement estimate came from for GPT4-based Copilot, especially for CRUD apps using popular frameworks, which while game changing is far from the great efficiency gains programming has seen in the past.

Compilers have conservatively led to 100x-1000x gains of efficiency at scale as have operating systems and various key libraries. GPT-assisted code generation doesn't yet seem to offer the same result. Even if it did, if history is our guide, this would lead to developers being able to generate far more economic value with less, which has consistently led to more developer jobs than before.

> Personally I think we are underestimating the potential, I feel like we are where the world was in 1990 looking at the emerging internet.

It doesn't feel correct to me to think of GPT-x as some technology similar to the internet as much as it is a way of experiencing the already existing internet, like a smart, context-aware search for specific details (though there aren't any good analogies for the role LLMs play/will play in tech moving forward.) If search engines had continued to improve with the momentum they had in the early days, we should've expected some experience not unlike GPT a decade ago (or more.)

> However, small studios do because they are cheap and serve a purpose where it's just WordPress maintenance and HTML POCs. Those jobs will disappear.

That seems very possible.

> Jobs are going to be lost, it's a fact. Even if it's just that junior who was learning code reviews and supporting basic development, that job will be around for maybe a year or two at the rate we are moving.

And how is that not a disaster for those who are out of work?

I don't know. It sounds like a rather substantial problem for those affected.

Reply on news.ycombinator.com