Hacker News Reader

Comment by JimmyRuska | Hacker News Reader

Having spent a lot of time with GPT-4 recently, it's possible the new generation will not have the same issues we did. For example they can write some code and say, "adjust for best practice".

Then "explain why <my code> needs to be <new code>"

Oh this is called <new concept>, give me an example of it. Give me another example. Give me another example. Give me use cases where this would be better. Give me a short story where it was done my way but it lead to a mishap in production and the protagonist learned <new concept>. Explain it like I'm 5. etc

Basically the new generation is the training neural network, and they can keep flashing context until they absorb the material. Sure, some people are lazy and will go as far as getting something barely working. But superpowers are available for those who fight for their place in the world

View on HN

> Then "explain why <my code> needs to be <new code>"

It would be nice, but until there's a breakthrough on having LLMs / other models generate, maintain and walk the user through a representation of their reasoning, these explanations are likely no more reliable than a random StackOverflow answer - except without the benefit of other users scoring / picking holes in the answer

Have you tried it yet? It is significantly better than a stackoverflow answer. I've been using copilot since release and have now integrated chatgpt into that workflow (thankfully copilot using chatgpt 4 and some tricks I figured out to treat copilot more like a chat bot are making it so I can avoid the chatgpt browser).

It may be significantly better than a random stackoverflow answer, but I haven’t seen it as being better than the top-voted stackoverflow answer (at least not without significant curation/prompting).

i think maybe you're using ai wrong. its not akin to stackoverflow answers, its more akin to a jr programmer that can bang out some code for you while you go to the kitchen or bathroom and then you fix a few mistakes and integrate it with your larger project

I have played with it a small amount. If I need to center a div in CSS, it’s faster than me remembering how to do that.

For anything significantly higher complexity, I don’t find that typing is anywhere near the rate-limiting factor. Figuring out how to decompose the problem, how the code should be structured overall, and other high-level concerns take far time/effort than banging out boilerplate.

Now, if I wanted to pretend to be a junior developer for three different companies at once, it would be a heaven-sent technological marvel.

interesting takes. i guess if you're using ai for basic things like that it makes sense to not like it

Enlighten me, please. What’s the killer use case where you’ve seen significant leverage from an AI tool? What was the business problem and what were the resulting prompts to your AI junior dev?

i am trying to solve many business problems as I am building apps that allow people to use stable diffusion and LLMs locally on their own hardware without the need to install anything and i'm doing it with compiled python that I have to ensure runs on various machines. this means i have to support customers. I'm also making a couple of small games. I also have to handle marketing.

the killer use case is augmenting my own workflow. i have fully integrated ai into my development pipeline. i am using chatgpt and i am using copilot. I almost never find myself turning to any other resource for information other than occasionally having to reference documentation for very obscure libraries. I also use an IDE.

this gives me time to concentrate on things such as my ci/cd pipeline, architecture, UX, testing, solving how to install bitsandbytes and triton on windows and other difficult problems related to build processes that don't have clear cut solutions.

I see a lot of skepticism like this on hackernews. People constantly downplaying how impressive this technology is. It's great to be critical. But most of the criticism is too stuck in the present. If it's already as good as many stackoverflow answers, what kind of potential will we see in 1 year, 3 years, a decade. This is massive paradigm shift in the way we program and this conversation thread will be look outdated in a year. So much of the tedium of programming has been eliminated in my day to day. I actually have ideas I pursue now that I know so much friction in the setup process is gone and I won't be severely blocked by bugs.

As a skeptic, this reads like "look how great Google search is at finding stackoverflow answers already; imagine how good it will be in a decade!"

i keep emphasizing this: i don't think we'll be programming any more down the line, i think we'll be engineering with an AI assistant like tony stark. obviously i could be mistaken but it looks like things are heading in that direction.

Have you tried giving it the problem statement and asking for a high level outline of the code and then iterate on that until it’s right and then ask it to generate pseudo code for one item at a time, correct that and then generate the code?

I get a lot of "ok, I've changed this, this and this because X, Y and Z" style answers even without prompting. It really wants to explain what it's doing, to the point where I have to sometimes say just shut up and give me the code. (But politely. Gotta be polite to our future AI overlords).

My theory is that people working inside frameworks (like me) are getting significantly better answers than people doing "freehand" coding because the training data's so much better in framework-world.

Yeah but you can run it, profile it and execute your test suite on it. If it passes and if it is more performant, aren't you done? I can try all the stack overflow answers and just pick the best one now.

No further breakthroughs are necessary, if you're experienced enough you have enough general context to tell when the advice is good and it's helping you understand a new subject and it absolutely is great at walking you through understanding something new in a broader domain you're already familiar enough with to detect when it's lost.

> Having spent a lot of time with GPT-4 recently, it's possible the new generation will not have the same issues we did. For example they can write some code and say, "adjust for best practice".

Not iteratively forever, the only feedback it will (eventually) get is it's own feedback, because humans will be using the AI best practices, and what it deems as best practice is almost certainly going to be too opaque to us mere mortals.

Nah, it'll still be human readable. that was the whole point of C and Java. Let the computers to the hard stuff in bytecode or asm. Compiler development will get even more complex though.

Training data is key, a lot of people will be employed in creating, curating, and labeling it. They will utilize AI to help them but it will be a partner. If AI reaches the point of creating and using its own training data then you can hang up your hat and call it a day because no human input or work will be needed at that point.

Yeah kind of an interesting journey. We went from people writing content for people, to bots writing content for people, to people curating content for bots (ultimately for people). The only permutation left is bots curating/writing content for bots and people are consuming only bot-generated content. In a way we are partially there since some content that was in these models was written by bots.

Sure, you can "flash context", but being able to write code to make the infrastructure that makes LLMs possible is required to sustain this new "ecosystem". I don't think every dev will be equally hit by this new paradigm, it's basically a two-tier system now, the LLM babysitters and the actual competent human coders (whose input probably gets fed into a LLM).

s/LLM/StackOverflow/g and squint, and you're basically describing the preceding decade.

Well, stackoverflow required competent users to participate. There's no need for that any longer. The "gap" will probably be addressed by LLM-designed tools that are only usable via LLMs and bigtech will be more than happy to put their own LLM-designed tools at work for their products, which means there'll be vendor lock at scale, so in the vein of the "digital realm emulating the analog realm" there'll be companies you'd have to specifically train for and none of it would carry over...

but this is assuming this current level of progress stagnates for at least the next five years. Sigh...

> There's no need for that any longer.

I don't think so:

LLMs have used Stackoverflow's answers for training. Now, if people would stop going there for questions, people would stop giving answers, so LLMs would have no data to train on for new problems.

I think unless there's a reward system for contributing to LLMs, the internet will mostly be devoid of new content. People will move to walled gardens. All you need is a short demo showing how something like GPT-4 with the internet access plugin can make a profile of you based on your publically available comments.

> LLMs have used Stackoverflow's answers for training

have used, perfect tense. But future tense? who knows.

We shouldn't assume future LLMs won't be able to parse documentation of new libraries and extrapolate full, detailed implementations.

This assumes that a future version won't have super-human coding abilities which can write code vastly better than any human. The language it writes in might not look anything like our current languages, because it might be able to write highly efficient code that it would be too slow and impractical for a human to write, and refactoring the entire codebase might happen automatically overnight rather than slowly across months.

In that world the skill won't be to code, but to accurately describe software/business requirements. Unless a future AI can do that too.

There always seems to be a lot of "We Can Do The Thing We Do Today But Faster" in these comments.

It misses that we won't be doing Old Thing. We'll be doing New Thing.

It's not a step change in an old medium. It's a qualitatively different change to a completely new medium, whose properties haven't yet been revealed.

It's like waving an iPhone at Babbage. An iPhone is so far ahead of an Analytical Engine that it offers a completely new and unimaginable set of affordances.

It's also faster, smaller, and more reliable, with much more memory - which is an interesting implementation footnote, but incidental to the fact that from a user POV it's a completely different class of device.

I agree, but that's a long time away, maybe five or six years. Until then (and even after) we'll need people that can interface with lower-level systems and also have prior experience. This also doesn't take into account countries that will effectively ban usage of companies like "Open"AI (which "Open"AI will be more than willing to enforce), and on top of a possible thin-client revolution, we'll see demand for human coders in the new tech-ghetto parts of the world.

Countries would be insane to ban companies like OpenAI. What a huge foot-gun that would be!

Imagine if a country banned computers in 1979 to protect jobs, other countries would have run circles around them and it would have had huge economic implications.

> For example they can write some code and say, "adjust for best practice".

And they’ll get code which isn’t good, just a parroting of bad ideas widely disseminated. A bit like Casey Muratori’s concept of fake optimisation: https://youtu.be/pgoetgxecw8?t=600

OT: Are you _the_ Jimmy Ruska (jimmyrcom on YouTube)? If so, thanks for your videos. I remember watching those as a teen, they were super inspiring and solidified my decision to pursue a career in IT.

I keep getting surprised every time I read that when someone posting on HN was a teen Youtube already existed.

These were the very early YouTube days :^) I'm turning 32 this year, started coding 19 years ago...

What you describe is basically how coders coming up in the last 10 years have been learning -- specific context from StackOverflow or wherever. Contrast this to the hierarchical textbook learning which came before it the intention of which (I think) typically was to impart knowledge which generalises well.

I think what we have now is more productive but results in worse programmers. I.e. you fix the thing in context, but learn little out of context and have to go back to the source of answers when the next thing comes up. The rate at which answers are sought increases, doesn't decrease. Etc.

Reply on news.ycombinator.com