Comment by ben_w | Hacker News Reader

Comment by ben_w | Hacker News Reader

Parent Full thread

> they're not even close to being AGI

Only because all three letters of the initialism mean different things to different people.

Existing LLMs won't do everything, but bluntly: good, we're not ready for a world where there is an AI that can do everything for $1-60/million words[0], and we need to get ready for that world before we find ourselves living in it.

ChatGPT-3.5 has a lot of weaknesses, but it can still do a better job of coding than a few of my coworkers demonstrated over the last 20 years. I'm listening to a German language learning podcast, and the hosts mentioned using it to help summarise a long email from one of their listeners. My sister has work anecdotes about it helping, and she's not in tech. Influencers, teachers, lawyers, Hollywood writers… well, "moral panic" doesn't tell you much… the game Doom was 30 years ago, and that had a moral panic that looks quaint given how much FPS games' graphics improved with each subsequent release, and I suspect ChatGPT-3.5 was to conversational AI what Doom was to 3D realtime gaming: the point at which people take note, followed by a decade of every new release being (wrongly) called "photorealistic".

[0] current pricing for gpt-3.5-turbo-1106 ($0.0010 / 1K tokens) and gpt-4-32k ($0.06 / 1K tokens) pricing: https://openai.com/pricing

> ChatGPT-3.5 has a lot of weaknesses, but it can still do a better job of coding than a few of my coworkers demonstrated over the last 20 years.

Whenever people say stuff like this I can't help but wonder what on earth kind of projects they work on. Even GPT4, while useful for things like reformatting or generating boilerplate code and stuff like that, it's still a far cry from any decent dev I've ever worked with, especially if you're not using a popular language like JS or Python.

My usual PRs at work are pretty big, complex pieces of code that all have to actually work when integrated with the larger system around it, no AI tool I've tried so far has come even close to acceptable here, other than for generating some boilerplate code that I would've written myself anyway. But even with the innocent-looking boilerplate there's always a weird gotcha that isn't obvious until you really analyze the code closely. It ends up saving nothing more than a few keystrokes, if that, yet people say all the time that they're generating entire pieces of software by gluing together code it spits out, which I find absolutely insane given my anecdotal attempts at it.

This can circumvented by going with more elaborate in-depth prompts, but at that point are you really saving on effort compared to the alternative? Is it really more efficient? By the time I have a prompt complex enough for it to spit out something good at me, I could've already bashed out the code myself anyways.

That's not even mentioning all the legacy shit you have to keep in mind for any one line of code, plus whatever conventions and standards your team uses and has etc.

I mean it works great for a function or whatever, but is that seriously what most people are working on? Simple, one-off independent function calls that don't interact in any way with anything within a larger system? Even simple CRUD apps aren't so well isolated.

Don't even get me started on the actual difficult part which is the whole preamble to creating the ticket in JIRA or whatever task management software you use where you're talking with stakeholders and planning out the work ahead, you're telling me you're paying 'Open'AI to do that whole rigamarole for you, and you're doing it successfully?

> Whenever people say stuff like this I can't help but wonder what on earth kind of projects they work on.

Terrifyingly, one of the bad human examples was doing C++. That person didn't know, or care to learn about, the standard template library; and they also duplicated entire files rather than changing access specifiers from private to public so they could subclass; and one feature they worked on was to support a change from storing data as a custom file format to a database, and the transition could take 20 minutes on some inputs even though neither loading before nor after this transition took more than milliseconds, and they insisted during one of the standups the code couldn't possibly be improved… the next day I looked at it for a bit, removed an unnecessary O(n^2) operation, and the transition code went back down to milliseconds. Oh, and a thousand(!) line long block for an if statement that always evaluated true.

The whole codebase was several times too big to fit into the context window for any version of any GPT model thanks to both this duplication and to keeping old versions of functions around "for reference" (their words), but if it had been rewritten to be more sensible it might just about fit into the biggest.

(My other examples were either still at, or fresh out of, university; but this person should have known better).

> Don't even get me started on the actual difficult part which is the whole preamble to creating the ticket in JIRA or whatever task management software you use where you're talking with stakeholders and planning out the work ahead, you're telling me you're paying 'Open'AI to do that whole rigamarole for you, and you're doing it successfully?

If it was all-round good, none of us would have jobs any more.

> Whenever people say stuff like this I can't help but wonder what on earth kind of projects they work on. Even GPT4, while useful for things like reformatting or generating boilerplate code and stuff like that, it's still a far cry from any decent dev I've ever worked with, especially if you're not using a popular language like JS or Python.

I mean this not overly sarcastically, but ... have you seen https://thedailywtf.com ? Between my own experiences, and that of some colleagues, I could probably put together at least a half-a-dozen WTF stories that would rival some of the best that site has to offer. There's enough really incompetent people in positions they shouldn't be in to the point that chatgpt - at this point - could realistically provide better output than more than a few of them.

Reply on news.ycombinator.com