supern0va

Sure, but there will be an upper bound after which we will be close to human level performance on the vast majority of tasks, and then at that point the focus becomes efficiency (or a continuing road to superintelligence for some tasks).

But regardless, compute will get to a point where human level intelligence close to as efficient as we are. You could argue it already is today, when you factor in the resources that the average person in the west already uses in terms of their overall impact on the planet.

>Things did not change in the past 10 years, I see no reason why it should change 10 years from now.

Has it not? Why do you say that?

Also, do we still require a Deep Blue sized supercomputer for chess? :)

>The amount of computations for a human to do the same tasks is thousands of orders of magnitudes less.

That may very well be true now. And in fact, this was true of more rudimentary calculations early on in computing history, where humans were definitely more efficient, particularly for more abstract mathematics. But Moore's Law comes at you fast. Even without more efficient compute, it's rather wild how much more efficient models are becoming these days just from algorithmic and training improvements.

So, maybe for now, certainly. Are you confident that will be the case in 5-10 years? And is that really your barometer for success?

>And when a human learns these things they usually remember how to, and are able to extrapolate that knowledge into new and fresh problem spaces.

That is certainly a limitation for now, but plenty of academic research is being done on how to address that in a more individualized way. That said, the models also have the advantage of synthesizing learnings from user interactivity back into a future release and essentially applying that globally, which is pretty neat.

There's also some cool techniques to sort of bridge the gap today, like compound engineering.

>Next time you get a new and a fresh and an inspiring idea, and you spend hours solving a unique problem nobody has ever done before. You can take comfort in the fact that a few months later some lame and uninspiring developer can write the same problem in a prompt and get the plagiarism machine to steal your work, just in a more lame and uninspiring way.

But that's the thing: it's becoming pretty clear that the "plagiarism machine" can probably take that same problem in a prompt, having never been trained on my code, and still solve it.

In that case...maybe it doesn't feel great to have someone copy my idea. But that is certainly not plagiarism in the way you mean it. And when you put ideas out into the world, you can't be certain that someone else won't copy and remix it into something new. That's kind of how the world works already, but we're just seeing the barrier to entry decline.

>"AI slop" caught on but "clanker" did not.

It caught on, sure, but not exactly in the way I expected. The wild popularity of "slop" as a term for AI eventually gave way to the genericization of the word "slop" to mean "content of low quality, regardless of source", and is seemingly being used as just a derogatory term for anything that people dislike (particularly by folks in left leaning communities). For example, I've seen people refer to (clearly human written) commentary from some political commentators as "slop".

You comment kind of reinforces the idea by the fact that you have to now say "AI slop" specifically to disambiguate it. It's kind of a fascinating little turn.

AI models decompose problems down into tiny pieces that exist in their training data, so in a sense, you're correct.

Though that's also what makes humans so good at solving problems as well, it turns out.

Also, slight tangent: but I do find the "clanker" insult kind of funny. I feel like it counter-intuitively makes the models sound cooler than they are, if anything. I love clankin' shit.

Just to be clear: that is quoted text from the source and not a statement I'm making, in case that's what you're suggesting here.

It's clear that the parent didn't bother to read the link they shared, which articulates exactly this. That's embarrassing.

From the link:

> They summarized their findings from the nine months:

> 1. Humans find GPT-2 outputs convincing.

> 2. GPT-2 can be fine-tuned for misuse.

> 3. Detection is challenging (detection rates of ~95% for detecting 1.5B GPT-2-generated text by RoBERTa).

> We’ve seen no strong evidence of misuse so far.

> We need standards for studying bias.

>

> All these points are valid, and OpenAI did a great job identifying potential risks, especially misuse and biases, at an early stage.

>The ARR were fine but showing skewed quarterly profitability numbers by slowing down research due to hitting compute capacity suggests otherwise.

I have to say, I find this really puzzling. We know for a fact that Anthropic are making bank on metered inference. That's their biggest source of profitability, we are seeing software companies start to majorly adopt coding agents over just the last few months.

Right as the biggest driver of enterprise adoption is accelerating, and it's tied to their biggest profit vector, you find it suspect that their profits are increasing significantly?

Also, can you clarify what you mean by "slowing down research" exactly? Do you mean they're not doing big pretraining runs? Less compute available for researchers? Scaled back RL?

>Also just to confirm, AI subscriptions are definitely being sold at a loss how big I don't know but these models are much harder to run.

Maximum usage of AI subscriptions is a loss, but do we actually know how that nets out? Has anyone done any research to try to figure that out?

On the surface, that's quite fair. However, there's one problem: it is much easier to make statements than to verify them, and that asymmetry is part of why the internet has been slowly eroding society.

It's useful/necessary to use past writing/arguments from an author to say whether they should actually receive any further critical evaluation, or be dismissed. We shouldn't say definitively "they're always wrong, so they're wrong now". However, it's reasonable to say: the author has a demonstrated lack of credibility, so we can probably assume they're wrong here, particularly if they have been wrong in this domain so many times before. Or if they happen to be correct, it's probably not strongly demonstrated by their work.

I highly recommend folks read Wired's profile on him: https://www.wired.com/story/ai-pr-ed-zitron-profile/

Tim Lee also pointed out that when Ed has posted details on some of his analysis, they have had some....oddities: https://x.com/binarybits/status/2034377838883700953

>He was responding to your comment

No, I could be mistaken, but I think he was clarifying his higher up comment:

>Imagine yourself in an isolation chamber. What are you thinking? Are you no longer conscious?

I like how in spite of the author explaining why (father of two small children that occupy his free time), you jumped to the most negative set of possibilities. Instead, it sounds like when he's with his children, he is focusing on them instead of on productivity, which is the opposite of what you're suggesting.

Also, if he instead chose to occupy his drive time with listening to a comedy podcast, or NPR, or even a technical podcast, I can't help but imagine you wouldn't give it a second thought, in spite of that being just as "productive" and "avoiding thinking about the tough things".

>LLMs just wait for a prompt, so they do nothing and are just frozen in place.

I'm not sure that's a compelling argument. Humans can be put into a similar state where they are unconscious and not thinking. Think of someone in a coma, for example, where we actually measure and confirm that there is no brain activity where they're in that state.

They are not actively conscious, but that doesn't nullify their consciousness from when they were awake, right?

>My objection is the strange idea that humans are automatons that are keyed off input like a clockwork machine and operate sequentially. This is clearly not the case.

Well, a few thoughts here. First, it's worth noting that the argument isn't necessarily that AI are conscious in the way that humans are, nor that humans are strictly automatons.

But I think the more interesting thing is that our understanding about consciousness has evolved quite a bit in just the last fifty to one hundred years. We used to think that only humans were conscious, but assumed that primates, cows, dogs, and other mammals were just automatons. Then we started to think: okay, maybe primates are conscious. Then eventually: well, dogs also seem to have consciousness, and then rodents, etc.

This has continued such that most people in the study of consciousness think all mammals are conscious, and the debate is shifting down to insects and other creatures that we do think/have thought of more as automatons. We don't actually know where to draw the line, because it's essentially impossible to really feel/know the inner states of other living beings.

In the face of all this uncertainty, Chalmers just points out that since we understand consciousness so little, that ultimately we should probably be less definitive in pronouncing which things do or do not have it.

Funny enough, the models seemingly go insane and decohere into noise output in the absence of sensory input, which is remarkably similar to what would happen to a human.

That said, I'm not sure I follow what you're actually asking here? I'll also note that I'm not taking a position one way or the other, just sharing a podcast and noting that an extremely reputable scholar on the subject of consciousness seems to have a bit more uncertainty and humility than many commenting here. ;)

I will say, I find it fascinating that there are some philosophers and consciousness researchers who seem to be less certain. I just listened to Chris Hayes interview David Chalmers this week, whose position seemed to be that it's probably not conscious, but that we can't be certain. And more than that: he seemed open to the idea that they may become conscious under further scaling/training/advancements.

It's a great interview, if you're interested: https://www.youtube.com/watch?v=NgDIG8u1-CA

Yeah, I have to admit to finding it somewhat ironic that some individuals accuse the "pro AI" folks of magical thinking, when it seems that escalating levels of magical thinking are being used by the "anti" crowd to suggest that the models can never achieve something akin to human intelligence (particularly in light of the fact that they have on certain dimensions done exactly that).

It's pretty clear that there are significant differences between their intelligence and human intelligence. But that doesn't mean there isn't some sort of intelligence here.

>I'm more interested in the conclusion that programming doesn't require thinking.

I suspect it largely has to do with how one defines "thinking". It seems like people like to implicitly define it in such a way as to require a human (or animal), but there are many examples of thinking/intelligence in nature that don't require a brain or even neurons.

I'm genuinely curious: without using the word "think" with all of its ambiguity, can you articulate what it is that we're doing that these models are not capable of? Because it's pretty clear (to me, at least) from the research, particularly a lot of the mechanistic interpretability work coming out of Anthropic, that the models are at least doing something akin to what we think of as thinking, even if it appears foreign to us.

Like, I'm not sure how you could read this and not see some spark of seems like thinking: https://www.anthropic.com/research/tracing-thoughts-language...

>If anything AI will be used to correct all the crappy human made code that is still being pushed due to the vanity of coders still pretending that they are better than AI at coding.

In my organization, this is already happening. We've been using LLMs to boost our test coverage without touching our human code, then use that as a scaffold to let it go through and refactor, clean up, and optimize, and then validating against both our tests and gold standard test datasets.

In our case, it's made a legacy codebase far more readable to our junior engineers, and the performance improvements (from using an autoresearch-style approach) has resulted in a six figure decrease in our compute spend for the production service we trialed this on.

I'm not sure an article that gives one paragraph summaries of the common anti-LLM talking points is really a substantial contribution to the conversation. This is essentially a snarked up version of a "Criticisms" section one would expect to find in a Wikipedia article on modern generative AI. It's fairly hollow unless you've had your head in the sand and are just getting up to speed on the current conversation.

Your statement seems to be implying (correctly) that LLMs can program, but just not as well as humans. If they're able to program presumably without "thinking" as you seem to be (implicitly) narrowly defining it, then why do you think that limits them to always being sub-par?

It seems like if they can do it, that there's no reason they can't eventually be trained to do it better up to and beyond human performance. It seems strange to suggest that thinking unlocks some nominal margin of "better" specifically that can't be overcome.

All of that aside, even if they can't outperform the top human programmers...what if they get to within a margin where they're still better than most? Isn't a 95th percentile programmer that can run 24/7 and continuously refine its work still going to ultimately come out on top?

I think you replied to the wrong parent.

>It is almost guaranteed that a 60-90B model can outperform current SOTA in coding tasks within 2-3 years.

I don't disagree, but how much of this ends up being distillation? I can't help but imagine that 4.8 was probably trained in part by leveraging Mythos.

If the very large models turn out to be very expensive to run relative to the benefits, it's possible that they could end up still being trained, but ultimately used as a tool to create smaller models that are nearly as effective.

I'm curious if someone here with a stronger background in the space has a similar intuition or not.

Interestingly enough, 4.7 actually did regress on a few benchmarks from 4.6, so it's more than just vibes.

What is the motivation for us users to lie about our experiences? It's to the degree now that people simply refuse to believe that I'm honestly describing my experiences with these tools?

I understand the motivations for the labs to lie, but what do you think mine is?

>Read the other comment in the thread. Your buddy literally confirmed exactly what I wrote.

Please engage in good faith. I commented that humans are the final step of the review process.

Ed actually seems to make some really serious errors in his work. Tim Lee called out a particularly egregious one here, though it's one of many: https://x.com/binarybits/status/2050562429709377986

That said, I really mean it when I say that I don't actually think Ed is a good choice for the anti-AI movement. I think an actual opposition is useful, but he ain't it.

I really recommend you read the Wired profile if you haven't yet and form your own opinion: https://www.wired.com/story/ai-pr-ed-zitron-profile/

>more like their own leak to WSJ and according to Ed Zitron

^ Apologies, the above read to me like you were saying that Ed himself was claiming that Anthropic leaked to the WSJ.

Yep! We have a review process where we have a few agents, each tuned to a particular domain of expertise (security, code quality, etc) which iterate until the feedback meets a certain threshold, at which point it goes over to humans for (hopefully) final review.

That said, I generally agree that you're correct: writing code in many ways has not been the biggest bottleneck. However, by removing much of that writing, it frees up engineers to work on the uniquely human things that are larger bottlenecks.

I had a few comments in a thread here touching on where I think most of the value has come from for us (which is largely search/understanding of our dependencies and making away team work far more viable, which aids with cutting through bureaucracy and the tendency for teams to push back on work): https://news.ycombinator.com/item?id=48298731

AFAIK, most predictions from several years ago were for...approximately now to within the next few years. Can you be more specific?

You criticized a very specific (and fake/misquoted) prediction, ignored the correction, and are now criticizing vague hand-wavey "predictions" that you have left unspecified.

Can you please stop with the angry/ranty replies and actually have a real conversation grounded in actual facts?

Now, having said all of the above...I'll also point out that these are predictions, not promises/guarantees. These people are being asked to forecast and are doing so. I hardly think they should be held responsible for not being literal oracles, but even so--please, at least quote them correctly/at all.

In short: be better than the hallucinations you're seen to call out from the models.

Fair enough, but I have to admit I'm puzzled about why you felt the need to then attribute it to Zitron?