FromTheFirstIn

Would you say those humans understand what they’re talking about?

Yes, I think we simply disagree. I think you know what LLMs are based on our other thread and if you think something important happens when you get a large enough context vector I don’t think there’s much I can say to change your mind. It seems unlikely to me!

Way better- thank you!

We’re in all the same threads lately!

I think if your definition of consciousness can fit these things then you’re more open minded than I care to be. Consciousness isn’t really guessing the next thing to say- it’s hard to say what it is, obviously, but blindly feeling forwards with each new conversation doesn’t seem like consciousness or learning to me.

The styling on the website makes me feel like my phone is a cylinder

While being slow to pass judgment or disregard an approach is a valuable trait in a senior Eng, I think 3 years is plenty of time to wait for proof of concept to pan out. It’s not panning out, it doesn’t seem on the verge of panning out, and soon the real cost is going to be passed on and the subsidies will end. LLMs see ripe to be the new IDEs, but not the new Engineers

To acquire new knowledge and build your understanding. They don’t understand so they can’t learn

You’re right, I’m very suspicious of HN when it comes to AI apologetics, but I shoulda trusted the parent commenter more.

Yeah, it not a linear cap (x% entropy doesn’t mean x% wrong) but it does seem like a hard cap. To be honest, the more I’ve understood scaling laws the more I think that the elephant in the LLM room is the entropy of the language. It explains why coding languages are so much more tractable (they’ve got WAY less entropy) and it explains why we haven’t seen a step function in capabilities for LLMs since GPT-4 outside of making specific toolings for particular contexts. I think E is coming to dominate and there isn’t a workaround for it.

This is a great read! I never realized the scale of the effort to find that BSD vulnerability- helps put things in perspective

Isn’t the limit exactly what you’re describing? There’s always uncertainty, and your asymptote can approach its limit but it does have a limit. That’s the limit to the intelligence. And this is just for cross entropy loss- even if you could get loss to 0, I’m still not convinced at all that an enormous semantic map and its convoluted geometries amounts to intelligence.

Agents are always combining the same underlying weights to their inputs, relying on the same maps of semi-semantic space and the relationships between those that it was leaning towards at training time. The fact that it’s successful in making lots of people have an Eliza effect doesn’t make it understand something. It’s simulating understanding based on an enormous corpus of text, much of which is people working through things or sharing an understanding of something. Unless you believe that all intellectual activity is about finding the space between words you shouldn’t believe LLMs have any chance at understanding anything.

The first sentence is “understand or learn any intellectual task that a human can.” Whatever you think of the benefits of LLMs, they don’t understand and they can only learn during the training period and with very minor adjustments in post training. So, no I don’t think any of these models are generally intelligent.

My understanding is that the true entropy floor of a language is intractable- regardless of context length there will be “unpredictable” tokens where cross entropy loss is bound to happen. Even with infinite parameters and data you’ll still have a chance at failing to predict the next token correctly a decent chunk of the time.

Also, linear gains in context length scale quadratically with compute because of attention, so depending on context growth means taking a bath on GPUs for as long as you can, right?

And sitting right next to the data and compute factors in every cross entropy loss equation is the entropy of the language, which is just a fixed constant. There’s such a hard cap on cross entropy loss training and I never hear it come up!

You’d have to really stretch the definition of AGI to make the current models fit

Cheating is always logical for the cheater unless they’re discovered and held to account. I’m not sure what your comment is pointing out besides that it’s possible, but worth saying: just because you can cheat and would benefit from cheating doesn’t mean you’re not culpable for cheating.

Buddy it’s not even going to replace 1%

But they are, so it’s demoralizing to be punished for not hitting the jackpot

Maybe slop?

The deciding power they’re talking about is the power to decide to use or not use an LLM.

It’s very funny to watch an AI bull deal with pushback. If people are gushing about this being able to take over all knowledge work in a span of a year and a half, isn’t it reasonable to hold it to a standard you’d hold an average developer to? If I said I wanted a cache that busted based on memory and instead it busted based on entries, I’d definitely tell the engineer they weren’t paying attention. I certainly wouldn’t be parading around the company telling everyone they’ll be a Principal Engineer in 18 months

This is such a great article- I’ve had so many runs where it’s felt like “why am I always getting this random card?” And now I’ll know! Thank you!

It always gives me Clash :(

If you’ve played the game it makes sense to have the seed be settable and shareable. In Slay the Spire it can be exciting to have an outrageously unlikely starting state or early option, and in order for players to share this with each other the seed has to be user controlled. It’s a big part of what gives the game its community!

Watch out, an AI bro is about to tell you you can’t read!

Their point is that labor they consider low impact or menial doesn’t drive returns, and therefore shouldn’t share in the returns. You’re right that the labor being physical is incidental, really they’re just classist/elitist and any job they consider beneath them would fit this model, while others wouldn’t. There’s a reason they chose a cleaner (and a woman!) instead of a product manager or CPA, though the quality is also unlikely to differ between spaceX and Walmart there.

Speaking of reading comprehension, they didn’t address the core argument of the person they were responding to, which is that labor that falls “beneath the fold” of this class line is not able to negotiate aggressively due to the inelastic costs of food, shelter, and basic necessities. It doesn’t matter how “high impact” you are, if you’re negotiating and need to eat you’ll accept any amount that lets you eat.

In fact, having impact or driving revenue is never the most important factor to reaping the rewards. Anyone who’s worked for a few years with their eyes open should reach this conclusion unless they have some strong motivation not to.

Google has made searching also include a default LLM response. There are also search results. When you search with Google, an LLM is prompted, for sure. It’s also pretty clearly a separate feature that’s been tacked on to search.

Just AI fandom