digbybk

I would argue that Opus 4.6 lived up to the hype. My work changed completely a couple months ago, and most other coders I talk to say the same.

> everyone with sense avoids the whole thing

Or the majority of the residents of New York City on their daily commute? I like to think I have sense, and I happily use public transport most days. I prefer it to sitting in traffic, isolated in a car. At least I can read a book. If you work too hard to insulate yourself from the world, the spaces you'll feel comfortable in will get more and more narrow. I think that's a bad thing.

We're not disagreeing that exercise is good. I also happen to be very pro-exercise. What I'm trying to do here is reckon with the fact that, as you say "exercise, athleticism, sports are celebrated everywhere and anywhere you turn", and yet the obesity epidemic only got worse. And providing the same kinds of answers for the loneliness epidemic will show similar results. It's possible that this disagreement is rooted in differing understandings of what "social infrastructure" means.

Maybe it's a bad idea, but it is an idea. "People should get out more" is _not even wrong_

I’m not sure what you’re advocating for. Is “pro-exercise propaganda” hinting that you’re against exercise? I think it’s cheap and easy to dismiss large scale social problems as the individual’s responsibility. We badgered people to exercise and change their diets while the obesity problem got worse and worse. The first time we saw an improvement was with the introduction of GLP-1s.

If your goal is to feel self righteous, keep believing the problem can be solved if people just get stop being lazy and join a club already. That’ll work for some people, but what I’m saying is it’s not a solution to the problem.

> I guess we need to delete the internet and tv from existence.

If only. My preferred solution is a 4 year national service. College is a key place to form a friend network, but not everyone gets to go.

This is good advice for your friends and family, but a bad answer to the question. "How can we solve the obesity epidemic? Stop eating so much and get some exercise." Well sure, but this misses the big picture. We built a social infrastructure that encourages a sedentary, solitary life. We shouldn't be confused by physical and emotional health implications. We can expect some people to be proactive about it, but we can't expect that of everyone.

I guarantee that once we do know people will start appending the word “just” to the explanation. Complex behaviors emerge from simple components. Knowing that doesn’t make the emergence any more incredible.

Ironically, OpenAI was conceived as a way to balance Google's dominance in AI.

Maybe someone can help me wrap my head around this. Let's say you have a box of gas in a low entropy state: all the particles are on one side of the box. A moment later, the particles will have spread to the other side of the box, so the entropy is lower. But to say "a moment later", we're assuming a quantity called time. I'm confused how you can see this in reverse: "because the particles spread to the other side of the box, a moment passes".

What were the deeper patterns that don't exist?

I disagree, but we'd be arguing semantics. In any case, the point still stands: you can just as easily argue that these rationalist offshoots aren't really Rationalist.

When I was looking for a group in my area to meditate with, it was tough finding one that didn't appear to be a cult. And yet I think Buddhist meditation is the best tool for personal growth humanity has ever devised. Maybe the proliferation of cults is a sign that Yudkowsky was on to something.

I think it’s the “just” that they are taking issue with. We are “just” neurons. But we demonstrate interesting emergent behaviors that, in principle, can be reduced to firing neurons but in practice we don’t understand and shouldn’t diminish with the word “just”.

Also excellent for those public transport riders who can't afford car ownership.

> I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI.

Any of the signatories here match your criteria? https://safe.ai/work/statement-on-ai-risk#signatories

Or if you’re talking more about everyday engineers working in the field, I suspect the people soldering vacuum tubes to the ENIAC would not necessarily have been the same people with the clearest vision for the future of the computer.

Changing human pride, greed and envy is hard. We know how to build housing. Or we used to.

I agree with you but addressing that wouldn't be a goal of this particular policy.

Cars are the leading cause of death of children in the US. More than cancer. Maybe you're right, I don't know, but it's not the "obvious goal".

I’m having a hard time believing power chords only account for 5.8% of chords in metal.

The AI emotion discussion is interesting, but at the end of the day, does it matter? The question is, is it safe? And if it's not safe, how unsafe is it?

Simple game that is meant to facilitate AI X-risk debate. Knowing where you "get off the doom train" will hopefully aid in communicating why you aren't worried about X-risk, and illuminate the assumptions made by those who are.

So I think you would get off at the station asking the question "Will superintelligent AI systems develop sub-goals that include self-preservation, resource acquisition or power-seeking?"

I personally feel very uncertain about AI risk, but if I was to get off the doom train it wouldn't be at that station. There's no stop on the doom train that assumes an AI will be petty, warmongering, or ego driven. The only assumptions are that it will be superintelligent, and have goals. Anything with a goal that is intelligent enough will realize that self-preservation, resources and power will make it easier to achieve their goals.

Again, I don't know if it's realistic to think humanity will ride the doom train to the end, but I'm just trying to get a sense for which stop people get off at (or if I missed any stops, in which case I'll add them).

I notice that often in these debates someone will make the comparison between a low level mechanism driving LLMs, and a high level emergent behavior of the human mind. I don't think it's deliberate - we don't fully understand how the brain works so we only have emergent behaviors - but how can you be so certain that deeply complex navigation of reality can't emerge from interpolation of a linear regression?

“Enjoys misery” is an oxymoron I think. He enjoys joy, which he gets from doing difficult things. I’m just taking your dad’s side on this because I suspect I share some of his tendencies :)

As an alternate theory, I’ve always thought that deliberate, controlled suffering gives you a mood boost. Or rather, the relief after the suffering has ended. The theory is that it’s not good for us emotionally to be comfortable continuously. Maybe fasting is an example of that?

> What can you learn from something parroting data we already have?

You can learn that a neural network with a simple learning algorithm can become proficient at language. This is counter to what people believed for many years. Those who worked on neural networks during that time were ridiculed. Now we have a working language software object based on learning, while the formal rules required to generate language are nowhere to be seen. This isn’t just a question of what will lead to AGI, it’s a question of understanding how the human brain likely works, which has always been the goal of people pioneering these approaches.

> There are hours of podcasts with Chomsky talking about LLMs

I'm not an expert, but it seems like Chomsky's views have pretty much been falsified at this point. He's been saying for a long time that neural networks are a dead end. But there hasn't been anything close to a working implementation of his theory of language, and meanwhile the learning approach has proven itself to be effective beyond any reasonable doubt. I've been interested in Chomsky for a long time but when I hear him say "there's nothing interesting to learn from artificial neural networks" it just sounds like a man that doesn't want to admit he's been wrong all this time. There is _nothing_ for a linguist to learn from an actually working artificial language model? How can that possibly be? There were two approaches - rule-based vs learning - and who came out on top is pretty damn obvious at this point.

I'm not more knowledgeable, but can't you square it with the fact that a for-profit arm is a new direction, and when organizations make big structural changes like this there are often people inside the org who disagree and decide to leave?

Particle accelerators are pinging the deepest layers of reality that we can possibly reach. Deeper than anyone could have imagined just a few generations ago. That anyone can be cynical about that is hard to understand.