jackb4040

The USS Liberty happened in the 1960s. It's downright silly to pretend the American state would ever have had a similar orientation towards a rival superpower vs its own unsinkable aircraft carrier.

Well it's important to you. It's evidently not important to the people you vote for, the people who fundraise off of this hostage situation, the people who have had trifectas multiple times during this era and have not even used them to permanently secure the "democracy", let alone the overwhelmingly popular policies with their base that would guarantee they win ever election but piss off their donors.

> half

They own the careers of all but a tiny handful of US politicians

It's hard for me to understand Google's redefinition of "world models" (basically: a txt2videogame modality for LLMs) as anything other than an attempt to bury Yann Lecun's definition of "world models", which includes an implicit bear case against LLMs and thus Google's stock price.

From the perspective of your career, the best thing to do is not have objections, or really any principles at all, in the first place.

This is so vague as to be impossible to pin down what you mean in order to refute it without you slipping out of it. But in very general terms, trust busting was a real thing that happened in this country once upon a time, yet would be unimaginable under the current ruling class.

Why would you expect training costs to come down in the future? It seems to me that the labs themselves have concluded they can't stop training due to competition, which is why they're now asking the government to step in and shut down their competitors.

How do we know this isn't just propaganda to force people to learn about the natural world, go outside, and make new friends?

It's a shame you've been downvoted, this is better satire than anything A24 has put out in years. Must be a large number of people who saw that post as a slightly exaggerated mirror held up to themselves and didn't like it.

I have a pretty negative opinion about LLM's coding abilities (and predictions about how they will improve). But I don't get a lot out of this type of argument. Compilers created a perfect, deterministic, provable abstraction so that we could have 99% of developers no longer know how assembly works and not suffer any consequences. Humans don't have to give up any control or make any quality compromises to deploy them as widely as possible.

If the author is contending that AI models will improve to a point that they actually make no mistakes while coding, and have that same level of reliability, then they're on exactly the same side as AI doom trolls just wearing a different costume.

Agree, and I am worried about what happens when that old code is written by LLMs, approved by a human who pretended to have read it, and now needs to be touched by another human who is completely reliant on LLMs to understand existing code.

Seems like a non-sequitur? "because math is formalizable", I agree it makes sense to then predict that data-driven AI models will be able to perform well on it. But where is the evidence that software coherence is formalizable? If it were then I would agree, but to me it seems that coherence in a software system relies on a ton of intuitive reasoning and familiarity with company culture, past failures, etc. it's like the least formalizable thing there is.

AI improves over time, sure, but it has differences and advantages and disadvantages over human intelligence. It's just a fundamentally different thing. In the same way calculators are superhuman, and improving, but can't write books. You yourself acknowledge to some degree the uneven nature of that improvement when you referenced formalizability.

Fantastic framework that I will be adopting, thank you!

One thing I will add: I'm sure the optimistic side will try to take the quantization that your (and other work like SlopCodeBench) are introducing and claim that we'll be able to fix the problem by training on that data, producing coding models that are rewarded for coherence.

I invite them to try! But my suspicion is that this will require a rather deep form of intelligence (world models) that the current language/data-based paradigm isn't really equipped to reach without requiring more energy than the Sun.

In the day since I posted this, I've seen a few good links on HN that touch on this question.

1. The Decoherence Trap: https://maevyn.codes/decoherence 2. SlopCodeBench, a benchmark that quantifies how models decohere over repeated change requests rather than a single one-shot challenge: https://news.ycombinator.com/item?id=50108717

I think the most important takeaway is that it takes time for these problems to manifest. If you start with a particularly coherent codebase it will take years for slop code to degrade it to a point where engineers are noticeably less productive.

I didn't downvote you, but if I had to guess why you were downvoted it's because it's very obvious to many working software engineers that decoherence is a real problem. In many ways that's what this OP was all about; not just loss of joy in programming for its own sake but loss of the ability to design deterministic, fully-understood systems that won't require constant off-the-books maintenance eating into your free time.

No, they're in post-colonial conditions and intentionally prevented from developing economically via a relationship of economic dependence on the imperial core. And as a result, they can't produce their own chemotherapy, insulin etc. and have to import it from developed parts of the world system.

If the imperial core spontaneously disappeared, they would begin developing - and that development would take the form of centralization and specialization and state-driven investment in underdeveloped sectors. Look at what Burkina Faso is doing right now for example: they went from contracts with French companies to export gold and uranium for a thousandth of their market value to now climbing the value chain by developing their own gold refineries.

To keep the example but circle back to the original point: If you lived in a future Burkina Faso that successfully developed its economy, proposing that they go back to small-scale production with worse economies of scale would be a step backwards, and yes the logical conclusion of that in any country is hunter-gatherers. There can be other important learnings from doing this as a constrained experiment within a development model, like resiliency. But thinking this is a development model in itself is a cargo cult mentality that misunderstands industrial production as some evil external imposition on humanity rather than the system that proved most effective at feeding the most people.

Those companies, and those people within otherwise decent companies, have existed since long before the advent of LLMs. Avoiding their influence is a career skill as important as the code itself. And I'm not saying you can just avoid them for your whole career, they're everywhere and are being supercharged by LLMs.

It's disappointing that the prevailing sentiment now is that architecture is less important than we thought. Software is always going to be getting more important, and all of it should be well thought out. If these people had these tools in 2004 they might've made thousands of fragmented buggy social media startups and actually slowed progress on that front.

Thank you! It's made me lose a lot of respect for my field how credulous everyone is to any argument that starts out with a fig leaf of pretending to be on your side.

"The AI labs are scared of their own models, surely they must be credible sources on how powerful those models are"

"They are asking the government for regulation, they must be even more responsible than the average company"

"They are claiming to be a software engineer who automated their job with AI, surely I have to buy their course"

There are many boilerplate pieces of software that an LLM can generate faster than a human, but nearly half a decade into the LLM era there is no evidence that it's leading to the apocalyptic end of software organizations. And that's all with the subsidy in place that is now disappearing. Everyone online is lying for money, and that money is coming out of the same investor-subsidized marketing budgets.

I think you're entitled to your opinion, but want to point out that there's 2 different axes on the graph here. You are unhappy about LLMs' affect on your working conditions, but also concede it's inevitable and the labs are at least somewhat sincere in their predictions of it displacing a significant amount of the value in the software engineering market.

It's not necessarily a given that the person you're replying to agrees on the latter, and therefore they might not be stoked on you starting off by agreeing with them but going on to make an argument that supports the companies responsible for the very thing they're complaining about.

I think there's a contradiction in you stating that management thinks LLM-generated output is "maintainable". They went to business school, they don't know what that means. It's your job to explain to them the consequences of shipping code faster than the people whose job it is to understand it can do so.

> The ability of people to freely associate

> solipsist

This is exactly the point they're making though. This is an exercise only available to the privileged. It is not available to people who haven't worked in tech for some amount of time, if not actively working in tech or holding equity like the homesteader above us.

It takes for granted the concentrated specialization of production that is modern, "on-grid" cities to produce things like microprocessors and food safety and insulin. This lifestyle is not available to masses of people, and we have a word for a society where everyone lives off-grid: "hunter-gatherers".

There can be other value in doing this, like Hundred Rabbits' exploration of how tech itself could be made more resilient given different macroeconomic conditions / incentives. But talking about this as if it's about how The Man just doesn't want you to be an independent free thinker is a misunderstanding of why most people live in the societies that they do.

LLMs are very useful, I use them every day as a software engineer to solve problems and search for information represented within the data available to them. But they are a specific type of intelligence, with many advantages and disadvantages vs human intelligence and it's not clear that just scaling or tweaking them without a theoretical, architectural change will make them more generally intelligent than humans (despite all US AI companies promising exactly that).

They are fundamentally based in language, and achieving deeper models of the world through language alone is deeply inefficient compared to the way humans model the world for years without any language at all. They do not learn at inference time. They don't have semantic understanding of the difference between their own output and other sources. etc etc.

That depth is the key for me. Of course they are capable of producing novel sentences that aren't in their training data, but the depth of that novelty is basically within the bounds of language itself. They are capable of more serious depth and more abstract reasoning than that, but I have experienced limits, which it then tries to surpass with tools to convert things it can't understand back into language (unit tests, LEAN) upon which it is trained.

Because I'm not an AI booster, my account is limited to 5 comments a day. So this is the last reply I'll be able to make today, if you want to continue the conversation we'll have to wait for tomorrow.

Open sourced by whom? Was this a standalone company or a spinoff of Figma or something? I imagine they must have been making serious money to afford that domain.

I do hobby work in Three.js and have always felt more comfortable using the declarative scene.add()-style API to manage my scene vs fiber / reactive stuff. But I imagine I'd feel differently if I were making 3d components tightly integrated with the state of the rest of a normal react app, rather than a fully immersive game scene.

I'm glad both apis exist.

But doesn't that just amount to labs intervening to teach the models to use a particular strategy to mask this one very obvious marker of the difference between their intelligence and biological intelligence? (And similar surface issues like using tool calls / reasoning for arithmetic, even though humans writing on the internet don't typically break show their work for multiplying two numbers)

The deeper architectural difference is still there, which manifests whenever you try to get the models to apply known techniques to modalities and problems outside their training data.

I don't quite understand the leap you're making between stochastic AI models for robotics (maybe with an LLM making api calls to it) and embodied AI / the rate of progress towards a singularity. Because they're both trained on a GPU? Up until 2022 GPUs were for video games and mining crypto, and neither of those produce a synergy that accelerates progress towards general intelligence either.

The American state is the most militaristic and violent state on the planet today, and the American people are the most armed in human history and regularly experience mass shootings and even school shootings.

Drone warfare relies on large numbers of drones overwhelming enemy electronic warfare countermeasures, in urban or open landscapes that haven't been specifically designed against it. A billionaire in a western country against a rag-tag group of guerrillas making drones from commodity radios is just about the least favorable conditions for it that could exist. You may catch a couple of them on accident up front, but they're very well-suited to adapt to it.

Effective altruist / rationalist doctrine is that the best thing a person can do is become ultrawealthy and use that wealth to fund technological progress. It is an ideology tailor-made for those who strive to become the richest person alive, and / or want to worship that person as a god.

When they speak of "existential consequences", it is not as members of humanity. It is a projection of their control over humanity. Personal risk for them is just being made into a regular human (no longer being ultrawealthy). They can't even contemplate physical risk. Case in point, this exact same class of people are buying up private islands and literally purchasing the blood of poorer people.

They would never accept your wager. It's the kind of thing Elon Musk would say "sounds interesting" to on Joe Rogan ten years ago, and then the only consequence after that would be his fanboys reminding everyone how cool it was that he said that.

Targeting? I think you meant to say "upstanding citizen in the vicinity when chemical reaction leads to AI-CEO-involved incident"

They shift between

"we need the government to halt model training for us (and potential competitors who are currently behind us) due to hypothetical sci-fi future bad things"

on the one hand, and then

"we should not face consequences for that time we helped a teenager plan a mass shooting"

on the other so effortlessly, so instantaneously. And virtually the entire media helps them do it!

Huh. Apparently if you Google "GoogleBook", the first 2 results aren't for GoogleBook but for Google Books.