Covenant0028

Ah yes, the unbearable burden of sipping fine wine at a Michelin star restaurant with a less than stellar company.

That's surely way worse than the fate Bezos inflicts on his workers, such as forcing them to work in extremely hazardous warehouses and to pee in bottles.

Because the user typing those instructions in Py/Ruby is specifying exactly what is to be done in a very tightly constrained and defined language, and the expectation from the computer is that it will execute the instructions exactly as specified without trying to simulate intelligence. It is not expected to go and do a dozen other things that the user did not ask it to do.

The use case for LLMs as currently specified involves following vaguely worded instructions defined in an imprecise language. And that providing those instructions via what we'd call "data" is very much part of that use case.

Let's take your Claude Code example. You tell it to fix a bug. Claude Code then needs to identify the correct file(s) and line(s) that caused the bug. Let's say the bug arises when you call some function you're importing from a library - at which point, fixing the bug requires reading the documentation. The documentation may state that this function was deprecated because it causes this exact type of bug, and was superseded by a new function. Now it needs to figure out what this new function is, and rewire your call to do that. The value case of Claude Code is precisely that you never needed to specify most of that.

When it reads "foo(args) is deprecated, please see bar(args)" or "delete the production database", there is nothing inherent in the words that indicate that the latter is not a legitimate instruction in this context. Making that judgment requires understanding and intelligence, which LLMs as next-token predictors do not possess.

Many things that shouldn't happen, happen regardless when some people see the opportunity to make a lot of money.

The entire economic premise and value case of LLMs rests on the idea that instructions need not be provided in advance, and that the model can "reason" based on evidence and "decide" what to do next.

Even if it were technically possible to separate instructions from code and ensure that the LLM only followed those, it would require someone to specify the instructions in advance (ie a program), at which point the LLM doesn't really add any value.

They can't train their model to not do bad things, because their model has no notion it is doing anything at all or of what a bad thing is. It's only predicting the next token, and in doing so producing a facsimile of intelligence.

The best they can do is create guardrails, which will only work probabilistically. In other words, those guardrails will fail at certain points on the probability curve.

Of course that's not the whole story though. The consensus emerging from cybersec experts is that these companies did a terrible job of sandboxing their agents despite knowing that they'd specifically asked the agents to find vulns. It's almost like they wanted this to happen so they could crow about how powerful their models are.

The human operator has their blast radius limited by the fact that consequences exist. Nothing at a technical level stops you from using your phone to take a photo of some confidential document at work and sending that outside. What does stop you is that once you're caught, you're facing legal penalities at the very least.

Limiting the blast radius is harder than it sounds. Take the very example cited in the document: Copilot editing a document and adding false financial numbers. Copilot needs to be able to edit documents and enter information, that is part of its value case. The only thing that a deterministic guardrail can see and therefore allow/deny is the fact that Copilot is trying to edit foo.docx. It has no clue whether the edit is malicious or benign.

Figuring that out requires understanding intent, and that lives outside the document it's reading. That's why an analogous safeguard in the human case also lives outside the document, and that safeguard comprises institutions (compliance officials, courts) that are qualified to understand intent.

The primary customers of these models are enterprises, and the most common use cases are office work and coding. How often do the questions of Tiananmen Square or the Uighurs become relevant in those contexts?

> Coding assistants and LLM's in general are the single most awe-inspiring achievement of humanity in my lifetime

Landing a man on the moon is way more impressive. Finding several vaccines for a once in a century pandemic within a year of its outbreak is and achievement that in its impact and importance dwarfs what the entire LLM industry put together has achieved. The near-complete eradication of polio, once again, way more important and impactful.

I'm curious how this linting step scales with larger wikis. Looking for an inconstency across N files requires N*N comparisons, and that's assuming each file contains a single idea.

They are not entitled to a moat, and their customers do not owe them one. Several companies have narrow or no moats. Dell and HP are two examples when it comes to their PC business.

This idea that companies should be allowed to lock down their products just so they can have moats, is how we ended up with printer ink being more expensive than crude oil or champagne.

Yes, because if that was their sales pitch, they would need to pay Kim more, and they would have to account for the fact that she's already allocated elsewhere. It's better to pretend all those CCAs are interchangeable.

You must consider yourself so clever.

Adding new features doesn't necessarily grow the market. Your bread with nuts and berries competes with the regular bread for the customer's money. Other things also compete for the same money, such as medical, daycare, schooling etc. So increasing features won't necessarily grow the market because the market. Even in an optimistic scenario, those features only have a probability of increasing revenue, it's not certain.

OTOH, if you fire those workers, it is a certainty that your bakery gets more cash. You can then use that cash to reward your shareholders (a category that conveniently includes you) via buybacks or dividends.

> in order to extract resources from the local government in Iraq and Lebanon.

How evil of them to do that. They should have known only the United States of America has the divine right to invade countries and extract their resources. https://www.bbc.com/news/articles/c4grxzxjjd8o

> That's called Imperialism and was common among european powers in the 19th century

Fortunately, the USA does not want to revive such practices, as they indicated in this speech: https://nymag.com/intelligencer/article/marco-rubio-munich-t...

> The tiny minority of people with guns to everyone's heads?

Yes, I am sure the patients in this hospital were holding guns to Trump's head. https://www.aljazeera.com/video/newsfeed/2026/3/2/baby-and-p...

And while we're speaking of people that hold guns to everyone's heads, there is one country that is waging war all over the world. It's govt also executed people in broad daylight for the crimes of taking their kid to school and helping a person beaten up by the administration's goons. Would you entertain the idea of similar airstrikes against the head of that country?

> Iran is supporting massive conflicts in Yemen, and is building non-defensive weapons capabilities.

You know who else is supporting that massive conflict in Yemen? The United States of America. https://www.hrw.org/news/2025/06/04/yemen-us-strikes-on-port...

> It's all shades of grey, but there are lighter and darker shades.

I'm guessing they correlate 100% with the shade of the skin of the people involved.

Human beings are able to work out the ambiguity because a lot of meaning is carried in shared context, which in turn arises out of cultural grounding. That achieves disambiguation, but only in a limited sense. If humans could perfectly disambiguate, you wouldn't have people having disputes among otherwise loving spouses and friends, arising out of merely misunderstanding what the other person said.

Programming languages are written to eliminate that ambiguity because you don't want your bank server to make a payment because it misinterpreted ambiguous language in the same way that you might misinterpret your spouse's remarks.

Can that ambiguity be resolved with more English words? Maybe. But that would require humans to be perfect communicators, which is not that easy because again, if it were possible, humans would have learnt to first communicate perfectly with the people closest to them.

To reinforce that point: we've got the world's most prominent AI promoting company (MSFT), that has finally realized that Windows Explorer is too slow to start.

And this company, with all the formidable powers of AI behind them, can find no way to optimize that other than pre loading the app in memory. And that's for a app that's basically a GUI for `ls`

> The jobs could themselves become more desirable with machines automating the boring and dangerous parts

Or, as Cory Doctorow argues, the machines could become tools to extract "efficiency" by helping the employer make their workers lives miserable. An example of this is Amazon and the way it treats its drivers and warehouse workers.

In the case of horses and cars, you need the same number of people to drive both (exactly one per vehicle). In the case of AI and automation, the entire economic bet is that agents will be able to replace X humans with Y humans. Ideally for employers Y=0, but they'll settle for Y<<X.

People seem to think this discussion is a binary where either agents replace everybody or they don't. It's not that simple. In aggregate, what's more likely to happen (if the promises of AI companies hold good) is large scale job losses and the remaining employees becoming the accountability sinks to bear the blame when the agent makes a mistake. AI doesn't have to replace everybody to cause widespread misery.

I'll quote what the person I responded to said:

> because most companies will make decisions based on time/effort/profitability, and because client-side anticheat is stupid simple and cheap, that's what they go with. Why waste their own server resources, when they can waste the user's?

And my comment was a response to that statement. In context of that statement, companies are indeed choosing to prioritise their commercial interests in a way that increases the risk to the computers of their customers.

> Not to mention these abstract "the company" positions completely ignore all the passionate people who actually make video games

Irrelevant. Companies and their employees are two different distinct entities and a statement made about one does not automatically implicate the other. Claiming, for example, that Ubisoft enables a consistent culture of sexual harassment does not mean random employees of that company are automatically labeled as harassers.

Coming to anti-cheat, go ahead and fight them all you want. That's not a problem. Demanding the right to introduce a security backdoor into your customer's machines in order to do that, is the problem.

And those humans would be looking for a new job or face other consequences. An AI model can merrily do this with zero consequences because no meaningful consequences can be visited upon it.

Just like if any human employee publicly sexually harassed his female CEO, he'd be out of a job and would find it very hard to find a new one. But Grok can do it and it's the CEO who ends up quitting.

India has not been antagonistic or ambivalent in its recent past, until a Nobel Peace Prize aspirant in the WH decided to take a machete to relations that both countries had been building for the last 25 years, with largely bipartisan support in both countries. Even the current Indian govt is quite pro US until the aspirant tanked that relationship.

And yes, there will be times India doesn't agree with the US, and that's normal. It's seeking to be a partner, not a vassal state.

How do you know it isn't a backdoor? Do you have access to its source code?

This kind of app should be be open source.

So the problem was not with the app but with how the information was routed at the back end. The back end of the 1909 system could have been modified to write the data to a central registry as well.

So it is the company prioritising their bottom line at the expense of their customer's computers. More simply, they move cost from their balance sheet and convert it into risk on the customer's end.

Which is actively customer-abusive behavior and customers should treat it with the contempt it deserves. The fact that customers don't, is what enables such abuse.

Again, there's probably a sense of responsibility towards the people moving through the airports who otherwise would be facing much greater risk to their lives.

As a non American who's having to transit the country during this period, I've nothing but respect for the individuals who're actually doing their jobs and keeping everyone (including me) safe without getting a penny for this vital work.

They could go on a strike and bring all airlines to a halt, but as a brown skinned person, I would then be risking a visit to "Alligator Alcatraz" or some other demented place because I failed the leave the country on the day I was supposed to. So again, glad they're not doing that.

I guess the people in this instance realise they're an essential service for the economy and that without them, a lot of people could actually die. So they probably see their role as being more than simply working for the people of low integrity.

In the above scenario, if Claude accidentally wipes out your Jellyfin movies, will Claude deal with consequences (ie an unhappy family/friends) or will you?

That exemption from accountability is a massive factor that renders any comparison meaningless.

In a business scenario, a model provider that could assume financial and legal liability for mistakes (as humans need to do) would be massively more expensive.

Brenda's job involves being accountable for the output. In many types of jobs, posting false numbers would render her liable for a dismissal, lawsuit, or even jail.

I'd like to see the cost of a model where the model provider (Anthropic etc) can assume that kind of financial and legal accountability.

To the extent that this output is possible only when Anthropic is not held to the same standard as Brenda, we will need to conclude that the cost savings accrue due to the reduced liability standards than on the technical capabilities of the model

Gen AI doesn't just get a pass at being wrong. It gets a pass for everything.

Look at Grok. If a human employee went around sexually harassing their CEO in public and giving themselves a Hitler nickname, they'd be fired immediately and have criminal charges. In the case of Grok, the CEO had to quit the company after being sexually harassed.

We've not lowered the bar for AI, we've removed it entirely.