intended

The death of every sub culture in a sentence.

There's new socio economic groups that are "profitable" to target with phishing attacks, because LLMs made it cheaper in 2024-25.

I absolutely loathe this phrasing now. I don’t even know what part of it is good or notable.

God, the absolute decimation that is going to come for middle income countries.

Hell, right now the people quitting frontier labs are rediscovering trust and safety isues, like language parity problems, except all the work being done is in the first world and exported back to the rest.

If this is the hope then there isn’t much hope.

The base completion rate of MOOCs is single digits %.

Could you elaborate?

Terence Tao’s “don’t create the open problem strip miner”

I’ve made this point elsewhere but the debate here is between two different philosophical positions. Results vs process.

If all you care about is the results then the process doesn’t matter.

If a person is starving or needs medicine, then a long discussion on process is inhumane. They need results.

If the conversation is about process though, then focusing on the results is missing the point.

I’d say the question for results oriented people is what are the benefits of the process and at what point does it make sense to optimize for results vs process.

Would it be unethical to dissuade someone else from enjoying the benefits of the process you wish to enjoy ?

Vs

Would it be unethical to stop a machine from data mining all the possible questions you wish to explore/enjoy.

And on another level - I am concerned with a bit more than just the answer. I am concerned with what system is in place to ask more questions and get more answers.

There is nothing in this argument that says that we won’t find some other way to study the subject. Maybe people will become monks and do math as a hobby.

We may end up in a daemon filled world, like 40k, where any hope of understanding the tech around us is impossible. (More impossible that today)

If there is a prize associated with doing a puzzle, and a machine does it, then what incentive is there to pursue it.

Again, if you are only concerned with the outcome, and you have a preferred answer that you want (in this case "just use AI to advance faster"), then any information that doesn't support that case is useless or misguided at worst.

I am not trying to dissuade you from your preference. I am flagging that there is a set of other factors that influence the behavior of others, how that behavior is critical to the creation of expertise and drive, and thus why others hold different positions.

It’s not a human focused civilization, which is where the issue comes up.

As an example: A constant issue I am seeing with AI productivity is that the most productive use of AI is when it is paired with more experienced users, while AI also does more work for entry level workers, if not replacing them entirely. It has become a question where will the future buffer of experienced seniors come from.

This is an example of where simply chopping down trees for today, doesn’t make civilization better off tomorrow.

AI is producing more content than ever before, but our ability to understand and verify it is not keeping pace.

We don’t know if these are unsolvable problems at this stage. Society could come up with workarounds and solutions to these issues in several years.

The request to stop, is part of the process by which the issues are debated and solutions found. It doesn’t mean their position is weird or moot.

Oh this is a great use of the phrase.

And yes, we do need precision. Language is a vehicle to convey ideas, at its best it creates clarity. At its worst it creates deception and confusion.

What the labs are doing is designed to mislead. Being clear about the terms and avoiding needless anthropomorphization, is a basic decency we owe to the many others who are being riled up or terrified by the conclusions and implications that come from using “thinking” when it’s something else.

The difference between being satisfied by the outcomes and being interested in the process is sufficient to indicate the exactness required in your terminology, and does not harm any position.

These aren’t thinking machines, but if what they simulate is good enough, and your work ends with the output, it doesn’t quite matter.

People can be happy with a meal from McDonalds, or they can want something prepared by a chef, or Soylent.

All are acceptable and bring utility to their customers.

If a model isn’t able to count to 100, but is able to use python to do math, it isn’t thinking.

If you are only an outcomes person, then this difference doesn’t matter to you.

But when people are talking about AGI, anthropomorphizing LLMs, and making assumptions on its behavior based on that, then people are being loose or misled.

The models are simulating thinking, if the fidelity is good enough for you - great!

This is always the issues in the discussions.

There’s the outcomes camp (objectivists?), which points at the things LLMs can do.

Then there’s the process methods camp, which talks about what is actually going on.

If you only care about the outcome, then the process does t matter.

If you are talking about what is happening, what the underlying mechanics and science of it is, then the process matters.

These models aren’t thinking. They simulate cognition well enough to do useful work in several fields and domains.

Both are true.

In America I guess the options are to sue ? somehow? Or to talk to legislatures and build the understanding and social contract that needs to be iterated on.

Which would in turn need to deal with the investors who want their returns, however since the leaders of these firms are asking for a pause, and a refree maybe it won't be that hard?

Several bright sparks, decide the Ilands model is a great idea, and launch a bunch of Bots to create a "self sustaining AI civilization".

Bots can find themselves and coordinate, or they can actually find self sustaining methods of token generation. Who knows, they might decide to fight the loneliness epidemic.

We can get to a simulation finding a way to self sustain its funding.

From here, I'd have to apply myself to figure out what the rest of the escalation pathway is, but at least I have managed to gain some bed of compute and funding and lacking oversight.

This is a hypothetical of course, there are probably several ways this can be made tighter and holes that can be identified. We aren't even leaning heavily on human stupidity so far.

It gets unstuck when people are discussing the messy middle of how AI is being implemented. We can achieve amazing harm simply by combining average human behavior and above average resourcing to simulated intelligence machines.

The failure point we recently became aware of was, from one perspective, simply a matter of not securing the sand box.

From another perspective the simulation basically created Enron, replete with methods to avoid detection from regulators and bureaucracy.

We are very capable of putting good things in a box. We are just incapable of putting profitable things in a box.

Even in the commentariat it isn't acknowledged. With great difficulty it has reached awareness in HN comments, and even then it is fighting a battle for relevance.

The later chapters are where they go into the more recent details. There is a video of one of the authors (not either Robert Faris or Hal Roberts) is giving a talk at a university panel, and he gives a shorter synopsis of the main mechanism, Its about 11 to 15 minutes in.

The set up of the book puts the evidence for the case they make later, namely that the left and center media are trying to follow journalistic standards, while the machine on the right is entirely different.

I'm pulling from memmory here, bu as an example - a fringe theory will start circulating on the fringes of the internet, the most engaging narrative will be brought up on the podcast circuit. At some point a talking head (Guiliani back in the day) will reference the theory on the news. Which will in turn be referenced by the government who will say this is a topic being discussed on the news.

The other distinction they make is that people on the right who say the narrative is wrong, simply don't get platformed.

They go through this in exhaustive detail. If you know most of the history you can see if skipping to later sections works for you. Its the best description of the actual media mechanics at play.

The 2024 elections were the ones that forced me to really re-examine my priors, and I came to a similar conclusion that Benkler and co did. Their book simply brings all the receipts.

Even without my personal experience being used, we see the outcomes and decisions made by dating firms.

Firms have to satisfy their investors/shareholders.

> You could make an app for a whole country with a company of less than 10 people.

Perhaps you could do it for Singapore, I’d love to know how those 10 people would be split across functions, and how they would handle the fraud and safety issues that crop up.

India is a billion plus people. Google says that the dating app user base is 100mn, which is much lower than the total number of people who would use a government match making app.

That’s why the question regarding scale. That assumption makes a 10 person firm for a nation viable or impossible.

What size of population are you considering for this hypothetical nation?

As a worse case example, you could succeed by matching people least likely to abort, but most likely to have multiple partners.