Anamon

My #1 reason to never, ever do that is that it also gives company admins the power to remotely wipe your personal device. Spending weeks just to set everything back up the way it was, just becauce someone clicked the wrong button (or some "AI" agent was hallucinating a compromise) -- no, thank you.

It may be novel, but I agree with parent that it's mostly in the convex hull of previous knowledge. I like the term remix to describe that! Progress and truly novel things often come from "out of the box", and we know how quickly LLM output quality degrades when it leaves the space spanned by its training data.

So I would also expect a plateau. I guess future progress will depend on if we'll have enough people who still enjoy free thinking and brooding over problems, when there's the tempting option to just ask for an instant result that does the job of the moment.

Why are you so sure about that? Research communities are (used to be?) very tightly-knit with a lot of communication, exchange, and networking. Arguably, the more niche the research area, the more tightly knit and personal.

This is what I hear most about from friends who stayed in academia and research: the pain of losing that. Conferences stopped being interesting, because there's barely an exchange of ideas anymore. Nobody trying out new approaches, everybody just pulling the same lever in an endless loop. Nobody with anything interesting to share, and no potential to learn.

Looking at some of the materials, they seem to have been doing CGI work, too, in later years.

I do mourn the lack of physical special effects, models, and puppets, though. The studios try to convince us that we couldn't tell the difference, but it's pretty obvious.

I already liked Project Hail Mary a lot after having seen it, but my respect for it went through the roof when I saw the role names in the closing credits: model makers, metal workers, puppeteers, animatronics...

I agree, but I'm less optimistic about how far from that scenario we really are. I wonder what we'll look back on as the more damaging effect of the LLM hype: the economic bubble, or the carelessness it caused in people using the systems.

The knowledge about the nature of LLMs, their strengths and weaknesses, what they can appropriately be used for and what they should definitely not be integrated into, seems woefully lacking in many places. Just think back to the recent, disconcerting stories from the military world. We started with lawyers not understanding that an LLM can and will make up fake case law. Now we're at high-ranking military personnel not understanding that LLMs can and will make up fake target identifications.

Maybe the best hope for awareness about this spreading is a gradual slope of worse and worse, but not-quite-catastrophic-yet, highly publicised incidents.

Maybe it's cynical, but I feel similarly and also wonder, what do these people see that I don't? Smart people with supposedly a lot of inside information. But when I hear them talking about LLMs with words like "thinking" or "intelligence", it seems like it either has to be fearmongering for ulterior motives, or otherwise I live in a completely different branch of reality from them.

Their models are still, and based on this technology always will be, token predictors. How is a text generation engine going to directly endanger human lives?

It won't. It only can if someone hooks up tools to them that can perform damaging actions. Someone has to build that toolset. Someone has to give those tools the affordances to take harmful actions. And then someone has to write a "harness" that uses the token predictor as an authoritative input for instructions. Doing so falls somewhere on a spectrum between gross negligence and malevolence.

From a distant enough viewpoint it might not look like a big difference, but I think it crucially shifts the responsibilities. What is the actual difference between driving a potentially dangerous system with inputs from an LLM versus inputs from a fuzzer, or a website with injection vulnerabilities? I think it's the ones building and configuring the tools downstream that are responsible for knowing the risks and build under the assumption of the worst happening.

Heaven knows I'm not one to defend the big LLM companies, but I don't see how this is OpenAI's responsibility. They're building and selling an autocomplete system. How is it their fault if people ask the autocomplete for harmful commands and then feed them into an over-privileged system to run them? (I know the waters are muddied a bit by some labs acting like this themselves right now).

As much as I don't agree with the authors framing of and terminology around LLMs, reading the article generously, maybe this is exactly what he felt, too. That it's not at OpenAI where his concern and insight is needed most, because the token predictor isn't actually the dangerous part. Maybe he'll look where he can bring safety consciousness downstream, where people are hooking up these outputs into systems and in ways that they shouldn't. Don't let the LLM pick your bombing targets. Don't give the LLM sufficient rights to the hospital's file storage to be capable of ransomware-ing it. Don't strap the loaded machine gun to the dog.

My bank does this, too. They also only say that the browser has "outdated components" without bothering to tell you what particular thing offended it. Maybe it's some library they both use that doesn't return the details?

They compared with different "landmarks", and those are usually built around where people are, or people settle around where the landmarks are. I also found it strange that they didn't at least share a hypothesis of their own. But that would be mine -- settlement density and "modern lifestyles" may correllate with some negative health effects, or on the other hand with growing older and therefore being more likely to die of cancer? Maybe they'd have found a negative correllation if they tested for distances to deserts, open plains, and mountaintops.

Aren't you worried about missing stuff?

Maybe I'm farther out of the average than I think, but for me, planning and reminders are one of the few things I want 100% conscious control over at all times. Reminders are for me to make sure I don't forget the important stuff, so hooking in any kind of probabilistic system there would feel antithetical.

We're at the point of one such being mainstream-popular again.

"I get it, the brain is just an LLM!"

> We shouldn't be bullying people delivering trillions of dollars of value to society.

Surely, you mean trillions of dollars of negative value?

Sounds plausible. In Germany, which is on the list, the show runs in syndication on ProSieben. They have their own VOD service and people can probably stream it there.

The people I always loved working with most were the ones that were relentless in their PRs. But I understand it's a culture that's difficult to achieve. For one thing, I think it needs teams that are relatively stable long-term. It can be challenging for people to not see professional criticism like that as personal attacks. Much easier if you know and have a feel for the people you work with, and you go have lunch together after the disagreements in the PR. Everybody benefits, including the customer, but it's a work environment that's increasingly more difficult to maintain.

> Does anyone solder their own microchips at scale anymore?

That comparison doesn't work because up until now, it was only the execution we automated or optimised. Now, we're trying to outsource the understanding itself. No one hand-crafts a microchip, but we know how they're put together. We can debug and improve the process. The suggestion now is to just trust the magical statistics machine and not care that the artifacts become unfixable and unreproducible if and when the magical machine fails.

> If companies want to compete in the world of AI generated code, human code review has to go.

This is a HUGE non-sequitur. It only follows if by "compete" you mean producing more LoC. How often does that translate to market fit or economic success?

This is the mindset that makes me want to leave this industry immediately. Somehow, an industry that already annoyed me with how much "mediocre is good enough" was an acceptable stance, with the emergence of LLM coding tools suddenly decided that "absolute dogshit is good enough" was just as acceptable, as long as everybody else is also fine with eliminating the few quality standards they might have had.

I was going to add this, had I not found a comment already mentioning it =)

The font kerning also usually tends to be way off, making the all-caps even more difficult to read.

I don't see this as inevitable at all. Voice as an interface has so, so many downsides. Privacy, inefficiency, making noise, fatigue, requirements to the environment, not to mention that at least I personally have many times of the day during which I really, really don't want to speak. In almost every circumstance, given all possible choices for input methods, voice would be dead last for me.

> currently changing multiple industries entirely

Four years in, I'm still waiting for a single example of this oft-repeated claim stated as fact.

The only industry I have first-hand experience in is software development, and LLMs are certainly not changing that "entirely". The implementation of quick, throwaway PoCs, maybe, but that ain't an industry.

I already am.

Even research is boring now. The drive towards understanding has almost entirely been replaced by throwing carp at the wall and seeing what sticks. "Researchers" publishing solutions they don't understand, developed through a process they can't share any insight into, solving problems nobody ever asked whether they are worth tackling.

This truly is the most boring, underwhelming timeline in tech and research that I could ever have imagined.

I gave it the benefit of the doubt given the point of the article, and the rest of it was fine, so I also believe it was meant as a joke.

A bit ironic, too, because the same article later quotes polls where a majority of respondents said they immediately stop reading an article and any future ones by the same author if their LLM detector goes off.

That's not true. Copyright applies whether the author says so or not. By default, anything you find on the publicly accessible web, you are NOT free to reuse.

That's why the GenAI companies try to talk their way out by claiming training is "Fair Use". It means they acknowledge that the materials they use are copyrighted and not legal to reproduce without a license.

You can download it and re-upload it on your own site. And that would be an example for something that, in almost all cases, you wouldn't be allowed to do (unless the creator explicitly grants you permission).

I don't think the point is to just create as much content as possible, either. The term "content" alone implies lack of quality.

IMHO, any LLM output is, and systemically forever will be, inferior to human creative output, if not entirely worthless. We can scale and industrialise the production of slop, sure, but who wants that and to what end? We're not getting the next cultural movement or artistic masterpiece from these systems no matter how often we pull the lever of the regurgitation blender machine. Because human intent is the whole point, and without it, there is no reason to make any of this.

I agree that it should be about creating the most value to the consumer. As a consumer, the value I get out of LLM writing or art is almost always exactly zero, and it makes it that much more difficult for me to find the things I would see value in.

Then the natural born humans should also be liable for any criminal acts the corporations founded or run by them commit.

The fact that they don't is crucial, and probably the very reason why the concept of corporations was invented.

That's a lot of optimism about the quality of LLM summaries which, just going by my own experiences with them, I cannot share.

The part about collating all human knowledge also reminds me of the optimism on the early Internet. I miss having such a positive and hopeful outlook on things, but again, can no longer bring myself to truly believe it. Maybe I'm just getting too old.

What I see as the most likely scenario is that for most users, this collation of knowledge will remain behind the oligarch gatekeepers, who can and will manipulate the output to suit their interests.

I know, I know, open models and such, but I don't see those making any dents against the oligopolies. People will still go to Google or OpenAI. No non-nerd is going to self-host a niche model, and the on-device inference most people will be using will be based on models provided by the same, manipulative oligarchs.

The degrading of Google's search result quality is actually a good indicator that this are heading towards more manipulation rather than unfiltered access to real, original human knowledge.

How do you see a possibility of avoiding model collapse? It seems to me to be an unavoidable consequence of two facts I consider pretty much irrefutable:

1) LLM content can at best be as good as the source material it was trained on. That's the upper bound. "Out of distribution" output of LLMs is mostly unusable.

2) LLM-generated content increasingly drowns out original content, online and elsewhere.

This spells "monotonically decreasing content quality" to me.

I find Kinsella's argumentation extremely weak, and his defense of it (e.g. his AMA) entirely unconvincing. Unless you susbscribe to a number of very daring assumptions which, in my view, do not at all track with our reality, his ideas just don't hold up.

That being said, in this scenario of yours, how much of that additional work do you figure would be actually original? If someone wants to make a living with music, why bother learning an instrument and writing songs if nobody can stop them from just recording someone else and selling that?

That's really the heart of this lawsuit, isn't it? If we give freeloaders the right to legally monetise other people's work, we will drown in bland, derivative, stolen garbage and drive out the actual creators, in the process also breaking the "business model" of the freeloaders. A real lose-lose-lose situation which I see as much more realistic and believable than Kinsella's libertarian dream world.

Not much need for extrapolation, either. We can just watch it happen right now. "Content production" on the web is definitely exploding, as per your point. But even in the mainstream, most people will agree that this flood of new, LLM-generated "works" is worth less than the originals they're sloppy pastiches of.

And it gets less likely the longer each use is.

I also doubt this. When I observe people out in town, most of them stare at their phones non-stop. To get to pick them up that often, they'd first have to put them down every now and then.

Not all of us are past that. Touchscreens have a variety of downsides that I think most people simply don't think about too much and put up with, and physical keyboards a number of benefits that people aren't aware of because they've been out of fashion for so long.

Less on the device and more on the keyboard. On Android, you can install third-party keyboards like any other app, and there's a wide range in quality. The default Android one is decent, but not among the best.

I've been strictly a Sony Xperia user for almost 15 years now, and Sony had its own keyboard app early on, which they later discontinued. Swipe typing on the Xperia keyboard is still the fastest and most precise I've ever been able to write on a phone, nothing else ever came close. Then they stopped development and moved over to a Microsoft keyboard, I think. I've never found another keyboard that did swipe typing as well as Sony.