If you told me 20 years ago that you could, by connecting a nuclear power plant to a virtually infinite number of GPUs, produce a machine capable of predicting the next token in a sequence, I'dve believed you, but also would have said that it would be an astoundingly huge waste of resources to use to get to that result.
People seem to ascribe some magic here that isn't happening, because they're surprised by the results.
I believe this is the result of drinking the kool-aid. At this point though, I'm really confused on how AI has any jobs left to do because blockchain was going to solve all the problems.
If we're focusing on energy consumption, using an LLM to generate a newspaper article probably uses less energy than a living journalist would use. The morning commute alone is probably "worth" a dozen articles.
The problem is different. First, that journalist is still alive and consuming resources, just out of a job. Second, because you now have a very cheap way to generate an infinite number of articles, and there are commercial incentives to do so, the "dead internet theory" has a good chance of coming true.
If you'd told someone with one of the first automobiles that they'd eventually be massively personally-owned, have access the majority of countries via paved road networks, and be refillable along the way, they'd have laughed at you.
And yet, that's what we've all lived in.
It happened because automobiles were massively useful.
It's difficult to imagine a scenario where basic-level cognition doesn't also scale, because a lot of the mental tasks we do every day are dumb and low-value.
For example they see a prototype and then assume in 10 years everyone will have one.
Or they look at a proof of concept and don’t realise that that last 10% of refinement actually takes 90% of the effort.
What people usually miss is that scaling is the problem, not the solution. Prototypes are easy in comparison.
Hence why cell phone took off (verse land lines) but flying cars didn't (verse ground cars).
Adding basic level creativity to automation seems like the former.
It's not going to change everything overnight, but it is going to impact everything into a new normal.
Meanwhile, my modern "AI" based Google keyboard autocorrect can't even handle the average case where only the first letter of the word was a mistake. It will only suggest corrections (and then auto-choose those corrections based on rules that change monthly it seems) that use the same first letter that I accidentally hit.
Also, my mother had a dirt cheap LG flip-phone in 1998 that, after 10 minutes of training, could dial any number and even dial any of your contacts by name with your voice.
In the mid-2000s, Microsoft had a BUILT IN function to Windows XP for dictation. You can see it do poorly in a demo, which is funny, but in actual use, without any cheating or prompting or massive compute resources, it correctly understood you 80% of the time. The state of the art in the industry could do much better than that without "AI" at the time, and cost a pretty penny for people with disabilities mostly. However, that dictation API is STILL IN WINDOWS, Microsoft just kinda hides it because they want you to use the Azure based one. It is super simple, with a powerful yet understandable API, available to any application that runs on Windows (like, say, games) and completely free to use. More importantly, if you have ANY knowledge about the expected use case for your app, you can feed the dictation engine an easy to generate "grammar" about the structure of commands, and it's accuracy can jump up to near perfection. When I played with it, I was able to build a voice controlled app with a single page of code and like an afternoon that easily turned my voice into text. It was literally harder to control the legacy Win32 APIs to inject input into queues for my use case: Voice controlled game input.