>Have we hit peak AI?
AI is at the same point cinema was at when pioneers of the genre showed short films in "salons" designed to frighten people with simple tricks.
This is hardly the peak. We're just at the beginning.
AI is at the same point cinema was at when pioneers of the genre showed short films in "salons" designed to frighten people with simple tricks.
This is hardly the peak. We're just at the beginning.
Btw, there was a sci-fi novel where the plot took place on a planet where winter came in periods too long for the natives of the planet to remember (they had short lifespans, or they had limited memories, I er, why, I can't remember!). I don't remember the title of the novel or the author and I've been trying to find it. I thought it was something by Ursula K. Le Guin, but it doesn't sound like it.
And no, it's not Game of Thrones. I think GoT took the idea of long-coming winters from the earlier novel, or it's just a coincidence (or cryptomnesia).
Anyway I'd appreciate if someone knows what I'm talking about and lets me know.
___________
The fact that organic brains exist proves the baseline of what's fundamentally possible and at very low power levels too if optimized well. We haven't reached that yet, and it's impossible to say what the upper limit is.
LLMs might not get there as an end-to-end solution but they'll will certainly be a significant part of one.
Not Asimov's Nightfall?
Nightfall and the movie Pitch Black also go in that direction.
Unforutnately Helliconia is not the one. I've read quite a bit of Brian Aldiss, but not that one. I think I would have remembered reading the Winter book in particular, judging from the discussion of the "Wheel of Kharnabar" in Wikipedia:
>> The Wheel is an extraordinary revolving monastery/prison built into a ring-shaped tunnel with a single entrance and exit, powered entirely by the efforts of the prisoners pulling it along by means of chains set into the outer wall. Once a prisoner enters a cell of the Wheel, it is impossible for him to leave until its full ten-year rotation has passed.
That's the kind of thing that sticks in my head. Anyway I remember bits of the plot and there wasn't so much intrigue and armies and whatnot. The native society was a primitive society that lived in yurts or tepees and didn't have much technology. I think they were supposed to be less intelligent than humans.
I don't think I'll ever forget Nightfall! :)
The Left Hand of Darkness is set on a planet with perpetual winter, though it somewhat warms up at times. Maybe that's what you're thinking of?
I'm 100% certain it's not the AI peak, but i'm also confident that we are close to the local maxima, at least for transformer architecture. Maybe the following improvement will be in chaining architecture, or a brand new technique, or maybe progress will be made in parallel fields (even smaller processors!).
Charitably, i'll decide that the question meant "Have we hit peak GPT" and in this case, this is a legitimate question.
I say that because i find GPT4 extremely useful. Even if GPT4 (in it's hypothetical un-nerfed state some would argue)* is the peak LLM i think we have a ton of room to grow even within that peak.
Locally running larger context windows, multiple domain experts in parallel and hooked up to a local repository of available IO actions would give a very significant "advancement" within the same conceptual "peak LLM". Now will we be able to get there on consumer hardware? Not idea. However i don't think i need more than GPT4 to find the whole technology extremely useful.
Moderately cheap to run large context windows[1] entirely locally is the "next big thing" for me. I get a fair bit of value from GPT4 and Phind. The technology has sold me already.
[1]: I should amend, i'm speaking loosely about context windows. What i mean is the ability to feed it refined datasets locally to have it's "local knowledge" change, which may be less about context windows and more about refined training. I'll leave that to the experts to debate, i am assuredly nothing close.
Yes. sorry my point wasn't well formulated (i'm not an expert either: i've trained LLMs on receipts in 2016, and quickly decided i hated that). I'll try doing it better by dividing the two idea i put in that post:
- i think that GPT4 is probably close the peak generalist GPT, meaning that maybe Bard can surpass it (or a chatGPT4.5 or something) but the improvement won't be stellar.
- If future improvement won't come from the transformer architecture itself (i absolutely can be wrong on that), it'll come from what we interface with it.
Absolutely context windows will probably a great way to have "specialists" transformers, but alos linking it with different "kind" of AI might push it too (i've read a post from Wolfram a while ago that sold me on interlinking different AI models/languages/state machine). I'm sure specialist or even large users haves tons of ideas.
I got some value from GPT3.5, enough for me to buy access to GPT4. with a caveat: i'm pretty sure that for work-related questions i lost a bit more time than i "gained" overall. because my mastery of the technologies we use and very specific needs that GPT cannot know. But in the last three weeks or so, i'm pretty sure it was overall a net benefit at work, as i know better when i should use it.
With larger context windows, we can use traditional search algorithms to cherry pick an educational context for the LLM for few shot learning. If that's paired with more, better and annotated data, that will take us quite far.