I'm sure he means well and is genuine in his aspirations, but what's outlined for GA is framing LLM's as quasi-gods, which they absolutely are not. I wish him the best, and look forward to being proven wrong.
It's only going to get crazier.
As you read this some poor person is starving. Humanity already possesses the ability to identify said person and send them aid. How is AI going to help here?
The ultimate fallacy is that all technological progress will benefit mankind. That will be true until it is not.
By the way, I donate monthly to GiveWell and the shrimp welfare project. Do you donate monthly to starving people? Most people don't, and the reason is simple: at the end of the day, most people just don't care that much about helping a starving person far away. They also don't care much about things like that, animals in factory farms, or earthquakes that kill hundreds of thousands. But ideally, we could find a way to empower people such that the minority who do care can make a big difference.
I'm genuinely not sure if you're referring to my comment or the one I responded to haha
> ideally, we could find a way to empower people such that the minority who do care can make a big difference
is it a coincidence, that in your ideal world, you would be empowered, because you care, as evidenced by your testimony of things you do that make you think you care? in my ideal world, wanton suffering would be minimized, regardless as to the path it took to get there; requiring 1 specific path involving people who think they are good, demarcates our ideals.
These are two vastly different statements, and the the war and diplomacy end is almost trivial by comparison, it would absolutely be solved before we solve all math in even the most steelmanned version.
and 30 years ago a computer beat Gary Kasparov, if you'd listened to Hans Moravec you wouldn't be surprised that the first thing that gets automated is intellectual domain expertise.
Things are going to get crazy when it can figure out how to walk into a random house a and brew a cup of coffee, not do math
And LLMs still suck at art and writing.
Nobody that understands automated proof checking was claiming that.
There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.
You need a problem where you both know what the solution looks like or can otherwise very quickly and efficiently determine that a solution is correct, but at the same time can't work out a correct solution with a similar amount of effort/time/cost as it took to determine how to verify a solution.
Most problems don't match that criteria. You usually either have a problem with a known method of solving, or you have a problem with no clear way of verifying the solution besides the act of finding the solution itself which would involve in some way proving it is correct, or you have a problem where verifying a solution takes a very long time or has a high cost or even can't be done more than once, so you need to try to determine the best solution without being able to actually test or verify.
Basically all problems just don't fit the "hard to solve but easy to verify" criteria to a degree that makes llms a good fit. On the other hand, there are so many problems that even a tiny fraction is a relatively large number.
a) No one ever said that.
b) Your comment shows a lack of understanding of the notion of truth.
There is no way for the LLM to bruteforce the search space any better than a human. What it can do better, tho, is to make connections between seemingly (for us) unconnected notions and join them, then verify if that's right.
Your view is not only wrong but also condescending in this day and age.
They put an LLM in a loop, with a reward function, and keep trying to get a higher score. The reward function for AlphaEvolve is “did this code get a better score or faster”, for math research it’s “did it write a LEAN proof”.
I’m open to hearing that I’m wrong but I don’t think I am. I agree that LLMs make connections that humans wouldn’t, but eventually you need to verify those because otherwise the connection it made may as well be a lie unless proven otherwise.
(I'm trolling, I'm not sure what you're asking or why)
LLM trained on you and fine-tuned on you. It does make sense in a way!
It is not what I want from LLMs, I think... but it might work.
Gwern obviously knows what he talks about. It doesn't seem like a Gas Town-like fever dream.
Always such certainty in these dismissals. And whenever you dig in, it always seems to be "well my super-human coding assistant made this dumb mistake!", along with a heavy dose of https://mastodon.social/@falseknees/116477394235883790
I don't think anyone who has spent any time around me would describe me as having anything but a pretty clear-eyed view of the current abilities of LLMs, I don't think anyone would seriously consider me to be in a period of mania or psychosis, and it also seems almost inevitable to me that we are a year or two away from LLM-cognition being substantially more competent in almost every regard from human cognition, and therefore in the foothills of the singularity.
The extraordinary claim that requires extraordinary proof is that this will somehow all blow over, not that we're heading towards a singularity here.
(Secondary question: what do you mean with singularity?)
As an offering of me engaging in good faith, a controversial opinion of mine: I legit think arpanet going online and starting the networking all of humanity into a massive coupled complex system fits the definition of singularity of "the moment after which predicting what will happen becomes hard to impossible", although that of course heavily depends on your definitions of prediction and hard/impossible.
“What LLMs can’t do, only humans can” feels like a God of the Gaps situation: people have to keep moving the goal posts because the more recent models keep unlocking more territory that was previously a “well they’ll never be able to do this!” holdout
I would actually expect the sea changes as you describe it in your first criteria to continue with 1) vision, audio and video natively integrated 2) continued scaling of e2e rlvf for workflows with large scale labeling efforts 3) ASICs and widescale deployment of diffusion models leading to speed ups
But as of right now, I still expect these models to need humans to prune the output to the gold and set up the harness right for both the novel bits, and for the boilerplate to be cohesive with the global intent.
Which is of course an amazing potential boost in productivity, but still a sigmoid flattening.
As for your second criteria that includes cost, I think we might every well see this coming soon, but it's difficult to estimate with the efficiency gains still possible.
Thanks for engaging:-)
You are literally arguing that LLMs are quasi-gods in the same comment you claim a 'clear-eyed view' of current LLMs.
Every bubble has its believers; that's why we have bubbles. There have been so many times somebody very smart has thought we were on the verge of the rapture, the coming utopia, the workers' paradise, power too cheap to meter, flying cars, settling the moon.
Could this bubble finally be the one? Sure. You could finally be the guy who is vindicated. And tomorrow the people I grew up around could be vindicated by Jesus's return to earth.
I can't prove either of you wrong, because you're not making rational arguments. You've had powerful experiences that have convinced you of something. There is no point in trying to discuss it rationally. The only thing people without faith can tell the faithful is what I said 30 years ago when I got out: I guess we'll see.
But my prediction is that we'll see an AI version of the Great Disappointment: https://en.wikipedia.org/wiki/Great_Disappointment
Many will stay faithful. The date of the singularity will keep getting pushed back. There will be another AI winter, but some will always see summer as just around the corner.
I guess we'll see.
I think this is what is driving the hype, the mania, and what increasingly appears to be insanely high investment. It's not about the underlying technology, which is objectively impressive, even if we're not certain of the true cost or utility of it. It's about selling the dream of Olympian omnipotence to investors, implicitly promising that IPO stands for Install Planetary Overlord.[1] And who doesn't want to be on the right side of that? (That there is no 'right side' in such circumstances does not seen to occur to anyone involved in the decisionmaking.)
1. Not my coinage, but that of SF author Charlie Stross in The Jennifer Morgue
At least he's pursuing something more novel than yet another "sandboxes for agents".
Not sure how you can feel sure anyone can get acquired even if their company had a path to profitability, much less for ones that absolutely don't.