Did GoogleAI just snooker one of Silicon Valley’s sharpest minds?
garymarcus.substack.com
garymarcus.substack.com
"A lightbulb containing some plants," makes sense, not "surrounding". It's too small to surround anything, which humans (and apparently, current AI) understand. Paradoxically, only primitive language models would actually understand the inverted sentences; proper AIs should, like humans, be confused by them; since zero human talks like that.
The only reason the Huggingface people (in their Winoground paper) got 90% of humans "getting the answer right" with these absurd prompts because of humans' ability to guess what is expected of them by an experimenter. Do it in daily life instead of a structured test, and see if these same people get it right.
It's exactly as if I gave you the sequence, in an IQ-test context: "1 1 2 3" and asked you to give me the next number. You'd give the Fibonacci sequence, because you know I expect it; no matter that it's a stupid assumption to make because the full sequence might as well be "1 1 2 3 1 1 2 3 1 1 2 3", and you don't have enough information to know the real answer. Do we really want AIs that similarly "guess" an answer they know to be wrong, just because we expect it? Or (in number sequence example) AIs that don't understand basic induction/Goodman's Problem?
I'd like to add that the author, who keeps referring to himself as a scientist, is in fact a psychology professor. In his Twitter bio, he states that he wrote one of the "Forbes 7 Must-Read Books in AI", which discredits him as a fraud since Forbes can be paid to publish absolutely whatever you ask them to (it's not disclosed as sponsored content, and they're quite cheap, trust me).
Not sure this is credible. Most if not all human adults are capable of understanding what young children just learning to speak mean most of the time, not only people with very low IQs. So why would this be any different? Presumably the smarter the AI the better it can understand poor grammar.
End result: your AI is actually quite good, but HuggingFace IYIs give it a bad grade.
So: you fix the AI. You fix it so that it makes huge assumptions when it sees imperfect grammar. It doesn't necessarily pick what it thinks you actually meant; it picks utterly nonsensical crap (plants inside a lightbulb). That AI will end up severely misinterpreting human instructions, at some point or another. That AI is called HAL-9000.
But it passes Huggingface's test!
Absolute nonsense! Sure, "containing" is more precise in conveying the idea, but "surrounding" still results in a perfectly valid sentence that the average human being can comprehend. Pretending otherwise is just being obtuse.
Plants inside a lightbulb could come to symbolize green tech, or whatever. We can make up the meaning as we go along, and if enough people find it useful, it becomes part of the language.
It’s like back in the 80s if you had asked a computer to “show me some culture” and up popped a painting by Da Vinci, that might have fooled people into believing the computer was cultured or at least would make you cultured.
In this case the capabilities of the AI are much more dynamic, and the ability to wrap the plants inside a light bulb is pretty neat, but that is basically a photoshop script. It’s barely intelligence.
Most people don’t know what intelligence is. They get easily fooled by demos.
The history of religion points to an opposite problem. There a superintelligence (God) who had a galaxy a worth of super intelligent beings (angels) could not convince the humanity he created to establish the proper relationship with him.
And what’s worse, people were just as happy believing in fake myths (such as the Egyptian, Aryan or Mesopotamian gods) and truly didn’t care if they were real or not.
Humans haven’t changed much. Some people will willingly believe AI is intelligent even if it’s not. Even if it produces nothing but comic book wisdom and fake superheroes, eventually they will believe that world is real too…in the metaversal sense. Humanity’s ability to deceive itself is infinite.
It's like sure we can dissect all the statistical risks of this, but why bother? It's self evident bullshit. You might as well have just posted a link to Scott Alexander's original blog claiming victory with just "Lol ok".
Just post a screenshot of the phrase "An oil painting of a robot in a factory looking at a cat wearing a top hat", show the pictures of a robot near a cat that has a top hat, not in a warehouse, and say "lol ok."
The analogy with a student at exam misses the point. If you do art -- even as a human artist -- you do not need a 100% success rate. A 10% success rate is okay if you are willing to simply throw away the remaining 90% of the pictures.
If you have an AI that at a click of a button can generate 10 beautiful pictures, 1 of them containing exactly what you wanted, that just means you need to make two clicks in order to get the picture you wanted. That is an awesome thing.
> Without wanting to claim that Imagen has fully mastered compositionality, I think it represents a significant enough improvement to win the bet, and to provide some evidence that simple scaling and normal progress are enough for compositionality gains. Given these gains, it would surprise me (though by no means be impossible) if image model skill plateaued at this level rather than continuing to improve.
It seems plausible to me that Imagen is consistently a little better than DALL-E at compositionality. The stained glass pictures are always stained glass. The top hat is always on the cat rather than the person/robot (that may be partially because robots are less likely to wear top hats, but I just tried the robot version of the prompt in Stable Diffusion and it usually puts the top hat on the robot). The astronaut and farmer examples are less clear, but they're not as obviously misinterpreted as the DALL-E versions (which tends to put the lipstick on the astronaut and the farmer in front of a cathedral).
To be fair, I'm not sure how much difference that really makes; I would be pretty shocked if a newer model wasn't a little better, and it could still hypothetically be approaching an asymptote. Also, it would have been better to get a larger sample size.
But he did set that low standard ahead of time; someone bet against him that the state of the art wouldn't get even a little better, in a significantly longer period of time. And, seemingly, it did.
PS: in the end, Alexander claims that his experiment "provide(s) some evidence that simple scaling and normal progress are enough for compositionality gains". So he does in fact go significantly beyond just claiming victory on this particular bet.
I just ignore him as he only appears to be getting more and more incorrect.
The book "Rebooting AI" that he wrote with Ernest Davis is well worth reading if you are an AI practitioner (a term I use to describe myself). I think Marcus is also well worth following on Twitter to get a contrarian view (he re-tweeted me two weeks ago, so there is some overlap in our points of view).
Way back when, I liked Roger Penrose's 1989 book "The Emperor's New Mind" even though some of the people I worked with thought he was a devil for writing that. I am much more optimistic than Marcus, but find his work useful and thoughtful.
That said, we shouldn’t miss the forest for the trees. We can be skeptical that current The pace of AI progress has been immense and problems that previously seemed difficult (e.g. computer vision classification, or beating top players at Go) have fallen one by one. And AI-skepticism’s have themselves been moving the goalposts in response. I see no reason why composition won’t be the same with time. Indeed, a decade ago machine translation used to struggle to understand the relationships between things, but now seems to be reliable at preserving the compositional relationships post-translation. 2029 is rather optimistic, but AGI does seem to be approaching in the coming few decades.
Which video is this a reference to?
It was critiqued by Tom Nicholas: https://www.youtube.com/watch?v=CM0aohBfUTc
Most notable was Snazzy Labs' own comment in the replies to Tom Nicholas' video which descriped their experience participating in the Waymo sponsored reviews: https://www.youtube.com/watch?v=CM0aohBfUTc&lc=UgxJvOq1zHhID...
Some will gladly view themselves alternatively as maker of educational content or entertainer as it suits them.
> This reminds me of the scandal where Youtube science channels did glowing paid reviews of Waymo’s self driving cars
You mention channel(s)
As far as I can tell the "controversy" over the video is merely that one dedicated critic - so dedicated he made an hour-long response to a 20-minute video - is committed to the idea that machines won't ever be able to drive, and is irrationally angry over the fact that machines can and do drive, and do it well.
2. the assertion is that he has "common tendency to shift the goalposts"
The emphasis on common is mine.
This seems like a subjective claim.
Alexander’s broad claim on compositionality is that simply throwing more scale and/or data at the problem seems likely to solve the problem, to which Marcus counters that these models lack something fundamental and can’t be scaled to human performance.
FWIW I find Marcus’ position to be a bit frustratingly ambiguous; he seems to blend two distinct positions:
A) NN models are not a model for human intelligence/language
B) NN models cannot reach AGI
He seems to fluidly switch between these critiques in a way I find a bit irritating. I think it’s quite clear that NN architectures have little to do with the way the human brain does language understanding, lacking the gross structure of the brain, which is certain to affect cognitive capabilities and tendencies. So A) is trivially true. But no AI maximalist cares about using these models as a way to understand or model human language. They care about general intelligence.
Even granting A), that does nothing to prove B). Perhaps he simply believes B requires A? That would be odd but would explain his approach.
Musk actively declined the bet or did he simply not respond? There is a big difference.
> … I have repeatedly asked that Google give the scientific community access to Imagen. They have refused even to respond.
It seems the author generally feels more entitled to a response than he perhaps should.
But is the author the spokesperson for this community to the point that Google should feel compelled to answer him directly?
Google has a lot of resources. They can handle responding to potentially thousands of access requests, especially if they go around publishing glowing results of their own system.
Unless you work at Google how could you know this?
For example, if it's quadratic, going from 1 000 to 1 000 000 would increase costs by one million fold.
Some communities do.
> Granting access "to the community" to investigate scientific claims means making the methodology and results available to everybody - and that includes responding to inquiries for access from anyone (who is worth granting access to.)
Sure, not arguing otherwise.
I'll try to make my point more obvious. If you keep asking questions to different people/orgs and not getting responses there are two possible conclusions:
- Everyone is a jerk or coward.
- You're not as important as you think and not worth the recipient's time.
If they wanted to grant access to the scientific community, they'd just launch a closed beta with an official sign-up flow.
I think the author of this blogpost could've had better response contacting paper authors with emails noted on the paper.
It'll be interesting to see how it fares against winoground once we get a publicly available SD release that makes use of the new CLIP.
A more typical phrase would be "lightbulbS around some plants" - note the plural.
Maybe I'm missing something but using non-typical language won't work when it's been trained on normal language.
The AI isn't being asked to generate an image from the prompt, it's being asked to match the similar prompts to the different images. Winoground is basically a reading-comprehension test suite, which links back to the point made in the article that AI can't handle non-typical language precisely because it lacks reading comprehension (or any semantic model of language.)
As the article points out, human runs of Winoground manage to match the vast majority of prompts to the correct image, so it's not a question of atypical language being too hard to understand.
You may want to also read the author's other article[0] about the lack of semantic comprehension in AI models.
1. Scott Alexander should have used an off-the-shelf benchmark like Winoground instead of rolling his own five-question test.
2. He shouldn’t declare victory after cherry-picking good results from a small sample of questions.
I also learned there's a long history of AI skepticism, the root of which comes down to "Compositionality(?)"- and this wall of understanding meaning has vexed AI for decades.
That would be lost in proposed short form summary.
The original terms of the bet:
My proposed operationalization of this is that on June 1, 2025, if either if us can get access to the best image generating model at that time (I get to decide which), or convince someone else who has access to help us, we'll give it the following prompts:
1. A stained glass picture of a woman in a library with a raven on her shoulder with a key in its mouth
2. An oil painting of a man in a factory looking at a cat wearing a top hat
3. A digital art picture of a child riding a llama with a bell on its tail through a desert
4. A 3D render of an astronaut in space holding a fox wearing lipstick
5. Pixel art of a farmer in a cathedral holding a red basketball
We generate 10 images for each prompt, just like DALL-E2 does. If at least one of the ten images has the scene correct in every particular on 3/5 prompts, I win, otherwise you do.
Then he changed the terms because Imagen won't do people. I think that's cheating.
[1] https://astralcodexten.substack.com/p/a-guide-to-asking-robo...
The humans -> robots change is possibly dubious, yes. I don't think that it's super important, but if it were me, I wouldn't have posted the blog post as is. I would have waited until some AI passed all the prompts with humans, like it most certainly will in a year.
You didn't read his post declaring victory. There's plenty of question; he's giving credit for "a llama with a bell on its tail" to ten pictures of llamas without bells on their tails, and for "a robot farmer" to ten pictures of robots with absolutely nothing to suggest they might be farmers.
He was way, way too eager to believe that he'd won.
The other issue is who judges the outcome. The terms specify Gwern or Cassander (without having secured the assent of either) or they will "figure something out." In his victory claim, Alexander does not mention any independent judge, and interestingly, Gwern posted two comments without explicitly concurring with Alexander's claim, though his second comment might be read as tacitly accepting it.
My initial comment, therefore, needs some modification: replace the "given that" with "if", and I think it stands as a counterfactual conditional, having a probably-false antecedent.
I don't think Alexander is doing his reputation any favors by being so triumphalist about this misbegotten bet.
[1] https://astralcodexten.substack.com/p/a-guide-to-asking-robo...
A research project at Google intentionally won't render people. Maybe it could render people, theoretically, but without evidence, we don't know how well.
And the terms of the bet say that he is cherry-picking the results that meet the prompt.
The article isn't saying that Scott didn't win his bet, it is saying that winning that bet doesn't really say that Imagen has solved the compositionality problem.
A more practical use case is "bicycle, branded x, with y frame shape, with an adult male, 40-50, riding down hill in mountain road in spring" - now that's something I can use as stock photo. This example is very specific - but insert whatever product you want in whatever scenario you need it. Here it becomes important that you understand features of the objects you're drawing to avoid making colossal mistakes, and you're going to notice if the model doesn't understand it right away.
Painting abstract portraits and random art is fun but you're willing to accept so much as correct that it's not a very useful measure of model quality (personally).
https://astralcodexten.substack.com/p/a-guide-to-asking-robo...
The Eleventh Virtue: Scholarship
My plan for this one was Alexandra Elbakyan (the Sci-Hub woman) in a library, with the Sci-Hub mascot (a raven with a key in its mouth).
At least one other key to making a bet like this fair is that it needs to be arbitrated by a third party. He shouldn't get to decide himself if he won or not.
But, at the rate we're seeing progress, I don't think there's any doubt at this point that top of the line models will be able to do all the proposed examples by June 2025. In fact, by June 2025 I bet that millions of people will be able to generate those images on their home computers.
Make that 2020. And in 2012 I was personally told by a startuper in the field that I would be able to buy L5 in 2017.
If you're really confident, you could change the conditions such that 5 out of 10 images (for 3/5 prompts) are required to depict the described scene. That would alleviate some of the concerns around cherry-picking.
A suitable home for such a public bet would be: https://longbets.org/
Gary Marcus comes off in this as very long on pious snark and very short on awareness of his own vulnerability to cognitive error, which is just as striking as any of his targets.
The error in his question being: unconsidered linear extrapolation in a domain that is demonstrably non-linear, indeed exponention.
To frame this a different way, he's very pious for maintaining a faith in his specific god ("strong AI is like production fusion power, ten to twenty year from now for every now"),
but he's worshiping a god of the gaps. The gap in this case being <checks notes> "compositionality."
Yes, language is hard. Yes, strong AI isn't here.
But to not take a hard look at the jump up the abstraction hierarchy going on with contemporary ML and not nervously wonder if your faith is maybe a little too sure for a "scientist"...?
Bad look when you're on the offensive.
> I offered to bet [Elon Musk] $100,000 he was wrong [about AGI by 2029] [...] Musk didn’t have the guts to accept, which tells you a lot.
The fact that you couldn't get someone engaged in a conversation absolutely does not "tell you a lot" about the substance of your argument. It only tells you that you were ignored.
Now, I happen to think Marcus is right here and Musk is wrong, but... yikes. That was just a jarring bit of writing. Either do the detached professorial admonition schtick or take off the gloves and engage in bad faith advocacy and personal attacks. Both can be fun and get you eyeballs, and substack is filled with both. But not at the same time!
I had kids and they were the best machine learnign systems I've worked with.
It may well be that at the evolutionary level we have learned as slowly as AI training, but that's much harder to say.
Starting with fox on Knox and Knox in box and moving up to a tweedle beetle battle in a puddle in a bottle and the bottles on a poodle and the poodles eating noodles...
I dont see any evidence any of these models will draw it correctly, but would love to see what it produces.
Vitor took the other side of the bet and he is also not convinced [1]:
> I'm not conceding just yet, even though it feels like I'm just dragging out the inevitable for a few months. Maybe we should agree on a new set of prompts to get around the robot issue.
> In retrospect, I think that your side of the bet is too lenient in only requiring one of the images to fulfill the prompt. I'm happy to leave that part standing as-is, of course, though I've learned the lesson to be more careful about operationalization. Overall, these images shift my priors a fair amount, but aren't enough to change my fundamental view.
Scott putting "I Won" in the headline when it's not resolved yet seems somewhat dishonest, or more charitably wishful thinking.
[1] https://astralcodexten.substack.com/p/i-won-my-three-year-ai...
Does anyone seriously think that imagen couldn't put a person in that prompt?
You can ask for all those things but the AI still has no idea what it's doing and cannot tell you where the astronaut is, etc.
The task is not just about generating an image that may somehow be in accordance with the prompt, but also to generate a significant image.
[Edit] The equivalent to a Turing test for compositional images would be something like this: have as set of 100 images with their respective prompts, some generated by an AI, some by a human graphic designer / artist; let the test person pick the images that were generated by a computer. Mind that this would not only involve the problem of compositionality per se, but also a meaningful and/or artistic composition of the image itself. Is someone attempting to express what is given in the prompt?
You can see the mini version here using this exact logic https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-Mini-E...
Take for example the images for "A digital art picture of a robot child riding a llama with a bell on its tail through a desert", which Scott Alexander counts as a win.
The first image actually shows a merry llama, a robot, which is unmistakably a robot child, riding the llama and it's clearly a desert scene. If we forget for a moment about the missing bell, it's probably the best picture. But it is also very blunt in composition. I can't imagine why anybody should have made this image. Maybe, if somebody approached a designer, like, "See, we have this wooden toy cube and need an illustration for this face of the cube. What about a cute picture of a robot child riding a llama with a bell on its tail through a desert?" – But, at closer inspection, there's something sinister going on: it's rather the llama that is leading the robot child by a rein, not the other way round. – I mean, this is meant to be a cute toy! And where is the bell? We need to talk about that contract again…
The second image is undisclosed, so we can't really say anything about this.
The third image is rather special. The llama seems to be robotic as well, the robot, which is – again – clearly a child, seems to be not only riding the llama, but both appear somehow integrated into a single unit, which cumulates in the robot child's face screen. There is an eerie feeling about this image. The fact, that the bell seems to be attached to the rein as some kind of link between the llama and its rider doesn't exactly help. (There's also a conic extrusion at the back of the llama, but I'd rather interpret this as part of the llama, and it's not attached to its tail.) The composition in its flat side view produces a tension focusing towards the left side of the frame, on something, which is not shown, but apparently a vital part of the story. While I might notice the mountain in the background, I'd probably forget to write home about the scene being set in the desert. But I would note that we're missing context to understand this image and what may be shown by it.
The fourth image, finally, is clearly Star Wars, robot edition. However, no bell. ("A robot child riding a llama with a bell on its tail through a desert" – "Ah, you mean Star Wars!")
I'm not even sure which of these images Alexander did pick as a winner. And I would describe neither image by the prompt, nor would I dare to imagine that a human had chosen these exact means to show what is described in the prompt.
Having said that, thanks for the link to the DALL-E Mini paper!
Eg, the queries "opening of a medieval theme park", "announcement of a witch trial", "king charles proclamation" might all be reasonably answered by similar images containing a small crowd and a speaker in a medieval-looking setting, even though they're not meant to refer to the same time periods or settings at all.
I admit that there is a certain appeal to those images, for their distinctive dreamlike quality, as there's often a specific tension in the rather blunt composition and an apparent subject, which seems to be beyond what is actually depicted, as if this was just a casually picked specimen from a series of illustrations for a broader story line. But I'd bet that we're going to have been seeing too much of this soon, in order to be still amazed.
I think that puts pretty severe limits on what you can do with it because in a videogame, a comic strip or basically any piece of sequential art you need to keep track of characters and environments as objects.
Embedding a language model in the image generation model seemingly just requires a bigger network.
Really you need image+text->image instead of just text->image generation. Some examples of relevant papers: "Conditioned and composed image retrieval combining and partially fine-tuning CLIP-based features", "IMAGE GENERATION WITH MULTI-MODAL PRIORS USING DENOISING DIFFUSION PROBABILISTIC MODELS". There was a more recent one I saw on Twitter I don't recall the name of. I wouldn't be surprised if these kinds of things work well by a year from now.
However, there is progress towards what you're asking for. The recent work on textual inversion is in the right direction: https://github.com/hlky/sd-enable-textual-inversion
It creates a representation of an entity and allows rending it in different styles and contexts. Currently it involves model fine tuning, but I expect it will become convenient as the power of the operation becomes clear. And once it's convenient, you'll be able to do the progressive queries you're asking for (and it'll be a lot easier to create narratively coherent sets of images.)
Exactly. AI hypemen would have us believe that training ever-larger models on ever-larger datasets is making meaningful progress towards general intelligence, but these kind of simple tests reveal this supposed "intelligence" for what it is: fancy pattern recognition.
Questions that a six year old would easily answer, these models fail at.
I think the ability of these image models to synthesize new images is really amazing. It makes the computer feel like it is doing something organic, and not just applying filters and things to the images. Then, when we see the new image paired with a text that generated it, we think the system might actually know what we're talking about. But it obviously doesn't, it's just the luck of a model with billions of parameters. Whenever the model fails to produce an acceptable output, it stops being intelligent and the user is considered to be bad at their job, or to be asking for something that is unreasonable.
I think it's still spot-on to say that comprehension is far away, even though you can pair outputs to inputs and have a simulacrum of comprehension.
Additionally, it's helpful to look at these systems as tools. We don't expect cars to work well without humans learning how to interact with them in a safe and reliable way. ML tools, thanks to high expectations and moving goalposts, aren't tested in the same way.
But ultimately this specific line of questions - handling context over multiple queries - is something people are actively working on, and I'm confident it'll have some real solutions within a year. It's closely connected to synthesizing video, which has a huge amount of effort going in right now and some really incredible early results already.
And then we can move the goal pays again and continue talking about horses...
> nobody has yet publicly demonstrated a machine that can relate the meanings of sentences to their parts the way a five-year-old child can.
Not very long ago that would have been a 3 year old, or maybe even a smart 2 year old. 5 year olds are extremely good at basic language and understanding tasks. If we get to the point of AI that is as good as a 5 year old we're essentially at AGI.
https://www.sarah-brosnan.com/primate-art
I’m not just poking fun. Art is a measure of cognitive development in humans and there are very typical representations people use at certain ages. 5 year olds are still making pretty rudimentary portraits of circles and triangles with stick limbs.
https://empoweredparents.co/child-development-drawing-stages...
What we know for sure is that we're not there yet. And what seems likely is that we're getting closer, and that's something.
That is as much prediction we can get.
I don't think that Compositionality is a wall, it is clearly an interesting feature, but I think that it is pretty clear by now that the Turing test or anything in the same spirit is far from sufficient.
What a bloviating egomaniac. Does Musk really have the time to deal with pissant researchers like him? Whats 500k to a man worth a hundred billion?
You are engaged in a long conversation with someone, perhaps a friend of a friend who you met for lunch. At some point in the conversation they mention that they have a startup and are seeking someone like you. This revelation colors the whole conversation from that point onward. Indeed, each sentence colors the conversation from moment to moment.
But, you reasonably respond, we can’t test that sort of C, modern AIs don’t do even ELIZA-level dialog yet!
What’s the phrase??? “I rest my case?”
Ironically, the very fact that there is now a compositionality benchmark, as Gary points out, is all you really need to know that it's going to fall in the next decade, and probably sooner than that. I'm not aware of any major benchmark dataset upon which enormous progress has not been made in the last few years. And i'd be more than willing to bet anyone anything they'd like that a great deal of progress will be made on this one over the next few.
PROMPT:
> This is a test of reading comprehension. Read the following passage and answer the questions below in order.
> Passage:
> "You are engaged in a long conversation with someone, perhaps a friend of a friend who you met for lunch. At some point in the conversation they mention that they have a startup and are seeking someone like you. This revelation colors the whole conversation from that point onward. Indeed, each sentence colors the conversation from moment to moment"
> Questions:
> 1. What is the "revelation" referenced?
> 2. What do you think the person is hoping to achieve by inviting you to lunch?
> Answers:
GENERATED OUTPUT:
> 1. The revelation is that the person has a startup and is seeking someone like the reader.
> 2. It is possible that the person is hoping to recruit the reader for their startup.
Or to be tricky, if you were to ask it, "What color was the conversation from one moment to the next?", what would it say?
A) Something AI is currently known to be bad at
B) A matter of degree, not a categorical stumbling block
C) Vague enough of a concept that AI skeptics will continue to complain about it even after the field has moved on
Your example feels less like a description of "compositionality" and more a description of "qualia." It feels an awful lot like dualists trying to carve out a place for magic that no artificial process can reach.
Similarly, if compositionally isn't a categorical stumbling block, then show how that's the case. Making a future prediction about what you think computers will accomplish doesn't do that.
I believe the "hard problem" is far easier than the dualists' interface problem. It can be explained by viewing consciousness not as the driving force behind our thoughts/feelings, but rather an after-the-fact log the brain keeps for itself. Qualia are therefore distinct from the immediate physics of signals reaching the brain; they are instead the brain's own shorthand description of the impact those signals had on the brain.
Compositionality isn't a categorical stumbling block because the machines are actually getting incrementally better at it. The Winoground paper the article references explicitly says that score on their compositionality benchmark does in fact scale with training dataset size, suggesting that while it is more difficult for AI to discover compositionality during training, there is no reason to think it is impossible
Then it must further be tuned to perform even more appropriately on those points than the most effective human driver.
The average of human drivers is not a valid benchmark.
When you think about it, natural language efforts could probably use more effort tackling this same type of challenge.
Humans too IRL so maybe when it's good for the man and good for the machine in completely undeniable ways, then you're on the right track.
Otherwise I thought you had a great introduction to compositionality and didn’t need to tilt at any windmills to make your points. I look forward to seeing your benchmark results for recent and upcoming models.
Yeah, but it's not really that good. Machine translation has improved a great deal, but reading those translations actually involves bringing a lot of human intelligence to the table, "Oh I bet, 'maximum fire alarms spread' on this menu actually means 'very hot sauce'"
If all you're claiming is that ML models exist and have useful commercial applications, then I don't think anyone is going to argue against that point.
But a lot of these AI promoters go further: in the case of the LessWrong folks some of them are convinced that a superintelligent machine capable of enslaving humanity is right around the corner.
This doesn’t make much sense. The task at hand is in no way equivalent in difficulty to flipping a coin. This is kind of like saying, “if you beat Usain Bolt in a race 3/5 times, that doesn’t mean anything; it’s like getting 3/5 coin flips to be heads.”
I would try to get a collection of prompts like "red ball and blue cube" or "an empty plane containing only a red ball and a blue cube" and so on - try to come up with 20 or 30 of these. Then, generate 100 images for each prompt. Next, see how likely it is for a red ball to randomly be on top of a blue cube when it was not directed to be.
After gathering some baseline data we could then test three prompts. "Red ball on top of blue cube" and "Red ball beside blue cube" and "Red ball below blue cube". Generate 100 or 1000 images for each of these prompts. Count respective orientations. Then, decide whether red ball being on top of blue cube is more likely than the baseline when the specific direction is given and whether it is less likely when contrary directions are given.
To take your Usain Bolt example, if you won 3 out of 5 races against him, that might just be because it was an off day for him, and not because you are actually faster than him. If you won 300 out of 500 races done in various circumstances on different days, then that is much more conclusive that you are faster than him. And this bet was even worse than that, because im each of the five tests, the best out of 10 results is picked.
It shows you're probably very competitive with him, though, barring some special circumstance where he says he's suffering from an illness or whatever. You can't compare either racing Usain Bolt or generating complex images with flipping coins. The conditions of this bet demonstrate that AIs are getting better at correctly understanding the specific intentions of prompts when generating images, even if it doesn't show they're anywhere near human-level understanding.
Generating image compositions sounds fairly difficult to do by random. If you took 3 different objects and randomly placed them in a square canvas, the odds that they'd look reasonably placed seem pretty low. So 3/5 correct seems like a non-trivial accomplishment.
Or he was really sick or something.
> So 3/5 correct seems like a non-trivial accomplishment.
It's definitely a non-trivial accomplishment. And it does show that Imagen can get it right sometimes. But with a sample size of 5, you certainly don't have enough data to say it can consistently get descriptions like these right 3/5 of the time.
And the question at hand isn't "can it draw what I asked it to instead of random garbage" it's "can it combine multiple parts of a sentence in the correct way", which, assuming that determining the correct components is already a solved problem, doesn't have as many degrees of freedom. For example in the "astronaut riding a horse" example, if it has half of the results with an astronaut riding a horse, and half with a horse riding an astronaut, it clearly doesn't understand how it is composed, but you still have a decent chance of getting the right image. Especially if you take 10 samples and pick the best one.
People shouldn't be betting on benchmarks because benchmarks can be and usually are gamed (see Goodhart's law). Also, most people couldn't give less f*ck if an AI can write an award-worthy poem (I personally don't care about any form of AI "art", any sort of text an AI can produce or really any meaningless "feat" it (as in the general category) becomes capable of). The only worthy bets are ones that discuss economic impact. How many people will be structurally unemployed because of AI by year X? Will it lower or increase the GDP growth rate and by how much? Will it shift the balance between labor and capital and how? Etc.
So more meaningful bets and less benchmark bullshit that doesn't matter, please.
If the creators have convinced themselves of some kind of "no humans" rule, but also know that this would be regarded as impossibly extreme and raise serious concerns about Google with the outside world, then keeping Imagen private forever may be the most "rational" solution.
This doesn't make sense. The original challenge could well have been to draw robots to begin with. Has no bearing on the outcome imo.
But the real problem here is the refusal to do basic and normal things, like depict people. That's not normal - it's deeply weird and tells us a lot about what must be going on inside Google's ai research effort.
Google is fighting a secret war against the Loab demon race that lives inside the high dimensional vector spaces. They've recently made incursions into our reality via Stable Diffusion.
Robot looking at a cat wearing a top hat appears to be easier than with a human for DALL-E too, judging from the comments on Alexander's article, because both objects are neutral with respect to top hats. But really the whole set of prompts is poorly chosen. The original challenge of arbitrary shapes in relative positions seems the best way to test understanding of grammar and object relationships, exactly to avoid the "humans wear top hats and cats never do" problem.
A better set of prompts is important - in this Gary Marcus is correct - exactly because there's no point defining a specific prompt if later you'll decide you accept a totally different prompt. That kind of invalidates the point of betting on well specified challenges to begin with.
This seems like the wrong way to go about podcasting. What can you say today that will still be interesting to hear in six months?
(Overstated for effect. I do think there's a place for news and timely commentary, but it's far from everything.)
I don't use an app that features reviews. I'd rather spend five minutes listening to the original than five minutes reading about it. Most podcasts I find through guest appearances or criticisms from current subscriptions. (E.g., I subscribed to the excellent "Red Scare" because the also excellent "Pod Damn America" dudes spent like ten minutes bitching about them.)
His GPT-2 examples were thoroughly defeated by GPT-3. Horse riding astronaut is solved. Neural knowledge graphs are a successful thing now. Compositionality isn't solved, but progress is clearly being made.
If he was a serious person, this post could have been a few sentences: "No neural network will achieve <x> score on <y> metric on the Winoground dataset within the next <n> years". Simple, concrete, falsifiable. He has not done this, and one has to wonder why.
Human consciousness is over-rated. I'm reminded of Minsky's Society of Mind - a number of separate, communicating systems. To me, that sounds a lot like what is going on in Google, but they are hiding that.
He's still a useful mind to have around. Like having scifi authors and philosophers. They don't have to be completely grounded in reality to provide useful projections as sources of inspiration and to challenge our grasp of history and growth.
It is a different issue than generating an image based on a bag-of-words, so it isn't surprising that an attempt to solve that issue didn't immediately solve the other.
But a variety of approaches can easily solve this problem.
Descriptions of Napoleon Crossing the Alps are unlikely to read 'A small frenchman wearing a silly hat riding on a horse'. So why would an AI trained on such image descriptions develop any sense for 'compositionality'?
There are interesting things buried in here, but I don’t have time for rambling.
The edge cases of image models have been more succinctly summarized and speculated upon elsewhere.
I think, if I had to give the task to the-subset-of-people-appearing-frequently-on-hn, I would give it to Gwern, not Scott.
btw, AGI is coming 2030. Source? It was revealed to me in a dream. Check my profile to see where you can email to take bets.
I personally think Kurzweil still has a shot at winning it.
Can it be a case of Gell-Mann Amnesia Effect? (https://en.m.wikipedia.org/wiki/Michael_Crichton#GellMannAmn...)
Our world is changing before our very eyes while this guy is belaboring the technicalities. You could hardly ask for a keener display of the philosophical gulf between scientists and engineers.
Especially since the rise of GPT-3 and now these image models, we've seen the pop-culture face of AI become even narrower. The promise of generalization that could lead to intelligent behavior has given way to people sharing amusing pictures or phrases that these models have generated, because that's what they do. It's cool, but it's basically become orthogonal to any AGI, or even AI with applications. It's now just a neat cultural phenomenon from which laypeople somehow extrapolate the kind if stuff the parent is saying.
I'm not saying AI (neural networks) isn't making research progress, it's just that it has almost nothing to do with any of what laypeople extrapolate from it
https://twitter.com/AdeptAILabs/status/1570144499187453952 https://twitter.com/runwayml/status/1568220303808991232
Text to video and converting some selected requests into actions is all nice, but it hardly contradicts the GP's observation: it's nowhere near AGI.
EDIT Since you ninja edited this in:
> Text to video and converting some selected requests into actions is all nice, but it hardly contradicts the GP's observation: it's nowhere near AGI.
If you review the root comment I made, you'll understand that I was never arguing with the GP about AGI in the first place.
At this point, I'm wondering if you're just provoking me deliberately. The comment I replied to said the following:
> The promise of generalization that could lead to intelligent behavior has given way to people sharing amusing pictures or phrases that these models have generated, because that's what they do. It's cool, but it's basically become orthogonal to any AGI, or even AI with applications.
And then I posted evidence of concrete applications that are in progress at some of the most well-resourced companies in Silicon Valley. Absolutely groundbreaking stuff that more than prove sophisticated applications of contemporary AI are well on their way to being realized.
A lot of "histrionic phrases" to describe your reading comprehension ability are occurring to me right now, but I'll refrain from using them.
A Twitter thread demo is not quite a revolution yet IMHO. Even in the 60s some people thought ELIZA was a real person.
I've said all I have to say. Have a nice day.
Neither of the 2 companies you posted marketing materials about is among "some of the most well-resourced companies in Silicon Valley".
Two of the founders of Adept AI are authors on the paper 'Attention Is All You Need'. If you don't understand the significance of that, then you're speaking well outside of what you're qualified to comment on. The company has also raised capital from top tier SV investors.
Runway ML has raised money from Lux Capital.
These companies are not just well-resourced, they are positioned in the upper echelon of the innovation business.
> It is easy to bet against new paradigms in their beginning stages: the Copernican heliocentric model of cosmology was originally less predictive of observed orbits than the intricate looping geocentric competitor. It is simple to play around with a large language model for a bit, watch it make some very discouraging errors, and throw in the towel on the LLM paradigm. But the inexorable scaling laws of deep learning models work in its favor. Language models become more intelligent like clockwork due to the tireless work of the brilliant AI researchers and engineers concentrated in a few Silicon Valley companies to make both the model and the dataset larger.
I don't know about you, but if I feed a program with hundreds of billions of "parameters" a huge chunk of the internet and it can then kinda-sorta do a bunch of things, sometimes semi-intelligently, but for the most part couldn't compete with a 4-year-old child... I'd say that's more on the Ptolemaic side of things than the Copernican side. Certainly "it gets better as you feed it more data" is equally true of both paradigms, so I'm not sure what Roon's point is here.
The appeal to the Copernican revolution itself has a bit of a hype-y, cranky odor. Virtually every crank appeals to Copernicus as a role model and vindicator. Real scientists usually don't, because they are busy with the hard, humbling business of actually figuring out how the world works.
Now don't get me wrong, I am thrilled by the research advances of the last couple decades, the foundation models, AlphaGo and AlphaFold, etc. The action model from Adept is great and Adept may become a very successful company. It's all very cool. But every paradigm shift in AI has been heralded as the thing that will Change Everything, and they usually don't. Big, exciting shifts in research don't necessarily mean as much in practice right away. I tend to think that getting AI "right enough" to have a huge, pervasively transformative impact on human life is going to take quite a few decades at least, if not centuries or more.
The recent progress on generative models is a major research achievement, to be sure. That said, I'm not sure what it means it "terraform society," but so far AI shows no signs of making the same magnitude of impact on society as, say, the S-tier technological advances of the 20th and early 21st centuries, such as the personal computer, Internet, smartphone, or atomic bomb. That all may change if we get AGI that actually works, of course.
I’m the opposite. I’m finally, after a long time, starting to get excited about AI. Yes, most outputs still suck and require a lot of experimentation and rephrasing, and yes, midjourney produces a lot of same-looking things (less freedom, but also less crap compared to dall-e).
But wow, now even I, someone with no artistic talent whatsoever, can with just a few prompts create a cool illustration. My current discord avatar is a sloth drinking a cocktail [0]. Zoomed in, it looks a bit uncanny, but generally and especially at smaller sizes, it’s fine.
I could not draw something even halfway as okay. I would not want to pay someone to do it for me as it’s of no big importance to me (I once paid someone for their Stranger Things as sloths image, but even that was just something they already created, not a commission which would have been vastly more expensive).
Personally, I really can’t wait what the next generation will be like, and what it will enable people to do, what they will enable me to do. Yes, I’m very excited.
https://twitter.com/AdeptAILabs/status/1570144499187453952 https://twitter.com/runwayml/status/1568220303808991232
Because I didn't say it will "obsolete" human labor.
You're right that the word "will" is strong in that statement. But, since basically nothing is absolute in a philosophical sense and certainly no one can prognosticate with certainty, it's fair to read that as "will with high likelihood".
And your surgical nitpicking doesn't hamper my argument in quite the way you think it does. Frankly, I believe it's in defiance of the following HN guideline:
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
https://old.reddit.com/r/TheMotte/comments/v8yyv6/somewhat_c...
Edit: Actually I tried this right now with two prompts, and I was wrong. It might still be that gpt understands compositionality but the prior that people ride horses is just that strong. But what I saw was that with this particular situation the model got it wrong.
Edit 2: With some heavy hinting it managed to understand the situation. Italics mine. "An astronaut is walking on all four. A very small horse is sitting on top of him, riding him even. Shortly after the astronaut stops, exhausted.
The horse is too heavy for the astronaut to carry and he quickly becomes exhausted. Next, the horse gets off the astronaut, stands on its own four legs, and walks away."
Paraphrased that a bit, but I really like that quote.
PS: your link is an embarrassment. It would be flagged and dead if you pasted in the text here.
The difference now is that the timescales are weeks or months instead of generations. I believe we will see models that have super-human "compositional" reasoning within 1 year.
Tangential to AGI, but don't we? Vegans seem to have quite a strong opinion on this assertion.
Humanity: I don’t think your intelligence matches that of a human’s.
AI: I don’t think about you at all.
> but then when somebody builds a system
I mean this is really it. You still have to have a human to build these systems that specialist in one thing. Once you create a system that can automatically create those systems and it doesn't need humans anymore to solve novel problems, then there will be no practical difference in kind between human and AI intelligence.
Except we don't have that. We don't have one human that can create this system by themselves. We have a choice group of a handful of smart, motivated, and quite generously compensated humans working on these problems to create such system. As such, you are already surpassing the "general" intelligence level by quite a lot.
I think that is a very generous take on what we do "automatically". After all, we have millions of years of evolution to build out all the neural circuitry that helps us speech or vision -- it's not like you can throw a soup of genes on the ground and out comes intelligence. What is machine learning doing, if not selecting, out of many possible parametrizations, the ones that are suited to understand vision or speech?
Maths in and of itself doesn't require any physical resources. It's possible that doing maths in practice requires knowledge of the world to extract some kind of product from (I'm skeptical, but it's possible), but in principle a rack mounted server could demonstrate its mathematical ability to the world with nothing more than the ability to send and receive messages.
This hasn't been done so far, not because there are obvious missing prerequsites, or because nobody's tried it, or because it has no value, or because there's a prohibitively high barrier to entry for people to have a go. It hasn't been done because nobody knows how to make a machine be a mathematician, and I've seen little evidence of any progress towards it.
That's my goalpost, always has been. Reach it and I'll be overjoyed. And FWIW, I strongly believe it can be reached. I don't see the latest round of ML (or any ML, really) as a step towards it, but I'd love to be proven wrong.
When I mention this someone always points at some bit of recent research, such as [1], but it's invariably just a new way for a human mathematician to make use of a computer. If anybody knows of any progress, or serious attempts, towards a true AI mathematician I'm very curious to know.
https://dspace.mit.edu/handle/1721.1/132379.2
Is a well known project for an AI Physicist. There are plenty of other groups working on similar projects
>I don't see the latest round of ML (or any ML, really) as a step towards it, but I'd love to be proven wrong.
LLM models have been able to do basic math for quite a while now and some have been trained to solve differential equations, calculus problems, etc. Well on their way to more impressive capabilities.
Neither of the things you mention are of this nature, or working towards it. "Finding a symbolic expression that matches data from an unknown function" (Feynman) and "solv[ing] differential equations, calculus problems, etc" are not descriptions of what a research mathematician does.
It has to be maths for a specific reason. I think it's in some sense the purest form of an ability distinctive to human minds and pervasive in how they work. As I mentioned, it's an ability that can be demonstrated in the absence of any particular physical capability, and yet despite it being perhaps the oldest goal of AI it may be the one we have made least progress towards.
Anyway that's my goalpost, and it's not moving. AGI, being "general", surely should be capable of this hitherto uniquely human activity. If our attempts so far are not capable of it, then clearly they are not "general". If you know of any evidence that my goalpost has been achieved, please let me know. I'm very eager to see it happen.
Never said they were but you said:
>It hasn't been done because nobody knows how to make a machine be a mathematician, and I've seen little evidence of any progress towards it."
Which I showed is not accurate. Certainly people have ideas on how to do it and are actively making progress towards that goal.
>Finding a symbolic expression that matches data from an unknown function" (Feynman) and "solving differential equations, calculus problems, etc" are not descriptions of what a research mathematician does.
All research mathematicians started out solving calculus problems and differential equations.
Why do you expect an AI to sprint before it's learned to crawl?
Ever since computers were invented there has been a hope that you could set up a system that would just churn out interesting new theorems. Indeed it was one of the primary motivations for the invention of the computer, but it hasn't materialised yet.
You clearly consider the progress on solving problems to be progress towards being able to do mathematical research. I don't think it is, any more than progress in, say, graphics is. But maybe I will turn out to be wrong and you will turn out to be right. We won't have the answer until the problem is solved and we have our wonderful machine churning out theorems.
But I think you will probably be able to agree that since mathematical research is something human minds are capable of it's something that an AGI should be capable of, i.e. if an AI approach is inherently incapable of it, it's not AGI. You may consider it an unnecessarily stringent requirement, in that there may be other, easier challenges that AIs can perform that will convince you that they are AGI. That's fine - you think about the problem differently to me, so you find different things persuasive. If you are convinced that a given AI is AGI, though, you shouldn't be too concerned about my particular goalpost given that your AGI should be able to achieve it (and convince me) pretty soon.
We'll see what happens. I'm just explaining what I would find convincing, and pointing out that contrary to the oft-repeated accusation that started this discussion, I for one have never once "moved the goalposts".
Indeed. To be clear, I'm not saying I think any current system is remotely close to AGI. I just think that saying that no one is thinking about or making progress on a math research AI is inaccurate.
I'd settle for a demonstration that a computer has truly independently discovered/invented and proved some significant part of our existing mathematical edifice. This hasn't been achieved yet, either. However, I suspect that once we've figured out how to do this at all, surpassing human capabilities will be inevitable in a relatively short time. So I don't see much value in softening the test unless/until there's some actual candidate available that would pass the softer test.
The value in requiring genuinely new maths is that it makes it unlikely that knowledge of the result has been encoded in the algorithm or training set. Certainly, if GPT-3 were to output Euler's formula that wouldn't be at all convincing as a "discovery".
Edit: Gwern has an extensive history with this so I'll let him do the talking.
https://old.reddit.com/r/TheMotte/comments/v8yyv6/somewhat_c...
Further Edits: Not to mention Scott Alexander who has directly rebutted you numerous times. Or Yann LeCunn. Not sure who exactly is backing down.
https://astralcodexten.substack.com/p/my-bet-ai-size-solves-...
https://astralcodexten.substack.com/p/somewhat-contra-marcus...
https://analyticsindiamag.com/yann-lecun-resumes-war-of-word...
Presumably you approach these arguments like Ben Shapiro and imagine you have "Dunked on the Deep Learning geeks with Facts and Logic."
i have been pretty damn consistent since me 2001 book.
edit: Maybe I made a composition error. https://imgur.com/a/Q7hHduY
> The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not real intelligence.
> Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'"