Yeah, GLM have been beating Anthropic on the pelicans for a while now.
(I suspect that's more of an indication that Anthropic have chosen not to waste resources training on animals riding vehicles, personally.)
(I suspect that's more of an indication that Anthropic have chosen not to waste resources training on animals riding vehicles, personally.)
The reason I thought this was an interesting benchmark is because it’s a non-image generating model creating an image using SVG code, so it kinda spans capabilities.
If an AI lab trained a model specifically for animals riding bicycles it seem trivial to modify the prompt and determine if it was trained specifically for that or if it’s generalized a skill and can also generate a proper orangutan walking on stilts or an armadillo on a skateboard, this sort of thing?
Google Gemini have openly boasted about their animals on vehicles results! https://x.com/JeffDean/status/2024525132266688757