To clarify this test is purely a PASS/FAIL - unsuccessful means that the model NEVER managed to generate an image adhering to the prompt. So as an example, Midjourney 7 did not manage to generate the correct vertical stack of translucent cubes ordered by color in 64 gen attempts.
It's a little beyond the scope of my site but I do like the idea of maintaining a more granular metric for the models that were successful to see how often they were successful.
Cool site btw! Thanks for sharing.
Actually, search engines do this this too: Google something with many possible meanings -- like "egg" -- on Google, and you'll get a set of intentionally diversified results. I get Wikipedia; then a restaurant; then YouTube cooking videos; Big Green Egg's homepage; news stories about egg shortages. Each individual link is very unlike the others to maximize the chance that one of them's the one you want.
Half a year ago, that was sort of possible for some genius really bent on making it happen. A year ago, that was unthinkable. Today, it's a matter of drag&dropping a workflow to a fresh ComfyUI install and downloading a couple dozen GB of img2vid models.
The returns on R&D are not diminishing, the progress is just not happening everywhere evenly and at the same time.
Sorry to dunk so hard, but your example of technology stagnating is actually an example of breakthrough technological innovation deep into a product’s lifecycle: the very thing you were trying to say doesn’t happen.
If you want to doubt that it was in fact a not a turning point you'd need to provide very strong arguments.
In my view the reason the iPhone felt so new was almost entirely the incredibly responsive capacitive touch screen with a finger ui, everything I'd used before it did resistive and preferred pen for detail. Pen actually is better for detail so in some ways it was that more than anything else that turned the device from a creation device to a consumption device which was whole new way of thinking about smart personal devices.
Of course it was also sold in a decent package too where Apple did deals that ensured it was available with good mobile internet plans which were also unusual at the time.