Google's best Gemini demo was faked
techcrunch.com
techcrunch.com
The Google video demo is hugely faked, to the point of being smoke & mirrors.
The inputs to the AI are still frames, not video.
The input images are different to the video frames, often substantially different.
The prompts were provided as text, not audio as shown.
Both the prompts and responses are significantly longer.
The latency is much higher than shown, which appears to be near real-time.
Etc...
The demo they showed would be a mindblowing improvement in LLM technology and applicability to use-cases such as controlling a home robot.
Instead, we were shown what is essentially a short science fiction movie "inspired" by possible future capabilities, not current capabilities.
Google literally said so: "The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers."
PS: Their other benchmark results are also highly suspect, because there is a high chance that Gemini trained on the exam question-answer pairs inadvertently. They even admit this in several places, such as in their technical paper. So... they knew the results are bogus and published it anyway!
Idk how the Gemini demo (a thing that actually does work and do the things displayed, just not in the exact way shown) is "equivalent" to a literally non functioning car...
> The inputs to the AI are still frames, not video.
A video is just a sequence of frames. The input is always going to be frames when it actually goes into the model, and you don't need every single frame to understand what's happening in the video.
> The prompts were provided as text, not audio as shown.
That's trivial to do now. Using Whisper, you can just turn voice into text and do the exact same thing. They don't really need to demonstrate that.
So sure, they definitely embellished, made it seem real-time and as if it didn't need more target-specific prompting per task. But saying it is completely fake is foolish.
Except they showed their AI the 3 most important frames (rock paper scissors) and not the hundreds in between.
The demonstration was a machine intelligence picking out meaning from a video.
The reality was a HUMAN using their meat brain to pick out the meaningful still frames and feeding that in to an AI that couldn’t have completed the demonstrated task on its own!
This is like making a demo of a robot cleaning a house, without acknowledging the janitorial staff doing the actual cleaning off-camera.
It’s absurdly fraudulent and should never have been made public.
Videos like this for such an existential product ought to have been reviewed by the CEO. After all, Google’s future relevance as a corporation depends on it.
It was requested, made, reviewed, approved, and then published.
A faked video of science fiction wishful thinking for major product launch.
Just wow.
> Steve Jobs also faked
So which is it?
If Google releases a new model that they say can read prompts directly from your brain and they release a demo where they show a person sit quietly and prompt the model just by thinking, and later it's going to be revealed that there was a person sitting behind the camera and furiously typing all queries on a laptop, would you still defend Google? This is honestly ridiculous.
Are you suggesting that next week Gemini will indeed be able to do everything it was shown to do with the latency that was shown?
Also, when the iPhone was released it did all the things that were “demoed” (I don’t remember the presentation so I don’t know if it was demoed the way Google did with Gemini or promised with slides, etc).
Are you suggesting that when Google releases Gemini next week it will be able to do the things that was faked in the demo?
Lots more discussion:
I can't pay for GPT-4 premium the equivalent of it is Bing that like helllllll slow compared to bard