Genie 4
deepmind.google
deepmind.google
It's very good at maintaining consistency for things that stay in frame, but I don't know that this points to it developing a strong "world simulation" like they claim.
> While it can maintain coherent environments over short sessions, visual details and object permanence can still shift or drift over time—especially as a simulation runs longer, after multiple complex prompt edits, or when navigating throughout the scene. Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are active areas of ongoing research.
I suspect it is, because the 2D -> 3D models I've seen are terrible, but more importantly the 2D -> 3d models are terrible. Noteworthy because I suspect the number of people who have 3D scans laying around is tiny.
What you're describing is vaguely how video games work, to my understanding. There may be interesting parallels if that becomes a technique (like how distant objects are sometimes still rendered, but at a lower resolution/polygon count).
Your understanding is broadly correct. There is a 3D world, and the GPU computes how it looks like from a particular viewpoint.
In essence, objects are painted back to front, so that in the final image every pixel shows the nearest visible object. The details get complicated, but the idea is simple.
Just because they happen to be throwing in the option to sign up to to play with the Genie 4 into an Ultra plan, doesn't mean that this is a material part of the value prop for Gemini.
Presumably, they can learn something from how niche Ultra users interested in tech demos engage with it. If that's an upside, it's really unclear it why it would make sense for "a reveal" to promise something too expensive to deliver to such Ultra users. (None of which means that the suggestion that you are responding to is technically reasonable or possible--just that it is unclear that one should dismiss the idea because one views this through the lens of a consumer "product reveal"
Just because OpenAI had to crawl back from Sora to focus resources, and Anthropic proudly proclaims in their filing that they have been avoiding promising creative opportunities for the same reason, doesn't mean that Google can't continue to do lab/research stuff.
It would be odd if they didn't.
Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.
Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?
Or render a low fidelity 3d model for coherence and then raytrace that onto a 2d frame that is then rendered at higher fidelity.
they got this working decently well for video over a year ago, maybe closer to two years ago at this point, time goes fast. so i assume it'll translate to progress in Genie too
I noticed several inconsistencies in things that stayed in frame. See the wall on the bridge in the Van Gogh example as they walk past it to see the other side.
Kind of makes me think that autonomous vehicles will eventually gather better dataset (although limited to roads!), since they do have 360 degree cameras and lidar, so they can track objects before & after they are shown on certain view.
The model already ingests text, so you just to attach a tiny VLM head that reviews the image and read / writes scene continuity text tokens.
Once you have that you can probably just RL with scoring on continuity issues and eventually the VLM head will learn to update dense continuity tokens?
So many fun wierd experiments to run if you have some compute to mess with. There are competent open video diffusion models that you could test this concept easily enough.
or even if you have a vast line of sight but a guy goes behind a tree
Would this be impressive in terms of convincing you to buy something or teach you something or entertain you?
Genuinely asking.
Obviously yes. I mean have you seen what entertains us? And have you seen what a TV ad looks like? It's not a high bar and this is way above it.
Btw your comment is anti-ai slop. Slinging mud with no real criticism or even a personal observation to be found.
This doesn't show up as much where the verifiable reward is "code compiles and tests pass" but you can still absolutely see it in things like movies, prose, or slide decks. Even frontier models are pretty bad at keeping track of narrative threads even as the technical quality drastically improves.
This begs the question of whether the gap can be closed just with more compute (presumably yes, or at least errors can be minimized beyond human capacity to see them) and is that going to be worth it with current tech (possibly no).
the average human cannot vividly recollect a walk through a familiar neighborhood, much less invoke a coherent fantasy of going on such a walk. do they contain a world model?
Humans also get details wrong when they do stuff like this. So right away, we learn this isn't a limitation of AI. If you want to say that getting that wrong is "proof they don't have a world model", that cuts both ways - it means humans do just fine despite not having one of those either.
But that also lends us insight into fixing the issue on merit! If humans also obviously lack a world model, how do we cope with this? Sometimes we use "live action" and reality forces consistency. Sometimes we use editors and QA. Sometimes it ends up on a website listing every little shot-to-shot inconsistency in every single TV show and movie ever made.
And now you have a few clear ideas on how to train the next generation of image models to do better! And you can probably dramatically improve your own prompting by providing reference images, using consistent descriptions, etc..
So, hardly irrelevant.
Our perception in realtime needs to be better than our ability to remember things from the past.
also > souls don't exist
yes i know, but if you think you're no better than an LLM then why don't you put your personality into a system prompt and spare us the carbon footprint
That's only because you cannot see me.
There are still things I can do much better than AI, making videos just happens not to be one of them.
for the time being we've all grown up to value hard work and that's a depth vector we subconsciously consider when watching something, when we watch an AI video it's just pixels, maybe it will change with future generations as what's real and what's not makes no difference
Anti-AI crowd is anti AI. Maybe they were (mostly) happy with status quo and don't like change. Maybe scared for the future. Or any other reason. Went from "AI can't even tell what 2+2 is", to "AI can't even tell whether these simply connected smooth 4-manifolds are merely homeomorphic or actually diffeomorphic after stabilization" without batting an eye. The goalposts will keep being moved and the dismissal will continue.
Whether it's a product that anyone actually wants is another matter, of course - as is whether they'll ever be able to zoom out and bring the entirety of the original vision into focus if this more limited version does become a success.
> Prompt asks for saxophone
Looks inside:
> Sound of a clarinet
Maybe Genie 5 will get it.
Edit: Okay, I read the prompts and they use negative phrasing a lot, so I guess it does work, but boy, are these prompts laborious. Requirements are repeated ad nauseam. This model seems to be quite tone deaf.
What do they mean "Real" here ?
> movies are going to become video games and video games are going to become something unimaginably better
Sometimes I really wanna skip 10 years into the future and see where all this eventually leads.
Sounds glib, but honestly ... more Ads :/
> ... more Ads
That will soon likely no longer be the case.
Do you actually think this is going to happen *to you* or are you just speculating about all the idiot dupes out there?
What is obvious to you/us is not obvious to most..
Perhaps only half of "ads" are actually marked as such
So, it won't change that much from today, the difference won't be as big as the difference between 2000s ads and today
It will be a nightmare! haha
This is the kind of detached from reality, devoid of humanity statement only a tech bro could come up with.
So would I. But I don't not want to experience it as a movie. It'd be something separate. Not an upgrade.
Which makes Sam's comment the inane blather of a pitchman. We don't need to bend over and try to steelman marketing dross. He doesn't care if that statement means nothing, why should we?
Movies come from books, but people still read.
But I get the point.
Altman thinks that movies "will become videogames". As in, he sees media as some sort of ranked ladder where everyone wants to climb as high as possible. He does not understand what movies are or why people like them.
You're projecting pretty hard here. That's what you think of when he says he thinks of them as a videogame. You do know that "games" on netflix are basically movies? See the recent https://www.netflix.com/games/82112833
You seem quite out of touch with what the state of media and games in general if the first thing you thought of was a game with a ladder.
Instead of better it's become unimaginably worse since games can no longer serve as a means of self expression because the second you express anything worthwhile 100 people copy it immediately ... without you. It's like Disney (whose whole business model is copying public domain stories and music and ... but taking out the source and everyone who ever had anything to do with it). Except there's no longer one highly selective company destroying the world's culture one piece at a time, but 10000000 idiots looking to make a quick buck, destroying culture and self-expression like a fire raging in the rainforest.
There will be no more "celeste" games where you join for the platforming but stay for the (politically offensive to all sides) story, because AIs won't discuss the story as it's politically incorrect. Celeste is fundamentally about depression and anxiety, due to being transgender, and is about offending both sides of the argument. First, the main character is transgender, with zero explanation or need for explanation how far that goes, which of course offends conservatives. Second the goal of the main character is to run away from any provided help and professionals (which the director did too) and even under the most extreme of circumstances only reluctantly accept help from a family member that then promptly dies making things "worse".
The Qwen and other Chinese models (and ...) will refuse to discuss being transgender at all, and any western models will shut down any discussion of running away from help hard (because otherwise they get held responsible for the consequences, as has already happened)
(oh and that running away from professional help in psychological issue is actually observed to be a positive thing in almost all cases, in studies. The main way to make psychological problems worse is to let them fester and running away from help is time and time again observed to be far more productive than going to professionals and expecting them to fix things. Because that's an extreme human bias: it can never be (allowed to be) the right reaction to NOT do something about a problem, especially when countless people are paid to "solve" the problem. In other words: if you take people who expressed intent to commit suicide and compare the groups who "get help" and the group that successfully shut down help, less people actually go through with it in the second group, and worse than that, the more intense the help, the better the odds the patient ends up killing themselves. Right to the point that doctors sometimes assist such patients because after 2 years of attempts people live in constant misery and have caused enough medical damage that they will have constant pain or worse the rest of their life. And yet people keep forcing help on patients. Usually in response to an actual suicide where no help was provided. Then more help gets built, and 5 years later we yet again observe ... more suicides. But tell someone with an actual problem to avoid help and you'll be skinned alive for that)
So AI could never make a game like that. Not because it actually can't, or isn't capable, but because there already is a strong political bias, in every model you might use to make it and neither side is going to back down on their policies. The Chinese Communist Party is not suddenly going to change it's mind about transgenderism. And the western models are not suddenly going to openly discuss running away from psychological help, not even as part of a story.
So really, we've just lost these games.
Needle in a Haystack Simulator is the new Threes! https://www.theverge.com/22914955/threes-2048-ketchapp-copyc... A very simple game to clone that gets 10,000 headlines because creators have always been anxious about cheap clones. And, anxiety drives the clicks that pay the bills.
Meanwhile, local models are already extremely capable and slowly shrinking their gap vs. frontier models over time. And, it has been long established that the censorship trained into those models can be disabled easily with open-source tools. De-censored models will happily dig into topics that push beyond the limits of sanity, let alone political pearl-clutching.
Besides that. If you want to write about some topic and AI doesn't want to help you, you still have your fingers. Type like a human and you haven't lost anything compared to when Celeste was made.
Needle in a haystack isn't a game. It's a way to stimulate your brain for 30 seconds at best. It's not a game.
Sounds wonderful. May as well skip this and just spend my life hooked up to an IV drip of drugs.
[1] reality is also somewhat in question in the films universe
At least that's my recollection. I've watched the matrix a few times :)
It's worth remembering that the Matrix is an allegory for systems of labeling and control (notably, gender), which make life (and specifically life with others) easier, though not necessarily better. The Matrix sustains the large human and machine populations amidst Earth's ecological collapse; people who choose Zion choose a harder life where they are free to be themselves. However, when Zion grows large enough to threaten the system, the system comes after its residents and wipes them out.
One person's freedom is tolerable, even necessary for the system to continue; everyone's freedom leads to everyone's doom. The movies have some weird-but-rational implications.
...Sorry, I lost track of my point, which is that we're already in The Matrix. The Matrix is society.
One individual human (Cypher) did choose to be plugged back in, though it didn't work out for him.
Reality is very much blurry
simulation becomes real enough
Riding a real horse isn't just about the horse ride, right? It's about the trip to the horse farm. The small but real danger. If you're doing it on some sort of multi-day trip, it's about the memories you make with others, the romantic time spent with your partner at night, or if you're single maybe the hope of meeting a romantic partner. For some maybe it's even about doing things like cleaning the stables, which isn't intrinsically fun, but probably adds to the feeling of bonding with the horses.
(I've never ridden a horse. I'm surely wrong about some of those things)
There's also the "abundance issue" with simulations. Have you ever looked at your unplayed games on Steam, or a folder full of ROMs, and just had a hard time choosing or enjoying anything, because (a) there's so much there, nothing feels special and (b) even when you're enjoying one game, part of you is thinking that perhaps the next game is even better?
If things truly progress to Matrix-style simulation, I guess a lot of that is "solvable" by "just" manipulating/injecting memories. But I think that's how far a simulation would need to go in order to supplant real life experiences entirely.
Or, there's the even darker route, also explored by much scifi -- reality gets so shitty that nobody can really even aspire to an actual horse ride anyway, so simulations are all anybody has. A lot of people already live some kind of version of this anyway due to lack of funds.
Sorry for no coherent point. Just rambling!
Right now we're fully on track for one of several dystopias.
Llms wont be able to create good games. Consider for a sec that there's like 500 "Great" games ever made, maybe more but not much, in some genres that number is less than 10. There is no training data, and its largely a matter of taste and culture that cant be represented/distilled into vector embeddings.
Even if you have a great game idea, it takes a ton of time and money to even get it off the ground, let alone get it coded correctly and maybe get the reach you want. A LOT of flash in the pan ideas probably fizzled out faster than they needed (most recently Among Us and Fall Guys come to mind) because they were built to ship, not to easily modify.
Tech like this could lower the barrier to smaller and more creative teams or even individuals and get some better competition in the area. Fighting games KNEW that rollback was the way looooooong before AAA bit the bullet and started making it standard, and they still fuck it up (and it probably would've taken longer if COVID didn't force the issue and punish those lagging).
That said 2 of the largest hurdles i've seen in everyone's "magically ideal" game are enemy AI and Plot.
Interesting and well designed enemy AI is still hard, and even games that did it well built the whole game around it (FEAR) and the system is vastly less impressive without that focus (Just Cause). I don't think we're anywhere near modern AI solving the issue of making a good AI system for a game due to design space problems, but it might.
Plot though...that's tricky. Everyone always says how cool it would be for everything to be like Mass Effect or more, with all choices mattering and everything interacting, but someone has to write that. AI is not turning out well though out plot points that don't feel like the equivalent of Bethesda mission creep.
Point of this whole ramble being that in arcadey/simple stuff I think we'll see a lot more great games (and would argue we already were with the growth of the indie scene over the last 15 years thanks to steam and other tools), but there's still going to be entire genres that are just miserable to develop.
Because now their competition outnumbers them 1000 to 1, the competition isn't devs or artists but anyone with a phone, who can be from any part of the world who may not even speak the same language. Many to most of these solo "devs" aren't interested in games at all, and just hoping to rack up sales, ads, hype a crypto shitcoin launch or whatever. These slop games all have flashy attention grabbing visuals, slick product pages, five minute promo videos, with the games boasting 500+ hours of gameplay... of which 99% are recycled borderline unplayable trash.
The indie market for lemons stifles indie devs who struggle to find any audience, with modest dividends going to the most deceptive, scammiest solo sellers churning out 10+ titles per day (mostly 1:1 copycats of GTA/AAA games, psuedo-porn games, gambling games, meme/news cycle one-off games, etc). Middlemen like Steam print more money than ever as purchase volume increases while the indie gaming market becomes closer to the mobile games industry today.
The vast majority of the force multiplier value from AI is captured by the handful of AAA conglomerates who follow the tech monopoly strategy of maintaining market share by constant acquisitions and collectively control the most valuable IP while constantly reducing headcount thanks to models like Genie.
But who knows. It just seems like wishful thinking whenever I hear talk about how AI will empower the little guy to take on Goliaths considering the huge amounts of capital driving AI research and products are coming from those very same giants.
Mappy, weirdly enough, holds up well. Bubble Bobble, still good. Mortal Kombat 1, meh.
Some of my favorite classic games, almost unplayable now. Tastes have changed dramatically.
Movies are the same. Very few movies hold up after multiple decades. Some do. But even some great ones don't.
To us, games like BG3 or E33 are masterpiece, but let take a look the most profitable games, GTA V, that is their "masterpiece". R* Knows this and high chance GTA 6 will continue this trend, a passable story combined with top notch graphic and physic, which both AI is really good at already
It's not the same at all.
If they were really the same, why was there so much uproar over how annoying Shorts was when it was originally added?
How can you honestly think YouTube in 2016 was practically the same as Tik Tok? Why would these other social media companies spend so much trying to change to copy their same strategy if they were all already the same?
Short form content is radically different today than what it was in 2016. It's produced differently. It's presented differently. It's consumed differently. The audience is different. It's pervasiveness in society is different.
God what an annoying thing to say. Suggesting that movies will essentially become games, or that future video games will be unimaginably better (presumably because they are driven by generative world models) betrays such a surface level appreciation for what makes art and culture and all this humanity so beautiful as it is.
The entire point of movies is the deliberate choices of the filmmakers! Movies will not be improved by our ability to choose what the main character looks like or add a style filter. Super Mario 64 is great because it is an expression of the very hardware that constrains it. Inevitable future versions where anything can happen because someone hooked the ROM up to a generative world model won't be better for it.
I agree that this technology is absolutely world changing, and am cautiously eager to see what comes of it, but it just bums me out, perhaps needlessly, that the people at the helm can be so artless, unreflective. Whatever.
It turns out, some people like that. A lot of people like it, presumably because it's fast and stimulating, and everyone's a sucker for wowing FX even as movie FX have gotten worse and more or less converged with video games.
What's lacking in these movies isn't so much plot, it's character and thoughtfulness. They tell us nothing about ourselves.
From an art history class, long ago, I remember the explanation of how classical Greek culture degraded into depictions if "pathos", visible in changes in the sculpture, and how that heralded more or less the decline of the civilization. It seems obvious to me that we've undergone a similar transition. And the thing is, it's not the technology itself, which could be used to produce meaningful art. It's the lack of thought on the part of the creators and the audience.
Exactly. I am not de-facto against the use of AI in art. But artful use of the medium is sparse at the moment.
It’s so true. The only time I watch one of these movies is when I’m stressed and exhausted and want something so dumb I’ll fall asleep in 20 minutes.
When it comes to these “worlds” that are generated, there’s no there there. I’ve tried various of these and after three minutes I close the browser. Walking around my neighborhood is a million times more interesting. Whereas the creator-constrained and -designed experiences (GTA comes to mind) actually deliver something engaging.
I can name several, as well as something profound to be discovered in the execution of the MCU as a whole, but perhaps more salient is why so many people feel the need to tear down the straw man version of Marvel that they have in their minds.
...Oh, fine, I'll give you one:
"What is grief, if not love persevering?"You occasionally see thoughtful movies. Many of them are unsurprisingly based on books. The horror genre has become this weird eddy of creativity too with interesting characters and plots and some surprisingly good acting. Of course you’ve historically had innovation in that genre, often ignored because it’s horror and thus low brow by definition.
My take on horror being creative is that it’s like old school punk, pulp novels, zines, underground music. The key there is that these are genres and styles that give you permission to suck, and therefore permission to try stuff.
If it must be high art or it must gross nine figures, you can’t suck. So you can’t innovate.
"Permission to suck" made me think of something even more fundamental that's relevant to the AI art conversation, and often forgotten; that the best art, the most creative work, comes when you're working around constraints. Those could be the constraints of a medium, if you're chiseling in stone and need to make certain trade-offs; or they could be monetary constraints (you can't afford a cast of 40,000 extras for a war movie, so you turn your focus on the intense local experience of a few characters). In a "necessity is the mother of invention" kind of way, having limitations forces you to think more carefully about every move, and that kind of thought is what can make something a small masterpiece instead of a large sloppy shitshow.
Given an unlimited budget, most directors will lose their heads and do terrible things, even with a good script. Having no limits brings out the worst impulses in any creative endeavor. If "AI filmmaking" comes to stand in for "unlimited budget", then I think the trend in the future will be toward smaller, more human-scale art, because artists will find constraints to work within as a means to focus.
Maybe I'm not saying this as well as I could.
Why would you look for that in any standard action movie?
Not every movie needs to awaken your soul to some big revelation about the universe.
The industry is called "the entertainment industry" for a reason. People are able to be entertained without deep philosophical introspection.
Marvel and DC are the modern versions of ancient Greek mythology, with a huge cast of mortals and demigods. The origin stories of the characters are all morality tales. In the comics, flying around, using superpowers and blowing stuff up was sugarcoating for the real stories, which revolved around the internal conflicts of the characters.
You wouldn't necessarily look for deep meaning or personal growth in a standard action movie, obviously, but somehow I feel like even "Die Hard" delivered more humanity per beat than anything Marvel has made in ages. I don't know why we need to set the bar so low.
It's the universal themes of love, longing, caring. That's what's missing in today's slopfest sequelities of modern movie making.
I hope new tech lowers the cost of production so much it allows those with taste, vision and care to topple the mess of modern 'storytelling'. We are getting the technology, let's hope we can have the imagination.
The concept feels more like a video game than a movie in the show, but presumably you could program the simulation to happen around you instead of you moving through it.
We still have podcasts but you think radio led to video games?
As an example, while Hitchhikers Guide to the Galaxy is available in at least 6 different forms of media, the movie is by far the worst.
An awful lot of mainstream movies do have fairly trite plots, with a predictable film style. That's not entirely hate, sometimes I really want to see Michael Bay blow up half of LA. I don't think the next Fast and Furious movie would suffer horribly from AI writing, it's never really been an "art" kind of series anyways.
Younger people I know, and market to sometimes, literally avoid "plot based media" in favor of brainrot. Akin to what my generation used to call "sooo random"
People catering to it earn a lot. They don't identify as directors, but they are. They don't identify as artists either. They identify as influencers and brands.
You will never be the generation they engage with. I don't expect to base any of my choices in following trends looking for opportunities to be contingent on your affinity for movies and first/third person games.
Even when it comes to spending, they are a powerhouse because parents nationwide and worldwide give them their weekly allowances in platform credits, with no further accountability on how it's spent. There is a large audience already, trends that are full economies within a few months instead of needing to wait an entire 12 to 20 year generation for the audience to have independent purchasing power.
That's what is happening and why someone with even greater perception than me would say things that seem out of touch, but are closer to reality than you'd realize.
GenZ is culturally destitute. It’s shocking and sad. Every time I think no I must just be old and I don’t know, I go and dig and… very little. There’s just not much there except brain rot and blather and music that’s absolutely derivative of previous music at best. At worst it’s musical brainrot, like “phonk” inspired stuff.
I fear GenA will largely be in the same boat.
It’s scrolling and algorithms. People blast weapons companies and stuff like Palantir or Flock or whatever here, and for rational reasons, but infinite scroll plus engagement algorithms has done more harm to humanity at scale than any of those.
yes, and its Gen Alpha that says it. they're just young though, but they are in aggregate an independent spending powerhouse
I still feel like the cultural output of GenZ or lack of it is worrying. Compared to every previous generation by their age, I just don’t see a lot.
I will wait though. Maybe they will get over whatever it is.
And video killed the radio star. Kids these days, man.
What's GenZ doing that is comparable to what any previous generation was doing in their 18-30 age bracket years artistically, musically, culturally, etc.?
I don't think it's their fault BTW. I think it's previous generations fault for creating the system that stole their attention. Every previous generation passed to the next generation a vocabulary and a set of tools to build new culture... until GenX and Millennials (my own cohorts) passed tools to stare at small rectangles and scroll.
This is something your average person can play around with. The filmmakers have been fumbling the ball a lot lately (not all the time, though) and I can't wait to see what rises out of the average populous. Lots of crap expected, for sure, but there will be gems in there that I'm willing to bet will blow away what we've seen in the industry. Kind of how indie game companies are eating AAA's lunch lately.
The new Terra 6.1 is pretty good at doing user testing as well via computer use, going through all the features and noting any bugs or inconsistencies.
It feels like we might be on a trajectory that takes us to a place where someone can just say "I'm bored, make me a new game" and then provide minimal feedback like "this isn't fun, make it better" and get an entertaining game without a lot of iteration. Of course we'll still have games that come from human ideas as well but vibe games seem like they're here to stay.
I'm not sure if this is a limitation of the tech or whether it will eventually have "taste" in such nebulous domains as "what is fun." They still can't write funny jokes despite trillions of dollars of training because even though there's plenty of funny material and discussions on what funny is, I guess it's still kind of a human thing.
Currently, if you want better results, you have to prime the model with what I like to think of as a Context Mood Board consisting of material that you select to express a gestalt. Turning reasoning off also helps since it forces it to rely on what's already in the context.
This is a lot of manual work to set up but you can save it as something that gets loaded when needed. For open weight models you can create a LoRA based on a dataset that represents what you consider to be good taste.
The point is that it is possible to do even though it's currently not available from the major providers. They're more focused on maintaining their lead in capability, taste takes a backseat. At some point it will become more important and then we'll probably see either a curated selection of styles to choose from or they'll just keep them hidden and automatically route you based on your request and your user profile.
If you want a preview of the difference this can make try out some image generation models in ComfyUI and compare the results from the same prompt using the original base models versus the fine tunes and LoRAs created by the community. It's incredible what a difference it can make in the output and what you're seeing is the design sense or artistic taste of the person that did the fine tuning and the artists whose material was selected as input.
Looking even further ahead, there's a huge amount of undiscovered talent out there in people who lack the technical ability to produce art but have a very good sense of taste. Somehow getting access to their preferences would advance things even more. There's still a lot of training to be done so we're hardly at the limit of what is possible for instilling taste or artistic sense in LLMs.
That came from you. And it’s a deep thing.
Yeah, it is kinda wild how the infinitude of possibility suddenly makes everything so dull.
Also, does anyone want movies to become games? I don't. I want movies to be movies. Good movies. With interesting stories, beautiful artistic cinematography, great characters, moving paintings as Lynch said it.
If I want games, I'll play games.
Either the desert of the real, or something unimaginable to Mk 1 humans.
Definitely wouldn't have been one of my picks to show off my fancy new model.
"Which one!?"
I've seen weird stuff in images that OpenAI sends me in it's promotional emails too. I would think that if you were trying to show off you'd make sure there's none of these obvious AI issues in what you choose to show off with. Maybe they don't really care? "Gemini choose 10 scenes that would be impressive for our announcement and add them to the post."
As he passes by the tower, it becomes round, by 0:40
What on Earth is audio-like music? Do they mean music-like audio?
It seems like a good descriptor to me! ;)
A proprietary version of this technology is already training Waymo's robot cars. Waymo blogged about their DeepMind-provided world model a year ago.
> It’s so obviously not representative of the real world.
That entirely depends on the use-case and level of granularity needed, doesn't it?
This blog is suggesting that we could train robots using only a world model.
> Do you have a basis for this idea, or it's a gut-feel?
My basis is that I’m a real human and these videos look far more like bad video games than the real life I’ve experienced.
> Isn’t the Waymo version used for some small portion of the dataset to cover edge cases that they have no real data for?
The usecases in the blog post[1] paint a much more expansive picture of how Wayne used Genie 3: including enabling transfer learning to convert single-camera dashcam footage into simulated full-take sensor data (multi-camera and Lidar), which is extremely valuable.
> My basis is that I’m a real human and these videos look far more like bad video games than the real life I’ve experienced
1:1 fidelity isn't required for a lot of training tasks; for instance, computer vision can do a lot with 32x32 resolution images, which is trash quality to humans. I think the potential customers for this technology will constrain the model to fit their use cases.
1. https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-f...
When your training simulation fails to reproduce certain features of the real-world environment, your control model is likely to have trouble with them, so more realistic simulations are helpful, but even fairly unrealistic training simulations are pretty useful.
In general, training neural networks to control robots is going to require simulations, ideally differentiable simulations. Even if they aren't differentiable, though, you can SGD your simulated training runs across a data center in a way that you just can't with training runs on Roombas.
They clearly know how to train robots. Unless Genie is a misdirection...
Conversely, try to think of examples where Generative AI has betrayed a lack of creativity or care from the creator, and thus left a sour or ambivalent or impression.
I find the former impossibly hard, and the latter depressingly easy. Do others feel the same?
If you mean world models like Genie,I don't know. If you mean the latter, I have definitely used software that I have enjoyed that I do not believe would have been possible without generative AI.
Fluxer is the first Discord competitor that when you show to someone they can't say "well Discord does X better", and it was largely made using AI.
Herdr is designed for AI usage, but I actually really like it even for just normal terminal multiplexing. It was surely made with a lot of AI.
I have seen people make incredible things at my work that have actually solved multi-year long pain points that we've never had the resources to address before.
I have used AI to patch and revive a bunch of small services like chat bots or web services and such that I had made and had decayed into failure over the years.
Sure, it's easy to point at people using AI as a megaphone for their lack of care. But I don't think it's impossibly hard to find people doing good things with it either.
When software is hard there's a kind of filter; a good piece of software means a human/team behind it with grit and care and user empathy. You can have some* confidence in the continued stability and maintenance of the software. (*relatively speaking, and with huge error bars).
Without knowing the details of Fluxer, if someone tells me about a vibecoded Discord clone, my immediate thoughts are like: - how much thought and care went into this? - how much work from Discord has been cannibalized? - will the authors be around next week to keep maintaining and making improvements and thinking deeply about their users?
I have also seen a lot of fantastic demos and stuff with AI coding. I use AI coding all day. I just think there's a weird gap still, where code and images and text are all "cheap", but none of this has translated into anything I personally use or care about.
Didn't know that before, but considering its abysmal failures to scale, yeah in hindsight it does make a lot of sense. (one of) My group(s) tried to move and we came back to Discord rather quickly.
It took the medium of film ~50 years to find Hitchock and ~75 to find Spielberg.
I'm encouraging others to do the same and be honest with themselves about the direction we're headed.
The predominant way people engage with this technology right now is through slop content in their instagram feeds.
(Btw I say all this as someone that has worked on generative models for 10 years).
If you want to get inspired go to civitai and look through the top stuff and be wowed. Maybe they didn't draw it, but they figured out how to make it happen and when you look at enough generated stuff you can tell the average from the holy shit. It's easy to get inspired by what people can come up with and want to do something yourself, which is a positive impact I'd say.
I can't personally relate, and I think its kind of telling that I'm being down-voted for simply asking people to engage with this thought experiment.
I don't think it would be a good user experience. I just crave the feeling of navigating such 'world'.
sure, it's still a cool accomplishment but i can understand being a little bit frustrated / not as excited for such an announcement.
Whining that I, a random Internet user with limited budget, usecases that the researchers don't care about, and no connection to the researchers, don't get technical details (??) or access (??) on day one is just profoundly goofy shit.
Obviously being able to use these things right away would be more exciting. But to be unable to get excited by advancements we can't immediately use is... quite entitled.
Why not expect it? It's what most model releases do.
Well, I didn't go knocking on Google's door asking them to give me access to their fancy models. It's them who came up to us saying "hey, look at the amazing model we have here; too bad we won't tell you anything about how it works nor even let you access it". Entitled you say? Then maybe don't dangle a carrot right before my face, and then complain that I would like to eat it?
"Don't talk publicly about things I personally cannot use right now" is peak entitlement, yes.
I never said anything like this? My point wasn't that "I can't use it, therefore it's bad"; my point was that as-is there's pretty much nothing exciting about this. Just because this exists inside Google doesn't really mean or do anything; it doesn't meaningfully advance the state-of-art precisely because nothing about this is public.
Do you remember Sora[1]? Announced in pretty much exactly the same way -- grandiose webpage with fancy demos, zero technical details, limited availability and then... remind me what happened? Oh, right, they completely shut it down, and in the end it didn't "advance" squat. Might as well just never existed. So remind me why exactly I should be excited about this one here...?
> erhmygord I can't even buy this car right this second for nearly nothing and use it to drive a thousand miles per hour through a kindergarten parking lot?!?!
And I wish I had thought to grab that nick years ago…
When tweetles battle beetles… AND…
Only open weights matter.
We should distill everything else we can into open weights.
These are exciting for robotics and industrial perhaps, but they have no consumer use case until the lid is blown off and consumers can do whatever they want with them.
So much societal progress is behind held up because the leading labs want to keep a stranglehold on the tech. Imagine if every PhD or hacker had access to the raw weights.
Which stems from the fact that most people have never created or contributed anything meaningful.
It used to be a lot different.
Look at companies trying to build more things that pull value into their private cloud offerings.
It's just a lame gimmick, this is not 2024 anymore, people have started to get sick of this AI thingie, especially when it comes to video/image generation. That Simon guy with his pelican is the exception that can only be seen on tech forums like this one.
As to why US (and probably also some Chinese) companies have started pouring hundreds of billions into AI-image generation is beyond my comprehension, but on the other hand I'm no economic genius.
If this is the best “frontier” they can achieve, then it’s no wonder they haven’t done anything groundbreaking in the space yet that produces practical results.
I feel refusal'd just *looking* at this page.
Gah. Go iterate. Listen to customers, adapt, iterate some more etc etc. That's how products are made.
(Maybe I should reallocate some of my GOOG stock into ANT after all.)
It’s where I get the most value, and my expectations are very low, no attachment if it goes away
Of course they are mining me for data anyway, but they were doing that before already (and I do have a Gmail account, although I’ve almost completely stopped using Chrome - btw, it’s crazy that you can’t add plugins/extensions to Chrome on Guest mode)
I suspect you're reading the base prompt, but the video also demonstrates adding further prompts to change what is happening as the scene progresses.
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
It's where (ex-)"AI luminaries" go to die. It's a way to make glacial, slow progress in the correct direction while never actually getting anywhere.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone (some model) who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
If it's the latter, I don't see a 'hole'.
This has already happened to other ML fields. For example, there's not much point anymore to ASR models (just feed audio into LLMs) and the space that TTS models occupy is fast disappearing. Oh and it's not just that Qwen 3.6 is a good ASR in addition to being good at coding mario. Doing things this way, not using an ASR model is fundamentally more expressive. A whole bunch of companies started up on ASR + TTS "towers" (lots of models, Pipecat style) as they're called, and they're dead companies walking. Their tech has been surpassed by just bog-standard generic LLMs in a way that they will never get the upper hand again.
A chain of models approach, a separate ASR, will never do this: "you said you agree but I can hear you're sarcastic, so I'll explain again". Also getting a conversational tower to switch languages when the user does that (or when it makes sense in a language course) is not quite impossible but very, very hard. And yet feeding audio directly into Qwen will almost magically solve these problems better than I will ever do it (lightly finetuned). It's fundamentally better in every way (except perhaps speed, of course. Although arguably such a tower needs an LLM too, it'll just have way less tokens)
This point has long been passed for image recognition. Feeding a photo or screenshot into a full LLM along with a request on what to do beats the crap out of any OCR, image recognition, ... model every time these days.
And now there's also hints of world models just being bypassed. Just giving a finetuned model the last 50 video frames (surprisingly, that fits in context, especially at reduced resolution), movement statistics, motor power, status, ... of a robot and asking what to do actually works (with heavy finetuning, but I think it'll start working without). Well ... AND then distilling that knowledge into a much smaller model because B200 racks are more than a bit to heavy for a logistics robot to carry around. But we'll get over that.
The results so far from genie have been stunning, and its not obvious to me that another approach is guaranteed to end up better for all use cases.
Yes! Hence my comment about world models being where (ex-)"AI luminaries" go to die. They had a great idea and pushed it, advancing their career, but as you point out this does not provide even them with a better perspective on where things are going. When it comes to the next big thing they have no better chance than a fresh grad student has.
But simulation (in this round, world models) is how you get the current thing to perform 1% better. So they work on that, and because of their reputation loads of money and resources pours into it because they chose it ... and then the solution comes from a different place entirely.
Except things are changing too, and world models like Google and Meta and Mistral and ... are pursuing appear to be getting bypassed by the current tech. It's starting to look a lot like the "internal world model" of a transformer that you've shown 1 million youtube movies is better than a separate world model, probably because the resources they pour into their general chatbot models are still an order of magnitude above what they can pour into their world models.
I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events.
We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff.
As an aside, why fall into the role of being an apologist for google-hype and Big Tech?
When you have a conversation, you have an idea in your mind of how the conversation is going to go. You predict what the other person is going to say before they say it.
When you are driving, you predict what the other drivers are going to do, you predict what signs say, how long you have until the green. If you see two red lights attached to the back of a car flash on, you may predict that the car is about to slow down and you start slowing down before their speed has changed.
Forget video games and movies. I believe these models will be that inner predictive model for robots.
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine.
I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal.
And no, this didn't: https://www.youtube.com/watch?v=V2K_OLiZ_RM
Some sort of simple primal generative AI construct that just feeds a conscious experience for one. You. Nothing else is real. No one else is real.
Something like that would be an infinitely simpler, than running the physics and conscious experiences of this unimaginably large universe we think we're in.
One possible use for these is to facilitate lab -> this -> real world for robot models. Basically better than lab / algorithmically designed environments and cheaper / easier to train on than real life. Something that "looks" like real life and can scale, to apply RL for robotics to.
Anyway, let's take your theory to its logical conclusion: the product development between real and simulated soft tissue behavior is much bigger than Exponential Time being a hypothesis and being proven. There isn't some effective acceleraltruist path here. Dario Amodei was a biotech dropout. The thing you are betting on which Google is not doing anyway, maybe they will destroy the descriptive single cell sequencing paper mill, but they're not going to be able to simulate the fat inside a BMI 30 abuela undergoing a gallbladder removal.
It's a dystopian business use case that I hope will get obliterated in the near future as the generations that believe in this stuff and are fascinated by it age out of the population.
The original Genie was a great piece of work with a real sense of purpose - to enable agents to learn about the real world in the way Dreamer or Schmidhuber's paper sketched. But now most of the people that drove that mission have left Deepmind (Jack Parker-Holder, Tim Rocktaschel, Demis) and it's become this instead.
The military capabilities don’t even need to be mentioned.
I am more interested in how to make consistent "participants" and "environment" with multiple clips so they look like one longer clip.
Will that require same dev/coding harness and spec-kit-like engineering flow?
Like create a spec on how a movie will look like, feed contexts, create tasks so that multiple clips look congruent?
A visual makeover is great but what happens when millions of people fall in love with virtual characters who seem to deserve that love?
In a nutshell, (with my memory of 30-40 years), the more and more realistic a scene becomes, the less the brain applies it to ourself. It becomes this definite place, definite person, rather than ourself or our surroundings projected into other characters.
All these scenes, that I guess are amazing, they all are creepy to me.
It's easy for them to show demos of abstract things because they don't necessarily have to follow real-world rules, but anything modeled after real world falls short of expectations.
- in Genie 4 the character's suit stopped making sense in the back where it now has a weird fold?
- the controls don't control the character even anymore as now its controlling some kid-height character in first person instead?
- Everything looks like Unreal Engine.
This does not look good.
Either way, the sense of believability was rapidly (and smoothly) lost.
The only place I see this applicable is for visual artists prototyping
So yea that one expensive ad would probably have a positive impact compared to a tidal wave of slop. Quality > Quantity after all.
Video games are interesting because of very specific constraints the developers impose on the player.
I know I could run the content through a fingerprint detector. However, I'm a professional software engineer...not a normal user.
What's the ethical impact of this technology??
Almost all the AI gen videos I have seen have extremely weird camera movement when panning around, it usually just looks way too smooth/uncanny.
[1]: https://www.minid.net/2026/8/2/the-actor-was-the-expensive-p...
Question I have is when will they be able to render something more then 30 seconds.
And how much will it cost? There has to be a HUGE amount of computing power behind this. As a video game developer I live in fear and anticipation.
Real nice prompting there, Google. It also still looks like a video game rendering.
Would be fun to muck around with world models. Are there any that I can try for under £200 p/m without being part of a special project?
The painting->interactive scene looks astonishing though
> audio-like music
That's a hyphen. What's "audio-like music" -- music that is almost sound but not quite? Music made from raw audio? :-)
audio, e.g. music
audio — like music
Hyphens should be used to link compound/suffix words, not em dashes.
Yes, "-like" is commonly used as a suffix, in which case a hyphen is correct. But describing music as "audio-like" is clearly nonsense (music isn't "like" audio, it is audio), so that's not the sense in which it's being used here.
For clarity, the full sentence is:
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
audio: music, ...
Looks like a hiccup, which created an odd and ambiguous sentence.
My favorite kind of music.
Every time I'm about to be excited about AI the cynic in me slaps me back to reality. I imagine the endless possibilities of AI for disabled people, then realize that it's going to leave millions of people disabled instead.
> AI Ultra (and thus Genie) are not available for Workspace companies
Who is this for?
I’ve also wondered what they mean by “world model”, whereas I interpret them as being about predicting outcomes - something an LLM cannot reliably do.
Is this new model an LLM or a predictive outcome model (as you say a la Yann Lecun), or some hybrid?
An LLM generates four alternative prompts for what could happen next and there's a real time vote running for each option. It's trippy, like watching someone's dream.
Right now LLMs only have text, images, and videos to train from. Anything it understands about physics is from an abstract level. It's like learning how to calculate throwing a baseball before you've ever even thrown a baseball. Imagine having to learn force = mass x acceleration before you've even picked up a ball.
What this world generation Google is doing appears to be something Nvidia has spoken about too. Letting LLMs/AI learn from a world physics simulation. Meaning LLMs having the opportunity to "experience" physics, the same way you intuitively know how to throw a ball after attempting a few times yourself.
If an LLM can better understand how our real world works the quality of its output improves -- theoretically.
As far as your taxes go, you can create an intelligent workflow right now to do that for you. Don't need AGI for that.
for all the a.i misery with hardware prices i.e ram.
such a wonderful world we live in now.
Show them or I'll assume this is pure marketing.
I'm very impressed!
If such a system were possible, it seems like it would be far better at consistency and realism, since it could be fine tuned and hand crafted to best represent the desired world. It could be used to make interactive experiences, games, perhaps even feature films!
Alas.
It doesn’t matter how impressive your demo is if it takes a literal data center to run.
Actual innovation can’t be seen at the let’s just throw money at it level. But, it can be seen at the that’s cool. I can run it on my laptop level.
In the same way, seeing that something is possible gives hope to the people who are making those open models that you want.
I read a book, watch a movie, and play a game, to get the point of view and creative decisions by the people who make them. I don't want shadowy projections of my own mind and whims creating the experience. I can conceptually understand why some people may _think_ they want a "genie" that just grants every wish, but my suspicion is that much like other slop deluging us, they will quickly see that just getting whatever you want doesn't make interesting or good
Yah, meta is discouraged here but why are we as a community putting up with this level of meta disruption? I guess I'm supposed to adapt to the new community but there's nothing there there.
I will stick with the more technical posts here and avoid anything to do with a new AI model and the complete and shocking thoughtlessness of the commenters (most of them).
[edit: grammar and spelling]
You can't just remove the societal/cultural aspects of art without neutering its emotional appeal and impact. Ripping all humanity of out movies/video games is not going to result in higher quality art.
The market has literally nothing to say about what is worthwhile artistically. It just says what people are willing to buy.
Until you've spend $100-150 on their platform. Then you get 250 per day until you spend $2000 and it becomes properly unlimited.
It is really annoying. I'm literally trying to pay you for a service and you won't let me.
... I'm suddenly reminded of Iain M Banks' "Surface Detail".
If these are the cause of why I have to pay 2-4x I used to for a computer, then Google should go fuck themselves.
> Genie form [sic] can simulate scenarios
It's kind of amazing: the juxtaposition of such advanced generative AI and such terrible speech-to-text coming from the same parent company.
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
(Emphasis mine). Like, yeah, I'd sure hope the music is similar to audio.
I wonder if this was a de-LLMese pass gone wrong, and the original was "enabling audio -- like music, ambient noises, and interactive dialogue -- to be simultaneously generated with the world.
It's why if you search a practical task you will often see results link to specific parts of a video.
So, Stadia + World Models = New generation of DRM for games.
Ai generated games suck even more, no skill; no taste, there is no TLC put into them
Then there is the economics of it, cost prohibitive to (re)generate a scene and ship that over internet pipes
https://www.denofgeek.com/games/why-google-stadia-failed-shu...
> beforehand...
is about the craze, the ChatGPT moment and beyond, when GPUs demand went exponential, not when various types of Ai algorithms were available
... history is helpful ... google didn't - and still doesn't - understand gamers, same for all the other Ai hucksters, gamers are among the most anti-Ai
The latency was no good even on fast internet for even slow games. It is and was pointless.
That made no sense and wasn't a viable business model.
There are so many options for games now that just sticking with the biggest AAA companies seems like a waste.
October? Yeah, just release the "Spiderman saves Halloween" prompt.
This is a wet dream for big studios.
We'll be riding the huge wave of indie titles that are incoming.
The problem will be separating the wheat (people with good stories and concepts enabled by Ai) from the chaff (people with no taste slopping out trash with Ai)
I mean - reviewers!
been checking out the archives lately, nothing recent excites me
I also love thrifting and artifact hunting.
Also, lmao, they're already trying to nickel and dime you and sell you microtransactions that took 5 minutes to make for 10 bucks. If you think they're going to pay the costs for running a world model for every single person, lol.
Do you have even the slightest idea of the level of expectation that even the most casual gamer has?
This isn’t achieving much beyond prototyping. They’re burning so much money for no reason.
Is it feasible? What Dyson sphere level of power would be needed to maintain such a simulation? Is that level of energy available to whomever is running the simulation?
Is there a positive ROI? What does this advanced civilization get out of running such an expensive simulation?
Is it moral? The simulation involves sentient creatures who are capable of deeply contemplating their own existence, capable of feeling rapturous pleasure and indescribable pain and loss, and ostensibly capable of making moral choices that range from heroic to depraved. Doesn't there come a point where the beings in a simulation are so close to the beings in "base reality" that the same moral principles apply? If you press a button that kicks off a simulation where humans can torture and murder each other, behaving essentially identically to humans in "base reality," aren't you in some sense facilitating those deaths?
Reductionist but I feel like that’s how it would go.
Why would you assume energy is relevant to whoever turned on our simulation? If I created a simulation where the only source of energy was cotton candy, my creations might ask "but where would our creator even get enough cotton candy to run a simulation like this", not realize that it's absurd to think that the creator of a simulation has any of the same constraints that exist inside the simulation.