OpenAI: Sora: First Impressions
openai.com
openai.com
Roughly what you would expect, good for artsy pieces where you don't need the model to generate anything very specific, but not very useful for most work since most work you want that control.
In other words it will be used for very similar things as current image generators, like intro scenes, short one offs, concept art etc.
We seem to be on a timeline where most of the significant use cases that the model doesn't handle well today is less than 2 years away from significant improvement.
My (completely baseless) guess is that within 2 years we begin to see "high budget" feature length productions beginning to move towards a cost saving model which fully allocate the production budget to primarily virtual content.
In less than a few years time there will almost certainly be a vast ecosystem of production and post production tools to give creators the controls to reliably create and fine tune their shots.
How would the golden standard text-to-video user interface would look? And I have been thinking on this for years, even before the current generative AI boom, and I wonder if it could generate like a 3D representation of the scene that you described, like there would be a file where you could very easily change things around, as if that thing had been created on Blender or whatever, but very very user-friendly and easy to edit things.
It will seem silly what I'm going to say, but the ideal interface, it reminds those movies people did using the game "The Sims", and how you could very easily move objects, and move the camera, and so on. What I'm trying to say here is that I would imagine these models creating a 3D representation of the scene, and the movie-making process ends up being somewhat similar to how could you could customize objects/people in that game.
Basically you use Sora to generate a promising scene, then you ask it (or another model) to turn that scene into a scene graph in a text file.
It will make mistakes, but it could work similarly to the Python interpreter in ChatGPT--it can iterate until everything is OK. Maybe there could even be some adversarial stuff where the scene graph is rendered on the fly to compare it to the generated clip, etc.
And then you can use you standard toolset to edit it, probably enhanced with a copilot model to automate as much as possible.
I’ll be impressed when ILM talks about it.
Pair that with the increasing legal copyright headwinds in terms of sucking in the world's data, and I can see this flatlining.
All that processing power is expensive, and these companies are yet to have a clear path to monetization. Stability AI is already circling the drain.
This has the "let's back the cash truck up to lock in the market share" of the streaming wars vibes, until they realized it's just not sustainable.
I wish it was clearer how Sora was used by each artist and how it impacted the provided examples. (I think I see some Sora generated output but I'd imagine it's not as clear cut in artistic works.)
Finally, software that makes images that don’t quite look right. The use cases for these will be unending
* What does Sora actually do? * What does it not do? * What will it likely be useful for? * And finally, what will be needed to actually replace the majority of video generation use cases?
Anyone into experimental film making can see this is basically the ultimate creative tool.
If you can't see the creative possibilities here you just don't have much imagination.
OpenAI doesn't have even that because they're an AI company, not a VFX company. Besides not even understanding the needs of their users, they see this model as a neat intermediate result on their path to the AGI, and as a progress report to raise more money. They're really interested in advanced emergent behavior it exhibits, not in artistic tools. For this reason they've never bothered to fix all the artifacts DALL-E 3 gives, let alone add any tooling to it. Sora will be the same, and its quality doesn't even remotely approach to what is required in production. It's more of an experiment.
What you see in the OP is simply a marketing material made by OpenAI in an attempt to make them look less nefarious to creatives by appealing to authority (took them quite long to understand that, usually they're superb at marketing). I can guarantee you won't see any real use of it in production because it's just not what OpenAI in there for. They already probably have another better model in the making, anyway.
Models made by actual VFX software companies will have a chance to be used because they care about the usability. Models made by Stability (SD3 has the same architecture as Sora, although it's for image generation) also have a chance because they are open-weights and have tremendous amounts of tooling around them. Models from OpenAI, unlikely.
I have always wanted to play around with experimental film making but not enough to find a group of actors let alone pay them.
The negative reaction to this reminds me of when rap was new. "It is not music, they aren't even singing!"