DreamWorks Animation to release MoonRay as open source
awn.com
awn.com
Really impressed that Dreamworks is making this move. They've made some of my favorite films of all time (the Dragons franchise genuinely changed my life and is a major part of why I love filmmaking, film scores, and 3D rendering) and I'm glad to see they're doing this. I can't wait to try MoonRay in Blender once a community integration exists!
[0]https://openmoonray.org/about
[1]https://jo.dreggn.org/path-tracing-in-production/2017/Moonra...
[2]https://www.awn.com/animationworld/how-moonray-became-hidden...
EDIT: I've also come across this excellent blog post written by an engineer who worked on MoonRay for 4 years: http://rendering-memo.blogspot.com/2019/
And this paper recommended by erichocean in their reply below: http://www.tabellion.org/et/paper17/MoonRay.pdf
It's bone-headed. They could have partnered with a cloud vendor and maintained control and a large percent of the revenue. Instead, Amazon can now turn this into an AWS offering, drive mindshare to the AWS offering, and then bleed DreamWorks of whatever competitive advantage this offered them. AWS makes money, DreamWorks loses talent and leadership that moves over to work at AWS.
Open sourcing big software with an enterprise market and giving it a permissive license is stupid, because you can't compete. Look at pretty much every permissively licensed database vendor. Elastic's recent fight comes to mind.
In ten years, when this stuff becomes easy enough for small studios to run (something my startup is working on), the need for big studios like DreamWorks to pursue projects funded by institutional capital evaporates. Everyone will be making real time 3D animation. If you look carefully, you can see the genesis of that trend now.
If I were in a leadership position at DreamWorks, I would be staffing up how to make these offerings broadly available. How to make tools that put more people in the director seat. That sounds like the mission of a company named "Dream Works". Let more people dream. I guess this does it, just not in a way that they'll be using your company to do so.
Big disruptions are coming to the creative fields. This kind of move is like giving away the keys to Amazon for free. (I mean, they are buying Hollywood studios left and right. If you don't see it, you're asleep at the wheel.)
Having a rendering engine that's really good is useless unless you have the talent to use it to make a movie, and Dreamworks is going to maintain their branding and reputation for a long time regardless of whether their tech is open source or not.
They aren't "giving away the key" because the render engine is not the key. It's a complement, and they're simply commoditizing their complement.
While I agree that high quality art is going to become completely commonplace over the next decade or two, I really don't think it's tied in any way to one company releasing their internal render engine to the public.
You can have the most advanced visual effects and still make a terrible movie. And you can make a great movie with slightly outdated effects and still wow people (Mitchels vs The Machines is a great example).
Dreamwork, being in the business of movie-making and not technology as a service, only benefits from an open renderer atmosphere. Pretrained talent, open source collaboration and contributions to improving their software and PR wins.
Side note, that movie was spectacular. I have poster for it on my apartment wall!
There are so many people that want to direct, yet the current Hollywood regime only has so much room at the top to grow into. That's going to change.
And if you doubt that people want to be creative, just look at TikTok. Really, YouTube should have been our first evidence of this.
For now.
It's their job to know where the market is heading, and the world of 3D animation is about to be blown wide open for smaller teams to accomplish what the big incumbents have done for decades.
Tooling will be one of the areas where money is made, and they just gave it away for free.
> Dreamworks is going to maintain their branding and reputation for a long time regardless of whether their tech is open source or not.
This is where we fundamentally disagree. Films currently require institutional capital and large teams to make, and that's about to change in a big way. The film industry is about to undergo something vaguely resembling what the news and journalism industry underwent as access to the tools of creation and distribution become more widespread and accessible.
The studio system exists because making movies is hard. It only needs to exist while that remains the case.
Reading your bio clearly shows why you have this POV. Keep on hustling, someday maybe it’ll be true!
Soon enough, convergent-rendered Val Kilmer is stumble-crawling through an extensive duct system containing numerous big city bank branches, relentlessly harried by a xenomorph Al Pacino.
We haven't seen the license yet so might be prudent to hold off on the panic until then.
"How to Train Your Dragon: The Hidden World" had around a couple dozen people in the art department, around 3 dozen in the sound department, over 150 in visual effects, around 100 in animation, around 150 in music, and another 150 additional crew.
I'd expect paying so many people is what makes these movies expensive and beyond the reach of smaller less well funded studios.
Pure inspiration?
All the above?
Nice touch on Apache license, really shows they're paying attention to open source community
I used to joke that F5 owes me a job and 2 month's salary for all the free QA we did for them back when their session affinity code was just a checkbox on the PR material. If they were more open that might have played out a bit differently.
While it’s true that Lattner worked for Apple while he created Clang and that his personal contributions were probably biased, it’s always been an outside of Apple project with independent contributors.
>One of Clang's main goals is to provide a library-based architecture, so that the compiler could interoperate with other tools that interact with source code, such as integrated development environments (IDE). In contrast, GCC works in a compile-link-debug workflow; integrating it with other tools is not always easy. For instance, GCC uses a step called fold that is key to the overall compile process, which has the side effect of translating the code tree into a form that looks unlike the original source code. If an error is found during or after the fold step, it can be difficult to translate that back into one location in the original source.
https://en.wikipedia.org/wiki/Clang#Background
Lattner gave a tech talk at Google introducing LLVM 2.0 and explaining some of Apple's rational for creating Clang.
> While it’s true that Lattner worked for Apple while he created Clang and that his personal contributions were probably biased
But it wasn’t silo’d into Apple and then released to the open source field, it was created as an open source project under the LLVM Developer Group from it’s inception.
So yes, his contributions and goals were to make it targeted for Apple’s needs; but there were plenty of independent contributors that were focused on just making it a good C/C++ compiler front-end for LLVM and were more focused on making it a general competitor to GCC.
It would be more accurate to refer to is as an Apple sponsored project.
I could explain it but I don't have the energy to respond to a gotcha today. This has nothing to do with the conversation in the grandparent or the whole thread really.
Just don’t spread your mistruths.
It sure is. Especially when you nerdsnipe yourself and try to drag a conversation with you.
> Just don’t spread your mistruths.
You are very invested in pretty much nothing at all. What mistruths am I spreading? That Apple appreciated people fixing bugs that affected them? Oh. No.
Sorry am I missing some context on Apache License? Considering HN has been pro GPL / APGL for quite some time.
A company that doesn't make their digital art tools available to artists learning to be artists risks stalling the recruitment pipeline because all other things being equal, artists will prefer to work at a house already using tooling they know rather than be forced to learn some other thing that does something they can already do with another toolchain.
Back when I worked at Pixar on the RenderMan team, I was the one who implemented the cpuTime and sampleCount AOVs (i.e., passes) [0] in RenderMan RIS as well as some of the other diagnostic AOVs. And SPI's Arnold also acquired a pair of equivalent AOVs at some point [1]. The cpuTime AOV wasn't exactly novel then either, as I recall; I think there'd been something like it back in the old REYES days.
I'd also contributed a bit to the render time heatmap and added the sample count heatmap in the XML stats files [2]. Those are more thumbnail resolution than the AOVs, but on the other hand they were always-on for all generated stats files. The also displayed with nice little histograms.
One of the things that I learned in my years at Pixar was that rich diagnostics, statistics, or metrics are one of those key things that separates production renderers from other renderers. Pipeline folks, render wranglers, and TDs thrive on them.
[0] https://rmanwiki.pixar.com/display/REN24/Arbitrary+Output+Va...
[1] http://fundza.com/tishela/PDF/TransitionToRayTracingAtSony.p...
https://docs.blender.org/manual/en/latest/render/layers/pass...
I also wonder now how much work it was to disentangle it from their infrastructure? A lot of internal studio tools and libraries tend not to work outside of their studio's intranet -- not so much from direct copy protection, but more just because of heavy assumptions about filesystems and servers.
Definitely. I think that "you don't know until you profile" applies just as much to optimizing renders as it does to optimizing code.
> Of course this was pre-pathtracing, so it showed up differently.
Yep, I remember those days of distribution ray tracing. There were all sorts of tricks to keep the ray trees under control, like lowering the number of glossy or shadow samples the deeper you were in the tree. I don't miss them. Pathtracing made things so much easier to tune.
At the same time however, does their renderer give them that many advantages? As someone who works on a (sort of) competing proprietary renderer, it's a lot of work and effort to do it and support it, and maybe they want to build a community around that from smaller studios and compete with Renderman a bit for mindshare?
But there's a huge advantage to being the first high-end renderer to go open source, so lots of small studios will adopt it if it's as solid as it looks.
It'll become available in Blender.
Maintenance will then get extended to the community, including (most likely) support for alternate shading schemes (MaterialX/Lama has decent momentum right now, converging with Sony's OpenShadingLanguage).
Apache 2.0 is an ideal license for this kind of project.
Studios don't really have idle hours on their render farms.
EDIT: I thought you were replying to my comment about the Lucas Presidio Campus
When I was on the design team for the Lucas Presidio Campus, we were collapsing all the various divisions to the new campus, aside from Skywalker...
I was the LV/Datacenter/network designer for the project.
One of the early design requirements was to 1) have fiber to the desktop and 2) have every machine on an interconnect matrix which brought the edge desktop connections as close to the core as possible such that every workstation would become a part of the ender cluster when idle/at night/on-demand etc...
This didnt happen but it was a goal back in ~2004
--
I recall them telling us when Jobs came to visit and they had the "Death Star" <-- render cluster they were working on... Lucas apparently saw no future and sold it to Jobs for ~$10mm?? and thats where Pixar got it start.
That was an amazing fun project to work on.
---
Oh - and during the design sessions, Cisco borked the design review for their (joke of a response) to the RFP, the then CIO for LucasArts in this design meeting with ILM, Lucas, Cisco others -- he says to the entire team in dead seriousness
"When can I have power-over-fiber to the desktop" with a look like "Ha! GotchA!
And everyone just blinked..
They also weren't really set up for the kind of strict isolation and security that offering resources to external parties would require.
Translation: we think making this open source will bring in significant outside contribution so we can get a better renderer with less investment ourselves while not having to pay external providers
Open source is a good strategy for the runners up trying to compete, you get the good will from giving something away in exchange for other people improving your product for free.
MoonRay likely has the best volume rendering in the world, certainly that's open source.
Only Weta's internal renderer is in the same league.
That fact might have allowed them to take short-cuts.
How do you know what Manuka's capable of? :)
They open something up, let people use it, and hope that some people might contribute back. Cool, sounds great to me. They aren't forcing me to contribute back to the project. Seems pretty win-win.
You're not going to get a free work force just by open sourcing your very context-specific code. Most companies/people learned this ~22 years ago.
If every animation studio started using MoonRay because of this, they'd have won.
Surely the win for DreamWorks is the competitive advantage of having aspiring creators be familiar with their pipeline. If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. (The only way this isn't true is if you subscribe to the belief that more MoonRay use → more MoonRay experts → more DreamWorks movies → more profit. That's probably not sound. This probably won't impact how many movies they put out, just how good they look (after benefiting from open source work) and how easy it is to hire for the movies that they put out.)
Other studios could also contribute improvements, which the original studio could benefit from. That's one of my motivations for open-source: I'm selfish and I enjoy the benefits from the efforts of others making my software better, should they find it useful.
(Note that this could mean that if you thought your software was putting you at an advantage, you might not want to open-source it, but you could also believe that even if it is better, your competitors won't / can't switch to it).
But if the improvements to MoonRay drive down production costs for movies, the entire industry wins.
They're in the movie business after all - speed of production saves a lot of money.
Over the years they've bankrolled a lot of development around Linux on the desktop and have one of the most astonishing NFS infrastructures I've ever directly layed hands on.
They've talked about their love of open source over the years, especially at Red Hat summit.
(I spent quite a bit of time working with them and their former CTO is currently a teammate of mine).
If you can't, legally, nbd.
https://www.usenix.org/legacy/event/lisa10/tech/slides/kamat...
Now, switching gears to talk about that presentation. ;)
I'd call out a few things.
1) Note on slide 17 the call out to _multiple sites_. Yup. You got it. There is a single view of the file store. At a practical level to the end user the sync time between sites is trivial to non-existent.
2) automount, automount, automount (Slide 25). The automount configuration is stored in LDAP providing a convenient way to correlate and sync data which is already integrated with the name service switch/NSS (man 5 nss).
3) Caching (Slides 22-24). This is where a lot of the magic happens. It's this cache tier which _presents_ as if it's a single file store.
One conceit I will make (which is glossed over a bit) is that the use of DNS and suffix searching is critical in a situation like this. It allows each site to use configuration management to push out relevant config files (e.g. /etc/resolv.conf) which can then present a search order like "site1.us.example.com us.example.com example.com". Thus, when you do a query for the unqualified name "ldap" (or even better "IN SRV _ldap._tcp") NSS will hand that off to the resolver which will dutifully check for the following (in order) until it gets a positive match or fails: ldap.site1.us.example.com, ldap.us.example.com ldap.example.com. This allows for a DNS based fall back as well as the added benefit of always hitting the local system (instead of a remote site).
Also, make sure you check out the stats on Slide 28.... 4 PB... in 2008... before they had to render a movie for each eye... (3D animation!)
I speculate that having this open source increases its popularity, which makes more likely that any given potential hire or VFX company will have experience with it, which makes it easier for DreamWorks to ramp up new hires or contract out to third party companies.
When I was at EA, we switched from a proprietary UI tool (which was quite well suited for consoles) to Flash (which wasn't) entirely because it eased staffing problems even though the technology itself was a worse fit. The best tool is often the one that people you can hire already know.
It might even still be in use at Cartoon Network, it's been a while since I last heard from my former co-worker in the dialup-delivered Flash cartoon mines who ended up as the animation director for Foster's and Titans but he sure was still using an old version of Flash/Animate long after the browser plugin was retired.
There's probably a lot of lessons in Flash's status as a tool that did a ton of different things half-assedly.
I remember someone at EA talked to Adobe about getting access to the Flash (.fla) file format so that our UI tools could read it directly to export into the in game format instead of reading the already compiled and compressed .swf format. And they were like, "Uh... you really don't want to do that. It's like a memory dump of the entire application state. It's a giant mess. We're really sorry."
By all accounts, it seemed like the Flash codebase was a dumpster fire. But so many amazing little games and UIs were built using it.
"We are currently preparing the packages for public release, and MoonRay will be available for download from Github soon."
Since free software is so important to many people and one of the biggest questions about how the digital society should work, it's crucial that we explore the possibilities of releasing free software. I'm tired of the current situation where on one side we have a ton of software and on the other side people who can't use that software because of how unethical or impractical it is. We need to find something that works for everyone and this is a step in that direction.
This is just a continuation of that.