Netflix Japan announced that it used AI-generated art in a new short anime film | Hacker News Reader

Netflix Japan announced that it used AI-generated art in a new short anime film

creativebloq.com

39 pointsamrrs62 comments

I am very disappointed in the reporting and discussion on this specific occurrence.

In the trailer Netflix released, you can SEE what they did with AI. Skip to 3:07. https://twitter.com/NetflixJP/status/1620357552025538561?ref...

First they input a pretty detailed hand drawn sketch. The AI turned the sketch into a rough approximation of detailed art, and then they iterated it a second time. Then they painted over the top, by hand, recompositing the elements in the process. (You can see the moon move.)

What happened here is somewhere in the realm of previz and rotoscoping. The AI fleshed out details in the previz, and then it was painted over. That is a perfect application of technology, without cutting out the humanity.

And yet the whole story is about "controversy." There is no analysis anywhere of the process. This feels a little more like a "proofreaders mad at spell check" moment in history. And it would appear that nobody participating in the discussion, watched the making of. What I see is a powerful tool that helped augment an artist. Visual effects is already a stressful race to the bottom industry. Tools and technology that make life easier and less stressful for creators should be applauded. This kind of democratization gives more people, (who cant afford teams) the ability to bring their ideas, dreams, visions, and creativity to life. There is plenty to be fearful of with the coming "everything is AI generated nonsense apocalypse, nothing is original anymore" but this isnt it.

The AI (+Human) part of the credits is weird tho.

I had to look up what opinion of rotoscoping was at the time. If anyone has other sources, Id love to see them.

"After producing three shorts that became massive hits, Fleischer would go on to create a whole series of shorts called Out of the Inkwell. He would even go on to make his own animation company called Fleischer Studios, which would in turn give the world characters like Betty Boop and Popeye. Walt Disney eventually adopted Fleischer’s Rotoscoping technique for Snow White and the Seven Dwarves (and other films thereafter) after the exclusivity patent expired in 1934. Disney would even have the cast act out the scenes required by the animators so they could be used for reference instead of using their own facial expressions or movements."

"Despite Rotoscoping making animation much easier, Disney’s animators were not fond of the new style and referred to it as a crutch. When they did have to use the technique, Disney’s crew would often only use it as a minor basis for their work and often exaggerated proportions and heavily elaborated expressions to go beyond the scope. What the animators couldn’t argue however, was that Rotoscoping (especially for Snow White) allowed them to have consistency with the animation that would have been impossible otherwise, thanks to the live-action stand-ins."

https://www.cgmagonline.com/articles/features/history-art-ro...

The cat's out of the bag on AI art. There's gonna be regular drama about it for the next few years and then it will just become part of life.

Same with other applications of AI over time.

It’s funny… the more I look at it, the more it seems that the prevailing sentiment regarding AI was that white-collar workers thought it would take blue-collar jobs… and now it is looking closer to the reverse.

I remember ~12 years ago coming across the concept of "the easy things are hard, the hard things are easy" (in the context of a robotics class).

The example I remember from the time was the relative ease of translating the decision trees of pharmacists (who in the US today earn a doctorate degree before practicing) into code versus catching a softly lobbed tennis ball with a robotic arm. A kid in elementary school can do the second task using their own arm without giving it any thought, but creating a hardware/software system to track the moving object, estimate trajectory, extrapolate future location, decide which rotations/translations need to be applied about the joints of the shoulder/arm/elbow/wrist to move your hand into the right position to make the catch, and then to actuate your hand closed at the right time with the right pressure is quite involved.

I think the class component here is underrated for sure.

Technology has done a lot more do destroy blue-collar jobs over the last century, and so white-collar people have been pretty chill about it. But I think you're right that we'll see a change around that.

ChatGPT, for example, is pretty bad at creating excellent prose with reliable information. But it's extremely good at creating a bullshit memo that nobody will really read or act upon. And I think there are a lot of people out there with bullshit jobs that will find themselves at risk.

> Technology has done a lot more do destroy blue-collar jobs over the last century

It'd be interesting to know if this is true. My gut says it might be the opposite. Maybe many "unskilled" jobs are moved out of the U.S. for example, but that's attributable to geopolitics not technology.

Here's some useful data for you: https://www.bls.gov/opub/ted/2016/employment-by-industry-191...

But note that this is only non-farm employment. Currently about 1% of the US's population works in agriculture. At around the same time, 31% of people did farm work: https://www.bls.gov/opub/mlr/2016/article/the-life-of-americ...

I'll note that offshoring is also driven by technological change. In particular, cheap global shipping and communication. It's true a bunch of manufacturing jobs went to places with cheaper labor. But if those jobs had stayed here, there would still be a lot less of them because our higher labor prices make it more worthwhile to spend on factory automation.

Another area where technology makes a big difference is in household labor. In 1910, 14.5% people were engaged in either domestic or personal service. Now it's 1%. We owe a lot of that to things like the washing machine and the dishwasher.

But what happened to the people who had those blue-collar jobs? Surely they didn't all die. Did they become white-collar workers? If so, wouldn't that be a good thing?

Sure! So far we have always found things for people to do. That's probably good.

But maybe not always. We are also seeing rising inequality and increased capital concentration. So there's a risk that we might head in the direction of, say, preindustrial England, where to survive people have to labor to make the rich happy.

There's also the phenomenon of bullshit jobs: https://en.wikipedia.org/wiki/Bullshit_Jobs

As the relationship between labor and value becomes less and less direct, it becomes easier for waste to creep in. E.g., a manger who over-hires not because the new people are needed to do anything, but because their status rises with their headcount.

So hopefully as all of these white-collar workers are forced to find new employment, we will find socially positive things for them to do. But I don't think it's guaranteed.

I've tried to get people to care about professional drivers over the years without success. Now that it's their jobs, they care.

Millions of Professional Drivers Will Be Replaced by Self-Driving Vehicles

https://www.nbcnews.com/business/autos/millions-professional...

I don't see why we need to lock in current professions. There weren't millions of professional drivers before there were cars. Jobs change over time, we shouldn't try to preserve them just to run a jobs program. If we want to make sure everybody has income, just give everybody income.

I don't think it's just about jobs. It's alright when the machines are driving the cars or running the factories, but when the machines are also painting the pictures, writing the poetry, composing the symphonies, what exactly are the humans for?

> but when the machines are also painting the pictures, writing the poetry, composing the symphonies, what exactly are the humans for?

Consuming. The machines will create culture, humans will consume it.

I'd hope, we'd be there more to maybe pursue scientific/research endeavors. Sure the ai can do this too, or enhance it - but imagine if every living human being were a scientist, or engineer working on solving the nature of the universe, FTL drives, fusion, expansion of the human race into space, etc?

No more wars, etc - just intellectuals working in intellectual fields pursuing whatever their curiosity takes them.

Is animation a white-collar job?

In the context of the Japanese animation industry, I don't think so. Animators there are worked to the bone for a pittance and are usually contractors, so they don't even enjoy the benefits a full-time employee would.

Either way I think the difference between e.g. driving and animating is the passion component. For the most part, I don't think anybody really enjoys working as a bus driver for example… in many ways it's a brutal and thankless job. By contrast people who work in animation tend to really, really love animation and pour themselves into their craft.

It certainly creates problems for drivers to have their job automated away, but for the most part nobody is being denied of earning a living with their passion in the process. By replacing artists with AI you're eliminating the possibility for many artists to produce art because they're going to have to find some other kind of work which will commandeer the majority of their time and energy instead, leaving little for creative pursuits.

Ideally it's the menial and mindless things that get automated, so long as the displaced workers are provided for. It's squandering of human potential on a massive scale for people to have to do those things just to exist.

Makes sense – physical labor is about robotics + AI, whereas mental labor is just the AI.

Actually a lot of physical labor didn't even need AI. A lot of things are just repetitive motion, which was way easier, now we have ai, ai + better facts + robots and add the ability to ai to create/iterate better algorithms for their own ai, and you basically get us to AGI and the singularity.

Some reality TV shows already feel AI generated. I don’t think people will notice.

And because the passage of time implies progress, this will be inherently better than what came before. /s

It will be significantly better for everyone who consumes media.

Imagine if you can get movies custom made to your own personal preferences?

So literally generic content replacing art? Just tell the machine what you expect and like and then your expectations can be satisfied on-demand! Sounds like cultural oblivion and I will hold my tongue on the quality of your aesthetic judgment.

No literally the opposite. Instead of having to create blockbusters that appeal to everybody and therefore none at all, we can create stuff that appeal to smaller audiences and try new things.

If you are the type of person who complains about the fifth transformer movie, or ask why Hollywood doesn't want to take a try on new things, the cheaper a production can be made, the better the ods that somebody can make it.

In a world where you need 200 million dollars to create a game, only studios of that size can make the games, and we never get Kerbal Space Program, Rimworld or Minecraft.

As a quick example of movies: Pearl Harbor would have been a great war movie if it had toned down the rom-com part by about 80%, but then it would have lost a big part of the audience. On the other hand, it would have been a great romantic movie if it had turned down the war content by about 80%, but then it would have lost the other part of the audience.

As it is, it is too long, and not a good movie in the first place.

Now it appears to me from your comment that you want to find a way to hate this no matter the merits, but honestly most people will want to watch things they like, and then be delighted and surprised. Game of Thrones did as well as it did because of the plot twists, not because it was predictable. The CGI and general budget went up for the last seasons and they are universally looked upon as the worst ones.

As a challenge for you: which movie, that you consider art, sold at least 10 million tickets in its original theater run?

I think this over values a lot of what is produced for tv and streaming, which I could at best describe as filler. A ton of cultural production has little to no artistic value, but exists to be basically slightly different and fresher than last years iteration. Why watch last years hallmark or lifetime movie when this years is the same one but refilmed? The same can be said for at least half of the Netflix/CBS output.

I don’t disagree about the artistic merit or lack thereof of most “content”. Will AI produced content be worse than 98% of what’s on Netflix? Probably not! But this will drastically change the economics of content creation, such that something that has the chance to be actual art will be dramatically more expensive than that which can benignly occupy a user’s attention as they alternate between staring at a screen and flipping through their phone. Give it a generation, or even just a decade (look at how rapidly audience tastes have degenerated since 2014 or so) and no one will be able to conceive of “content” being anything else. Who is going to sign on to fund popular entertainment with artistic ambition, eg The Fabelmans, or art house fare, eg Tar, in such a world? Such a future, a world without popular or even middle brow art, is dystopian to its core (and indeed it may already be our present) which is why I fail to conceal my contempt for the aesthetic values of anyone who would eagerly embrace it.

Does “art” occupy much of the collective attention to begin with? If we look at best picture nominees by box office gross, something like 6/10 weren’t seen by anyone to begin with, and that was an unusually low percentage historically.

Exactly.

You'll get one Kubrick/Spielberg/George Lucas every decade. But the vast majority of what's produced is another dumb Star Wars movie or another Marvel cash grab.

You are looking only at mass distributed titles

You should google the definition of generic.

> Sounds terrible and I will hold my tongue on the quality of your aesthetic judgment.

Thank god! Your opinion would devastate me!

Anyway, I'm super excited to be able to prompt an AI to "create another kubrick movie in the modern era" or "make a seinfeld episode about living under covid" or something.

A better large language model would generate a script 100x better than any of the current hollywood writers (who do a pretty piss poor job as it is...)

I was referring to the genre fiction versus literary fiction distinction, something which would be very evident if your understanding of the term wasn’t limited to the first hit of a google search.

You're either changing your argument or you suck at writing. Which is it?

Obviously everyone knows what the genre of fiction means. But that wasn't what you said in the slightest lmfao.

No, half the joy for most people who socialise is sharing the experience with others. I also couldn't care less if you were to show me a chess match and tell me it was Stockfish vs AlphaZero but would if it were Hikaru vs Carlsen simply because I am interested in the work of others much more than an algorithm even if it were to be as good.

> half the joy for most people

You should talk to the average Netflix user and see if they can name any of the writers/director/producers.

They might be able to name a couple of the main actors/actresses.

No one cares about the producers/director for the vast majority of media.

Some small % of people care (who are interested in the production or whatever) and I'm sure they'll continue to watch purely human generated content.

The vast majority of people will move on because AI generated media will be 100x better than the garbage scripts that hollywood writers put out.

I agree that there are a few directors who are incredibly high quality (your tarantino, kubrick, etc.) and people of those caliber will continue to be successful.

But yeah, I'd much rather have an AI generate me a movie than watch Disney's next cash grab star wars movie (that has almost comically bad writing).

It probably will be at least somewhat better, but not for this reason.

Better for who?

Probably a net improvement for humanity overall.

Animation has always been about minimizing cost and time while looking passable, and those are exactly AI's strengths.

Hot take: As someone who hardly ever watches TV/streaming, I’m glad the machine is destroying itself. People need to find a better way to live their lives than to come home from a shitty day job that wasted the first half of their day, and waste the next half watching dumb shit.

Lmfao in what universe is AI-generated media going to reduce the amount of movies/tv shows people consume?

It's going to do the exact opposite...

True, instead we should be submitting our crappy "hot takes" on the internet.

> To avoid paying artists? Not at all. It claims the move was to "help the anime industry" amid an ongoing labour shortage.

Once the labour shortage is resolved they'll go back to paying artists right?

Never let a good crisis go to waste

The monthly Netflix fee won't be dropping so straight into the profit margin it goes.

What's interesting is that between AI art generation and content platforms with monetization like (Youtube, etc.) - it could actually level the playing field for content creators creating actual high quality anime.

Currently large studios likely have the upper hand because they can have teams of artists on staff to handle the background art, intro sequences and the supporting HR and management overhead.

A truly talented artist with storytelling and a distinct visual art style would likely have to seek out such a studio to gain access to these capabilities.

Now they will be able to just create and share their content - focusing primarily on storytelling and determining the visual style as well as marketing.

Similar to how cost reductions in movie quality digital cameras has improved the short film, indie creators ability to match the quality of Hollywood films.

Netflix Japan cites the use of AI as a reason for the shortage of animator talent, but the cause is that wages are too low and the use of AI is not a fundamental solution, and this will hollow out the industry in the future because new people who are not yet "creative talent" will not be trained.

Incidentally, Rinna Corporation in the text is a Microsoft affiliate whose business is the chatbot 'Xiaona' developed by Microsoft Research Asia.

The problems with AI-generated art seem like a catch-22. AI is inferior to human artists because it's not creative because it only remixes past work. But if you as an artist are doing more than remixing past work, why would you be afraid of being replaced by something that can't do what you do? If what you do is genuinely valuable then the market will reflect that and using AI tools will lose companies money instead of making it.

An ATM can't do everything a bank teller can do, but it can replace 90% of them and halve the wages of the rest.

An ATM can't do everything a bank teller can do, but it can replace 90% of them and halve the wages of the rest.

What a fantastic example, because it's completely and utterly wrong.

https://www.aei.org/economics/what-atms-bank-tellers-rise-ro...

I’ve often wondered why banks keep needing to open new branches. We have online banking and ATMs. The one real time I needed to visit a bank branch in recent years was to open a bank account for my son and it was an absolute CHORE the way the bank handled new account creation.

I upvoted your post, because that's how I feel as well, but that still reads as: "Tell me that you haven't spoken to a senior citizen without telling me..."

It might be surprising to some, but old people actually also have needs and many of the modern solutions we create just do not work for them.

Elderly individuals also have needs and often struggle with modern solutions designed for a younger demographic.

My neighbor, who is elderly, frequently asks for help with his phone and struggles with the shift towards technology and the use of apps during the COVID-19 pandemic was especially painful time for him. He prefers to stick to traditional methods and is content with living a simple life, rather than adapting to new technological advancements.

He will keep going to physical bank branches until they close the last one in our city.

That’s fair. But presumably the people who become senior citizens as time marches forward will be more comfortable with tech than present day senior citizens. There would be a tipping point where less people need in person services.

There are people retiring in the next decade, who entered the workforce in 92, when the internet began to take off, or even earlier but they grokked it pretty quick because they worked in tech fields, or programming, etc.

I'm 43, and am a zennial, born analog, grew up digital. Most of the people I know who aren't technical, can do way more than the people their same age could 10 or 20 years ago. New tech pops up, they are still adopters, or I am anyways. I'm slowing down a little I guess (re: TikTok), but I still 'get' it, and could use it if I chose to.

My point is, I think we're just about already there. My wife's dad who never had a smartphone before, or a laptop, etc got one after her mom died in 2020, and he's using it like its always been attached at the hip.

and it also won't try to sell on me a mortgage application when it sees the balance in my bank account, and just finish my deposit or withdrawal at the same quick speed

bye humans.

One longer-term issue with this in the arts/coding space is that "remixing past work" is often the apprentice-level labor that's necessary on the road to originality and expertise. If we eviscerate the work available for entry-level artists and junior developers to practice on, we can expect much less practiced master-level talent in 10, 20, 50 years.

And it's not clear that there's a practical economic incentive to stymie this short-term optimization and its long-term negative consequences.

>it's not creative because it only remixes past work

I have a feeling that even everything human beings create derives only from their own experience.

Uniqueness can only exist in a bubble. Some tribe in Africa who's never been touched by the real world, may have unique stories - perhaps about constellations with different names than we've ever heard or some other weird concept. E.g. the Mayan Calendar was unique to the Mayans.

The more dispersed we become, the more global (cats out of the bag on that), the more we are just a hodgepodge of our experiences. Its literally impossible to write something that hasn't been written before, either fully, or just by mashing maybe 4 different stories together.

Even back in Shakespeare's day this was very true the majority of his work - and even the words he contributed to the English language were actually taken from Greek/Roman plays, and ideas and their language.

Art imitates life imitating art imitating life, etc.

Given how many animes don't have the budget to actually finish the anime, do the practical and economic thing.

Given how many anime don't even need a big budget because they're already a bit saturated (such as isekai), they should cheapen those types of anime and allocate the remaining resources to the higher quality ones (in terms of story/narrative).

Crediting a ai wrangler as "+ human" does seem like a cheeky slap in the entertainment industry.

Imagine being an artist in 2040. You wake up, hop onto the work tablet, and you are paid to upload creative inputs into the Machine Learning backplane, so that various automations can use automated metrics from audience testing to learn what the next big product is going to be. You can't prove part of you is in the next movie, but deep down, you hope you were. The Company gives you a digitally signed token for your Boost-Resume, a sort of C.V. for artists that indicates a statistically likelihood your art was involved with a successful financial venture. At .065 yours is impressive and almost guarantees you a spot at any hourly wage backplane creative input "Art Unit Engineer", your official title. At Thanksgivings and Christmases, you can impress small children by slowly and poorly recreating their favorite franchises, although you do it less and less as their optics are increasingly monitored by automation for various infringements. Fortunately, you won't get in trouble, per se - merely docked in your paycheck with a realtime license fee for your ketchup and napkin doodles.

Don't think that'll even exist in 2040.

Instead, it'll be more like...

User wakes up and opens up the Netflix App.

Netflix starts showing a bunch of AI generated movie trailers that are custom tailored to the user.

User swipes right/left on which movie trailer looks interesting to him.

If he swipes right, then Netflix starts generating the movie JIT as the person is watching it.

If the person gets bored at all, then he can go back and swipe right/left on some more trailers.

Eye tracking and heart rate monitor determine when the feed is losing your interest and adjusts accordingly.

If it is a choose your own adventure movie, it wouldn't even need swipable trailers. It could just detect what part of the screen you looked at and push the story that way.

Based. Unfathomably based.

Reply on news.ycombinator.com