Artistic style transfer for videos [video] | Hacker News Reader

Artistic style transfer for videos [video]

youtube.com

51 pointsmmastrac11 comments

Paper here: http://arxiv.org/pdf/1604.08610v1.pdf, and possibly a better video here: https://www.youtube.com/watch?v=Khuj4ASldmU&feature=youtu.be

The video linked is much better IMO.

I can imagine making good use of techniques like these for some of my game ideas.

It would also enable films like A Scanner Darkly to be made with much smaller animation budgets. https://m.youtube.com/watch?v=eke5VnpNcNk

Like the "morph" fad of the early 90s and the Photoshop/Kai filter fads -- I feel like I'm just waiting for a Geico commercial or OK Go video to use this so we can call it done and move on.

This stuff kinda looks like crap and there seems to be a race to demo as much ugly, unfinished trippy stuff as possible. With the web, I wonder if people will burn out on it before agencies get a chance to mine it to sell cars and insurance.

There certainly is a higher bar now for this sort of publishing. Watching these amateurish videos -- poorly paced and awkwardly chopped together -- just doesn't fly in 2016.

I think you're incorrect - this is the future of animation.

Whilst these are pasted together samples - imagine the power of animating a piece, then being able to adjust the style at will. This takes a ton of manual work out of that, and quite magical to me.

This isn't animation, it's painting. And animation is composited in layers so it wouldn't need any of this (poorly done!) scene segmentation. You really think Hollywood hasn't done these sorts of obvious shader experiments and tossed them out because... well, they look like schlock and middle America doesn't get the art history reference anyway?

If you want to make something look like an oil painting dig out SIGGRAPH papers from 20 years ago. If you want to cop the style of a reference painting (fad!) then code it up properly inside Blender and render something. Even if artists decide to use this approach, this isn't the way the technology will be deployed in animation.

How about dynamic games or real-time augmented reality that paints your world in any style? There's countless applications beyond what an artist would want tweak by hand

Surely one could use this to previsualize stylistic changes before committing to repainting by hand?

That really isn't the way it works. Hollywood spends millions developing a look. If they want to see a mockup, they spend the bucks, because the magic really does come from the human element. If anything, seeing a robot do it poorly turns off producers. It's anti-inspirational. It implies something is either cliche or about to be commodified. That's the last thing they want before embarking on a multi-year, couple hundred million dollar artistic endeavor -- whether movie or game.

There are tech teams inside these companies (of course) and they do take input from academia (Gollum's skin...) but, having been there, it most certainly doesn't look like this.

Very strange to see this here. I was personally blown away. Did you watch the right video?

And even if you dismiss the quality, there's still a huge opportunity to use this to inspire "real" art. It's bigger than a fad.

Morphing hasn't gone anywhere, you've seen it thousands of times since the "fad" of the 90's and not noticed it. Similarly, Kai's work from that era was groundbreaking in terms of UI and visual possibilities and has inspired countless people to further challenge our preconceptions of seeing. Novelty fades rapidly, but to dismiss techniques entirely on that basis seems questionable. I remember the shader work mentioned from SIGGRAPH in the 90's. I loved that stuff, still think its great. Neural visualization isn't that. Yeah, these videos are amateurish - they're exuberant tech demos of possibilities that are at once unique and banal. I'd say the same of any technology

Reply on news.ycombinator.com