So I've been playing with style transfer for about six months. I even combined it with another technique to actually get something that I liked enough to have printed on canvas (https://github.com/j2kun/art-21-logo)

And as much as I love the math, the computer science, and the tech behind it, I have to admit that the novelty wore off quick. Not only does a style-transferred image not have the power of the original, as another commenter said (we can debate about why), but the longer you look at it the more you see the ugly artefacts. I understand it's not perfect, and that there will be improvements, but to me that's what's primarily holding back style transfer. Not runtime or memory constraints, but whatever first principles need to change to get a global unity of style in the image.

Of course, if you introduce the human hand to fix the artefacts (which I think is great, and I know people are working on this), then style transfer can still be super useful. I can definitely see this becoming a new photoshop/illustrator integration.