Technologists have produced a 3D-printed painting in the style of Rembrandt | Hacker News Reader

Technologists have produced a 3D-printed painting in the style of Rembrandt

bbc.com

127 pointsmdturnerphys62 comments

If it were a -human- painting a fake Rembrandt, no matter how perfect an imitation, it would be called "a very very good imitation of Rembrandt" and nobody would claim that they'd in any sense brought Rembrandt back to life. Calling a very good forged Rembrandt a "new Rembrandt" would be pretty silly.

Calling one generated by a computer a "new Rembrandt" seems just as silly. Just because it was composed by an algorithm doesn't make it Rembrandtier than one painted by an expert forger.

Pretty cool as a technical achievement, though.

Human artist would most likely inject their own take into "imitation" art, but if the computer's art was purely derived from an artist past work, it's only inspired by the artist work; this is in theory, even the code to enable this would most likely have biases that would result in artifacts in the "new" art.

From http://www.catb.org/jargon/html/koans.html#id3141241

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

“What are you doing?”, asked Minsky.

“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.

“Why is the net wired randomly?”, asked Minsky.

“I do not want it to have any preconceptions of how to play”, Sussman said.

Minsky then shut his eyes.

“Why do you close your eyes?”, Sussman asked his teacher.

“So that the room will be empty.”

At that moment, Sussman was enlightened.

Yes, I know, and agree.

Ugh. I see the point in general, but the eye closing is a really broken analogy, and there is a difference between a bias and a preconception.

Yes, the difference between bias and preconception is exactly what, or are you being bias?

Seriously, lookup the meaning of the terms and see if what they mean are what you believe them to mean.

A preconception is an idea. A bias is much more general. If you're putting in a table, and you don't make it quite flat, you have biased the table, but you have not instilled preconceptions in the table.

It's not the case that randomly-generated weights will not have biases. The point is that if you generate weights randomly the computer will still have biases, you just don't know what they will be.

Only if the training set was the artists thoughts and feelings. And the model was his nervous system and body mechanics. The images alone are not enough data to truly mimick the next painting he would have produced. It's an average of the existing work.

Yes, it's more accurate to say that they have an AI forger.

Maybe some money in that ;)

They just have to teach it to sign the paintings ;)

If anyone hasn't see F for Fake, I highly recommend it.

Probably relevant to the discussion: https://en.wikipedia.org/wiki/Tim's_Vermeer

Computers are our new babies. We applause and cheer for anything trivial they come up with.

I think it might just be a simple as interpreting your work in the most significant way increases exposure and potential grant money.

The problem is when parents start seeing baby's poop splatters as artistic expression. The equivalent with computers would be to call bugs features?

I think the goal of the project was to make a painting that is statistically indistinguishable from his other paintings. Another painter painting in his style would still inject some of their own style and technique into it. If you looked closely or took statistics from two painters trying to mimic the same style, you would be able to distinguish them.

Here they went to great lengths to try to make it very accurate. Everything from the shape of eyes and faces and the patterns of brush strokes.

Statistically indistinguishable by what measure? One of many we could come up with, no doubt. If an expert were to look closely at this, perhaps they would also distinguish them, by something the statistics didn't quantify.

An expert forger would be trying to make a painting that is literally indistinguishable from an actual Rembrandt, which is much harder. The GP's point was that something that can fool an algorithm is no more deserving the label "new Rembrandt" than something that can fool a human.

By what measure? The measure of as many experts as you can find.

Copying a specific painting is a very different thing. It's hard for entirely mechanical and chemical reasons. Trying to make a new painting that blends in is much more interesting.

I don't mean copying a specific painting, but creating something that looks to human experts like a previously undiscovered Rembrandt. This would be a lot more impressive than creating something that fools a particular statistical algorithm.

What has suggested to you that the goal of this program is not creating something that looks to human experts like a previously undiscovered Rembrandt? (With emphasis on 'looks'; they're not trying to trick forensics)

They don't seem to have asked a lot of human experts for input. I think if they had done, the finished article would look different.

In fact it's nearly impossible to nail down provenance of any painting without an historical paper trail.

Auction houses regularly pay academics to make statements about provenance, but the statements are always questionable if there's no definitive record of sales and ownership.

The goal is something that looks like a real Rembrandt. Not something that has a convincing paper trail, or something that correctly carbon dates. The point is not to trick anyone, but to ask the question "based solely on your knowledge of how Rembrandt paints, can you tell if this is one of his?"

Check out Orson Welles "F is for Fake" - illuminating film about authenticity and value.

We changed from the baity title to the typically more accurate subtitle.

Very true. Only Rembrandt could paint a "new Rembrandt". This is just a very good imitation - not that that makes it any less impressive as a technical achievement.

Well, not entirely. Imagine a perfect replica of Rembrandt's brain connected to an artificial body, a painting produced by this machine would be a Rembrandt.

Now imagine reducing the correlation between the artificial brain and Rembrandt's brain until you reach the level of correlation to that of the machine described in the article. There will be a precise point where the paintings produced by the device will stop being Rembrandt's.

That is not to say that this machine is producing Rembrandt's. I am only saying that your reasoning is too simplistic.

Not sure why you're getting downvotes. Sure, the argument comes off as sort of sophomoric in the face of the indisputable fact that there was an actual Rembrandt and this painting was produced neither by him nor by a low-fidelity replica of his brain; but I think it's interesting to think about. If we wanted to get real sophomoric, we'd apply the Ship of Theseus/Heraclitus/Lucretius line of thought to whether even a Rembrandt is a Rembrandt: is what I made yesterday made equally by the me of today? Cross-pollinating the brain-replication discussion with the multiverse discussion is pretty wild to think about: what is a copy of a dynamic thing?

Thanks, for your reply. I think I'm being down voted because the idea is too general for most people to understand, also they get caught up in the emotional back patting that goes on whenever people get a chance to show their allegiance to the "group think". I'm getting pretty sick of being correct and yet ignored. As a younger man I thought I was just stupid and wrong, that was bad. Now I am older I realise that most other people are wrong and that I am usually right, I am not sure which is worse.

Have you misunderstood the meaning of sophomoric? You are describing your own interesting philosophical thoughts as sophomoric. Is this because people have told you that they are? Have you considered the possibility that the complexity and breadth of your thoughts in this area make them feel inadequate?

I am sensing this odd meme developing, which seems to be rampant in the singularity crowd, that a computer imitating a person accurately enough to deceive another human is equivalent to the person being imitated. So much so that it is even considered to have a similar value to that person. As in, "oh good, we have the computer now and we don't need to care that Rembrant (or whomever is being mimicked) is dead anymore".

It seems more that we now have a snapshot of Rembrant as he was when he died. Assuming that the man were immortal somehow, his style would continue to progress, each painting building upon the last. If this algorithm were to do that, it'd just be deviating from what we now consider a Rembrant. I don't think we'd appreciate when an algorithm spitting out pictures akin to Picassos Rose Period suddenly decided that Cubism was more his thing.

The computer didn't learn how to paint or even compose or even choose subjects like Rembrandt did. With the help of a team of experts, it assembled premade parts of Rembrandt's earlier paintings into a fancy collage under tight constraints. The heightmap algorithm was probably more interesting than the one that made the picture.

I will admit this has the same freaky uncanny valley eyes as a Rembrandt painting. So, if they can leave the program alone and produce and endless stream of these things it is different than a single physical imitation.

However, I suspect this was highly supervised, and curated aka they picked the best example. So, it's much closer to a painter using a new brush type than an independent thing.

I feel like it's misleading of them to say that a computer painted the picture. That makes it sound like all works of Rembrandt have been fed into a Machine Learning algorithm and finally the computer spit out this piece.

After having watched the YouTube video, I fell like actually the picture was more or less photoshopped together from existing Rembrandt paintings. The role of the algorithms was "only" to generate the most average / representative of his works.

I'm guessing recent progress in texture synthesis [1] and content generation [2] using deep convolutional nets made this possible. It would be interesting to see what was actually fed into the generating system. That would be a piece of art truly speaking magnitudes.

[1] http://arxiv.org/abs/1505.07376

[2] http://arxiv.org/abs/1511.06434

Those techniques aren't capable of generating images at high resolution that even remotely look as crisp as what was done here.

You can read on their website [1] what they did, and seems that it involved a lot of manual labour still.

[1] https://www.nextrembrandt.com/

I used to paint old masters portraits as a hobby (including Rembrandt). I think it's a bit disingenuous to describe this as 'painting'. On top of ma2rten's criticism that it basically photoshopped previous examples to create a most representative work, I'd add that 3D printing with texture is not painting. Rembrandt, in particular, was known for painting with countless layers of semi transparent glazes to give his paintings a sense of depth and an ethereal glow. He'd also mix his pigments with bindings aside from pure linseed oil, sometimes adding chalk or glass. 3D printing the final result with none of that process makes for a very limited reproduction.

Also, I'll add that I think it's very impressive, but I think they're overselling it here.

The 3D printed element is the least interesting part of the process because it's literally superficial. There's no reason you couldn't set up a robotic brush painter next to a robotic pigment factory to recreate all of the above.

However - what's interesting to me is that the only way to make this work was by defining the features that were going to be included, analysing and averaging them separately, and gluing them together to create a whole.

This wasn't a "Throw everything into an RNN and see if something useful falls out" project. It deliberately separated specific target features.

That works well for an artist with very consistent and narrow clothing, lighting, and background choices, and limited model poses.

It might not work so well for artists who included a much wider range of imagery - which is probably most of them.

I'm also getting the feeling that they are being dishonest about the techniques they've used.

It's also very misleading to say that it's in the style of Rembrandt when there are so many factors that make his works unique missing from this painting.

It's an impressive achievement. But compare the computer generated image[1] with a real Rembrandt of a similar subject at high resolution.[2] Compare the detail in the neck ruff. Look at the iris of the eyes. (Why do Rembrandt's people have no eyelashes?)

[1] http://ichef-1.bbci.co.uk/news/950/cpsprodpb/4B28/production... [2] https://upload.wikimedia.org/wikipedia/commons/3/3e/Johannes...

I think your first link is not high-res enough. I don't know why but the website breaks the image into tiles: www.nextrembrandt.com/static/img/painting/painting_XX.jpg where interesting values for XX are 17, 18, 22, 23, 24, 28, 29, 33. (Full range is 01-40 inclusive.)

youtube video showing the process https://www.youtube.com/watch?v=IuygOYZ1Ngo

Thank you. A lot more information in that video than in the article. Looks like it was half human and half algorithm. Still very interesting way of making imitation paintings.

They said they incorporated the height map from the original.

How did they obtain them? Do museums/galleries/owners let scientists or engineers study the layers of paint of the physical paintings?

Yes they let scientists study the layers.

You can create hight maps by casting light from each side of the painting and using the shadows for your hight map calculations.

Edit: this (old) tutorial shows how you can do it yourself: http://zarria.net/nrmphoto/nrmphoto.html

I think a flat bed scanner is great for creating those maps. Just scan your peace 4 times each time rotating it 90 degrees.

Off-topic, but how do you guys feel when pretending to code for the camera?

elephants do really good abstract impressionism as long as you tell them when to stop.

http://www.getty.edu/education/teacherartexchange/archive/No...

Not to say that rembrandt = abstract impressionism. Still, the humans tagging features is kind of a copout. The 'grain of salt' for these projects is to look at the variation of outputs when you tweak the model inputs. In this case it's going to be a head with oil paint texture 9 times out of 10 (and the tenth will be cecilia jimenez 'I fix').

It won't be long until a computer can 'participate' in a genre and really contribute to art. This project, while cool, isn't there yet.

Take a look at these:

* Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis (http://arxiv.org/abs/1601.04589)

* A Neural Algorithm of Artistic Style (http://arxiv.org/abs/1508.06576)

They do much more than photoshopping. It's actual synthesis of new images in the style of a desired painter.

neural blending side-by-side on p6 of the first paper is really cool

If it was that easy, why didn't they produce more paintings?

I have the feeling that a lot of human work still went into this painting (besides writing the algorithm).

I'm interested in how they 3d printed this picture. Also how they colored it.

I'm curious why I cannot get hi res scans of old masterpieces I can use as wallpaper on my monitor. Doing image searches only turns up low res versions.

I'd guess that licensing such scans for quality reproductions is a source of revenue for museums.

I can't even find hi res scans of works held by the Library of Congress.

Check out: https://www.rijksmuseum.nl/en/search?s=objecttype&p=1&ps=12&...

They require you to make an account, but they have tens of thousands of public domain photographs of many of the artworks in their collection.

That looks awesome! Thanks!

It would be cool if we could upload a photo and then have it oil painted by a computer in Rembrandt style.

Yes, and I imagine it would be a short step to having e.g. your portrait painted in the style of an artist of your choice for a fraction of the cost. I wonder could there be any intellectual property issues arising from this down the line?

Check out deepart.io -- Can do this today

You mean Deep Style-style ?

https://www.reddit.com/r/deepstyle/

Everybody keep calm - the Sublime Text 2 instance used to code in the video was registered.

Reply on news.ycombinator.com