Artbreeder – Extend Your Imagination with GANs
artbreeder.com
artbreeder.com
The goal was to make a tool that reflected the collaborative and explorative aspects of creativity. Also to make GAN's (and high-dimensional spaces more generally) accessible to everyone in a fun way.
Let me know if you have any questions :)
Maybe you should make it!
Check out the yearly resources.
There are other ways to do generative writing though, mostly with prompts.
Given that nVidia has a pretty clear license saying that their model and all derivates may only be used non-commercially (https://github.com/NVlabs/stylegan2/blob/cec605e0834de5404d5...) and Artbreeder is clearly commercial (https://artbreeder.com/pricing) but is using StyleGAN (https://artbreeder.com/i?k=755521d9fe07456286a06f06), do you feel that Artbreeder is intentionally violating nVidia's license?
For what it's worth, I fully support your project and I nearly launched something similar myself. But many people choose not to do projects such as yours due to licensing considerations, and I'm wondering in general how to reconcile the moral dilemma. On one hand, nVidia almost certainly doesn't care. On the other hand, they did say very clearly "thou shalt not use this model, or any derivative thereof, commercially."
(It's tempting to believe the license only covers the code, not the model. But the same license appears in their download drive: https://drive.google.com/drive/folders/1QHc-yF5C3DChRwSdZKcx...)
If you click "Edit Genes" (e.g. on the upper-right of https://artbreeder.com/i?k=755521d9fe07456286a06f06) you'll find a bunch of sliders. StyleGAN uses sliders like this to control various facial features. For any given StyleGAN model, you can find a slider for a given attribute as follows: Create a classifier for that attribute (for example, "blond hair" probability), generate 50,000 or so outputs from the model, classify each image, then plot a fitted line through all 50k latents. You now have a blonde hair slider for your specific model.
Ok, but sliders aren't unique to nVidia's licensed model. How do we check?
Suppose Artbreeder used a model that was trained from scratch. Since it has a random initialization, its latent space wouldn't be compatible at all with nVidia's FFHQ model. Sliders on Artbreeder wouldn't work with nVidia's model, and vice-versa. A "blond hair" slider for Artbreeder would result in nonsense when applied to nVidia's model.
Artbreeder has an API, through which you can get the latent vector of any given image. You can use this to extract Artbreeder's sliders as follows: Extract the latent vector for a given Artbreeder image; move a slider to the maximum setting; extract the latent vector for the result; subtract the two latent vectors. Presto, you now have the slider.
If you add the resulting slider values to nVidia's FFHQ model, you will find that they control the same facial features on both Artbreeder and nVidia's FFHQ model. They don't match exactly – Artbreeder appears to be fine-tuned from nVidia's base model – but it's very close. So it's clearly derivative, and thus falls under nVidia's "no commercial usage for derivative work" restriction.
First off, I am not a lawyer but have consulted them. A lot of this is a gray area and not well defined. As you mention, there is the code, the given models, models fine-tuned on those models, models trained from scratch, using styelgan reimplementations, etc. Even with the license being included there does not mean that the output of a model trained from scratch counts as derivative work. And technically I just sell things like upscaling, keeping images private, google sync etc and provide unlimited image creating for free (not sure if that would hold up in court but I think it's relevant). So, no I do not think I am violating the license. But some lawyers may feel differently.
Also, I try hard to cite everything involved. Google was thrilled by my use of the biggan model and how it showed off the model. With the amount of money I have spent on Nvidia cards, I hope they are happy too :)
and then image ownership is a whole other question. Everything in arbreeder is CC0 to keep things simple https://artbreeder.com/terms.pdf
Best, Joel
If so, was that a recent change? When I checked several months ago, your latent space sliders were compatible with nVidia's FFHQ model: https://news.ycombinator.com/item?id=23149739
The only explanation seemed to be that Artbreeder was fine-tuned from nVidia's released FFHQ model.
To rephrase the question: It seems like you were using nVidia's model (or a fine-tuned version) at some point in the past. At the time, you were also selling subscriptions via your pricing page. Do you just sort of ignore the legal implications and hope that it never becomes an issue?
(That's sometimes an effective strategy.)
I could make a document that slowly became more illegible after each new visitor, just for fun!
One bit of UX feedback - I played about with children, cross breeding and editing genes and never figured out a mental model for what interactions caused a derivative image to be previewed vs being used to update the current seed image.
Clicking on images often saved them, though I'm not sure what that means for me, the user. Do I need to save a child image in order to derive from it? Even if it's just an intermediate result I want to play with? When I breed images, I see the image on the left is updated, but it doesn't seem to update the seed image being used in the 'Children' tab, or the genes available for editting? I ended up saving and then re-opening images as a relatively safe workflow, but it would have been a lot clearer if there was more UI feedback showing me what's a preview and what's an input at any point.
All those questions were rhetorical btw, just talking through my user experience :)
In that vein, I've created a webapp [1] that runs entirely client-side using TensorFlow.js, and let's you generate videos of latent interpolation, images and tweak with parameters using a less-conventional generator trained on many different datasets.
best, joel
Jean Luc, is that you?
not needed. That's where. Better hope you have ownership of some shares in the companies that are generating these things!
> I can't stop thinking how making art is at anybody's fingertips and yet means so much less.
art has always meant something, but only to those who feel it. There's no intrinsic value, no utility to art cept by the eyes of the beholder.
I think we will also see that art is not entirely about communication, but also to a big deal simply created in the eye of the beholder.
I've had this vision that one day you may write a simple text and when done with it one will be able to choose a command and the same content of the text transforms to the style of Italo Calvino or Hemingway or GG Marquez or whatever you want. At the end of the vision I asked myself what it would mean. My answer was complete vulgarization of anything. This general trend we're heading to where anybody can do anything effortlessly reminds me of Arthur Clarke's Childhood's End. Great book btw.
Also, often my favorite part of artbreeder is when artists take what they save as the inspiration or building blocks for full works. It's really an inspiration tool, but saying inspiration-breeder is a mouthful.
More generally, I think computation can meaningfully augment human creativity by providing surprise and break us out of our loops.
Best, Joel
My theory is that because this is built be using pre-trained BigGAN models, they do not have access to the encoder to convert an image to the latent space, but instead only to the generator that converts the latent space to an image.
Accordingly, they can only remix parameters in latent space, but not locate the latent space that a real-world photo would reside at.
What do you mean? Click for example "create" -> "portrait" -> "upload".
I signed up and explored what the free account permits [3,4]; however, I have to say I feel the same about this project as I did about "NLP Tool for Technological Research" (30 days ago) [2]. Unless you can upload an image you're familiar with, something you've worked on, I just don't understand what I'm seeing.
Personally, I enjoy studying composition. I enjoy abstract works of Suprematists, Cubists, Futurists, Abstract Expressionists. All of which I think could be more or less reimagined with computational means.
So for my WOW, I want to see what Artbreeder does to an image I'm already familiar with, or how it can imitate other visual styles.
In the last case I really need to see the input, because if the results appear similar to the input there will be copyright issues. In one of my experiments [5] I used the mixed images feature. This provides a graph which gives the appearance of showing how the image was created. Then again, not seeing any code, I don't know for sure what else was used which may or may not be subject to copyright. I'm not saying the project should be open source, but maybe there is some detail that can be exposed, and used to recreate the effect with a different image. If you can do that, then QED the result is not cribbed from un-cited sources. It's just computational pixel-pushing (heck, is it?).
I did some graphic design work early in my career and when we "appropriated" an image to be part of a photoshop composition, the rule of thumb was always you should not be able to reference the source from what you sampled. Some people spoke about 5-10% of an image; however, if you were doing something with any significant visibility and didn't pay for the source, the first rule was make it unidentifiable.
I don't have any particular need for images right now. This is just my first impression based on my experience and what I can't get enough of--more abstract art, or something that takes an image I've studied, have intimate familiarity, and says, This will look much better upside down! haha
[1]: https://artbreeder.com/lineage?k=03b855cd8c852c56b338
[2]: https://news.ycombinator.com/item?id=22847429
[3]: https://artbreeder.com/lineage?k=ee55675001b83a64f71f