Hacker News Reader

Comment by johnfn | Hacker News Reader

Which one are you comparing against? I've tried hundreds of prompts between SD and DALL-E and get comparable results. Midjourney was lagging for a while, but the new --testp parameter is really remarkable, which, in my view, makes it superior not only to Stable Diffusion but also to DALL-E as well.

View on HN

An easy example of DALL-E superiority is its ability to combine two different concepts together.

For example, DALL-E performs extremely impressively on prompts in the format of “a still of homer Simpson in The Godfather” (replace character and movie as you wish). with the other two it’s a lot of misses

Yeah you're right that Stable Diffusion produces garbage for that prompt.

I'd love to see a site with lots of examples of the same prompt fed into various models, I assume someone has already made that.

> Yeah you're right that Stable Diffusion produces garbage for that prompt.

I dunno, I generated 20 images from that prompt locally and got three good ones[1].

https://imgur.com/a/rZ6wOEF

What? None of the people in these images are even remotely recognizable as Homer Simpson.

What would you count as a pass then? A literal rendering of the cartoon Homer Simpson on top of a still from the actual Godfather film?

Check out DALL-E results for similar prompts:

https://twitter.com/Dalle2Pics/status/1534718848137560064?re...

Ah, that is much better, definitely.

Have to let the AI experts speculate on why SD goes nuts there because it definitely knows what "The Godfather (1972)" means (if you ask for e.g. 'A still of Patrick Stewart in "The Godfather (1972)"' you get one - which I believe DALL-E can't do because of their facial restrictions?)

With StableDiffusion I can buy a used RTX 3090 on eBay for $650, tell the model to generate 5,000 images, and then review each one until I find what it is I'm looking for.

Turns out a shitload of misses are acceptable when it only takes 4-7 seconds to generate an image from a prompt. 5000 generations on an RTX 3090 takes around 7 hours +/- 30 minutes, by the way.

What I've been doing is generate maybe 100 images, pick the best one, and then generating another 100 from that, using --init-image ("good" image file name) and --init-image-strength 0.2 (or so), either with the original prompt or a slightly tweaked one.

Those are the params I use in ImaginAIry, mileage may vary if you're using a different package.

It's a bit ironic bringing a 7 hours of RTX 3090 run as a cost saving given that it's like 3KWh of electricity, which costs more than DALL-E's already outrageous prices.

Is your math ok?

3KWh is like $0.5 USD ...

DALL-E would give you like two pictures for that price, LOL

I live in Texas. 3 KWh of power is $0.29 for me.

In France that would be 0.51 EUR ...

While this is likely true for this specific prompt, I think that cherry-picking a single prompt that DALL-E outperforms SD on is not super indicative of anything. I've conversely found a large number of prompts where SD outperforms DALL-E, either in aesthetic quality or just following directions! I think you'd really have to compare both of them across a large number of prompts of different types to be sure.

Can you share the prompts you see where SD consistently outperforms DALL-E?

Have you poked around lexica.art?

Sure, try this one:

> house interior, friendly, playful, video game, screenshot, mockup, birds-eye view, top down perspective, jrpg, 32 bit, pixel art, black background

SD absolutely demolishes DALL-E on this one. SD produces really nice-looking output, with a high degree of consistency. DALL-E produces incoherent nonsense.

To say nothing of the fact that you have lots of sliders to configure jsut how closely or loosely it follows your prompt. And choice in sampling methods.

You can't just compare SD and DALL-E performance on prompts alone, because SD gives you a lot more levers to steer it in the direction you want.

>An easy example of DALL-E superiority is its ability to combine two different concepts together.

This is a con for some prompts. As an example, I asked for a painting of an elephant and a dog drinking tea together. The result was a dog with an elephant nose next to a teapot.

A similar misfire was the word 'porcupine' which drew pigs, I guess because porc is in it? Anyway, it's idea-blending is a little too aggressive.

Start your prompt with "group photo of" then list the elephant and the dog. If you try this across many images, group photo will result in about 2x as many keeping the subjects separate.

from dall-e: https://i.imgur.com/RHiOjuM.png

I would argue that none of these follow the prompt. they all represent a goodfather frame in simpson stile, which is not about placing homer in a godfather still.

My experience is that with prompts that fit into OpenAI's limiting content policy DALL-E text2img results are usually much better. And I use SD like 95% of the time, so it's not the case that I would be more used to DALL-E.

I need some examples because I don't really see it for the vast majority of usecases.

Here I wanted to illustrate the game Waffle[0], first attempt with Dalle was pretty good, not true for SD:

https://labs.openai.com/s/rCzJwauuiaIj1Pd3IyJGaHS3

Here I wanted an illustration of a nuclear plant in a japanese landscape, first attempt with Dalle produced multiple good results. I tried SD and MJ (back when MJ didn't use SD) as well, had trouble even with multiple attempts:

https://labs.openai.com/s/FxhxtMFe3kFS8msV8vekRAJ3

There are others, but anyway I think my examples are not important since it will be always easy to cherry pick prompts that yield the best results in model X.

In my experience SD is good at producing (especially non-photo-realistic) art that looks pretty and DALL-E is better at following a specific prompt when I know what exactly I want.

Of course I recognise your experience might (and probably does) differ.

[0] - https://wafflegame.net/

Reply on news.ycombinator.com