Hacker News Reader

It’s not generating images. This looks like an LLM like GPT-3.

It’s (probably) generating text in some encoding that represents a view hierarchy along with basic layout attributes (alignment etc)

Probably fine-tuned some on human-created examples with the same format.

So the prompt provides the example format and then merges in the user’s request. LLM spits output that is then parsed and transform into css/figma etc.

That’s almost surely how it is done. It is how we do it and it works well. We are just unsure it needs all that complexity; it seems generating ‘educated guesses’ works much better. Probably a combination will be a winner.

Many AI image generation can do text now. Especially some unreleased ones like Muse from Google.

What does behind the scenes might isn’t necessarily 1:1 with old image generation stuff either way.

Seems to me like this is not generating a simple image of the layout, but is generating the parameters to define the layout.