Hacker News Reader

Comment by sp0rk | Hacker News Reader

Did you generate a bunch all at once before starting to get some idea of what the natural variance looks like? I would think it's important to verify some level of progression over time, because with the current four it seems entirely possible that the examples could have all been generated at the same time with no changes to the model.

View on HN

Also unclear if he's sampling at temp=0. Looks like he doesn't set a temp? https://github.com/adamkdean/gpt-unicorn/blob/8ad76ec7161682... So not sure what he's really doing.

Then it's using the default value which is temperature=1.0, which is by no way deterministic (not that temperature=0.0 is either, but it's more likely to give similar responses to similar prompts, than 1.0 is)

Aren’t they using the March 14 model like the general public?

It’s frozen in time, there are no updates to it..

All of these will be drawn using the same model until they push a new update, or you switch to a different GPT

But I already think they proved the point that the generation is random enough that it would be extremely difficult to track progress this way.

GPT's output is by default somewhat random. If you ask the same exact question several times, you'll potentially get several different answers. Each successive word in the output is chosen from a distribution of possibilities -- that distribution is fixed, but that actual sample chosen from the distribution is not fixed. See, e.g., https://platform.openai.com/docs/api-reference/completions/c...

When using the openAi end points you can set the temperature, k_top etc. That will allow you to tone down the randomness. Passing a temperature of 0 would mean same input always has the same output.

When I ask GPT-4 to draw a unicorn, I get this:

https://i.imgur.com/HQ1Vjp0.png

And when I ask it to draw Mona Lisa, I get this:

https://i.imgur.com/gsBU49w.png

So far as I can see, the only difference is that the GitHub repo uses the API; the ones that I have were rendered using the web chat UI. Which makes me wonder if they're using the bleeding edge model for the chat.

That's a really interesting question!

I haven't yet gained access to the enhanced chat features with image outputs. I'm using the API with default parameters, with the gpt-4-0314 model, outputting SVG.

It's not a perfect experiment, but we'll see how it gets on over time.

To be clear, I'm not using multimodal chat, either. I just told it to output an SVG snippet in the chat and then manually copy/pasted it to view.

Reply on news.ycombinator.com