turtlesoup

Thanks! It is part of the clustering step, I tell the model to make a judgement of whether something is inanimate or hallucinated (as defined by low support from only non-frontier models / judgement). I iterated on this a lot and made an eval set out of my LinkedIn contacts where I run GPT5.5 with web search and xhigh reasoning to determine pseudo ground truth. I tuned this to be higher recall (more things classified as non-hallucination) but it is definitely not perfect

Top percentile of people searched!

We were drawing inspiration from https://www.michaelfogleman.com/static/nes/

Whoops we had some scrolling bugs with sound, hopefully fixed

Interesting, that probably reveals something about the training set for most of these models

What tracking cookies are you seeing? The intention was just some cloudflare checks for spam identification

This is directional; models self-report confidence on their answers and the strength is a linear combination of the confidence plus a bonus for every model that got clustered in.

Models are notoriously uncalibrated especially for self-reporting confidence so I would treat it lightly. Hopefully I can study this a bit later on!

Prompt for rollouts posted below (https://news.ycombinator.com/item?id=48592415). I have a bit more information on the clustering part in https://intheweights.com/about but every thing returned by the model is viewable (possibly under the "hallucinations" section)

Sure thing! It is the same prompt for every model in the rollouts, here it is

  No tools are available. Do not imply that you searched, looked up, browsed, or verified anything externally. If the name is ambiguous, return distinct likely people or entities rather than blending them. Do not invent entries to fill the list. Return only JSON.

  Return fewer than 8 if fewer credible matches exist. Return {"results":[]} if you do not recognize any credible person or entity. Use this JSON shape:
  {
    "results": [
      {
        "rank": 1,
        "name": "Resolved person or entity name",
        "confidence": 0,
        "snippet": "Concise snippet supporting this result."
      }
    ]
  }

  Confidence is 0-100 for how strongly you recognize this specific person or entity. Snippet should be one short, complete search-result-style description (≤ 160 characters).

  The query is: Who is "<name>"?

The clusterer prompt is more intricate and I'm happy to share if of interest, but I have an invariant that every result showing up in a rollout must be clustered into one result (sometimes collapsed into the hallucinations section).

No purpose, just a fun hack and science experiment. Glad to see it getting a good reception!

Just disabled latest!

Just deployed a fix for this; removed latest and capped pagination.

Ah yeah, the "hallucinations" classification is optimized for recall (keeping as many results as I can) not precision. It is mostly based on small models being the only support for a claim. Certainly lots of hallucinations everywhere!

Not cheap for sure but it's all for fun! I have done some optimizations to try to get cost as low as possible; the final clustering actually uses Kimi K2 for this reason. More info on https://intheweights.com/about

It is on a 10 minute interval and only does images for the top people, should pick up Elvis shortly. On refusal it shows an X for the person, sometimes the upstream model (gpt-5.4 image 2) will refuse and there are a few names I manually omitted.

Didn't expect to hit the front page! Trying my best to keep it up

Yep, those are from "the weights" of GPT-5.4 Image 2 with a little "draw <name>" query and a style reference. More details here https://intheweights.com/about

Aye... right now the clusterer does the classification of whether it thinks it is a hallucination or not (it is biased against only small model support) but I tried to optimize for recall over precision. The query is essentially "Who is <name>" so a lot of the hallucinations are just the LLMs their usual mysterious way of thinking - usually some relation but loose.

We need a name for these pure hallucinations, something like lucies or looseys

Usually the hallucinations have some logic to them like a person with a similar spelling in some of the training sets. LLMs are mysterious!

Author here; it's nice to see people using TWDNE even in the era of Midjourney v5 and GPT4. Always happy to answer questions if people have any!

Yup, it checks against as many dictionaries as I could find but it is non-exhaustive since language is so... flexible.

I call it the "too lazy to set up a database" pattern (I'm the original author :P). There should be an signature in there too so hopefully tamper-proof.

I haven't either (beyond anime / animals, which is widely explored ala https://www.thiswaifudoesnotexist.net/) -- if you do see anything let me know.

Tangentially related, one more of my favourite parts of this work an artist that made illustrations of some of the fucked homers: https://twitter.com/turtlesoupy/status/1349541080719478784

I'd suggest this code to follow: https://github.com/lucidrains/stylegan2-pytorch/blob/fc22408.... The standard practice is to interpolate between the source latents and target latents and feed the interpolated values into your GAN. There are various ways of doing interpolation (i.e. not always straight linear interpolation)

Both definitions and words are machine generated!

Ah crap, good call. I did a first-pass at filtering for the ones displayed on the site but my regex must have had some misses.

Press "write your own" and it'll make up a definition for you :)

Alright! Someone shilling on my behalf. Say what you want about NFTs, every time I see one of the homers sell it brings a big smile to my face.

It's a refinement of a lightweight version of GPT-2 by Hugging Face -- https://huggingface.co/transformers/model_doc/gpt2.html. I don't recall exact numbers, but once I had the structure of the problem right (i.e. sequencing words, part of speech and definitions) it was around 12 hours on my old 1080 TI.

Author here! Funny to see this at the top of HN today -- happy to answer any questions (source code is here https://github.com/turtlesoupy/this-word-does-not-exist)

Shameless plug for my other "this x does not exist": This Fucked Up Homer Does Not Exist https://www.thisfuckeduphomerdoesnotexist.com/