Hacker News Reader

I strongly suspect the inability to do negation comes from the fact that the training dataset contains virtually no images tagged with negated descriptors. It wouldn't surprise me to see it correctly handle negations that might be more common in image tags (e.g. "a car not running" might be a sufficiently common phrase for "broken down car").

View on HN

Seems straightforward to check something like LAION-400M's captions to see how often negations show up.

If it's simply a lack of data, that should be easy to fix. Given an image caption, every other caption is a relevant negation: 'a photo of a happy dog' is also not 'a photo of a sad cat', and can be turned into the caption 'a photo of a happy dog and not a photo of a sad cat'. So lots of obvious data augmentation and synthetic data approaches to apply there to boost negation.

The problem is that most things don't have specific antonyms. The "not a cat" label could be applied to literally any image without a cat in it.

Exactly. That's why it's data augmentation.

Reply on news.ycombinator.com