I would love to see the prompt / image data sent to GPT-4o!
you are given a reference and three candidates, which one of the candidates do you think is a match to the reference? Only output its identifier or a code when none is found
Not exactly that but something along those lines.
Then one "user" message per car (reference + candidates) with image + text indicating the type (reference or candidate) and an identifier (can be as simple as the index for the candidates).