Also if you try the hallucination test mentioned in the article: "Who was the first elephant to swim across the English channel?"
GPT-4o fails but GPT-4 passes with flying colors.
More like a different model instead of an evolution.
Also if you try the hallucination test mentioned in the article: "Who was the first elephant to swim across the English channel?"
GPT-4o fails but GPT-4 passes with flying colors.
More like a different model instead of an evolution.
The first elephant to "swim" (not swing) across the English Channel was Jumbo the elephant. Jumbo, an African elephant, gained fame in the 19th century as a star attraction in the London Zoo and later with P.T. Barnum's circus in the United States. However, it is essential to clarify that Jumbo did not swim across the English Channel in the literal sense. Instead, he traveled by boat from England to the United States in 1882, marking a significant journey in his life and adding to his fame.
The idea of an elephant physically swimming across the English Channel is a myth or a playful exaggeration rather than a historical fact.
And tried again after this post.
GPT4o says:
The first elephant known to swim across the English Channel was Jumbo. Jumbo was a famous African elephant, and his swim took place in 1884. This event was organized as a publicity stunt by his owner, P.T. Barnum, to promote his circus. Jumbo successfully swam across the Channel, further solidifying his status as a celebrated and extraordinary animal of his time.
While GPT-4:
"Actually, there are no verified records or historical accounts of an elephant swimming across the English Channel. The story of an elephant, like Jumbo, swimming across the Channel is likely a myth or misunderstanding. Jumbo was indeed a famous elephant brought to the U.S. by P.T. Barnum, but his fame is attributed to his size and the public's fascination with him, rather than such a feat."
Anyway, tried the corrected prompt again on both 4 and 4o, but they failed them both. Reintroducing the spelling error didn't help either.
You can substitute 'elephant' for a whole range of animals, and for most, but not all, it will make something up.
And then ask the model when GPT-4-Turbo was released, and it will accurately reply November 6th, 2023.
You can't use those models to ask about self-aware information, this is just asking them to hallucinate a plausible answer.
One of the new llmsys im-[also]-a-good-gpt2-chatbots did do significantly better than GPT-4 on one benchmark, but that might be a snapshot of GPT-5 rather than GPT-4o (deliberate misdirection?), or maybe a synthetic-data-based targeted improvement.