No analogy needed. It's actually because "Yes it exists" is a linguistically valid sentence and each word is statistically likely to follow the former word.
LLMs produce linguistically valid texts, not factually correct texts. They are probability functions, not librarians.