> How many of the letter N does the word "alienation" contain.
Mix and match letters and words. It'll hallucinate answers in the largest majority of cases.
What I really want is an LLM that simply tells me "I have no idea sorry." Or some mechanism by which it can provide a confidence score. Until we get there I'm wary of using them for anything other than cursory research or as a fun generation tool.