You mean you correctly guessed without testing at all - and got lucky. I'm not sure this is the same thing as 'solving' it.
It's the same reaction I have to the Monte Hall problem: I don't have to think or be clever to get the right answer. So naturally you couldn't effectively teach me anything by simply posing the question, and any information you collect by doing so won't accurately reflect what I learn or how well I think. You'd just be testing topical familiarity.