The "25 horses" question is much less useful, but gives you an idea of how the candidate approaches a problem. Getting the right answer is great, but seeing that someone understands the concept of combinatorics or some kind of process of elimination is much more valuable.
You really want a technical question so that you can see someone CODE the solution, or better yet, give them some task to do that's apropos - that part of the argument is fair. In past companies we've done all three, weeding out terrible candidates with the coding questions, weeding out mediocre candidates with thought-process questions, and selecting only the best with a programming simulation.
(echo 'ZW5naW5lZXJpbmdAcm9ja3RoZXBvc3QuY29t' | base64 -D if you're curious, and the answer to the horse problem is 7 IIRC)