The ones who appear stuck could be the reasonably clever ones who know that the question's purpose is to reject poor-fit candidates. The extra time is needed to think about how to "game" the question to avoid looking like a poor fit, not because the question has a large scope. (And that includes avoiding haplessly looking like a poor fit even when the candidate knows he or she is actually a good fit, just by fumbling a rejector question.)
Even those who could instantly blurt out a honest answer aren't going to do that, if they know what is good for them. For instance "On Monday, I start getting paid $150K per year to work on new widgets for my toy open source GUI library, that nothing uses other than the demo calculator, text pad and mini paint program."
All the discussion stimulating questions about what languages you'd like to be using and such are just pieces of rope to hang yourself with. Especially the reversal "what would you absolutely not want to do?" You probably don't want to give a passionate discussion about things you refuse to do.
Suppose the job you're interviewing for is in fact close to ideal. You want to be doing exactly what they're doing and paid about what they pay. You have the skills and are in fact a great fit. But, oops, you can't say that, because it's indistinguishable from the bullshit that someone completely naive would blurt out.
All this kind of thinking is going to chew up many seconds of time, and create awkward silences.