There's usually a lead for an interviewing process like this, and I don't think it would be that hard to say something like "hey can I have a word for a minute" as you're transitioning between groups.
Even if there is, that doesn't mean he'll know who the lead is. I have been in interview where I have gone from person to person with no one in two consecutive sections. As far as I could tell there was no one person to discuss sensitive issues with.
Wow, interesting, thanks. What did you think of that process--did you end up hiring on there?