That is certainly not the impression I got from reading the OP :
"At one point when he was asked to move to another conference room he decided he had enough and said that he was done with the interview and wanted to leave."
"When he went to leave the lead jumped into the elevator with him and asked him why he didn't want to continue"
We're forced to interpret a 2nd hand retelling of the events, but everything points to this individual leaving relatively abruptly. Why else would someone feel the need to run after him into the elevator?