It was tempting to tell him that we should just wrap this up if he had more important things to do.
Especially at a "small startup", this is a good sign. Makes me think there was more work than employees which is why the CEO was interviewing and doing customer support.
I had a friend, haven't seen him in a while then met for a coffee and a chat. 10 times in 20 minutes he would pick up a call in the middle of the conversation and "hold that thought".
Made a huge effort not to tell him to go f himself at the end but that concluded our "friendship".
This is not a thing that anyone should need to be told
If something comes up, find a new host or give the candidate a mid-session break.
maybe the CEO is swamped and needs to hire people so he can dedicate attention.
I'm assuming the best, but if my assumptions are true, the CEO should have said upfront that they are growing fast and he may need to personally help with support during the interview.