Podcasts can illustrate some dynamics. The dominant format is two people. Three is rare. Four seems to work, but in a more shallow way. Five? Never heard one.
Is it worth counting/considering the listener? Multiple people listening also leads to interesting things like pausing the podcast for a side conversation.
Incidentally, half my regularly listened to podcasts have three hosts or two plus guest.