I suspect this is related to why a string quartet is the right number of musical voices. Two violins, viola, and cello give you a very fulfilling number of separate ideas to track without overwhelming you.
In terms of perception, I'm not sure there's much of a relationship to a human conversation. To make things equal, the string players would need to take turns soloing while the others wait more or less silently to respond, each with their own solo response. You'd be bored out of your gourd if string quartets were written that way.
But more to the point, the vast majority of time in a string quartet is devoted to two or more of the players producing phrases of music in parallel, and that is musically coherent and pleasing to the players and audience. Most humans cannot track two humans speaking in parallel at all. That alone tells us that music cognition is a very different phenomenon than speech cognition.
In short, I'm not sure why a string quartet would be considered the optimal genre for humans to produce music together. And even if it is, the reasons why are even less likely to do with the protocols around human speech cognition, and certainly not with some bizarre equivalent of the "theory of mind" associated with the musical phrase produced by one of the instruments[1].
1: Small digression-- In Elliott Carter's 2nd String Quartet he actually started with a concept that each instrument was a kind of "character" in a play among the quartet. In this case, the problem with OP's metaphor becomes obvious even in the introduction-- the homogenous timbre of a string quartet makes it difficult to hear the differences among the characters. (IIRC I think even Carter admitted this.)
On the other hand the second and more famous dining (and conversation) club founded by Dr Johnson had originally 9 members, and gradually grew from that to dozens. Although many including Johnson may have not been entirely happy with the expansion.
For those who do not know counterpoint, you have only three motions a voice (a horizontal line of music, traditionally sung) can make relative to another voice (move closer, apart, or to hold steady) combined with limited voice ranges (say, a doubling of frequency, or so) and limited interval choices (seven, or so) within an octave or frequency doubling, and the voices are very close to one another but only rarely cross one another, on top of all that various rules systems that forbid or frown on such things as the tritone, parallel fifths, and so on into the weeds such that with more than a few voices you quickly run out of valid options for all the voices to move independently.
Interestingly, I like the 5-piece versions of all 3 of these: add a keyboardist to the rock band, a piano or harpsichord to the classical string or woodwind quartet, a female vocalist to the acapella group. Having two leads lets you do much more intricate countermelodies and harmonies.
A "classical" rock band consists of 4 utterly different instruments from a tonal perspective, and is thus nothing like 4 humans talking. Same thing for jazz - and its why you can have multiple instruments performing simultaneously and in ways that are not obviously connected to each other.
I have not seen a good band with more than 4 people on stage simultaneously.
Any vaguely disco-adjacent band will have more than 4 people on stage because there will be at least keyboards and horns in addition to drums, bass, guitar and vocals. Even a band that simply adds an additional person player percussion to a typical 4 piece exceeds your limit yet can wonderfully enhance the music.
If you haven't seen any of those bands, then that's your loss, but provides no reason to try to generalize about the right size for a live band.
You pick really popular names but in most regular bands when there is more people there is worse link between them...
Glenn Branca Orchestra.
> I have not met a religious programmer
> Donald Knuth
There are good bands like that but not most.
I've seen bigger bands but they suck in comparison.
However your link looks about jazz.
A common rock band is rarely good with more than 4 members because people lose unity and it's just technically harder and people are rarely professionally trained.