Yeah this tracks with what I've seen too.
As a note people with 100+ direct reports are not really managing them. Often it'll be indirect as in there are 100+ people in a hierarchy below you. You might only mange 10 people but they manage 10 people and so on.
In terms of reporting your "team" is all 100 people even though you may have never interacted with half of them other than an introduction.