This is not 100% accurate. Who the collective members of a server follow affects the content shown in searches and feeds to some degree.
For example, compare https://hachyderm.io/tags/cybertruck to https://mastodon.social/tags/cybertruck
As you can see, the bigger server 50% more posts, 40% more participants, 25% more posts today.
As you surf your "local server", you don't get "everything" from the Fediverse. If you care about that, you likely want to join the "biggest" server you can.
Personally I don't care too much about this. There's more content added to the Fediverse than I have time to consume, and so seeing a slice based on who I currently follow and the server I utilize is plenty for me.
But it also makes sense if it bothers people who don't want to miss out on some zeitgeist.
A very small one has a large likelihood to disappear. Each instance has different moderation rules. On a small instance, you don't even see entire threads and lots of other stuff does not sync properly.