I mean we have no problem looking at historical society and drawing a distinction between nobility and the masses. Why does that logic fall apart in modern day societies
In the past (and present!) this false narrative has been used to push mostly bad things: nativism, xenophobia, and really dumb economic policy. By falling for this false distinction you are priming yourself to be susceptible to demagoguery and similar bad ideas.
If you look at reality you'll find that most distinctions between "the elites" and "the people" collapse and there is substantial overlap between both groups, both in membership and in their political self-interest.
In The People vs The Elite, there are people and there are elite people.
People are people and elite people are people, therefor the people are the elite! Qed