The vast majority of people care.
> We were nomads before we settled in cities, and it's only the designs of the empowered few that ever made the idea compulsory.
Reasoning from pre-agrarian living patterns is, quite frankly, hippy nonsense. And no, we didn't settle in cities because of "the designs of the empowered few", but because agriculture leads to more permanent, prosperous settlements, which attract raiders, and settling close together allowed for common defense. In other words, as soon as people earned a living by their own planning and sustained effort, (as opposed to merely collecting the bounty of the earth) they settled down and drew borders to protect what they had built from people who wanted to just show up and reap the rewards of their effort, at their expense!
> I also lived where I could see Tijuana from my back yard and all the pearl clutching and self-fanning over "illegal immigrants" is a giant crock of blustery nonsense.
We can't have borders because you could see Tijuana from your back yard?
> We have bigger problems than normal folks just trying to live their lives.
Defending borders is the most basic function of the state. It quite literally does not have anything better to do than to defend its borders.