State borders are not any more arbitrary than the border between Canada and the US. In the US, "States" are sovereign states that gave up some limited amount of sovereignty to join the Union. They are not simply administrative divisions like in other countries. The Constitution established a limited role for the federal government of the United States and reserved all other powers to the individual states.

It may seem like a historical relic now, but each state joined the union under the premise (and likely binding law/treaty) of retaining some sovereignty. Each state has it's own laws, courts, police, military, etc. It's a feature, not a bug, that rural states can prevent the cities from imposing a tyranny of the majority on them at a federal level via the senate. Just as the house of representatives based on population can prevent a bunch of rural states from imposing their will on the high population ones.

Take for instance Nevada. If the majority of the population of Nevada wants legal gambling, they should be able to have it. A pure federal democracy would allow 9 states to make a law banning gambling nation wide. That's far less likely to happen in the current system as low population states have reason to band together and prevent any federal over riding of the freedom of their state's citizens to self govern.