The founders thought so. It's called the US Senate.
This is known as the Connecticut Compromise[0] and, if it had not been adopted, we likely wouldn't have had a Constitution.
[0] https://en.wikipedia.org/wiki/Connecticut_Compromise
Edit: Added detail WRT the Connecticut Compromise.
The wisdom of the Senate was quite literal--30-year minimum age limit, 6-year terms, and an expectation that states would send elder statesmen as it was legislatures that selected their senators. As Alexis De Tocqueville described it,
> At a few yards' distance from this spot is the door of the Senate, which contains within a small space a large proportion of the celebrated men of America. Scarcely an individual is to be perceived in it who does not recall the idea of an active and illustrious career: the Senate is composed of eloquent advocates, distinguished generals, wise magistrates, and statesmen of note, whose language would at all times do honor to the most remarkable parliamentary debates of Europe.
> The founders thought so. It's called the US Senate.
No, they were trying throw a bone to states like Connecticut, Rhode Island and New Jersey, which were land-locked out of Western expansion. It was the more 'rural' states that actually preferred to balance Congressional power on population.
[ed: As the more rural states saw their own populations likely to increase over time. The Senate's existence had absolutely nothing to do about balancing rural vs urban interests].
As GP said above -- a counterweight to to heavily populated areas. Which the more rural areas were at the time.
The more rural states (like Virginia at the time) had most of the population, which is why they preferred representation based on population.
The argument was one of population (whether current or expected), as writings by all involved in it will tell you explicitly.
Regardless, the US Senate was created with two senators from each state provided equal representation to each state in that house.
Which was and still is, a counterweight to heavily populated areas. Rural or urban really doesn't make a difference.
Edit: Clarified rural vs. urban importance WRT US Senate representation.