That was how the US Senate originally operated. You would elect your state legislature (where your vote is much less diluted than at the federal level) and they would elect your Senator.

Early 20th century populism took that away. It also meant that the states now have no representation in the federal legislature, which led to an almost immediate federal takeover of basically everything.