In the US there are multiple levels of law enforcement that generally have their own jurisdictions. At the national level, there is the Federal Bureau of Investigation (FBI) to handle crimes that occur across state lines or large crime webs, like the Mafia. Then there are state-wide police, some states call them highway patrol, others state police. Probably the most famous are the Texas Rangers. These have the jurisdiction of the entire state. Next down the line is the county Sheriff, typically for enforcing county-wide laws. In many areas that don't have a local police force, the county sheriff are the most local police force they have. Finally there are city police like Los Angeles Police Department (LAPD) or New York Police Department (NYPD). These departments handle crime and law enforcement within a city's limits. These range from very large like the above examples to small departments like this article.

Generally if a crime occurs within a city it's that city police that will investigate, if it is outside the city within a county it may be the county sheriff or state police, and across state lines, the federal government gets involved. Obviously this can cause some confusion or fighting over jurisdictions. On the flip-side a local police department can help the needs of the community specifically. For example, where I live we have a small police department that responds immediately if there is a problem when you call them. They often help elderly residents that are having trouble, will drive by your house when you're on vacation, etc. Depending on states and cities local laws, sheriffs and police chiefs may be hired or elected.

Culturally, the United States has been oriented towards local control and biased against federal control. The states traditionally have been very independent of the federal government. For example most state constitutions are very long compared to our relatively short national constitution. In the western part of the united states, as communities were developed and grew, the need for local law enforcement grew with those communities, often before they were really a part of a state or the federal government had any control. So the legacies of those localized police and law enforcement entities has lived on.

I'm less familiar with how Europe's or other countries' police came to be, but typically those countries are far smaller than the United States and generally have more centralized federal power, which means having a centralized police force makes more sense.