There's a lot more than just Orcs. Bilbo and the Dwarves almost get eaten by trolls in The Hobbit. There's intelligent giant spiders. The occasional Dragon.
But frankly Orcs are bad enough. They're cunning, actually genuinely intelligent tool using predators. They're mainly carnivorous by preference, but can probably survive on stuff that would make you sicken and die if they have to. As soon as a rural population gets established, Goblins will descend on it and hoover it all up to grow their population, but then probably die out when the food supply runs down again. So they have a boom-bust population cycle that keeps human populations suppressed.
The area in the north surrounding the Shire used to be a great kingdom, but it was wiped off the map by hordes of Orcs and Trolls and such fomented by the Witch King. There may also be a magical element to the success of the Orcs and other such creatures.
Whatever the logic or reasons, the fact is Middle Earth is severely under-populated compared to Europe and always has been. It's demographics are far too thin to support the kinds of developments we had, and every time they established kingdoms that might turn that around, they got violently crushed.
Bree is heavily fortified and paranoid for good reasons. The mission for the rangers isn't to militarily defend the region as a garrisoning force, it's to suppress any news or investigation of the Shire to keep it secret. As long as everyone thinks it's just a few scattered villages, as it has been for most of it's history, there's not enough there to send a big force of Orcs for. If the goblin king of the mountains knew how much bread, cheese and ale there was there for the taking, it would have got the scorched earth treatment long ago.